Introduction

Multilingual knowledge bases such as Wikipedia (Wikipedia, 2016), WordNet (Bond and Foster, 2013), and ConceptNet (Speer and Havasi, 2013) are becoming essential sources of knowledge for people and AI-related applications. These knowledge bases are modeled as knowledge graphs that store two aspects of knowledge: the monolingual knowledge that includes entities and relations recorded in the form of triples, and the cross-lingual knowledge that matches the monolingual knowledge among various human languages.

The coverage issue of monolingual knowledge has been widely addressed by recently proposed embedding-based techniques, which provide simple methods to encode entities in low-dimensional embedding spaces and capture relations as means of translations among entity vectors. Meanwhile, the problem of applying these techniques on cross-lingual knowledge remains largely unexplored. Such knowledge, includinginter-lingual links(ILLs) that match the same entities, andtriple-wise alignment (TWA) that represents the same relations, is very helpful in synchronizing different language-specific versions of a knowledge base that evolve independently, as needed to further improve applications built on knowledge bases, such as Q&A systems, semantic Web, and Web

search. In spite of its importance, this cross-lingual knowledge remains largely intact. In fact, in the most successful knowledge base Wikipedia, we find that ILLs cover less than 20% entity alignment.

Leveraging knowledge graph embeddings to cross-lingual knowledge no doubt provides a generic way to help extract and apply such knowledge. However, it is a non-trivial task to find a tractable technique to capture the cross-lingual transfers. Such transfers are more difficult to capture than relational translations for several reasons: (i) a cross-lingual transfer has a far larger domain than any monolingual relational translation; (ii) it applies on both entities and relations, which have incoherent vocabularies among different languages; (iii) the known alignment for train- ing such transfers usually accounts for a small percentage of a knowledge base. Moreover, the characterization of monolingual knowledge graph structures has to be well-preserved to ensure the correct representation of the knowledge to be aligned.

To address the above issues, we first propose a multilingual knowledge graph embedding model

MTransE, that learns the multilingual knowledge graph structure using a combination of two component models, namely knowledge model and alignment model. The knowledge model en- codes entities and relations in a language-specific version of knowledge graph. We explore the method that organizes each language-specific version in a separated embedding space, in which

MTransE adopts TransE as the knowledge model. On top of that, the alignment model learns cross-lingual transfers for both entities and relations across different embedding spaces, where the following three representations of cross-lingual alignment are considered: distance-based axis cal- ibration, translation vectors, and linear transformations. Thus, we obtain five variants ofMTransE

based on different loss functions, and identify the best variant by comparing them on cross-lingual alignment tasks using two partially aligned trilingual graphs constructed from Wikipedia triples.

While the MTransE solely relies on the structured knowledge for cross-lingual learning, it would be promising to enhance the corresponding learning process with the literal descriptions of entities that are stored in many KGs (Ji et al., 2017; Lehmann et al., 2015; Mahdisoltani et al., 2015). These descriptions comprise an alternative view of entities that potentially bridges two languages, since the descriptions of an entity in different languages often share a lot of semantic

EN triple: (Ulugh Beg, occupation, astronomer)

FR triple: (Ulugh Beg, activité , astronome)

who concentrates their studies on a specific question or field outside of the scope of Earth...

Un astronome est un scientifique spécialisé dans l'étude de l'astronomie...

Inter-lingual Link (ILL): (astronomer@EN, astronome@FR)

Figure 3.1: A simple example which shows triples, an ILL, and entity descriptions in a multilin- gual KG (DBpedia). The French description for astronome means an astronomer is a scientist specialized in the study of astronomy, which contains much fewer content details than the English description forastronomer.

information. However, it is non-trivial to characterize and utilize such information for cross-lingual learning, as this requires the model to learn to match descriptions across different languages with inadequate labels, while conquering the inconsistency of literals in content details, grammars, and word orders (as shown in Fig. 3.1). Moreover, aggregating semantic relatedness of descriptions from words of different languages is another challenge.

To address these issues, we propose a novel co-training-based approach KDCoE to enhance the semi-supervised learning of multilingual KG embeddings. KDCoEiteratively trains two com- ponent embedding models on multilingual KG structures and entity descriptions respectively. A KG embedding model jointly trains a translational knowledge model with a linear-transformation- based alignment model to encode the KG structure. A description embedding model employs an attentive gated recurrent unit encoder (AGRU) and multilingual word embeddings to characterize multilingual entity descriptions, and is trained to collocate the embeddings of cross-lingual coun- terparts. The co-training is processed on a large Wikipedia-based trilingual KG, for which a very small portion of ILLs is used for training. During each iteration of co-training, both models al- ternately propose a set of most confident new ILLs to strengthen the supervision of cross-lingual learning, which leads to gradually improved accuracy on cross-lingual inferences. Experimen- tal results on entity alignment confirms the effectiveness ofKDCoEthat significantly outperforms previous models, while those results on zero-shot alignment and cross-lingual KG completion also show wider usability of our approach.