Top PDF Embedding Uncertain Knowledge Graphs

Embedding Uncertain Knowledge Graphs

Embedding Uncertain Knowledge Graphs

Embedding models for deterministic Knowledge Graphs (KG) have been extensively studied, with the purpose of cap- turing latent semantic relations between entities and incor- porating the structured knowledge they contain into machine learning. However, there are many KGs that model uncer- tain knowledge, which typically model the inherent uncer- tainty of relations facts with a confidence score, and em- bedding such uncertain knowledge represents an unresolved challenge. The capturing of uncertain knowledge will bene- fit many knowledge-driven applications such as question an- swering and semantic search by providing more natural char- acterization of the knowledge. In this paper, we propose a novel uncertain KG embedding model UKGE, which aims to preserve both structural and uncertainty information of rela- tion facts in the embedding space. Unlike previous models that characterize relation facts with binary classification tech- niques, UKGE learns embeddings according to the confidence scores of uncertain relation facts. To further enhance the pre- cision of UKGE, we also introduce probabilistic soft logic to infer confidence scores for unseen relation facts during train- ing. We propose and evaluate two variants of UKGE based on different confidence score modeling strategies. Experiments are conducted on three real-world uncertain KGs via three tasks, i.e. confidence prediction, relation fact ranking, and re- lation fact classification. UKGE shows effectiveness in captur- ing uncertain knowledge by achieving promising results, and it consistently outperforms baselines on these tasks.
Show more

8 Read more

Traversing Knowledge Graphs in Vector Space

Traversing Knowledge Graphs in Vector Space

Path modeling. Numerous methods have been proposed to leverage path information for knowl- edge base completion and question answering. Nickel et al. (2014) proposed combining low-rank models with sparse path features. Lao and Cohen (2010) used random walks as features and Gard- ner et al. (2014) extended this approach by us- ing vector space similarity to govern random walk probabilities. Neelakantan et al. (2015) addressed the problem of path sparsity by embedding paths using a recurrent neural network. Perozzi et al. (2014) sampled random walks on social networks as training examples, with a different goal to clas-
Show more

10 Read more

OpenKE: An Open Toolkit for Knowledge Embedding

OpenKE: An Open Toolkit for Knowledge Embedding

We release an open toolkit for knowledge em- bedding (OpenKE), which provides a unified framework and various fundamental models to embed knowledge graphs into a continu- ous low-dimensional space. OpenKE prior- itizes operational efficiency to support quick model validation and large-scale knowledge representation learning. Meanwhile, OpenKE maintains sufficient modularity and extensibil- ity to easily incorporate new models into the framework. Besides the toolkit, the embed- dings of some existing large-scale knowledge graphs pre-trained by OpenKE are also avail- able, which can be directly applied for many applications including information retrieval, personalized recommendation and question answering. The toolkit, documentation, and pre-trained embeddings are all released on http://openke.thunlp.org/.
Show more

6 Read more

Context Dependent Knowledge Graph Embedding

Context Dependent Knowledge Graph Embedding

Perozzi et al. (2014) and Goikoetxea et al. (2015) have proposed similar ideas, i.e., to gener- ate random walks from online social networks or from the WordNet knowledge base, and then em- ploy word embedding techniques on these random walks. But our approach has two differences. 1) It deals with heterogeneous graphs with differen- t types of edges. Both nodes (entities) and edges (relations) are included during knowledge path ex- traction. However, the previous studies focus only on nodes. 2) We devise a two-stage scheme where the embeddings learned in the first stage will be fine-tuned in the second one, while the previous studies take such embeddings as final output. 2.2 Modeling LCPs
Show more

6 Read more

Knowledge Graph and Text Jointly Embedding

Knowledge Graph and Text Jointly Embedding

We examine the embedding approach to reason new relational facts from a large- scale knowledge graph and a text corpus. We propose a novel method of jointly em- bedding entities and words into the same continuous vector space. The embedding process attempts to preserve the relations between entities in the knowledge graph and the concurrences of words in the text corpus. Entity names and Wikipedia an- chors are utilized to align the embeddings of entities and words in the same space. Large scale experiments on Freebase and a Wikipedia/NY Times corpus show that jointly embedding brings promising improvement in the accuracy of predicting facts, compared to separately embedding knowledge graphs and text. Particularly, jointly embedding enables the prediction of facts containing entities out of the knowledge graph, which cannot be han- dled by previous embedding methods. At the same time, concerning the quality of the word embeddings, experiments on the analogical reasoning task show that jointly embedding is comparable to or slightly better than word2vec (Skip-Gram). 1 Introduction
Show more

11 Read more

Semantically Smooth Knowledge Graph Embedding

Semantically Smooth Knowledge Graph Embedding

This paper considers the problem of em- bedding Knowledge Graphs (KGs) con- sisting of entities and relations into low- dimensional vector spaces. Most of the existing methods perform this task based solely on observed facts. The only re- quirement is that the learned embeddings should be compatible within each individ- ual fact. In this paper, aiming at further discovering the intrinsic geometric struc- ture of the embedding space, we propose Semantically Smooth Embedding (SSE). The key idea of SSE is to take full ad- vantage of additional semantic informa- tion and enforce the embedding space to be semantically smooth, i.e., entities be- longing to the same semantic category will lie close to each other in the embedding s- pace. Two manifold learning algorithms Laplacian Eigenmaps and Locally Linear Embedding are used to model the smooth- ness assumption. Both are formulated as geometrically based regularization terms to constrain the embedding task. We em- pirically evaluate SSE in two benchmark tasks of link prediction and triple classi- fication, and achieve significant and con- sistent improvements over state-of-the-art methods. Furthermore, SSE is a general framework. The smoothness assumption can be imposed to a wide variety of em- bedding models, and it can also be con- structed using other information besides entities’ semantic categories.
Show more

11 Read more

Knowledge Graph Embedding for Ecotoxicological Effect Prediction

Knowledge Graph Embedding for Ecotoxicological Effect Prediction

Ontology alignment. Ontology alignment is the process of finding mappings or correspondences between a source and a target ontology or knowledge graph [10]. These mappings are typically represented as equivalences among the entities of the input resources (e.g., ncbi:taxon/13402 owl:sameAs ecotox:taxon/Carya). Embedding models. Knowledge graph embedding [22] plays a key role in link prediction problems where the goal is to learn a scoring function S : E × R × E → R . S(s, p, o) is proportional to the probability that a triple hs, p, oi is encoded as true. Several models have been proposed, e.g., Translating embeddings model (TransE) [5]. These models are applied to knowledge graphs to resolve miss- ing facts in largely connected knowledge graphs, such as DBpedia [14]. Embed- ding models have also been successfully applied in biomedical link prediction tasks (e.g., [3, 2]).
Show more

18 Read more

Knowledge Graph Embedding with Numeric Attributes of Entities

Knowledge Graph Embedding with Numeric Attributes of Entities

Recently, a number of Knowledge Graphs (KGs) have been created, such as DBpe- dia (Lehmann, 2015), YAGO (Mahdisoltani et al., 2015), and Freebase (Bollacker et al., 2008). KGs encode structured informa- tion of entities in the form of triplets (e.g. hM icrosof t, isLocatedIn, U nitedStatesi), and have been successfully applied in many real- world applications. Although KGs contain a huge amount of triplets, most of them are incomplete. In order to further expand KGs, much work on KG completion has been done, which aims to predict new triplets based on the existing ones in KGs. A promising group of research for KG completion is known as KG embedding. KG embedding
Show more

5 Read more

Incorporating Domain Knowledge into Medical NLI using Knowledge Graphs

Incorporating Domain Knowledge into Medical NLI using Knowledge Graphs

Tay et al., 2018). With these attempts, even though NLI in domains like fiction, travel etc. has pro- gressed a lot, NLI in medical domain is yet to be explored extensively. With the introduction of MedNLI (Romanov and Shivade, 2018), an expert annotated dataset for NLI in the clinical domain, researchers have started pursuing the problem of clinical NLI. Modeling informal inference is one of the basic tasks towards achieving natural lan- guage understanding, and is considered very chal- lenging. MedNLI is a dataset that assists in as- sessing how good a sentence or word embedding method is for downstream uses in medical domain. Recently, with the emergence of advanced con- textual word embedding methods like ELMo (Pe- ters et al., 2018) and BERT (Devlin et al., 2018), performances of many NLP tasks have improved, setting state-of-the-art performances. Following this stream of literature, Lee et al. (2019) in- troduce BioBERT, which is a BERT model pre- trained on English Wikipedia, BooksCorpus and fine-tuned on PubMed (7.8B tokens in total) cor- pus, PMC full-text articles. Jin et al. (2019) pro- pose BioELMo which is a domain-specific version of ELMo trained on 10M PubMed abstracts, and attempt to solve medical NLI problem with these domain specific embeddings, leading to state-of- the-art performance. These two attempts show a direction towards solving medical NLI problem where the pretrained embeddings are fine-tuned on medical corpus and are used in the state-of-the-art NLI architecture.
Show more

6 Read more

Knowledge graph embedding by dynamic translation

Knowledge graph embedding by dynamic translation

ABSTRACT Knowledge graph embedding aims at representing entities and relations in a knowledge graph as dense, low-dimensional and real-valued vectors. It can efficiently measure semantic correlations of entities and relations in knowledge graphs, and improve the performance of knowledge acquisition, fusion and inference. Among various embedding models appeared in recent years, the translation-based models such as TransE, TransH, TransR and TranSparse achieve state-of-the-art performance. However, the translation principle applied in these models is too strict and can not deal with complex entities and relations very well. In this paper, by introducing parameter vectors into the translation principle which treats each relation as a translation from the head entity to the tail entity, we propose a novel dynamic translation principle which supports flexible translation between the embeddings of entities and relations. We use this principle to improve the TransE, TransR and TranSparse models respectively and build new models named TransE-DT, TransR-DT and TranSparse-DT correspondingly. Experimental results show that our dynamic translation principle achieves great improvement in both the link prediction task and the triple classification task.
Show more

10 Read more

TransGate: Knowledge Graph Embedding with Shared Gate Structure

TransGate: Knowledge Graph Embedding with Shared Gate Structure

Embedding knowledge graphs (KGs) into continuous vec- tor space is an essential problem in knowledge extraction. Current models continue to improve embedding by focus- ing on discriminating relation-specific information from enti- ties with increasingly complex feature engineering. We noted that they ignored the inherent relevance between relations and tried to learn unique discriminate parameter set for each relation. Thus, these models potentially suffer from high time complexity and large parameters, preventing them from efficiently applying on real-world KGs. In this paper, we follow the thought of parameter sharing to simultaneously learn more expressive features, reduce parameters and avoid complex feature engineering. Based on gate structure from LSTM, we propose a novel model TransGate and develop shared discriminate mechanism, resulting in almost same space complexity as indiscriminate models. Furthermore, to develop a more effective and scalable model, we reconstruct the gate with weight vectors making our method has com- parative time complexity against indiscriminate model. We conduct extensive experiments on link prediction and triplets classification. Experiments show that TransGate not only out- performs state-of-art baselines, but also reduces parameters greatly. For example, TransGate outperforms ConvE and R- GCN with 6x and 17x fewer parameters, respectively. These results indicate that parameter sharing is a superior way to further optimize embedding and TransGate finds a better trade-off between complexity and expressivity.
Show more

8 Read more

Improving Knowledge Graph Embedding Using Simple Constraints

Improving Knowledge Graph Embedding Using Simple Constraints

Embedding knowledge graphs (KGs) into continuous vector spaces is a focus of cur- rent research. Early works performed this task via simple models developed over KG triples. Recent attempts focused on either designing more complicated triple scoring models, or incorporating extra information beyond triples. This paper, by contrast, in- vestigates the potential of using very sim- ple constraints to improve KG embedding. We examine non-negativity constraints on entity representations and approximate en- tailment constraints on relation represen- tations. The former help to learn compact and interpretable representations for enti- ties. The latter further encode regularities of logical entailment between relations in- to their distributed representations. These constraints impose prior beliefs upon the structure of the embedding space, without negative impacts on efficiency or scalabil- ity. Evaluation on WordNet, Freebase, and DBpedia shows that our approach is sim- ple yet surprisingly effective, significantly and consistently outperforming competi- tive baselines. The constraints imposed in- deed improve model interpretability, lead- ing to a substantially increased structuring of the embedding space. Code and data are available at https://github.com/i ieir-km/ComplEx-NNE_AER.
Show more

12 Read more

Knowledge Graph Embedding via Dynamic Mapping Matrix

Knowledge Graph Embedding via Dynamic Mapping Matrix

Knowledge graphs are useful resources for numerous AI applications, but they are far from completeness. Previous work such as TransE, TransH and TransR/CTransR re- gard a relation as translation from head en- tity to tail entity and the CTransR achieves state-of-the-art performance. In this pa- per, we propose a more fine-grained model named TransD, which is an improvement of TransR/CTransR. In TransD, we use two vectors to represent a named sym- bol object (entity and relation). The first one represents the meaning of a(n) entity (relation), the other one is used to con- struct mapping matrix dynamically. Com- pared with TransR/CTransR, TransD not only considers the diversity of relations, but also entities. TransD has less param- eters and has no matrix-vector multipli- cation operations, which makes it can be applied on large scale graphs. In Experi- ments, we evaluate our model on two typ- ical tasks including triplets classification and link prediction. Evaluation results show that our approach outperforms state- of-the-art methods.
Show more

10 Read more

Learning to Update Knowledge Graphs by Reading News

Learning to Update Knowledge Graphs by Reading News

The dimensions of all embeddings (words, enti- ties, and relations) are all set to 128, and the hid- den dimension is 256. We use a single layer en- coder, as we find that more layers do not bring any benefit. The basis number of basis-decomposition is 2. We replace the word embeddings of entity mentions in the text by the entity embeddings for better alignment of word embedding space and en- tity embedding space. The entity embeddings and relation embeddings are pre-trained using R-GAT, and the word embeddings are randomly initialized. We set dropout (Srivastava et al., 2014) rate to 0.5. The batch size in our experiments is 1. We use Adam optimizer (Kingma and Ba, 2014) with a learning rate of 0.001 for training.
Show more

10 Read more

Uncertainty Theory Based Novel Multi Objective Optimization Technique Using Embedding Theorem with Application to R & D Project Portfolio Selection

Uncertainty Theory Based Novel Multi Objective Optimization Technique Using Embedding Theorem with Application to R & D Project Portfolio Selection

But in reality, sometimes investors have to deal with the uncertainty which acts neither randomness nor fuzz- iness. In order to deal with such type of uncertainty, Liu [24] founds uncertainty theory as a branch of mathemat- ics. Subsequently, Liu [25] proposes uncertain process and uncertain differential equation to deal with dynamic uncertain phenomena. In addition, uncertain calculus is introduced by Liu [26] to describe the function of uncer- tain processes, uncertain inference is introduced by Liu [26] via the tool of conditional uncertainty and uncertain logic is proposed by Li and Liu [27] to deal with uncer- tain knowledge. Liu [28] proposes an uncertain pro- gramming including expected value model, chance con- strained programming and dependent-chance program- ming to model several optimization problems. Till now, several research works [29,30] have been done in this area, but none has considered the R & D project portfolio selection problem in the uncertain environment. Basi- cally, till date, no embedding theorem based optimization technique is proposed in uncertainty theory.
Show more

11 Read more

Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures

Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures

Embedding applications to relational learning constitute a huge field to which it is impossible to do justice, but one general difference between our approaches is that e.g. a matrix factorization model treats the embeddings as objects to score re- lation links with, as opposed to POE or our model in which embeddings represent subsets of proba- bilistic event space which are directly integrated. They are full probabilistic models of the joint set of variables, rather than embedding-based approx- imations of only low-order joint and conditional probabilities. That is, any set of our parame- ters can answer any arbitrary probabilistic ques- tion (possibly requiring intractable computation), rather than being fixed to modeling only certain subsets of the joint.
Show more

10 Read more

Jointly Embedding Knowledge Graphs and Logical Rules

Jointly Embedding Knowledge Graphs and Logical Rules

made by embedding models, via integer linear pro- gramming or Markov logic networks. In their work, however, rules are modeled separately from embed- ding models, and will not help obtain better embed- dings. Rockt¨aschel et al. (2015) proposed a joint model that injects first-order logic into embeddings. But their work focuses on relation extraction, cre- ating vector embeddings for entity pairs, and hence fails to discover relations between unpaired entities. This paper, in contrast, aims at learning more pre- dictive embeddings by jointly modeling knowledge and logic. Since each entity has its own embedding, our approach can successfully make predictions be- tween unpaired entities, providing greater flexibility for knowledge acquisition and inference.
Show more

11 Read more

Knowledge Based Semantic Embedding for Machine Translation

Knowledge Based Semantic Embedding for Machine Translation

Our proposed KBSE relies on the knowledge base. To get the semantic vector of source sentence, our semantic space should be able to represent any necessary information in the sentence. For ex- ample, since our designed knowledge base do not have tuples for number of objects, some results of our KBSE generate the entities in wrong plurali- ty form. Since our KBSE consists of two separate parts, the Source Grounding part and the Target Generation part, the errors generated in the first part cannot be corrected in the following process. As we mentioned in Section 3.3.1, combining KB- SE with encoder-decoder can alleviate these two problems, by preserving information not captured and correct the errors generated in source ground- ing part.
Show more

10 Read more

TransG : A Generative Model for Knowledge Graph Embedding

TransG : A Generative Model for Knowledge Graph Embedding

In spite of the success of these models, none of the previous models has formally discussed the issue of multiple relation semantics that a relation may have multiple meanings revealed by the entity pairs associated with the corresponding triples. As can be seen from Fig. 1, visualization results on embedding vectors obtained from TransE (Bordes et al., 2013) show that, there are different clusters for a specific relation, and different clusters indicate different latent semantics. For example, the relation HasPart has at least two latent semantics: composition-related as (Table, HasPart, Leg) and location-related as (Atlantics, HasPart, NewYorkBay). As one more example, in Freebase, (Jon Snow, birth place, Winter Fall) and (George R. R. Martin, birth place, U.S.) are mapped to schema /fic- tional universe/fictional character/place of birth and /people/person/place of birth respectively, indicating that birth place has different meanings. This phenomenon is quite common in knowledge bases for two reasons: artificial simplification and nature of knowledge. On one hand, knowledge base curators could not involve too many similar relations, so abstracting multiple similar relations into one specific relation is a common trick. On the other hand, both language and knowledge representations often involve ambiguous infor- mation. The ambiguity of knowledge means a semantic mixture. For example, when we mention “Expert”, we may refer to scientist, businessman or writer, so the concept “Expert” may be ambigu- ous in a specific situation, or generally a semantic mixture of these cases.
Show more

10 Read more

Estimating node connectedness in spatial network under stochastic link disconnection based on efficient sampling

Estimating node connectedness in spatial network under stochastic link disconnection based on efficient sampling

(Fushimi et al. 2018). Although our method can be applied to general networks in prin- ciple, we target mainly spatial networks because urban road structures can be naturally regarded as uncertain graphs and few existing studies focus on such networks. In our previous study (Fushimi et al. 2018), our method-connectedness centrality-defines the connectedness of each node as the expectation of the number of reachable nodes and attempts to extract nodes with high connectedness even when the graph is separated into several connected components by a link disconnection. In order to extract multiple nodes with high connectedness, we enhanced this method to group connectedness cen- trality, which selects nodes so as to maximize our objective function in a greedy manner. For a road network, the group connectedness centrality can be used to estimate instal- lation sites for evacuation facilities, as these must be accessible to neighboring residents even when the roads are blocked due to floods, landslides, or the collapse of houses and telegraph pillars.
Show more

24 Read more

Show all 10000 documents...