Knowledge graph embedding
Knowledge graph embedding is a class of methods that represents the entities and relations of a knowledge graph within a continuous mathematical space. An embedding model assigns each entity a numerical representation and defines relation-dependent operations that score candidate facts. The resulting score functions support statistical analysis of graph structure, especially link prediction, in which an unobserved edge is ranked according to its compatibility with known edges.
A knowledge graph is commonly expressed as a set of triples ((h,r,t)), where (h) is the head entity, (r) is a relation, and (t) is the tail entity. A triple such as ((\text{Paris},\text{capital of},\text{France})) records a directed relational assertion. Embedding replaces the discrete symbols with vectors, matrices, tensors, or related continuous objects whose parameters are estimated from observed triples. It therefore connects representation learning with statistical relational learning.
Formal framework
Let (\mathcal{E}) denote a finite set of entities and let (\mathcal{R}) denote a finite set of relation types. A knowledge graph is a subset
[ \mathcal{G}\subseteq \mathcal{E}\times\mathcal{R}\times\mathcal{E}. ]
An embedding model associates each entity (e\in\mathcal{E}) with a representation (\mathbf e) in a space of dimension (d). The representation of a relation depends on the model family and may take the form of a vector, a linear transformation, or a higher-order tensor. A scoring function
[ f_r(h,t)=f(\mathbf h,\mathbf r,\mathbf t) ]
assigns a real value to each possible triple. Depending on convention, either a larger or a smaller value indicates greater compatibility with the graph.
Parameter estimation contrasts observed triples with corrupted triples obtained by replacing the head or tail entity. Under a margin-based objective, the model separates the score of an observed triple from that of a corrupted counterpart. Under a probabilistic objective, the score is transformed into a conditional probability and optimized through cross-entropy or a related loss. This contrastive procedure reflects the absence of explicit negative facts in most knowledge graphs, although an unrecorded triple is not logically equivalent to a false statement.
Embedding dimensionality determines the size of the continuous representation but does not by itself determine logical expressivity. The scoring function constrains which relational patterns can be represented, while the training data determine which of those patterns are statistically supported. Regularization further limits parameter magnitude and reduces dependence on incidental properties of the observed graph.
Historical development
Continuous models of multi-relational data developed from matrix factorization, latent feature models, and tensor decomposition. In 2011, Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel introduced RESCAL as a factorization of a three-way relational tensor. RESCAL represents every relation by a full matrix, allowing different relation types to transform the same entity space in distinct ways.
The translational formulation TransE was introduced in 2013 by Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, Oksana Yakhnenko, and You Watanabe. In TransE, a valid triple is modeled by the approximate vector equation
[ \mathbf h+\mathbf r\approx\mathbf t. ]
This construction interprets a relation as a translation between entity representations. Its limited parameter count made the distinction between entity embeddings and relation-specific transformations particularly explicit, while its geometry restricted the treatment of reflexive relations and relations connecting one entity to many alternatives.
Later translational models altered the geometry rather than abandoning the translation principle. Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen formulated TransH by projecting entities onto a relation-specific hyperplane before translation. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu developed TransR, which maps entities into relation-specific spaces before applying a translational score. These formulations separated the role of an entity in the graph from its representation under a particular relation.
Bilinear and complex-valued models established a separate line of development. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng introduced DistMult, whose diagonal relation matrices provide a compact bilinear score. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard subsequently introduced ComplEx, which uses complex-valued vectors and conjugation to distinguish the two argument positions of an asymmetric relation.
Principal model families
Translational models
A translational model evaluates the displacement between a transformed head entity and a tail entity. The TransE energy function is commonly written as
[ f_r(h,t)=\lVert \mathbf h+\mathbf r-\mathbf t\rVert_p, ]
where (p) usually denotes the first or second vector norm. Low energy indicates that the embedded entities satisfy the relation-dependent translation.
This geometry represents inversion by reversing a translation and represents compositional regularities through vector addition. It cannot assign distinct destinations to the same translated point without placing those destinations near one another. Consequently, relations with multiple valid tails can cause unrelated entities to occupy similar regions of the embedding space.
RotatE replaces real-valued translation with elementwise rotation in a complex vector space. Each relation has unit-modulus components, and a compatible triple satisfies
[ \mathbf t\approx\mathbf h\circ\mathbf r, ]
where (\circ) denotes elementwise multiplication. The rotational representation directly accommodates relation inversion and permits certain compositions to correspond to multiplication of relation phases.
Bilinear models
Bilinear models represent a relation as an operator acting between two entity vectors. RESCAL uses the score
[ f_r(h,t)=\mathbf h^{\mathsf T}\mathbf R_r\mathbf t, ]
where (\mathbf R_r) is a full matrix. The matrix can encode interactions between every pair of latent dimensions, producing substantial representational capacity at a parameter cost proportional to (d^2) for each relation.
DistMult restricts (\mathbf R_r) to a diagonal matrix. Its score is unchanged when the head and tail are exchanged, because scalar multiplication is commutative. The model therefore cannot distinguish the direction of a genuinely asymmetric relation through its scoring function alone.
ComplEx retains a factorized score while moving entity and relation embeddings into the complex number domain. The score takes the real part of
[ \langle \mathbf h,\mathbf r,\overline{\mathbf t}\rangle, ]
where the bar denotes complex conjugation. Conjugation makes the score sensitive to argument order and removes the symmetry imposed by DistMult.
Tensor-decomposition models
A knowledge graph can be represented as a three-way binary tensor whose axes correspond to head entities, relation types, and tail entities. Tensor-decomposition models approximate this sparse structure through lower-dimensional factors. RESCAL corresponds to a relation-specific matrix factorization, whereas DistMult and ComplEx impose more constrained decompositions.
TuckER applies a Tucker decomposition with a shared core tensor. Entity and relation embeddings interact through this core, which determines how latent entity dimensions combine under each latent relation dimension. Several earlier bilinear models arise as restricted forms of this framework, making tensor decomposition a common mathematical language for comparing their expressivity.
Learning from incomplete graphs
Knowledge graphs generally follow an open-world assumption: failure to record a triple does not establish its negation. Standard embedding objectives nevertheless require contrastive examples, so training commonly constructs negative candidates by replacing one endpoint of a known triple. This operation introduces false negatives whenever a corrupted triple is valid but absent from the stored graph.
Negative sampling distributions influence the learned representation because most randomly generated triples are easy to distinguish from recorded facts. More concentrated distributions produce candidates that resemble known graph structure, although they also increase the probability of treating an unrecorded fact as negative. Self-adversarial sampling assigns greater weight to corrupted triples that the current model scores highly, thereby coupling the sampling process to the model’s evolving errors.
Graphs also contain systematic frequency variation. A small number of entities participate in many triples, while numerous entities occur rarely. Frequently observed entities receive more direct parameter updates and are consequently easier to position from structural evidence alone. Sparse entities depend more strongly on shared relational patterns or on external information supplied through text embedding, attributes, or graph neighborhoods.
Evaluation
Link-prediction evaluation removes a recorded triple and asks the model to rank its missing entity among all entities in the graph. For tail prediction, the candidates have the form ((h,r,e)); head prediction analogously ranks ((e,r,t)). Mean reciprocal rank averages the inverse rank of the correct entity, while Hits@(!k) records the proportion of queries for which a correct entity appears among the first (k) candidates.
Filtered evaluation removes other known correct triples from the candidate set before computing a rank. Without filtering, a model can be penalized for ranking one valid answer above the particular valid answer selected for the test query. Filtering remains dependent on graph completeness because a correct but unrecorded candidate is not removed.
Benchmark performance can be affected by reciprocal relations and near-duplicate edges. If a training graph contains ((h,r,t)) and a test graph contains an inverse relation connecting (t) to (h), prediction may reduce to recognizing a correspondence already present in the data. Dataset construction therefore determines whether an evaluation measures broad relational generalization or reconstruction of recurring graph patterns.
Evaluation through rank statistics does not directly measure probability calibration or semantic validity. A high-ranking triple can reflect a regularity in the stored graph without constituting a justified factual assertion outside that graph. Conversely, a semantically correct triple may receive a low rank when its relation pattern is absent from the training structure.
Expressivity and interpretation
The expressivity of an embedding model concerns the relational configurations representable by its scoring function. Symmetry requires that exchanging the head and tail preserve compatibility, while antisymmetry requires the reverse triple to receive a different score. Inversion links two relations whose arguments occur in opposite order, and composition links a sequence of relations to another relation. Model families encode these configurations through different algebraic constraints.
An embedding coordinate usually lacks a fixed human-readable meaning. Rotations, reflections, and other transformations of the latent space can preserve every triple score while changing individual coordinates. This non-identifiability prevents a coordinate from being interpreted independently of the full scoring function and its equivalent parameterizations.
Structural embeddings also inherit distinctions and omissions contained in their training graphs. They summarize observed relational regularities rather than supplying an independent account of causation or truth. When embeddings are used in a downstream machine learning system, their behavior depends jointly on the source graph, the objective function, and the mechanism that converts scores into decisions.
Relation to graph neural networks
Graph neural networks compute representations by aggregating information from neighboring nodes, whereas classical knowledge graph embedding assigns an independently optimized vector to each entity. The distinction is associated with inductive behavior. A parameter assigned directly to an entity provides no representation for a previously unseen entity, while an aggregation function can construct one from attributes and neighboring edges.
Relational graph convolution extends neighborhood aggregation by using relation-dependent transformations. Hybrid systems combine such aggregation with translational or bilinear decoders, so the encoder derives entity representations from local structure and the decoder scores candidate triples. In this setting, knowledge graph embedding denotes the scoring geometry as well as the learned vectors themselves.