From Knowledge Graphs to Box Embeddings

Knowledge graphs represent knowledge symbolically through entities and relations. Once knowledge is written in this form, several natural problems arise: we may want to predict facts that are missing, infer facts that follow from known facts and rules, or answer queries involving several relations. Knowledge graph embedding provides a
geometric approach to these problems by representing symbolic knowledge in a continuous space.

This blog follows one question:
$$\text{How should relational knowledge be represented geometrically?}$$
The simplest answer is to represent entities by points and relations by transformations between points. TransE provides a canonical example of this idea. However, a point is naturally suited to representing a single location, while relational knowledge often describes sets of possible entities.

Query2Box therefore moves from points to regions by representing query answer sets as boxes. BoxE takes a different step and asks how boxes can represent relations themselves. BoxLitE extends this geometry toward logical and ontological knowledge, while Convex BoxE turns attention from representation to the optimization problem used to learn it.

The resulting narrative is
$$\text{symbolic knowledge}\longrightarrow\text{points}\longrightarrow\text{boxes}\longrightarrow\text{relational and logical geometry}.$$

1. Knowledge Graphs and Their Tasks

A knowledge graph is relational data. In its standard binary form, a fact is represented by a triple
$$(h,r,t),$$
where $h$ is the head entity, $t$ is the tail entity, and $r$ is the relation connecting them. For example, a simple ordered structure may contain the facts
$$a\leq b,\qquad b\leq c.$$
The graph itself is the data. Once the data are given, different tasks can be posed on top of it. Three tasks are particularly useful for understanding the development from vector embeddings to box embeddings:

prediction, inference, and query answering.

Prediction asks for facts that may be missing from the observed graph.
Given a partial triple such as
$$(a,\leq,?),$$
a knowledge graph completion method ranks candidate entities that could fill the missing position. More generally, knowledge graph completion, or KGC, attempts to predict plausible triples that are not currently observed.

Inference asks a different question. If the relation $\leq$ is known to be transitive, then
$$a\leq b,\qquad b\leq c$$
entails
$$a\leq c.$$
Here the new fact is not merely predicted because it resembles observed
data. It follows from the facts together with a logical rule.

Query answering asks for all entities satisfying a possibly more complex relational condition. A query may ask for every $x$ such that $a\leq x$, or may combine several relations through conjunction and existential quantification. The output of such a query is therefore naturally a set of entities rather than a single entity.

These three tasks can be summarized as
$$\text{predict}\qquad\text{infer}\qquad\text{answer}.$$
Knowledge graph embedding, or KGE, is an approach to such problems. It replaces symbolic objects with geometric representations and turns relational structure into geometric conditions. Individual methods such as TransE, Query2Box, and BoxE are models within this broader approach. The first question is therefore how a symbolic relation can become geometry.

2. TransE: Relations as Translations

TransE gives one of the simplest answers. Entities are represented by points in a vector space, while a relation is represented by a translation vector. For a true triple $(h,r,t)$, the model seeks
$$e_h+r\approx e_t.$$
A symbolic relation has therefore become a geometric transformation. Starting from the point representing $h$, the relation vector $r$ moves toward the point representing $t$.

This idea is attractive because it converts knowledge graph completion into a geometric scoring problem. A candidate tail $t$ is plausible when $e_t$ lies close to the translated point $e_h+r$. Thus the symbolic question
$$(h,r,?)$$
becomes the geometric question of which entity point lies near a target point.

The simplicity of this representation also reveals its limitation. For a fixed head $h$ and relation $r$, the expression
$$e_h+r$$
defines essentially one target point. Suppose a relation is one-to-many and both $R(a,b)$ and $R(a,c)$ are true. TransE then requires
$$e_a+r\approx e_b$$
and
$$e_a+r\approx e_c.$$
Consequently, $e_b$ and $e_c$ are pushed toward the same location even when $b$ and $c$ should remain distinct. Similar geometric difficulties arise for other general relational patterns.

This does not mean that vector embeddings can only represent functional relations. Many later KGE models use richer scoring mechanisms. The important observation here is narrower: the point-to-point translation geometry of TransE gives a fixed pair $(h,r)$ essentially one geometric target.

This suggests a different question. If a query may have many valid answers, why should its geometric representation be a single point?

3. Query2Box: Queries as Regions

A query naturally defines a set. For a one-hop query
$$R(a,?),$$
there may be several correct answers $b,c,d,\ldots$. More complex queries can define even larger answer sets. Representing such a query by one point therefore forces a set-valued object into a point-valued representation.

Query2Box changes the geometric object. Instead of representing a query by a point, it represents the query by a box. Schematically,
$$\text{query}\longrightarrow\text{answer set}\longrightarrow\text{box}.$$
A box in $d$ dimensions can be written as
$$B=[l_1,u_1]\times\cdots\times[l_d,u_d].$$
Entity embeddings remain points, but a query is represented by a region.

Candidate entities can then be ranked according to whether their points lie inside the box or how far they lie from it. One query can therefore geometrically accommodate many answers without forcing them to collapse to a single target point.

The move from points to boxes also provides natural geometric operations. Intersection of boxes can represent conjunction of conditions, while the size and location of a box encode the set of possible answers associated with a query. The central conceptual change is therefore not simply from vectors to a different geometric shape. It is from a point-valued representation to a set-valued representation.

Query2Box uses boxes to represent answer sets of queries. This leads to a broader question. A binary relation is itself a set, but it is a set of pairs,
$$R\subseteq\mathcal E\times\mathcal E.$$
Can boxes represent the relation itself rather than only the answer set generated by a particular query?

4. From Query Boxes to Relational Boxes

4.1. BoxE: Relations as Boxes

A binary relation has two argument positions,
$$R(h,t).$$
If boxes are to represent the relation itself, a natural first step is to distinguish these positions. BoxE therefore associates a binary relation with two boxes,
$$R\longmapsto\left(R^{(1)},R^{(2)}\right),$$
where $R^{(1)}$ corresponds to the first argument position and $R^{(2)}$ to the second. A first attempt might declare a fact $R(a,b)$ valid whenever
$$e_a\in R^{(1)},\qquad e_b\in R^{(2)}.$$
This representation identifies which entities can occupy each argument position, but it loses information about which entities can occur together.

Consider a relation containing only
$$R(a,b),\qquad R(c,d).$$
The two-box condition requires $e_a$ and $e_c$ to lie in $R^{(1)}$, while $e_b$ and $e_d$ lie in $R^{(2)}$. But the same conditions would also accept
$$R(a,d),\qquad R(c,b).$$
The relation has effectively been factorized into a Cartesian product of two marginal sets. The boxes tell us who can play each role, but not who can play that role with whom.

BoxE introduces bump vectors to restore this interaction. Each entity $a$ is represented by a base position $e_a$ together with a bump vector $b_a$. For a binary fact $R(a,b)$, the points tested against the relation boxes are not simply $e_a$ and $e_b$, but
$$e_a+b_b\in R^{(1)},\qquad e_b+b_a\in R^{(2)}.$$
The representation of $a$ used in the fact now depends on its partner
$b$, and the representation of $b$ depends on $a$. In particular, $R(a,b)$ tests $e_a+b_b$, whereas $R(a,d)$ tests $e_a+b_d$. These two contextualized points need not occupy the same location.

The conceptual roles of the two ingredients can therefore be separated:
$$\text{boxes}\longrightarrow\text{admissible argument regions},$$
$$\text{bumps}\longrightarrow\text{interaction between arguments}.$$
Query2Box and BoxE thus use boxes for different purposes.

Query2Box maps a query to a region containing its possible answers. BoxE uses boxes as components of a representation of relations and uses bumps to make entity representations depend on the other entities occurring in a fact.

4.2. BoxLitE: From Relations to Logical Knowledge

BoxE raises the expressive power of relational geometry, but a knowledge base may contain more than relational facts. It may also contain conceptual and ontological knowledge. For example,
$$Student\sqsubseteq Person$$
states that every student is a person.

Regions provide a particularly natural representation for such statements. If concepts are represented by boxes, the logical inclusion can be reflected geometrically as
$$B_{Student}\subseteq B_{Person}.$$
BoxLitE develops this direction by combining relational facts with conceptual knowledge. Instead of using geometry only to score missing links, boxes also encode structures such as concept inclusion and role constraints. Geometric containment can therefore correspond to logical inclusion.

This reconnects box embeddings with the inference task introduced at the beginning of the blog. The geometry is no longer used only to predict which triples are plausible. It is designed so that part of the logical structure of the knowledge base is reflected directly by relations between geometric regions.

The progression can now be seen as
$$\text{answer sets}\longrightarrow\text{relations}\longrightarrow\text{logical knowledge}.$$

4.3. Convex BoxE: From Representation to Optimization

The previous developments concern representation: what geometric objects should represent entities, queries, relations, and concepts? Once a representation has been chosen, another question remains: how should its parameters be learned?

Knowledge graph embedding models are typically trained by solving optimization problems over their geometric parameters. These problems are often nonconvex. Convex BoxE turns attention from the expressive role of boxes to the optimization structure induced by their parameterization.

The question becomes

Can a box-based knowledge representation be learned through convex optimization?

This introduces a second reason for studying boxes. Their value may come not only from their ability to represent sets and logical inclusion, but also from the possibility of constructing learning formulations with stronger optimization properties.

The development from TransE to box-based models can therefore be understood through a sequence of increasingly rich questions. TransE asks how relations can become geometric transformations. Query2Box asks what geometry should represent a set of answers. BoxE asks how boxes can represent general relations without losing interactions between their arguments. BoxLitE asks how the same geometry can encode logical and ontological structure. Convex BoxE asks whether these representations
can also lead to better structured learning problems.

The move from vectors to boxes is therefore not merely a change of geometric shape. It is a change in what geometry is asked to represent:
$$\text{points}\longrightarrow\text{sets}\longrightarrow\text{relations}\longrightarrow\text{logical structure}.$$

From this viewpoint, box embeddings provide a bridge from symbolic relational knowledge to a geometry in which prediction, query answering, reasoning, and learning can increasingly be expressed within the same representational language.

References

A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, and O. Yakhnenko.
Translating Embeddings for Modeling Multi-relational Data. NeurIPS,
2013.
H. Ren, W. Hu, and J. Leskovec. Query2Box: Reasoning over Knowledge
Graphs in Vector Space Using Box Embeddings. ICLR, 2020.
R. Abboud, I. I. Ceylan, T. Lukasiewicz, and T. Salvatori. BoxE: A
Box Embedding Model for Knowledge Base Completion. NeurIPS, 2020.
D. Lourenço, H. Morgan, A. Ozaki, M. Pavlovic, and M. Sallinger.
BoxLitE: A Faithful Knowledge Base Embedding Based on Convex
Optimization. 2026.
H. Morgan. From Translations to Boxes: Convexifying Knowledge Graph
Embedding Approaches. Diploma thesis, TU Wien, 2026.