OpenAI's Entity Disambiguation: Not Memorizing Entities, but Composing Meaning from ~100 'Types'
OpenAI built an entity-disambiguation system where a neural network decides whether a word belongs to each of ~100 automatically discovered, non-exclusive 'types', then composes those types to infer which object the word refers to. Versus per-entity memorization, this 'type layer' is more general, cheaper, and more robust—useful for disambiguation in retrieval, knowledge graphs, and RAG.
Bottom line
A word can mean many things ('Apple' the fruit or the company? 'Jordan' the name or the country?). OpenAI's trick: rather than memorizing each entity, let the model decide which 'types' a word belongs to, then compose types to pin down its meaning.
How it works
- The system automatically discovers ~100 'types'—note 'discovers', not hand-designed categories.
- These types are non-exclusive: a word can belong to several at once ('Paris' is a 'city', a 'place name', and possibly a 'person name').
- For a given word, the network decides membership in each type; the combination of yes/no answers infers which specific object it refers to.
Why this is better
- More general: types are a more abstract intermediate representation than entities. Learning 'city', 'company', 'person' transfers to never-seen entities—per-entity memorization can't learn new words.
- Cheaper and more robust: entities number in the millions and keep growing; types are ~100 and relatively stable. Composing a small stable set beats maintaining a huge entity table.
- Closer to real semantics: the non-exclusive design acknowledges language's ambiguity—a word really can belong to multiple categories.
Engineering takeaways
For teams in retrieval, knowledge graphs, and RAG, 'types-first, entity-second' is a reusable disambiguation paradigm:
- In RAG, mapping ambiguous query words to types first can improve precision and recall.
- In knowledge-graph construction, a type layer can anchor entity linking and reduce direct-match noise.
Replacing rote memorization with transferable intermediate representations is a common thread in modern representation learning.