Skip to content
AI DemystifiedHow AI actually works
Lesson 06 of 22Foundationsintermediate4 min

Embedding Space

Meaning as geometry

What are embeddings?

Models map tokens to points in a high-dimensional space. Similar meanings land close together. That geometry is how systems know “dog” is nearer “cat” than “banana” without a hand-written dictionary. Below you will probe a fixed 2D teaching map and try vector analogies. Both use pure helpers the tests exercise (nearestNeighbors, sameCategoryShare, vectorAnalogy).

Teaching projection only

Distances and analogies on this page use fixed teaching vectors in 2D/3D. They are not live cosine scores from a production embedding API. Real spaces have thousands of dimensions, noise, and domain drift.

Meaning as geometry

Tokens become vectors. Nearby vectors usually share usage contexts, so "dog" sits closer to "cat" than to "banana". Distance is a usable stand-in for relatedness.

Learned, not hand-coded

Nobody draws the production map by hand. Training moves points so words that show up in similar sentences end up nearby. Clusters are a side effect of prediction, not a dictionary.

Retrieval uses neighbors

Search, clustering, recommendations, and RAG all lean on nearest-neighbor lookup. Geometry turns "find related chunks" into a distance query.

This page is a 2D sketch

Real model embeddings often have thousands of dimensions. The map here is a fixed teaching projection, not PCA of a live model. Use it to learn the idea, then measure real systems on your data.

Neighbor probe

Pick a word. We report its 5 nearest neighbors in the teaching plane and how many share its category (a simple cluster-coherence check). Radius for the map UI is 25 units.

“dog”· Animals
  • catAnimalsd=7.1
  • lionAnimalsd=8.5
  • snakeAnimalsd=11.2
  • rabbitAnimalsd=12.2
  • birdAnimalsd=14.1
Same-category share (k=5)
100%

5 of 5 nearest neighbors share the category. In a well-clustered region of this toy map, that share is high.

Teaching metric only · 50 fixed points · not a model embedding

Animals
10
avg intra d=14.0
Fruits
8
avg intra d=10.7
Vehicles
8
avg intra d=10.6
Tools
8
avg intra d=7.2
Emotions
8
avg intra d=8.0
Weather
8
avg intra d=9.8

Toggle categories, then click a word to inspect neighbors. Distance is Euclidean in this toy plane: closer usually means more related meaning. The map reuses the same nearestNeighbors helper as the probe above.

Loading embedding map…

Why this matters for products

Embeddings power search, clustering, recommendations, and retrieval-augmented generation (RAG): find relevant document chunks by similarity, then ground the model's answer on them. Geometry turns a closed-book guess into an open-book exam.

Continue to RAG

Key Insight

Even in thousands of dimensions, the principle holds: meaning becomes geometry, and geometry enables nearest-neighbor retrieval. That is the bridge from tokens to RAG and search products. When a neighbor list looks wrong, the bug is often the embedding model or the chunking strategy, not magic missing from the vector database.

Check your understanding

1/3

What is an embedding?

Sources, scope & review statusVerified Jul 29, 2026

Claims this lesson makes

  • Stable conceptEmbeddings place learned representations in a vector space where some semantic relationships become geometrically useful.

    Scope: The plotted coordinates and arithmetic presets are curated teaching projections.

  • Stable conceptLinear word analogies are a limited diagnostic: the standard test has known methodological weaknesses, and learned geometric patterns can encode social bias.

    Scope: An exact answer in the arithmetic lab is guaranteed by its hand-authored coordinates and is not representative of every trained embedding.

Next content review: 2027-07-29. Source links establish the lesson's claims; interactive numbers remain labeled simulations unless explicitly identified as measured data.