JobsDart LogoJobsDart

How vector embeddings work in RAG: dimensionality, metrics, and semantic spaces

Vector embeddings translate human language into geometric coordinates where semantic meaning equals proximity. Here is how high-dimensional math powers modern semantic search.

JobsDart Editorial4 min read

Key takeaways

  • An embedding is an array of floating-point numbers mapping human concepts into high-dimensional geometric space (typically 768 to 3,072 dimensions).
  • Concepts with similar semantic meanings cluster together in vector space regardless of whether they share exact vocabulary.
  • Cosine similarity measures the angle between vectors (invariant to document length), whereas dot product considers both angle and magnitude.
  • Matryoshka Representation Learning (MRL) allows developers to truncate embedding dimensions by 50% or more with negligible loss in retrieval recall.
  • Mixing embedding models or upgrading an embedding model without re-indexing your entire database corrupts similarity scores due to embedding space drift.

From raw text to geometric coordinates

Computers do not understand the conceptual meaning of words, sentences, or technical skills. They only understand numbers. For decades, text search relied on lexical indexing—counting exact keyword frequencies using algorithms like TF-IDF and BM25. While fast, lexical matching fails when a job seeker searches for "frontend engineer" and an open vacancy specifies "React developer".

Vector embeddings solve this fundamental limitation. An embedding model takes arbitrary text—a word, a paragraph, or an entire resume—and transforms it into a fixed-length array of floating-point numbers called a dense vector representation. In this coordinate space, geometric distance directly mirrors semantic meaning.

When looking at vector embeddings explained through geometry, phrases that express similar underlying intent land near each other in space, enabling machines to understand relationships, analogies, and conceptual synonyms.

  • Dense vectors capture semantic intent rather than literal character strings.
  • High-dimensional coordinate geometry groups conceptual synonyms naturally together.
  • Enables cross-lingual, semantic, and fuzzy concept retrieval across millions of documents.

Similarity metrics: Cosine vs Dot Product vs Euclidean Distance

Once two pieces of text are converted into dense vector representations, how does a vector database determine how closely related they are? Search engines use three primary mathematical metrics.

Comparison of Vector Distance and Similarity Metrics
MetricFormula ConceptWhen to UseSensitivity to Text Length
Cosine SimilarityCosine of the angle between two vectors (-1.0 to 1.0)General text and document searchInvariant (length-normalized)
Dot Product (Inner Product)Sum of pairwise element productsWhen embeddings are pre-normalized to unit length (L2=1)Sensitive to vector magnitude
Euclidean Distance (L2)Straight-line physical distance between vector coordinatesClustering and spatial anomaly detectionSensitive to magnitude and scale

Dimensionality trade-offs and Matryoshka embeddings

Vector dimensionality represents a constant engineering balancing act between semantic precision and infrastructure cost. Early embedding models used 384 or 768 dimensions. Contemporary models frequently output 1,536 or 3,072 dimensions per chunk.

Higher dimensions capture finer semantic nuance, but they quadruple RAM requirements and disk storage while slowing down approximate nearest neighbor (ANN) search across millions of vectors.

Modern architectures overcome this dilemma through Matryoshka Representation Learning (MRL). MRL trains embedding models such that the most critical semantic information is packed into the first 256 or 512 dimensions. Developers can truncate 1,536-dimensional vectors to 512 dimensions, reducing vector database storage costs by 66% while preserving 98% of retrieval accuracy.

The risk of embedding drift in production

A critical mistake in enterprise RAG architectures is treating embeddings as static, interchangeable strings. Embedding spaces are model-specific coordinate systems.

If you embed your document repository using an OpenAI model and subsequently embed incoming user queries using an open-source BGE or Cohere model, the resulting vectors exist in completely separate, non-overlapping universes. The cosine similarity scores will be random noise.

Furthermore, when upgrading an embedding model from version 2 to version 3, teams cannot perform a rolling update on the fly. The entire database must be re-embedded in a dual-write migration before switching search traffic over.

  • Never mix vectors generated from different embedding models in the same search collection.
  • Always record the embedding model name and dimension version in document chunk metadata.
  • Plan dual-write migration pipelines whenever upgrading embedding model versions in production.

Frequently asked questions

What is the difference between dense and sparse embeddings?

Dense embeddings map text into dense floating-point vectors where every dimension holds a non-zero value representing abstract semantic concepts. Sparse embeddings (such as BM25 or SPLADE) map text to vocabulary-sized arrays where only words present in the document have non-zero weights.

Why is cosine similarity preferred over Euclidean distance for text search?

Cosine similarity measures directional angle rather than Euclidean distance, preventing longer documents containing more tokens from being artificially penalized simply because their vector magnitude is larger.

Can embedding models handle code and technical documentation effectively?

General text models struggle with syntax, but modern state-of-the-art embedding models are explicitly trained on GitHub repositories and documentation, achieving high accuracy on programming queries.

Further reading

Check this against your own resume

Scan your CV against a real job description, or build a parse-safe one from scratch. Your first scan costs nothing.

Free ATS checkerAI resume builder

Keep reading

AI EngineeringHybrid search vs vector search: why pure vector RAG quietly failsPure vector search is great for conceptual synonyms, but notoriously bad at finding product SKUs, error codes, and exact names. Hybrid search combines BM25 and vector embeddings to give you the best of both worlds.
AI EngineeringRAG reranking explained: why vector similarity fails and how cross-encoders fix itVector similarity compresses complex documents into a single dot product, missing nuances and negations. Adding a second-stage cross-encoder reranker solves retrieval precision.
AI EngineeringHow to build a production RAG application from scratchBuilding a prototype RAG script takes twenty lines of code; building one that works reliably in production requires careful document ingestion, chunking boundaries, and vector indexing.

Referenced in these guides

All career guides