Embeddings

What it is

An embedding is a list of numbers that represents the meaning of a piece of text. The sentence 'Abdur built a RAG chatbot for a real-estate company' becomes a vector of 384 floats. A different sentence with similar meaning, such as 'he developed a retrieval-based assistant for property listings', becomes a vector that sits close to the first one in that 384-dimensional space.

The model that produces these vectors is trained so that distance means semantic similarity, not keyword overlap. That is the whole trick: once text is a vector, 'find the paragraph that answers this question' becomes 'find the vector nearest to this one', which is plain arithmetic.

The model I use for this site is all-MiniLM-L6-v2 from the sentence-transformers library. It runs on CPU, loads in a couple of seconds, produces 384-dimensional vectors, and handles up to 256 word pieces per input. OpenAI's text-embedding-3-small produces 1536 dimensions and is more accurate on long or technical text, but it costs money per call and needs a network round trip.

Why it's needed

Keyword search fails the moment the user's words differ from the document's words. A recruiter asking 'has he done any work with vector databases?' would never match a paragraph that says 'LangChain + ChromaDB chatbot' with plain string matching. Embeddings close that gap because 'vector databases' and 'ChromaDB' land near each other in the vector space.

Every RAG system, semantic search feature, recommendation engine and clustering job starts with embeddings. If the embeddings are poor, nothing downstream can fix it, so understanding what they are and what they cannot do is the first real AI-engineering skill.

How it works

• Load an embedding model once at startup. Loading per request is the most common performance bug I see in beginner code.
• Call model.encode() on a list of strings. Batching is much faster than encoding one string at a time.
• Each string becomes a fixed-length vector regardless of its length; a 5-word sentence and a 200-word paragraph both become 384 numbers.
• Normalize the vectors (unit length) so cosine similarity reduces to a dot product. sentence-transformers can do this with normalize_embeddings=True.
• Store the vectors next to the original text. You retrieve by vector but you show, and feed the LLM, the original text.
• Always embed queries with the same model that embedded the documents. Vectors from two different models are not comparable.

Code example

from sentence_transformers import SentenceTransformer
import numpy as np

# Load once, reuse for every request (this takes ~2s, encoding takes ~ms)
model = SentenceTransformer("all-MiniLM-L6-v2")

sentences = [
    "Abdur built a RAG chatbot with LangChain and ChromaDB.",
    "He developed a retrieval-based assistant for property listings.",
    "The recipe needs two eggs and a cup of flour.",
]

# normalize_embeddings=True gives unit vectors, so dot product == cosine
vectors = model.encode(sentences, normalize_embeddings=True)
print(vectors.shape)  # (3, 384)

# Compare the first sentence with the other two
for i in (1, 2):
    score = float(np.dot(vectors[0], vectors[i]))
    print(f"similarity(0, {i}) = {score:.3f}")

# Typical output: ~0.6 for the paraphrase, ~0.05 for the recipe

Where I used this

Use cases

• Semantic search over documentation, tickets or a knowledge base
• The retrieval step of every RAG pipeline
• Deduplication and clustering of similar records
• Recommendations: 'users who read this also read' without any user history
• Classification by nearest labelled example when you have little training data

Common mistakes

• Mixing models: embedding documents with one model and queries with another gives meaningless similarity scores.
• Ignoring the token limit: MiniLM silently truncates after 256 word pieces, so the end of a long paragraph is never seen. Chunk first.
• Embedding on every request: the model load belongs at startup, not inside the endpoint.
• Expecting exact matches: embeddings capture meaning, so 'Python developer' and 'Django engineer' score high, but a product code like 'SKU-4471' will not retrieve reliably. Use keyword search for that.
• Treating the dimension count as quality: 1536 dimensions is not automatically better than 384. Benchmark on your own data.

Interview questions