Cosine Similarity & Semantic Search

What it is

Cosine similarity measures the angle between two vectors. If they point in the same direction the score is 1, if they are perpendicular it is 0, and if they point in opposite directions it is -1. For text embeddings, a score near 1 means 'these two texts mean nearly the same thing'.

It ignores vector length and only looks at direction. That matters because the magnitude of an embedding can drift with input length or repetition, while the direction carries the meaning. Two paragraphs about Django deployment point the same way whether one is 30 words and the other 300.

Semantic search is just cosine similarity applied at scale: embed the query, compute the score against every stored vector, sort descending, take the top results. With normalized vectors that is a single matrix multiplication, which is why a NumPy array can serve retrieval for thousands of chunks in under a millisecond.

Why it's needed

Retrieval needs a number that says how relevant a chunk is to a question. Cosine similarity is that number. It is cheap, works well for high-dimensional embeddings, and most embedding models are trained with it as the objective, so it is the metric their geometry was shaped for.

Understanding it also tells you what the scores mean and do not mean. A 0.42 is not 'forty-two percent relevant'; it is only comparable to other scores from the same model. That understanding stops you from hard-coding a threshold that works for one model and silently breaks for another.

How it works

• Formula: cos(a, b) = (a · b) / (||a|| × ||b||). The dot product divided by the product of the lengths.
• With unit-length vectors the denominator is 1, so cosine similarity equals the dot product. Normalize once at indexing time and skip the division forever.
• For a query against N chunks, stack the chunk vectors into an N × d matrix and compute matrix @ query. The result is N scores in one operation.
• np.argsort(scores)[::-1][:k] gives the indices of the k highest-scoring chunks.
• Scores are relative: compare them against each other within one model, not against a universal threshold.
• Beyond a few hundred thousand vectors, brute force gets slow and you move to an approximate index (HNSW in FAISS, Qdrant, pgvector or Chroma).

Code example

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

chunks = [
    "Abdur migrated Django monoliths to FastAPI microservices on AWS.",
    "He built a LangChain and ChromaDB chatbot for a real-estate company.",
    "Lahore is the capital of Punjab province in Pakistan.",
]
# Index once: unit vectors so dot product == cosine similarity
index = model.encode(chunks, normalize_embeddings=True)

def search(query: str, k: int = 2):
    q = model.encode([query], normalize_embeddings=True)[0]
    scores = index @ q                      # one score per chunk
    top = np.argsort(scores)[::-1][:k]      # highest first
    return [(chunks[i], round(float(scores[i]), 3)) for i in top]

for text, score in search("any experience with vector databases?"):
    print(score, text)

# The ChromaDB chunk should rank first even though the query never
# says 'ChromaDB': that is semantic search.

Where I used this

Use cases

• The retrieval step in RAG
• Finding duplicate tickets, questions or product listings
• 'Related articles' and 'similar items' features
• Matching candidate resumes against a job description
• Clustering feedback or reviews by topic

Common mistakes

• Using Euclidean distance on unnormalized embeddings and wondering why long chunks always lose.
• Hard-coding a threshold like 'only keep chunks above 0.7'. Score distributions differ per model; set thresholds from your own data or skip them.
• Recomputing the chunk embeddings on every query. Encode once, store, reuse.
• Comparing a query vector to chunk vectors from a different model: the scores are random noise.
• Assuming the top score is always relevant. If nothing in the corpus matches, the top result is still returned; check the score gap or let the LLM say 'not found'.

Interview questions