Vector Databases (ChromaDB)

What it is

A vector database stores embeddings alongside their text and metadata, and answers 'give me the k nearest vectors to this one' quickly, even with millions of rows. It adds the things a plain array lacks: persistence, updates and deletes by id, metadata filtering, and an approximate index so queries stay fast as data grows.

ChromaDB is the easiest one to start with. It runs in-process, persists to a folder, embeds documents for you if you let it, and its API is three calls: create a collection, add documents, query. pgvector adds the same capability to Postgres you already run. Qdrant, Weaviate and Pinecone are dedicated servers for scale. FAISS is a library, not a database: blazing fast search with no persistence or filtering of its own.

For this portfolio I deliberately did not use one. A few dozen paragraphs fit in a NumPy matrix and brute-force cosine search takes microseconds. At B1 Properties, with listing documents that changed and a chatbot that needed filtering, ChromaDB was the right tool.

Why it's needed

Brute-force search over an array is O(N) per query and has no update story. Once you have more than a hundred thousand chunks, or chunks that change daily, or several tenants whose data must stay separate, you need an index, metadata filters and upserts. That is the job a vector database does.

Choosing one is also an engineering-judgement signal. Reaching for Pinecone to search fifty paragraphs is over-engineering; sticking with a NumPy array for ten million product descriptions is under-engineering. Knowing the thresholds is the skill.

How it works

• Create a collection, which is a named set of vectors that share one embedding model. Never mix models in a collection.
• Add documents with stable ids, their text, their vectors and metadata such as source, section or tenant. Re-adding the same id upserts, which is how you update.
• Query with a vector (or let Chroma embed the query text) and n_results. Optionally pass a where filter on metadata to narrow the candidates first.
• Under the hood an HNSW graph index finds approximate nearest neighbours in logarithmic time instead of scanning everything.
• Persist to disk with a persistent client so restarts do not re-embed the corpus.
• At scale, move to a server deployment (Chroma server, Qdrant, pgvector) so several app instances share one index.

Code example

import chromadb
from chromadb.utils import embedding_functions

# Persistent local store; the folder holds the index between restarts
client = chromadb.PersistentClient(path="./chroma_store")
embed = embedding_functions.SentenceTransformerEmbeddingFunction(
    model_name="all-MiniLM-L6-v2")

collection = client.get_or_create_collection(
    name="portfolio", embedding_function=embed,
    metadata={"hnsw:space": "cosine"})

paragraphs = open("knowledge_base.txt", encoding="utf-8").read().split("\n\n")
collection.upsert(                                  # upsert = add or update
    ids=[f"kb-{i}" for i in range(len(paragraphs))],
    documents=paragraphs,
    metadatas=[{"source": "knowledge_base.txt", "pos": i}
               for i in range(len(paragraphs))],
)

res = collection.query(
    query_texts=["experience with AWS and Docker"],
    n_results=3,
    where={"source": "knowledge_base.txt"},         # metadata filter
)
for doc, dist in zip(res["documents"][0], res["distances"][0]):
    print(round(dist, 3), doc[:90])

Where I used this

Use cases

• RAG over large or frequently updated document sets
• Multi-tenant assistants where each customer's data must be filtered separately
• Product and listing search with semantic matching plus attribute filters
• Long-term memory for agents, storing past interactions as vectors
• Image and audio similarity search using multimodal embeddings

Common mistakes

• Using a vector database for a tiny corpus. It adds a dependency, a persistence folder and a failure mode for no retrieval gain.
• Random ids on every ingest run, so each deploy duplicates the whole corpus and retrieval returns the same chunk three times.
• Mixing two embedding models in one collection after a model upgrade. Re-embed everything into a new collection and switch.
• Forgetting the distance metric. Chroma defaults to L2; set the space to cosine if that is what you evaluated with.
• Treating the vector store as the source of truth. Keep the original documents elsewhere; the index should be rebuildable.

Interview questions