Embeddings & Vector Search
Turn text into searchable vectors
🔑 Key Concepts
- What embeddings are — Dense vector representations where semantically similar text is close in vector space. 1536 dimensions for OpenAI embeddings.
- Generating embeddings — client.embeddings.create(model='text-embedding-3-small', input=[texts]). Returns a list of float vectors.
- Cosine similarity — dot(a,b) / (norm(a) * norm(b)). Ranges from -1 to 1. Higher = more similar. The standard metric for semantic search.
- ANN indexes — Exact search is O(n). Approximate Nearest Neighbours (HNSW, IVF) scales to millions of vectors with minimal quality loss.
💡 Practice: Try implementing each concept yourself before moving on. Reading about RAG and building RAG are very different things.