Quick answer: Vector embeddings are lists of numbers that represent the meaning of a piece of text (or an image, or audio) as a point in high-dimensional space. Text with similar meaning gets mapped to nearby points, so you can measure how related two pieces of text are by measuring the distance or angle between their vectors. This is what makes semantic search, recommendations, and RAG retrieval possible.
If you have ever wondered how a search box finds the right document even when you did not use any of the words in it, embeddings are the answer. They convert language into geometry, and geometry is something computers are very good at comparing.
From text to vectors
An embedding model takes text and outputs a fixed-length array of floating point numbers, for example 384, 768, or 1536 of them depending on the model. That array is the vector. Each dimension does not map to a human-readable concept, but taken together the vector encodes meaning learned from training on huge amounts of text.
The key property: meaning becomes position. "How do I reset my password" and "I forgot my login credentials" share almost no words, but a good embedding model places their vectors close together because they mean nearly the same thing. Keyword matching would miss that entirely.
# Conceptually, an embedding is just a list of floats
"reset my password" -> [0.021, -0.144, 0.087, ..., 0.006] # 768 numbers
"forgot my login" -> [0.019, -0.139, 0.091, ..., 0.004] # very close
"best pizza in Chicago" -> [-0.201, 0.330, -0.052, ..., 0.180] # far away
How similarity works
Once text is vectors, "how related are these two things" becomes "how close are these two points". The standard measure is cosine similarity: the cosine of the angle between two vectors. It ranges from -1 (opposite) through 0 (unrelated) to 1 (identical direction). It ignores magnitude and looks only at direction, which is what you want for meaning.
import numpy as np
def cosine_similarity(a, b):
a, b = np.array(a), np.array(b)
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
# Higher = more semantically similar
print(cosine_similarity(reset_vec, forgot_vec)) # ~0.9
print(cosine_similarity(reset_vec, pizza_vec)) # ~0.1
Some setups use Euclidean distance or dot product instead. If your vectors are normalized to unit length, cosine similarity and dot product rank results identically, which is why many vector databases normalize on ingest and then use the cheaper dot product.
| Metric | Measures | Notes |
|---|---|---|
| Cosine similarity | Angle between vectors | Most common for text, ignores magnitude |
| Dot product | Angle and magnitude | Equals cosine when vectors are normalized |
| Euclidean (L2) | Straight-line distance | Sensitive to magnitude, less common for text |
What models produce embeddings
You do not train an embedding model yourself. You call one. The landscape splits into hosted APIs and open models you run yourself.
| Type | Examples | Trade-off |
|---|---|---|
| Hosted API | OpenAI, Cohere, Voyage embedding endpoints | Easy, pay per token, data leaves your box |
| Open weights | sentence-transformers, BGE, E5 families | Free to run, you own the infra |
The practical rule: pick one model and use it for both indexing and querying. Vectors from different models live in different spaces and are not comparable. If you re-embed your corpus with a new model, you must re-embed your queries too.
How embeddings power search and RAG
Embeddings are the retrieval engine underneath semantic search and RAG. The flow is the same in both:
- Embed every document (or chunk) ahead of time and store the vectors.
- At query time, embed the query with the same model.
- Find the stored vectors closest to the query vector.
- Return those documents (search) or feed them to an LLM as context (RAG).
# Indexing
chunks = ["...", "...", "..."]
vectors = model.encode(chunks) # each chunk -> a vector
store.add(vectors, metadata=chunks)
# Querying
q_vec = model.encode(["how do I reset my password"])[0]
hits = store.search(q_vec, top_k=5) # nearest neighbors by cosine
for h in hits:
print(h.score, h.text)
At scale you do not compare the query against every vector one by one. Vector databases use approximate nearest neighbor indexes (HNSW and friends) to find close matches fast, trading a tiny bit of accuracy for a large speed gain.
Common pitfalls
Embeddings are easy to misuse in ways that quietly wreck retrieval quality.
- Mismatched models. Indexing with one model and querying with another produces garbage. Same model, always.
- Forgetting to normalize. If your metric assumes unit vectors and you skip normalization, similarity scores get distorted. Normalize on ingest if your store expects it.
- Wrong chunk size. Embedding a 10,000-word document as one vector blurs many topics into a single point, so nothing matches well. Embedding single sentences loses context. A few hundred tokens per chunk is the usual sweet spot, and the strategy is worth care: see what are chunkers and text chunking.
- Dimension mismatch. Your vector store's configured dimension must match the model's output size, or inserts fail. A 768-dim index will reject 1536-dim vectors.
- Assuming embeddings understand your jargon. General models may not separate domain terms that look similar. Test on your own data before trusting the scores.
Where clean web data comes in
The quality of your embeddings is capped by the quality of the text you embed. Feed in HTML full of navigation menus, cookie banners, and ad markup, and those tokens pollute every vector, pulling unrelated pages together because they share boilerplate. Clean, structured text produces cleaner vectors and sharper retrieval.
That is why the ingest step matters. When you pull web pages into an embedding pipeline, get them as clean Markdown first:
curl https://link.sc/v1/fetch \
-H "Authorization: Bearer lsc_..." \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/docs/guide", "format": "markdown"}'
Then chunk and embed the result. Garbage in, garbage vectors out, so the cleanup is not optional.
The takeaway
Vector embeddings turn meaning into coordinates, and coordinates are comparable. That single move unlocks semantic search, recommendations, and the retrieval half of RAG. Pick one model, use it consistently, mind your dimensions and normalization, chunk sensibly, and feed it clean text. Get those right and the geometry does the rest.
Feeding web content into an embedding pipeline? Get clean, structured text from any URL with link.sc.