WTF is Embeddings?
Lists of numbers that represent the meaning of text (or images) so similar things end up close together.
Also called: embedding, vector embeddings
TL;DR
An embedding turns "How do I reset my password?" into a vector of, say, 1,024 numbers. "I forgot my login" lands nearby even though it shares no words. That's what makes semantic search, RAG, clustering and recommendations work.
How it works
An embedding model maps input to a fixed-size dense vector; similarity is usually measured with cosine similarity or dot product. Vectors are stored in a vector index for nearest-neighbour search. Practical concerns: chunk size, the embedding model's domain fit, dimensionality versus cost, and re-embedding everything when you change models, since vectors from different models aren't comparable.
Origin
- Coined / introduced by
- Mikolov et al. (word2vec) popularised learned word embeddings
- When
- 16 Jan 2013
- Story
- Learned vector representations of words became mainstream with word2vec; modern models embed whole passages.