How to choose an embedding model (criteria, not names)
The right embedding model is the one that works best on your data, not the one at the top of a leaderboard. Here's the decision by criteria — evaluate on your own golden set, weigh dimensionality against cost, and mind the rest.
'Which embedding model should I use?' has a boring, correct answer: the one that scores best on your data. Leaderboards like MTEB are a starting shortlist, not a decision — a model that tops a general benchmark can underperform on your legal or technical corpus. So the skill isn't naming a model; it's knowing the criteria to judge by.
Evaluate on your own golden dataset
The one criterion that dominates all others: build a small golden set of real queries paired with the documents that should be retrieved for them, and measure retrieval quality per candidate model — Hit Rate (did the right doc make the top k?) and MRR (how high did it rank?). This is the same evaluate-retrieval-separately discipline from that post, applied to model selection. A weekend building this eval set saves you from shipping a model that benchmarks well and fails on your actual queries.
Dimensionality vs cost
Bigger embedding vectors can capture more, but every dimension is storage and every comparison is compute — at millions of vectors, dimensionality is a real bill (the vector-search-internals post). Higher-dimensional isn't automatically better for your task; measure whether the extra dimensions actually lift your Hit Rate before you pay for them. Some models support Matryoshka representation, where a single embedding can be truncated to fewer dimensions with graceful degradation — so you can store a shorter vector and only pay for full dimensionality where it earns its keep.
The rest of the checklist
- Multilingual needs — if your content or queries span languages, you need a multilingual model, and you must eval it per-language, not just in English.
- Domain fit — a general model may miss the vocabulary of a specialised corpus; test domain-tuned options against it on your golden set.
- Hosted vs self-hosted — a hosted embedding API is easy but sends your text to a third party (a dealbreaker for confidential data) and meters every call; a self-hosted open model keeps data in your boundary and caps cost at your hardware. For sensitive corpora, self-hosting is often the deciding factor, not accuracy.
- Consistency — you must embed queries and documents with the same model; changing the model means re-embedding everything, so treat the choice as somewhat sticky.
Don't pick an embedding model from a leaderboard — pick it from your own Hit Rate on your own queries, then weigh dimensionality, languages, and where your data is allowed to go.