What is the best way to cache embeddings at scale in 2027?

By Kory White · Updated Nov 2026 · 6 min read

Rows of small wooden card-catalog drawers with brass label holders and pulls

Direct answer

The documented pattern is a key-value cache keyed on the model and the exact text, placed in front of the embedding call, so each unique piece of text is embedded once per model. LangChain's CacheBackedEmbeddings and RedisVL's EmbeddingsCache both work this way, and both treat the model name as part of the identity of an entry.

At scale, four decisions matter: how the key is built (model namespace plus a strong hash), where entries live (a shared store such as Redis rather than process memory), how long they live (TTL), and how you fill the cache (batches). Storage cost for the vectors themselves is a separate problem, with its own tools. Vendor docs are our sources; there is no single benchmark that crowns one design, so we label our own reading as judgment.

What an embedding cache is, and what it is not

An embedding cache stores the vector an embedding model returned for a piece of text, so the next request for the same text skips the model call. RedisVL's guide describes its EmbeddingsCache as storing and retrieving embedding vectors with their associated text and metadata, useful for applications that frequently compute the same embeddings. LangChain's reference describes CacheBackedEmbeddings as an interface for caching results from embedding models.

This is different from a semantic cache, which stores whole LLM responses and matches new prompts by similarity. Redis points to that as a separate guide, listed under Cache LLM Responses. An embedding cache only matches identical text, so it returns exactly what the model would have returned.

Judgment: because the match is exact, an embedding cache has no accuracy trade-off to tune. The only tuning is hit rate, cost and freshness.

Why it pays: cost and latency

OpenAI's embeddings guide says usage is priced per input token. Its table, assuming about 800 tokens per page, lists 62,500 pages per dollar for text-embedding-3-small, 9,615 for text-embedding-3-large and 12,500 for text-embedding-ada-002. Every repeated page of text that you embed again is spend you could skip.

Redis lists the use cases where this helps: search applications caching frequently searched queries, recommendation systems, API services that generate embeddings through paid APIs, batch processing of datasets with duplicate texts, and chatbots caching common user queries. Its guide's own benchmark shows 0.4564 seconds without caching and 0.0619 seconds with caching, reported as 7.37x faster. The guide's summary separately cites 15-20x faster with batch operations.

Not settled: those two speedups come from a small demo on one machine and do not agree with each other. Treat them as a sign of direction, not a number to plan capacity with. Measure your own hit rate and latency.

The key: model plus text, hashed with a strong function

LangChain's from_bytes_store reference gives a namespace parameter that avoids collisions with other caches and says to set it to the name of the embedding model used. That matters because the same text produces a different vector under a different model. RedisVL's entries carry a model_name field alongside the content and the embedding for the same reason.

The same page covers hashing. The default key encoder uses SHA-1, and the docs warn that SHA-1 is not collision-resistant, so a motivated attacker could craft two different texts that hash to the same cache key. They advise new applications to use one of the alternatives (blake2b, sha256 or sha512) or a custom strong encoder. They also advise creating a new cache if you change the key encoder on an existing one, to avoid collisions or duplicate keys for the same text.

Judgment: the collision warning matters most when users can submit text that later becomes a shared cache key. For a purely internal corpus the risk is lower, but picking a strong encoder on day one is free, while changing it later means rebuilding the cache.

Where to keep it: a shared store, with TTLs

LangChain's CacheBackedEmbeddings works with any store that accepts string keys and lists of floats, so the backing store is your choice. For a fleet of workers, a shared network store is what lets every worker see the same cache. Redis's guide lists a running Redis instance (Redis 8+ or Redis Cloud) as the prerequisite for its cache.

RedisVL supports a global time-to-live set when the cache is created and a per-entry TTL override. It also provides batch methods (mset, mget, mexists, mdrop) and async versions of all methods for async applications. Expiry is how you bound the size of the cache and how you retire entries you no longer want.

Judgment: the sources do not say what TTL to use. A reasonable starting point is to match it to how long a text stays in your corpus, and to rely on the model-in-the-key rule rather than TTL for model upgrades.

Documents versus queries, and batching

LangChain's reference says that by default only document embeddings are cached; to cache query embeddings too, you pass a query store (or set query_embedding_cache to True to share the document cache). Its batch_size parameter is the number of documents to embed between store updates, so a long indexing job writes to the cache as it goes instead of only at the end.

Documents. Corpus text is embedded once and re-embedded only when it changes, so this is where repeated spend sits in an indexing pipeline with duplicate or unchanged text.
Queries. Redis names frequently searched queries and common chatbot queries as cache candidates. Opt in to that cache deliberately, since LangChain leaves it off by default.
Batches. Use the batch methods (mget to check, mset to write) so a job with thousands of texts does not make one network round trip per text.

Judgment: the document-versus-query split is our reading of defaults. Whether query caching is worth it depends on how often your users repeat the same query.

Shrinking what you store: dimensions and quantization

A cache of vectors still costs memory. OpenAI's guide says the default vector length is 1536 for text-embedding-3-small and 3072 for text-embedding-3-large, and that a dimensions parameter can reduce it. If you shorten vectors, treat the shortened vector as a different embedding setup in the cache key, since the cached and live vectors must match.

For the vector database that holds the embeddings for search, Qdrant's quantization guide lists scalar quantization (float32 to 8-bit integer, 4x compression with minimal accuracy loss), binary quantization (up to 32x) and product quantization (up to 64x). It suggests keeping the original vectors on disk and the quantized vectors in RAM, and re-running its accuracy comparison on your own collection before fixing parameters.

Judgment: quantization is a search-index decision, not a cache decision. Cache the full-precision vector if you may want to re-quantize later, and verify recall on your own data.

A practical reading, labeled as judgment

What follows is our synthesis, not a procedure any one source gives. Build the key from the model name, any dimension setting and a strong hash of the exact text. Put the entries in a shared store with a TTL that reflects how long the text stays relevant. Fill and read in batches. Cache documents first, and add queries only after you see repeats in real traffic. When you change the embedding model, let the new namespace start empty rather than mixing vectors.

What this page does not claim: it does not say what hit rate you will see, which store is fastest, or that one vendor's cache is best. The sources are product documentation, each describing its own tool, and none compares them.

Questions people ask

What is the best way to cache embeddings at scale?

The pattern in LangChain's and Redis's documentation is a key-value cache in front of the embedding call, keyed on the model and the text, stored in a shared durable store, with TTLs and batch reads and writes. No source we read benchmarks designs against each other.

What should the cache key contain?

LangChain says to set the namespace to the name of the embedding model used, and hash the text with a key encoder. Its default is SHA-1, which it warns is not collision-resistant; it lists sha256, sha512 and blake2b as alternatives.

Does LangChain cache queries too?

Not by default. Its reference says only document embeddings are cached unless you pass a query embedding store or set query_embedding_cache.

Can I set an expiry on cached embeddings?

In RedisVL, yes: a global TTL when you create the cache, and a per-entry override. Entries can also be dropped in batches.

How much does caching save?

It depends on how often your text repeats. Redis's own demo reports 0.4564 seconds uncached against 0.0619 seconds cached, 7.37x faster, but that is a small demo. OpenAI prices embeddings per input token, so every skipped repeat avoids that spend.

How do I reduce the memory the vectors take?

Qdrant documents scalar quantization at 4x compression, binary at up to 32x and product at up to 64x. OpenAI's dimensions parameter shortens vectors at the source. Test recall on your own data either way.

Sources

LangChain reference: CacheBackedEmbeddings · Default behavior, store interface, query caching, batch_size
LangChain reference: CacheBackedEmbeddings.from_bytes_store · Namespace, key encoders, SHA-1 warning
Redis docs: RedisVL Cache Embeddings guide · EmbeddingsCache fields, TTL, batch and async methods, benchmark, use cases
OpenAI API docs: Vector embeddings · Per-token pricing, pages per dollar, default dimensions
Qdrant docs: Quantization · Scalar, binary and product quantization compression figures
Hero photo: "Wooden Card Catalog Furniture" by MarkBuckawicki, CC0, Wikimedia Commons · Credit for the photo at the top of this page

Facts were read from the linked pages on November 7, 2026. These are product documentation pages, each describing its own tool, and benchmarks in them are small demos. Sections marked as judgment are our reading, not a source claim.

Top photo: MarkBuckawicki / CC0 via Wikimedia Commons