- Published on
Perplexity Releases pplx-embed-v2-context-9b-preview: A Contextual Embedding Model That Retrieves Answers and Their S...
- Authors
- Name
- DeepHorus
Perplexity Research and turbopuffer have released pplx-embed-v2-context-9b-preview, a contextual embedding model built for retrieval-augmented generation pipelines. Unlike conventional retrievers that return a single "gold passage," the model embeds each chunk with the full document in view and learns to surface the answer together with the context needed to verify it.
The release is deployable today as a self-hosted preview: weights are available on Hugging Face under the MIT license, and loading requires transformers>=5.4.0 with trust_remote_code=True. The model is not yet accessible through the Perplexity API, and the model card warns that weights and interface may change without backward compatibility.
The motivation addresses a known weakness in RAG. Long documents are split into chunks, and chunks frequently depend on entities, headings, or definitions stated elsewhere, while traditional training marks only one chunk as correct and treats every other chunk as a negative β including sentences that make the answer checkable. Perplexity also cites coarse binary labels, LLM annotation costs that grow linearly with dataset size, and labels tied to a single chunking strategy.
The training signal is the core innovation. A query-aware context compression model acts as teacher, scoring every token in the query and document together; those scores are pooled per chunk as the mean of the top n tokens and converted into a temperature-scaled softmax target, optimized via forward KL divergence plus an InfoNCE document loss inspired by ColBERT's MaxSim. Each batch samples a random chunking strategy, and the teacher runs only during training, adding no inference latency or storage.
The model builds on an in-house 9B ColBERT retrieval model, projecting to 2048 dimensions with Matryoshka training supporting 1024 dimensions, plus quantization-aware training enabling native int8 embeddings. The release is a model soup trained on roughly 430 datasets covering over 50 languages, with no ConTEB data.
Evaluation uses context-bench, a privately held turbopuffer benchmark of 2,099 queries, 38,894 documents, and 2,458,072 sentence chunks with exhaustive ranking. Perplexity reports its 1024-dim int8 (1 KB) embeddings slightly exceed voyage-context-4 at 2048-dim float32 (8 KB) on its chunk-retrieval suite, signaling possible cost savings for production deployments.
For RAG practitioners, the preview suggests flexible chunk boundaries without re-annotation and cheaper, lower-latency retrieval β though preview status means benchmarks and interfaces may still shift.
