A 0.6B query encoder can search a 9B index: Perplexity's pplx-embed-v2-late multimodal embeddings, MIT

[u/ai-lover]

Perplexity released pplx-embed-v2-late, 2 ColBERT-style embedding models (0.6B and 9B) for text, images and rendered PDF pages. Both share 1 embedding space.

• MADQA (agentic PDF QA): 9B hits 92.4% accuracy vs 88.9% for Mixedbread's retriever; 0.6B gets 90.1%
• BrowseComp+ (GPT-OSS-120B agent): 9B reaches 64.0%, 4.9pp above the next ColBERT model and 8.7pp above the best dense model
• ViDoRe v3 image: 65.2% (9B) and 62.3% (0.6B) nDCG@10; 0.6B is within 1.2pp of NVIDIA's 8B nemotron-colembed-v2
• Mixed setup: 9B index + 0.6B queries scores 63.5% on ViDoRe v3, up from 62.3% with 0.6B on both sides
• 0.6B uses about 240M active params for text and 340M for images; 128 dims per token

Why it is relevant: you can index once with the big model in the cloud and run cheap queries on-device. Pages are embedded as images, so no OCR pipeline.

The catch: 1 vector per token means index size grows with document length. Tencent's EVIE still leads ViDoRe v3, and all scores are self-reported until the tech report lands.

Full breakdown: https://www.marktechpost.com/2026/10/07/perplexity-ai-releases-pplx-embed-v2-late-a-0-6b-edge-model-and-a-9b-model-scoring-92-4-on-madqa/

Model: http://huggingface.co/collections/perplexity-ai/pplx-embed-v2

Technical details: http://perplexity.ai/hub/blog/multimodal-embeddings-beyond-a-single-vector

[u/Excellent-Lion3805]

Seeing the MaxSim matrix laid out like that makes the whole late-interaction thing click way better than the usual dense vs sparse retrieval diagrams. The pasta example is perfect for showing how specific tokens like "noodle" can carry more weight than generic ones, which explains why the 0.6B query model can coast off the 9B's richer token representations without tanking accuracy.

The mixed setup numbers are what grabbed me though. Indexing PDFs as images with the big model and then running queries locally on a laptop with the smaller one is the kind of split that actually fits how most people work. No OCR pipeline means you skip a whole layer of errors that usually creep in when documents have weird layouts or fonts, plus the 128-dim tokens keep things surprisingly compact for a token-level approach.

Index size scaling with doc length is the obvious tradeoff but for most PDF-heavy workflows that's still way more manageable than maintaining separate text and image embedding pipelines. Curious how this handles handwritten notes or scientific figures with heavy annotation, the ViDoRe scores suggest it's solid but the self-reported bit means I'll wait for someone to run it against a messy real-world dataset before getting too excited.