Visual Document Retrieval
Retrieving document pages by embedding the page images themselves with a vision language model, instead of extracting text first.
Tier Grades
Tier A: Frontier AI labs
feasible
Google, NVIDIA, Alibaba and Cohere ship multimodal embedders that index page images.
Tier B: Indonesian AI labs
feasible
ColPali is trained from an open VLM with a benchmark built by a 7-author team, a scale an Indonesian lab could follow; local capacity is not in the sources.
Tier C: Institution units such as ITB
feasible
Building an Indonesian ViDoRe-style test set and fine-tuning ColPali is within an institution unit's reach (7-author paper).
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
ColPali is released with open models, data and code, so scanned-document RAG proofs of concept can use it directly.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeColPali is a vision language model producing multi-vector embeddings from page images; parameter count and compute are not in the abstract.
- DataViDoRe benchmark of page-level retrieval tasks across domains, languages and settings, released with ColPali.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 7 authors on ColPali.
- Time horizonMonths.
- Shelf-life signalsLate-interaction multi-vector retrieval (ColPali) versus single-vector multimodal embedders (Qwen3-VL-Embedding, Cohere Embed v4), RAG over scanned and visually rich documents without an OCR step
- Pipeline roleTier A and open labs release page-image embedders; Tier C builds Indonesian document test sets; Tier D applies them to RAG over scanned forms, reports and manuals.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
ColEmbed V2embeddingexemplar
jina-embeddings-v4embeddingexemplar
ColQwen2 v1.0embeddingexemplarColPaliembeddingexemplarjina-embeddings-v5-omni-nanoembeddingproduced byjina-embeddings-v5-omni-smallembeddingproduced by
Qwen3-VL-Embedding-2Bembeddingproduced byQwen3-VL-Embedding-8Bembeddingproduced by
BGE-VL familyembeddingproduced byQwen3-VL-Rerankerrerankerproduced byQwen3-VL-Reranker-2Brerankerproduced byQwen3-VL-Reranker-8Brerankerproduced byLlama Nemotron Embed VL 1B v2embeddingproduced byLlama Nemotron Rerank VL 1B v2rerankerproduced by
nomic-embed-multimodalembeddingproduced byColPali v1.2embeddingproduced by
Cohere Embedembeddingproduced byEmbed English v3.0embeddingproduced byEmbed Multilingual v3.0embeddingproduced byEmbed v4.0embeddingproduced byEmbed v5.0 Proembeddingproduced by
Gemini Embedding 2embeddingproduced byLinks
R&D TopicVisual Document Retrieval
ModelsColEmbed V2NVIDIA, embeddingColPaliIlluin Technology, embeddingColQwen2 v1.0Illuin Technology, embeddingjina-embeddings-v4Jina AI, embedding+18 more
ColEmbed V2NVIDIA, embeddingColPaliIlluin Technology, embeddingColQwen2 v1.0Illuin Technology, embeddingjina-embeddings-v4Jina AI, embedding+18 moreUse Casesnone recorded
Who Runs Itnone recorded
Models 22
Cohere EmbedCohere, embeddingproduced byColPaliIlluin Technology, embeddingexemplarQwen3-VL-RerankerAlibaba (Qwen), rerankerproduced byColPali v1.2Illuin Technology, embeddingproduced byColQwen2 v1.0Illuin Technology, embeddingexemplarEmbed English v3.0Cohere, embeddingproduced byEmbed Multilingual v3.0Cohere, embeddingproduced byEmbed v4.0Cohere, embeddingproduced byEmbed v5.0 ProCohere, embeddingproduced byGemini Embedding 2Google, embeddingproduced byQwen3-VL-Embedding-2BAlibaba (Qwen), embeddingproduced byQwen3-VL-Embedding-8BAlibaba (Qwen), embeddingproduced by+10
Qwen3-VL-Reranker-2BAlibaba (Qwen), rerankerproduced byQwen3-VL-Reranker-8BAlibaba (Qwen), rerankerproduced byBGE-VL familyBAAI, embeddingproduced byColEmbed V2NVIDIA, embeddingexemplarjina-embeddings-v4Jina AI, embeddingexemplarjina-embeddings-v5-omni-nanoJina AI, embeddingproduced byjina-embeddings-v5-omni-smallJina AI, embeddingproduced byLlama Nemotron Embed VL 1B v2NVIDIA, embeddingproduced byLlama Nemotron Rerank VL 1B v2NVIDIA, rerankerproduced bynomic-embed-multimodalNomic AI, embeddingproduced byPapers
- Unifying Multimodal Retrieval via Document Screenshot EmbeddingXueguang Ma et al., EMNLP2024 (per arXiv comment), 2024 founding
- ColPali: Efficient Document Retrieval with Vision Language ModelsManuel Faysse et al., ICLR 2025 (per arXiv comment), 2024 state of the art
- VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality DocumentsShi Yu et al., arXiv preprint, 2024 state of the art
- M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document UnderstandingJaemin Cho et al., arXiv preprint, 2024 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2407.01449 https://arxiv.org/abs/2407.01449
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Visual Document Retrieval", https://aurion.semai.id/topics/visual-document-retrieval/, records of October 2026.