Document Layout, Table Structure and Key Information Extraction
Models that use text, layout and image together to classify documents, find tables and pull fields from forms.
Tier Grades
Tier A: Frontier AI labs
feasible
Microsoft and other big labs set the LayoutLM line.
Tier B: Indonesian AI labs
feasible
Fine-tuning LayoutLM-class models on Indonesian forms fits an Indonesian lab.
Tier C: Institution units such as ITB
feasible
A 5-author team produced LayoutLMv3; a university unit can work on it.
Tier D: Organisations such as SEMAI (PoT, PoC, PoV)
feasible
Field extraction from invoices and claims is a common proof of value.
Where this comes from
Tiers: a) frontier AI labs, b) Indonesian AI labs, c) institution units such as ITB, d) organisations such as SEMAI delivering proof of technology, concept and value. Each rationale cites the anchors below.
Anchors
- ComputeTraining compute is not in the sources read.
- DataNot in the sources read.
- Team size (proxy)Proxy only (author count of an exemplar paper, not a headcount): 5 authors on the LayoutLMv3 paper (arXiv 2204.08387).
- Time horizonYears.
- Shelf-life signalsDocument VLMs absorb layout models, Multilingual layout models cover Bahasa Indonesia forms
- Pipeline roleTier A releases layout models; Tier C builds local form datasets; Tier D builds invoice and claims extraction pilots.
Where this comes from
Team size is a proxy (author counts of exemplar papers), not a headcount.
Who and What
Exemplars
Observed actors
Linked models
PP-DocLayoutV3OCR and documentexemplar
LayoutLMv3OCR and documentexemplar
DonutOCR and documentexemplarTable TransformerOCR and documentexemplar
Nemotron Parse 1.1OCR and documentproduced byLayoutLM document QA (Impira)OCR and documentproduced byDiT (Document Image Transformer)OCR and documentproduced byLayoutXLMOCR and documentproduced by
SDMG-ROCR and documentproduced byLayoutLMv2OCR and documentproduced byLayoutLMOCR and documentproduced by
LORE table structure recognitionOCR and documentproduced by
Unified Detector (text and layout)OCR and documentproduced by
PDF-Extract-Kit 1.0OCR and documentproduced by
Surya and MarkerOCR and documentproduced byLinks
R&D TopicDocument Layout, Table Structure and Key Information Extraction
Models 15
LayoutLMMicrosoft AI, OCR and documentproduced byLayoutLMv2Microsoft AI, OCR and documentproduced byLayoutLMv3Microsoft AI, OCR and documentexemplarDiT (Document Image Transformer)Microsoft AI, OCR and documentproduced byDonutNaver, OCR and documentexemplarLayoutXLMMicrosoft AI, OCR and documentproduced byNemotron Parse 1.1NVIDIA, OCR and documentproduced byPDF-Extract-Kit 1.0OpenDataLab, OCR and documentproduced bySurya and MarkerDatalab, OCR and documentproduced byTable TransformerMicrosoft AI, OCR and documentexemplarUnified Detector (text and layout)Google, OCR and documentproduced byLayoutLM document QA (Impira)OCR and documentproduced byPapers
- LayoutLM: Pre-training of Text and Layout for Document Image UnderstandingYiheng Xu et al., KDD 2020 (per arXiv comment), 2019 founding
- FUNSD: A Dataset for Form Understanding in Noisy Scanned DocumentsGuillaume Jaume et al., arXiv preprint, 2019 founding
- LayoutLMv3: Pre-training for Document AI with Unified Text and Image MaskingYupan Huang et al., arXiv preprint, 2022 state of the art
- DocLayNet: A Large Human-Annotated Dataset for Document-Layout AnalysisBirgit Pfitzmann et al., KDD 2022 (per arXiv comment), 2022 state of the art
- PubLayNet: largest dataset ever for document layout analysisXu Zhong et al., arXiv preprint, 2019 state of the art
Where this comes from
The papers to read first: where the direction began, its current best, and a survey when one exists.
Sources
- arxiv.org/abs/2204.08387 https://arxiv.org/abs/2204.08387
To cite this page: Project AURION, AI Use-Cases Research & Innovation, "Document Layout, Table Structure and Key Information Extraction", https://aurion.semai.id/topics/document-layout-and-key-information-extraction/, records of October 2026.