Document AI: layout-aware extraction, OCR, and tables
Real documents are scans, PDFs, and forms — not clean text. Getting structured data out of them needs OCR, layout-aware models, and table extraction. Here's the stack before an LLM ever sees the content.
Before RAG retrieves anything or an LLM reasons about anything, something has to turn a messy document — a scan, a PDF, a form — into clean, structured text. That's Document AI, and in document-heavy domains it's a stack in its own right that quietly sets the ceiling on everything above it.
OCR: pixels to text
Scanned pages and images need OCR to recover the text and, importantly, its position on the page. Tesseract is the open-source workhorse; AWS Textract and Azure Document Intelligence are the managed options, and they go beyond characters to detect forms and tables directly. Position matters because layout is meaning — which the next step depends on.
Layout-aware models: text alone isn't enough
A document's meaning lives in its layout — headings, columns, a form's field-value pairs, a table's grid. Flatten it to a stream of text and you lose that. Layout-aware models like LayoutLM read the text together with its bounding boxes (and often the page image), so they understand structure a plain text model would erase. Toolkits like Docling and Unstructured wrap this into a pipeline that turns a PDF into typed structural elements.
Tables, figures, and formulas
The non-prose parts are the hardest and the most valuable. Table extraction recovers rows and columns as actual data rather than a jumble of cell text; figure and formula extraction pull out diagrams and equations. For drawings, chemical structures, and dense figures, multimodal models (VLMs) that see the image directly are increasingly the right tool.
Why it decides your RAG ceiling
Garbage in, garbage out: if extraction mangles a table or silently drops a form field, no amount of clever retrieval, reranking, or prompting downstream can recover it. In document-heavy work the extraction layer is where accuracy is won or lost — and it's why section-aware chunking (from the advanced-RAG post) depends on getting structure right here first.
Everyone obsesses over the model and the prompt. In document-heavy AI, the boring extraction layer at the bottom quietly sets the ceiling for everything above it.