Document Format Support

TLDR

In the 2024-2025 AI landscape, document format support has evolved from simple character extraction to Intelligent Document Processing (IDP). The primary objective is no longer just "reading text" but "preserving semantic structure" to feed Retrieval-Augmented Generation (RAG) pipelines. Modern engineering favors toolchains like Docling and Unstructured.io, which convert legacy formats (PDF, DOCX) into "LLM-ready" Markdown or JSON. By utilizing Vision-Language Models (VLMs) like LayoutLMv3, developers can now maintain hierarchical metadata, complex table structures, and correct reading orders, ensuring that Large Language Models (LLMs) receive contextually accurate data.

---

Conceptual Overview

The historical approach to document support was rooted in Optical Character Recognition (OCR) and basic stream parsing. While effective for digitizing archives, these methods often produced "alphabet soup"—a flat string of text stripped of its visual and structural context. For a human, a multi-column PDF is easy to navigate; for a traditional parser, the text from the left column often intermingles with the right, destroying the logical flow.

The Semantic Gap in Document Parsing

The "Semantic Gap" refers to the loss of meaning that occurs when a visually structured document is flattened into plain text. In the context of LLMs, this gap is catastrophic. LLMs rely on the proximity and hierarchy of tokens to infer relationships. If a table's headers are separated from its data, or if a footnote is injected into the middle of a paragraph, the model's reasoning capabilities are severely compromised.

Modern Intelligent Document Processing (IDP) seeks to bridge this gap by treating documents as structured data objects rather than flat files. This involves:

1. Layout Analysis: Identifying the spatial coordinates of text blocks, images, and tables.
2. Reading Order Recovery: Determining the sequence in which a human would naturally read the content, especially in complex layouts like scientific journals or financial reports.
3. Structural Tagging: Assigning semantic roles (e.g., H1, caption, list_item) to extracted elements.

The Rise of LLM-Ready Formats

The industry has converged on Markdown as the preferred intermediary format. Markdown is token-efficient, human-readable, and retains enough structural markers (hashes for headers, pipes for tables) to guide an LLM's attention. By converting a PDF into structured Markdown, we provide the LLM with a "map" of the document, allowing it to distinguish between a primary argument and a tangential sidebar.


---

Practical Implementations

Building a production-grade ingestion pipeline requires moving beyond simple libraries like PyPDF2. Modern pipelines integrate specialized parsers that handle the heavy lifting of layout reconstruction.

Leading Toolchains: Unstructured and Docling

Two frameworks currently dominate the open-source ecosystem:

1. Unstructured.io: This framework uses a "partitioning" logic. It breaks documents into "elements" (e.g., Title, NarrativeText, Table). It is highly versatile, supporting over 20 file types, including .eml, .msg, and .pptx. Its modularity allows developers to swap out OCR engines (like Tesseract or PaddleOCR) depending on the document's complexity.
2. Docling (IBM Research): Docling is a specialized, high-performance tool specifically optimized for PDF-to-Markdown conversion. It excels in speed and accuracy for technical documents. Unlike general-purpose parsers, Docling focuses on the "LLM-ready" output, ensuring that tables are reconstructed into clean Markdown syntax that RAG systems can easily index.

The Conversion Workflow

A standard implementation follows these steps:

• Step 1: Normalization. Converting various inputs (DOCX, HTML, PDF) into a unified internal representation.
• Step 2: Layout Parsing. Using heuristic or ML-based models to detect headers, footers, and page numbers (which are often discarded to prevent noise in RAG).
• Step 3: Table Extraction. This is the most difficult stage. Tools must recognize cell boundaries and spanning rows/columns to produce a valid CSV or Markdown table.
• Step 4: Metadata Enrichment. Injecting source-level data (e.g., document_id, page_number, section_title) into the extracted text chunks.
• Step 5: Validation via Comparing prompt variants (A). Once the text is extracted, it is critical to validate the quality. This is done by Comparing prompt variants (A) to see how the LLM performs on the extracted data. For instance, one might test a summarization prompt on "flat text" vs. "structured Markdown" to quantify the improvement in accuracy provided by the IDP pipeline.

Code Example: Basic Partitioning with Unstructured

from unstructured.partition.pdf import partition_pdf

# Partitioning a complex PDF into semantic elements
elements = partition_pdf(
    filename="financial_report.pdf",
    strategy="hi_res", # Uses layout model for better accuracy
    infer_table_structure=True,
    chunking_strategy="by_title", # Semantic chunking
)

# Exporting to Markdown for LLM consumption
for element in elements:
    print(f"Type: {element.category} | Text: {element.text[:50]}...")

---

Advanced Techniques

As documents become more "visual-first" (e.g., infographics, complex dashboards), traditional parsing reaches its limit. Advanced techniques leverage the power of computer vision and deep learning.

Vision-Language Models (VLMs) and LayoutLMv3

LayoutLMv3 represents a breakthrough in document AI. Unlike traditional models that only look at text, LayoutLMv3 uses a unified architecture to process text and image patches simultaneously. It employs "Spatial Awareness," where the model learns the relative positions of words on a page.

When a VLM like Gemini 2.5 Pro or GPT-4o processes a document, it doesn't just "read" the characters; it "sees" the bolded text, the proximity of a caption to an image, and the hierarchical indentation of a list. This multimodal approach is essential for:

• Complex Forms: Understanding the relationship between a label and an input field.
• Nested Tables: Correctly associating data points in multi-layered financial statements.
• Mathematical Notation: Preserving the structure of equations that are often garbled by standard OCR.

Semantic Chunking Strategies

Traditional RAG pipelines often use "Fixed-Size Chunking" (e.g., every 500 characters). This is a significant anti-pattern. If a chunk boundary falls in the middle of a sentence or a table row, the context is severed.

Semantic Chunking uses the document's structure to define boundaries. A chunk should ideally represent a single "logical unit," such as:

• A single section (from one H2 header to the next).
• A complete table and its preceding descriptive paragraph.
• A list and its introductory sentence.

By aligning chunks with the document's natural hierarchy, the retrieval engine can return more coherent context to the LLM, reducing hallucinations and improving the "Faithfulness" metric in RAG evaluation.

---

Research and Future Directions

The future of document format support is moving toward a "Machine-Readable-by-Default" web and enterprise ecosystem.

The /llms.txt Standard

A new proposal, /llms.txt, suggests that websites should provide a simplified, Markdown-based version of their content at a standardized path. This is analogous to robots.txt but designed for LLM crawlers. Instead of forcing an LLM to parse complex HTML/JS, the site provides a clean, structured text map, significantly reducing the compute required for data ingestion and improving the quality of the "world knowledge" available to models.

Domain-Specific Languages (DSLs) and DeonLang

In specialized fields like law or compliance, even Markdown is sometimes insufficient. The rise of DeonLang and similar DSLs allows for the representation of legal nuances—such as obligations, permissions, and prohibitions—in a format that is both human-readable and logically verifiable by an AI. This suggests a future where "Document Support" includes the ability to translate natural language documents into formal logic.

Native Multimodal RAG

We are approaching an era where the "Extraction" step may be bypassed entirely. In Native Multimodal RAG, the system indexes the raw images of document pages. During retrieval, the model "looks" at the page image and the text simultaneously. This eliminates the errors introduced during the conversion to Markdown/JSON, though it currently requires significantly higher computational resources and specialized vector databases capable of storing multimodal embeddings.

. Column 2: Modern (IDP, Markdown, Semantic Chunking, LayoutLMv3). Column 3: Future (Native Multimodal, /llms.txt, DSLs, Direct Image Indexing). The rows compare 'Context Preservation', 'Computational Cost', and 'Accuracy'.)

---

Frequently Asked Questions

Q: Why is Markdown preferred over JSON for LLM context?

While JSON is excellent for programmatic data handling, Markdown is often more "token-efficient" for LLMs. Markdown uses fewer structural characters (like braces and quotes) to convey hierarchy, allowing more actual content to fit within the model's context window. Additionally, LLMs are extensively pre-trained on web data (Markdown/HTML), making them naturally adept at interpreting its structure.

Q: How do I handle multi-column layouts in PDFs?

Standard parsers read left-to-right across the entire page, mixing columns. To solve this, you must use a tool that performs Layout Analysis (like Unstructured's hi_res strategy or Docling). These tools identify columns as separate bounding boxes and sequence the text within each box before moving to the next, preserving the logical reading order.

Q: What is the role of "Comparing prompt variants" (A) in extraction?

Comparing prompt variants (A) is a validation technique. After extracting text from a complex format, you run the same task (e.g., "Summarize this table") using different prompt structures or different versions of the extracted text. If the LLM succeeds with the structured Markdown version but fails with the plain-text version, you have empirical proof that your extraction pipeline is adding value.

Q: Can LLMs handle raw images of documents instead of text?

Yes, multimodal models like GPT-4o and Gemini 1.5 Pro can process images directly. However, this is expensive and slow for large-scale RAG. Most production systems still prefer extracting text to Markdown for the "Retrieval" phase and only use the raw image for the final "Generation" phase if high visual fidelity is required.

Q: What is the best way to extract tables from scanned documents?

For scanned documents, a hybrid approach is best. Use an OCR engine (like Amazon Textract or Azure Document Intelligence) that specifically supports table structure recognition. These services return a JSON object representing the table grid, which you can then convert into a Markdown table for your LLM pipeline.