The Benchmark That Started a Fight
The original post compared MinerU 2.5, Granite-Docling, and PaddleOCR-VL. Commenters demanded their favorites get added. The sequel expanded to eight parsers: MinerU 2.5 (1.2B VLM), Granite-Docling (258M VLM), PaddleOCR-VL (0.9B VLM), XBerg 1.0 (text-layer parser), HURIDOCS PDLA, LiteParse, Chandra (Datalab’s OCR model), and LightOnOCR-1B. All the raw outputs, test files, and scripts are available in the pdf-parser-bench repository.
The results were decisive. Chandra swept all 14 capabilities. Perfect score. Zero failures. Not “pretty good” or “acceptable for production”, 14 out of 14 faithful extractions.
What “Perfect” Actually Means
Here’s where it gets interesting. We’re not talking about clean, digital PDFs with standard fonts. The benchmark included genuinely nasty documents: real merged-cell HTML tables, LaTeX equations (both display and inline), a 1909 cursive handwriting sample, and a page with an actual stain obscuring text.
Chandra handled all of it. The merged-cell tables came out as proper HTML. The LaTeX rendered correctly. The 1909 cursive, which killed every classical OCR tool, was read nearly perfectly. And on the stain, it did the thing that matters most: it skipped the unreadable section instead of guessing what was there.
That last point deserves emphasis. LightOnOCR-1B, which otherwise impressed for its size, wrote fluent, confident, entirely fabricated text over that stain. In a RAG pipeline, that’s not just a parsing error. That’s your system confidently retrieving information that doesn’t exist. The risks of over-relying on AI in technical workflows extend far beyond code reviews, they’re lurking in your document ingestion layer.
The Massacre in the Handwriting Column
The handwriting category exposed the fundamental divide between approaches. XBerg, LiteParse, and HURIDOCS PDLA returned nothing usable, cursive defeats classical OCR entirely. Granite-Docling leaked raw DocTags into its output, producing what can charitably be called “technical debt in text form.” PaddleOCR-VL read most of the content but invented an aristocratic “Maulevrier” where the document clearly said “Maude.” A hallucination, even a small one, corrupts downstream retrieval.
The broader OCR landscape shows similar gaps. AIMultiple’s DeltOCR Bench found that even top-tier solutions struggle with handwriting, GPT-5 leads at 95% accuracy, but the range spans from 46% to 95%. Printed media is even more competitive, with Gemini 2.5 Pro, Google Vision, and Claude Sonnet 4.5 all sharing the top score at 85%. Chandra’s performance on the 1909 cursive isn’t just beating competitors, it’s operating in a different class.
The Elephant in the Room: 91 Seconds Per Page
Now for the part that makes this benchmark feel less like a victory lap and more like a warning shot. Chandra took 91 seconds per page on an L4 GPU. LightOnOCR-1B managed 7.9 seconds per page on the same hardware.
The benchmark author was explicit about this caveat: Chandra belongs to a different class of model. This isn’t a fair comparison on speed. But fairness doesn’t matter when you’re building a production pipeline.
The math is brutal. If you’re ingesting 100,000 document pages, a realistic enterprise workload, Chandra at 91 seconds per page means over 105 days of continuous single-GPU processing. LightOnOCR-1B would finish the same job in about 9 days. The commenter who asked for speed and cost data in future benchmarks nailed the real issue: it’s entirely possible to get 10% better results using 1000% more compute, and that tradeoff matters when you’re paying the bill.
This tension connects directly to broader industry patterns. The quiet edge AI revolution is partly driven by this exact problem, smaller models that run efficiently locally are winning practical deployments even when larger models technically perform better. And the limitations of running powerful models on low-end hardware only compound the issue when you try to scale.
What Chandra’s Architecture Tells Us
The Chandra repository doesn’t reveal everything about the architecture, but the capability profile is telling. Perfect scores on LaTeX preservation, HTML table structure, and cursive handwriting suggest a model trained specifically on document understanding rather than general-purpose OCR. This isn’t a vision-language model that happens to read text, it’s a purpose-built parser.
That distinction matters for practitioners. General-purpose VLMs like PaddleOCR-VL and Granite-Docling are getting better at document tasks, but they still treat PDFs as images with text in them. Chandra apparently treats documents as structured artifacts with layout, typography, and semantic relationships that need preservation.
The “skip the stain” behavior is particularly revealing. Most OCR systems will guess when they encounter illegible content. Chandra’s ability to recognize that a section is genuinely unreadable and omit it, rather than hallucinate, suggests an uncertainty-aware architecture. For document pipelines where accuracy matters more than completeness, that’s arguably more valuable than raw extraction speed.
The Competitive Landscape Is Shifting
MinerU, the parser powering hexread’s production service, didn’t embarrass itself but didn’t dominate either. The benchmark positions it as solid mid-pack. XBerg’s CPU-only efficiency is attractive for specific use cases, but it collapses on anything that isn’t clean digital text.
The real story is the emergence of a clear quality hierarchy. At the top, you have purpose-built document parsers like Chandra that handle complex layouts flawlessly but demand serious GPU time. In the middle, you have efficient VLMs like LightOnOCR-1B that deliver good results at practical speeds but hallucinate under pressure. At the bottom, you have classical OCR pipelines that simply cannot handle the messy reality of real-world documents.
This is the same pattern playing out across AI infrastructure. The architectural challenges in distributed AI systems with open-weight models extend to parsing, you’re constantly choosing between model capability, computational cost, and deployment complexity. There’s no free lunch, but the menu just got more interesting.
Should You Switch Your Pipeline?
That depends entirely on what you’re processing. If your corpus is modern, digitally-generated PDFs with standard layouts, the smaller, faster parsers will serve you fine. LightOnOCR-1B’s 7.9 seconds per page with real LaTeX and clean pipe tables is genuinely impressive for its size.
But if you’re dealing with historical documents, academic papers with complex equations, or anything with legacy formatting, Chandra’s perfect score matters. A 91-second page cost is worth it when the alternative is silently corrupting every equation in your scientific corpus or hallucinating text in your legal document archive.
The benchmark author’s comment about Chandra being “in a different class” is accurate but incomplete. It’s not just a different class of model, it’s a different philosophy of document parsing. Instead of extracting text and hoping the structure survives, Chandra appears to understand the document as a whole. That’s the future of this space, and the speed will catch up.
The Takeaway
This benchmark doesn’t just tell us Chandra is good at parsing PDFs. It tells us the parsing quality gap between purpose-built models and general-purpose alternatives is far wider than most people assumed. And it tells us that hallucination, not raw accuracy, is the real enemy in document processing pipelines.
For teams building RAG systems, the lesson is uncomfortable but clear: your retrieval quality is capped by your parser’s ability to faithfully represent the source document. You can have the best embeddings, the most sophisticated reranking, and the perfect prompt template, but if your parser introduces errors before the data ever reaches the index, you’re building on a foundation of AI-generated confidence that has nothing to do with reality.
The benchmark repo is public. The scripts are there. Run the tests on your own documents before you commit to a parser. Because the next document that goes through your pipeline might be the one where a hallucinated word changes everything.