OCR’s Memory Wall Just Crumbled: Why Page-by-Page Parsing Is Now a Legacy Pattern

The developers who chop PDFs into individual pages just to OCR them are running a workaround, not a solution. They know it. They feel it every time cross-page context gets lost, every time that for-loop spits out garbled output at the boundary between pages, every time a 30-page document takes longer to process than a 5-pager in a non-linear way.

The community has normalized this. Reset memory every page. Process in isolation. Glue the text back together. Pray it makes sense.

Unlimited OCR just made that whole approach look like a medieval torture device for throughput. The paper, quietly dropped by Baidu, doesn’t just improve OCR performance, it re-architects the core attention mechanism so that parsing 40+ pages in a single forward pass uses less memory than most models use for a single page.

Let’s dissect exactly how they pulled it off, because the implications extend well beyond OCR into any task where static reference inputs meet dynamic long-horizon generation.

The Fundamental Bottleneck Nobody Wanted to Admit

Every transformer-based OCR model you’ve used suffers from the same crippling flaw: the KV cache grows linearly with output length. Standard Multi-Head Attention (MHA) dictates that after generating T tokens, your cache size is L_prefill + T. For a 20-page document, that output might run 50,000 tokens. Your GPU memory goes along for the ride, ballooning until either the OOM killer intervenes or generation slows to a crawl.

The industry response has been to build janky orchestration layers. Chop the document. Process each chunk. Assemble. But this isn’t a solution, it’s a patch that introduces context gaps at every seam.

The human approach to copying a book is fundamentally different. We don’t re-read everything we’ve written. We glance at the immediate context and the source material. We maintain a continuous, bounded cognitive state. Baidu’s insight was brutally simple: why is nobody building attention mechanisms that mirror this?

R-SWA: The Mechanism That Makes KV Cache a Fixed Cost

Reference Sliding Window Attention is the architectural centerpiece here, and it’s elegant precisely because it changes almost nothing about how attention computes, only what it computes over.

The mechanism splits every generated token’s attention field into two segments:

1. The Reference Window (m): All visual tokens and the prompt. This is globally visible to every generated token and remains fixed. For a 20-page PDF using DeepEncoder’s Base mode, that’s roughly 5,120 tokens, compressed from 1024×1024 images down to 256 tokens per page.
2. The Causal Sliding Window (n): The preceding n output tokens only. Default is 128. This window slides forward as generation progresses, evicting the oldest token’s key-value pair with each new token.

The KV cache management maps directly to a FIFO queue of capacity m + n. Every new token evicts the (m+1)-th position. That’s it. No complex sparse data structures. No approximate attention. Just a fixed-size queue running standard scaled dot-product attention over a bounded context.

The cache ratio compared to standard MHA:

C_R-SWA(T) = L_m + min(n, T) ≤ L_m + n

When T ≫ n, this converges to approximately (L_m + n) / T, approaching zero as output length grows. For a 50,000 token output with 5,120 prefix tokens and a 128-token window, you’re looking at roughly 5,248 tokens of cache versus 55,120 under standard MHA.

That’s a 90.5% reduction.

The DeepEncoder: Why Visual Tokens Must Stay Static

DeepSeek OCR’s DeepEncoder was already impressive, cascading SAM-ViT’s window attention with CLIP-ViT’s global attention to achieve 16x token compression. A 1024×1024 PDF page compresses to 256 tokens. For multi-page processing, this compression is critical.

But there’s a subtle architectural choice that makes R-SWA work: visual tokens do not undergo state transitions. They are encoded once, before decoding begins, and remain static throughout the entire long-horizon process. This distinguishes R-SWA from linear attention variants like Mamba or RWKV, which apply recurrent state updates to all tokens, progressively blurring visual features.

The visual representation stays pristine because it never gets fed through the recurrent update path. The decoder tracks parsing progress purely through state transitions within the causal sliding window of generated text.

What the Benchmarks Actually Reveal

The numbers on OmniDocBench v1.5 tell a story that’s almost too clean:

A 6.22-point overall gain over the baseline. Text edit distance cut nearly in half. Formula recognition jumping 9.24 points. And this on a model that uses the same DeepSeek OCR weights, with only 4,000 steps of continued training on 2 million document samples, with the encoder frozen.

The subcategory analysis is where it gets interesting. Reading order edit distance on newspapers dropped from 0.217 (DeepSeek OCR) to 0.134. Notes improved from 0.089 to 0.018. These are precisely the document types with irregular, complex layouts where global attention tends to dilute focus across irrelevant spatial regions.

The phenomenon appears to be that bounded local attention forces the model to route information more efficiently through the sliding window. Rather than attending to distant tokens that might mislead, the model maintains a tighter, more coherent generative state.

The Speed Cliff: Why Constant TPS Matters for Production

The theoretical TPS comparison under ideal concurrency reveals the practical impact:

The crossover point is around 512 tokens. Below that, R-SWA’s overhead is negligible. Above it, the advantage compounds linearly with output length.

But the stability is the real production win. DeepSeek OCR’s TPS declines steadily, eventually showing a sharp spike when the KV cache length crosses alignment boundaries and data transfer efficiency drops. R-SWA’s latency is flat. Flat means predictable SLAs. Predictable SLAs mean tighter resource provisioning and more honest pricing.

For production systems handling millions of document pages daily, this isn’t a nice-to-have. It’s the difference between infrastructure that auto-scales gracefully and infrastructure that burns GPU hours unpredictably during peak loads.

The Open Architecture Shift

Baidu released the model and weights under an MIT license. The arXiv paper explicitly acknowledges DeepSeek OCR, DeepSeek OCR-2, and PaddleOCR. The inference setup supports both standard Transformers and SGLang, with the SGLang server configuration including the all-important custom logit processor for n-gram repetition control.

Launching the server:

python -m sglang.launch_server \
    --model baidu/Unlimited-OCR \
    --attention-backend fa3 \
    --page-size 1 \
    --mem-fraction-static 0.8 \
    --context-length 32768 \
    --enable-custom-logit-processor \
    --disable-overlap-schedule

The --mem-fraction-static 0.8 parameter combined with the bounded KV cache means you can provision GPU memory far more aggressively than with standard attention where cache growth is unpredictable.

For engineers looking to integrate this, the inference pattern for multi-page documents is refreshingly straightforward:

model.infer_multi(
    tokenizer,
    prompt='<image>Multi page parsing.',
    image_files=pdf_to_images('your_doc.pdf', dpi=300),
    output_path='your/output/dir',
    image_size=1024,
    max_length=32768,
    no_repeat_ngram_size=35, ngram_window=1024,
)

No chunking. No assembly. One shot, start to finish.

Where the Architecture Breaks

The current 32K context limits prefill length. At DeepEncoder’s 256 tokens per page, that caps input at roughly 125 pages before running out of prefix space. The long-horizon evaluation tested up to 40+ pages, with edit distance staying below 0.11.

But the paper’s stated roadmap includes 128K context training, which would push the practical limit well beyond most real-world document collections. The longer-term plan involves a prefill pool architecture, essentially teaching the model to fetch prefill KV chunks on demand, simulating human page-flipping behavior.

The other limitation is resolution. Multi-page mode uses 1024×1024 (Base mode). For documents with extremely small text or fine print, the single-page Gundam mode (dynamic resolution) is better suited. You can’t have both high resolution and massive page counts without trade-offs in the prefill stage.

The Generalization Thesis

R-SWA is not an OCR-specific hack. The paper explicitly identifies ASR, machine translation, and video captioning as natural application targets. Any task with static reference input and dynamic long-horizon output benefits from this pattern.

The implication is worth sitting with: for years, the industry has been scaling context length as a brute-force strategy. Build bigger caches. Process more tokens. Pay more memory. Baidu’s approach says the opposite, constrain the memory, bound the computation, and let the architecture’s efficiency speak for itself.

Alternative open-weight VLM models for structured document extraction have been pushing in similar directions, but none have solved the KV cache growth problem at the architectural level.

What This Means for the Pipeline

If you’re building document AI pipelines today, the calculus just changed. The standard advice has been to process documents page-by-page or use sliding window approaches that overlap chunks. Both introduce complexity around document boundaries, duplicate processing, and context discontinuity.

Open-source alternatives for complex document parsing pipelines have emerged, but they’ve been fighting the same fundamental constraint. R-SWA breaks that constraint at the mechanism level.

The production implication is that you can now build ingestion pipelines that treat an entire document as a single atomic unit of processing. Cross-page references? Handled. Reading order continuity? Baked in. Memory provisioning? Predictable regardless of document length.

Idempotency patterns for OCR pipelines handling document reprocessing become significantly simpler when there’s no chunking logic to reason about.

The Takeaway

Unlimited OCR is not the final word on document parsing. The 32K context cap is real, and the resolution trade-off between single and multi-page modes requires careful management.

But the architectural pattern matters more than the specific model. R-SWA demonstrates that bounded-memory attention isn’t just a compromise, it can outperform full attention on the same benchmarks while using a fraction of the resources. The model learned to maintain effective long-range coherence through a 128-token window because the architecture forced it to route information efficiently.

This is the direction efficient long-context AI has to take. Not scaling until your hardware breaks, but designing constraints that make scaling irrelevant.