- Published on
A 3.4B MoE Document Parser With Built-In Speculative Decoding for Low-Budget GPUs
- Authors
- Name
- DeepHorus
Jina AI has released jina-ocr-v1, a visual document parser that converts PDFs, scans, tables, charts and invoices into clean Markdown in a single pass. The model, built by the Elastic-backed company, targets low-budget GPUs such as the NVIDIA L4. It combines a 3.4B-parameter mixture-of-experts architecture with built-in speculative decoding. The open weights are about 6.8 GB in BF16 and run on Transformers or vLLM.
The model post-trains DeepSeek-OCR and keeps its two efficiency components. DeepEncoder uses SAM, a 16x convolutional compressor and CLIP-L to turn a 1024Γ1024 page from 4,096 patches into 256 visual tokens. A dynamic-resolution mode adds up to nine local tiles at 100 tokens each, capping a page at 1,156 visual tokens. The decoder is DeepSeek-3B-MoE with 12 layers, 64 routed experts and two shared experts, activating about 570M parameters per token.
Jina AI reports scores of 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR-Bench. The license is CC BY-NC 4.0, meaning commercial use requires contacting Jina AI. For research and non-commercial deployment, the model is available now.
The FastMTP speculative decoding head applies one dense draft block recursively for three steps. The decoder verifies drafts greedily and accepts the longest matching prefix, committing 2.73 tokens per step on average. Because the committed text always equals plain greedy decoding, the speedup is lossless.
Throughput is the main result. On one A100 40 GB at concurrency 32, jina-ocr-v1 parses 2.57 pages per second, the highest of 14 systems measured, versus 1.22 for olmOCR-2 and 0.38 for chandra-ocr-2. On an NVIDIA L4 at batch size 1, eager decoding rises from 42.7 to 83.1 tokens per second, a 1.95x speedup. With CUDA graphs, K=1 delivers 185.6 tokens per second.
The model does not lead on accuracy; PaddleOCR-VL-1.6 and HunyuanOCR-1.5 score higher on OmniDocBench, while chandra-ocr-2 and dots.mocr score higher on olmOCR-Bench. Post-training adds 7.4 points over the DeepSeek-OCR backbone on olmOCR-Bench. Jina AI emphasizes that its output is the shortest among systems scoring above 83.
Researchers can deploy jina-ocr-v1 through Jina Reader or directly on their own hardware. The company says the design makes high-quality OCR feasible on budget GPUs. Commercial users will need a separate agreement with Jina AI.
