Overview
Wei, Sun, Li (DeepSeek-AI, arXiv:2510.18234, Oct 2025) — Investigates compressing long text contexts via optical 2D mapping. Text is rendered as images, then compressed into far fewer vision tokens than the equivalent text tokens.
Architecture
- DeepEncoder: Core compression engine maintaining low activations under high-resolution input
- Decoder: DeepSeek3B-MoE-A570M processes the compressed vision tokens
Key Results
| Compression Ratio | OCR Precision |
|---|---|
| <10x | 97% |
| 20x | ~60% |
- Surpasses GOT-OCR2.0 (256 tokens/page) using only 100 vision tokens
- Outperforms MinerU2.0 (6000+ tokens/page average) with <800 vision tokens
- Production throughput: 200k+ pages/day on a single A100-40G
Implications
- Historical long-context compression: Could compress entire document histories into manageable token budgets
- Memory forgetting mechanisms: Lossy compression naturally implements forgetting
- LLM training data: Can generate training data at scale from document images
Related
- simplemem-lifelong-memory — Complementary approach to context management (semantic compression vs. optical)
- compact-guide-to-rag — RAG systems that could benefit from optical context compression