WigtnOCR

Pseudo-label distillation compresses a 30B teacher's parsing intelligence into a 2B student via LoRA fine-tuning on 2,667 Korean government document pages, achieving #1 retrieval performance across 6 parsers.

Overview

A VLM-based document parser specialized for Korean government documents. This research project originated from a practical need to parse Korean government public documents for a B2G (government-facing) RAG service. In a B2G environment where the end user's document structure is unknown in advance, the only certainty was the domain: Korean government public documents. The goal was to build an SLM-based document parser that could accurately read and structure these documents while meeting practical infrastructure constraints (limited GPU, cost).

Beyond the "bigger is better" paradigm, the central question in AI engineering today is: how to efficiently transfer LLM intelligence into SLMs. We applied the knowledge distillation paradigm pioneered by Orca (Microsoft, 2023) — training an SLM to learn from an LLM's reasoning process — to the domain of Korean government document parsing.

By LoRA fine-tuning Qwen3-VL-2B-Instruct on 2,667 Korean government document pages, WigtnOCR matches or exceeds a 15x larger Teacher model (30B) in parsing performance and ranks #1 in retrieval across 6 parsers, validating the end-to-end causal chain: "structured parsing → improved chunking → better retrieval." Model weights, training data, and evaluation code are all released as open source.

Technical Challenges — Why Existing Parsers Fall Short

Pure OCR Limitations: Traditional OCR tools like PaddleOCR can recognize text but fail to understand document structure. In our evaluation, they extracted 3-30x less text than WigtnOCR and missed most tables, forms, and complex layouts in Korean government documents.

Rule-based Parser Limitations: Rule-based parsers like PyMuPDF4LLM offer fast text extraction but near-zero structural recognition. They cannot preserve the hierarchical structure of legal articles (sections/clauses/items) or handle mixed layouts of tables, diagrams, and text.

Latest VLM Parser Limitations: State-of-the-art VLM parsers such as dots.ocr (RedNote) and olmOCR (AI2) are primarily trained on English and Chinese documents. They are not optimized for Korean government documents, which feature complex tables, forms, stamps, mixed scanned/digital pages, and multi-column layouts.

30B Model Deployment Limitations: While 30B-parameter VLMs deliver excellent parsing quality, they require dual GPUs and have slow inference, making production deployment difficult. A 2B model can be served on a single GPU with fast inference and realistic edge deployment.

Pseudo-GT Pipeline

Stage 1 — Pseudo-GT Generation: PDF page images were fed to Qwen3-VL-30B-Instruct (Teacher) to generate structured markdown. A total of 4,501 pages were processed: 3,637 pages from 10 KoGovDoc documents + 864 pages from 39 arXiv papers. Initially, the 30B-Thinking model was used but produced unstable outputs (think tag contamination, token truncation), leading to a switch to the Instruct model. Finding: For document transcription, instruction-tuned models are more stable than reasoning models.

Stage 2 — GT Quality Verification: Qwen3.5-122B was used as a judge on a 5-point scale. Critically, evaluation was performed text-only without the original images — not asking "does this match the original?" but "is this output usable as training data?" This design prevents circular evaluation bias that would occur if a VLM evaluated another VLM's visual interpretation. A text-only LLM judge independently assesses structure, tables, completeness, hallucination, and consistency across 5 dimensions. Pass rate: KoGovDoc 75.1%, arXiv 73.8%. Pages scoring below 3 were excluded from training.

Stage 3 — Data Refinement: Document kogov_008 comprised 53% of all data — controlled via max_doc_ratio=0.25. Reasoning residue (English thought processes from the 30B-Thinking model) contaminating the GT was discovered — 20 pages deleted, 257 cleaned. Final dataset: train 2,667 + val 294 pages.

OmniDocBench Evaluation

Evaluated on OmniDocBench (CVPR 2025) — 1,355 PDF pages with human-annotated ground truth — comparing 4 models.

• •Text NED: Matches 30B Teacher (0.288 vs 0.289)
• •Table TEDS: Overall #1 — 0.649 (vs Teacher's 0.523, +12.6pp)
• •Reading Order: Exceeds 30B Teacher (0.211 vs 0.227)
• •vs Base 2B — Text NED +21%, Table TEDS +16%, Reading Order +30%
• •Student matches or exceeds 30B Teacher in 4/5 categories — validating pseudo-label distillation

Chunking Quality & Retrieval Evaluation

To verify whether "structured parsing actually produces better chunks," we evaluated 6 parsers using the BC/CS metrics from MoC (ACL 2025). Semantic chunking (BGE-M3) was used as the primary comparison strategy — both parsers chunk using the same method, with only the input text's structural quality differing, ensuring a fair comparison.

BC/CS quality doesn't guarantee retrieval performance. Stage 8 measures the final retrieval performance to complete the causal chain. Pipeline: Semantic chunking → BGE-M3 vectorization → FAISS search, evaluated on 564 queries.

• •WigtnOCR ranks #1 in Hit@1 (0.739), Hit@5 (0.855), and MRR@10 (0.788)
• •vs PaddleOCR: Hit@1 +22.7pp; vs 30B Teacher: +2.3pp
• •MinerU is BC/CS #1 but Retrieval #5 — chunk boundary quality doesn't predict retrieval; text richness and structural fidelity matter more for end-to-end RAG performance