Why Small-Parameter OCR Multimodal Models Can Achieve High-Accuracy Document Parsing

1. OCR Multimodal Models Solve More Than “Character Recognition”

Traditional OCR mainly focuses on recognizing text in images: given an image as input, it outputs the text content and the location of that text. In real business scenarios, however, users are often not satisfied with merely “recognizing the characters.” They expect the system to understand the structure of the entire document, for example:

• Which text belongs to headings and which belongs to body text;
• What row-column relationships exist in tables;
• How images, captions, and footnotes correspond to one another;
• In what order multi-column layouts should be read;
• How formulas, code, stamps, and handwritten content should be represented;
• How to distinguish headers, footers, tables of contents, and body text in scanned documents.

Therefore, modern document parsing systems usually need to perform text recognition, layout analysis, reading-order recovery, table structure reconstruction, formula recognition, and image-text semantic understanding at the same time. Such systems can be regarded as multimodal document understanding models that combine OCR with vision-language models.

In terms of task form, the input is no longer just a local text region but a page or even an entire document image. The output is not only a character sequence but also structured text, coordinate information, table markup, heading hierarchy, and other layout elements. In other words, the system needs to answer two questions:

1. What is written in the image?
2. What role does this content play in the document?

The second question is often harder than the first.

2. What a Public Case Demonstrates

A public research case used a small multimodal document parsing model with about 1.2 billion parameters and achieved a score of 95.69 on a public document parsing benchmark. Its previous version scored 92.98, an improvement of 2.71 points.

From a horizontal comparison, the model’s total score exceeded that of several general-purpose vision-language models with much larger parameter counts. Some large models have tens of billions or even hundreds of billions of parameters, yet on the vertical task of document parsing, their scores were lower than that of the small specialized model.

On a difficult-sample subset, the small model scored 94.08, while the second place scored about 92.01, a lead of more than 2 points. This result is particularly noteworthy because complex documents often contain low-resolution scans, dense tables, multi-column layouts, special fonts, complex formulas, and mixed languages. Performance on hard samples often reflects practical usability better than the overall score on ordinary samples.

These numbers do not mean that parameter scale is unimportant. Rather, they show that in vertical scenarios with clear task boundaries, high-quality data, and relatively stable output formats, training data and task design may be more decisive than blindly increasing model size.

3. Performance Bottlenecks May Come from Data, Not Architecture

Many model optimization efforts begin by considering more parameters, a different network architecture, or a longer context window. For document parsing tasks, however, model performance is also directly affected by the following factors:

• Whether training samples cover real document types;
• Whether annotations are accurate and structurally consistent;
• Whether table, formula, and layout information is fully preserved;
• Whether hard samples have been specifically constructed;
• Whether the training objective aligns with the final output format;
• Whether the training data contains many duplicate, low-quality, or incorrect samples.

If a model has never seen complex cross-page tables, adding parameters does not necessarily enable it to reconstruct table structures reliably. If training labels mix headings, body text, and footnotes together, the model will struggle to learn reliable hierarchical relationships. If training data mainly comes from clean electronic documents, performance may degrade noticeably when the model is deployed on low-quality scans.

Therefore, the optimization direction for document parsing systems can be summarized as follows:

Make training data closer to real input, make labels closer to the final output, and make difficult cases appear more fully during training.

4. Four Key Stages of Data Engineering

4.1 Build a Comprehensive Data Classification System

Document data should not be divided simply into “images” and “text.” It should be stratified according to actual parsing difficulties, for example:

• Ordinary printed documents;
• Multi-column layout documents;
• Table-intensive documents;
• Documents with many formulas and special symbols;
• Low-clarity scanned documents;
• Documents with complex backgrounds or watermarks;
• Documents with mixed languages;
• Long documents and cross-page documents;
• Documents with mixed images and text, or dense captions.

Classification is not only for counting data volume. More importantly, it helps analyze where the model fails. Only after failure types are clearly identified can data be supplemented in a targeted way.

4.2 Improve Annotation Quality and Structural Consistency

Document parsing labels are usually not simple plain text but sequences with structure. For example, an abstract form like the following can be used:

<title>Document title</title>
<body>First paragraph of body text</body>
<table>
  <row><cell>Field One</cell><cell>Field Two</cell></row>
</table>
<formula>Formula content</formula>

Real projects may not use the exact tags above, but the core idea is the same: let the model learn the mapping from visual content to structured results.

Annotation specifications must be unified. In particular, the following questions should be clarified:

• How should heading levels be represented;
• How should row spans and column spans in tables be handled;
• Whether headers and footers should be output;
• How images and captions should be associated;
• How line breaks, spaces, and punctuation should be normalized;
• What representation should be used for formulas;
• How unrecognizable content should be marked.

If different data sources use different rules for the same type of content, the model will learn conflicting patterns during training. This can manifest as unstable output formats, confused structural tags, or large differences in parsing results for the same page across multiple runs.

4.3 Strengthen Hard Samples and Boundary Samples

Ordinary samples help the model learn basic capabilities, but difficult samples often make the real difference. Common hard samples include:

• Sticky, blurry, and skewed characters;
• Missing table lines or excessively dense lines;
• Dense layouts with small font sizes;
• Text colors that are close to the background color;
• Occlusion by complex stamps;
• Images, text, and tables appearing in an interleaved manner;
• Unclear reading order in two-column or multi-column layouts;
• Cross-page tables and broken paragraphs;
• Multiple languages and symbols on the same page.

Hard samples should not appear only in the test set. They should enter the training set through data augmentation, manual curation, and feedback from failure cases. Otherwise, a model may score highly on ordinary samples but degrade quickly when encountering real business documents.

4.4 Perform Data Deduplication and Quality Filtering

Large-scale data does not equal high-quality data. Too many duplicate samples can cause overfitting, and incorrect labels can directly contaminate the training objective.

Common data governance processes include:

1. Perform near-duplicate detection on images and text;
2. Remove samples that cannot be opened, are severely damaged, or have empty content;
3. Detect illegal nesting and format errors in labels;
4. Check whether images and annotations correspond one-to-one;
5. Count distributions across categories, resolutions, and languages;
6. Downsample high-frequency templates;
7. Upsample scarce but important scenarios.

The goal of data governance is not simply to reduce data volume but to increase the density of effective information.

5. Why Small Models May Be More Advantageous in Vertical Tasks

Small specialized models usually have three advantages.

First, the output objective is clearer. A document parsing model does not need to cover all open-domain knowledge. Instead, it is optimized around recognition, layout, and structured output, so its task scope is more concentrated.

Second, inference costs are lower. A smaller parameter count means memory usage, latency, and deployment costs are easier to control, making the model suitable for batch document processing, edge-device deployment, or high-concurrency services.

Third, the training objective is easier to align. General-purpose models need to handle dialogue, reasoning, knowledge question answering, image understanding, and many other tasks, whereas a specialized model can concentrate more training resources on tables, formulas, reading order, and structural labels.

However, small models are not superior in every task. If a scenario requires complex knowledge reasoning, cross-document analysis, long-chain question answering, or open-domain image understanding, large general-purpose models may still have advantages. A reasonable engineering approach is often not a simple either-or choice. Instead, specialized models handle high-frequency, standardized parsing, while stronger general-purpose models handle a small number of complex reasoning tasks.

6. Evaluation Should Not Rely on a Single Total Score

Evaluation of a document parsing system should be broken down into at least the following dimensions:

• Text recognition accuracy;
• Coordinate or region localization accuracy;
• Layout classification accuracy;
• Reading-order recovery accuracy;
• Table structure matching;
• Formula recognition accuracy;
• Document-level structural completeness;
• Output format validity rate;
• Inference latency and resource usage.

Total scores can be used for horizontal comparison but cannot replace component-level analysis. For example, a model may be strong at text recognition but unable to reconstruct tables correctly. It may also achieve accurate table structures while having a high error rate on low-clarity scans.

In addition, the evaluation set should include both ordinary samples and difficult samples, with stratified statistics by business scenario. Before deployment, validation should also include real internal enterprise data, because public data and production data may differ significantly in scan quality, template distribution, language types, and privacy handling.

7. A Layered Architecture Is Recommended for Deployment

A maintainable document parsing system can usually be divided into the following modules:

1. Input preprocessing layer: perform rotation correction, resizing, image enhancement, and page segmentation.
2. Visual encoding layer: extract visual features such as text, regions, lines, and layout.
3. Multimodal parsing layer: jointly understand image content and spatial relationships.
4. Structured output layer: convert results into a unified text, table, or markup format.
5. Quality validation layer: check tag closure, table column counts, page order, and completeness of key fields.
6. Human review layer: sample and correct low-confidence or high-risk documents.
7. Data feedback layer: feed error cases back into the training and evaluation system.

The quality validation layer is especially important. Even if the model recognizes content largely correctly, incomplete structural tags, misaligned table columns, or reversed reading order can still affect downstream retrieval, question answering, and data ingestion.

8. Conclusion

The development of OCR multimodal models shows that document intelligence is not just about “making models bigger.” In relatively well-defined tasks, a carefully designed data system, unified annotation specifications, sufficient hard-sample training, and rigorous evaluation breakdowns can enable small-parameter models to reach a very high level of specialization.

For developers, three lessons are especially worth reusing:

• First establish data classification and error analysis systems, then decide whether to expand model scale;
• Optimize tables, formulas, layout, and reading order as independent capabilities;
• Measure systems using real business documents and end-to-end structured results, rather than looking only at character recognition rates.

Ultimately, the competitiveness of a high-performance document parsing system often comes from the coordination of models, data, evaluation, and engineering pipelines, not from competition over a single parameter count.