Why Multimodal Vision Arrives Late: Vision Token Costs and Engineering Paths

    This article explains why services positioned around low pricing tend to delay their vision capabilities in multimodal models. It breaks down why vision tokens are expensive and how they can be compressed, walks through the engineering costs and strategic trade-offs behind vision features, and closes with practical guidance on budgeting and architecture for building cost-efficient vision applications—covering tiered encoding, caching, fallback, and observability.

    1. The Phenomenon: Why Do Low-Priced Model Services Arrive “Late”?

    On August 21, 2026, a large-model service known for extreme low pricing finally launched its first vision-understanding API. This release had three notable details:

    • The name carried an experimental suffix, hinting that the capability was still iterating rapidly;
    • It supported only images, not video;
    • Weights were not open, and access was API-only.

    Around the same period, comparable products moved at different paces:

    • Some vendors made their flagship model a native omni-modal model from day one;
    • Some followed the conventional “language first, multimodal later” order, with flagship models focused on text and code, while vision was handled by a separate multimodal series;
    • Others filled in text, image, and video all at once.

    Why is there such a large gap in the timing of multimodal capability among top-tier vendors? A common explanation is “insufficient technical reserves.” But looking at the patterns of technical evolution and product positioning, this explanation is too simplistic. A judgment closer to the essence is: for a service positioned around low price, the difficulty of vision is not “whether it can be done” but “whether the cost can still be borne after it’s done.” This determines three things: launch timing, launch form, and openness strategy.

    This article first breaks down why vision tokens are expensive and how they can be compressed, then analyzes the engineering costs and strategic trade-offs, and finally provides cost-budgeting and architecture-design practices for developers.

    2. Vision Tokens: The Cost “Amplifier” of Multimodal Inference

    2.1 Why One Image Turns into Thousands of Tokens

    When mainstream multimodal models process an image, they generally go through three steps:

    1. Split the image into regular small blocks (patches), commonly 14×14 or 16×16 pixels;
    2. Have a vision encoder (usually a vision Transformer) encode each patch into a vector;
    3. Through one or more projection layers, map it into the language model’s word-embedding space and concatenate it into the text token sequence for joint computation.

    From this, the number of tokens an image occupies is approximately determined by the following formula:

    1. Image vision tokens ≈ ⌈image width / patch side⌉ × ⌈image height / patch side⌉ ÷ downsampling ratio

    For a 1024×1024 image with a patch size of 14, a single image is 73×73 ≈ 5,300 tokens. Add positional enhancement or multi-scale slicing schemes, and a single high-resolution image can easily reach tens of thousands of tokens. If a single request includes multiple images, or processes high-resolution long images or document screenshots, the token count stacks up linearly.

    2.2 The “Multiplication Effect” of Vision Tokens

    These vision tokens participate in prefill and attention computation together with text tokens. The computation of Transformer self-attention grows quadratically with sequence length, and long sequences bring significant memory and latency overhead. For an API service billed per token, the impact of vision tokens runs through the entire pipeline:

    • Encoder inference: the vision encoder must run once per patch;
    • Sequence concatenation: long sequences raise attention computation costs;
    • Cache overhead: every vision token must retain its corresponding Key/Value in the KV cache, and multi-turn conversations keep accumulating them.

    More critically is the competitive dimension. Vision capability is often used for differentiation: for the same feature, whoever is cheaper has the better cost-performance ratio. Once inference costs spiral out of control because of vision, the cost-performance advantage is quickly eroded.

    2.3 First Quantify Costs with an Estimator

    Rather than staying at qualitative judgment, it’s better to write costs as a computable model. The following is example code to estimate the number of vision tokens per image at different resolutions and compression ratios, along with the relative cost:

    1. import math
    2. PATCH = 14 # Vision encoder patch side length, commonly 14 or 16
    3. def vision_tokens(width: int, height: int, compress: int = 1) -> int:
    4. """Estimate the number of vision tokens for a single image.
    5. compress is the downsampling ratio, meaning every compress patches
    6. are fused into 1 vision token.
    7. Does not account for extra info such as slicing or watermarks; actual
    8. values follow the server side."""
    9. grid_w = math.ceil(width / PATCH)
    10. grid_h = math.ceil(height / PATCH)
    11. return max(1, (grid_w * grid_h) // compress)
    12. def compare(width: int, height: int):
    13. base = vision_tokens(width, height) # no compression
    14. print(f"Resolution {width}x{height}")
    15. for c in (1, 4, 16, 64): # 1, 2x2, 4x4, 8x8 fusion
    16. n = vision_tokens(width, height, c)
    17. print(f" Compression {c:>2} -> {n:>6} tokens, "
    18. f"vision encoding overhead down to {n / base:6.2%}")
    19. if __name__ == "__main__":
    20. compare(1024, 1024)
    21. compare(3000, 4000) # original photo taken by a phone

    The output is as follows:

    Resolution Compression ratio Vision tokens Vision encoding overhead (relative to no compression)
    1024×1024 1 5358 100%
    1024×1024 4 1339 25.00%
    1024×1024 16 334 6.23%
    1024×1024 64 83 1.55%
    3000×4000 1 61321 100%
    3000×4000 4 15330 25.00%
    3000×4000 16 3832 6.25%
    3000×3000 64 958 1.56%

    These numbers show:

    • A phone’s original photo, uncompressed, exceeds 60,000 vision tokens for a single image;
    • After 4×4 (16×) downsampling, it drops to under 4,000, less than 7% of the original.

    If there are millions of requests per day averaging 10,000 tokens per image, the vision portion alone must process tens of billions of tokens per day. This also explains why low-priced model providers treat “compression” as the entry threshold for multimodal capability rather than an optimization item after launch.

    3. How Can Vision Tokens Be “Saved”?

    Vision token compression mainly follows five technical routes. The first four focus on “within a single inference,” while the last focuses on “cross-request reuse”:

    1. Control resolution and slicing strategy: not every image is worth high-resolution processing. First do low-resolution global encoding, then apply local high-resolution slicing to text-dense regions, letting the model “zoom in to see details” on demand. This balances small-text recognition and overall understanding while avoiding token bloat from whole-image high resolution.
    2. Vision token downsampling and aggregation: through structures such as pooling and cross-attention resampling, fuse many adjacent patches into a few representative tokens. The original pixel information is already compressed at the middle layers, so the loss to the model is limited.
    3. Sparse and linear attention: let each token interact only with a local window or a small number of global tokens, reducing attention’s quadratic complexity to near-linear.
    4. Visual primitivization: instead of indiscriminately feeding the whole image into the model, first extract structured primitives such as points and bounding boxes through detection or segmentation, then use these primitives as “reasoning-capable tokens” in the chain of thought. The object the model thinks about changes from “the whole image” to “what things are in the image, where they are, and what relationships they have.”
    5. Cross-request caching and reuse: build an embedding cache keyed on the hash of the same image, reuse the already-encoded visual representation across multi-turn conversations, and avoid repeated computation.

    Route 4 deserves separate explanation. According to a public study’s description, a 756×756 image, after being processed by this scheme, retains only about 81 representations in the vision KV cache. Compared with the common “over a thousand tokens per image” scheme, the token magnitude differs by about tenfold. Its compression ability comes from two points:

    • Visual primitives retain only task-relevant objects, rather than uniformly encoding all pixels;
    • Sparse attention further compresses the information redundancy between tokens.

    The cost is equally obvious, elaborated in Section 4 below. It should be emphasized that these routes are not mutually exclusive; engineering practice usually combines them, for example “resolution slicing + token downsampling + sparse attention + cache reuse.”

    4. Why “Saving Money” Is Itself Heavy Engineering

    4.1 Visual Primitivization Requires Reconstructing the Chain of Thought

    Traditional vision tokens are “passive input”—the model processes whatever it receives. Visual primitivization requires the model to learn a whole new way of reasoning: first determine which objects are needed and where they are in the image, then reason around those coordinates and bounding boxes.

    This brings changes on two fronts:

    • The chain of thought must be redesigned: how structured information such as coordinates, boxes, and points enters the reasoning chain—the model must first “locate” and then “judge”;
    • Training data must be specially constructed: it requires a large number of image-reasoning pairs with object-detection annotations (bounding boxes, keypoints).

    So the model not only answers “what’s in the image” but also “in which region, based on what clues, this conclusion is reached.” This localizable, verifiable reasoning better fits engineering needs such as autonomous-driving scene analysis, industrial quality-inspection localization, and document field extraction—but its data requirements far exceed general image-text pairs.

    4.2 Building Your Own Data Pipeline: Long Cycle, High Cost

    According to the same public study’s description, to support this route, the research team specifically crawled about 100,000 object-detection datasets, keeping only about 30,000 after two rounds of filtering. This funnel itself illustrates the difficulty of data engineering:

    1. Raw collection ~100,000 object-detection datasets
    2. │
    3. ├─ Round 1 filtering: format compliance, complete annotations, license compliance
    4. ▼
    5. ├─ Round 2 filtering: annotation quality, category coverage, difficulty distribution
    6. ▼
    7. Usable training samples ~30,000+

    By comparison, the traditional multimodal route can directly reuse off-the-shelf vision encoders and image-text pretraining results, with a much shorter cycle. A self-built pipeline is slower, but in exchange offers complete control over data distribution and reasoning form.

    4.3 Evaluation and Regression Must Also Keep Up

    After vision capability goes live, the evaluation set must cover dimensions such as OCR, chart understanding, spatial relations, fine-grained recognition, and hallucination rate. Every compression-strategy adjustment requires a new round of regression to avoid “saving tokens but losing accuracy.” This is an ongoing cost that further extends the launch cadence.

    5. Strategic Trade-offs: Agent First, Vision as a “Loop-Closed Patch”

    Beyond cost, product positioning also affects the launch order of multimodal capability.

    Vendors that make multimodal the flagship selling point usually bundle vision, audio, and video capabilities together as core differentiation for external promotion. Agent-centric teams, on the other hand, position vision as the “perception layer,” with a clear purpose: to let the Agent “see the results of its own execution.” Typical scenarios include:

    • GUI operation: recognize interface state from screenshots and determine whether a click took effect;
    • Form and document processing: verify filled content and extract key fields;
    • Result validation: run visual regression checks on generated visualizations.

    In this kind of architecture, vision is a link in the Agent’s execution loop, not an independent product selling point. So vision arriving somewhat later does not affect the mainline product iteration; conversely, the maturity of Agent capability will also force vision capability to be completed.

    This “perceive-decide-execute” loop can be described with pseudocode:

    1. def run_agent_with_visual_feedback(task, max_steps=5):
    2. state = init_state(task)
    3. for step in range(max_steps):
    4. # Text-only reasoning: plan the next action
    5. plan = llm.plan(task, state.history)
    6. if plan.is_finished:
    7. return state.result
    8. # Execute action: call tools, operate the interface, etc.
    9. obs = execute(plan.action)
    10. # Vision perception: screenshot/image -> structured description
    11. # Optionally do low-resolution global encoding first, then
    12. # local upscaling on key regions
    13. visual = vision_model.describe(obs.screenshot, focus=plan.focus_regions)
    14. # State update: write the visual observation back into the
    15. # context for the next round of decision-making
    16. state.update(observation=visual, action=plan.action)
    17. # Reflection and retry
    18. if visual.has_error and step < max_steps - 1:
    19. state.append_reflection(visual.error_hint)
    20. raise AgentMaxStepsReached(task)

    This loop has several engineering points:

    • Vision descriptions must be structured: output decidable fields such as “whether a button appeared, error text, form validation prompts” rather than a piece of natural-language description, so it is easy for programs to consume;
    • Introduce validation and retry: vision judgments can be wrong, so confidence thresholds and a maximum retry count are needed as safeguards;
    • The context must be lean: write only key regions and structured conclusions into history to avoid unbounded accumulation of vision tokens.

    6. Engineering Practice: How to Design Low-Cost Vision Multimodal Applications

    For enterprise developers, rather than agonizing over when a given vendor launches its vision API, it is better to focus on “how to make vision affordable and effective within your own business.”

    6.1 Budgeting: Allocate Vision Tokens by Scenario

    Scenario Typical input Resolution strategy Vision tokens per image Optimization focus
    Document OCR and field extraction Scans, contracts, receipts Local high-resolution slicing Hundreds to thousands Layout analysis localization + regional slicing
    Chart/data-visualization understanding Screenshots, report charts Medium global resolution Hundreds Structured extraction over pure vision
    General image Q&A User-uploaded photos Default medium resolution Hundreds to thousands Downsampling + cache reuse
    GUI Agent observation Consecutive operation screenshots Low resolution + upscaled key regions Tens to hundreds Incremental screenshots, structured vision descriptions
    Fine-grained recognition (defects, parts) Industrial camera originals High-resolution slicing Thousands Rule/small-model pre-filtering, then large-model review

    The budgeting method is simple:

    1. Cost per request ≈ (text tokens + vision tokens) × input unit price + output tokens × output unit price
    2. Daily cost ≈ cost per request × daily requests × (1 + retry rate)

    Note that retries and multi-turn conversations bill vision tokens repeatedly—which is exactly where the value of cache reuse lies.

    6.2 Tiered Vision: Coarse First, Then Fine

    Do not treat all images the same. A recommended “coarse filter + fine inspection” two-tier architecture:

    1. Image input
    2. │
    3. ├─ Low-resolution global encoding -> determine task type and key regions
    4. │ │
    5. │ ├─ Global info sufficient -> answer directly (low cost)
    6. │ └─ Details needed -> high-resolution slicing of key regions -> fine reasoning

    This approach keeps the average number of vision tokens at a level matching task complexity, rather than paying for the worst case.

    6.3 Caching and Reuse

    • Hash images and cache vision embeddings, reusing them directly across multi-turn conversations;
    • Merge multiple questions about the same image into a single request to avoid repeated encoding;
    • For structured documents, extract text and tables first, then hand them to the language model; use vision only for layout understanding.

    6.4 Degradation and Fallback

    • When the vision service is unavailable, degrade to text-only processing or human review;
    • Set up a human review queue for low-confidence results;
    • Use a rules engine for high-certainty judgments (for example, “does a red alert box exist”), reserving the model for ambiguous judgments.

    6.5 Full-Chain Observability

    Monitoring metrics should include at least:

    • Vision token share and cost per request;
    • Accuracy comparison before and after compression;
    • Hit rate and retry rate per scenario;
    • Distribution of vision-related errors.

    These metrics should be connected to a unified logging service and alerting system, so that both cost anomalies and accuracy regressions can be detected in time.

    7. Conclusion: Cheapness Is a Capability, Not Just a Price

    Back to the original question: why do model services positioned around low price delay multimodal capability? The answer can be summarized in three layers:

    1. Cost layer: vision tokens are inherently expensive, and the low-price positioning dictates that compression must be pushed to the extreme before daring to open up;
    2. Engineering layer: the visual-primitivization route requires reconstructing the chain of thought and building your own data pipeline, with a cycle far longer than “plugging in an encoder”;
    3. Strategic layer: Agent-first teams treat vision as a loop-closed patch rather than a flagship selling point, so the cadence naturally differs.

    For developers, the real takeaway is not “who launches first” but: the value of multimodal capability does not depend on how many modalities it supports, but on whether it can reliably complete tasks at a given unit cost. Whoever can compress vision tokens to a level matching business value can go further in multimodal applications. Low price itself is a capability that must be designed.