Why Vision Models Arrive Late: How Low-Cost LLMs Complete Their Multimodal Stack

    The delayed release of vision capabilities from cost-focused LLM providers is often a deliberate engineering decision rather than a sign of weak technology. This article examines why visual tokens are expensive, how structured visual representations and sparse attention can reduce inference costs, and why affordable image understanding is especially valuable for closing the execution-observation-correction loop in AI Agents.

    1. An Industry Pattern Worth Examining

    Over the past year, large language models have followed noticeably different paths toward multimodality. Providers pursuing an “all-in-one flagship” strategy often introduced image understanding—and sometimes video understanding—early, making these capabilities a central launch feature. By contrast, major language model providers built around extreme cost efficiency, whose users primarily choose them for low inference prices, have generally lagged in vision.

    Even when these providers eventually release vision-enabled APIs, the models are often labeled experimental. Their capabilities may be limited to static images, with video support unavailable and model weights kept closed.

    It is easy to interpret this pattern as a sign of weak technology or slow execution. However, the inference-cost accounting tells a different story. The delay is often a deliberate technical decision: visual tokens are expensive, while low pricing is central to these providers’ market position. Adding vision without controlling its cost could undermine the entire product strategy.

    This article examines the issue from three technical perspectives: why visual tokens are expensive, which compression approaches the industry is exploring, and what role vision actually plays in Agent applications.

    2. The Cost of Visual Tokens: Why Images Consume So Many Resources

    2.1 One Image Can Equal Thousands of Text Tokens

    Most vision-language models use a combination of a vision encoder, a projection layer, and a language model backbone. A standard-resolution image—for example, one around 768×768 pixels—is divided into patches by the vision encoder and can easily produce hundreds or thousands of visual tokens.

    High-resolution images, multiple-image inputs, or configurations designed to preserve fine-grained information for OCR-level tasks can push the count into the thousands or even tens of thousands.

    More importantly, these visual tokens do not affect only the input stage. During autoregressive generation and long-context inference, they remain in GPU memory as part of the KV cache.

    • Memory usage: KV-cache size grows linearly with the number of tokens. A large visual-token footprint directly reduces the number of concurrent requests a service can handle.
    • Inference latency: Attention computation is sensitive to sequence length, so more visual tokens increase prefill latency.
    • Billing cost: In token-based pricing models, a single image can cost several times—or even tens of times—the equivalent amount of textual information.

    2.2 The Deadlock Created by a Low-Cost Positioning

    For a model provider whose core value proposition is low pricing, this cost structure is difficult to reconcile. Suppose its text-generation price is only a fraction of that of leading models. Users choose it precisely because it is affordable. If vision causes the cost of each request to increase severalfold, the provider must either raise prices and weaken its positioning or absorb the loss.

    The provider therefore needs to demonstrate that it can deliver vision at extremely low cost before releasing a broadly usable multimodal API. This is the first reason vision models may arrive late: the issue is not necessarily a lack of capability, but the fact that the economics have not yet been solved.

    3. Compression Strategies: Reducing Visual Tokens as Far as Possible

    3.1 From Patch-Based Representations to Structured Visual Primitives

    The industry is exploring several broad approaches to reducing the cost of visual tokens:

    1. Lower the resolution or increase patch size: This is straightforward to implement, but it can severely damage fine-grained information, such as small text and small objects.
    2. Pool or merge tokens: Visual tokens can be merged after the vision encoder produces them. This often achieves a compression ratio of roughly 2–4×, which may not be enough to fundamentally change the cost structure.
    3. Use structured visual primitives: This is the most aggressive and interesting approach. Instead of dividing an image into uniformly sized patches, the system treats structural information—such as key points and bounding boxes—as visual tokens that can participate directly in reasoning. The model no longer relies on a dense representation of pixel patches, but on a sparse collection of semantic spatial entities.

    Combined with compressed sparse attention, the third approach can reduce the number of visual KV-cache entries to a very low level. For an image at a moderate resolution, some research has achieved KV-cache compression ratios of several thousand to one. In other words, the visual cache required to process an image may be reduced to roughly the scale of dozens of text tokens.

    This can bring the marginal cost of a vision request closer to that of a text request, making multimodality more compatible with a low-price business model.

    3.2 The Cost: Rebuilding the Reasoning and Data Pipelines

    This approach also introduces substantial engineering costs.

    • The reasoning process must be redesigned. A conventional VLM pipeline looks like this: image encoding, continuous visual features participating in attention, and text generation. A structured-visual-primitive approach requires the model to first locate relevant regions in the image, generate structured visual tokens, and then reason over those tokens. The entire post-training reasoning paradigm must be reworked.
    • The data pipeline must be built from the ground up. Structured reasoning data of this kind is rarely available in existing corpora, so it usually has to be created specifically for the task. One practical approach is to collect large-scale object-detection datasets and apply multiple rounds of quality filtering. After filtering, only one-tenth to one-third of the original data may remain as usable training material.
    • The validation cycle is longer. Connecting an existing vision encoder can produce a working system within a few months. A structured-visual approach, by contrast, may require development cycles measured in quarters or even years.

    This explains why some vision models arrive late. The delay can result from choosing a more difficult path—one that few providers are willing to take, but that may be necessary to preserve the original cost model.

    4. Strategic Perspective: Multimodality as an Agent Completion Layer

    4.1 Two Different Product Strategies

    The difference in release timing also reflects two distinct product strategies.

    • The all-purpose flagship strategy: Multimodality is a flagship selling point. Image, text, and video capabilities are introduced early and promoted prominently, emphasizing broad coverage.
    • The Agent-first strategy: The provider continues to prioritize language reasoning and tool use, often building its own Agent execution framework. Vision serves mainly to let the Agent “see” the results of its actions—for example, checking whether a UI rendered correctly from a screenshot, confirming the state of an interface after code execution, or verifying the result of a document-processing task. In this model, vision completes the loop of execution → observation → correction.

    For the second strategy, video understanding and multimodal content generation are lower priorities. What matters is affordable, sufficiently capable static-image understanding that can be used repeatedly in large-scale Agent workflows without making every operation expensive.

    This returns to the core issue discussed earlier: only when visual-token costs are reduced aggressively can vision become a routine component of Agent workflows rather than a premium feature that is charged for on every request.

    4.2 What Developers Should Take Away

    The differences in release timing offer several broader lessons for developers:

    1. Calculate the token economics when evaluating multimodal APIs. Convert typical workloads—single-image question answering, multi-image document comparison, and repeated Agent screenshot loops—into estimated token costs. Then assess whether the advertised pricing is genuinely low. Compression ratios and KV-cache strategies may have a greater impact on real-world cost than parameter count.
    2. Treat an experimental label as a signal about capability boundaries. Experimental models often have limited interface stability and narrower capability coverage. For example, they may not support video or may operate under tighter quotas. Production integrations should include a clear fallback path.
    3. Evaluate vision in Agents through the complete feedback loop. The value of vision for an Agent lies in making execution observable. Evaluation should therefore focus on structured tasks such as locating elements, reading tables, and interpreting user interfaces, rather than relying only on generic image-captioning quality.

    5. Conclusion

    The late arrival of a vision model may reflect a combined decision about cost structure, technical architecture, and product strategy. For providers built around cost efficiency, the central question is not whether to add multimodality, but what cost structure can support it.

    Releasing vision becomes rational only when the expense of visual tokens is reduced enough to remain compatible with the provider’s low-price positioning. For developers, the more useful question is not which provider launched first, but how the underlying compression techniques are evolving.

    Structured visual primitives, sparse attention, and aggressive KV-cache compression are the technologies most likely to reshape the cost curve of multimodal applications.