How Multimodal Large Models Use Fewer Thinking Tokens for More Efficient Reasoning
Explicit chain-of-thought can improve reasoning but often increases decoding cost, latency, and resource usage. This article explains how a small number of dedicated thinking tokens can compress intermediate reasoning states for multimodal tasks, how models may be trained to use them, how implicit reasoning can be evaluated, and how to balance accuracy, efficiency, robustness, and interpretability in practical deployments.
1. Why Explicit Chain-of-Thought Increases Reasoning Costs
Chain-of-thought reasoning generally asks a model to generate a sequence of intermediate steps before producing an answer. This can help the model organize information and break down complex problems. It also makes parts of the reasoning process easier for people to read and inspect. However, readability does not necessarily mean efficiency: the longer the intermediate text, the more autoregressive decoding steps are typically required.
The cost is especially noticeable in multimodal tasks. A model may need to combine an image, a question, and contextual information while identifying objects, distinguishing attributes, filtering visual clues, and determining relationships. If every step is expressed in complete natural-language sentences, the reasoning trace may contain extensive descriptions that are not necessary for the final answer. Each additional generated token adds decoding computation. Longer outputs also increase response time and resource usage during inference.
The important question, therefore, is not whether the model needs to reason, but whether every part of that reasoning must be presented as long, human-readable text. One possible direction is to compress intermediate states into a smaller set of abstract representations, allowing the model to integrate information in its latent space before generating the final answer.
2. The Basic Idea of Using a Small Number of Thinking Tokens
Efficient reasoning frameworks built around this idea use a limited number of dedicated thinking tokens instead of a lengthy textual chain of thought. These tokens are not simply abbreviated versions of complete natural-language steps, nor do they mean that the model mechanically produces fewer outputs. They are better understood as compact representations that carry intermediate reasoning states: the model reads the multimodal input, encodes task-relevant information into these representations, and then uses them to generate an answer.
The process can be summarized as follows:
- Input understanding: The model receives an image and its associated question, then identifies the visual and linguistic information required by the task.
- Compact reasoning: The model stores, updates, or combines intermediate states through a limited number of thinking tokens.
- Answer generation: The model uses these states to make a judgment and produce a result for the user.
The key point is that this approach compresses explicit expression, not the reasoning task itself. The model still needs to extract visual clues, associate semantic information, and perform logical judgments. It simply does not need to spell out every intermediate step. If the compact representation preserves information that is useful for answering the question, it may reduce decoding length. If the compression is too aggressive, however, important clues may be lost and accuracy may decline.
3. How to Train Compact Reasoning能力
For thinking tokens to be useful, the model must learn to map task-relevant intermediate information into a limited implicit representation and correctly use that representation during answer generation. Training typically needs to balance two types of objectives.
The first is task performance: the model should produce answers that are correct, relevant, and complete. The second is the reasoning budget: the model should avoid relying on unnecessarily long intermediate text. If training optimizes only answer accuracy, the model may continue to produce lengthy reasoning traces. If it focuses only on restricting the token count, the model may not have enough capacity to integrate complex clues. A practical approach must therefore avoid treating “shorter” as automatically “better” and instead focus on the task value delivered per unit of reasoning budget.
Training data also affects compression quality. For simple recognition tasks, a small number of intermediate states may be sufficient. For questions that require comparisons across regions, quantity estimation, or the combination of multiple visual clues, training must teach the model to preserve different types of information. Evaluation should cover a range of task difficulties so that the model does not perform well only on simple examples while losing necessary reasoning steps in complex scenes.
4. How to Verify That Implicit Reasoning Is More Than Shorter Output
Looking only at the final answer makes it difficult to determine whether the model has developed a useful internal reasoning representation. A correct answer may result from effective reasoning, but it may also come from dataset bias, pattern memorization, or guessing. In addition to comparing task performance and output length, researchers should examine whether the thinking tokens encode information that can be meaningfully analyzed.
One possible research approach is to train an independent language-model interpreter that converts abstract tokens into more readable descriptions. Researchers can then analyze how those descriptions relate to task-relevant clues. For example, do they reflect key objects, attributes, or spatial relationships in the image? Do they match the requirements of the question? Do different tasks correspond to different types of intermediate information? This analysis can help determine whether latent representations are not entirely unobservable and can also provide clues for diagnosing errors.
However, text generated by an interpreter should not be treated as the model’s actual or complete internal process. It may be only an approximate reading of the implicit representation, and it may introduce biases from the interpreter itself. Explanation results should therefore be used as an analytical tool rather than final proof of reasoning correctness. A more reliable assessment combines task performance, changes in token budgets, input perturbation tests, and explanation analysis to determine whether the model is genuinely using relevant information.
5. Evaluation Should Consider Both Quality and Cost
Efficient reasoning should not be measured by output length alone. Evaluation should cover at least the following areas:
- Task quality: Check answer accuracy and completeness across multimodal tasks with different types and difficulty levels.
- Generation budget: Measure the number of tokens used during reasoning and observe how changes in the budget affect results.
- Response efficiency: Under the same computing environment and test conditions, compare generation overhead and response time. Different implementations and hardware can produce different results, so a reduction in token count should not be treated as a fixed proportional reduction in latency.
- Robustness: Change the wording of the question, visual details, or contextual information and observe whether the model consistently preserves the key judgment.
- Interpretability and diagnosability: Examine whether compact representations support error analysis and whether the resulting explanations are consistent with the evidence in the task.
Comparisons should also use consistent test settings and clearly distinguish model-generated intermediate content from the final answer. Only a joint evaluation of accuracy, reasoning budget, and real-world runtime efficiency can show whether a compression approach provides meaningful value.
6. Suitable Use Cases and Practical Boundaries
A small number of thinking tokens is particularly suitable for tasks in which the reasoning process contains repeated descriptions but the key state can be stored compactly. For example, a model may need to select relevant clues from an image and answer a clearly defined question, while the user has no need to inspect every intermediate judgment. In such cases, compressing explicit reasoning may improve efficiency.
This approach is not a universal replacement for explicit reasoning. Readable intermediate steps remain important in auditing, education, and scenarios that require step-by-step verification. For tasks with long evidence chains or high costs of error, excessive compression may make problems harder to locate and diagnose. When the data distribution changes or input quality is poor, the model may also fail to encode critical details reliably within a limited representation. System design should therefore allocate reasoning budgets according to task risk and explanation requirements rather than pursuing the shortest possible output.
A practical strategy is to use compact reasoning as the default path while retaining diagnostic capabilities. When the model has low confidence, the input is ambiguous, or the task requires review, the system can switch to a more comprehensive reasoning process or provide a more detailed explanation. This creates a controllable balance between efficiency and transparency.
7. Conclusion
Compressing a long chain of thought into a small number of abstract thinking tokens explores an alternative to expressing every reasoning step explicitly. Its value lies in reducing unnecessary generation while preserving the internal information required to complete the task. Its main challenge is ensuring that compression does not damage reasoning quality and developing reliable methods for evaluating implicit representations.
For developers and system designers, the central concern should be the overall trade-off among task performance, reasoning budget, runtime cost, and diagnosability. A small token budget does not automatically guarantee deeper reasoning, nor should it be viewed as a complete replacement for interpretable reasoning. With evaluations that cover diverse tasks and careful boundary design, compact reasoning can become a practical direction for improving the efficiency of multimodal models.
