Anonymous Multimodal LLM Revealed: 320B Total, 18B Active, and the Balance Between Coding and Agents
A previously anonymous multimodal model has been formally disclosed, revealing a sparse architecture with about 320B total parameters and roughly 18B activated parameters per token. This article examines its hybrid sparse and linear attention design, 1M-token context, coding and agent benchmark results, native visual feedback for coding workflows, local deployment considerations, and a staged adoption path for enterprises. The focus is not on parameter count alone, but on how capacity, inference cost, long-context stability, tool use, and visual understanding combine to support real production tasks.
1. From Anonymous Testing to Public Disclosure: Evaluation Is Moving Toward Real Workflows
A model that previously participated in public testing under an anonymous identity has now disclosed its technical background. During the anonymous phase, the model ranked near the top in actual weekly usage, suggesting that developers are shifting their attention from “how large is the parameter count” to “can the model reliably complete real tasks.”
For developers, a model’s value is not limited to single-turn question answering quality. Code generation, web debugging, automation, document analysis, and multi-step task execution often require the model to continuously understand context, call tools, observe execution results, and revise its approach based on feedback. As a result, whether a model is suitable for production usually depends on several dimensions:
- Inference cost per request: How many parameters are activated per token directly affects latency, VRAM usage, and throughput.
- Long-context stability: A longer context window is better suited to large code repositories, complex logs, and continuous tasks.
- Tool-calling capability: Can the model plan steps correctly, select tools, and handle exceptions?
- Multimodal understanding: This is especially important for screenshots, interface states, charts, and rendered output.
- Continuous operation: Agent tasks are usually not a single generation, but multiple rounds of “plan—execute—observe—revise.”
The model’s technical direction is built around these issues: reducing runtime cost through sparse computation while retaining coding, agent, and visual interaction capabilities.
2. Core Specs: Total Parameter Scale Is Not Equivalent to Activated Scale
Public information shows that the model has about 320B total parameters, but only about 18B parameters are actually activated during inference for a single token. This is a typical sparse expert architecture approach.
Traditional dense models usually use most parameters in every forward pass. A larger model may have stronger theoretical expressive capacity, but inference also consumes more compute and VRAM. Sparse expert models divide parameters into multiple relatively independent expert modules, and a routing mechanism selects a small number of experts to participate in computation based on the semantic features of the current token.
This can be abstracted as:
Input token↓Shared representation layer↓Router determines which experts suit the current token↓Activate only a small number of expert modules↓Aggregate expert outputs↓Generate the next token
The direct benefit is that the model can have a larger total capacity while carrying a smaller actual computation load per step. It is important to note that “18B activated parameters” does not mean the model is equivalent to an ordinary 18B dense model. Total parameter scale determines the range of knowledge and capabilities the model can hold, while activated parameter scale has more influence on per-step inference cost. They correspond to “capacity” and “computation,” respectively, and cannot be simply equated.
For enterprise deployment, this structure is particularly suitable for scenarios that:
- Require strong model capability but want to control per-request cost;
- Serve a large number of concurrent users;
- Need long context without accepting linear VRAM growth;
- Run agent tasks that last longer and involve more inference rounds;
- Need to run the model locally or in a private environment.
3. Architecture Optimization: Sparse Attention and Linear Attention Working Together
In long-context scenarios, the computation and storage pressure of standard attention grows quickly. Assuming a sequence length of N, traditional full attention usually computes pairwise token associations, with computational complexity close to O(N²). When context expands from hundreds of thousands of tokens to the million level, the attention matrix, key-value cache, and memory access overhead all become major bottlenecks.
The model uses a hybrid structure combining sparse attention and linear attention. The goal is to let different types of information use different computational paths:
- Use finer-grained attention for local or key regions to preserve high-quality semantic associations;
- Use more efficient linear processing for large-scale, strongly continuous information;
- Reduce repeated computation between irrelevant tokens through sparse access;
- Lower cache pressure in long documents, code repositories, and multi-turn interactions.
This design is not simply “reducing computation,” but redistributing the compute budget between accuracy and efficiency. For coding tasks, the model needs to pay attention to local variable relationships in the current function, as well as project-level directory structure, interface constraints, and call chains. Using only local attention may cause global information loss, while using only full attention brings enormous cost. Hybrid attention aims to strike a balance between precise local modeling and efficient global modeling.
Public data also shows that, compared with the previous-generation high-end model, its attention computation is reduced to about one-third, and the key-value cache size is roughly one-quarter of the previous generation. The key value here is not only lower VRAM usage, but also:
- Higher concurrency: The same accelerator card can hold more requests.
- Longer sessions: Agents can retain more complete historical operation records.
- Less cache movement: The inference system faces lower memory bandwidth pressure during multi-turn generation.
- Better continuous operation: Long-running coding and automation tasks are less likely to be interrupted by cache growth.
4. The Practical Meaning of 1M Context: Not Just “Fitting It In,” but “Using It Well”
A maximum context of 1M tokens is an important capability indicator for this class of model, but in practical engineering, simply expanding the window does not automatically solve long-document problems. The real difficulties include information localization, content compression, historical state management, and preservation of key facts.
In code agent scenarios, a million-token context may contain:
- Project directories and module dependencies;
- Multiple source code files;
- Build logs and runtime logs;
- Test reports;
- Historical modification records;
- User requirements and acceptance criteria;
- Page screenshots and interaction states;
- Tool-call results.
If all content is continuously appended in raw form, even if the model can receive it, it may face problems such as attention dilution, key information being buried, and increased inference latency. Therefore, practical applications should combine layered context management:
1. Short-term context
Store the current task, the most recent turns, the latest tool results, and unresolved issues so the model can respond quickly to the current step.
2. Mid-term memory
Store interface constraints, failure causes, modification plans, and test conclusions already confirmed in the current task to avoid repeated attempts.
3. Long-term knowledge
Store project-level information through code indexes, document retrieval, or structured databases, and recall relevant content only when needed.
4. State summaries
When context approaches the threshold, compress lengthy processes into structured summaries, for example:
Task goal: Fix the page layout issue on narrow-screen devicesCompleted: Located component A and style file BAttempted: Changed container width and resolved horizontal scrollingUnresolved: The modal still overflows on secondary pagesNext step: Check the modal positioning context and media query rules
This approach of “raw records plus structured summaries” is usually more stable than relying solely on an ultra-long window.
5. Coding and Agent Capabilities: From Generating Code to Completing Tasks
Public test results show that the model has made noticeable improvements in coding and automation tasks. For example, in a software engineering task evaluation, it scored 63.4, compared with 46.2 for the previous-generation model; in an automation task evaluation, it reached 48.8, compared with 26.2 for the previous generation; and in a high-difficulty coding capability test, it reached 29.0, close to the 29.5 of a high-end closed-source model.
These numbers should not be understood as “the model can fully replace developers,” but as an improvement in the model’s success rate across complex task chains. A mature coding agent needs to complete at least the following steps:
- Understand natural-language requirements;
- Analyze the existing project structure;
- Locate relevant code;
- Plan the scope of modification;
- Generate patches or new files;
- Run builds, tests, or static checks;
- Read error messages;
- Continue modifying based on feedback;
- Summarize changes and verification results.
The most easily overlooked part is the closed loop after step 6. A model that only generates code often cannot guarantee that the code actually runs; a model with tool-calling and error-correction capabilities is closer to a software engineering agent in the practical sense.
Therefore, during implementation, it is advisable to place the model within a clear execution framework:
Requirement parsing↓Task decomposition↓Retrieve code and documentation↓Generate modification plan↓Execute changes↓Run tests or builds↓Analyze output↓Revise and verify again
To control risk, the execution framework should also add permission boundaries. For example, reading code can be automated, but deleting files, modifying production configurations, and executing high-risk commands should require approval or sandbox isolation.
6. Vision Enters the Coding Loop: From “Reading Screenshots” to “Checking Results”
Another important feature of the model is that it uses native visual capabilities in the coding process, rather than treating visual understanding only as a standalone question-answering function.
Traditional code models usually reason only from text, code, and error logs. When tasks involve web layout, graphical interfaces, game scenes, or 3D simulation, many problems cannot be judged directly from source code. For example:
- Whether elements are occluded;
- Whether fonts overflow their containers;
- Whether buttons are actually within clickable areas;
- Whether animations drop frames;
- Whether 3D objects clip through each other;
- Whether rendered results match design expectations.
With visual feedback, an agent can run the following loop:
Write or modify code↓Launch the application and capture interface images↓Identify layout, color, hierarchy, and interaction state↓Determine whether it meets the goal↓Locate possible code causes↓Continue modifying and re-rendering
This shifts the model from “writing code based on descriptions” to “writing code and checking results.” For front-end development, visual feedback can help identify style hierarchy, responsive layout, and interaction-state issues; for games and 3D applications, it can help check scene composition, object positions, and rendering effects.
However, visual feedback also has limitations. A screenshot reflects only one moment and one viewpoint, and cannot replace full automated testing. Therefore, best practice is to combine visual checks with unit tests, end-to-end tests, accessibility checks, and performance monitoring.
7. Local Inference and Deployment: From Model Files to Usable Services
Public information shows that the model can already adapt to multiple mainstream local inference frameworks. For deployment engineers, the real concern is not a specific tool name, but the following engineering factors:
1. VRAM planning
Total parameter count, activated parameter count, weight precision, context length, and concurrency all affect VRAM requirements. Especially in long-context scenarios, the key-value cache may become the main source of usage.
2. Quantization strategy
Low-bit quantization can reduce VRAM and bandwidth pressure, but it may affect complex reasoning, code generation, and visual understanding quality. It should be validated against business tasks, not judged only by changes in model size.
3. Parallelism
Large models usually require tensor parallelism, pipeline parallelism, or hybrid parallelism. The parallel strategy should be selected based on hardware interconnect bandwidth, batch size, and service latency targets.
4. Service scheduling
For agent tasks, request lengths and generation rounds vary widely. The server should support dynamic batching, request priorities, timeout control, and session isolation to prevent long tasks from slowing down short requests.
5. Security isolation
If the model can execute code or call system tools, the execution environment should be placed in a sandbox, with restrictions on file system, network, process, and system permissions. Complete logs should also be retained for auditing and troubleshooting.
8. Suitable Scenarios and Adoption Guidance
This type of high-capacity, low-activation-cost multimodal model is suitable for:
- Enterprise code assistants;
- Q&A over large codebases;
- Automated testing and defect repair;
- Web interface generation and visual acceptance;
- Log analysis and fault localization;
- Joint understanding of documents, screenshots, and tables;
- Prototyping for games and 3D scenes;
- Long-running, multi-step task agents.
For practical adoption, a phased strategy can be used:
Phase 1: Offline evaluation.
Prepare the enterprise’s own code, documentation, logs, and screenshot samples, and evaluate code correctness, tool-call success rate, context recall accuracy, and visual judgment stability.
Phase 2: Read-only assistant.
First enable code retrieval, explanation, issue localization, and testing suggestions, while temporarily disallowing the model from directly modifying production files.
Phase 3: Controlled modification.
Allow the model to generate patches in an independent branch or temporary workspace and automatically run checks.
Phase 4: Closed-loop agent.
After permission approval, sandboxing, rollback, and auditing mechanisms are mature, gradually enable multi-step automated repair tasks.
Conclusion
The formal disclosure of this previously anonymous model reflects a shift in large-model competition from simply pursuing parameter scale toward comprehensive competition in computational efficiency, long context, multimodal interaction, and task closed loops.
The 320B total parameters provide substantial capacity, while the 18B activated scale reduces per-step inference cost; hybrid attention reduces long-context pressure; visual capabilities allow the model to observe the real results of code execution; and tool-calling and automatic correction mechanisms move it further from a “code generator” toward a “software engineering agent.”
For enterprises and developers, model selection should not focus only on leaderboards or individual scores. It should be validated against real tasks: whether the model can understand existing code, reliably call tools, discover its own errors, maintain state through long workflows, and continue operating within safe boundaries. Only by combining model capability, inference infrastructure, and engineering governance can multimodal large models become reusable, maintainable, and auditable productivity tools.
