Large Model-Assisted Chip Architecture Exploration and Hardware-Software Co-Design in Practice

Large model training and inference place increasingly high demands on compute, bandwidth, and energy efficiency. When general-purpose processors face very large parameter counts, long-sequence attention, and mixed-precision computation, they often struggle to satisfy throughput, latency, and power consumption targets at the same time. More teams are therefore exploring model-chip co-design: the algorithm side defines key operators and dataflows, while the hardware side customizes compute units, memory hierarchy, and interconnect topology so that the two sides adapt to each other. This approach can improve effective compute per unit of power, but the gain depends on the specific workload, process node, and toolchain, and must be measured on a per-project basis.

1. Why Co-Design Matters

In a traditional flow, the model team first determines the network structure and then maps operators onto existing chips, while the hardware team makes architectural trade-offs based on historical workloads. When the two sides proceed independently, mismatches easily arise: the algorithm has already changed, but the chip has not caught up.

Co-design starts at the beginning of a project by translating key bottlenecks in training and inference into hardware constraints, such as attention computation, MoE routing, and KV Cache access. At the same time, it reserves configurable space in the microarchitecture. In this way, some model adjustments can be completed through firmware or microcode, reducing the number of respins. This premise holds only when the reserved space covers new operators, new bit widths, or new storage requirements, and when timing and power still have sufficient margin. Changes beyond that scope still require a revision or a new tapeout.

2. Four-Stage Process

The first stage is workload profiling. The team needs to collect real training and inference traces and measure operator time share, memory access patterns, precision distribution, and communication topology. Sampling should cover a complete training cycle and typical inference peak periods; otherwise, the statistics may deviate from reality. Metrics must be defined consistently across the team—for example, operator time share should be calculated against the total time of the same batch, and bandwidth utilization should be normalized to peak bandwidth. Acceptance thresholds should be set against historical baselines rather than copied directly from external numbers.

The profiling results directly determine which units the chip should strengthen. When KV Cache access is intensive, adding high-bandwidth on-chip SRAM can be considered. When matrix multiplication accounts for a high share, a reconfigurable systolic array can be considered. When the proportion of sparse computation is high, low-precision tensor cores can be considered.

The second stage is architecture exploration. The team can use large models to perform semantic retrieval and pattern induction over historical design documents, RTL code, and simulation logs, quickly generating candidate microarchitectures. Large models are good at converting natural-language performance goals into constraints. For example, “reduce attention latency by 40% at a 128K context” is broken down into specific constraints on bandwidth, parallelism, and cache capacity. These constraints must also be cross-checked against timing, power, and area reports from EDA tools; they cannot rely on model judgment alone.

Before candidate designs enter subsequent stages, they should pass the following gates: human review to confirm architectural assumptions and performance goals; RTL lint checks and formal verification to eliminate obvious structural problems; PPA regression to compare whether area, power, and performance are within budget; and data isolation and log auditing to confirm that design data has not left the authorized scope. If any item fails, the design should be sent back for re-evaluation.

The third stage is hardware-software co-simulation. The team first runs a complete training step on a behavioral model to verify numerical stability and gradient correctness. It then moves down to the RTL level and uses the synthesized netlist to evaluate real timing margin. The results of the two simulation levels need to be aligned. If the performance gap between the behavioral model and the netlist remains persistently large, it indicates that the surrogate model needs calibration.

The fourth stage is closed-loop iteration. Bottlenecks found in simulation should be fed back to the model team, which adjusts operator fusion strategies or quantization schemes and then guides the next chip revision. Whether the iteration cycle can be shortened depends on workload characteristics, toolchain maturity, and verification coverage. The team should record the time consumed in each round and use actual data to judge the effect rather than relying on estimates.

3. Key Capabilities for Co-Design

The first is a unified intermediate representation. The model graph, operator library, and chip instruction set need to map to the same IR so that optimization suggestions generated by large models can land directly in the compiler or hardware description language. The IR should at least include operator semantics, tensor layout, precision, memory access and communication constraints, and timing and power metadata. Version numbers and compatibility policies must also be defined. For example, new fields should be backward compatible, deprecated fields should retain a transition period, and compiler and RTL versions should be recorded in sync.

The second is a differentiable performance model. The team can use lightweight surrogate models to predict area, power, and latency, enabling architecture search based on gradients rather than pure trial and error. The prediction error of the surrogate model needs to be reported continuously and compared with synthesis results on a regular basis. When the error exceeds the agreed range, conclusions can only be used for preliminary screening and cannot directly serve as design decisions.

The third is a cloud-native collaboration environment. RTL simulation, formal verification, and power analysis can be containerized; waveform and coverage data can be stored in object storage; regression tasks can be triggered through message queues; and exceptions can be detected in real time by logging services and monitoring alerts. This environment can be deployed on general cloud infrastructure. For stages that depend on specific commercial tool licenses, cost and portability need to be evaluated separately.

4. Implementation Challenges

The first is data quality. Chip design documents often contain many internal terms and legacy conventions, and large models need domain fine-tuning to accurately understand timing constraints and physical design rules. Fine-tuning data can come from reviewed design documents, confirmed constraint files, and sanitized historical logs. Sanitization rules must be explicit, access permissions should be granted on a least-privilege, per-project basis, and process parameters must not leave the isolated environment.

The second is verification completeness. The cost of fixing hardware errors is far higher than that of fixing software. RTL generated by large models may introduce hard-to-detect race conditions or power hotspots, so formal methods and random simulation must be combined. The coverage level required before tapeout should be written into explicit admission criteria at project kickoff.

The third is cost control. Tapeout costs at advanced process nodes are high, and the specific amount varies by process node and project scale. The team should use surrogate models during architecture exploration to keep area and power within budget and reduce the risk of large-scale rework later.

The fourth is talent structure. Engineers who understand both model algorithms and the digital backend remain scarce. Internal knowledge bases and large-model-assisted Q&A can lower the barrier to cross-team collaboration, but they cannot replace experienced engineers making key judgments.

5. Incremental Implementation Path

Teams do not need to start with full customization. A more prudent approach has three steps.

Step one: optimize the software stack on existing general-purpose chips. Use large models to assist in generating operator fusion and scheduling strategies, and validate the gains with measured data.

Step two: if the gains are clear and the bottleneck is concentrated, start small-scope customization. For example, add a dedicated accelerator only for the attention module while reusing mature IP for the rest.

Step three: decide whether to expand the customization scope based on the results of the first two steps.

Version management is the foundation of this path. Every model structure change should correspond to a chip configuration snapshot, which should record the model version, chip configuration, toolchain version, test set, and key metric results. If a new version has problems, the team can roll back to the last usable version based on the snapshot and confirm the conditions required for rollback in advance, such as whether the old version’s build artifacts are still reproducible.

Security and compliance also need to be designed in advance. Design data should be processed in an isolated environment, and access logs should be fully recorded and audited regularly. Sensitive process parameters should be classified, and the range of personnel who can access them should be restricted.

6. Outlook

As model scale continues to grow, co-design may shift from one-off customization to continuous evolution. Future chips may contain more reconfigurable logic, dynamically adjusting data paths and precision at runtime according to workload. Elastic cloud compute can handle large-scale architecture search, while lightweight inference chips can be deployed locally for low-latency services.

Regardless of how the technology route changes, one principle is worth keeping: models define requirements, hardware provides the foundation, and tools close the verification loop. Where verification completeness and cost constraints are satisfied, collaborative design can move forward.