Homogeneous Fusion RISC-V Chip Architecture for Edge AI: An In-Depth Look
As large language models and robotics move to the edge, a single system must deliver both strong general-purpose compute and powerful AI throughput—without the complexity of traditional heterogeneous designs. This article examines the homogeneous fusion architecture, which integrates a high-performance RISC-V CPU cluster and an AI-enhanced AI-CPU cluster on one chip, running a single Linux OS with unified scheduling. It covers the value of homogeneity, the role of the RVA23 specification in ecosystem compatibility, software-hardware co-design for LLM inference, the layered software stack, and the roadmap and challenges ahead for this emerging paradigm.
1. Introduction: The Core Tension of the Edge AI Era
As large language models (LLMs) and robotics migrate to the edge, the industry faces a key tension: a single system must provide both strong general-purpose computing capability and powerful AI throughput, while ultimately shipping as a scalable, cost-effective commercial product. The traditional approach is a heterogeneous “CPU + discrete AI accelerator” design, but this introduces a series of engineering problems: complex driver adaptation, costly memory copies, and a fragmented software stack.
To resolve this tension, a computing paradigm known as homogeneous fusion has been gaining attention: under a unified RISC-V instruction set architecture and extension framework, a high-performance general-purpose CPU cluster and an AI-first “AI-CPU” cluster are integrated on the same chip. Both run the same Linux operating system, with threads or tasks as the basic scheduling units for unified scheduling and resource governance. The result is a system-level delivery model of “one platform, one software stack, one operational plane.”
A useful analogy is the big.LITTLE design widely adopted in smartphones during the mobile internet era, which pursued high performance and low power under a single OS. In the AI era, homogeneous fusion pursues high performance and high AI throughput simultaneously, while compressing the system fragmentation costs of heterogeneity into a manageable range.
2. Architecture Overview: Homogeneous, Not Heterogeneous
2.1 Two Compute Clusters
A typical chip of this kind contains two clusters:
- High-performance CPU cluster: Handles control-plane duties, system services, and complex business logic. It emphasizes single-thread and multi-thread performance, ensuring system responsiveness and ecosystem compatibility.
- AI-CPU cluster: Also built on RISC-V cores, but with microarchitectural enhancements for AI workloads. Through instruction set extensions (such as processing units for matrix operations and fixed-point/low-precision computation), it executes tensor operations natively—no discrete accelerator or proprietary runtime required.
2.2 The Value of “Homogeneity”
Compared with the traditional CPU + NPU heterogeneous approach, homogeneous fusion differs in several fundamental ways:
- A single address space: The AI compute units share the process model and virtual memory system with the general-purpose CPUs. AI tasks can be invoked like ordinary functions with low overhead, eliminating cross-device memory copies and synchronization costs.
- A single operating system: One Linux kernel manages all compute resources, with no need to maintain a separate driver stack, firmware, and runtime for AI accelerators.
- Unified scheduling: The OS scheduler treats the AI cluster’s compute as schedulable resources, governing priority, bandwidth, and affinity at thread/task granularity—critical for latency-sensitive scenarios such as robotics.
3. RVA23: The Foundation of Ecosystem Compatibility
RVA23 is the most important specification baseline for RISC-V application processors, defining the instruction sets and extensions that application-class processors should implement. For edge AI chips, fully implementing RVA23 means:
- Plug-and-play software ecosystem: Mainstream Linux distributions, compiler toolchains, and language runtimes run directly, with no secondary porting.
- A standardized extension path: Standardized extensions such as bit manipulation and vector operations provide a uniform interface for low-level AI operator optimization.
- Avoiding fragmentation: The biggest historical risk for RISC-V has been ecosystem fragmentation caused by vendor-specific custom extensions. Following RVA23 ensures custom extensions build on a common baseline, reducing downstream adaptation costs.
At the implementation level, enabling RVA23 involves pipeline support for vector extensions, verification of architectural features such as interrupts and virtualization, and coordinated upstreaming with the kernel and compiler communities—a long-term effort spanning hardware design through system software.
4. Software-Hardware Co-Design for LLM Inference
LLM inference is a workload constrained by both memory bandwidth and compute. Running LLMs efficiently on resource-limited edge hardware typically requires co-design across multiple layers:
4.1 Instruction Layer: Matrix and Low-Precision Extensions
The AI-CPU cluster uses dedicated matrix processing unit extensions that support common low-precision data formats (such as 8-bit integers and 16-bit floating point), sinking core operators like GEMM (general matrix multiply) into hardware. Compared with relying on vector extensions alone, dedicated matrix units can significantly increase peak compute within the same area and power budget.
4.2 Compiler Layer: Triton-Style Operator Generation
On the software side, a Triton-style operator compilation approach lets developers describe operators in a Python-like domain language, which the compiler then automatically maps to target extension instructions—greatly reducing the cost of hand-writing kernels. A typical operator development flow looks like this:
@kerneldef matmul_kernel(a, b, c, M, N, K, BLOCK: constexpr):pid = program_id(0)# Blocked loads, dot-product accumulation, write-back...
The compiler translates the blocking logic into matrix extension instruction sequences, performing loop optimization and memory access scheduling.
4.3 System Layer: Memory and Scheduling Governance
LLM inference is highly sensitive to memory capacity and bandwidth, so system-level mechanisms—KV cache management, memory bandwidth quotas, and task affinity settings across compute clusters—directly affect throughput and latency. The advantage of a unified OS is that these governance mechanisms can reuse existing kernel facilities such as cgroups and CPU affinity, rather than depending on accelerator-specific management channels.
5. Software Stack and Resource Governance
A complete edge AI software stack typically has four layers:
- Kernel layer: Context-switching support for AI extensions, performance counters, and scheduler enhancements;
- Runtime layer: A unified operator execution framework handling operator scheduling, memory pool management, and multi-cluster coordination;
- Compiler layer: Operator compilers and graph optimizers that automatically lower higher-level models to extension instructions;
- Framework and application layer: Interfaces with mainstream inference framework formats and exposes standard APIs upward.
For resource governance, the AI cluster and general-purpose cluster can be modeled as two classes of schedulable resources, using quotas and priority policies to ensure critical tasks (such as a robot’s control loop) are not starved by LLM inference.
6. Evolution Path and Challenges
From first- to second-generation products, improvements typically come along three dimensions: stronger single-thread and multi-thread general-purpose performance, higher peak AI compute and memory bandwidth, and more complete RVA23 compliance and software stack maturity. First-generation deployments have already validated the feasibility of this approach—general-purpose CPUs ensure compatibility and maintainability, while ISA-integrated AI compute reduces deployment complexity.
Challenges remain, of course: how to keep improving matrix unit efficiency without inflating die area, how to make compilers generate operators approaching hand-written limits, and how to gracefully balance real-time requirements against throughput within a single kernel. These are questions that future iterations must continue to answer.
7. Conclusion
The homogeneous fusion paradigm represents a pragmatic path for edge AI chip design: built on the openness of RISC-V and the standardization of RVA23, it “internalizes” AI capability as instruction set extensions rather than bolt-on devices, and uses a unified OS and software stack to eliminate heterogeneous fragmentation. For robotics, industrial vision, and other scenarios that must run both business logic and large models at the edge, this paradigm offers a reference answer that balances performance, cost, and engineering maintainability.
