Selecting an Octa-Core AIoT Platform: Compute Power, Bandwidth, and Real-World Testing

    Octa-core AIoT platforms are widely used in industrial control, edge computing, and smart security applications. However, spec sheets often omit test conditions, leading to incorrect hardware selection. This article provides a systematic approach to evaluating these platforms by analyzing power consumption, NPU compute power, interface and memory bandwidth, and essential vendor test requirements to ensure accurate deployment and performance assessment.

    Industrial control, edge computing, and smart security projects commonly use octa-core AIoT platforms as their main controllers. However, the numbers on specification sheets often lack explicit test conditions, making direct comparisons prone to selection errors. This article explains how to evaluate each metric in the order of power consumption, compute power, bandwidth, vendor test items, and real-world deployment testing. Unless otherwise specified, all numbers in this article are calculation examples and do not represent the actual test results of any specific chip.

    Key Takeaways

    • If your business primarily involves long-term light loads, such as sensor aggregation or lightweight communication, prioritize the standby power consumption of efficiency cores and the overall device power consumption curve.
    • If your business primarily involves peak inference, first examine the available operator ratio of the NPU and memory bandwidth utilization, then compare the nominal TOPS.
    • If your business primarily involves multi-channel video, first calculate the frame data bandwidth, then look at the headroom of the interfaces and memory.

    1. Process Node and big.LITTLE Architecture: What Determines the Power Ceiling

    The process node affects transistor density and leakage power. A more advanced node typically integrates more transistors in the same area and has lower static leakage. For devices that need to run for extended periods, this affects standby power consumption and heat generation.

    However, 8nm and 12nm are merely node names. The process libraries, packaging methods, and power management designs of different foundries vary significantly, making it impossible to judge power consumption solely based on the node name. Even within the same process, the chip design itself alters the final power consumption. Therefore, selection should be based on measured power consumption.

    The big.LITTLE heterogeneous architecture is a common design for octa-core platforms. Big cores are suited for bursty workloads such as system boot, complex UI rendering, and model loading. Little cores are suited for continuous workloads like background synchronization and lightweight communication. The EAS (Energy Aware Scheduling) scheduler in the Linux kernel allocates tasks to different cores based on the load.

    During selection, verify whether your workload can engage the big cores on demand. Observe the utilization and frequency switching logs of each core while running a load script. If the load consistently falls on the little cores, no matter how high the peak compute power is, it remains unused, making the standby power paid for the big cores unjustifiable.

    Power consumption and performance data provided by vendors must include test conditions. A qualified test report should contain at least the following:

    • Chip model, firmware version, and operating system version
    • Measurement point (e.g., whole device, SoC core, or standby power)
    • Load type (e.g., video resolution, bitrate, and frame rate, or inference model and input size)
    • Peripheral and network status (e.g., screen brightness, Ethernet connection status)
    • Ambient temperature and cooling solution
    • Testing tools and their sampling frequency

    When comparing across platforms, the firmware version, cooling conditions, load script, and sampling frequency must be identical. If any of these differ, the numbers cannot be directly compared. You can request the vendor to provide data for both the previous generation and the target platform under the same conditions, noting the test date.

    It is recommended to use the following table to record each test for future comparison:

    Record Item Example Content
    Platform & Firmware Platform name, firmware version, OS version
    Load Script Script name and version, execution duration
    Measurement Point Whole device, SoC core, or standby
    Peripheral Status Screen brightness, network port status
    Environment Ambient temperature, cooling solution
    Tools Tool name, sampling frequency

    2. NPU Compute Power: Deconstructing Nominal TOPS

    NPU TOPS numbers are typically calculated using low-precision formats like INT8 or INT4. Vendors often convert based on bit-width, so the INT4 number for the same chip is usually higher than INT8. However, this does not mean your model will yield the same benefits using INT4. Whether the drop in precision is acceptable must be verified using your own models and data.

    Sparse and dense calculations must also be viewed separately. Sparse computing only takes effect when the model weights are pruned sufficiently and the compiler supports it. The sparse TOPS advertised are usually theoretical upper limits and cannot be treated as daily inference capability.

    Operator coverage is equally critical. If the NPU does not support a certain operator in your model, that operator will fall back to run on the CPU. Data must be shuttled back and forth between the CPU and NPU, significantly increasing latency and power consumption. Before selection, confirm that all operators in your model can be executed on the NPU and request an operator support list from the vendor.

    You can use a rough formula to filter candidate platforms:
    Available Compute Power ≈ Nominal TOPS × Proportion of NPU-runnable Operators × Actual Compute Utilization

    Here, the compute utilization must be obtained through actual testing and cannot directly adopt the vendor’s numbers. Many NPUs are practically limited by memory bandwidth, causing compute units to wait for data frequently. Therefore, during testing, you should simultaneously record memory bandwidth utilization, not just throughput numbers.

    When requesting NPU test results from vendors, it is recommended to fix the following items and organize them into a table for multi-vendor comparison:

    Item Content to Record
    Nominal TOPS Data format (INT8, INT4, or FP16), dense or sparse metric
    Tested Model Model name, version, input size
    Batch Batch size per inference
    Quantization Format Bit-width and quantization method for weights and activations
    Precision Metric drop compared to the floating-point baseline
    Latency Average and tail latency (e.g., P99) for a single inference
    Power Consumption Whole device or NPU power consumption during inference
    Available Operator Ratio Proportion of operators in the model that can fully run on the NPU
    Software Environment Inference framework, compiler version, input preprocessing method, tail latency statistical method

    3. Interface and Memory Bandwidth: From Theoretical Values to Headroom

    Theoretical bandwidths of interfaces and memory must be calculated against your business data volume. The formula for theoretical memory bandwidth is as follows:
    Theoretical Bandwidth (GB/s) = Data Rate (MT/s) × Single-channel Bit-width × Number of Channels ÷ 8 ÷ 1000

    Note two points. First, the DDR data rate is twice the clock frequency. For example, a DDR4 with a clock of 1600MHz has a data rate of 3200MT/s; you must use 3200 in the calculation. Second, the bit-width is calculated per single channel.

    Example 1: DDR4-3200, single-channel 64-bit.
    3200 × 64 ÷ 8 ÷ 1000 = 25.6 GB/s. Dual-channel would be 51.2 GB/s.

    Example 2: LPDDR4X-4266, two 32-bit channels, meaning a total bit-width of 64-bit.
    4266 × 64 ÷ 8 ÷ 1000 ≈ 34.1 GB/s.

    Theoretical values for interfaces also need conversion. The theoretical rate of Gigabit Ethernet is about 125 MB/s, with actual payload throughput lower than this. PCIe 3.0 is about 0.985 GB/s per lane in one direction. USB 3.0’s 5Gbps, after 8b/10b encoding, is about 500 MB/s.

    Business Data Volume Comparison Example

    Assume 4 channels of 1920×1080, 25fps video, with a pixel format of NV12. The data per frame is about 1920×1080×1.5 ≈ 3.11 MB.

    Assume each frame undergoes one write and two reads: written once after decoding, read once for encoding or display, and read once more for AI preprocessing. This assumption must be modified according to your actual data flow.

    The bandwidth requirement for a single channel is approximately: 3.11 MB × 25 × 3 ≈ 233 MB/s.
    For 4 channels, the total is approximately: 233 MB/s × 4 ≈ 0.93 GB/s.

    Add to this the weight reads and intermediate feature map read/writes for NPU inference. This part must be filled in using the actual test results from Section 2 and cannot be replaced by theoretical values.

    The actual available memory bandwidth is typically lower than the theoretical value due to refresh overhead, read/write switching, and access patterns. You can estimate the headroom using the following formula:
    Headroom Coefficient = Theoretical Bandwidth × Effective Efficiency ÷ Total Bandwidth Requirement

    The effective efficiency must be determined through actual testing. If you temporarily lack data, you can use an assumed value, such as 0.6, and replace it after testing. The headroom coefficient during the design phase can start at 2 as a baseline, with the specific value adjusted based on concurrency and burst loads.

    Memory Component Verification Checklist

    The “memory component” in the title needs separate verification. Memory with the same capacity and speed can have different dies, temperature grades, and packaging, leading to different platform compatibility and long-term supply risks.

    Verification Item Content to Verify Description
    Generation & Type DDR4, LPDDR4X, LPDDR5, etc. Must be within the SoC memory controller’s support range
    Data Rate Nominal rate vs. actual running rate Subject to the SoC vendor’s compatibility list
    Bit-width & Channels Single-channel bit-width, number of channels Directly determines theoretical bandwidth
    Die & Capacity Die model, single die capacity, number of dies Must be re-verified if the die model changes
    Temperature Grade Commercial, industrial Industrial scenarios must confirm the operating temperature range
    ECC Whether supported, whether enabled Affects reliability and available capacity
    Supply & Batch Supply cycle, batch management, substitution rules Avoid rework caused by mid-production component changes

    4. Test Items to Request from Vendors

    When communicating with vendors, it is recommended to list the following content as a written checklist and request them to reply item by item:

    1. Firmware, operating sys