From Punch-Hole to Under-Display: How On-Device AI Is Reshaping Device Hardware
A wave of recent supply-chain reports points to three converging shifts in mobile hardware: display form factor (a single punch-hole plus under-display infrared recognition), imaging (a mechanical variable aperture with a new sensor), and chip architecture (wafer-level near-memory packaging). This article treats those rumors not as fact but as a lens on one underlying driver — on-device AI moving from a "feature demo" to an "always-on default capability." It explains the engineering rationale, trade-offs, and implementation approaches behind each thread, then covers device-cloud collaboration, on-device model miniaturization, a layered evaluation baseline, and privacy-by-design. It is aimed at developers, architects, and product technology leads who need to turn hardware options into measurable, user-perceivable experiences.
Recent months have seen a cluster of reports on upcoming device-hardware moves. Aggregated supply-chain information suggests that a leading device maker plans to expand its smartphone lineup from five models to at least seven within roughly two years. Its next-generation Pro series, expected in fall 2026, is said to change in four main areas:
- The front of the display adopts a single circular punch-hole, with the face-recognition module moved beneath the screen;
- At least one rear camera adds a mechanical aperture and switches to a new-generation image sensor;
- A new generation of wafer-level chip packaging moves the memory module closer to the processor;
- The overall exterior stays close to the previous Pro design.
Separately, industry commentary has pushed back on talk of an “AI bubble,” arguing that no bubble exists and that major opportunities lie ahead.
None of this is confirmed by the manufacturers, and this article makes no factual claim about it. Even so, the reports sketch a clear technical picture: a shift in form factor, a shift in optics, and a shift in architecture. These are not isolated selling points but three external expressions of the same underlying force at the device edge — on-device AI moving from a “feature demo” to an “always-on default capability.”
This article sets aside the capital-markets “bubble debate.” From an engineering standpoint, the question becomes measurable: can the per-unit compute cost of on-device AI keep falling with scale while usage stays healthy? The sections below follow the three hardware threads and break down their principles, costs, and implementation methods, for developers, architects, and product technology leads.
1. The Common Logic Behind Three Hardware Threads
1.1 From “can do” to “commonly used”
Early on-device AI showed up as isolated features — face unlock, scene recognition, voice wake — with three things in common:
- Low compute;
- Loose latency requirements;
- Occasional triggering.
The rise of generative AI has pushed long-tail tasks onto the device: real-time voice interaction, photo-intent recognition, call summaries, and local document Q&A. These tasks need models resident in memory and called frequently. On-device AI therefore faces three constraints: memory capacity, memory bandwidth, and power/thermal budget. This drives two lines of evolution:
- Raise data-acquisition efficiency: let sensors see more clearly and transfer faster;
- Shorten data-movement distance: put memory closer to the compute units.
1.2 Where the three threads converge
| Thread | Hardware change | Core problem solved | Engineering focus |
|---|---|---|---|
| Form factor | Punch-hole + under-display IR recognition | Coexistence of a full front display and front-facing recognition | Transmittance, display compensation, calibration |
| Optics | Variable aperture + new sensor | Conflict between light intake and depth of field; high dynamic range | Actuator reliability, exposure strategy, ISP algorithms |
| Architecture | Near-memory advanced packaging | Memory wall, data-movement energy, local inference | Bandwidth and latency, thermals, yield and cost |
All three share one trait: the payoff rests on system-level coordination, and no single-parameter gain delivers the experience on its own. Under-display IR recognition, for instance, depends simultaneously on the screen’s pixel structure, optical coatings, face algorithms, and display-compensation strategy. Near-memory packaging likewise requires the chip microarchitecture, memory controller, inference framework, and compiler to work together. That is the basis for the discussion below.
2. Display Form Evolution: From Punch-Hole to Under-Display
2.1 Three concepts not to conflate
- Punch-hole display: a dedicated circular or pill-shaped opening within the display area for the front module. Light does not pass through the display layer, so imaging is clear, implementation is simple, and cost is low. The cost is a visual sense of “incompleteness,” and the UI must reserve space in the status-bar area.
- Under-display camera: the camera sits beneath the display panel and images through the gaps between pixels. The image suffers brightness loss, color shift, glare, and reduced sharpness, so it suits low-fidelity scenarios such as video calls and beauty selfies.
- Under-display infrared recognition (the core change in these reports): the IR module used for face unlock and payment moves beneath the screen. IR recognition relies on a 2D IR image plus depth information (parallax or time-of-flight); it demands less “detail sharpness” than visible-light photography but very high signal-to-noise ratio, contrast, and repeatability — and unlock and payment scenarios are extremely sensitive to failure rate.
2.2 Why under-display IR is harder
- The transmittance-window trade-off: an IR-transparent region must be opened in the screen, usually with special pixel design (lower pixel density, adjusted cathode patterns) to improve IR-band transmittance. This region has to be balanced repeatedly against display quality and overall appearance.
- Signal-to-noise and glare: lower transmittance directly weakens IR image contrast; the periodic pixel array can also diffract light, producing bright spots or stripes in the IR image that hurt feature extraction.
- Calibration and aging drift: assembly tolerances, panel batch differences, thermal drift, and long-term material aging all shift the baseline of the IR image. The device needs factory calibration and periodic re-establishment of its reference state during operation.
2.3 Reading the transitional form
“Single punch-hole + under-display IR” is essentially a compromise between experience and difficulty: the visible-light front camera, which depends most on image quality, stays in the punch-hole to preserve selfie and video clarity; the IR recognition, which is less demanding on detail but extremely demanding on reliability, sits beneath the screen to gain display completeness. As cathode patterning, transparent conductive layers, pixel arrangement, and compensation algorithms mature, punch-hole and under-display approaches may coexist for a long time, configured differently by product tier.
2.4 Illustrative display compensation
In practice, display and image compensation for under-display regions usually revolve around “calibration data + lookup + local filtering.” The pseudocode below only illustrates the idea:
import numpy as npdef compensate_display(frame: np.ndarray,mask: np.ndarray,gain_table: np.ndarray) -> np.ndarray:"""Brightness compensation near under-display / punch-hole regions on the display-driver side (pseudocode)frame: linear-light image to display (H, W, 3)mask: under-display region weight map (H, W); 1 means fully inside the regiongain_table: per-channel compensation gains from factory calibration (H, W, 3)"""gain = 1.0 + mask[..., None] * (gain_table - 1.0) # no compensation outside the regionreturn np.clip(frame * gain, 0.0, 1.0)def compensate_camera_capture(raw: np.ndarray,mask: np.ndarray,ring_kernel_size: int = 5) -> np.ndarray:"""Suppress glare and stripes in the under-display image on the ISP side (pseudocode)Idea: first run bilateral / guided filtering along guide-image edges to suppress stripes,then replace the masked anomalous bright spots with a mean estimated from surrounding pixels."""from scipy.ndimage import uniform_filter # mean filter for illustrationguide = rawsmooth = uniform_filter(raw, size=ring_kernel_size, mode="nearest")detail = raw - smooth# Suppress high-frequency bright spots in the detail layer only inside the under-display regionsuppressed = raw - mask[..., None] * np.clip(detail, 0.0, None)return np.clip(suppressed, 0.0, 1.0)
2.5 Platform reuse behind product-matrix expansion
A leading maker expanding from five models to seven or more carries engineering weight no less than a hardware overhaul:
- Configuration-matrix explosion: combinations of chip tier, memory capacity, screen size and form factor, and camera set multiply test and maintenance costs.
- Layered design: adopt a “core platform + hardware abstraction layer + capability manifest” approach so upper-layer software adapts to differences by querying device capabilities, rather than scattering model checks through the code.
- Illustrative capability manifest:
# device_capabilities.yaml (illustrative)device_tier: prodisplay:front_camera: hole_punchunder_display_infrared: truecamera:variable_aperture: trueaperture_steps: [f/1.4, f/2.0, f/2.8, f/4.0]ai_runtime:memory_on_package: truemax_model_size_mb: 1500supports_int4_weight_only: true
- Quality assurance: build continuous integration and automated regression across the configuration × device × scenario matrix, paired with unified telemetry and online monitoring to drill down by model into crash rate, jank rate, unlock success rate, and power draw.
3. Imaging System Upgrade: Variable Aperture and New Sensors
3.1 What variable aperture solves
The imaging strategy of a fixed-aperture phone is pulled by two conflicting needs:
- Low light wants a large aperture for more light intake;
- Bright light and landscape scenes want a small aperture for more depth of field and a shorter shutter to freeze motion.
Some devices use electronic shutter, high shutter speeds, and ND filtering to “simulate” a small aperture, but this cannot truly widen depth of field and can introduce rolling-shutter artifacts. A mechanical variable aperture drives blades to change the opening, yielding three direct benefits:
- Controllable depth of field: stopping down keeps foreground and background sharp, with a slight starburst effect;
- More flexible shutter: no need to rely on extreme shutter speeds in bright light, reducing rolling-shutter artifacts;
- More stable close-ups: paired with telephoto or macro, it more easily achieves sufficient in-focus range.
The costs are equally clear:
- It needs a micro stepper or shape-memory-alloy actuator, adding thickness, power draw, and failure points;
- Blade edges collect dust, and repeated actuation risks dust ingress or sticking;
- At small apertures, the diffraction limit begins to show. The Airy disk diameter is about 2.44·λ·N, where N is the f-number. With a visible wavelength of λ ≈ 0.55 μm, going from N = 2.0 to N = 4.0 grows the Airy disk diameter from about 2.7 μm to about 5.4 μm. If pixel size is near or below this value, sharpness drops noticeably. So a smaller aperture is not always better — there is an engineering-optimal range.
3.2 A system approach to “more light at night, no overexposure by day”
The typical traits of a new-generation sensor are higher quantum efficiency, larger full-well capacity, and dual-gain readout and other wide-dynamic-range techniques. But the sensor is only the starting point; the full imaging chain runs roughly as follows:
- Multi-frame capture and alignment: capture multiple frames with different exposure combinations and align them via optical flow or feature matching to suppress handshake and scene motion;
- Fusion and tone mapping: composite a wide-dynamic-range image in the fusion domain, then apply local tone mapping to darken highlights and lift shadows while preserving local contrast;
- Denoising and detail enhancement: denoise adaptively per region based on a noise model (photon shot noise + read noise) to avoid a smeared look in shadows;
- Coordinated exposure strategy: schedule aperture steps, shutter, gain, and multi-frame strategy together, rather than deciding each with fixed rules;
- Sustained performance management: in long-video and burst scenarios, control imaging-pipeline power and sensor/processor temperature to prevent throttling from causing quality jumps.
3.3 Illustrative variable-aperture exposure strategy
from enum import Enumclass Aperture(Enum):F14 = 1.4F20 = 2.0F28 = 2.8F40 = 4.0def choose_aperture(scene: dict, lens_sharpness: dict) -> Aperture:"""Variable-aperture exposure strategy (pseudocode)scene: {lux, subject_distance, motion_level, need_bokeh}lens_sharpness: {aperture: relative sharpness coefficient} from lens factory calibration"""lux = scene["lux"]if scene["need_bokeh"] and lux > 200: # bright portrait, prioritize bokehreturn Aperture.F14if scene["motion_level"] == "high" or lux > 5000:return Aperture.F28 # strong light or motion, prioritize DoF and shutterif lux < 10: # very dark, prioritize light intakereturn Aperture.F14if 0.1 < scene["subject_distance"] < 0.3: # close-up, widen depth of fieldreturn Aperture.F40# Normal scenes: pick the step balancing light intake and sharpnessreturn max(Aperture, key=lambda a: lens_sharpness[a.value] / (a.value ** 2))
4. Chip Packaging and the On-Device AI Compute Foundation
4.1 The memory wall: the most realistic on-device AI bottleneck
In traditional designs, compute units connect to memory through board traces in a star topology centered on the processor. Lately, relying on process scaling and architectural tuning alone to raise compute has struggled to keep pace with the growth of model size and data volume, and the “memory wall” has grown prominent:
- Latency: a longer round-trip distance means higher access latency;
- Energy: data movement is a sizeable share of system power. It is widely held that off-chip memory access costs one to two orders of magnitude more energy than on-chip cache, with exact figures varying by process and interface;
- Area and signal integrity: crowded board routing and worse crosstalk also squeeze battery space.
For on-device AI, inference repeatedly moves model weights and activations. So “closer, wider, and more efficient memory” improves real-world experience more than simply stacking compute.
4.2 Wafer-level packaging: moving memory next to the processor
Wafer-level packaging completes most steps — bump formation, redistribution, packaging — at the wafer level, rather than dicing into individual dies and packaging one by one. Its advantages include:
- Each package is closer to bare-die size, improving PCB space utilization;
- Fan-out technology enables denser interconnects and better electrical performance;
- Combined with 2.5D/3D stacking, through-silicon vias (TSVs), and hybrid bonding, a package can integrate the processor and memory.
The “wafer-level multi-chip module packaging” in the reports corresponds in industry to a typical form of integrating multiple dies within a single package — for example, packaging low-power memory dies together with the processor, shortening the interconnect between memory and compute from the “centimeter” scale to the “millimeter” scale.
Concept distinction: Packaging does not change the instruction set and does not automatically improve AI performance. Near-memory packaging translates into “lower latency, higher bandwidth, and better efficiency” only when the chip design reserves on-die/in-package memory controllers, matching physical interfaces, and a software stack (compiler, inference engine, memory allocator) that adapts to them.
4.3 Benefits and costs
| Dimension | Traditional board layout | Near-memory advanced packaging |
|---|---|---|
| Interconnect distance | Centimeter scale | Millimeter scale, some even micron scale |
| Signal integrity | Long traces, prone to crosstalk | Short interconnects, more controllable signal quality |
| Space utilization | Occupies PCB area | Integrated in-package, saves space |
| Thermals | Main components spread out | Concentrated heat, harder thermal design |
| Cost and yield | Mature, controllable cost | Multi-die integration and test, higher cost |
| Serviceability | Replaceable, easy to repair | High integration, hard fault isolation and rework |
Thermal design is the most easily overlooked constraint of near-memory packaging. Memory is temperature-sensitive, while the processor concentrates heat under sustained load. Thermal design must therefore proceed at three levels:
- Inside the package: thermal interface materials and die layout;
- Device level: vapor chambers, graphite sheets, and other heat-spreading materials;
- System level: dynamic power and thermal scheduling.
4.4 From parameters to experience: building a layered evaluation baseline
Parameters like “how many TOPS” or “how much memory” do not directly reflect user experience. It is better to build a layered evaluation baseline around on-device AI scenarios:
- Component level: memory bandwidth, access latency, and sustained performance under continuous load;
- Task level: time to first token, tokens per second, end-to-end time for typical tasks, and energy per task;
- Experience level: recognition accuracy, crash rate, temperature rise and throttling behavior, power draw, and compliance of privacy-related metrics.
5. Device-Cloud Collaboration: Letting Local Compute and Cloud Capabilities Each Play to Their Strengths
5.1 Why device-cloud collaboration is needed
| Scenario trait | Suggested placement | Reason |
|---|---|---|
| Real-time voice wake, face unlock, initial image understanding | On-device | Latency-sensitive, and raw data should not leave the device |
| Long-text summarization, complex reasoning, cross-document Q&A | Cloud | High model-size and compute demands |
| Batch offline processing (photo organization, log summaries) | Mainly on-device, cloud as supplement | Schedule during idle periods to balance power and accuracy |
| Model iteration and data reflow | Cloud | Needs centralized storage, evaluation, and staged rollout |
5.2 Common methods for on-device model miniaturization
- Quantization: e.g., converting FP16 weights to INT8 or even INT4 (with INT4 weights + high-precision activations, “weight-only quantization” cuts weight size to about one-quarter of FP16). Watch for accuracy regression on long-tail tasks;
- Knowledge distillation: use a large model as the teacher to distill a smaller model, then continue fine-tuning for the target task;
- Structured pruning and low-rank factorization: reduce parameters and compute;
- Operator and hardware adaptation: align model structure with NPU operator support to avoid frequent fallbacks to the CPU that drag down overall latency.
5.3 Illustrative routing strategy
from dataclasses import dataclass@dataclassclass Request:task_type: strestimated_tokens: intprivacy_level: int # 0-2, 2 means highly sensitivelatency_budget_ms: int# Capability table: task type -> (on-device model handle, max processable tokens)LOCAL_TABLE = {"asr": (local_asr, 30_000), "vision": (local_vision, 4_000)}def route(req: Request):handler, limit = LOCAL_TABLE.get(req.task_type, (None, 0))on_device_ok = (handler is not Noneand req.estimated_tokens <= limitand req.latency_budget_ms < 500 # prioritize local for real-time tasks)if req.privacy_level == 2 or on_device_ok:return ("local", handler(req) if handler else local_fallback(req))return ("cloud", cloud_client.chat(req)) # complete data masking before cloud
5.4 The cloud side: a general capability-category view
On-device AI does not mean the cloud has nothing to do. The cloud mainly handles training and iteration, elastic inference, data governance, and observability, which break down into the following general capability categories:
| General capability category | Typical role in the on-device AI chain |
|---|---|
| Object storage | Store training data, model files, and version snapshots |
| Message queue | Decouple tasks and balance inference-request traffic peaks |
| Container platform | Deploy and elastically scale inference services to handle traffic peaks |
| Logging service | Aggregate on-device metrics and masked logs |
| Monitoring and alerting | Track latency, success rate, and model quality; trigger degradation or rollback |
For example, with Baidu AI Cloud, developers can compose architectures from these general capability categories and align them with the privacy policies, data masking, and data-minimization principles in on-device SDKs.
5.5 The privacy and compliance baseline
- Minimal collection: report only necessary data, preferring structured metrics over raw data;
- On-device-first processing: for sensitive information such as faces, voiceprints, and precise location, process on-device by default;
- Masking and de-identification: data that must be uploaded should be de-identified and aggregated before transmission;
- Auditable and revocable: keep processing logs for audit and give users an off switch and a data-deletion entry point;
- Compliance alignment: design data-processing flows to meet the personal-information protection requirements of the region where the business operates.
6. Common Misconceptions and Rational Expectations
- “Punch-hole means under-display”: they are different approaches — the former favors image quality, the latter favors display completeness — and in the near term they will likely coexist.
- “A smaller aperture is always better”: small apertures hit the diffraction limit; aperture choice must weigh lens quality, pixel size, and the scene together.
- “Near-memory packaging = 2× performance”: packaging gains require chip design and the software stack to work together; a packaging upgrade alone is not a performance gain.
- “Bigger numbers mean smoother”: memory capacity, memory bandwidth, and thermal capability must match; a weak link in any one turns paper compute into something the user cannot feel.
This article makes no call on the “AI bubble” debate. From an engineering view, rather than debating the aggregate, it is better to land on observable metrics: the actual usage rate of on-device AI tasks, energy and latency per task, and the downward trend of per-task cost with scale. Whether a technology roadmap survives the cycle ultimately depends on whether it can keep delivering measurable, reproducible, user-perceivable value.
7. Conclusion
Overall, the three threads — display form, imaging optics, and near-memory advanced packaging — are together carrying on-device AI from “usable” to “delightful.” What they share: gains come from system-level coordination, not single parameters; the difficulty lies in the multiple coupling of optics, thermals, power, compute, and the software stack; and success ultimately rests on engineering execution.
For developers and architects, start from four areas:
- Build a layered evaluation baseline and drive decisions with data, not parameters;
- Design a device-cloud collaboration architecture so local compute and cloud capabilities each play to their strengths;
- Connect the full chain of “hardware-capability query → inference-engine adaptation → model quantization and compression” as early as possible;
- Treat privacy and compliance as a design constraint, not an after-the-fact patch.
For enterprise decision-makers still watching the AI investment hype, rather than chasing concepts, invest effort in measurable capability accumulation — device-cloud inference pipelines, evaluation systems, data governance, and cost models. A new round of terminal-hardware upgrades has begun, and what ultimately decides success is who can turn these technology options into a stable, secure, user-perceivable product experience.
