Display, Imaging, and On-Device AI in Flagship Mobile Devices: How System-Level Design Is Evolving

1. Full-Screen Evolution: From Exposed Sensors to Under-Display Integration

Early full-screen designs usually relied on a large cutout or a dedicated area at the top of the screen. That space housed the front camera, proximity sensing, ambient light sensing, and identification modules. This approach is mature and reliable, but it breaks up the display. The break is most noticeable during full-screen video, gaming, and immersive reading.

Under-display sensing aims to place some optical or infrared components beneath the display panel, so the screen looks more complete. The goal is to balance display quality against sensing accuracy.

A display is built from several layers, including the emissive layer, the driver layer, the encapsulation layer, polarizing structures, and the protective cover. For an identification module that must emit and receive specific light signals, these layers add three problems: light loss, scattering, and diffraction. Poor handling can cause the following:

1. Weaker signal strength: Screen materials absorb or block part of the light, so the receiver gets a weaker usable signal.
2. More pattern distortion: The pixel arrangement and electrode structure can disturb the light path and affect the recovery of depth or structured light information.
3. Display consistency issues: If the sensor area uses a special pixel layout, its brightness, color, or graininess may differ from the surrounding area.
4. Higher reliability requirements: Local changes in the screen structure may affect long-term aging. Development must assess stability under high and low temperatures, drops, humidity, and long periods of display use.

Under-display solutions therefore require the display panel, optical module, image processing algorithms, and system security modules to be optimized together. On the hardware side, the target wavelength band must have higher transmittance. On the software side, signal calibration, noise suppression, and liveness detection algorithms are needed to improve recognition reliability in complex environments.

From a product standpoint, moving some sensors under the display while keeping a front camera cutout is a safer transitional path. It improves the front-facing visual completeness and keeps the mature selfie, video call, and portrait capture capabilities. It also builds manufacturing and algorithm experience for more demanding under-display camera solutions later.

2. Why Front Camera Cutouts Will Persist

Even as under-display sensing matures, front cameras will likely keep their own cutouts for some time. Cameras demand more image quality than identification sensors do.

Front cameras handle video calls, selfies, portrait blur, short video recording, augmented reality interaction, and remote collaboration. Users are sensitive to sharpness, skin tone reproduction, backlit performance, dynamic range, and low-light noise. If the camera sits entirely under the display, common problems include:

• Lower image contrast;
• Glare or haze in highlight areas;
• Detail resolution limited by the pixel structure;
• More visible noise in low light;
• Imaging inconsistencies across different display states.

The front camera therefore tends to evolve along a path of reducing the visible area, optimizing the cutout position, improving under-display light transmission, and adding computational photography compensation. Development teams should not focus only on shrinking the cutout. They also need a complete image quality evaluation system that covers at least still photos, video, portraits, backlighting, night scenes, mixed indoor lighting, and motion capture.

3. Mechanical Variable Aperture: More Physical Control Over Mobile Imaging

In rear camera systems, the mechanical variable aperture is an important hardware enhancement. Unlike a fixed aperture lens, a variable aperture uses a mechanical structure to adjust the effective opening of the lens. This changes how much light reaches the sensor.

Aperture size affects both exposure and depth of field. In a simplified form, the light throughput entering the lens is roughly proportional to the square of the aperture diameter:

Light throughput ∝ aperture diameter²

A larger aperture admits more light per unit time. This helps shorten exposure time and reduce ISO in dark scenes, and it reduces blur caused by hand shake or subject motion. A larger aperture also produces shallower depth of field, which creates a more obvious separation between the subject and the background.

A smaller aperture admits less light, but the depth of field increases. In bright daylight, high-brightness scenes, or scenes where more foreground and background detail must stay sharp, a smaller aperture lowers the risk of overexposure and extends the range of sharp areas in the image.

Compared with purely software-based background blur, a mechanical aperture offers the following advantages:

1. Real control over light intake: Software can adjust exposure parameters, but it cannot change how much light the lens actually receives. A mechanical aperture improves exposure conditions at the optical level.
2. More natural depth-of-field transitions: Software portrait blur depends on subject segmentation and depth estimation, which can fail at complex edges. Optical depth of field forms naturally and handles hair, transparent objects, and fine outlines more faithfully.
3. Better tolerance in bright scenes: In very strong light, closing the aperture reduces the chance of sensor saturation and preserves more information for later high dynamic range merging.

A variable aperture does not automatically make a camera better. Space inside a mobile device is extremely limited, so mechanical parts must balance thickness, drop resistance, service life, noise, and power consumption. Development should focus on testing blade consistency, repeat positioning accuracy, action stability at low temperatures, and wear after long-term use.

4. New Image Sensors: Balancing Night Light Capture and Daytime High Dynamic Range

The core job of an image sensor is to convert light into an electrical signal. The image processing chain then turns that signal into the final photo or video. Mobile imaging has long faced a basic conflict. Dark scenes require collecting as much light as possible, while bright scenes require avoiding overexposure in local areas.

To address this, sensors and image processing systems are usually optimized from several directions.

1. Improving Effective Light Sensitivity

Larger photosensitive pixel areas generally help collect more photons, which improves the signal-to-noise ratio. Lens light transmission, pixel structure, charge conversion efficiency, and readout noise also affect the final low-light image quality.

In night photography, the system should not simply raise brightness. It must balance brightness, noise, color accuracy, and motion sharpness. For example, a long exposure can brighten the image, but people, vehicles, or hand shake can cause motion blur. Aggressive noise reduction can produce a cleaner image but may erase texture detail.

2. Extending Dynamic Range

Dynamic range describes a device’s ability to keep detail in both shadows and highlights. In high-contrast scenes, such as a portrait by a window, a sunset, stage lighting, or neon signs at night, a sensor with limited full-well capacity can lose highlight detail and produce large areas of pure white.

Common approaches include multi-frame exposure merging, dual conversion gain, staggered exposure readout, and local tone mapping. The goal is not to make the image look brighter. The goal is to keep as much information from the real scene as the display can show.

High dynamic range processing can be understood as a constrained fusion process:

Final image = shadow exposure information + midtone exposure information + highlight protection information

Real algorithms must also handle motion detection, ghost suppression, white balance consistency, and local contrast control. If multi-frame alignment fails, the image can show double edges. If local enhancement is too strong, the result can look unnatural, with smeared textures or halo artifacts along edges.

3. Building Scene-Aware Imaging Strategies

High-quality imaging systems need to understand the scene. Night portraits, document scanning, moving pets, concert lighting, and backlit landscapes each have different priorities. Based on ambient brightness, subject motion, image content, and user actions, the system can choose different exposure, focus, noise reduction, and merging strategies.

Scene recognition should serve image stability and should not change true colors too much. For professional users and content creators, predictable, consistent, and repeatable output often matters more than aggressive automatic enhancement.

5. How Advanced Packaging Supports On-Device AI

The key value of on-device AI is running some inference tasks directly on the device. This reduces dependence on remote services. It lowers response delay caused by network fluctuations, and it keeps some sensitive data on the device, which improves privacy and offline usability.

Running larger models locally, however, requires efficient data transfer between the processor, graphics processing unit, dedicated AI accelerator, and memory. In traditional designs, the compute chips and working memory sit at some physical distance from each other. Frequent data movement adds latency and energy cost.

Wafer-level multi-chip module packaging takes a different approach. It combines multiple functional chips with shorter interconnect paths through tighter packaging. For mobile devices, the potential benefits include:

• Shorter data transfer paths for some workloads;
• Lower energy use for frequent high-speed data exchange;
• Better latency when compute units access memory;
• A hardware foundation for running larger local models;
• Higher system integration within limited device space.

Advanced packaging is not a simple performance upgrade. Because multiple chips are packed closely together, thermal design becomes more complex. On-device AI inference often involves sustained computation and heavy memory bandwidth use. If the cooling design is insufficient, rising temperature can force the device to lower its clock speed, which causes performance fluctuation.

The device system should therefore balance tasks dynamically. For example, low-priority tasks can run when the device is idle or charging. Real-time interactive tasks should get priority for fast response. Long, heavy generation tasks need limits set by power and temperature policies.

A simplified on-device task scheduling logic can be written as follows:

If the task requires immediate feedback:
    prioritize low-latency local inference
Else if the task is computationally heavy and the network connection is stable:
    choose the processing location based on privacy level, energy budget, and user authorization
If device temperature keeps rising:
    reduce concurrency, shorten inference length, or defer non-urgent tasks

6. Which Problems On-Device AI Should Solve First

For device makers and app developers, on-device AI should go beyond proving that a device can run a model. Features should be designed around clear user value. Tasks that suit local processing usually have the following characteristics:

1. Sensitivity to response time: Examples include real-time captions, voice wake-up, input prediction, image preprocessing, and interaction assistance. These tasks need to minimize waiting time.
2. Involvement of personal data: Examples include photo library analysis, personal document summarization, offline translation, and local search. Processing these locally can reduce how much data leaves the device.
3. Need for offline use: In areas with poor network coverage, where data costs matter, or when users turn networking off, local capability offers clear advantages.
4. Controllable input size: Limited by memory, power, and heat, mobile devices suit tasks with bounded length and clear goals better than unlimited large-scale computation.

On the software side, the on-device AI capability should be split into five layers: model management, data preprocessing, inference execution, result post-processing, and privacy control. This reduces coupling between business features and the underlying hardware. It also makes it easier to replace or extend models as chip capabilities improve.

7. Conclusion: Device Innovation Is Shifting From Individual Parameters to System Engineering

Full-screen design, variable apertures, new image sensors, and advanced chip packaging look like separate fields. Each one, however, aims at the same goal: improving a mobile device’s sensing, computing, and interaction naturalness within limited size, battery capacity, and thermal headroom.

Under-display sensing brings the front of the device closer to a complete display. Keeping a front camera cutout during this transition reflects a practical balance between image quality and form factor. Mechanical apertures and high dynamic range sensors strengthen how faithfully devices record scenes in complex lighting. Advanced packaging provides the foundation for stronger local AI computing.

Competition among future flagship devices will depend more on hardware and software working together. Development teams need to advance component selection, structural design, imaging algorithms, power management, model compression, privacy protection, and user experience together. Only then can individual technologies become stable, reliable, and noticeable product value.