HarmonyOS On-Device AI Assistant Upgrade: Multi-Agent Collaboration and Voice-Driven Document Generation

1. Background: On-Device AI Shifts from Q&A to Task Execution

As mobile device computing power increases and device-cloud collaborative architectures mature, on-device AI assistants are no longer limited to simple Q&A or information retrieval. They are beginning to take on more complex roles in task planning and execution. Particularly in scenarios like enterprise office work, academic research, and content creation, user expectations for AI have shifted from “providing suggestions” to “directly delivering a finished work product.”

A recent capabilities upgrade for the AI assistant within the HarmonyOS ecosystem introduced core changes in three main areas:

1. Enhanced PPT Generation: Supports custom layouts and voice-interactive input.
2. Multi-Agent Parallel Scheduling: Can simultaneously schedule up to a hundred intelligent agents to handle complex tasks.
3. Open External Skill Integration: Expands capabilities into office collaboration, information management, and other domains through a universal skill integration mechanism.

The combination of these capabilities essentially pushes on-device AI from a “single-model conversation” toward an “orchestrable intelligent task execution system.”

2. PPT Generation Upgrade: From Template Matching to Intent Understanding

The main pain point of traditional AI-generated presentations is that users input a text description, and the system outputs a result using a fixed template, which often deviates significantly from user expectations. Users must repeatedly adjust their text descriptions or manually modify the layout of each slide, yielding limited efficiency gains.

2.1 Custom Layouts and Template Constraints

The new version supports explicit constraints on layouts before generation. Users can specify the presentation type (e.g., business report, academic defense, creative showcase), and the system will match the appropriate information hierarchy and visual structure. Essentially, this moves the “design intent” to the input phase rather than leaving it for post-generation modifications.

From a technical perspective, this is similar to adding a layer of structured conditional input to the generation model. Users describe not only the content but also the structure in which it should be presented. For example:

- Type: Business Report
- Structure: Background Analysis -> Market Opportunity -> Solution Design -> Implementation Path -> Expected Revenue
- Style: Data charts prioritized, concise text

This structured description significantly reduces the uncertainty of the generated results, notably increasing the usability of the first draft. For professionals who frequently create reporting materials, this reduces not only generation time but also the hidden costs of repeated revisions.

2.2 Voice Interaction and Multi-Turn Intent Clarification

The value of voice input lies not just in “freeing hands” but in fundamentally changing the interaction mode. When typing, users tend to compress their requirement descriptions. Voice communication, being closer to natural conversation, can carry more complex logical expressions and contextual associations.

In the new version, users can describe their PPT requirements through voice dialogue, and the system supports multi-turn clarification. For instance, if a user says, “Help me make a presentation on the Q3 sales analysis,” the system can follow up with, “Should it focus on regional comparison or product line analysis?” or “Do you need to include channel cost data?” This interaction method makes requirement gathering more comprehensive, thereby improving generation quality.

Behind this capability is the collaboration of speech recognition, natural language understanding, and task planning modules. Voice is no longer just a substitute for text input; it has become a crucial means of building task context.

3. Multi-Agent Clusters: Key Architecture for Parallel Task Execution

3.1 From Single Agent to Agent Cluster

When processing tasks, a single AI agent is limited by its context window length and the number of reasoning steps. Its efficiency is noticeably inadequate for complex tasks requiring cross-domain knowledge and multi-step parallel research.

The multi-agent cluster mechanism introduced in this upgrade is a direct response to this bottleneck. The core idea is to decompose a complex task into several parallel sub-tasks, which are processed by multiple agents respectively and then aggregated.

Typical Application Scenarios:

• Literature Review Writing: A student needs to review core literature from the past five years in a specific research direction. This can be broken down into sub-tasks like “retrieval and filtering,” “classification and induction,” “method comparison,” and “research gap analysis.”
• Industry Research Reports: Professionals needing an in-depth report on a niche industry can process market size, competitive landscape, policy environment, and technology trends in parallel.
• Multi-language Information Integration: When analyzing materials in different languages and generating a unified format output, different language processing tasks can be executed simultaneously.

3.2 Task Decomposition and Scheduling Mechanism

From an engineering perspective, a multi-agent cluster needs to solve three core problems:

First, task granularity division. If decomposed too finely, the communication and merging costs between agents will outweigh the benefits of parallelism. If decomposed too coarsely, individual sub-tasks remain too heavy to reflect parallel advantages. A reasonable approach is to divide based on natural task boundaries, such as by data source, analysis dimension, or output chapter.

Second, agent role allocation. Different sub-tasks require agents with different capabilities, such as information retrieval, data analysis, or text generation. The cluster needs to allocate roles dynamically based on task characteristics rather than using a fixed, uniform model.

Third, result consistency and merging. When multiple agents process in parallel, output formats may be inconsistent, and information coverage may overlap. Unified validation and deduplication merging are required during the aggregation phase to ensure the quality of the final deliverable.

According to official disclosures, the current system supports scheduling up to 100 agents working in parallel, simultaneously processing approximately 1,500 execution steps. This scale is sufficient to cover the vast majority of personal office and lightweight team collaboration scenarios. Research and initial drafting that previously took hours can be compressed into minutes for an initial version delivery under the cluster mode, improving efficiency by roughly an order of magnitude.

3.3 Practical Considerations

Although the multi-agent cluster capability is powerful, users should still note the following:

• Clarity of task description directly impacts decomposition quality: If user input is too vague, the system cannot effectively break down sub-tasks, making it difficult to leverage the cluster’s advantages.
• Real-time validation and human intervention are still necessary: AI-generated drafts can serve as an efficient starting point, but critical data and conclusions still require manual confirmation.
• Applicable scenarios have boundaries: For tasks requiring deep original thinking or highly personalized expression, parallelly generated results may require more post-editing.

4. External Skill Integration: From Closed Tool to Open Platform

4.1 Significance of the Skill Integration Mechanism

The functional boundaries of any AI assistant are limited. Rather than continuously building in new features, providing a standardized external skill integration mechanism allows ecosystem participants to expand capability ranges. In this upgrade, the AI assistant supports connecting to external skill platforms, allowing users to invoke skill modules covering multiple domains with a single click.

The value of this mechanism lies in:

• Reduced user switching costs: There is no need to jump between multiple applications. Information retrieval, data processing, and message management can be completed within a single entry point.
• Composable skills: Different skills can be chained to form workflows, such as retrieving data, generating charts, and finally embedding them into a document.
• Ecosystem extensibility: More skills can be integrated via unified standards in the future, continuously expanding the AI assistant’s capability boundaries.

4.2 Typical Skill Invocation Scenarios

Taking office scenarios as an example, users can achieve the following through skill integration:

• Chat Software Binding and Message Processing: Linking the AI assistant with common communication tools to enable features like information summarization, to-do extraction, and auto-reply drafting.
• Data Query and Report Generation: Connecting to data sources via skill plugins, executing queries, and generating visualized results.
• Industry-Specific Information Processing: Invoking specialized data collection and analysis skills for niche sectors like agriculture, farming, or manufacturing.

These capabilities essentially expand the AI assistant from an “information provider” to an “action executor,” enabling it to intervene in users’ actual workflows.

5. Observations and Suggestions for Developers

For teams focusing on on-device AI application development, this upgrade provides several noteworthy signals:

5.1 Continuous Enhancement of On-Device Model Capabilities

In the past, many complex AI tasks had to rely on cloud-based models, making latency and privacy issues unavoidable. Today, on-device systems can support multi-agent parallel scheduling and highly complex document generation, indicating a substantial improvement in terminal-side inference capabilities. When designing applications, developers can more actively integrate AI capabilities into local workflows.

5.2 Multi-Agent Architecture Becomes an Application Layer Focus

Single-model conversation is no longer the competitive focus. The key to differentiation is how to organize multiple agents to collaborate efficiently around specific tasks. Developers are advised to focus on engineering problems such as agent orchestration, task planning, and result merging. These capabilities often determine the user experience more than the model itself.

5.3 Re-evaluating Voice Interaction

Voice is no longer just “a supplementary input method”; it is becoming a core interaction modality for task definition and context construction. For document generation and content creation applications, voice interaction combined with multi-turn clarification can make requirement gathering more complete and generation results more aligned with expectations. Developers should consider designing voice as a core interaction entry point rather than a simple add-on feature.

6. Conclusion

This upgrade to the HarmonyOS on-device AI assistant reflects the evolutionary trend of on-device AI moving from “being able to converse” to “getting things done.” The improvement in PPT generation focuses on intent understanding and interaction optimization; the multi-agent cluster solves the processing efficiency problem for complex tasks; and external skill integration opens a channel for capability expansion.

For general users, this means that in daily office work and study, they can describe needs more naturally and obtain deliverables in less time. For developers, it requires paying attention to the latest advancements in on-device AI capabilities and considering how to integrate patterns like multi-agent collaboration, voice-driven generation, and open skill integration into their product designs. When AI assistants truly become collaborative systems capable of executing tasks rather than simple Q&A tools, the form of productivity tools will undergo a deeper transformation.