Mobile AI Assistant App: Architecture Design and Best Practices

1. Background and Problem Definition

Mobile intelligent conversation assistants have evolved from simple Q&A tools into comprehensive entry points that support multi-turn dialogue, file understanding, image recognition, and voice interaction. For developers, the challenge lies not in calling a model API once, but in building a stable client-side experience around the model’s capabilities. Common challenges include:

• Frequent network fluctuations causing long replies to disconnect midway;
• Limited memory and battery on mobile devices, leading to laggy rendering during long conversations;
• Sensitive information in history and attachments requiring strict storage and transmission controls;
• The need for unified management of multi-device accounts, subscription statuses, and feature toggles.

This article does not focus on any specific product. Instead, it breaks down such applications into general technical problems and explains the design concepts and implementation key points one by one.

2. Functional Module Decomposition

A mature mobile conversation application typically includes the following six modules:

1. Conversation Layer: Responsible for the creation, switching, renaming, deletion, and retrieval of conversations.
2. Message Layer: Handles structured storage, streaming appending, and state management of messages. States include sending, generating, completed, and failed.
3. Input Layer: Supports multiple input types such as text, voice transcription results, images, and files.
4. Network Layer: Encapsulates request signing, timeout retries, streaming connections, and disconnection recovery.
5. Security Layer: Manages identity credentials, local encryption, and log desensitization.
6. Observation Layer: Collects crash information, latency metrics, error codes, and user feedback.

The clearer the module boundaries, the smaller the scope of changes required when switching model services or adjusting interaction methods.

3. Overall Architecture Design

A layered architecture is recommended. The presentation layer only handles rendering and interaction, the business layer processes conversation logic, the data layer manages local storage and remote synchronization, and the network layer shields protocol details from the server.

The schematic structure is as follows:

+-----------------------------+
| Presentation: Pages & UI    |
+-----------------------------+
| Business: Conv, Msg, State  |
+-----------------------------+
| Data: Cache, Sync Queue     |
+-----------------------------+
| Network: Stream, Retry, Auth|
+-----------------------------+
              |
              v
  Server Gateway -> Inference Service

The server side typically includes a gateway, authentication, rate limiting, conversation persistence, and inference services. To handle attachments, object storage can be used to save files, and message queues can asynchronously parse and index them. Logging services and monitoring alerts are used to track request chains and exception trends.

4. Implementation of Streaming Responses

Models usually return content segment by segment during generation. If the client waits for the complete result before rendering, users will experience noticeable latency. Streaming rendering must achieve incremental parsing, throttled refreshing, and interruptibility.

Schematic code (pseudo-code):

function startStreamingReply(conversationId, userMessage):
    message = createAssistantMessage(conversationId, status = "generating")
    buffer = ""
    lastRender = now()
    connection = openStream(endpoint = "/v1/chat/stream", body = {
        conversation: conversationId,
        input: userMessage.content
    })
    connection.onChunk(chunk):
        buffer = buffer + chunk.text
        // Throttle UI updates to avoid frame drops
        if now() - lastRender >= RENDER_INTERVAL_MS:
            updateMessageContent(message.id, buffer)
            lastRender = now()
    connection.onComplete():
        updateMessageContent(message.id, buffer)
        setStatus(message.id, "completed")
    connection.onError(err):
        setStatus(message.id, "failed")
        saveDraft(message.id, buffer) // Preserve received content as draft

Three points to note during implementation:

• Render Throttling: Refreshing the UI on every received chunk will cause frame drops. Updates should be merged at fixed intervals, with the interval value determined through real-device testing.
• Interruptibility: When the user clicks stop, the client should proactively close the connection and retain the partially generated content.
• Draft Preservation: Save the received content during network interruptions so the user can choose to retry or continue generation.

5. Conversation Context Management

A model’s context window is limited, and long conversations cannot infinitely send all historical messages to the server. There are three common approaches:

1. Sliding Window: Retains the most recent few rounds of messages. Simple to implement, but early key information is easily lost.
2. Summary Compression: Generates summaries for older history, replacing the original text with the summary. This balances cost and coherence.
3. Key Information Extraction: Separately records user-specified preferences, constraints, and facts, attaching them to every request.

These three strategies are often used in combination. Schematic code:

function buildContext(messages, tokenBudget):
    pinned = messages.filter(m => m.pinned)
    recent = []
    used = tokenCount(pinned)
    // Iterate backwards to find recent messages that fit the budget
    for m in reverse(messages where not m.pinned):
        cost = tokenCount(m)
        if used + cost > tokenBudget: break
        recent.prepend(m)
        used = used + cost
    return concat(pinned, recent)

Tokenization rules depend on the specific model, and the client can usually only estimate the token count. Therefore, the server should re-verify before final inference and return a clear error code if the limit is exceeded, rather than silently truncating.

6. Weak Network and Offline Strategies

Mobile network environments are complex, and applications must define behaviors for each state in advance.

• Check network status before sending. When offline, place the message in a pending queue and mark it as unsent in the UI.
• Use exponential backoff for request failures, limiting the maximum number of retries to avoid draining battery and data.
• For operations requiring idempotency (like deleting a conversation), use a client-generated unique request identifier to prevent duplicate execution.
• Save conversation indexes and recent messages locally. Offline users can still browse history but cannot initiate new generation requests.

Upon network recovery, sync tasks should replay the queue in chronological order and handle conflicts. Conflict rules should be defined during the design phase, such as prioritizing server timestamps.

7. Security and Privacy Design

Security design should follow the principles of least privilege and defense in depth.

1. Credential Storage: Access tokens should be stored in the secure storage provided by the mobile OS, not in plaintext config files or regular logs.
2. Transmission Encryption: All interfaces should use encrypted transmission and verify server certificates.
3. Local Data Encryption: Conversation databases and attachment caches should be encrypted using device keys, with keys managed by the system keystore.
4. Log Desensitization: Only log request latency and error codes. Do not log complete conversation content, phone numbers, ID numbers, or other sensitive fields.
5. Permission Requests: Permissions for microphone, photo gallery, and file access should be requested only when the user actually uses the corresponding feature, with a clear explanation of the purpose.

Additionally, the application should provide options to clear local data, export conversations, and delete accounts to meet relevant compliance requirements.

8. Performance Optimization Suggestions

• List Virtualization: Long conversations should only render messages in the visible area to reduce view node counts.
• Lazy Loading: Show thumbnails for images and attachments first, downloading originals only when needed.
• Index Design: Index conversation IDs and timestamps to ensure stable response times for history retrieval.
• Request Merging: Combine multiple small-scale sync requests into batch requests to reduce network wake-ups.
• Memory Reclamation: Release rich text caches for invisible messages when switching conversations.

Performance targets should be based on real-device data. It is recommended to conduct stress tests on mid-to-low-end devices, not just verifying on high-end models.

9. Observability and Quality Evaluation

After launch, continuous monitoring of the application state is necessary. It is recommended to establish the following metrics:

• First Token Latency: The time from the user sending a message to the first character appearing in the UI.
• Generation Completion Rate: The proportion of normally completed replies out of all requests.
• Exception Distribution: Statistics of failure causes categorized by error code, network type, OS version, and app version.
• Crash Rate: Tracked separately by version and device model.
• User Feedback: Tracking likes, dislikes, and text feedback on replies for regression analysis.

Metric collection must comply with privacy regulations, reporting only aggregated data and anonymous identifiers.

10. Implementation Suggestions

1. First implement conversation management, streaming rendering, and basic error handling to form a usable version, then gradually add capabilities like attachments and voice.
2. Put server addresses, timeout values, and feature toggles into remote configurations for easy gray releases and quick rollbacks.
3. Establish a unified error code mapping table to translate server errors into user-friendly prompts.
4. At the deployment level, general container platforms and object storage cloud resources can be used. If you choose Baidu AI Cloud, you can use its general-purpose services directly without customizing specific capabilities for this scenario.