Mobile AI Assistant Architecture: Package Size, Model Routing, and Cross-Device Scheduling

Mobile LLM assistant usage is shifting from the web to phones. Phones operate under unstable networks, limited memory and battery, and users who are sensitive to storage consumption. Developers must handle package-size control, model scheduling, multi-device synchronization, and scenario expansion within these constraints. This article reviews the practices, boundaries, and risks of these areas from an engineering perspective. It does not recommend specific products.

How Can the Install Package for a Mobile AI Assistant Be Reduced?

Install package size directly affects download conversion, but there is no universal answer to “how small is small enough.” Before discussing package size, teams need to define the measurement basis: is it the raw install package (APK/IPA) size, or the size users actually download after app store splitting and compression? Does it include dynamic libraries, all ABI architectures, and bundled resources? Different definitions can produce results that differ by several times. Without a reproducible measurement environment, any number is only a rough order-of-magnitude reference; teams should measure against their own baseline.

Lightweighting is mainly achieved through three things.

1. Reuse core logic across platforms. Non-UI logic such as network communication, encryption, and state management can be implemented in Rust or C++ and exposed to Android and iOS through FFI. This reduces duplicate code across the two platforms, but memory management and the threading model across FFI boundaries need attention; otherwise, hard-to-diagnose crashes can be introduced.

2. Fetch large dependencies on demand. Libraries related to on-device inference (such as .so files) and model resources can be large and do not have to ship with the main package. The rules differ between the two platforms:

• Android: native libraries can be loaded lazily through dynamic feature modules or split mechanisms. Downloaded code must pass signature verification and version compatibility checks and comply with app store policies.
• iOS: app stores prohibit downloading and executing executable code that changes app functionality. What can be downloaded on demand is data such as model weights, vocabularies, and configuration files, or system-provided on-demand resources can be used. Native libraries should ship with the package or be handled within what platform review allows.

1. Keep the UI layer native. Cross-platform UI frameworks add package size and startup overhead. Native interfaces combined with lightweight rendering components usually balance startup speed and interaction smoothness. Whether to adopt cross-platform UI should depend on team staffing, target platforms, and performance requirements.

How Should Multi-Model Routing Be Designed?

A mobile assistant usually connects to multiple models to balance response speed, reasoning depth, and cost. A common model matrix is as follows:

Model type	Suitable tasks	Characteristics	Fallback strategy
Flagship	Complex reasoning, code generation, long-text analysis	High quality, but also high latency and cost	Switch to Standard after quota is exhausted or timeout
Standard	Everyday Q&A, writing, general summarization	Balanced quality and speed	Switch to a small on-device model when cloud is unavailable
Lightweight	Short Q&A, instruction confirmation, completion	Low latency, suitable for high-frequency calls	Acts as the ultimate fallback and is always available

Routing decisions should not rely on a single model. A layered approach is recommended:

1. Rule-based routing: route directly based on explicit features such as input length, presence of code blocks, or long file uploads. Overhead is minimal.
2. On-device classification: when rules cannot decide, a small on-device model returns a complexity label and confidence. Its size, single-inference latency, and accuracy need to be measured; this article does not assume specific values. Classification time should be included in the time-to-first-token budget; if it exceeds the budget, skip classification.
3. Cloud fallback: when on-device confidence is below a threshold, send the request to a cloud classifier or use the Standard model directly.
4. Misclassification recovery: when users are unsatisfied with an answer, provide an entry point to “retry with a stronger model” and log misclassification samples for threshold tuning.

Offline scenarios need separately defined behavior. Without a network, only the local model can be used. The app should clearly indicate the current capability scope and avoid silent degradation that makes users mistake local results for cloud results.

Billing and permissions should also be handled centrally at the gateway. Free users use the Standard model by default and have a context-window cap; paid users can unlock the Flagship model and longer context. The gateway needs to handle authentication, quotas, and rate limiting. When a request exceeds limits, it should return an understandable message rather than a raw error.

What Are the Boundaries of Cross-Device Sync and Command Scheduling?

Multi-device sync involves two types of data, and they should be designed separately.

Conversation history and settings can use incremental sync. Long-lived connections such as WebSocket can push updates in real time, while a local database (such as SQLite) stores an offline copy. CRDTs are suitable for merging concurrently modifiable state, such as tags, favorites, and reading progress. But they cannot automatically resolve all conflicts. For example, whether an instruction sent simultaneously from two devices should execute, and how many times, cannot be decided by CRDTs; business rules are required.

Command scheduling cannot rely only on eventual consistency. When a phone remotely directs a desktop Agent to perform code generation, builds, or tests, the following problems need to be solved:

• Idempotency: each command carries a unique ID; the desktop must not execute duplicate receipts.
• Offline queue: when the desktop is offline, commands enter a queue and are delivered in order after reconnection, with an expiration time.
• Status reporting: execution logs, code diffs, and task status are reported in stages to avoid large files filling the connection.
• Notifications: a local notification is triggered when a task completes, but notification content should not include sensitive code or keys.

Risk control should be front-loaded:

• Sensitive commands (such as deleting files, pushing code, or changing permissions) must require a second confirmation on the phone.
• The desktop Agent runs with least privilege, exposing only the project directory and necessary commands.
• Logs are sanitized before upload, removing tokens, passwords, and personal information.
• The initiator, time, parameters, and result of each command are written to an audit log and retained long enough for traceability.

In architecture, scheduling should not remain embedded in the chat app indefinitely. As workflows become more complex, embedded scheduling causes the main app’s dependencies and permissions to expand quickly. Splitting scheduling into a separate collaboration app or process while keeping chat and interaction in the main app is a common evolution path. After the split, the interface between the two needs to remain stable.