Multi-Model Integration and Filing for Embodied Large Models: An Engineering Practice
This article describes an engineering approach to integrating multiple large language models. It covers a unified abstraction layer, routing based on task type, health, budget, and quality, and context window management. It also explains what China's generative AI service filing requires, and the safety, logging, and rollout controls needed for embodied large models that drive physical devices.
1. Why Integrate Multiple Models
Models from different vendors differ clearly in parameter scale, training data, inference speed, context window length, and domain-specific capability. Intelligent workflows often combine conversation, code generation, structured extraction, and image understanding. A single model rarely performs best on all of these tasks, so integrating multiple models has become a basic architecture requirement for many teams.
Multi-model integration mainly addresses three problems.
Complementary capabilities. A lightweight model can handle simple intent recognition, while a larger model handles complex reasoning. When each task is matched to a suitable model, output quality and response speed are usually both maintained.
Service availability. A single provider may become unavailable during traffic peaks, network instability, or version upgrades. With several upstream channels, traffic from a faulty channel can switch to a backup channel, which keeps the business running.
Cost control. Billing standards differ widely across providers. In some scenarios, a smaller model already meets requirements. For example, if a classification task costs one-tenth as much per call on a small model as on a large one, and the accuracy gap falls within acceptance limits, the small model should be preferred. Cost benefits must be measured with your own business data. Unit prices alone do not justify the choice.
2. Designing a Model Abstraction Layer
Before integrating multiple models, build a unified abstraction layer. Downward, the layer hides each provider’s HTTP communication details, authentication methods, and error codes. Upward, it gives business code one consistent calling interface. Business code therefore does not need to know which model runs underneath.
The abstraction layer usually has three core components.
- Unified request builder: maps internal parameters (prompt, temperature, frequency penalty, maximum output tokens) to the field names and structures each provider requires.
- Response normalization module: converts raw responses from different models into a standard object that contains the text, token usage, finish reason, and model ID.
- Error mapping and retry module: defines standard exception types, such as timeout, rate limit, and authentication failure, and decides whether to retry based on the error type.
The code below shows the call flow. It is illustrative and should not be used in production as written. get_adapter and each adapter must be implemented for the specific provider, and dependency versions should be pinned in the project.
import timefrom dataclasses import dataclassclass RateLimitError(Exception):"""Raised when the provider returns HTTP 429 or an equivalent rate-limit error."""class AuthError(Exception):"""Authentication failed. Retrying has no effect."""@dataclassclass ModelResponse:text: strtotal_tokens: intfinish_reason: strmodel_id: strclass UnifiedChatModel:def __init__(self, provider_config: dict, max_retries: int = 3):self.adapter = get_adapter(provider_config["type"]) # Illustrative: implement per providerself.config = provider_configself.max_retries = max_retriesdef chat(self, prompt: str, temperature: float = 0.7) -> ModelResponse:request = self.adapter.build_request(prompt, temperature, self.config["model_name"])for attempt in range(self.max_retries + 1):try:raw = self.adapter.execute(request)return self.adapter.parse_response(raw)except (TimeoutError, RateLimitError):if attempt == self.max_retries:raise# Exponential backoff, capped at 8 seconds. Add random jitter in production.time.sleep(min(2 ** attempt, 8))# AuthError is not retried and is raised directly.
The code has two key points. First, it retries only on timeout and rate-limit errors. Authentication failures are raised immediately, which prevents invalid requests from consuming quota. Second, the retry count and backoff cap should live in configuration so each provider can be tuned separately.
3. Intelligent Routing Strategies
The routing layer sends each request to a suitable model. Common decision bases include the following.
| Decision Basis | Signal Source | Example Strategy |
|---|---|---|
| Task type | Lightweight classification model or rules | Code requests go to the code model group |
| Health status | Latency and error rate statistics | Remove a node when its error rate exceeds the threshold |
| Budget quota | Per-business-line usage metering | Degrade a business line once it exceeds its daily budget |
| Historical quality | Online evaluation scores | At equal prices, shift traffic toward the model with higher accuracy |
Health-aware routing is the easiest to put into practice. Teams can use the starting parameters below and adjust them based on their own traffic data.
- Statistics window: Calculate P99 latency and error rate over a rolling 5-minute window. Do not make removal decisions when the sample count is too small.
- Removal condition: If, for 3 consecutive windows, P99 exceeds twice the baseline or the error rate exceeds 5%, remove the node from the routing pool.
- Recovery condition: After removal, send a probe request every 30 seconds. After 20 consecutive successes, restore traffic and assign only a small share at first.
These numbers are starting points, not industry standards. Before launch, replay historical logs to check whether the thresholds would remove healthy nodes during normal fluctuations.
The routing layer must also handle two edge cases.
Cold start. A newly integrated model has no history, so the routing layer cannot judge its quality. Start it with about 5% of traffic, then increase its weight gradually based on evaluation scores once enough samples have accumulated.
All models unavailable. When every backup model is also unavailable, the system should return a clear degraded response, such as asking the user to try again later, and record an alert. It should not silently return empty content, because the business team would struggle to notice the failure.
4. Context Window Management
Context window lengths differ across models. As conversation history grows, a short-window model may return errors or lose key information. Teams need a context processing pipeline with three steps: concatenation and truncation, summary compression, and tiered storage.
Trigger conditions. Compress the context when the total token count after concatenation exceeds 80% of the target model’s window. Alternatively, compress every 10 conversation turns. Either condition is sufficient to trigger compression.
Compression method. Use a summarization model to generate an incremental summary of earlier conversation turns. The summary should keep names, figures, dates, user constraints, and unfinished tasks. The summary becomes new system-level context, and the original conversation moves to long-term storage.
Summary quality checks. After generating a summary, check three things: the summary is no longer than 20% of the original text, key entities and figures are preserved, and the constraints the user explicitly stated still apply. Sample historical sessions and compare these three items manually or with rules, then calculate the retention rate.
Fallback on compression failure. If the checks fail, the system should fall back to truncation. That means keeping the system prompt and the most recent N turns, and logging the fallback so it can be analyzed later.
Tiered storage. Short-term working memory stays in the prompt. Long-term memory goes to a vector database or key-value store and is injected only when needed. Retrieved results also need length limits so they do not crowd out space for the current reasoning.
5. What Generative AI Service Filing Means in China
In China, filing for generative AI services (备案) is primarily based on the Interim Measures for the Administration of Generative Artificial Intelligence Services and related rules. The cyberspace administration authorities handle the filing. The filing applies to generative AI services offered to the public and to their providers. It is not a one-time capability certification for any specific model.
Three points help clarify what filing involves.
- Filing is not model certification. Filing materials may include a description of training data sources, content safety mechanisms, test question sets, and test results. These materials show that the service provider has submitted the required information. They do not constitute third-party certification of the model’s safety.
- Filing has a defined scope. After filing is complete, the provider may offer the service to the public within the approved scope. Functions, user groups, or deployment forms beyond that scope require a new assessment.
- A model does not receive a general commercial license through filing. When different teams use the same model in different scenarios, each team must determine whether its own service requires filing and whether the approved scope covers its use.
Embodied AI products involve more complex service forms. Model outputs are action instructions that directly control physical devices, and a product may be subject to AI service regulations, industrial equipment rules, and consumer electronics requirements at the same time. The specific filing obligations, material requirements, and review procedures should follow the announcements of the cyberspace authorities and the relevant industry regulators. Consult qualified compliance professionals. This article does not compare how strict the review standards are.
6. Technical Elements and Risk Controls for Embodied Large Models
A general-purpose embodied large model aims to let one set of model parameters adapt to multiple hardware platforms and a wide range of instructions. Common technical elements include the following.
Cross-modal perception fusion. The model must fuse data from RGB cameras, depth sensors, tactile sensors, and inertial measurement units (IMUs), and align features from these sources into a unified representation of the environment. Unsynchronized sensor clocks directly degrade alignment quality, so timestamps must be corrected at the data ingestion layer.
Decoupling action primitives from motion control. The model first breaks a natural language instruction into discrete action primitives, such as grasp, move, place, and rotate. A lower-level control module then converts these primitives into motor commands. This layered design makes it easier to test each layer separately and to add safety limits at the low level.
Sim-to-real transfer. Randomizing object textures, lighting, and friction coefficients in simulation can improve policy robustness in real environments. After transfer, run small-scale validation on real hardware. Simulation results cannot replace physical testing.
Experience case library. After a robot completes a task, the system can encode the perception and action sequences as a case and store it in a searchable experience library. When a similar scene appears, the planner retrieves historical cases as references. Cases should pass manual or rule-based review before being stored, so that erroneous actions are not reused.
Physical actions carry far higher risk than text output, so the control layer needs its own safety mechanisms. At minimum, include the following: speed and torque limits for actions, an emergency stop channel, restricted zones and virtual fences, and a manual takeover entry point. These mechanisms should run independently of the large model, so hardware can be stopped safely even if the model produces abnormal output.
7. Governance and Log Auditing
Completing filing is only the starting point for compliance. During ongoing operation, teams must handle four areas: content auditing, operation rate limiting, log retention, and model updates.
Real-time content auditing. Deploy filtering layers at both the model input and output. The input filter checks user instructions for prohibited content and injection attacks. The output filter checks generated text and action sequences for boundary violations.
Operation rate limiting and circuit breaking. When the number of consecutive operations on a device exceeds a threshold within a time window, the system should raise an alert and pause execution until a person confirms. Set thresholds separately for each device type and task type.
Log redaction and minimal retention. Audit logs need to record inputs, outputs, and inference parameters. These often contain personal information, such as facial images, voice recordings, precise locations, and phone numbers. Redact logs before writing them, and keep only the fields needed for auditing. Set the retention period according to compliance requirements and internal policy, and delete logs automatically when they expire. Assign access permissions by role, and record the operator and purpose of each query.
Canary rollout. When iterating on a model, first run it on a small set of devices and monitor task success rate, number of exceptions, and number of safety alerts. After the metrics reach predefined baselines, expand the rollout gradually. Set the baselines before the canary starts, not after the results are in.
8. Pre-Launch Checklist
The checklist below can serve as the basis for a launch review. Teams should set the numerical acceptance criteria based on their own business risk.
| Check Item | Acceptance Criteria |
|---|---|
| Filing status confirmed | A written opinion or filing certificate has been obtained, and the service form matches the approved filing scope |
| Routing health checks | P99 and error rate thresholds are configured and verified through fault injection testing |
| Cold start strategy | Initial traffic share and promotion conditions for new models are written into configuration |
| Context compression | Key fact retention in sampled summaries meets the team’s target |
| Log redaction | Sampled logs contain no phone numbers, facial images, or precise locations |
| Rate limiting and emergency stop | Alerts and blocking trigger when thresholds are exceeded, and the emergency stop channel is tested and effective |
| Canary release | After running on a small set of devices for the agreed period, metrics reach baselines before traffic is increased |
9. Recommended Implementation Order
Teams can proceed in the following order.
- Confirm the filing boundaries first. Define the service form, user groups, and deployment scenarios before designing the system.
- Build the abstraction layer and fallback mechanisms first, then integrate models one at a time.
- Calibrate routing thresholds using historical data. Do not launch with default parameters.
- Complete the safety controls during system design, and keep them independent of the model.
- Expand gradually through canary releases. Use the metrics agreed in advance to decide whether each step can continue.
Multi-model integration addresses the reliability and cost of model calls. Filing addresses whether a service can legally be offered to the public. Both require ongoing verification and cannot be treated as a one-time configuration task.
