Rethinking In-House Model Development in Embodied AI: Where Should R&D Budgets Go?

Embodied AI R&D has long depended on specialized data collection, model training, and task tuning. Teams need robots to perceive environments, understand instructions, plan actions, and recover after execution failures. Because real scenes involve variable objects, lighting, viewpoints, and manipulation conditions, these capabilities are usually built up through extensive experimentation.

As general-purpose large models improve in language understanding, visual reasoning, and task planning, robotics teams now have another option: instead of training every capability from scratch, they can let a general model handle high-level understanding and decisions, while dedicated robot systems handle action generation, control, and safety constraints.

This combination can lower the development barrier for some tasks and make capabilities that once required specialized training easier to call on. It does not mean embodied model development has lost its value. What has changed is the focus: from “can we train a model?” to “which capabilities should be reused, and which must be developed specifically for the robot and the scene?”

Why Combined Approaches Improve Task Performance

Robot tasks typically involve multiple stages: understanding natural language instructions, recognizing scenes and objects, decomposing task steps, selecting manipulation targets, executing actions, and verifying results. General models can participate in several high-level stages and reduce the system’s dependence on single-task data.

For example, given an instruction such as “put the items on the table into the specified container,” the system must first identify the items and container, infer the operation sequence, and then invoke low-level control for grasping and placement. A general model can help with instruction interpretation, task decomposition, and visual semantic association; robot-specific modules handle kinematic constraints, grasp posture, force control, and action execution. When the two fit together well, teams may not need to train a complete model for every language expression and scene variation.

The key is not that the model “does everything” for the robot. The system’s capability boundary is redrawn. High-level task understanding can borrow general capabilities, while low-level action control still needs reliable robot technology. If this is ignored, fluent-looking offline reasoning may not translate into stable operation in a real environment.

Higher Benchmark Scores Do Not Equal Production Readiness

When a standard benchmark improves after adding a general model, it suggests the solution is using existing semantic and reasoning capabilities more effectively. It does not prove the system is ready for deployment.

Benchmark tasks usually have clear goals and limited environment scope, while real-world object arrangements, equipment states, and user expressions are more diverse. A model may produce a reasonable language plan, but the execution layer can still be affected by vision errors, robot arm precision, communication latency, and physical contact. A single success is not long-term stability; robots must also handle repeated execution, anomaly recovery, and task interruption.

Therefore, teams should break evaluation results into multiple dimensions rather than looking only at the total score:

• Task success rate: whether the objective was actually completed, not just whether a plausible plan was generated.
• Generalization: whether performance remains stable after object, instruction, or layout changes.
• Execution reliability: whether actions are repeatable and failures can be identified and recovered from.
• Safety: whether the system avoids actions that are out of bounds, dangerous, or in conflict with on-site constraints.
• Operating cost: whether the overall cost of inference, data processing, equipment operation, and maintenance is acceptable.

These indicators help teams see where the improvement comes from and reveal weak points that the combined pipeline has not covered.

Self-Developed Models Still Matter, But Their Goals Need Redefinition

“Should we still build our own model?” is not a simple either/or question. The value depends on whether the business has requirements that general models cannot meet.

If tasks involve fine manipulation, strict latency, specialized sensors, complex safety boundaries, or stable operation on specific equipment, specialized models and control algorithms remain important. A company’s accumulated scenario data, failure cases, and equipment knowledge can create advantages that are hard to obtain from general capabilities. Conversely, if a team is simply duplicating general-purpose semantic understanding and routine task planning, large-scale in-house model training may not generate returns that justify its cost.

A more practical approach is to separate “self-developed model” into different levels. Basic perception and language capabilities can be evaluated for reuse. Motion control, state estimation, and safety policies related to device structure should be prioritized as core robot capabilities. The parts that have the greatest impact on business outcomes should be judged by data and experiments, not by assumption. Self-development does not have to start from pretraining; it can also mean data adaptation, module optimization, policy learning, or system-level integration for a specific task.

Budget Should Shift From Model Scale to Capability Gaps

When the technical route changes, budget planning should change with it. If most funds continue to go to model training, the real constraints may be overlooked: data quality, robot execution, system integration, and field validation.

Teams can start by building a capability inventory that records performance and failure causes at each step of the current task chain. If the main issue is instruction understanding, they can evaluate whether a general model meets the requirement. If failures are concentrated in grasp localization, action stability, or adaptation to environment changes, resources should go to robot data, perception systems, and control capabilities. Then run small-scale controlled experiments under the same tasks, equipment, and evaluation criteria to compare a pure specialized approach, a general-model-assisted approach, and a hybrid approach.

Budget should also include ongoing operating costs, not just training costs. For example: does the system require extra inference resources? Do network or device condition changes affect execution? Does a model update require safety revalidation? Are logs and replay mechanisms available when exceptions occur? These engineering issues directly determine whether a solution can move from the lab to long-term operation.

A Layered Architecture Can Reduce Route-Change Risk

A robust system design can split robotic tasks into relatively clear layers:

1. Task understanding layer: convert user instructions into structured goals and detect missing information or ambiguity.
2. Planning and decision layer: determine task steps, check preconditions, and adjust the plan based on execution feedback.
3. Action execution layer: map the plan to executable device operations and handle motion constraints and real-time state.
4. Safety and monitoring layer: restrict dangerous actions, monitor execution status, and pause or enter recovery when anomalies occur.
5. Evaluation and data layer: record input, decisions, actions, and outcomes to support problem analysis and later improvement.

Layering does not mean modules are independent. It means responsibility boundaries are clear. A general model can participate in task understanding or planning, but it should not bypass safety checks and directly control the device. The low-level execution module should verify whether an action is feasible and refuse to execute when constraints are violated. This makes it easier to adapt when model capabilities change or a supplied solution changes, without rebuilding the whole system.

Conclusion: Move From Model Chasing to System Value Validation

The progress of general-purpose large models is putting pressure on embodied AI because some abilities that once required specialized training can now be obtained more quickly through combined architectures. But this shift is better understood as a restructuring of R&D work, not the end of in-house development. The core question is not “should we train a larger model?” but “what is the most expensive and least stable capability gap in the current task pipeline?”

Only by placing benchmark scores, real execution, reliability, safety, and cost in the same evaluation system can a company decide which capabilities should be reused, which parts must be self-developed, and where next year’s budget should go. Rather than making a one-time bet on a model route, teams can use modular architecture and small-scale validation to adjust investment continuously. The goal is to make every R&D dollar correspond to a clear scenario problem and a testable improvement target.