How General-Purpose Foundation Models Are Reshaping In-House Embodied AI Development

    General-purpose foundation models are advancing faster than embodied AI models, forcing robotics teams to reconsider where to invest in proprietary research. This article examines how to divide capabilities between external general-purpose models and in-house systems, covering hybrid architectures, model evaluation, budget allocation, organizational changes, data loops, and the execution-layer capabilities that remain essential for reliable robots.

    Over the past year, general-purpose foundation models have advanced noticeably faster than major paradigm breakthroughs in embodied AI. For robotics teams, the pressure is not simply that competing models have more parameters or higher benchmark scores. The more fundamental change is that the entire research and development path is being rewritten.

    Capabilities that once required large datasets, substantial compute, and years of algorithm engineering can now be partially covered by general-purpose models through in-context learning, tool use, code generation, and task orchestration. This raises a practical question: should embodied AI companies continue making major investments in proprietary foundation models? Should next year’s in-house model budgets remain unchanged?

    This article does not evaluate any specific product. Instead, it treats the issue as a general technical problem: when a general-purpose foundation model becomes the foundation for broad generalization, how should embodied AI teams divide the boundary between externally sourced capabilities and proprietary core capabilities? How should they design hybrid architectures and reorganize budgets and teams to avoid continuing to invest heavily in the wrong direction?

    1. The Shift in Development Paradigms

    Traditional embodied AI development typically includes data collection and labeling, imitation or reinforcement learning, task-specific post-training, simulation evaluation, real-robot deployment, and iterative improvement. This process has a long feedback loop, high costs, and strong sensitivity to task distributions. Improving a model from 80 to 82 points in a specific scenario may require multiple rounds of data collection, training adjustments, and real-world validation.

    General-purpose foundation models follow a very different iteration cycle. Their improvements often come from larger-scale pretraining, stronger reasoning, and better tool-use capabilities, which can be released on relatively short schedules. For embodied AI teams, the most direct impact is that some tasks no longer need to begin with a dedicated policy trained from scratch. They can instead be handled through language instructions, environment summaries, task decomposition, and tool calls.

    Even when final execution still depends on local robot policies, a general-purpose model can take responsibility for higher-level cognitive functions such as understanding the task, breaking it into steps, generating a plan, and recovering from failures.

    This creates two major changes. First, the comparison target for algorithm teams has changed. They are no longer competing only with other embodied models, but also with the general cognitive capabilities of general-purpose foundation models. Second, the budget structure is changing. If high-level generalization can be obtained through API calls or private deployments of general-purpose models, the marginal return from building the same general cognitive capabilities in-house may decline.

    2. Evaluation Signals: From Native Embodied Models to General Models with Robot Agents

    Public evaluations are already showing a clear trend: more teams are submitting combined solutions based on a general-purpose foundation model and robot-agent orchestration rather than relying only on a native embodied foundation model.

    The basic design is straightforward. A general-purpose model interprets natural-language instructions and scene summaries, then generates a task plan or executable action sequence. Local policies handle real-time control, safety constraints, and low-level execution. The evaluation framework measures success rate, robustness, and recovery capability.

    This combination can generalize effectively across some open-world tasks. A native model trained on a relatively small set of task data may achieve a significant improvement after a general-purpose model is introduced for task decomposition and contextual reasoning.

    This does not mean that a general-purpose model can replace the entire robot control stack. Rather, it is taking over part of the cognitive layer, reducing the amount of generalization that the native embodied model must handle on its own.

    For engineering and R&D leaders, the evaluation signal is important. If high-scoring systems increasingly use hybrid architectures, continuing to train a large, all-purpose native embodied foundation model in isolation may result in high investment, slow iteration, and an increasingly misaligned comparison target.

    3. Capability Decomposition: Why In-Context Learning Matters

    Among the capabilities demonstrated by general-purpose models, in-context learning deserves particular attention from embodied AI teams. In-context learning allows a model to form an initial strategy for a new task without updating its parameters, using only instructions, examples, environment descriptions, or interaction history included in the prompt.

    For robotics, this means a model may generate an operational plan from a language instruction and an environment summary without additional task-specific post-training or a large collection of task demonstrations.

    Previously, achieving similar capabilities with an embodied model often required collecting task demonstrations, constructing multitask datasets, performing post-training or fine-tuning, validating in simulation, and then transferring the result to a real robot. In-context learning moves part of this process into the inference stage and reduces the cold-start cost for new tasks.

    However, it has clear limitations. Tasks requiring strict real-time performance, force control, fine manipulation, safety constraints, or long-horizon stability still need local policies and control systems as a fallback.

    General-purpose models are also improving in spatial understanding, commonsense reasoning, task decomposition, and failure recovery. When these capabilities are combined with robot execution layers, they create a division of labor: general-purpose models handle high-level cognition, while local policies handle execution.

    As a result, the focus of proprietary development should shift from repeatedly rebuilding a general cognitive foundation to building execution capabilities and data feedback loops that are difficult to replace.

    4. Is In-House Model Development Still Necessary?

    The question of whether to build in-house cannot be answered with a simple yes or no. It is more useful to divide the system into four layers.

    4.1 General Cognitive Layer

    This layer includes language understanding, task decomposition, commonsense reasoning, and tool use. Mature general-purpose models can often provide these capabilities through prompt engineering, retrieval-augmented generation, agent orchestration, and private deployment.

    The barrier to developing this layer from scratch is extremely high. When budgets are limited, teams should avoid blindly competing with large general-purpose model providers.

    4.2 Embodied Execution Layer

    This layer includes real-time control, motion planning, force control, grasping, navigation, and safety constraints. It is closely tied to the robot’s morphology, sensors, actuators, and operating environment, making it a core area for in-house development.

    A general-purpose model cannot replace millisecond-level closed-loop control.

    4.3 Data Feedback Layer

    This layer covers real-robot data collection, simulation data generation, failure-case recovery, automated labeling, and evaluation-set construction. It determines the long-term competitive advantage of an embodied AI team.

    Even when a general-purpose model is used, high-quality scenario data is still required to constrain its outputs, validate its plans, and train local policies.

    4.4 Morphology Adaptation Layer

    Different robot bodies, degrees of freedom, sensor configurations, and task domains vary significantly. Teams therefore need proprietary adapters, skill libraries, and safety policies.

    The general-purpose model provides generalization, while the local system provides reliability.

    The resulting conclusion is that in-house development remains necessary, but the target of that development must change. Budgets should move away from repeatedly training general cognitive foundation models and toward the execution layer, data feedback loops, evaluation systems, and safety constraints.

    5. Budget and Organization: From Compute Scaling to Closed-Loop Data

    When general-purpose models can handle part of the cognitive workload, the structure of in-house investment should change. In the past, the largest budget items may have included pretraining compute, labeling teams, and model-development positions. Going forward, a larger share should be allocated to data feedback loops, evaluation infrastructure, real-robot operations, and safety systems.

    Pretraining teams do not necessarily need to be eliminated. Their roles can evolve into three categories:

    1. General-model application and orchestration teams, responsible for prompting, tool calls, retrieval, and routing across multiple models.
    2. Local policy and skill teams, responsible for real-time control, execution, and morphology adaptation.
    3. Evaluation and data teams, responsible for task-set construction, regression testing, and failure analysis.

    The value of an algorithm engineer is no longer measured only by the ability to train a large model. It is increasingly measured by the ability to integrate general-purpose capabilities into a robot system reliably.

    During budget reviews, organizations can use a layered capability ledger:

    • General cognitive capabilities can be measured by inference cost, latency, and private-deployment cost.
    • Local execution capabilities can be measured by improvements in real-robot stability and safety.
    • Data feedback loops can be measured by task coverage and failure-recovery rates.

    This approach helps prevent teams from building in-house models simply for the sake of ownership. At the same time, it avoids the risks of complete dependence on external models, including latency, availability, and data-security concerns.

    6. Engineering Implementation: A Reference Hybrid Architecture

    A practical hybrid architecture commonly includes task routing, general-purpose model planning, local skill execution, safety monitoring, and log feedback.

    1. def execute_task(task, observation, budget):
    2. if task.is_safety_critical or task.requires_hard_realtime:
    3. return local_policy.execute(observation)
    4. if task.needs_open_world_reasoning:
    5. summary = compress_observation(observation)
    6. plan = general_model.plan(task.instruction, summary)
    7. plan = safety_filter(plan)
    8. return local_policy.execute_plan(plan)
    9. return skill_cache.lookup_or_execute(task, observation)

    Four engineering considerations are particularly important.

    First, observation compression. Robot sensors generate large volumes of data. Images, point clouds, and joint states need to be compressed into summaries that a general-purpose model can interpret, reducing latency and cost.

    Second, safety filtering. Plans generated by a general-purpose model must pass rule checks, collision detection, and permission controls before execution.

    Third, caching and distillation. Frequently used tasks can rely on cached plans, or outputs from a general-purpose model can be distilled into a smaller local model to reduce dependence on online inference.

    Fourth, evaluation regression. Teams should establish offline task sets and real-robot regression suites to monitor success rate, recovery rate, and latency across the hybrid architecture.

    At the infrastructure level, teams can use general-purpose object storage for multimodal data, message queues to decouple data collection from training, container platforms to deploy inference services, and logging and monitoring systems to track task failures.

    For cloud deployments, general cloud platforms such as Baidu AI Cloud may provide common capabilities including object storage, container platforms, logging, and monitoring. However, specific choices should be based on latency, data compliance, and cost assessments rather than on the pursuit of a particular model.

    7. Conclusion: In-House Development Is Being Redefined

    The rapid progress of general-purpose foundation models has created real pressure for the embodied AI industry. However, the question of whether to build in-house should not be reduced to continuing or giving up.

    A more accurate conclusion is that the expansion of general-purpose cognitive capabilities is shrinking the space for redundant proprietary development. At the same time, real-time control, safety constraints, data feedback loops, and morphology adaptation remain irreplaceable capabilities for embodied AI teams.

    Whether next year’s in-house model budget is at risk depends on where the money goes. If it continues to focus on general cognitive pretraining under the old paradigm, returns may decline. If it is redirected toward hybrid architectures, data feedback loops, evaluation systems, and local execution capabilities, in-house development may become even more valuable.

    The central challenge for embodied AI teams is not simply that general-purpose models have become powerful. It is whether they can quickly shift the center of their R&D strategy: letting general-purpose models serve as the foundation for broad generalization, while proprietary systems provide reliable execution and continuous improvement.