Why Robots Still Need a Dedicated Embodied Model Even When General LLMs Are Strong

1. Reframing the Question: Not “Can It Be Done?” but “Who Should Do It?”

General-purpose multimodal models have climbed a steep capability curve in recent years. They can read manuals, decompose tasks, generate action scripts, and even complete some pick-and-place flows in simulation. That has led the industry to ask the same question over and over: if general models can already “solve robot tasks,” why should robots maintain their own model stack?

The question itself is framed poorly. It equates “a tool has one viable path to the task” with “this tool is the optimal engineering choice for the scenario.” A more rigorous framing breaks it into three successive questions:

• Capability ceiling: Does the capability a general model currently demonstrates cover the critical path of robot operation?
• Engineering fit: Under latency, compute, power, and cost constraints, can it enter the real-time control loop?
• Capability redundancy: Are the irrelevant capabilities it must carry to satisfy robot requirements crowding out resources for the critical ones?

Once these three are separated, the picture becomes clear. The strengths of general models are semantic understanding, long-horizon planning, and cross-domain knowledge transfer — precisely the upper-layer requirements of a robot system. The lower-layer requirements of a robot — high-precision perception across physical modalities, sub-centimeter spatial reasoning, hybrid force/position control, and fast recovery from failure — are not native strengths of general models, nor do they emerge naturally by scaling up parameter counts.

2. Capability Configuration, Not Parameter Count: An Embodied Model Should Be “Not Too Big, Not Too Small”

When discussing the shape of an embodied model, it is easy to misread “size” as parameter count. A more meaningful measure is the capability configuration table: which capabilities must have no weak points, and which should be deliberately excluded.

Capability dimension	Importance to robot operation	Notes
Task semantic understanding and decomposition	High	Can be handled by a general model and pushed down through an interface
Long-horizon planning and tool selection	High	Upper-layer capability; tolerant of second-level latency
Dense depth perception and calibration error compensation	Very high	Determines grasp point accuracy; requires native modeling
Tactile/force representation	Very high	Almost no corresponding corpus on the digital side; must be built in-house
Body dynamics and self-occlusion reasoning	Very high	Strongly coupled to the specific mechanical structure
General code engineering ability	Low	No direct causal relation to the task closed loop
Open-domain chit-chat and multi-turn Q&A	Low	Consumes context and compute budget

Treating “general coding ability approaching that of frontier general models” as a goal for an embodied model is a classic metric mismatch. The reverse is the real risk: treating distance, touch, and force feedback as optional enhancements. For tasks such as high-precision assembly, flexible harness insertion, or fragile-item sorting, the loop cannot converge without physical-modality feedback.

“Embodied-native” does not mean bolting an action output head onto a digital model. It means reconstructing three things at once: input, output, and learning objective.

• Input reconstruction: Native support for point clouds, depth maps, tactile arrays, joint torques, and proprioceptive state, with explicit modeling of the noise characteristics of each modality.
• Output reconstruction: Not just discrete action labels, but pose increments with uncertainty, contact force limits, and termination conditions.
• Objective reconstruction: Task success rate and physical-constraint violation rate as the primary supervision signals, rather than text similarity or trajectory fitting error as the only objective.

3. Three Constraints Imposed by the Physical World

Constraint 1: Inputs Carry Error, and the Error Structure Differs by Modality

No sensor is error-free. Depth cameras produce holes and edge flying points on reflective, transparent, and thin-walled objects. Tactile sensor arrays exhibit baseline drift, hysteresis, crosstalk, and unit-to-unit variation. Joint encoders have quantization error, and calibration parameters drift slowly with temperature and mechanical wear.

The key point is not whether error exists, but that the error is structured and modelable. Engineering responses include:

1. Explicitly modeling uncertainty so downstream policies know where not to trust perception.
2. Cross-validating modalities: using tactile contact events to correct depth-based distance estimates, and using body kinematics constraints to reject visual outliers.
3. Making representations robust at the representation layer — training the model on noisy data rather than assuming clean inputs.

This also explains why tactile representation deserves to be built as an independent, long-term capability: tactile signals have high bandwidth, strong temporal structure, and a semantic space that is not shared with vision, so vision-side pretraining results cannot be reused directly.

Constraint 2: Action Feedback Is Irreversible, and the Cost of Trial and Error Is Asymmetric

The greatest convenience of the digital world is rollback. A failed compile can be retried; an unsatisfactory model output can be resampled. The physical world has no undo: crushing a strawberry, scratching a panel, or bending a workpiece costs immediately and cannot be recovered.

This imposes three design constraints:

• Confidence thresholds must be explicit. When perception is uncertain, the correct behavior is to slow down, request a retry, or ask for help — not to commit at maximum likelihood.
• Force limits must be hard constraints. Contact operations need a safety boundary independent of the policy layer; they cannot rely only on the model having “learned” gentle handling.
• Recovery policies must be first-class citizens, equal to the main policy. Failure detection, state rollback, and re-alignment should be designed in as primary components, not as exception branches.

Constraint 3: Closed-Loop Latency, Compute, and Data Distribution

A single inference pass of a general large model typically takes hundreds of milliseconds to seconds, while force-control loops usually require millisecond-level refresh rates. This means a general model cannot enter the control loop directly; it can only sit at the perception–planning layer. The reasonable shape is layered:

# Illustrative layered architecture (pseudocode, expressing responsibility boundaries only)
class EmbodiedStack:
    def __init__(self):
        self.planner = TaskPlanner()           # Can use a general LLM, second-level / task-level
        self.perception = PhysicalPerception()  # Depth + tactile + proprioception, with uncertainty
        self.policy = ActionPolicy()            # Embodied-native model, 10~50 Hz
        self.guard = SafetyGuard()              # Force limits, workspace, emergency stop
    def step(self, instruction, obs):
        plan = self.planner.decompose(instruction)   # Low-frequency call
        phys = self.perception.encode(obs)            # High frequency; outputs mean + sigma
        if not self.guard.is_reachable(phys):
            return self.replan(plan)
        action = self.policy.act(phys, plan)
        action = self.guard.clip(action)              # Hard constraints take priority over policy output
        return action

Data is a hard constraint on the other side as well. Robot data is expensive to collect, expensive to label, and varies widely across embodiments, which makes cross-platform transfer difficult. A common approach is a layered data recipe: simulation and synthetic data establish basic physical priors, teleoperation demonstration data aligns embodiment and task distribution, and real run data is used for closed-loop fine-tuning and mining failure cases. Raw data volume is often in the TB-to-PB range, so the engineering stack needs object storage to hold raw streams, message queues to decouple peak shaving from collection and backhaul, container platforms to run parallel training and simulation rollback jobs, and logging plus monitoring and alerting to trace the failure attribution of each run. The maturity of this data loop usually affects final results more than the choice of model architecture.

4. How to Evaluate Whether an Embodied Model Is “Good Enough”

A single success-rate metric hides problems. At least five categories of measurement are recommended:

• Task success rate and completion time, reported in tiers by object category and pose difficulty.
• Perturbation robustness: performance degradation curves under target pose offsets, occlusion, and changes in lighting and reflectivity.
• Physical-constraint violation rate: number of collisions, proportion of maximum contact-force violations, self-collision duration.
• Uncertainty calibration quality: whether the confidence the model reports matches its actual failure rate.
• Cross-embodiment and cross-scene transfer cost: the data volume and fine-tuning steps needed to add a new platform.

The fourth is the most often overlooked, yet it directly determines whether the system can be deployed safely. An overconfident model that fails frequently is far more dangerous than one that “knows it doesn’t know.”

5. Common Pitfalls and Conclusion

Three pitfalls are worth watching for. First, treating general benchmark scores as proof of embodied capability. Second, assuming that scaling up parameters will automatically fill in physical-modality perception. Third, treating world models or vision-language-action models as the end state — the former leans toward “predicting consequences” and the latter toward “fusing information,” and both are necessary components rather than complete answers.

The conclusion can be stated in one sentence: progress in general models genuinely raises the capability ceiling of robot systems, but it raises the cognitive and planning layer; the contact and force-control layer must still be handled by models designed natively for the physical world. The evaluation standard for an embodied model is not whether its capabilities are comprehensive, but whether the critical capabilities have no weak points and no resources are wasted on non-essential ones — not too big, not too small, covering exactly what the physical world demands.