Physical Intelligence: Moving From Planning to Reliable Execution

Progress in Physical Intelligence Is Not Only About “Thinking”

As general-purpose models become better at understanding natural-language instructions, breaking down tasks, and generating action plans for robots, physical intelligence may appear to be crossing an important threshold. But this progress should not be judged only by whether a robot understands what it is supposed to do. It must also be judged by whether the robot can perform the task consistently and reliably in the real world.

More precisely, robotic task completion involves at least two related but distinct components:

• Cognition and planning, which handle goal understanding, object recognition, and step selection.
• Perception and control, which use sensor feedback to adjust motion and manage real physical contact between objects.

Improvements in the first component may make robots better planners. The second determines whether those plans can withstand the conditions of real environments.

As a result, early signs of a breakthrough in physical intelligence often indicate stronger task understanding and action planning. They do not automatically mean that robots have achieved reliable manipulation.

1. A Successful Plan Is Not the Same as a Completed Task

Consider a task such as clearing a cup from a tabletop. A robot first needs to identify the cup and the table, determine whether the cup should be moved and where it should go, and then plan the steps for grasping, carrying, and placing it. These are tasks of understanding and planning.

During execution, however, the cup may stick to the table, the grasping point may be occluded, or the object may slip after the gripper closes. Even if the robot identifies the correct target and chooses the right sequence, the task can still fail if the gripping force is inappropriate or if the arm collides with another object while moving.

This illustrates that a correct plan and successful manipulation are different metrics. Planning quality can be evaluated by whether the selected task steps are reasonable. Manipulation success must be measured through actual outcomes in a physical environment.

A single video showing a successful result cannot establish that a system can reproduce the same outcome consistently under identical conditions. It also cannot reveal whether the system can detect errors and recover when failures occur.

2. Why Real-World Object Interaction Is Harder

Natural-language tasks can often be described using abstract concepts such as “pick up,” “wipe,” or “put into.” A robot, however, must convert these concepts into continuous motion while handling the constantly changing relationship between its actions and the environment.

Contact states are difficult to predict precisely. An object’s weight, surface friction, shape, and placement all affect the outcome of manipulation. Vision can help estimate where an object is, but it cannot always directly determine whether an object is securely grasped, whether a brush is actually touching a surface, or whether excessive force is being applied.

Errors accumulate across action sequences. A deviation in one action may be minor, but position, orientation, and force errors can compound over multiple steps. A few millimeters of error during grasping can cause a placement failure. An unstable placement can then affect later actions. The longer the task, the more the system must continuously verify its state instead of generating one complete plan and executing it without correction.

Environmental changes weaken fixed strategies. Objects may be moved, lighting may change, or new obstacles may appear on a table. These changes can make an original action plan unsuitable. Real-world environments are not as fully controlled as demonstration settings, so systems need the ability to observe again, reassess the situation, and adjust their actions.

3. From Plan Generation to Closed-Loop Control

Bridging the gap between planning and execution does not simply require more detailed plans. The key is establishing an effective feedback loop. A more complete execution process can be summarized as follows:

1. Observe the current state: Use vision, touch, or other sensor signals to estimate object position, orientation, and environmental changes.
2. Choose a short-horizon action: Determine the next step based on the current state rather than relying entirely on a long plan generated at the beginning.
3. Execute and collect feedback: After performing an action, observe the result again and determine whether the object moved or whether the grasp succeeded.
4. Detect deviations and recover: If the actual outcome differs from the expected outcome, update the state estimate, adjust the action, or replan.

This cycle of observe, act, and observe again prevents the system from placing all uncertainty on a one-shot plan. For tasks requiring precise manipulation, the control loop must also be timely. The environment may change during execution, and the system must be able to correct its actions based on feedback rather than discovering deviations only after an entire motion sequence is complete.

Language models and other high-level models are well suited to task semantics, step organization, and commonsense reasoning. That does not mean they can replace low-level motion control on their own. In practice, systems usually need a clear division of responsibility:

• High-level decision-making determines what should be accomplished next.
• Low-level control determines how to complete it with appropriate speed, direction, and force.

The connection between these layers also requires clear information flow and mechanisms for handling exceptions.

4. How to Determine Whether Capabilities Have Actually Improved

Physical intelligence should be evaluated through reproducible, statistical task testing rather than isolated demonstrations. At a minimum, evaluation should consider the following areas.

Task completion rate. Run the same task repeatedly under the same conditions and record the proportion of successful completions. One successful run only shows that the system can handle one particular situation; it does not demonstrate stability.

Range of scene coverage. Test whether object positions, orientations, occlusions, and environmental layouts vary. If a system works only when objects are arranged in fixed locations, its performance may still depend heavily on a predefined setup.

Robustness during execution. Assess whether the robot can detect and recover from small deviations or disturbances during a task. Measuring only the final result may conceal whether the system depends on extensive human intervention.

Task-chain length and error propagation. Success in a simple single-step operation does not mean that a robot can reliably perform multi-step household tasks. Evaluation should cover both short tasks and longer workflows, observing whether earlier errors affect later actions.

Failure cost and safety. In homes, warehouses, and production environments, failure does not only mean that a task remains unfinished. It may also cause object damage, equipment collisions, or risks to people. Safety constraints, stop mechanisms, and exception handling should therefore be evaluated alongside success rates.

Two metrics that are often confused should also be distinguished. One measures the number of unfamiliar scenarios in which a system can operate. The other measures how many tasks the system can complete within those scenarios. Even if both rates appear high, the combined coverage may still be substantially lower than intuition suggests.

Evaluation should disclose test conditions, task definitions, the number of repeated trials, and the criteria used to classify failure. Broad claims such as “strong adaptability” should not replace verifiable results.

5. What Evidence Is Still Needed to Move From Demonstration to Usable Systems?

A demonstration in which a robot completes several actions in sequence can show the potential of integrating perception, planning, and control. However, it is not sufficient on its own to prove general-purpose capability.

More convincing evidence includes:

• Repeated testing across different objects and environments.
• Records of failure types and recovery outcomes.
• Clear disclosure of whether human teleoperation or manual resets are required.
• Comparisons between training environments and test environments.
• Stability results from extended operation over time.

Claims of “self-improvement” or “continuous improvement” require the same level of caution. A system that updates its policy using data is not necessarily able to improve its success rate autonomously and safely in real-world environments. It is still necessary to verify whether data collection is sufficient, whether updates produce measurable gains, whether performance changes can be reproduced, and whether the improvement process introduces new safety risks.

For developers, a practical approach is to break a task into observable substeps and define success and failure conditions for each step. Next, establish sensor-feedback and state-update mechanisms so that the system can distinguish between “the action has physically succeeded” and “the action was merely executed.” Then, through repeated testing, gradually increase object variation, environmental disturbances, and task length. Failures should be categorized by cause rather than reduced to a single overall success rate.

Conclusion: Real Breakthroughs Depend on What Robots Can Do With Their Hands

Physical intelligence may indeed advance significantly as models improve in understanding and planning. But knowing what should be done is only one part of a complete task pipeline. A robot’s practical execution capability depends on whether it can perceive contact, correct errors, adapt to changes, and remain stable across repeated runs.

For this reason, the most meaningful indicators are not how smooth a single demonstration looks. They are the success rate of contact-rich manipulation, performance in unfamiliar environments, recovery after failure, and whether test results can be independently repeated.

Planning is becoming a foundational capability. Making those plans work reliably in the physical world remains a core challenge for physical intelligence.