Why LLM Agent Capabilities Are Determined by Post-Training

    This article explains why the capabilities of AI agents increasingly come from the post-training stage rather than model scale or pre-training alone. It covers the division of labor between pre-training and post-training, why agent abilities concentrate in post-training, the trade-offs between minimal and standard tool-space designs, the hidden risk of overfitting to a single agent skeleton, and practical guidance for building general, robust, and maintainable agent systems.

    1. The Paradigm Shift: From “How Big the Model Is” to “How the Model Is Used”

    Early on, judging a model’s quality often meant looking at parameter count and training compute. But as the benefits of scale taper off, a clearer engineering reality has emerged: once pre-training has packed world knowledge into a model, that knowledge is relatively fixed, while how the model actually invokes that knowledge to complete real tasks is largely determined by the post-training stage.

    • Pre-training: Self-supervised learning over large corpora, typically with a next-token prediction objective. It shapes language ability and world knowledge, and determines what the model “knows.”
    • Post-training: Methods such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). It shapes behavior patterns and task preferences, and determines what the model “does and how it does it.”

    You can summarize it in one line: pre-training sets the ceiling of knowledge, while post-training determines how behavior is realized in practice, and the two cannot substitute for each other.

    2. Why Capabilities Concentrate in Post-Training

    From an engineering perspective, post-training is the stage where you “teach the model to do things.” It usually targets three goals: aligning with human intent, unlocking reasoning and tool-use abilities, and constraining output format and safety boundaries.

    A key conclusion: the distribution of post-training data directly determines the distribution of the model’s capabilities. If the training samples are heavily concentrated on a single interaction paradigm, the model will show strong ability in that paradigm while clearly degrading in others. This is both the source of post-training’s power and the source of its risk—capability gains and capability narrowing often come from the same design.

    3. “Minimal” vs. “Standard”: Trade-offs in Tool-Space Design

    In a concrete agent system, what the model can do depends on how many tools or skills it “sees.” Two common design approaches appear in practice:

    • Minimal configuration: Exposes only a very small number of actions (for example, single-digit tools). The action space is small, ambiguity is low, and routing decisions are simple, which makes high success rates easy to achieve on well-defined tasks, though coverage is limited.
    • Standard configuration: Exposes a large number of tools and skills (up to dozens). Coverage is broad, but the model needs stronger selection and planning abilities, and the probability of routing errors rises accordingly.

    An illustrative tool registration structure (general pseudocode):

    1. # Minimal configuration: expose only core actions
    2. minimal_actions = ["search", "summarize"]
    3. # Standard configuration: expose a richer toolset
    4. standard_actions = [
    5. "search", "summarize", "compute", "render_chart",
    6. "read_file", "write_file", "call_api", "plan", ...
    7. ]
    8. def register_agent(action_space):
    9. # The larger the tool space, the higher the selection/planning burden
    10. for action in action_space:
    11. agent.register(action)
    12. return agent

    The essence of this trade-off is “focus vs. coverage.” When task boundaries are clear, the minimal configuration is often more stable; when tasks are open-ended and require cross-tool collaboration, the standard configuration has greater potential but also demands more from the model’s selection and planning abilities. Neither is absolutely superior—the key is to align the tool space with the real task distribution.

    4. Overfitting to a Specific Skeleton: The Hidden Cost of Generality

    If the post-training stage invests too heavily in one specific agent skeleton (that is, a fixed prompt structure, action format, and calling convention), the model is prone to “overfitting” to that skeleton: it performs excellently under that convention, but once you switch skeletons, performance may drop noticeably.

    This is highly similar to overfitting in traditional machine learning—a mismatch between the training distribution and the inference distribution damages generalization. Its typical symptom is that the model is not “smarter,” but rather “more used to this one way of interacting.”

    Approaches to mitigate this include:

    • Multi-skeleton training: Expose the model to a variety of prompt structures and action formats to improve robustness;
    • Data diversity: Mix samples across different task distributions and tool sets;
    • Distribution alignment: Ensure the interaction conventions in the evaluation environment match those in the training environment;
    • Generalization evaluation: Cross-validate across multiple skeletons rather than scoring on a single configuration.

    5. Practical Recommendations for Building General, Robust Agent Systems

    Overall, differences in model capability come less and less from “theoretical knowledge reserves” and more and more from domain understanding, problem modeling, and tool-orchestration ability. Here are a few actionable recommendations for developers:

    1. Choose the tool space by task boundary: use the minimal configuration for well-defined tasks and the standard configuration for open-ended tasks, avoiding the “one umbrella to cover everything” approach.
    2. Decouple domain knowledge from orchestration logic: the knowledge layer handles “knowing,” while the orchestration layer handles “how to combine tools to complete tasks.”
    3. Design for generalization: train and evaluate across multiple skeletons and tool sets, and stay alert to overfitting on a single configuration.
    4. Build an evaluation loop: define clear metrics (success rate, number of steps, cost, latency) and keep running regressions.
    5. Make good use of general infrastructure: use a container platform to deploy agents, a message queue for asynchronous tasks, object storage to manage intermediate artifacts, and logging and monitoring/alerting services to trace call chains and anomalies—these general capabilities let you focus your effort on orchestration and post-training itself.

    An illustrative agent invocation skeleton (general pseudocode):

    1. def run_agent(task, tool_space, max_steps=8):
    2. state = {"task": task, "trace": []}
    3. for step in range(max_steps):
    4. action = model.plan(state, tool_space) # Select and plan
    5. if action.type == "tool":
    6. result = tool_space.execute(action) # Execute the tool
    7. else:
    8. return action.answer # Finish and return the result
    9. state["trace"].append((action, result))
    10. return summarize(state["trace"])

    6. Conclusion

    As parameter scale stabilizes, the focus of competition among agent systems is shifting from “how big the model is” to “how to train and orchestrate it.” Understanding the division of labor between pre-training and post-training, weighing the minimal and standard tool spaces, and staying alert to overfitting on a single skeleton are what let you build truly general, robust, and maintainable agent systems amid rapid model iteration. Mastering this methodology of post-training and orchestration matters far more than chasing any specific model—because what is truly scarce is the ability to model the problems in your own domain and turn them into orchestratable tasks.