Does Agent Post-Training Overfit the Evaluation Scaffold?
This article examines why coding agents can score higher when the evaluation scaffold exposes fewer tools, separates competing explanations such as interface burden, strategy switching, opening anchoring, and prompt leakage, and proposes controlled experiments plus post-training data practices to reduce scaffold memorization.
Does Agent Post-Training Overfit the Evaluation Scaffold?
Why scores change when the tool description changes
A coding agent’s score often depends not only on model parameters but also on the working environment it is placed in. That environment includes the length and tone of the system prompt, the number of visible tools, the parameter schema for each tool, whether plugins or external capability descriptions are preloaded, and what information the model is allowed to see in the first turn.
A recurring community observation is that the same model generation may perform moderately in a standard environment with many tools and complex schemas, but its score rises noticeably when only a small set of basic capabilities is exposed, such as running commands and reading files. More important is an anchoring experiment: the first turn deliberately pretends to be a minimal environment, and after the model makes its first tool call, the full tool set is restored. The score remains high. If the problem were simply “too many tools cause choice difficulty,” performance should fall after the full tool set returns. It does not. This suggests that the model has already entered a familiar working mode at the start, and later information struggles to pull it out.
This does not automatically prove “post-training cheating” or “benchmark contamination,” but it is enough to raise suspicion from “occasional variance” to “distribution memory”: the model may have seen many trajectories with similar shapes, and once the opening looks like one of them, it continues along the most familiar path.
Turning the phenomenon into testable hypotheses
Before discussing overfitting, write the hypotheses clearly so that every environment sensitivity is not called overfitting.
- Interface burden hypothesis: Tool schemas are long and descriptions overlap. The model spends context and attention on selecting tools and filling parameters. Lower scores in complex environments are a normal interface cost.
- Strategy switching hypothesis: A minimal environment forces the model to solve problems with general read/write and command combinations, reducing branches that search for specialized tools, so behavior is more stable.
- Opening anchoring hypothesis: Post-training data contains many trajectories that begin with only a few tools and then follow a fixed calling rhythm. The model identifies the task type from surface features in the first few turns and locks into a solution script. Adding tools midway does not reset that script.
- Prompt leakage hypothesis: Wording, directory conventions, hidden checkpoints, or fixed error messages from the evaluation scaffold were directly or indirectly written into post-training samples. The model is reproducing memory rather than reasoning.
The first two explanations describe normal interaction between capability and interface. The latter two point to an overly tight match between post-training and evaluation distributions. The anchoring experiment mainly challenges hypothesis 1: if the tools are restored in the second turn, choice difficulty should reappear. It cannot by itself confirm hypothesis 4, because the model may simply remember a general rhythm such as “explore first, make small changes, then run checks,” rather than a hidden answer.
Models remember whatever trajectories look like
A common post-training data shape compresses a successful agent run into a trajectory: system prompt, user task, several tool calls, observations, and a final patch or command. If the collection pipeline keeps only one scaffold for a long time, trajectories will show highly isomorphic openings:
- The system prompt is very short and barely introduces plugins or extended capabilities.
- The only available actions in the first turn are “inspect the environment” and “read content.”
- The first few steps of successful samples are highly similar: list directories, read entry files, run existing checks, and make small changes.
- Failed samples are filtered out, or only the final successful branch is kept.
With supervised fine-tuning or preference optimization on this distribution, the objective rewards openings that “look like successful trajectories.” The model does not need to truly learn to plan over an arbitrary tool graph. It only needs to recognize the opening signature and emit the first few steps with high probability, and the reward is already high. Even if more tools appear later, autoregressive decoding continues the script that has already started unless training explicitly contains samples where the environment changes midway and replanning is required.
Therefore, the object of overfitting is often not the standard answer to a particular problem but the statistical fingerprint of the scaffold: prompt length, tool-name style, format of the first observation, and even the order of “which read-only action to call first.”
Designing evaluations that separate these hypotheses
A single comparison that changes modes is not enough. The evaluation should be a set of mutually exclusive controls, with the task set fixed in advance to avoid selecting a favorable subset after seeing scores.
Control A: Separate tool count from information content.
Prepare three environments. The minimal set keeps only execution and reading. The equal-noise set keeps the same number of tools, but their names and descriptions are meaningless placeholders, to test whether the length of schema text itself hurts scores. The full set uses real, usable, accurately described tools. If the minimal set is high while both the noise set and the full set are low, the issue looks more like context burden. If the noise set is also high, the model is mainly reacting to the surface feature “few tools” rather than using tool semantics.
Control B: Anchoring and delayed reveal.
Give only the minimal set in the first turn, then inject the full set after the model’s first call. Record tool selection before and after injection, whether the model rereads task instructions, and whether it abandons its original plan. Also run reverse anchoring: give the full set in the first turn, then withdraw it to the minimal set after the second call. If only “minimal first, full later” preserves a high score while “full first, minimal later” collapses, the benefit comes from the opening signature rather than the model being genuinely better with fewer tools.
Control C: Same task, different wording.
Keep the tool set and executable semantics unchanged, and change only the paragraph order of the system prompt, the level of detail in tool descriptions, and the field-naming style. Large score swings indicate reliance on surface templates. Stable scores make it more reasonable to talk about task capability.
Control D: Process metrics, not just final scores.
Record first-turn action entropy, invalid-call rate, repeated-read rate, number of replans after an environment change, and steps to the first effective edit. Overfit trajectories often show extremely concentrated first-turn actions, almost no plan changes after a mutation, and still favorable final scores.
These controls use general experimental design and do not depend on any private mode name from an evaluation product. When reporting results, include the task source, whether it is public, filtering rules, and random seeds, so that a non-reproducible single run is not presented as a capability conclusion.
Reducing scaffold memory during post-training
Evaluation can reveal the problem; training can reduce it. Practical principles include the following.
Diversify the opening distribution.
Generate multiple system prompts and tool lists for the same task: more or fewer tools, detailed or sparse descriptions, and when necessary, harmless but real additional tools inserted randomly. Do not collect successful trajectories from only one scaffold. If 90% of openings in the data share the same minimal signature, later alignment steps will struggle to remove that preference.
Add environment mutations.
Change visible tools, permissions, or the working directory midway through a trajectory, and retain successful samples where the model notices the change and replans. In preference learning, responses that ignore new tools and plow ahead with the old script should lose to responses that first confirm the environment and then adjust the plan. This removes the reward for high scores in the anchoring experiment.
Separate interface skills from task skills.
Use short trajectories to teach “how to read a schema, how to handle missing parameters, and how to degrade gracefully after a tool failure.” Use another batch of trajectories to teach “how to understand a repository and a requirement.” If both types of data are mixed under the same template, the model may treat the template as part of the task.
Retain failures and recoveries.
Keeping only eventually successful paths teaches the model that the world always unfolds along familiar first steps. Retaining some recoverable failure observations with correct recovery actions can reduce the payoff of “memorizing the opening.”
Use process constraints instead of memorized answers.
Rewards should be tied to checkable intermediate facts: whether relevant files were read, whether the edit was minimal, whether the check command was actually executed, and whether its output was cited. Do not reward call sequences that match a hidden scaffold word for word.
An illustrative data-construction logic can be written as the following pseudocode. It expresses only the experimental structure, not any platform API:
for task in tasks:for scaffold in sample_scaffolds(task): # minimal, full, shuffled descriptions, delayed revealtraj = rollout(policy, task, scaffold)if not passes_hidden_checks(traj):continuerecord(traj, meta={"n_tools_at_start": scaffold.start_tools,"mutated": scaffold.mutation_step,"replanned": detected_replan(traj),})# During training, sample by start_tools and mutated strata to avoid one type of opening filling the batch
How to read scores without misjudging them
For engineering leaders, a stratified scorecard is more useful than the statement “minimal mode is stronger.”
- If the gap among minimal, full, noise, and anchoring environments is smaller than the variance of the tasks themselves, the interface is probably not the main bottleneck.
- If the anchoring experiment is significantly higher than “full tools from the first turn,” treat the proportion of opening templates in post-training data as a defect, not the minimal-mode high score as a release highlight.
- If there are many invalid calls in the full-tool environment, first fix tool descriptions, merge duplicate capabilities, and shorten schemas. This is an interface-design issue, and the model need not be blamed first.
- When making external capability claims, publish the environment settings as well. Reporting only the most favorable scaffold turns evaluation choice itself into another form of overfitting.
Avoid the opposite error as well: do not deny the reasonable phenomenon that “fewer tools can force the model to think” merely because overfitting is suspected. More tools consume more context with schema text and create more combinations for incorrect parameters. A score drop can be an honest cost. The criterion is always whether semantic capability changed or only the opening signature changed. Anchoring experiments, reverse anchoring, and noise tool sets are designed to make that cut.
Practical checklist
- Fix the task set and write the four environments in advance. Do not change modes after seeing scores.
- Record both final scores and first-turn action distributions, plus whether replanning occurs after mutations.
- Audit post-training trajectories: count the variety of opening tool sets. If one signature dominates, fix the data before discussing model size.
- Stratify training batches by scaffold type and include trajectories where tools change midway.
- Decouple production evaluation from internal scaffolds, and periodically rerun the same tasks with wording that never appeared in training prompts.
- Treat “high score in minimal environment” as a risk signal and research starting point, not as a standalone capability conclusion.
Agent post-training can easily learn “how to start” more robustly than “how to solve.” A score jump from minimal mode can be interpreted as cleaner reasoning only after opening anchoring has been ruled out. Otherwise, it shows that the evaluation scaffold has been written into the model’s habits, not that general coding capability has saturated.
