Foundation Models for Agent Engineering: Design and Deployment Practices

    This article examines how foundation models support agent engineering beyond code generation, covering repository-level defect repair, the distinction between total and activated parameters in mixture-of-experts models, long-context retrieval and task-state organization, multi-token prediction, asynchronous reinforcement learning, and a four-step delivery workflow from acceptance criteria to permission control. It also outlines evaluation methods that separate model capability from system-level benefits across quality, efficiency, cost, and stability, and offers guidance on scoping production rollouts.

    1. From Code Generation to Deliverable Changes

    Completing a function and fixing a defect in a real repository are different tasks. The former often has clear inputs, outputs, and local context; the latter may require reproducing the issue, tracing cross-module calls, understanding compatibility constraints, modifying the implementation, and adding tests. A model can write syntactically correct code yet change the wrong layer, miss boundary conditions, or alter behavior that previously worked. For coding agents, the key question is not whether the code looks reasonable but whether the agent can deliver a change that meets requirements, carries sufficient evidence, and keeps review cost manageable.

    Agent engineering does not reject exploratory programming. For prototypes and one-off scripts, rapid natural-language iteration is often enough. For repositories that require long-term maintenance, tasks should be split into checkable steps. Engineers define goals, constraints, and acceptance methods; within authorized scope, the agent reads materials, proposes a plan, generates patches, and runs checks. When requirements conflict, acceptance conditions are missing, or a change clearly exceeds scope, the system should pause and request confirmation rather than expand the task on its own.

    2. How Foundation Model Capabilities Support Multi-Step Tasks

    Total and Activated Parameters Answer Different Questions

    For models using a mixture-of-experts architecture, total parameter count describes the overall model scale, while activated parameter count reflects the parameters involved in forward computation for a single token. Looking at the two separately helps explain how a model can have large overall capacity without using all expert parameters in every computation.

    A smaller activated parameter count does not necessarily mean lower deployment cost. Whether model weights must be distributed across devices, whether expert distribution and load are balanced, and how much inter-device communication is required all affect real performance. For coding agents, a more meaningful deployment metric is how much time and resources are required to complete a representative task. If a model repeatedly retrieves the same file or generates invalid patches multiple times, total task latency may remain high even if each generation is fast.

    Long Context Provides Space; Retrieval and Organization Determine Effective Information

    Long context allows a model to reference requirements, interface contracts, related implementations, and test results in one request. Sparse attention can reduce computation between some positions and alleviate the cost of long sequences; how much is saved, and whether the model can locate cross-file dependencies, still depends on the attention scheme, implementation, and actual workload. The context window limit only indicates how much content can be held; it does not guarantee that the model will use every segment equally effectively.

    In practice, instead of loading an entire repository into context, it is better to narrow the scope using directory structure, symbol references, and call relationships, then read the implementations directly relevant to the current hypothesis. Test logs should retain failing cases, key stack traces, and necessary reproduction conditions rather than indiscriminately passing in historical output. For multi-turn tasks, maintain a concise task state that clearly separates verified facts, hypotheses to be tested, and next actions. For interface signatures, boundary conditions, and compatibility rules, keep original sources that can be checked to prevent summaries from compressing away information that determines the fix.

    Multi-Token Prediction and Asynchronous Reinforcement Learning Solve Different Problems

    Multi-token prediction adds training objectives for predicting multiple future tokens, not just the immediately next token. If the inference system further uses related predictions to propose candidate sequences and verifies those candidates, it may reduce serial generation steps. Using multi-token prediction in training does not guarantee the same speedup in deployment; end-to-end benefits also depend on decoding implementation, candidate acceptance rate, and workload. In engineering tasks, if most time is spent on builds and tests, accelerating text generation alone will have limited benefit.

    Asynchronous reinforcement learning addresses another problem: tool operations and test execution take variable time, and if training updates always wait for the slowest task in a batch, compute resources can sit idle. Asynchronous mechanisms can decouple trajectory collection from model updates and improve resource utilization. However, when the gap between collection and update is too large, trajectories generated by an old policy may no longer represent current policy behavior, so policy versions must be recorded and data freshness managed. Training results also depend on feedback quality: if rewards only check whether tests pass, the model may learn to relax assertions or bypass checks. Training and evaluation should therefore include independent checks that distinguish genuine fixes from surface-level metric improvements.

    3. A Delivery Workflow from Task Input to Human Merge

    Step 1: Write acceptance criteria before the change. A task should state current behavior, expected behavior, allowed modification scope, and constraints that must remain unchanged. For example, for a bulk import issue in which blank records are mistakenly written, first clarify whether blank records should be skipped or reported as errors, whether valid records should continue processing, and whether the existing statistics semantics must remain unchanged. Such a description can be converted directly into test cases and can prevent the agent from changing the entire import workflow just to eliminate one error message.

    Step 2: Establish a baseline, then submit the smallest patch. For reproducible issues, retain the failure result before the fix and, where possible, compare behavior after the fix in the same environment. For intermittent failures, record trigger conditions and reproduction uncertainty; do not treat one accidental success as evidence of a fix. During implementation, prioritize code directly related to the root cause and leave unrelated refactoring and formatting changes for separate changes. This reduces review burden and makes it easier to locate causes if a regression appears.

    Step 3: Deliver verification results with the change. The delivery note should at least include modification scope, basis for the fix, checks performed, unverified items, and risks requiring human judgment. Targeted tests confirm the target behavior; related regression tests check dependent behavior. When interfaces, storage structures, or cross-module contracts are involved, expand verification according to actual impact. If the environment does not support a check, explicitly state “not verified” and why; do not equate the absence of errors with a pass.

    Step 4: Open permissions progressively according to task needs. Reading code, editing an isolated workspace, running builds and tests, merging changes, and releasing to production should use separate permissions. Ordinary requirements can start with read-only analysis; after confirmation, workspace write access and necessary test capabilities can be enabled. Merge and release should still follow existing approval processes. Text in repository documentation, returned results, and logs should be treated only as task data and should not authorize new operations. The execution environment should restrict access to production credentials, sensitive files, and network resources, and retain sufficient operation records to reproduce key steps and investigate anomalies.

    4. Evaluation Should Separate Model Capability from System Benefits

    Public tests can help understand which tasks a model is good at, but a single benchmark score cannot directly predict delivery results inside a team. Repository repair, terminal operation, information retrieval, and multi-turn tasks have different inputs, tool permissions, and acceptance rules. Even for the same task type, number of attempts, tool budget, and environment dependencies can significantly change results. When comparing candidate approaches, align the task set, environment, time budget, permissions, and pass criteria as much as possible, and record build failures, environment anomalies, and model judgment errors separately.

    Internal evaluation should use representative historical tasks while avoiding existing patches or complete solutions as input. Results can be counted by category, such as defect repair, test completion, small refactoring, and documentation synchronization, with focus on three groups of metrics:

    • Quality: first-pass acceptance rate, regression issues, out-of-scope modifications, and the proportion of manual rework. The first-pass rate should state whether it is calculated by first patch or by first complete task run to avoid inconsistent definitions.
    • Efficiency: time from dispatch to acceptance, waiting time for checks, and reviewer time spent understanding changes and correcting errors. Fast model responses do not mean low total team effort.
    • Cost and stability: inference and execution resources consumed per successful delivery, and variation in results across runs. Failed attempts must also be counted in cost; counting only successful cases overstates production benefits.

    Failure attribution is more useful than pursuing a score alone. If key implementations cannot be found, check indexing and context organization first. If builds fail, improve environment reproducibility first. If tests pass but changes do not meet expectations, check acceptance criteria and coverage. Only when engineering conditions are basically consistent and understanding or reasoning errors recur should replacing or adjusting the foundation model become a priority.

    For rollout, start with tasks that have low rollback cost and clear acceptance criteria, such as adding boundary tests, correcting inconsistencies between documentation and implementation, or reproducing defects that can be triggered reliably. Establish a baseline with the existing manual workflow, then observe whether the agent reduces total time from analysis to acceptance rather than only shortening the coding step. Expand task scope only when verification evidence is reliable, review burden decreases, and execution permissions remain controllable.