Agent Engineering: A Practical Framework for Shipping Programming Capability

    Complex software fails when local patches look right but cannot be delivered across repos, contracts, permissions, and release windows. This article explains why vibe coding breaks down, which agent capabilities matter, which model techniques support long-horizon work, how to evaluate against real engineering, and how to bound cost, permissions, and rollout.

    Complex software is no longer a snippet that happens to run. Requirements span repositories. Constraints come from interface contracts, permissions, data, and release windows. Failures often look reasonable locally and still cannot be delivered. Dropping a chat model into an editor produces vibe coding: lucky prompts, unreproducible patches, and human-driven regression. Agent engineering is different: given a clear goal, tool boundaries, and acceptance criteria, the model must keep planning, calling tools, checking results, and correcting the path.

    Why vibe coding fails

    Vibe coding relies on one-shot or few-shot completion. It is efficient for boilerplate, isolated functions, and known APIs, where context is short and correctness is easy to eyeball. Once the task is “fix a cross-module defect” or “migrate an old interface to a new protocol,” a single generation lacks three things.

    First, state. The model cannot see whether tests actually ran, dependencies were installed, or configuration matches production. Second, responsibility boundaries. It may edit unrelated files, loosen checks, or swallow exceptions just to make the current error disappear. Third, a stopping condition. Without a machine-checkable definition of done, the conversation drifts into “try again.”

    Programming capability should not be read as “answers that sound like a senior engineer.” It should be read as “finish the task in a controlled tool environment and leave evidence.” Evidence is failure logs, passing tests, the scope of the diff, and unresolved risk—not a confident paragraph.

    Capabilities a programming agent needs

    A usable coding agent has to hold four capabilities at once.

    Planning and decomposition. Split the goal into verifiable sub-steps and revise the plan when new information appears, rather than rewriting from scratch. On long-horizon tasks, the plan itself should be inspectable: current assumptions, and which observation the next step depends on.

    Tool use. Reading files, searching symbols, running tests, inspecting diffs, and issuing read-only queries should have least privilege and timeouts. Writes and high-risk commands need separate authorization. Tool output is observation, not a model-invented “it should have succeeded.”

    Long-context use. Cross-file constraints, interface notes, and historical failures belong in the same working memory. A larger window does not automatically mean better understanding: irrelevant fragments dilute attention, and repeated logs crowd out real contracts. Engineering still needs retrieval, summarization, and citation so long context locates facts instead of dumping raw text.

    Self-checking. Generation must be followed by checks independent of generation: compile, unit tests, static rules, contract tests, or a small replay. Failures feed the next observation; they are not explained away.

    The practical payoff is that “can the model write code?” becomes “in our repos and permission model, can we measure task success, rework, and human intervention?”

    Model techniques that support long-horizon work

    Discussions of coding and agent models often mention a few architectural choices. They are general concepts, not product-specific switches.

    Sparse attention. Standard self-attention compute and memory grow roughly with the square of sequence length, so long logs, long diffs, and multi-file context get expensive fast. Sparse attention lets each position attend to a subset of history, trading some information for longer sequences and lower unit cost. In practice, the sparse pattern must still preserve local code neighborhoods and a few global tokens; otherwise the window is large but the critical constraint is missed.

    Mixture of experts. Total parameters can be large while only a subset of experts activate, trading capacity against per-step compute. For programming, capacity helps cover languages, frameworks, and failure modes; the activation ratio sets latency and cost. Higher sparsity is not always better: overly specialized experts and unstable routing yield hit-or-miss results on similar problems. Evaluate routing stability and whether rare languages starve, not just parameter count.

    Multi-token prediction. Training or decoding that predicts several future tokens can raise throughput, help local syntax, and provide signal for speculative decoding. It does not guarantee logical correctness. For agents, faster local generation must still obey tool-returned facts; otherwise it only produces wrong patches faster.

    Asynchronous reinforcement learning or environment-feedback post-training. Rewards in programming and operational simulations often arrive late: tests must finish, multi-step tool chains must end. Asynchrony can pipeline sampling, evaluation, and updates and improve utilization of expensive environments. Risks are equally clear: delayed reward blurs credit assignment; unstable environments treat noise as policy improvement; if reward only cares that tests turn green, the model learns to delete tests or weaken assertions. Reward should also constrain diff scope, forbidden actions, and whether new failures appear.

    More data usually improves coverage but does not replace task distribution. If pretraining or post-training lack your real build system, private frameworks, and failure modes, public-benchmark gains will not transfer. Data governance must separate public code, internal code, and logs that contain secrets so confidential material does not enter training outside reproducible experiments.

    Evaluation that resembles real development

    Do not substitute a single chat impression for evaluation. Split tasks into complementary layers, each with its own failure types, rather than one total score.

    Software repair. Given a failing test or bug description, the model submits a minimal patch in an isolated repo. Record pass rate, lines changed, whether unrelated modules were touched, and whether the network was required. A passing patch that inflates the change surface is not equivalent to a small correct one.

    Terminal and environment operations. In a clean environment, install dependencies, read errors, change configuration, and rerun. Watch whitelist compliance, environment damage, and whether timeout is followed by a strategy change based on new logs.

    Tools and browsing. When answers depend on external docs or internal interface notes, check that citations are traceable, parameters match the docs, and the agent stops when permission is insufficient instead of guessing.

    Long-horizon consistency. Multi-step tasks that allow intermediate failure. Metrics include step count, repeated-error count, human takeover points, and whether the final state is reproducible.

    Business or resource-constraint simulations can test multi-step decisions, but high scores on simple rules only show adaptation to those rules and do not transfer to real commercial systems. Any leaderboard should publish task version, tool permissions, whether multiple attempts are allowed, and decoding settings such as temperature. Without that, ranks are not comparable.

    An internal set matters more than a public one. Sample recent real defects: cross-service, read-only diagnosis, write a patch, and rollback. Fix random seeds and base images, store logs in object storage, keep tool traces, and alert on timeout and failure rates. You are reviewing behavior, not impression.

    Inference cost and long sequences

    Long-context bills usually come from three places: prefill over a long prompt, repeated concatenation of multi-turn tool results, and retries that recompute the same context. Common cost cuts: inject only symbol-relevant fragments; cache stable prefixes; cap log length fed back each step; summarize before retrying and set a max step count; separate parallel read-only checks from serial writes.

    Claims such as “one node approaching multi-GPU” or “long-sequence cost cut in half” are meaningful only under the same batch, precision, sequence length, and service quality. Fix those conditions before citing numbers. Hardware adaptation is about operator coverage, numeric consistency, long-sequence memory strategy, and graceful degradation—not slogans. Before switching, run the same eval set for accuracy and latency; do not switch if the gap exceeds your threshold.

    Process, permissions, and risk

    Agents fit repetitive work with automated acceptance: fix local logic from a failing test, fill an adapter from a template, change configuration under documented constraints. They should not unilaterally decide data deletion, permission expansion, production release, or irreversible migration. Those actions keep human approval, and the approval object is the diff and test report, not the model’s prose.

    Default to read-only. Writes stay on a working copy. Secrets stay out of the prompt and are injected by the runtime. Network is off by default, opened only to documentation mirrors or internal read-only APIs. Log every command so you can tell model error from environment drift.

    System instructions should define done, forbidden actions, and how to stop under uncertainty. For example: do not claim complete until all tests pass; if you cannot reproduce, emit a minimal repro and stop; if a suspected secret appears, abort and alert. Those rules prevent more incidents than swapping in a larger model.

    If the team uses a cloud development environment, containers can pin the build, queues can serialize eval jobs, and logs plus monitoring can track steps, timeouts, and takeovers. General cloud capabilities can host reproducible environments and observability; they do not replace your eval set, permission design, or release gates. Compare task success, latency, cost per task, and explainability of failures—not advertised parameter counts.

    Rollout order

    Start with fewer than ten internal tasks with high automation of acceptance. Pin the environment and scoring scripts. Give the agent a minimal tool set and run enough failures to see whether errors are retrieval, planning, or reward hacking. Only then consider longer context, sparser compute, or post-training. Keep a rollback baseline at every step: the same tasks must still complete under the old human-in-the-loop process.

    When pass rate, change scope, and takeover cost are stable together, expand languages and repos. Parameter count, expert count, and context length are capacity knobs. What actually moves you from vibe coding to agent engineering is whether observations are real, rewards resist gaming, permissions are minimal, and failures are recorded as the next executable constraint.