AI Coding with LLMs: Why Model Capability Alone Isn't Enough, and How Agent Frameworks Change the Result

    The same reasoning-capable LLM can perform very differently across coding tools. This article explains why: coding performance is a combination of model capability and the execution harness (agent framework) built around it. It breaks down the five core components of a mature harness—system prompts, tool chains, project context and memory, sub-agents, and permission management—explains why these factors let strong models deliver more, covers the trade-off between total cost and first-party native integration, and outlines a practical workflow for evaluating and building your own harness with a reproducible benchmark set and data-driven tuning.

    1. The Phenomenon: Why the “Same Model” Feels So Different Across Tools

    Most developers eventually run into a counterintuitive situation: the same LLM with strong reasoning ability performs unremarkably when plugged into Tool A, yet can independently carry out multi-file refactors and fix bugs automatically when plugged into Tool B. This raises a common question — should you use the coding tool shipped by the model vendor itself, or an established general-purpose agent execution framework?

    Answering that question requires dropping the “model is everything” assumption. What actually determines the coding experience is the collaboration between model capability and the execution harness. The model is the brain: it reasons and generates. The harness is the nervous system and the hands: it turns the brain’s intent into a complete loop of actions that can be executed, verified, and corrected. However strong the brain is, without good neural pathways it cannot perform fine motor skills.

    2. Core Concepts: Where Model Capability Ends and the Harness Begins

    Model capability refers to the model’s own reasoning depth, code comprehension, instruction following, context window size, and generation quality. This part is largely determined by the model itself, and developers have limited room to adjust it.

    The execution harness is the entire set of mechanisms built around the model. It decides what the model “sees, can do, and how it is constrained.” This is the part developers can genuinely tune in depth, and it is the largest source of difference between tools.

    A mature harness essentially does three things: give the model sufficient and precise information (context engineering), give the model safe and effective means of action (tool use), and establish a verifiable feedback loop (testing and self-correction). Each is examined below.

    3. The Five Core Components of an Execution Harness

    3.1 System Prompt: Defining the Model’s Role and Behavioral Rules

    The system prompt is the “constitution” of the whole harness. It determines whether the model strictly follows code conventions, whether it prefers minimal changes, and whether it reads and understands existing code before making edits. A well-designed system prompt makes things explicit: reuse dependencies the project already has, avoid pulling in unnecessary new libraries, run verification proactively after making changes, and so on.

    A poor system prompt lets the model improvise, producing code that looks plausible but does not run. A good one works like a strict technical lead, keeping the model on a reliable engineering track.

    3.2 Tool Chain: From “Talking About It” to “Doing It”

    The tool chain gives the model real capability to act. Typical tools include reading and writing files, executing commands, running tests, searching the codebase, and making network requests. The value of the tool chain lies in closing the “act — observe — correct” loop.

    1. Model decides → calls a tool (e.g., runs unit tests) → reads the result
    2. ↑ │
    3. └────────── fixes the code based on errors ←──────┘

    Without a tool chain, a model can only talk in the abstract. With a complete one, a framework lets the model self-correct through compiling, testing, and reading errors the way a real engineer would — and that is the dividing line between toy-grade completion and production-grade coding.

    3.3 Project Context and Memory: Helping the Model Understand the “Whole Project”

    Conversation context alone is nowhere near enough when facing a large codebase. Mature frameworks introduce a project context configuration file so the model understands the project’s tech stack, directory conventions, coding style, and common pitfalls before it starts the task. For example:

    1. # Project context configuration (example)
    2. ## Tech stack
    3. - Languages/frameworks, package manager, test framework
    4. ## Code conventions
    5. - Naming conventions, indentation, comment requirements
    6. ## Key rules
    7. - Do not introduce third-party dependencies that have not been reviewed
    8. - Run the full unit test suite before committing

    This kind of “project memory” substantially reduces the chance that the model repeats mistakes or drifts away from engineering conventions, making collaboration more stable and more predictable.

    3.4 Sub-Agents: Dividing and Isolating Complex Work

    For complex tasks that span multiple modules, mature frameworks support dispatching sub-agents. The main agent handles planning and coordination, while sub-agents each take on subtasks such as searching, coding, and testing, with their contexts kept isolated from one another. The benefit is twofold: the main context is not “polluted” by huge volumes of intermediate information, and different subtasks can proceed in parallel — improving both overall efficiency and accuracy.

    3.5 Permission Management: A Safe and Controllable Boundary

    AI-assisted coding has to answer the question of what the model may and may not do. A mature permission mechanism asks for confirmation before high-risk operations (deleting files, pushing code, accessing the network) while letting low-risk operations that can be automated run freely. Good permission design is the key to balancing efficiency and safety.

    4. Why a Mature Framework Lets a Strong Model Perform Better

    Taken together, a mature execution framework makes a strong model stronger because it systematically amplifies the model’s strengths while constraining its unpredictability:

    • A more precise system prompt → stronger instruction following
    • A more complete tool chain → a more genuine self-correction loop
    • Better project context → fewer misjudgments about project conventions
    • More sensible sub-agent scheduling → higher success rate on complex tasks
    • Stricter permission management → lower risk of costly mistakes

    This also explains the phenomenon at the start of this article: switch frameworks, and the experience can be completely different — and that difference usually has nothing to do with model capability.

    5. Weighing Total Cost Against Native Fit

    Beyond framework maturity, real-world selection also involves weighing two kinds of cost.

    First, total cost. On top of subscription fees, there is the hidden consumption driven by call volume and context length, plus the time your team spends learning and tuning the framework.

    Second, native fit. A model vendor’s first-party tool typically has the advantage of native alignment with its own models, out-of-the-box usability, and predictable reliability. A general-purpose execution framework typically has the advantage of more mature workflows, more flexible model switching, and a richer ecosystem of extensions.

    The rational approach in practice is to run A/B comparisons based on task quality and overall cost-effectiveness, rather than comparing subscription prices alone. Many developers end up pairing a mature execution framework with a cost-efficient model, or even mixing combinations at different stages of a workflow, to strike the best balance between quality and cost.

    6. Putting It into Practice: How to Evaluate and Build Your Harness

    1. Build a reproducible benchmark set. Pick 10–20 typical tasks your team has actually encountered (bug fixes, new features, refactors) and use them as a common yardstick for comparing combinations.
    2. Focus on loop capability, not one-shot generation. What matters most is whether the model can self-correct through tests, not whether the first version of the code “looks fine.”
    3. Capture project context. Write the tech stack, conventions, and rules into a configuration file so the model always starts from the reality of the project.
    4. Establish a permission and security baseline. Define clearly which operations can run automatically and which require human confirmation.
    5. Observe and tune continuously. Use the logging and monitoring/alerting services commonly offered by cloud platforms to record call traces, success rates, and cost distribution, and let that data drive framework optimization. General-purpose infrastructure from cloud providers — container platforms, object storage, message queues, and similar services, such as those offered by Baidu AI Cloud — can serve as a stable foundation for this kind of AI coding workflow.

    7. Summary

    “First-party tool or mature framework” has never been an either/or question. The real answer is: the model sets the ceiling, and the framework decides whether you can reach it. When you run a strong model on a general-purpose execution framework, the more mature the five dimensions — system prompt, tool chain, project context, sub-agents, and permission management — the more fully the model’s real capability is unleashed. The recommendation for developers is to use real tasks as the yardstick for comparison, use total cost and output quality as the criteria for selection, and keep accumulating your own project context and tuning experience. That is the fundamental path to turning a strong model into dependable productivity.