Building a Code Agent from Scratch with Python: Architecture, Implementation, and Best Practices

1. Why Understanding the Internal Mechanics of Code Agents Matters

As large language models continue to evolve, “AI that can write code” has become part of everyday development. However, most developers treat AI coding tools as black boxes and lack a deeper understanding of their architecture and runtime behavior. A real code agent is not simply “input a question, output code.” It is a complete system that can autonomously understand requirements, plan steps, call tools, execute code, and iterate based on feedback.

For developers who primarily use Python, learning the underlying implementation often faces two challenges. First, many mainstream mature implementations are closed-source products, making it impossible to inspect their internal logic. Second, even when open-source references exist, they are often based on other technology stacks and are difficult to borrow directly. Therefore, building an understandable and extensible reference implementation of a code agent from scratch with Python is a key path from “using it” to “understanding how it works.” This article focuses on that goal.

2. Core Concepts and Capability Boundaries of Code Agents

Before starting implementation, it is important to clarify the essential difference between a code agent and a traditional code-generation model. A traditional model performs a “one-shot mapping”: given a prompt, it directly generates a code snippet, and its capability is limited by the context window and a single inference pass. A code agent introduces a closed loop of “perception, decision, action, and feedback,” with the following core capabilities:

• Autonomous planning: Break down a complex task into executable sub-steps and adjust the plan dynamically.
• Tool use: Read and write files, execute commands, retrieve documentation, and run tests, rather than being limited to text generation.
• State memory: Maintain context during long tasks and remember completed steps and intermediate results.
• Iterative correction: Automatically adjust the approach based on code execution results, error messages, or test feedback.

It is important to emphasize that an agent is not “a stronger model.” It is a combination of “model plus engineering constraints.” The model handles reasoning and generation, while the engineering framework constrains behavior, provides tools, manages context, and ensures safety. Understanding this division of labor is a prerequisite for designing a good agent.

3. Overall Architecture Design

A maintainable code agent usually adopts a layered architecture. The following diagram describes the responsibilities of each layer in logical structure:

+--------------------------------------------------+
|                  User Interface                  |
|      Task input, progress display, result feedback, human confirmation    |
+--------------------------------------------------+
|                Agent Control Layer               |
|   Reasoning -> Decision -> Tool Call -> Observation -> Reasoning (loop)  |
+--------------------------------------------------+
|      Capability Layer: Tools | Context Management | Memory and Retrieval |
+--------------------------------------------------+
|      Execution Layer: Code Sandbox | File System | Command Execution |
+--------------------------------------------------+
|      Model Layer: LLM inference interface (can be replaced by any backend) |
+--------------------------------------------------+

The value of this layering is separation of concerns: the control layer focuses on “what to do next,” the capability layer handles “what can be done,” and the execution layer handles “doing it safely.” When the model backend needs to be replaced, new tools need to be added, or security policies need to be adjusted, the changes can be limited to the corresponding layer instead of affecting the entire system.

4. Breaking Down Core Modules

4.1 Requirement Understanding and Task Planning

The first step for an agent is not to write code, but to transform a vague natural-language requirement into a structured execution plan. A practical approach is to first ask the model to output a “task checklist” that defines the goal, constraints, and acceptance criteria, and then proceed item by item. The following is illustrative code:

def plan_task(requirement: str, context: dict) -> list[dict]:
    prompt = f"""
    Please break down the following development requirement into ordered, independently executable steps.
    Each step must include: goal, affected files, expected output, and acceptance criteria.
    Requirement: {requirement}
    Known context: {context}
    Return as a JSON array.
    """
    result = llm.complete(prompt, response_format="json")
    return json.loads(result)

The key to the planning stage is to keep the plan “revisable.” In real development, requirements often change midway. Therefore, the agent should not treat the plan as an immutable contract. Instead, it should reevaluate whether adjustments are needed after each execution step.

4.2 Tool System: The “Hands and Feet” of a Code Agent

The tool system is the core feature that distinguishes a code agent from an ordinary chatbot. The model itself cannot directly operate the file system or execute code; it must do so indirectly through tool calls. A well-designed tool should have a clear name, a clear input and output schema, and a detailed description.

from dataclasses import dataclass
from typing import Callable
@dataclass
class Tool:
    name: str
    description: str
    parameters: dict      # JSON Schema describing input parameters
    handler: Callable     # Function that actually executes the tool
def read_file(path: str) -> str:
    """Read the content of the specified file."""
    with open(path, "r", encoding="utf-8") as f:
        return f.read()
def run_command(cmd: str) -> dict:
    """Execute a command in a controlled environment and return the output."""
    ...
TOOLS = [
    Tool("read_file", "Read file content", {...}, read_file),
    Tool("run_command", "Execute a shell command", {...}, run_command),
]

The quality of tool descriptions directly affects the performance of the agent. Descriptions should be like interface documentation written for a new colleague: they should clearly explain what the tool does, when to use it, and what its parameters mean. In practice, tool granularity needs to be balanced. Tools that are too coarse make it difficult for the model to control them precisely. Tools that are too fine-grained increase the number of call rounds and consume context too quickly.

4.3 Execution Loop: The “Brain” of a Code Agent

The execution loop is the central hub of the entire agent. Its essence is to let the model repeatedly switch between “reasoning” and “acting” until the task is completed or a termination condition is triggered. The classic loop pattern can be summarized as follows:

def agent_loop(task: str, max_steps: int = 30):
    messages = [{"role": "user", "content": task}]
    for step in range(max_steps):
        # 1. Let the model reason over the current context and decide the next action.
        decision = llm.complete(messages, tools=TOOLS)
        messages.append(decision)
        # 2. If the model requests tool calls, execute them and add the results.
        if decision.tool_calls:
            for call in decision.tool_calls:
                observation = execute_tool(call)   # Execute safely
                messages.append({
                    "role": "tool",
                    "content": str(observation),
                    "tool_call_id": call.id
                })
        else:
            # 3. The model considers the task complete and returns the final answer.
            return decision.content
    return "Maximum steps reached; task not completed"

Several engineering points are worth emphasizing here:

• Termination conditions must be explicit: There should be a normal exit where the model declares completion, as well as fallback mechanisms such as maximum steps, timeouts, and exceptions, to prevent the agent from entering an infinite loop.
• Observations must be returned: The output of tool execution must be injected back into the context so that the model can “see” the result and make decisions based on it. This is exactly what makes the loop a closed loop.
• Errors should be feedback, not interruptions: When code raises an error or a test fails, the error information should be returned to the model as an observation so that it can correct itself, rather than terminating directly.

4.4 Context Management: Fighting “Memory Forgetting”

The context window of a large model is limited, while the intermediate artifacts of real development tasks, such as file contents, command outputs, and conversation history, can grow quickly. Without management, the context can quickly exceed the window limit, causing early critical information to be “pushed out.” Context management is therefore critical to the success of long-running tasks.

Common strategies include:

• Sliding window: Keep only the most recent interaction rounds and compress earlier content into a summary.
• Importance filtering: Prioritize retaining the task goal, confirmed decisions, and the files currently being edited, while discarding redundant intermediate outputs.
• External memory: Write the complete history to external storage and load it through retrieval when needed, instead of putting everything into the context at once.

The following is a simplified example of summary compression:

def compress_history(messages: list, keep_recent: int = 6) -> list:
    if len(messages) <= keep_recent:
        return messages
    older = messages[:-keep_recent]
    recent = messages[-keep_recent:]
    summary = llm.complete(
        "Please compress the following conversation history into key points, preserving important decisions and unfinished items:\n"
        + format_messages(older)
    )
    return [{"role": "system", "content": f"Historical summary: {summary}"}] + recent

4.5 Security and Sandboxing: An Unavoidable Line of Defense

A code agent can execute arbitrary code and commands, which means it is inherently high-risk. A single misjudgment or malicious input could delete files, leak sensitive information, or even damage the system. Therefore, security isolation is a prerequisite for production use, not an optional feature.

Core protection measures include:

• Execution isolation: Put code execution into a container or restricted sandbox to limit its access to the host system.
• Least privilege: Grant only the file and command permissions necessary to complete the task, and reject dangerous operations by default.
• Human confirmation checkpoints: For high-risk actions such as deletion, deployment, and network access, require manual approval before execution.
• Input validation: Perform basic injection detection on commands generated by the model to avoid command concatenation vulnerabilities.

When cloud resources are involved, follow the general principle of least privilege. Reduce risk through credential isolation, access control policies, and audit logs, rather than allowing the agent to hold unrestricted account credentials.

5. Development Process from Requirements to Implementation

By connecting the modules above, a practical development path can be divided into five stages:

1. Requirement analysis: Define the agent’s target scenarios, such as code refactoring, defect fixing, and test generation, and specify the capability boundaries and what the agent should not do.
2. System design: Determine the layered architecture, tool list, model backend interface, and context management strategy, and produce interface contracts.
3. Minimum viable prototype: First implement the smallest closed loop of “reasoning, single tool, loop,” and verify basic feasibility.
4. Capability expansion and hardening: Gradually add tools, improve context management, and complete the security sandbox and exception handling.
5. Evaluation and iteration: Establish a quantifiable evaluation set, including task success rate, average steps, and token consumption, and continuously optimize based on data.

This process emphasizes “make it run first, then optimize,” avoiding excessive design at the beginning by pursuing completeness too early.

6. Common Challenges and Engineering Best Practices

In practice, developers often encounter the following problems:

• Uncontrolled loops: The agent repeatedly calls the same tool or falls into an infinite loop. The countermeasure is to set a maximum number of steps, detect repeated actions, and force an interruption.
• Tool overuse: The model calls tools randomly when they are not needed, wasting resources. The countermeasure is to optimize tool descriptions and clarify usage scenarios and prerequisites.
• Context explosion: Long tasks cause context overflow. The countermeasure is to introduce summary compression and external memory.
• Unreliable results: Generated code looks reasonable but cannot run. The countermeasure is to introduce a “write then test” feedback loop so that the agent can self-correct based on test results.

A best practice that runs throughout is: Treat the agent like a new colleague who needs constraints. Only by giving it clear tasks, appropriate tools, clear boundaries, and immediate feedback can it deliver stable results.

7. Conclusion

The value of a code agent lies not in how powerful the model itself is, but in whether a reasonable engineering architecture can connect the model’s reasoning ability to a real development environment. Starting from core concepts, this article systematically breaks down key modules such as the execution loop, tool system, context management, and security sandbox, and provides a reference implementation approach based on Python.

For developers, understanding this mechanism has multiple benefits. It enables more efficient use of existing intelligent coding tools and also provides a foundation for building customized agents for specific scenarios. The real direction for advancement is to master the principles and then, combined with specific business requirements, make the agent a trustworthy part of the development workflow under the premise of safety, controllability, and evaluability.