Building an LLM Code Agent in Python: A Hands-On Guide

AI-assisted programming has become an essential productivity tool in modern software development. Yet as coding assistants proliferate, many developers find themselves using these tools without truly understanding how they work under the hood. For Python developers in particular, the mainstream code agents are often closed-source, or their open-source versions are built in other languages, making it hard to learn from the source code or extend and customize them. This article aims to break down that barrier by showing how to build a fully functional code agent in Python from scratch.

1. The Core Architecture of a Code Agent

A code agent is not just a wrapper around an LLM that generates code snippets. It is a complex system that senses its environment, plans autonomously, executes code, and adjusts based on feedback. The key is combining the LLM’s reasoning capabilities with a real code execution environment, forming a closed loop of think – act – observe.

A typical code agent architecture includes the following layers:

1. Interaction layer: Accepts natural language requests from users and displays the execution process and results.
2. Planning and decision layer: Built around the LLM, it interprets intent, breaks down tasks, and generates code or tool-calling instructions.
3. Execution layer: Provides a secure sandbox that runs generated code and captures standard output, error messages, and environment state changes.
4. Memory and state layer: Manages multi-turn conversation context, stores intermediate variables and execution history, and provides the basis for subsequent decisions.

2. Requirements Analysis and System Design

Before implementing anything, it is crucial to define the system’s boundaries. Our goal is an autonomous agent that accepts natural language instructions, writes code, executes it, and repairs its own errors. The design should follow the principles of high cohesion and low coupling, with functionality organized into modular components.

Key design considerations include:

• Interface abstraction: Define unified interfaces for LLM calls and tool invocations, making it easy to swap underlying models or extend the toolset later.
• State management: Manage context as a list of messages so the LLM always has access to the full interaction history.
• Security isolation: Code must run in an isolated container or sandbox to prevent malicious code from damaging the host environment.

3. Building the Core Modules

3.1 Intent Recognition and Task Parsing

The agent’s first step is understanding what the user wants. This involves combining the user’s natural language input with the current context to construct an appropriate prompt. At this stage, the system should establish a clear role setting, telling the LLM that it is an agent capable of executing code.

3.2 Planning and the LLM Decision Engine

The decision engine is the agent’s brain. It is typically implemented using the ReAct (Reasoning and Acting) pattern or a Function Calling mechanism. Below is a simplified Python decision engine showing how to guide the model through reasoning and action:

import json
class DecisionEngine:
    def __init__(self, llm_client, tools):
        self.llm_client = llm_client
        self.tools = tools
    def think_and_act(self, user_query, history):
        # Build a prompt that includes tool descriptions
        system_prompt = "You are a code agent. Analyze the user's request and decide whether to write and execute code."
        messages = [{"role": "system", "content": system_prompt}] + history
        messages.append({"role": "user", "content": user_query})
        # Call the LLM
        response = self.llm_client.chat(
            messages=messages,
            tools=self.tools.get_tool_schemas()
        )
        # Parse the response to determine whether to reply with text or call a tool
        if response.tool_calls:
            return self._handle_tool_calls(response.tool_calls)
        else:
            return response.content

3.3 A Secure Code Execution Sandbox

The execution sandbox is the bridge between the agent and the real world. For security reasons, you should never run exec() or os.system() directly on the host machine. In production, you can use container platforms or WebAssembly sandboxes to isolate the execution environment. In Python, this can be done via subprocesses or containerized APIs:

import subprocess
class CodeSandbox:
    def execute(self, code_string):
        try:
            # In production, write the code to a temp file and run it in a restricted container
            # This is a simplified example showing how to capture output and errors
            result = subprocess.run(
                ['python3', '-c', code_string],
                capture_output=True, text=True, timeout=10
            )
            if result.returncode == 0:
                return {"status": "success", "output": result.stdout}
            else:
                return {"status": "error", "output": result.stderr}
        except Exception as e:
            return {"status": "timeout", "output": str(e)}

3.4 Context Memory and the Feedback Loop

Once the sandbox returns an execution result, the agent feeds it back to the LLM. If the execution succeeds, the model may proceed to the next step or report back to the user; if it fails (e.g., an exception is thrown), the model must analyze the error log, modify the code, and retry. This feedback loop is at the heart of the agent’s self-repair capability.

def feedback_loop(self, user_query, history):
    max_attempts = 5
    for _ in range(max_attempts):
        action = self.think_and_act(user_query, history)
        if action.get("type") == "text":
            return action["content"]
        elif action.get("type") == "code_execution":
            code = action["code"]
            exec_result = self.sandbox.execute(code)
            # Add the execution result to the history
            history.append({"role": "assistant", "content": f"Executed code: {code}"})
            history.append({"role": "tool", "content": f"Execution result: {exec_result}"})
            if exec_result["status"] == "success":
                history.append({"role": "assistant", "content": "Code executed successfully. Moving on to the next step."})
            else:
                history.append({"role": "assistant", "content": f"Error encountered: {exec_result['output']}. I need to revise the code."})
    return "Failed to complete the task after multiple attempts."

4. Engineering for Production and Best Practices

Moving from prototype to a production-grade code agent involves several additional engineering challenges:

1. Context window management: As task complexity grows, execution logs and code can exceed the model’s context window. You need a memory summarization mechanism to compress and distill earlier history while preserving key information.
2. Concurrency and state consistency: In multi-agent collaboration or long-running complex workflows, use message queues or state machines to manage task flow and ensure consistent state.
3. Cloud-native integration: When deploying agents in the cloud, you can use object storage for intermediate artifacts, logging services to monitor the agent’s reasoning and execution traces, and container platforms to dynamically scale sandboxes. General-purpose LLM platforms and cloud-native infrastructure—such as those offered by Baidu AI Cloud—can provide the compute and runtime environment needed for stable agent operation.

5. Conclusion

Building a code agent from scratch is well within reach. By understanding how LLMs reason, designing a sound system architecture, and implementing the core modules for perception, decision-making, execution, and feedback in Python, developers can create their own intelligent coding assistants. The process deepens your understanding of AI fundamentals and lays a solid engineering foundation for exploring advanced areas such as multi-agent collaboration and complex workflow automation. Combining theory with hands-on practice is the key competitive advantage in the coming wave of AI-driven programming.