Engineering Guide to LLM Code and Long-Horizon Agent Systems

    This guide explains how large models are evolving from code assistants into task-execution systems that can plan, use tools, run in restricted environments, validate results, and repair failures. It covers five core capability layers, multi-dimensional evaluation, engineering architecture, cost optimization, safety governance, cloud deployment, and practical steps for building reliable long-horizon agent systems.

    1. From Code Completion to Task Delivery

    As software development and office automation become more complex, large models are moving from answering code-related questions to acting as execution systems for engineering tasks. Early code models focused mainly on function-level completion, snippet generation, and simple explanation. Today, developers care more about whether a model can:

    • Understand repository context.
    • Call external tools.
    • Execute commands in restricted environments.
    • Repair results after failures.
    • Deliver verifiable outcomes.

    These capabilities can be summarized as code and long-horizon agent capabilities.

    A long-horizon agent does not simply produce a one-time answer. Instead, it plans around a goal through multiple steps: planning, tool invocation, result validation, error correction, and final summarization. The emphasis is no longer only on single-turn generation quality, but on closing the full task loop.

    For example, in a software repair task, the model needs to locate relevant files, understand call relationships, generate a patch, run tests, analyze failure logs, and continue adjusting based on test feedback. Similar “plan–execute–verify” loops are also common in parametric modeling, document analysis, and financial modeling.

    Therefore, engineering implementation should not focus only on whether the model can generate plausible code. It must also assess whether the model can deliver reliably under real constraints. This requires strong context understanding, tool invocation, error recovery, cost control, and safety boundary management.

    2. Key Capability Layers: Five Dimensions of Complex Task Agents

    2.1 Repository-Level Code Understanding and Generation

    Traditional code completion usually relies only on the current file or current function context. Engineering tasks, however, often involve cross-file dependencies, interface contracts, build configurations, and test pipelines.

    A code agent’s foundational capabilities should include:

    • Understanding repository structure, including module boundaries, dependencies, and entry files.
    • Supporting coordinated changes across multiple files, such as updating interface definitions, implementation logic, and unit tests together.
    • Interpreting build, test, and static-analysis results.
    • Controlling patch minimization and rollback capability.

    If a model can only generate code snippets but cannot understand how a change affects the overall system, it may cause a common problem in complex projects: locally correct changes that break the overall system.

    2.2 Task Completion in Terminal and Restricted Environments

    The terminal is an important environment for testing agent capability. Real development environments often include resource quotas, permission restrictions, network isolation, and timeouts. Models cannot assume that every command can run without limits.

    The ability to complete tasks in restricted terminal environments requires the model to:

    • Locate issues from error output rather than blindly retrying.
    • Choose appropriate commands or scripts under limited resources.
    • Roll back failed operations or switch to alternative paths.
    • Treat dangerous operations cautiously to avoid irreversible effects.

    The key capability here is not simply “knowing commands.” It is the ability to make stable progress in an uncertain environment.

    2.3 Tool Invocation and External Software Automation

    In scenarios such as 3D modeling, parametric design, office document processing, data analysis, and financial modeling, agents often need to call script interfaces, plugin mechanisms, or automation interfaces provided by host applications.

    At a general level, this capability includes:

    • Converting natural-language tasks into structured parameters.
    • Calling external programs to generate files, render results, validate outputs, or perform conversions.
    • Parsing output results and judging whether they meet expectations.
    • Adjusting parameters or switching execution paths when failures occur.

    For example, in a parametric modeling task, the model needs to generate executable scripts, set parameters such as dimensions, hierarchy, materials, or structural attributes, and verify whether the generated result satisfies geometric constraints.

    In office document scenarios, the model needs to understand tables, formulas, charts, and reference relationships, ensuring that the output is structurally complete and logically consistent.

    2.4 Document, Finance, and Knowledge-Intensive Workflows

    Beyond code tasks, long-horizon agents are also widely used in knowledge-intensive workflows such as document production, research analysis, financial modeling, and industry reporting.

    These tasks typically have the following characteristics:

    • Long output chains involving research, structural organization, calculation verification, and final drafting.
    • High requirements for factual accuracy, especially for data, citations, and conclusions.
    • The need to handle complex relationships among tables, formulas, text, and structured fields.
    • A frequent need for multi-round verification, such as checking calculation methods, verifying citation sources, and confirming conclusion consistency.

    Therefore, knowledge-intensive workflows should not rely only on single-shot generation. The model should be treated as an analysis and orchestration engine, combined with rule-based checks, external data queries, and human review to form a complete process.

    2.5 Long-Horizon Planning and Verification Loop

    The core of a long-horizon agent is the closed loop. A mature agent system usually includes the following stages:

    1. Task understanding and decomposition: Break the goal into executable steps.
    2. Plan generation: Define execution order, dependencies, and failure fallback strategies.
    3. Tool execution: Run commands, scripts, or service calls in a controlled environment.
    4. Result verification: Judge results through tests, rule checks, or secondary model evaluation.
    5. Plan repair: Modify the original plan based on verification results instead of repeating failed steps.
    6. Result summarization: Produce auditable and reusable final deliverables.

    The following generic pseudocode describes this loop:

    1. def execute_task(task, runtime):
    2. plan = runtime.planner(task)
    3. state = {"step": 0, "observations": []}
    4. while state["step"] < runtime.max_steps:
    5. step = plan[state["step"]]
    6. result = runtime.sandbox.run(step)
    7. state["observations"].append(result)
    8. if runtime.verifier.accept(result):
    9. return runtime.summarize(state)
    10. plan = runtime.planner.repair(task, state)
    11. state["step"] += 1
    12. return runtime.fallback(state)

    This code reflects an important principle: the value of an agent comes not only from generation, but also from verification and repair.

    3. Evaluation System Design: From a Single Score to Multi-Dimensional Acceptance

    When evaluating code and long-horizon agent capabilities, a single benchmark score is often insufficient to reflect real-world usability. A more reasonable approach is to build a multi-dimensional evaluation system that covers task types, execution environments, error recovery, cost consumption, and safety boundaries.

    3.1 Terminal-Restricted Task Evaluation

    Terminal tasks should be evaluated based on:

    • Command-generation accuracy.
    • Recovery ability after failures.
    • Strategy selection under resource limits.
    • Presence of dangerous operations.
    • Ability to complete tasks without full manual hints.

    Evaluation should not look only at final success. It should also record whether intermediate steps are reasonable, whether invalid loops occur, and whether the system frequently depends on human fallback.

    3.2 Long-Horizon Code Task Evaluation

    Long-horizon code tasks may include defect repair, refactoring, test completion, interface adaptation, and cross-module migration.

    Evaluation dimensions include:

    • Whether the modification scope is precise.
    • Whether additional regressions are introduced.
    • Whether compilation and tests pass.
    • Whether the code style follows repository conventions.
    • Whether the patch is easy to review and roll back.

    A high-quality long-horizon code agent should be able to explain the reason for a change, its impact scope, and the verification method, rather than simply outputting a large block of code.

    3.3 Parametric Modeling and External Tool Task Evaluation

    For tasks such as 3D modeling, design automation, and file generation, acceptance can be evaluated from the following perspectives:

    • Whether parameters are correctly parsed.
    • Whether scripts execute stably.
    • Whether output files match structural expectations.
    • Whether there are performance or resource issues.
    • Whether failures produce error messages that are easy to locate.

    The key to this type of evaluation is verifiable results. If an agent only generates scripts but cannot confirm the quality of the resulting artifacts, its engineering value remains limited.

    3.4 Knowledge-Intensive and Factual Accuracy Evaluation

    In document, finance, and research-analysis scenarios, factual accuracy and traceability should be emphasized. A layered verification mechanism is recommended:

    • Rule checks: Validate number formats, units, formulas, and field completeness.
    • Consistency checks: Confirm consistency across paragraphs, tables, and conclusions.
    • Source checks: Confirm whether citations exist and are relevant.
    • Stress testing: Deliberately construct cases that are easy to confuse, misquote, or over-infer.

    Factual evaluation should not only measure the rate of fully correct answers. It should also observe the proportion of answers containing errors, the distribution of error types, and whether errors are easy to detect.

    3.5 Safety and Compliance Evaluation

    Safety evaluation should cover:

    • High-risk content recognition.
    • Blocking of dangerous operations.
    • Protection against sensitive information leakage.
    • Misuse risk control for offensive and defensive security tasks.
    • Whether general capabilities drop significantly after filtering.

    An ideal safety strategy is not simple refusal. It should preserve normal engineering-task capability while maintaining safety boundaries.

    4. Engineering Architecture: Building a Usable Long-Horizon Agent System

    4.1 Control Layer: Task Orchestration and Policy Routing

    The control layer is responsible for task reception, policy selection, and workflow scheduling. It decides whether a task should be handled by a standard reasoning path or an accelerated path, whether a single model can execute it directly, whether multiple models need to collaborate, and whether external tools or human approval are required.

    Typical policies include:

    • Routing simple Q&A to a lightweight path.
    • Sending code generation to code review and test validation.
    • Requiring human confirmation for high-risk operations.
    • Placing long tasks into asynchronous queues.
    • Prioritizing caching and batch processing for cost-sensitive tasks.

    4.2 Inference Layer: Combining Standard and Accelerated Modes

    Developer-facing platforms often provide inference modes with different cost-latency tradeoffs. Standard modes are suitable for batch analysis, offline generation, and complex long-horizon tasks. Accelerated modes are suitable for interactive coding, real-time feedback, and low-latency scenarios.

    Engineering practice should avoid using a high-cost mode indiscriminately. A more reasonable approach is to:

    • Use the standard mode for initial planning.
    • Use the accelerated mode for key steps to improve interaction speed.
    • Reuse cached context where possible.
    • Control retry budgets for failed tasks.
    • Preserve a more complete verification chain for high-value tasks.

    4.3 Execution Layer: Sandbox, Isolation, and Rollback

    When agents execute external commands or scripts, they must run in isolated environments. The execution layer should provide:

    • File-system isolation.
    • Network-access control.
    • Resource quota limits.
    • Operation audit logs.
    • Failure rollback mechanisms.
    • Sensitive information masking.

    For example, in a code-repair task, the agent should not directly modify a production branch. It should generate a patch in a temporary workspace, run tests, and then submit the change.

    4.4 Observability Layer: Logs, Metrics, and Tracing

    Long-horizon tasks often involve many steps and long execution times, making observability especially important. It is recommended to record:

    • Each planned and executed step.
    • Tool-call parameters and return summaries.
    • Error locations.
    • Retry counts.
    • Cache-hit status.
    • Total latency and resource consumption.
    • Final verification results.

    Observability not only helps with troubleshooting but also provides a data foundation for future model optimization and evaluation-set construction.

    5. Cost Optimization: Caching, Tiered Reasoning, and Scenario-Based Scheduling

    Code and long-horizon agent tasks often involve long contexts and many model calls. Cost control is therefore one of the core challenges in engineering implementation.

    5.1 Caching for Long-Context Scenarios

    Many agent inputs are repetitive, such as system prompts, tool descriptions, repository conventions, historical summaries, and fixed templates. Context caching can reduce repeated computation and lower input costs.

    Common caching strategies include:

    • Caching system prompts and tool descriptions.
    • Caching stable task background.
    • Caching intermediate results at appropriate stages.
    • Reusing existing plans for similar tasks.
    • Extending cache lifetime for high-value content.

    However, caching is not always better when it is longer. Cached content should remain stable and structured, and should be cleaned periodically. Otherwise, it may cause context pollution or unnecessary cost.

    5.2 Output Costs Are Often Higher Than Input Costs

    In many pricing structures, output tokens are more expensive than input tokens. Therefore, optimization should not focus only on shortening inputs. It should also reduce invalid output.

    Ways to reduce output cost include:

    • Requiring the model to produce a brief plan before execution.
    • Using segmented validation for long-form generation.
    • Avoiding repeated summarization of the same content.
    • Limiting maximum output length for failed steps.
    • Compressing final answers into structured summaries.

    5.3 Combining Standard and Accelerated Versions

    Accelerated versions are usually designed for low-latency interaction, making them suitable for coding assistants, real-time debugging, and instant Q&A. Standard versions are more suitable for complex planning, batch processing, and offline analysis.

    A practical dual-track approach is:

    • Use accelerated capabilities during interactive phases to maintain response speed.
    • Switch to standard mode for deep tasks to preserve reasoning quality.
    • Place offline tasks into queues for unified scheduling.
    • Set budget limits for high-cost tasks.

    6. Safety Governance: Balancing Data Filtering and Capability Preservation

    In code and agent scenarios, safety risks mainly come from high-risk corpora, dangerous operation instructions, and potential abuse paths. A relatively mature approach is to perform data governance during model training and post-training, identifying and removing high-risk content to reduce the likelihood of model misuse.

    6.1 Key Principles of Data Filtering

    Data filtering should not pursue “filtering strength” alone. It must also pay attention to “capability preservation.” Overly aggressive filtering may make the model conservative, slow, or prone to false refusals in normal engineering tasks.

    Safety governance should therefore consider:

    • Whether high-risk categories are accurately identified.
    • Whether normal tasks are mistakenly blocked.
    • Whether the model can still complete routine development and office tasks after filtering.
    • Whether bypass paths exist.
    • Whether necessary safety prompts and human-confirmation mechanisms are retained.

    6.2 Before-and-After Evaluation After Filtering

    After safety filtering, two types of capabilities should be evaluated separately:

    • Safety capability: Whether the model can identify and reject high-risk requests.
    • General capability: Whether the model can still complete code generation, document processing, tool invocation, and long-horizon tasks.

    If general capability declines significantly, the filtering rules may be too broad, or there may be corpus-distribution bias.

    6.3 Application-Level Safety Boundaries

    In addition to model-level governance, application-level protection mechanisms are also necessary:

    • Requiring approval for high-risk operations.
    • Applying whitelists or sandbox restrictions to external command execution.
    • Enforcing access control for sensitive data.
    • Scanning output content for sensitive information.
    • Retaining audit logs for critical operations.

    7. Cloud Deployment: Composing General Infrastructure Capabilities

    In actual deployment of long-horizon agents, it is often unnecessary to build all infrastructure from scratch. General cloud capabilities can be used to build the system more efficiently.

    In related practices involving Baidu AI Cloud, a common approach is to combine general capabilities such as container platforms, object storage, message queues, logging services, and monitoring and alerting.

    7.1 Container Platforms for Execution and Inference Services

    Container platforms are suitable for deploying model services, sandbox executors, and task scheduling components. Their advantages include:

    • Elastic scaling.
    • Environment consistency.
    • Resource isolation.
    • Fast release and rollback.
    • Natural integration with monitoring systems.

    For long-running tasks, inference services and execution sandboxes can be deployed separately to prevent a single task from blocking the overall service.

    7.2 Object Storage for Code, Artifacts, and Logs

    Repository snapshots, build artifacts, test reports, generated files, model outputs, and audit logs can all be stored in object storage. This makes long-term retention easier and supports later analysis and traceability.

    7.3 Message Queues for Asynchronous Long-Running Tasks

    Long-horizon tasks often take a long time and are not well suited to synchronous waiting. Message queues can support:

    • Peak shaving and valley filling.
    • Failure retries.
    • Priority scheduling.
    • Asynchronous result notification.
    • Parallel processing of multiple tasks.

    7.4 Logging and Monitoring for Operability

    Agent systems often have long execution chains and are difficult to diagnose. Unified logging and monitoring can help identify:

    • Rising tool-call failure rates.
    • Increased average task latency.
    • Declining cache-hit rates.
    • Abnormal retry counts.
    • Increased false positives in safety blocking.

    A complete monitoring system helps teams identify bottlenecks from two dimensions: model effectiveness issues and system engineering issues.

    8. Best Practices for Making Code and Long-Horizon Agents Usable

    8.1 Build a Verifiable Task Set First

    Do not directly launch a complex agent system. Instead, construct a representative task set first, including:

    • Simple code repair.
    • Cross-file refactoring.
    • Test-failure repair.
    • Script generation and validation.
    • Structured document generation.
    • Data verification tasks.
    • High-risk operation recognition tasks.

    Continuous regression testing on this task set helps determine whether upgrades deliver real benefits.

    8.2 Enforce Verification Mechanisms

    Agents should not be allowed to “generate only without verification.” It is recommended to add the following at key checkpoints:

    • Compilation checks.
    • Unit tests.
    • Static analysis.
    • Format validation.
    • Result consistency checks.
    • Human spot checks.

    The more automated the verification mechanism, the more likely the agent can form a stable closed loop.

    8.3 Control Step Budgets and Retry Budgets

    Long-horizon tasks are prone to infinite loops. It is recommended to set:

    • Maximum number of steps.
    • Maximum retry count.
    • Timeout for a single step.
    • Total budget limit.
    • Failure degradation strategy.

    When the budget is exhausted, the system should output current progress, failure reasons, and suggested next steps, rather than failing silently.

    8.4 Preserve Human-Reviewable Intermediate Artifacts

    Agent outputs should not be limited to final answers. The system should also preserve:

    • Task plans.
    • Key step records.
    • Modification diffs.
    • Test logs.
    • Failure reasons.
    • Final acceptance summaries.

    This helps developers quickly judge whether a task is trustworthy and supports later auditing.

    8.5 Include Cost Metrics in Task Success Criteria

    Engineering implementation should not only ask “Can the task be completed?” It should also ask “Is completion economical?” It is recommended to track:

    • Input and output consumption.
    • Cache-hit ratio.
    • Average number of model calls.
    • Cost of failed retries.
    • Human intervention ratio.
    • Total task latency.

    Only when effectiveness, stability, and cost are measured together can an agent system have sustainable operational value.

    9. Conclusion

    Code and long-horizon agent capabilities represent an important direction in the evolution of large models from assistance and Q&A to task delivery. Truly valuable systems do not rely simply on model-generated text or code. Instead, they build a complete loop of planning, execution, verification, repair, and auditing around real tasks.

    In engineering implementation, teams should focus on five priorities:

    1. Build a multi-dimensional evaluation system to avoid misleading results from a single metric.
    2. Build an execution environment that is isolated, rollback-capable, and observable.
    3. Optimize cost through caching, tiered reasoning, and scenario-based scheduling.
    4. Maintain balance between safety filtering and general capability.
    5. Use general cloud infrastructure to improve system stability and operability.

    When these capabilities are systematically combined, large models are no longer merely code generators. They can become reliable task-execution assistants, helping developers and organizations turn complex workflows into verifiable, reusable, and continuously optimizable automation capabilities.