LLM Selection and Deployment in Coding Plans: Engineering Practices

    This article explains how to build a Coding Plan, a decision system for selecting, evaluating, routing, and deploying large language models across software development tasks. It maps tasks such as code generation, code review, test repair, repository-level changes, and tool use to capability tiers, cost tiers, and risk controls. Readers will learn how to profile tasks, run weighted evaluations on layered test sets, route requests by tier, control permissions and audit agent actions, and roll out changes with canary releases, monitoring, and rollback.

    As model iteration cycles keep getting shorter and coding tasks keep diversifying, teams do not need a static leaderboard that simply says which model is “stronger.” They need an executable, reproducible, and auditable Coding Plan. A Coding Plan is an engineering decision system. It maps tasks such as code generation, code review, test repair, repository-level changes, tool calls, and multimodal content generation onto different capability tiers, cost tiers, and risk control strategies. This article does not focus on any specific product. It starts from general technical capabilities and engineering practice, and explains how to complete model selection, evaluation, routing, launch, and continuous optimization.

    1. Define Task Profiles Before Reading Leaderboards

    The first step in a Coding Plan is not comparing models. It is defining the tasks. Different coding tasks place very different demands on a model. Single-line completion depends mostly on low latency and local context consistency. Single-file generation depends on syntactic correctness and readability. Cross-repository changes depend on long-context understanding, dependency analysis, and judgment of the change’s impact. Test repair depends on locating errors, reading failure logs, and verifying regressions. Agent-style tasks depend on tool calls, task decomposition, state management, and permission boundaries. If these tasks are averaged into a single score, the comparison will usually mislead.

    Teams should therefore start with a task profile table. The table should record the task type, input size, context sources, whether tool calls are allowed, the cost of failure, the latency budget, the cost ceiling, and compliance requirements. The clearer the task profile, the easier it is to put evaluation sets and routing rules into practice. For tasks involving 3D content generation or multimodal assets, teams should also record spatial consistency, asset format compatibility, reproducibility, and the share of outputs that need manual review. Anonymous canary builds and early versions should go through blind evaluation before they receive any production traffic.

    2. Selection Criteria: Six Hard Metrics and Three Soft Constraints

    The hard metrics are correctness, context utilization, tool-call reliability, latency and throughput, cost, and stability. Correctness is not only whether the model produces code. The question is whether the code passes tests, whether it introduces regressions, and whether it follows project conventions. Context utilization covers long files, cross-file dependencies, and tolerance for noise after retrieval augmentation. Tool-call evaluation covers parameter validity, retry behavior after failures, least-privilege permissions, and audit completeness. For latency and throughput, measure time to first token, full response time, and tail latency under concurrency separately. Cost should be calculated per task, not per call price alone. Stability covers output format consistency, completion rate for long tasks, and recovery from abnormal states.

    The soft constraints are security and compliance, observability, and vendor replaceability. Security and compliance require defined boundaries for code, logs, and prompts during transmission, storage, retrieval, and auditing. Observability requires recording the task ID, a summary of inputs, tool calls, approval results, and final artifacts. Replaceability requires that evaluation sets, routing rules, and prompt templates stay decoupled from any specific model, so that hidden dependencies do not make migration difficult. Weights should not be set arbitrarily. Calibrate them against business risk. High-risk repository-level changes should give more weight to correctness and security. Low-risk completions should give more weight to latency and cost.

    3. Build Tiered Evaluation Sets for Reproducible Results

    Evaluation sets should be divided into four layers: a golden set, a regression set, an adversarial set, and an online shadow set. The golden set contains high-quality tasks confirmed by humans and is used for stable capability comparison. The regression set covers historical defects and frequently changing scenarios, so that version upgrades do not introduce degradation. The adversarial set includes ambiguous requirements, noisy context, misleading error logs, and unauthorized tool calls, and tests robustness. The online shadow set collects the real task distribution without affecting users, and is used to calibrate offline conclusions. Each task should be linked to an acceptance method, such as unit tests, static checks, human scoring, or rule validation.

    The pseudocode below shows the idea. The weights are for illustration only and must be calibrated to the business.

    1. weights = {
    2. correctness: 0.30,
    3. repo_context: 0.15,
    4. tool_use: 0.15,
    5. latency: 0.10,
    6. cost: 0.10,
    7. stability: 0.10,
    8. security: 0.10
    9. }
    10. function evaluate(candidate, task_set):
    11. results = []
    12. for task in task_set:
    13. output = candidate.run(task.input, task.context, task.tools)
    14. metrics = {
    15. correctness: run_tests(output, task.tests),
    16. repo_context: judge_context_use(output, task.references),
    17. tool_use: validate_tool_calls(output.calls, task.policy),
    18. latency: measure_latency(output),
    19. cost: estimate_cost(output.usage),
    20. stability: check_format_and_retry(output),
    21. security: check_permission_and_leakage(output)
    22. }
    23. results.append(weighted_sum(metrics, weights))
    24. return aggregate(results)

    Evaluation reports should include the mean, the variance, failure examples, and manual review conclusions. A report that gives only an average score can hide long-tail failures, and production risk in coding tasks tends to concentrate in that tail.

    4. Replace Single-Model Binding with a Decision Matrix

    The output of model selection should not be “choose one model.” It should be “choose a capability tier for each task.” We recommend a combined strategy with four tiers: light, balanced, flagship, and high-speed. The light tier handles low-risk completion and formatting. The balanced tier handles routine generation and test repair. The flagship tier handles cross-repository changes and high-risk reviews. The high-speed tier serves interactive scenarios. The decision matrix should include task type, risk level, context size, tool permissions, latency budget, and cost ceiling, and it should keep manual approval checkpoints.

    Routing can be expressed as follows:

    1. function route(task):
    2. if task.risk == "high" or task.scope == "repo":
    3. return "flagship_with_review"
    4. if task.complexity <= 2 and task.latency_budget_ms < 800:
    5. return "light_fast"
    6. return "balanced"
    7. function execute(task):
    8. tier = route(task)
    9. plan = build_plan(task, tier)
    10. result = sandbox_run(plan)
    11. if result.failed:
    12. result = escalate(task, result)
    13. return review_and_audit(result)

    The thresholds above are examples and must be calibrated against real traffic. The value of routing is that it separates cost, latency, and risk into layers. It avoids sending every task to the highest capability tier.

    5. Tool Calls and Agent Workflows: Permissions, Sandboxing, and Auditing

    The main risk of a coding agent comes from what it can do, not only from whether its code is correct. Tool calls should follow the principle of least privilege. Reading, writing, executing, and network access should each be authorized separately, and high-risk operations should require approval. Every call should record the task ID, a summary of parameters, the approval result, and an artifact hash. Sandboxed execution should restrict file system access, network access, and environment variables. After a failure, the task should move to an isolated area rather than being retried directly. Audit logs should be traceable and searchable, and they should not store sensitive data in plaintext.

    1. call = {
    2. task_id: "...",
    3. tool: "read_or_write_or_exec",
    4. args_hash: "...",
    5. approval: "required_or_granted",
    6. result: "success_or_failed"
    7. }
    8. audit_log(call)

    For multi-step agents, task decomposition, tool selection, result validation, and rollback strategy should be part of the workflow itself, not left to a single prompt. Each step should produce verifiable intermediate artifacts. Otherwise, work that appears complete may not be possible to verify through regression testing.

    6. Cost, Latency, and Context Engineering

    Cost optimization cannot rely only on switching to cheaper tiers. More effective methods include context compression, retrieval augmentation, cache reuse, batching, and tiered routing. Context engineering should separate information into three types: content that must stay verbatim, content that can be summarized, and content that can be retrieved when needed. This prevents the entire repository from being added to the prompt indiscriminately. Latency optimization should focus on tail latency and retry amplification, using timeouts, graceful degradation, and circuit breakers. Evaluation and inference services can run on general-purpose cloud infrastructure, such as the container, object storage, logging, and monitoring and alerting services offered by Baidu AI Cloud. These services can isolate tasks, store evaluation artifacts, and keep traceable records.

    7. Evaluate 3D and Multimodal Tasks Separately

    When a Coding Plan covers 3D scene scripting, asset processing, and spatial layout generation, evaluation cannot look only at text correctness. Add metrics for spatial consistency, units and coordinate systems, completeness of asset references, reproducibility, and the cost of manual review. These tasks suit a generate-then-validate pipeline. The model first produces a structured description, rule-based checks run next, and manual spot checks follow. Do not replace geometry-level and asset-level acceptance with plain text scoring.

    8. Rollout Path: Canary Release, Monitoring, Rollback, and Continuous Evaluation

    Launches should follow four stages: shadow traffic, small-traffic canary, per-task ramp-up, and full rollout. Monitoring should track task completion rate, test pass rate, manual rework rate, tail latency, cost per task, tool-call failure rate, and security interception rate. Every change to a model tier or prompt template should trigger evaluation on the regression set, and one-click rollback should remain available. Review failure cases regularly and add new defects to the regression set, so that the process forms a closed loop.

    9. Common Pitfalls and Checklist

    Common pitfalls include: relying on public leaderboards and ignoring private tasks; using averages that hide long-tail failures; giving high-risk tasks to agents without approval; ignoring context cost and tail latency; letting evaluation sets drift