LLM Agents in Practice: Long Context, Task Decomposition, and Output Quality Control
This article explains why evaluating LLM agents comes down to system capability rather than raw model intelligence, and what it takes to make them deliver reliably in real workflows. It covers practical techniques for long-context tasks (retrieval, aggregation, entity consistency, numerical conservation, and explicit uncertainty), why long-horizon tasks break and how to counter it with contracts, checkpoints, and human-in-the-loop, hallucination control through a graph-extraction pipeline, multi-agent orchestration with contract-first design and runnable acceptance gates, command-line tool usability, cross-language inference-engine integration, cost control, and cloud capability reuse. It closes with a production checklist and the takeaway that the remaining weaknesses are long-horizon continuity and one-shot runnability, which engineering practices fix better than swapping models.
1. The Evaluation Target Has Shifted: From Model Capability to System Capability
LLM products today generally support long-context input, file understanding and authoring, and structured data analysis. Their integration surfaces have also expanded from a single chat interface to three forms: web, command-line tools, and programmable APIs. For developers and enterprise users, the real question is no longer “is a single turn of Q&A smart,” but “can the agent system reliably deliver across a complete workflow.”
The gap between these two questions shows up at three levels:
- Single-output correctness: Are the facts accurate, does the code run, and are conclusions traceable to sources?
- Multi-step task completability: When a task spans many turns or even days, can context and state stay consistent?
- Engineering usability of the output: Is the deliverable a “presentable fragment” or a “buildable, deployable, maintainable artifact”?
In everyday use, it is easy to confuse the first level with the third. Between code that “looks right” and code that “runs,” there is often a gap of environment dependencies, interface contracts, and boundary conditions. This is why an evaluation method must deliberately separate “generation quality” from “delivery quality.”
2. Long-Context Capability: Effective Context Is Far Smaller Than the Window
A long context window does not mean the whole window is used effectively. In engineering practice, it helps to design test cases across five dimensions, each with 5–10 fixed samples, forming a repeatable regression set:
| Dimension | Test method | Expected behavior |
|---|---|---|
| Key-info retrieval | Plant a unique parameter in the middle of a document of over 100,000 characters | Accurately restate that parameter |
| Cross-section aggregation | A timeline or metrics scattered across multiple places | Merge with no omissions and no conflicts |
| Entity consistency | The same character referred to by multiple names | Unify and merge into the same entity |
| Numerical conservation | Summing multiple rows in a table | Calculation matches the document’s basis |
| Uncertainty expression | The document genuinely lacks the information | Clearly answer “not provided,” not fabricate |
The last item is the easiest to overlook, yet it is the dividing line for reliability. A model willing to say “it’s not in the document” accumulates errors far more slowly over long tasks than a model that “always gives an answer.”
For combined tasks like “file understanding + content authoring + data analysis,” it helps to write the prompt as an explicit work order: what the input is, what the output structure is, and where the evidence to cite is located. A sample output constraint can be written as a JSON Schema fragment:
{"entities": [{"id": "e_001", "name": "", "aliases": [], "type": ""}],"relations": [{"from": "e_001", "to": "e_002", "type": "","evidence": "source excerpt or section location", "confidence": 0.0}],"unverified": [{"claim": "", "reason": "lacks source support"}]}
The key design choice is to make unverified a first-class output field. Having the model proactively hand over “the parts I’m unsure about” costs an order of magnitude less than having a person pick out errors one by one afterward.
3. Why Long-Horizon Tasks Break
Real engineering tasks often span hours or even days and need a person to add feedback in between to keep moving. This is not simply “insufficient capability,” but structural causes:
- Error compounding: At 95% per-step accuracy, 20 chained steps drop below 40% overall success.
- Context compression loss: After a long session is summarized, the details of early decisions (why this option was chosen) are erased, so later steps repeatedly overturn existing conclusions.
- Invisible environment state: Which files changed, how far the tests got, whether dependencies were installed — this external state is outside the model’s view.
- No self-check stage: After finishing, there is no build, no test, no smoke check, so errors only surface when a person takes over.
The corresponding fix is to move state out of the context and onto durable artifacts:
- Task contractization: Each phase has explicit inputs, outputs, and acceptance conditions.
- Checkpoint persistence: Write progress to a structured file so that after an interruption you can resume from the breakpoint instead of starting over.
- Human-in-the-loop (HITL): Turn “come back later and give feedback” from a defect into a design — explicitly pause before key milestones and let a person decide.
- Verifiable intermediate artifacts: At the end of each phase you must leave a checkable file, not a natural-language report.
A sample task state file:
{"task_id": "graph-2026-08-26","phase": "verify","milestones": [{"name": "schema", "status": "done", "artifact": "schema.json"},{"name": "extract", "status": "done", "artifact": "raw_relations.jsonl"},{"name": "normalize", "status": "done", "artifact": "entities.json"},{"name": "verify", "status": "running", "artifact": null}],"decisions": [{"at": "phase 2", "choice": "keep only binary edges between relations", "reason": "keeps visualization complexity manageable"}]}
The decisions field exists specifically to counter summary loss: recording “why we decided this” explicitly means the rationale survives context compression.
4. Hallucination Control: A Knowledge-Graph Extraction Example
Organizing the character relationships in a classical chapter-based novel is a typical example — the book has many characters linked into a network through shared roles, and in the first few rounds the model often returns relationships that are both incomplete and laced with fabricated edges. The reliable approach is to treat it as a data-processing pipeline, not a single conversation:
- Extract: Require every relationship to carry a location in the source text.
- Normalize: Build a character-alias dictionary; merge the multiple names for the same person into a single entity id.
- Merge and deduplicate: Compare programmatically, rather than letting the model judge “is this a duplicate.”
- Validate: Use code for graph-consistency checks; this step must be done by a deterministic program.
- Label: Any relationship that cannot be traced back to the source goes into
unverifiedand does not enter the main graph.
The validation logic itself is purely deterministic and suits a script rather than the model:
def validate(entities, relations):ids = {e["id"] for e in entities}problems = []# 1. Dangling edges: endpoints must exist in the entity tablefor r in relations:if r["from"] not in ids or r["to"] not in ids:problems.append(("dangling", r))# 2. Missing evidence: relations without source support must not enter the main graphif not r.get("evidence"):problems.append(("no_evidence", r))# 3. Self-loopif r["from"] == r["to"]:problems.append(("self_loop", r))# 4. Isolated-node hints: possibly missed edgesconnected = {r["from"] for r in relations} | {r["to"] for r in relations}for e in entities:if e["id"] not in connected:problems.append(("isolated", e["id"]))return problems
This reflects a general principle: anything a program can decide should not be left to the model’s self-assessment. Graph structural validity, numerical conservation, and field completeness all fall within deterministic checks.
Likewise, when building a site that organizes historical knowledge, if the model misses a key branch (for example, an entire clan or faction never appears), the root cause is usually that the prompt only gave a vague goal like “write an introduction,” instead of first asking it to produce an outline and then expand section by section. Changing it to “first output the table of contents, confirm it, then write the body” noticeably lowers the omission rate.
5. Multi-Agent Orchestration: Splitting Is Easy, Integration Is the Bottleneck
The idea of a parallel sub-agent cluster is appealing: a main agent decomposes the task and multiple sub-agents work at the same time. In practice, two failures are common:
- Output that won’t run: Each subtask’s code looks reasonable on its own, but when assembled the interfaces don’t match and dependency versions conflict.
- Long wait times: Serially dependent subtasks are wrongly parallelized, or agents wait on each other.
The cause almost always points to the same thing: missing interface contracts and mandatory integration-stage verification.
Contracts first
Before the main agent decomposes the task, first fix the data structures, function signatures, and configuration formats shared across subtasks as a common deliverable. Sub-agents are responsible only for the contract and do not invent their own interfaces.
# Produced by the main agent first, as a shared prerequisite for all subtasksCONTRACT = {"module_io": {"input": {"path": "str", "options": "dict"},"output": {"artifacts": "list[str]", "metrics": "dict"},"errors": ["ValidationError", "RuntimeError"],},"config_schema": "config.schema.json","dependency_lock": "versions are pinned, no floating ranges"}
Layered orchestration
Not every phase is worth parallelizing. A reasonable shape is “serial exploration + parallel implementation + serial integration”:
def orchestrate(task):plan = main_agent(task, mode="serial") # Serial: decompose and define contractscontracts = plan["contracts"]results = run_parallel( # Parallel: independent implementations[sub for sub in plan["subtasks"] if sub["depends_on"] == []],shared=contracts,)remaining = [sub for sub in plan["subtasks"] if sub["depends_on"]]for sub in remaining: # Serial: parts with dependenciesresults.append(run_subtask(sub, shared=contracts))return integrate_and_verify(results, contracts) # Serial: integration and acceptance
Runnable acceptance
A subtask’s definition of done should not be “generated code,” but “provided runnable evidence”: the build passes, unit tests pass, the smoke script produces the expected output, and logs show no fatal errors. Making these a gate that runs automatically when a subtask finishes intercepts most “won’t run” output before integration.
# Subtask completion gate examplebuild && unit-test && smoke-test --artifact-path "$ARTIFACT" || return_to_agent_with_logs
The return_to_agent_with_logs step matters: on failure, feeding the full logs back to the agent gives it a chance to self-repair; otherwise it will just make the same mistake a different way.
6. Usability Details of Command-Line Coding Tools
The command-line form is close to developers’ real working environment, but the usability pitfalls are concentrated and concrete:
Output rendering. Long text, diffs, and tables in a terminal easily get misaligned. You need to control line width, fold and truncate overly long output, and keep it readable in both light and dark themes. A rough interface is not a capability problem, but it directly affects willingness to use the tool and the efficiency of spotting errors.
Session isolation. Cross-session state bleed is a serious defect: an intermediate result from session A showing up in session B destroys trust in all output. Engineering-wise, you must ensure the session id maps one-to-one to a namespace, state directories are isolated per session, all writes are idempotent, and shared resources are locked. This should be a mandatory pre-release check.
Interruption and message injection. The ideal interruption model is tiered:
| Type | Behavior | Use case |
|---|---|---|
| Queued message | Consumed at the next turn after the current one ends | Adding requirements, extra constraints |
| Immediate interrupt | Terminate current inference and discard incomplete output | Direction is clearly wrong |
| Conditional injection | Continuously read messages while running, interrupt only when a condition is met | Mid-course correction, cost cutoff |
The third type is the most useful and the hardest to implement; the core is a lightweight event loop:
def agent_loop(session, policy):while not session.done:if policy.should_interrupt(session): # Poll for pending messages while runningsession.abort(reason=policy.reason())session.discard_partial()continuestep = session.next_step()if step.requires_human_decision: # Explicit pause at milestoneswait_for_human(session)session.execute(step)for msg in session.drain_inbox(): # Non-blocking fetch of queued messagesif msg.priority >= policy.threshold:session.abort(reason=msg.text)breaksession.enqueue_for_next_turn(msg)
A companion requirement is observability: log per-round token usage, latency, and tool-call counts, otherwise when a “subtask takes a long time” you cannot tell whether it is queuing, network, or the model itself being slow.
7. Cross-Language Engineering: An Inference-Engine Integration Example
One class of tasks is especially unfriendly to agents: connecting a call chain between a high-performance compiled language and an interpreted language, while producing a distributable package. Typical steps are compiling the inference engine, designing a cross-language binding layer, wrapping an API in the higher-level scripting language, and finally packaging it as an installable artifact. The difficulty is not writing code, but these error-prone areas:
- Memory ownership: Cross-language boundaries most easily produce dangling pointers and double frees. The binding layer should wrap native resources with RAII and fix the release responsibility at the creator.
- Exception and error-code mapping: Errors from the native layer must be fully mapped to the upper layer, including error codes, context, and readable messages, not all collapsed into “unknown error.”
- Thread model: The upper-layer runtime may hold a global lock; calling a blocking native function while holding the lock causes deadlock. You must explicitly release the lock before entering the native call and reacquire it after returning.
- Tensor and buffer layout: Shape, data type, and memory layout (row-major/column-major, alignment requirements) must be explicitly checked at the boundary; silent conversion produces numerical errors that are hard to locate.
- Reproducible builds: Pin dependency versions, fix compiler options, record artifact hashes, and run consistency checks in CI. Otherwise it is normal for the same source to compile into libraries with different behavior on two machines.
A minimal resource wrapper:
// Use RAII to make ownership explicit and avoid double frees across the language boundaryclass Session {public:explicit Session(const Config& cfg) : handle_(engine_create(&cfg)) {if (!handle_) throw EngineError(engine_last_error());}~Session() { if (handle_) engine_destroy(handle_); }Session(const Session&) = delete; // Disallow copying to prevent double freeSession& operator=(const Session&) = delete;Session(Session&& o) noexcept : handle_(o.handle_) { o.handle_ = nullptr; }Buffer run(const Tensor& input) {Buffer out;// Release the upper-layer global lock before entering the blocking native callScopedLockRelease release;int rc = engine_infer(handle_, input.raw(), &out.raw());if (rc != 0) throw EngineError(engine_last_error());return out;}private:EngineHandle handle_;};
Instructions to the agent should be explicit that “the deliverable is an installable package + passing test cases + a minimal runnable example,” not “complete the wrapping of the inference pipeline.” Writing the delivery standard clearly makes a real difference in output usability.
8. Usage and Cost Control
The cost structure of agent tasks differs from traditional calls: one multi-step task often produces dozens of model calls, and retries and rework amplify this. Practical general measures include:
- Budget caps: Set token and step limits per task; pause and request a human decision when exceeded, avoiding infinite consumption on a wrong path.
- Tiered routing: Use lightweight models for exploration, chat, and format conversion; use high-capability models for architecture design and difficult debugging.
- Context reuse: Cache stable system prompts and knowledge documents; repeated requests transmit only the delta.
- Parallelism control: Match the number of concurrent sub-agents to the quota so that the budget is not exhausted before integration.
- Failure classification: Categorize failures as “model capability,” “contract,” and “environment,” and decide the optimization direction by proportion. Most teams’ rework is actually concentrated in contracts and environment.
Insufficient quota is itself a quality problem — when the budget only allows two attempts, any “generated but won’t run” output is unacceptable, and here the value of quality gates far exceeds that of speed.
9. Connecting to Cloud Capabilities
When productizing an agent system, you can reuse the general capability categories that cloud platforms offer:
- Object storage: Host long documents, intermediate artifacts, and model-generated visualization results.
- Message queue: Carry async tasks and the interrupt-instruction channel, supporting the three-tier “queued message / immediate interrupt / conditional injection” mechanism.
- Container platform: Provide an isolated sandbox for agent-generated code and control its access to system resources.
- Log service and monitoring/alerting: Collect per-call chains, latency, and failure reasons, supporting cost accounting and anomaly alerting.
- Model service and gateway: Unified access, rate limiting, and routing, making it easy to switch between models of different capability levels.
In these scenarios, Baidu AI Cloud provides exactly the general capability categories above, and teams can combine them as needed rather than building sandbox, queue, and observability infrastructure themselves.
10. Production Checklist
- Build a fixed evaluation set covering long-text retrieval, cross-section aggregation, numerical conservation, and refusal-to-answer cases.
- Make uncertainty explicit: require the model to output an
unverifiedfield instead of forcing an answer. - For long-horizon tasks, adopt “contract + checkpoint + human-in-the-loop,” with progress persisted and resumable.
- Leave deterministic checks to code: graph consistency, numerical conservation, field completeness.
- For generation tasks, produce the outline or schema first, confirm it, then expand the body.
- Multi-agent orchestration follows “serial exploration, parallel implementation, serial integration,” with shared contracts defined first.
- A subtask’s definition of done includes runnable evidence: build passes, unit tests pass, smoke test passes.
- Command-line tools must guarantee session isolation and idempotent writes, and support tiered interruption.
- For cross-language engineering, make memory ownership, exception mapping, lock release, and reproducible builds explicit.
- Set a budget cap and failure-classification statistics for each task, and use data rather than intuition to decide the optimization direction.
Overall, mainstream LLMs today are at a level of “safe to do real work and unlikely to make obvious mistakes” on most routine development tasks. The weaknesses are concentrated in long-horizon task continuity and the one-shot runnability of multi-agent output. Neither is solved by simply swapping models; both must be filled in with engineering methods — task decomposition, contract design, verification gates, and externalized state. Treating an agent as a collaborator that needs explicit delivery standards, checkpoints, and feedback loops, rather than an all-purpose Q&A interface, is currently the highest-value way to use it.
