Why AI Coding Assistants Re-read the Full Work Log Every 30 Seconds

In intelligent coding systems that support background tasks, users often see brief status messages such as “Running tests,” “Analyzing files,” or “Waiting for execution results.” On the surface, these messages contain only a few words and seem unlikely to create significant compute overhead. In some implementations, however, the system may periodically send the background assistant’s complete work record back to the model just to generate that short status line.

This creates a technical pattern worth examining: the output is very short, but the input can be very long; the task itself may not have changed, yet the system still sends requests at regular intervals; users see only a status line, but they may not know how much context and inference resource it consumes behind the scenes.

1. The Core Issue: Short Output Does Not Mean Low Cost

The cost and compute load of an intelligent model call usually depend on input length, output length, model type, call frequency, and cache hit rate. Many people tend to judge whether a request is expensive by looking at the number of output words. In status-summary scenarios, however, input often accounts for the main share.

A single status-generation request may include:

• The background assistant’s complete prior conversation record;
• Tool calls that have already been executed;
• Command output returned by tools;
• File reads, modifications, or search results;
• Parameter definitions and invocation constraints for available tools;
• System instructions that guide the model to generate a summary;
• The current task objective and runtime environment information.

The final output may be only a few words such as “Running tests” or “Checking configuration file,” but the model must first understand the entire context before it can determine what happened most recently. These requests are therefore less like generating a few words and more like “reading a continuously growing work log and then writing a one-line summary at the top.”

A simplified relationship can be expressed as:

Single status request cost
≈ input context length × input unit price
+ output length × output unit price

In a real system, another layer should be considered:

Total cost
≈ number of calls × effective input per call × model price × cache-miss ratio

If the ratio of output to input is close to one in several thousand, most of the system’s resources are being spent on “re-reading history,” not on generating the status text.

2. Why Systems Check Every Few Tens of Seconds

From a product-experience perspective, periodic checking is easy to implement and offers good compatibility. A background task may execute commands, access files, wait for external processes, or perform multiple rounds of reasoning. The front-end interface needs to know whether the task is still running, whether it has stalled, and what happened recently.

The most direct implementation is timed polling:

1. The front end or orchestrator starts a timer;
2. When the interval is reached, it checks whether the background assistant is still active;
3. It reads the assistant’s work record;
4. It calls the model to generate a short status;
5. It sends the status to the user interface;
6. It waits for the next check.

This design has several clear advantages:

• It is simple to implement and does not require the underlying task to support a complex event protocol;
• It has good compatibility with different tools and execution environments;
• Even if the background task does not actively report progress, the system can infer progress from logs;
• It can provide continuous feedback during long-running tasks;
• It is easy to connect to an existing model-call pipeline.

But it also has an inherent flaw: the polling frequency is not tied to the rate at which the task changes. A task may produce no new action for tens of seconds, yet the system still triggers a check on schedule. If every check reads the complete history, cost accumulates with both time and context length.

3. Why Full-Context Mode Causes Repeated Reads

Early implementations often treat the background assistant’s work record as all the material the model needs for analysis. Each time a status is generated, the summarizer receives a new copy of the context.

Suppose a background task has run for a long time and its record contains:

• Multiple rounds of user objectives and model plans;
• Multiple command executions;
• Large amounts of compilation or test output;
• Several file fragments;
• Tool-call parameters and returned results;
• Error messages and the subsequent repair process.

As the task progresses, the work record keeps growing. If the system triggers a summary about every 30 seconds and passes the complete record from the beginning each time, the input size of a single call may continue to expand. Even if the summary content is always only three to five words, input consumption still keeps increasing.

More importantly, the system often includes tool descriptions in the request as well. Tool descriptions include names, parameter structures, parameter meanings, execution constraints, and return formats. For a background assistant that needs to call multiple tools, this content becomes a fixed overhead in every round of requests.

Full-context mode can therefore be abstracted as:

Round 1: read history H1
Round 2: read history H2
Round 3: read history H3
...
Round n: read history Hn

Here, Hn is usually larger than Hn-1. Without effective caching or an incremental mechanism, the system is effectively paying repeatedly for historical information that has already been processed.

4. Why Different Measurements Can Differ Greatly

When analyzing this type of problem, it is common to see situations where “one team measures a high share for status summaries, while overall telemetry shows a low share.” This does not necessarily mean that one side’s data is wrong. The key is that the measurement scope may be different.

At least the following scopes need to be distinguished:

1. A Single Team vs. All Users

For teams that run background tasks frequently and use complex models for long periods, status-summary calls may account for a high proportion. Across all users, many may not enable background assistants, or their tasks may be short-lived, so the overall proportion is diluted.

2. A Single Task vs. All Requests

If only a certain type of background task is observed, the share of status requests may be significantly higher than in the whole system. Global statistics usually also include a large number of requests for main conversations, code generation, file analysis, error repair, and other operations.

3. Estimated Spend vs. Request Count

The cost of one long-context request may be far higher than that of one short request. Counting only call volume cannot accurately reflect cost; counting only spend may also be affected by different model prices and caching strategies.

4. Whether Caching Is Included

If part of the historical context hits the cache, the actual billing or compute load may be lower than the raw input length suggests. Conversely, if the cache misses, the overhead of a complete re-read is exposed directly.

Therefore, when evaluating a status-summary system, at least the following metrics should be recorded together:

• Number of requests;
• Number of input tokens;
• Number of output tokens;
• Cache hit rate;
• Model tier used;
• Cumulative consumption per task;
• Share of status requests in total consumption;
• Share of rounds with no change.

Only when these metrics are placed within the same measurement scope is it possible to explain the differences accurately.

5. The Most Critical Optimization: From “Rereading the Full Text” to “Reading Increments”

The core solution to this problem is not simply changing the polling interval from 30 seconds to 60 seconds. It is reducing the material that must be reprocessed in each round.

1. Store the Last Summary Check Position

The system can maintain a checkpoint for each background assistant, recording the log position that had been processed when the last summary was generated. The next round reads only new content instead of rereading the entire history.

Last processed position: offset = 12000
Current log position: offset = 14500
Input for this round: new content from 12000 to 14500

If the new content is small, the model input drops significantly.

2. Use Rolling Summaries

When the history reaches a certain length, the system can first generate a structured stage summary, for example:

Task objective: fix test failures
Current stage: running regression tests
Completed: modified configuration file, rebuilt
Pending: locate two failing test cases
Latest error: dependency version mismatch

Subsequent requests only need to carry this short summary and the latest events. This preserves the global task information while avoiding repeated transmission of the raw log.

3. Separate Status Detection from Full Reasoning

A status line does not necessarily require a strong model or full context. Rules can be used first to determine:

• Whether there has been a recent tool call;
• Whether a command start or end event has appeared;
• Whether an error has occurred;
• Whether there has been no output for a long time;
• Whether the task has entered a waiting state.

Only when the rules cannot determine the answer should the model be called for semantic summarization. This “rules first, model as fallback” architecture can eliminate many low-value requests.

4. Trigger by Events Instead of Fixed Polling

If the underlying executor can emit events, the status can be updated at the following moments:

• A tool starts executing;
• A tool finishes executing;
• A new file is read or modified;
• A test starts or ends;
• An exception occurs;
• The background task enters a waiting state;
• The task completes or fails.

Event-driven updates match real changes better than fixed timers. When there are no new events, there is no need to generate a new status.

6. Model Choice Also Affects Actual Overhead

Status-summary tasks usually do not require full complex reasoning capability. Their main job is to identify the most recent action and compress it into a few words. The system can therefore consider a tiered strategy:

• Simple events are handled by local rules or a lightweight model;
• Ordinary status uses a lower-cost model;
• A high-capability model is used only when there is ambiguity, an anomaly, or a judgment spanning multiple rounds of context;
• Deterministic states such as task completion or failure are reported directly by the executor.

It is important not to look only at the model name or per-call price. Input length, latency, stability, and cache support also matter. If a cheaper model still reads an extremely long context every time, the overall cost may remain high.

7. Why Users Should See This Overhead

Although status messages are short, they can create continuous background consumption. If the interface does not show related information at all, users will find it difficult to determine where the cost comes from, and they cannot decide whether to disable high-frequency status updates.

A more transparent design can provide the following information:

• Whether background status summaries are enabled;
• The current task’s status-check frequency;
• The number of summary requests generated in the recent period;
• The input and output scale consumed by status summaries;
• Whether full context is being used;
• Cache hit status;
• Cumulative resource consumption per task;
• Whether the user can switch to a low-frequency or event-driven mode.

Not all of this information needs to be shown to ordinary users, but at least it should be queryable in advanced settings, task details, or a usage dashboard.

8. How to Estimate the Upper Bound of High-Frequency Status Checks

In legacy polling designs, a rough formula can first be used to estimate input pressure:

Input per minute
≈ checks per minute
× number of active background assistants
× context length per assistant

If the check interval is about 30 seconds, each active assistant triggers roughly two checks per minute. If multiple background assistants are running at the same time and each assistant’s context keeps growing, input volume can increase quickly.

When estimating total consumption further, the following should also be included:

Total consumption
≈ input per minute
× active minutes
× input price
× cache-miss ratio

This formula is not a precise billing formula, but it is suitable for capacity planning and risk alerts. In enterprise environments in particular, combinations of “long tasks, multiple concurrent assistants, high-capability models, and low cache hit rate” should be evaluated together.

9. A More Reasonable Implementation Architecture

A more mature status-summary system can be divided into five layers:

Layer 1: Event Collection

Record tool calls, file changes, command output, errors, and task lifecycle events.

Layer 2: Change Detection

Determine whether meaningful new events have appeared since the last status update. If nothing has changed, skip the update.

Layer 3: Context Compression

Organize historical events into a stage summary, keeping only key information such as objectives, progress, blockers, and recent actions.

Layer 4: Status Generation

Prefer rules and templates; when the answer cannot be determined, call the model to generate a brief description.

Layer 5: Cost and Observability

Record request count, context length, cache hit rate, model type, latency, and failure rate, and support querying by task, user, and time period.

The core idea of this design is that status updates should revolve around “what changed,” not around “how much time has passed since the last check.”

Conclusion

Rereading the complete work log every few tens of seconds to generate a few words of status is not a problem of model output being too long. It is a problem of poor context organization. It assigns work that should be handled by event systems, incremental logs, and lightweight rules to expensive full-context inference.

This design has some justification in early products: it is fast to implement, highly compatible, and does not require modifying the underlying execution pipeline. As the number of background tasks grows, logs become longer, and model capabilities improve, however, the resource waste caused by repeated reads becomes increasingly visible.

A better direction should be:

Full log rereading
→ incremental event reading
→ reuse of stage summaries
→ rules-first detection
→ model handling of complex cases
→ transparent cost information for users

For developers, judging whether a status system is efficient should not depend only on how short the status text is. It should track how much context is read each time, how often it is triggered, whether anything actually changed, and whether users can clearly perceive these background requests.