Using Ultra-Long Context in AI-Assisted Coding Projects

Introduction

Context windows in AI coding tools keep growing. After jumping from hundreds of thousands of tokens to the million-token range, developers can send a large share of a mid-sized codebase in a single request. A bigger window is convenient, but it also creates new problems: more information does not equal better understanding, cost and latency rise with input size, and models are more likely to drop constraints over a long session. This article focuses on practical engineering practices for using ultra-long context on real projects.

1. Know the limits of ultra-long context

1.1 Fitting in the window is not the same as using it well

A context window only measures how much input the model can receive. It does not guarantee that every segment is used with the same accuracy. A common observation is that material in the middle of a long prompt is recalled and reasoned over less reliably than content at the beginning or the end. Put critical constraints at the start or end of the task description instead of burying them in a large dump of files.

1.2 Cost and latency grow with input size

Input tokens drive both price and response time. In multi-turn chats, if full history is resent every turn, cost accumulates quickly. Estimate spend as input tokens per turn multiplied by the number of turns, not as a single request.

1.3 Density matters more than volume

Dumping an entire repository usually includes many unrelated files and dilutes the code that actually matters. Select material by relevance to the current task.

2. Prepare the codebase before you feed it

2.1 Organize material in layers

Prepare inputs in this order:

• Project overview: business goal, stack, and directory layout.
• Architecture conventions: layering, naming, and error handling.
• Interface contracts: public APIs, data structures, and key configuration.
• Core modules for the current task, plus their tests.

2.2 Filter unrelated content

Exclude dependency directories, build artifacts, logs, binaries, and secret files. Example filter:

EXCLUDE_DIRS = {"node_modules", "dist", "build", ".cache", "logs"}
EXCLUDE_SUFFIXES = {".png", ".jpg", ".zip", ".pyc", ".lock"}
SECRET_PATTERNS = {".env", "secret", "credential", "private_key"}
def should_include(path, size_bytes):
    parts = path.split("/")
    if any(p in EXCLUDE_DIRS for p in parts):
        return False
    if any(path.endswith(s) for s in EXCLUDE_SUFFIXES):
        return False
    if any(k in path.lower() for k in SECRET_PATTERNS):
        return False
    return size_bytes <= 200 * 1024

Adjust the rules per project. Always exclude files that may contain secrets, even if they are small.

2.3 Estimate token size

Chinese, English, and source code tokenize at different rates. Roughly, Chinese text often maps to about one to two characters per token; code varies more because of symbols and indentation. Before a large feed, measure with the provider’s tokenizer or API rather than guessing from character counts.

3. Control long tasks with checkpoints

3.1 Freeze state at key points

In a long session, early decisions can fade as context is compressed. The reliable approach is to write those decisions into project files, not only into the chat. Create checkpoints after:

• Requirements and design are confirmed;
• Interface definitions are frozen;
• Each module’s tests pass;
• You are about to start a new session.

Each checkpoint should list completed work, remaining work, key decisions and reasons, and known issues.

3.2 Checkpoint template

## Checkpoint #3
- Current goal: order-module refund flow
- Done: refund API, state machine, unit tests (12/12 passing)
- Not done: integration with payment callbacks
- Key decisions:
  1. Refund states only move one way, to avoid duplicate refunds
  2. Amounts are stored as integers in the smallest currency unit
- Known issues: retry policy after callback timeout is still open

3.3 Treat auto-compression with caution

Some tools summarize older turns when context nears the limit. Auto-summaries often keep the gist but drop exact numbers, edge cases, and rejected options. Treat checkpoint files as the source of truth for that detail. If auto-compression is on, re-check critical constraints before and after a compression step.

4. External tools and token cost

4.1 Tool descriptions consume context too

When you attach tools such as docs search, database access, or a browser, names, parameter schemas, and return payloads all enter the context. More tools leave less room for code and can make tool selection less accurate.

Practical steps:

• Enable only tools needed for the current task;
• Merge similar tools to cut duplicate descriptions;
• Trim return values to required fields and the first few lines.

4.2 Trimming return values

def trim_result(text, max_lines=50, max_chars=4000):
    lines = text.splitlines()[:max_lines]
    trimmed = "\n".join(lines)
    if len(trimmed) > max_chars:
        trimmed = trimmed[:max_chars] + "\n...[truncated]"
    return trimmed

Truncation discards information, so use it only when needed and keep the fields the current task actually requires.