How AI Coding Assistants Design System Prompts and Context Injection

1. From a Fixed Block of Text to Layered Assembly

An early approach was to hard-code one long instruction and send it unchanged when the model started. It is simple to implement, but it creates several problems. When tools are added or removed, the description drifts from the capabilities that are actually available. When the project environment changes, information written into the prompt quickly goes stale. After context is compressed in a long session, important constraints can disappear with it.

A more practical approach is to split the system prompt into two layers.

• Static layer: session-independent behavior rules, such as tool-selection principles, output style, and safety limits. It ships with a release, changes infrequently, and is a good fit for version control and review.
• Dynamic layer: content tied to the current session, such as descriptions of enabled tools, instructions for external services, a skill index, and runtime environment details. It is generated when the session starts or when state changes.

Keeping the layers separate leaves the static part stable and easier to test. The dynamic part keeps what the model sees aligned with the real runtime state.

2. The Static Layer: Four Kinds of Behavior Constraints

2.1 Prefer Specialized Tools

An agent often has both a general command-line capability and several specialized tools. Reading a file, for example, can be done by running a shell command or by calling a dedicated read tool. The prompt should explicitly prefer specialized tools, for three reasons.

1. Permission control: a specialized tool can check path scope and read/write attributes before the call. A general command is hard to constrain at that granularity.
2. Structured results: a specialized tool returns a consistent format, so the model parses it more reliably and is less likely to misread inconsistent output.
3. Observability: the arguments and results of a specialized tool can be recorded in full, which makes later review easier.

A priority rule can look like this:

Tool selection priority (example)
1. Find a file path        -> use a file-matching tool
2. Search file contents    -> use a content-search tool
3. Read file contents      -> use a file-read tool
4. Use a shell command only when none of the tools above can do the job

2.2 Output Style: Cut the Noise

Output-style rules exist to reduce wasted tokens. Typical requirements include answering directly, skipping emoji, skipping empty greetings and acknowledgements, and not restating what the user already said.

These rules look minor, but in coding workflows that call the model often, extra wording on every reply adds up to a real context cost. Developers can also find the important information faster. Brevity means removing filler, not dropping necessary explanation. When the task involves a risk judgment or the reason for a change, the model should still explain.

2.3 Memory Files: Discovery Order and Write Rules

Project memory files store long-lived information such as team conventions, build commands, and code style. A common discovery order is the user-level directory, then the project root, then the current subdirectory. Rules in a subdirectory apply only when working on files under that directory.

The design needs to make three things explicit.

• Load order and override rules: the closer a rule is to the current file, the higher its priority.
• Write rules: keep entries short and actionable, for example “the test command is …”, rather than long narrative.
• Size limits: memory content consumes the context window, so there needs to be a truncation or summarization strategy.

2.4 High-Risk Actions Need Explicit Authorization

For irreversible actions such as force-push, hard reset, and bulk deletion, the prompt should require the model to get clear user authorization before it acts. This is a behavioral safeguard. It does not replace the permission system itself.

A more reliable approach is to handle actions by risk level.

Risk level	Examples	Handling
Read-only	Check status, read logs	Run directly
Reversible writes	Edit workspace files	Explain before acting, and keep the change reversible
Irreversible	Force-push, delete data	Require a second confirmation

3. The Dynamic Layer: Keep the Prompt Aligned with Runtime State

3.1 Assemble Tool Descriptions from What Is Enabled

Each tool carries its own description. The assembler walks the tools that are currently enabled and appends those descriptions to the prompt. When a tool is disabled, its description disappears with it, so the model does not see a tool it cannot call.

def build_tool_section(tools):
    lines = ["## Available tools"]
    for tool in tools:
        if not tool.enabled:
            continue
        lines.append(f"- {tool.name}: {tool.describe()}")
    return "\n".join(lines)

3.2 Inject External Service Instructions Incrementally

When a session is connected to an external tool service, the server provides usage instructions. Injecting the full text on every turn creates a lot of repetition. A better approach is to record what has already been sent and add only the delta when the connection state or the instructions change.

After context compression, those instructions may have been dropped, so they need to be injected again. Otherwise the model loses how to use those tools.

def inject_server_instructions(session, servers):
    current = {s.name: s.instructions for s in servers}
    delta = {k: v for k, v in current.items() if session.sent.get(k) != v}
    if delta or session.needs_reinject:
        session.sent.update(current)
        session.needs_reinject = False
        return format_block(delta)
    return None

3.3 Skill Index: Progressive Disclosure

Each skill usually has three fields: a name, a short description, and the situations where it applies. Putting those three into the prompt lets the model know which capabilities exist. The implementation details are loaded only when the skill is actually invoked. The model can see the range of capabilities without the full skill text filling the context.

3.4 Environment Information

Details such as the operating system, the current date, and the working directory give the model a basic sense of the runtime. Judgments about “today” or “the past week”, and relative path references, depend on that information. Environment data should be injected when the session starts and updated when important state changes.

3.5 Lazy Loading and Tool Discovery

When there are many tools, putting a full description of every tool into the prompt at once uses a large amount of context. One approach is to inject only tool names and a lookup entry point. When the model needs a particular tool, it loads the full definition through a search interface.

This is a typical load-on-demand pattern. The cost is an extra lookup call. The benefit is a cleaner context and more focused attention from the model.

4. Static Text Versus Dynamic Assembly

Dimension	One fixed block of text	Layered dynamic assembly
Consistency with tool state	Easy to go stale	Stays in sync with what is enabled
Environment awareness	Maintained by hand	Injected automatically at session start
Recovery after context compression	Usually nothing to do	Needs an explicit reinjection
Implementation complexity	Low	Higher; needs state management
Testability	High	Assembly output needs snapshot tests

The cost of dynamic assembly is mostly implementation complexity and testing. For a smaller system, static text plus a few dynamic fields is usually enough. Full layered assembly is worth the investment when there are many tools, long sessions, and large differences in environment.

5. Engineering Practices

1. Keep static rules short. Each rule should correspond to a behavior that can be verified.
2. Snapshot-test the assembled prompt, and confirm that disabling a tool actually removes its description.
3. Record which dynamic blocks were injected on each turn, so problems such as “the model does not know a tool exists” are easier to trace.
4. When compressing context, mark critical constraints as non-droppable.
5. Enforce authorization for irreversible actions in the execution layer, not only as wording in the prompt.

6. Summary

A system prompt is essentially runtime configuration. Splitting it into stable behavior rules and dynamic blocks that change with the session balances consistency, context cost, and safety boundaries. For a team building an agent, the first step is to decide which content must stay fixed and which content should be generated dynamically.