A recently leaked system prompt reportedly grew to roughly 200,000 characters in about a month. This article explains why prompts bloat, the four categories of cost that follow — token spend and latency, instruction dilution and behavior drift, lost maintainability and verifiability, and a wider security exposure surface — and how to govern the system prompt as a software artifact: layered loading, explicit priority syntax, retrieval-based memory files, quantified metrics, and a supporting toolchain. It includes a full reference structure showing how such a prompt can be reduced from six figures to roughly 5,000 resident characters

1. Starting at 200,000 Characters: The System Prompt Is Becoming a Stack of Patches

A general-purpose AI assistant was recently reported to have shipped a new system prompt of roughly 200,000 characters, and the discussion that followed has been extensive. In the screenshots circulating in the community, a single collapsed code block holds the entire prompt; expanded, it runs to an extraordinary length. That alone would not have made developers pay attention — a rendering quirk explains very little. The real signal is the growth rate: in about one month, the system prompt expanded by nearly 70%.

For developers, the system prompt stopped being an ordinary string hidden on a server long ago. It is written by a team, revised after every incident, and it directly determines an agent’s behavioral boundaries and answer quality. When its size moves from a few kilobytes to hundreds of thousands of characters, the prompt has crossed out of “documentation” territory and into the scale of a software system.

1.1 What Is Driving the Growth

From public discussion and the prompt fragments shared by the community, the bloat comes from a handful of recurring sources:

1. Interface and product surface expansion: nearly every new surface, tool, or plugin that ships requires an added block of product description and interaction rules.
2. Finer-grained safety policy: rules for high-risk areas such as child safety, mental health, weapons control, and cybersecurity keep getting more detailed, and long-conversation scenarios have picked up their own dedicated constraints.
3. Cross-session memory: an entire specification for working memory was folded into the prompt, covering when to read, when to write, file formats, directory structure, and privacy boundaries.
4. Time-sensitive facts embedded as text: one example cited is a complete timeline of an app’s removal and restoration under export controls written directly into the prompt. Content like this is fundamentally an external fact and should not live permanently as static text.

1.2 Three Symptoms of the Bloat

Dimension	Symptom	Direct Consequence
Scale	~70% growth in a month, ~200,000 characters total	Context cost per turn keeps climbing
Structure	General principles, scenario rules, examples, and time-sensitive facts mixed together	Rules interfere with each other; conflicts are hard to locate
Governance	Accumulated by “found a problem → append a clause,” with no retirement mechanism	New rules pile up; old rules are rarely re-examined

2. Four Classes of Cost from Prompt Bloat

Much of the discussion reduced the problem to “it’s too long.” Length is only the surface. What matters more is the systematic cost the bloat creates.

2.1 Cost and Latency

The system prompt is re-sent on every turn. Even if 200,000 characters is not enough to blow past the context window of mainstream large models — which run from hundreds of thousands to over a million tokens — it still occupies input length on every single request, driving up token consumption and time-to-first-token.

Prompt caching, the common industry technique of caching and reusing a fixed prefix, can absorb some of that overhead. Its precondition, however, is a stable prefix, and a system prompt grown by constant appends is the textbook case of a prefix that never stops changing. Once the cache hit rate drops, the savings shrink dramatically.

The more insidious cost is hidden: model attention is not distributed evenly across tokens, and longer content in the middle of the context tends to be relatively weakened. When large volumes of rules are piled in the middle, you can get “written but not effectively followed” behavior — while the team misdiagnoses it as insufficient model capability and appends still more rules.

2.2 Instruction Dilution and Behavior Drift

As the number of rules in the prompt grows, the effective “attention share” each rule receives is spread thinner. Rules can also conflict implicitly. For example:

• A global instruction says “be concise and direct,” while a scenario spec says “product descriptions should be as detailed as possible.”
• A global instruction says “state honestly when information is unconfirmed,” while a specific scenario says “lead with the most likely answer.”

Without explicit priority and conflict-resolution rules, the model has to trade off between rules arbitrarily — the same class of question behaves differently across sessions, and production issues become hard to reproduce.

2.3 Maintainability and Verifiability

When the prompt becomes a single monolithic text of hundreds of thousands of characters, the team gradually loses three capabilities:

• Attribution: an abnormal answer in production can’t be traced back to the rule that caused it.
• Regression testing: after editing one rule, you can’t confirm whether other scenarios broke.
• Cleanup: nobody dares delete any old rule, because “what if removing it breaks something.”

The system prompt ends up as an ever-growing pile of technical debt.

2.4 A Wider Security Exposure Surface

A system prompt is effectively a complete behavioral specification: internal safety policy, compliance judgment criteria, the product roadmap, and even sensitive decision timelines such as takedowns and restorations may all be written into it. If the prompt leaks, or is induced into repeating itself under adversarial input, what is exposed is not just the rules themselves but the entire risk model and defensive logic. An attacker can use that to construct targeted bypasses.

Seen this way, “write everything into the system prompt” is not merely an efficiency problem. It is a security problem.

3. The Way Out: Govern the System Prompt as Software

3.1 Ask Three Questions First

Before any discussion of “slimming down,” every prompt change should answer three questions:

1. Necessity: does this information have to be resident in context? Could it become externally retrievable knowledge, or something a tool fetches at runtime?
2. Effectiveness: is there evaluation data showing the target metric actually improved after this rule was added? Rules without data behind them are often just psychological comfort.
3. Retireability: has the scenario it serves been retired? Has the rule been superseded by later clauses? Who has the authority to delete it?

3.2 Layer by “Scope × Lifecycle”

Much like code, the system prompt can be layered, with lazy loading controlling what stays resident:

Layer	Scope	Lifecycle	Loading Strategy	Typical Content
L0 Global safety baseline	All scenarios	Long-term	Resident	Non-negotiable safety and compliance red lines
L1 Core behavioral principles	All conversations	Long-term	Resident (trimmed)	Role definition, general answer principles, time-sensitivity verification requirement
L2 Scenario capability specs	Specific surface, tool	Mid-term	Injected on scenario entry	Tool capabilities and product interaction rules
L3 Domain details and cases	Specific domain	Short-to-mid term	Loaded when a trigger matches	High-risk domain rules, historical incident cases
L4 Time-sensitive facts	Changes with the outside world	Short-term	Not in the prompt	Compliance policy, regulatory changes, product status timelines

Two points carry the weight here:

• Trim what stays resident: keep L0 and L1 within a few thousand characters so a limited context budget covers as many scenarios as possible.
• Load everything else on demand: store L2 and L3 as separate files, selected by scenario routing or trigger matching.

Industry practice shows similar tradeoffs. Many AI coding tools split coding rules into “global rules” and “project-level rules,” keeping the former lean and resident while the latter is configured per project on demand.

3.3 Define Priority and Conflict Resolution

After layering, you still need priority between layers and between rules. A reasonably safe ordering:

1. Safety and compliance rules outrank everything and always win in a conflict.
2. When global principles conflict with scenario specs, the scenario spec applies only insofar as it does not lower the safety standard.
3. When the prompt conflicts with externally retrieved content, and the external content is stale or from an unreliable source, prefer the model’s internal knowledge or explicitly flagged uncertainty over blindly trusting it.

3.4 Tag Rules with a Unified Priority Syntax

Explicit tags inside the prompt are more reliable than using natural-language tone to signal rule strength. For example:

[P0-SEC]   Safety and compliance red line: never crossed under any condition.
[P1-CORE]  Global behavioral principles: followed by default; takes precedence over P2 in a conflict.
[P2-SCENE] Scenario spec: applies only in the matching scenario; must not lower P0/P1 requirements.
[P3-EX]    Examples and cases: illustrate rule intent only; impose no additional constraints.

With tags in place, a script can automatically check questions such as: does a P1 rule directly contradict a P2 rule inside the same scenario, and has any P0 clause been weakened by a later “exception” note. Going further, you can record each rule together with its ID, owner, introduction date, and verification result — giving the prompt a traceable change history like code.

4. Case Study: The Design Logic of Cross-Session Memory

The largest single addition is a cross-session “working memory” mechanism, specified in enough detail to resemble an internal product specification. It happens to illustrate one answer to prompt bloat: move “what needs to be remembered” out of the prompt body into an external file system, and keep only the access rules in the prompt.

4.1 Why a Memory File System Instead of Longer Summaries

Traditional cross-session memory uses one of two approaches: generate a summary at the end of a session, or stuff the full conversation history into context. The first loses a lot of information; the second is expensive. The memory file system treats memory as a maintainable data asset rather than a one-off context snapshot.

The specification opens with a telling statement about why to write: memory is written not because the user asked for it, but because the future you will need that context. Two things follow from that:

• Memory serves future tasks, not just the current conversation.
• Memory is an actively maintained commitment, which means there must be explicit write, update, and cleanup mechanisms.

4.2 Three Categories of Context Injection and On-Demand Reads

The mechanism does not load all memory every turn. It injects three categories by default:

1. profile: the body of the identity file, usually short.
2. preferences: the user’s preferences for answer style and collaboration.
3. memory_listing: the memory index, containing only paths, summaries, aliases, and sources — not bodies.

When specific content is needed, a read tool loads it on demand. Pseudocode (illustrative):

def build_memory_context(memory_root):
    return {
        "profile": read_text(memory_root + "/profile.md"),          # body injected
        "preferences": read_text(memory_root + "/preferences.md"),  # body injected
        "memory_listing": [                                        # index only
            {
                "path": rel_path,
                "summary": meta["description"],
                "aliases": meta.get("aliases", []),
                "source": meta.get("source", "user"),
            }
            for rel_path, meta in list_memory(memory_root)
        ],
    }

This is classic directory-style progressive disclosure: show the table of contents first, load the bodies on demand. The idea is widely used in documentation systems and design specifications. The benefits show up in three places:

• Context size stays bounded and does not grow linearly with the amount of memory.
• Irrelevant memories don’t interfere with current reasoning.
• The index itself can be stored structurally, which makes retrieval and access control easier.

4.3 Required Steps Before Saying “I Don’t Have That Information”

One rule in this mechanism is emphasized repeatedly: before answering “there is no information on that,” you must check the index and read the possibly relevant files.

Suppose the index lists /areas/billing-service.md, /people/pm-alice.md, and /topics/deployment.md. When a user asks about the architecture decision for the billing service, you must read /areas/billing-service.md first, even though none of those bodies were injected into context. The specification explicitly forbids using “not mentioned in this conversation” as sufficient grounds.

[Memory retrieval precondition]
1. Parse the entities (projects, people, domains) and time range in the user's question.
2. Match paths, summaries, and aliases in memory_listing to identify candidates (at most N files).
3. Read the candidate files in order.
4. Only when the index has no match and no candidate file is relevant may you answer
   "no information on that," stating the scope you searched.

This rule effectively simulates a retrieval protocol in the prompt: retrieve first, answer second, so that “not injected” is not mistaken for “does not exist.” Similar retrieval-before-answer constraints are common in enterprise knowledge base question answering.

4.4 Write Decisions: Filter Memory by Time Sensitivity

Not everything is worth writing to memory. The approach classifies information by time sensitivity and only keeps stable information for the long term:

Information Type	Example	Time Characteristic	Handling Strategy
Identity facts	Name, role, team	Stable for months or more	Write to profile
Preferences and habits	Answer detail level, code style	Stable for months or more	Write to preferences
Domain knowledge	Owned projects, long-term focus areas	Stable for months or more	Write to the corresponding domain file
Temporary task state	Current iteration progress, ad hoc context	Days to weeks	Keep in conversation context or the task system
Externally time-sensitive facts	Policy changes, release dates	Changes with the outside world	Retrieve in real time; never write to memory

The criterion is straightforward: information that will most likely still hold three months from now is a candidate for long-term memory.

In practice you can operationalize this further. If a fact’s source data updates more slowly than the memory’s expected lifetime — an architecture decision, for instance, changes on a quarterly cadence — it belongs in memory. If it updates faster — product prices, shift schedules — it should be fetched live by a tool. That way the memory system holds only “structural facts” and leaves volatile data to external systems.

4.5 Five File Types and Directory Conventions

The specification uses directories to separate information types, with each file mapping to one purpose and one lifecycle:

• /profile.md: stable identity information.
• /preferences.md: answer style and collaboration preferences.
• /areas/*.md: domains or projects owned long-term.
• /topics/*.md: recurring subject-matter knowledge.
• /people/*.md: relevant collaborators and their associated preferences.

Organizing memory by file path rather than one large text gives you clear ownership and independent updates. It also supports incremental writes naturally — an update in one domain does not ripple into other files — and makes per-category access control and cleanup straightforward.

4.6 Shared Across Entry Points and Concurrent Writes

Multiple product surfaces — a chat interface, a coding assistant, an office collaboration assistant — may share one memory file set and modify it simultaneously, which is a very realistic engineering challenge. Three common risks:

1. Two surfaces read the same old version of a file and each commit; the later write overwrites the earlier one.
2. The same fact appears in multiple files and gradually diverges into contradictions.
3. Content deleted by one surface is written back from another surface’s cache.

A safer approach is to add a version number and last-modified timestamp to each memory file and validate the version on write. On conflict, merge by “append rather than overwrite,” or mark the item as pending confirmation for the user to decide. Pseudocode (illustrative):

expected = read_meta(path).version
new_text = merge(base=read_text(path), incoming=change, strategy="append_section")
ok = write_if_version(path, new_text, expected=expected)
if not ok:
    conflict = detect_conflict(read_text(path), change)
    mark_pending(path, conflict)   # queue for user confirmation

Alongside this, you need a single-source-of-truth convention: each fact is maintained only in the most specific file that owns it, and other locations keep references only, so multiple copies don’t drift apart.

4.7 Memory Lifecycle and Privacy Boundaries

Any long-running memory system eventually faces forgetting and privacy. At minimum, the design should cover:

• Expiry and archiving: periodically review memories that haven’t been updated in a long time and classify them as “still valid,” “needs confirmation,” or “archive.”
• Correction: when a user corrects information, update the existing entry instead of appending, so old and new content don’t coexist.
• View and delete: memory should be transparent to the user, viewable, and deletable.
• Minimization and de-identification: don’t proactively record sensitive personal information unrelated to the task, and de-identify where necessary.

“Correct rather than append” is frequently overlooked in prompt practice, and it is the key defense against memory drift.

4.8 What Actually Stays in the Prompt

Worth emphasizing: what this mechanism writes into the system prompt is not the memory content itself but the rules about memory — when to read, when to write, how to organize, how to resolve conflicts. That is precisely what keeps prompt size under control:

Content goes outside and loads on demand; the prompt keeps only access rules and judgment criteria.

5. Full Example: How a 200,000-Character Prompt Shrinks to 5,000

Below is an illustrative system prompt structure (rule bodies omitted) showing how these principles land in practice.

# [P0-SEC] Safety and compliance baseline (resident, ~800 chars)
- Handling principles for high-risk requests: child safety, mental health,
  weapons, cybersecurity;
- Processing order for high-risk requests: safety baseline > global
  principles > scenario spec.
# [P1-CORE] Core behavioral principles (resident, ~1200 chars)
- Role: general-purpose AI assistant, playing the role defined by the
  product description of the surface it runs in;
- Answer principles: accurate, concise, direct; state the basis when
  uncertain;
- Time-sensitivity verification: for volatile information such as prices,
  versions, policies, and regulations, verify via tools;
- Language: match the user's language by default.
# [P2-SCENE] Scenario specs (loaded on demand, 1000-3000 chars each)
- chat-assistant-spec.md    <- search, file upload, citation conventions
- coding-assistant-spec.md  <- code change flow, dependency introduction, testing requirements
- office-collab-spec.md     <- handling of documents, spreadsheets, presentations
- browser-ops-spec.md       <- safety boundaries and confirmation mechanisms for page actions
# [P3-EX] Domain details (trigger-based loading)
- high-risk-domain/*.md     <- loaded when the risk classifier matches
- incident-cases/*.md       <- referenced only when the same class of problem appears
# [MEM] Memory system rules (resident, ~1000 chars)
- Inject profile, preferences, and memory_listing by default;
- Before answering "no information on that," search the index and read candidates;
- Write criterion: facts that will most likely still hold three months from now;
- Concurrent writes use version validation; on conflict, append or defer to the user.
# [EXT] Time-sensitive facts
Not written into the prompt; fetched via retrieval tools, with source and
fetch time noted.

In this structure, the resident portion — L0, L1, the memory rules, and the scenario routing table — comes to roughly 5,000 characters. Scenario specs, domain rules, memory file bodies, and time-sensitive facts all move outside and load on demand.

The character counts above only indicate orders of magnitude; actual figures need to be measured against your team’s rule text. The before-and-after comparison summarizes to:

Content Category	Before	After	Loading
Safety baseline and global principles	Mixed throughout one continuous rule text	Trimmed to a few thousand characters	Resident
Scenario and product descriptions	All resident	Split per scenario	Injected on demand
Memory bodies	Stuffed in as summaries or repeatedly injected in full	Separate files	Read on demand
Domain rules and incident cases	Appended one at a time	Categorized and archived	Loaded on trigger match
Time-sensitive facts (e.g., takedown timelines)	Written in directly	Not in the prompt	Retrieved live by tools

To be clear, reducing resident length does not mean discarding information. It means each piece of information appears at the moment it is most needed.

6. Measurement: Give Prompt Governance Something to Stand On

Without metrics, there is no governance. At minimum, establish the following:

• Resident length and growth rate: count characters and tokens of the resident prompt weekly, and set a growth threshold — for example, a resident portion above 8,000 characters must go through review.
• Prefix cache hit rate: measures prefix stability and cost efficiency.
• Instruction adherence rate: design test cases per key rule and track the adherence percentage.
• Out-of-bounds rate: sample and measure high-risk scenarios weekly.
• Attribution cost: average time from discovering a production issue to locating the specific rule behind it.

You can also build a rule-level A/B evaluation: split the prompt into independent rule modules, add or remove exactly one module per run, and compare metrics on a fixed test set. A rule is kept only when the metrics genuinely improve, which avoids adding rules on intuition.

To keep evaluation close to real usage, the evaluation set can come from sampled production logs plus human-annotated high-frequency questions. Each rule should carry a “verification status” field — such as “verified effective,” “pending verification,” or “recommend retirement” — and be included in periodic review.

7. Enterprise Adoption: Toolchain and General Cloud Capabilities

Treating the prompt as software requires a supporting engineering toolchain:

1. Version control: keep prompts as files in the code repository, with every change going through review. Reviews should focus on three things: is it resident, can it be made retrievable, and does it carry a verification result.
2. Automated checks: add resident-length budget checks, sensitive information scans, and P0-versus-P2 conflict detection to CI.
3. Layered loading: the scenario routing layer decides which spec files to load based on entry point, intent, and risk classification.
4. Observability: record the content fingerprint and length injected on each request, to make issues reproducible and costs traceable.
5. Memory data management: memory files need persistent storage, version history, and audit logs. Where retrieval is involved, an index can combine vector search with keyword search.

On the cloud, these capabilities can be carried by general-purpose product categories:

• Object storage: hold memory files and snapshots of every prompt version.
• Message queues: asynchronously drive memory index updates, expiry checks, and conflict detection jobs.
• Container platform: host runtime services such as scenario routing and retrieval, scaling elastically with traffic.
• Logging service: centrally collect the content and length injected per request, supporting fast search and issue reproduction.
• Monitoring and alerting: alert on resident-length growth, anomalous out-of-bounds rates, and sudden cache hit-rate drops.
• Access control and audit: isolate memory data per user and per tenant, keeping all memory operations traceable.

On Baidu AI Cloud, for example, a team can store prompt and memory file version snapshots in object storage, track core metrics such as resident length and out-of-bounds rate with the logging and monitoring services, and absorb peak load from memory index updates through the container platform’s elasticity.

8. Practical Advice by Role

Developers: treat the system prompt as code you write and maintain. Set a length cap on the resident portion, and require every new rule to answer two questions: can it be loaded on demand instead, and how will its effectiveness be verified.

AI application architects: separate three classes of information — the safety baseline that must stay resident, scenario specs that load on demand, and time-sensitive facts fetched live — and make the conflict-resolution order explicit.

Enterprise leaders: pay attention to sensitive information exposure and compliance risk inside the prompt. Establish a prompt review process so every new rule has a clear owner and a verification result.

9. Conclusion

A 200,000-character system prompt is less a “leak incident” than a mirror. It reflects an engineering problem that becomes universal once intelligent systems reach scale: when rules accumulate in natural language, the system gradually loses controllability.

The fix is not writing less but placing things correctly: keep stable rules in the prompt, defer scenario specs until they are needed, hand memory content to the file system and retrieval, hand volatile facts to tools, and put every change through version control and quantitative evaluation.

Prompt engineering is shifting from writing prompts to governing prompts. The sooner a team builds layering, measurement, and regression into the process, the better it can keep the system stable and controllable while capabilities iterate quickly.