A 200,000-Character System Prompt: What Prompt Bloat Costs and How a Cross-Session Memory File System Fixes It

1. Background: An Iteration That Only Adds

According to leaked system prompt text circulating in the community, one international model vendor’s new flagship model has a system prompt of roughly 200,000 characters. That is up from about 120,000 in the previous-generation flagship, an increase of about 67% over a span of only about a month. These numbers come from screenshots and text circulating online and have not been confirmed by the vendor. They should therefore be treated only as a sample for observing engineering trends in large models.

Placed side by side, the two generations show that the new content falls into fairly typical categories:

• Persistent-memory specification: A “memory file system” specification of more than 200 lines, with detail close to an internal product specification. It covers read/write timing, file formats, directory structure, privacy boundaries, and conflict handling.
• Model routing notes: New routing rules, such as routing high-risk requests from a lightweight model back to the flagship model.
• Product capability notes: Separate usage specifications for the command-line coding tool, the collaborative office suite, the browser, spreadsheets, presentations, design, and tags.
• Interaction rules: Details covering search, files, generative outputs, writing, citations, and tool calls.
• Safety details: Added reminders on child safety, mental health, weapons, and cybersecurity, plus new requirements for handling long conversations.
• Style rules: Many concrete language-style and interaction rules, such as when to check in with the user proactively and how to control length.
• Policy timeline: Even the complete timeline of a feature’s deprecation and reinstatement is written into the always-on prompt.

Every increase has a clear motive. When a problem appears in production, a rule is added. When a new feature ships, a section is added. When different product surfaces each go their own way, each gets its own specification. Each item looks reasonable on its own. Stacked together, however, the prompt has gone from a “code of conduct” to a “compilation of regulations.”

2. The Cost of Bloat: Why a Prompt Cannot Grow Without Limit

2.1 Rule Dilution and Conflicting Instructions

The more instructions there are, the more thinly each one’s “weight” is spread, and the greater the chance that they contradict each other. Suppose “stay concise” and “before answering, you must check the memory files and fully explain your basis” exist at the same time. The model’s behavior then becomes unstable. Longer content also makes “lost in the middle” more likely. In a very long context, the model tends to be more sensitive to what comes first and last, and content in the middle is used less effectively.

2.2 Cost, Latency, and Cache Invalidation

The system prompt usually sits at the very front of the context, so it is reprocessed on every turn of a conversation. Many inference services support prefix caching. When the prefix stays unchanged, later requests can reuse the already-computed intermediate state (the KV cache), which reduces compute overhead. But every change to the prompt can invalidate the cache for everything after it. Frequent patching therefore amplifies both latency and compute cost.

2.3 Maintenance and Evaluation

Once the prompt reaches 200,000 characters, it becomes hard for engineers to explain exactly which rule made the model make a given decision. A fix often adds another exception clause on top of the existing rules, so the rules keep multiplying along with the problems. Verifying before launch that rules do not conflict and cause no regressions also takes a great deal of testing.

In practice, a feedback loop makes the problem worse. Retrieval tools and skill descriptions tell the model through documentation when to use a given tool. Call frequency then rises, and teams tend to write more accumulated experience into documentation. This is often one of the main reasons specifications keep growing.

3. The Cross-Session Memory File System: The Most Worth Studying Design

Of all the additions, the one with the most engineering value is a cross-session memory mechanism. It organizes a user’s context into a set of files under a single directory. Different surfaces can all access these files and may even modify them at the same time:

• the conversational client
• the command-line coding tool
• the collaborative office suite

In effect, this gives the model a piece of “working memory.”

3.1 Directory Convention: Five File Categories for Long-Term Context

Memory is divided into five categories by purpose:

• profile.md: stable identity information, such as name, role, job title, work location, and long-term responsibilities
• preferences.md: preferences for how answers are given, such as language, level of detail, and output format
• areas/: domains the user is responsible for over the long term, such as an ongoing project or a technical direction
• people/: information about people the user collaborates with on an ongoing basis
• topics/: temporary topics or interests, which have a shorter shelf life

This is paired with a clear criterion for what gets written. According to the leaked text, the core standard is this: whether information is worth writing down depends on whether it will still be useful three months from now. This is exactly what separates a memory system from an ordinary conversation log. It keeps “long-term, still-relevant working context,” not a running record of everything.

/memories
├── profile.md            # Stable identity: name, role, long-term responsibilities
├── preferences.md        # Answering preferences: language, level of detail, format
├── areas/
│   ├── flutter-project.md   # Ongoing cross-platform mobile project
│   └── ...
├── people/
│   ├── partner.md        # Core team members
│   └── ...
└── topics/
    ├── food.md           # Topics such as food preferences
    └── ...

According to the presumed metadata design, each file also carries a brief description, aliases, and a source. Together, these act as a lightweight index.

3.2 Read Strategy: Index First, Then Full Text

Injecting all memories on every turn would only make the prompt bulkier, which runs counter to the goal of controlling bloat. This design therefore reads on demand. Each turn injects only three things:

• <profile>: the full text of profile.md, since identity is the most frequently needed context
• <preferences>: a summary of answering preferences
• <memory_listing>: the path, brief description, aliases, and source of every memory file. The file contents are not shown, so the model gets “a table of contents rather than the books.”

When the model needs details, it calls a memory read tool to fetch them on demand. The following is an illustrative example:

# Memory context injected on each turn (illustrative)
<profile>
Name: Li M. | Role: Backend Engineer | Team: Trading Platform R&D
</profile>
<preferences>
Answer in Chinese by default; for code questions, give the conclusion first, then an example; do not add long summaries unless asked.
</preferences>
<memory_listing>
- /areas/flutter-project.md   [Alias: Mobile Engineering] Source: coding tool
  Contents: Long-term technical decisions and architecture conventions for a cross-platform mobile project
- /areas/payments.md          [Alias: Payment Pipeline]   Source: conversation
  Contents: Stability design and capacity planning for the payment gateway
- /people/partner.md          [Alias: Collaborator]       Source: conversation
  Contents: A colleague frequently involved in day-to-day collaboration, and the modules they own
- /topics/food.md             [Alias: Food]               Source: conversation
  Contents: Taste preferences and dietary restrictions
</memory_listing>

Suppose the user asks: “What architecture did we finally settle on for that cross-platform mobile project of mine?” The model handles it as follows:

1. It looks through the listing and locates /areas/flutter-project.md.
2. It calls the memory read tool to fetch the file’s full text.
3. It answers based on that content, rather than refusing on the grounds that “this information is not in the context.”

The prompt stresses one rule again and again: before answering “I don’t have this information,” the model must first check the listing and read any file that might be relevant. This rule clearly separates “I can’t” from “I didn’t look,” which directly improves reliability.

Even more notable, the same directory serves multiple product surfaces. A technical convention the user asks the model to remember in a chat can also be read in the command-line coding tool. The experience comes close to “one person, one memory.”

3.3 Write Timing and Lifecycle

On the write side, the prompt sets an important principle. A memory is written not because the user explicitly said “remember this,” but because “you will need this context in the future.” This turns memory from a note the user specifically asks for into working memory that the model maintains on its own, and it makes the model responsible for deciding what is worth keeping.

The full flow usually works like this:

1. A “memory extraction” step runs when a session ends.
2. The model filters out the long-term information from that session.
3. It updates existing files or creates new ones.
4. It generates or refreshes the corresponding listing entry.

The illustrative logic is as follows:

def consolidate_memories(session) -> None:
    # 1) Extract candidates: only facts, preferences, and decisions with lasting value
    candidates = extract_long_term_facts(session)
    # 2) Classify each candidate and locate its target file (profile / preferences / areas / people / topics)
    for item in candidates:
        path = classify(item)              # determine the target path from the memory specification
        if not is_long_lived(item):        # still useful after three months? otherwise, drop it
            continue
        existing = load_memory(path)       # if it already exists, merge incrementally rather than overwrite
        merged = merge_conflicts(existing, item)   # resolve conflicts according to the agreed rules
        write_memory(path, merged)         # attach source information when necessary
    refresh_index()                        # update memory_listing

Different categories of memory call for different lifecycle strategies:

Category	Typical content	Criterion	Recommended strategy
profile.md	Name, role, long-term responsibilities	Unchanged for months	Keep stable; record changes
preferences.md	Language, level of detail, and format preferences	Valid over a fairly long term	User can override at any time
areas/	Ongoing projects and technical directions	Weeks to months	Archive when the project ends
people/	Collaborators and their responsibilities	Months to years	Update as the organization changes
topics/	Topics, interests, and temporary preferences	Days to weeks	Clean up automatically once expired

4. Why Keep Memory in Files Instead of the Prompt

The core idea of this design is this: the prompt carries “rules,” and the memory files carry “facts.” Rules should be few and stable, while facts are numerous and change constantly. If facts are written straight into the prompt, it will grow steadily the longer it is used and fall back into bloat.

A rough estimate shows the difference. Suppose a user has accumulated 50 memory files of about 300 Chinese characters each. Injecting all of them on every turn takes about 15,000 characters. At a rough rate of 0.6 to 1 token per Chinese character, that is about 9,000 to 15,000 tokens.

If each turn injects only the identity, a preferences summary, and a listing of about 200 characters, it takes only a few hundred characters. When a memory is a hit, reading 1 to 2 files usually keeps the total under 2,000 characters. The more turns a conversation has, the wider the gap becomes.

More importantly, prefix caching works best with content that stays the same. Moving frequently changing facts outside the prompt both keeps the context short and protects the cache hit rate.

5. Enterprise Practice: Building Layered Memory

To build similar capabilities in your own product, you can follow the idea of “layered context” and group content by how often it changes:

Layer	Content	Change frequency	Injection strategy
Global rules layer	Safety baselines, code of conduct, tool-calling rules	Low	Always in the prompt; version-controlled
Identity and preferences layer	User profile, answering preferences	Low to medium	Always included, as a summary
Long-term memory layer	Memory files for domains, people, topics, and so on	Medium	Listing first; read on demand
Session context layer	Current conversation, retrieved snippets, tool results	High	Assembled dynamically; compressed promptly

In practice, it is worth focusing on the following points:

1. Never change a rule only by adding to it: Before adding a new rule, ask “which existing rule already covers this?” Revise an existing rule rather than appending to it where possible. Also tag every rule with the reason it was introduced and the issue it addresses.
2. Split the prompt by role: Coding, office, and browser surfaces differ greatly from one another. Have them share a “common core,” then load “surface-specific modules.” This keeps all specifications from being crammed into a single prompt.
3. Unify tool interfaces: Memory reads and writes and index maintenance should be abstracted into common interfaces. Otherwise, each surface will implement its own version and the data will become inconsistent.
4. Version concurrent writes: When multiple entry points modify the same memory at the same time, use a version number or optimistic locking to prevent them from overwriting each other. Attach a source and a timestamp to each write so changes can be traced and rolled back.
5. Build in privacy and compliance: Memory files may contain personal information. Users should be able to view, export, and delete them, sensitive information should be masked, and retention periods should be set according to business needs.
6. Test before you expand: Build a test suite that covers memory reads and writes, rule conflicts, and long-conversation regressions. Run baseline tests before adding or changing a rule, and keep monitoring results after launch.

At the infrastructure level, general-purpose cloud services can fully support this kind of design:

• Object storage: stores memory files and version snapshots
• Message queue: triggers memory extraction after a session ends, asynchronously
• Vector retrieval and knowledge base: semantic search over large amounts of unstructured content
• Log service: records prompt versions and retrieval hits
• Monitoring, alerting, and container platform: track changes in call volume, latency, and cache hit rate

If you use Baidu AI Cloud products, such as the Qianfan model platform and general-purpose services like object storage, log service, and monitoring and alerting, you can follow the same layered approach. This lets you build similar long-term memory capabilities in your own business systems.

6. Conclusion

At its core, a 200,000-character system prompt is what an iteration style of “fix every problem by adding, never by removing” produces over time. Every patch has a reason, but together they cost rule dilution, cache invalidation, and growing maintenance difficulty. By comparison, the memory file system points to a more sustainable path:

• Use layering to control what is actually injected on each turn.
• Use index plus on-demand reading to balance context length against complete information.
• Use a time-based write test to keep memory quality high.

For developers and enterprises, rather than adding more to the prompt, it makes more sense to treat the prompt as a rulebook and memory as an archive. Keep the rules concise and stable, and manage the archive in tiers. This may be the key step in moving large models from “longer prompts” to “better context engineering.”