Engineering LLM-Powered Open-Ended Text Games

    This article covers the engineering discipline needed to keep LLM-powered open-ended text games maintainable at scale. It explains why generative models shift the real bottleneck from writing code to clarifying requirements and verifying behavior, then walks through practical techniques: tiered model routing with cost optimizations such as context trimming and result caching, a three-layer architecture that keeps open-ended gameplay controllable, a reputation system that turns player attitude into a computable filter, an action-point economy that makes stat differences felt, engineering guardrails for AI-generated code, and content-safety boundaries. It closes with three takeaways for practitioners.

    Once LLMs Landed, the Bottleneck Moved

    Once general-purpose LLMs became available, the first thing they replaced wasn’t architecture design but the mechanical step of turning already-decided logic into code. As a result, the real efficiency bottleneck shifted:

    Stage Past time sink Current time sink
    Requirements clarification Coding Requirements clarification
    Implementation Coding Review and acceptance
    Integration Coding Data contract consistency
    Long-term maintenance Coding Regression test coverage

    The conclusion is straightforward: models made “writing” cheap, which paradoxically made “thinking it through” and “verifying it’s correct” more expensive. This is especially true for open-ended sandbox gameplay—when players can “do whatever they want,” the state space is nearly infinite, and if the structure isn’t designed well, the faster you generate, the faster technical debt piles up.

    Tier Your Model Usage: Spend Compute Where It Actually Matters

    Tasks of different difficulty have vastly different model requirements. Putting architecture review and single-line log completion in the same tier is the biggest waste of both cost and latency.

    A recommended coarse-grained tiering:

    • Light tier: completion, formatting, boilerplate, comments, test-case drafts, log messages. Characteristics: short context, a single correct answer, and errors that are immediately obvious.
    • Heavy tier: cross-module refactoring, data-structure changes, root-cause analysis of tricky bugs, balance tuning. Characteristics: open-ended answers, broad impact, and high cost of mistakes.

    The routing logic doesn’t need to be complex—a few interpretable signals are enough:

    1. TIERS = {"light": ("fast-model", 4096), "heavy": ("strong-model", 8192)}
    2. def route(task):
    3. score = 2 * count_terms(task, ["cross-module", "interface", "refactor", "compatibility"])
    4. score += 1 if len(task.prompt) > 800 else 0
    5. score += 1 if task.file_count > 3 else 0
    6. score += 1 if task.has_acceptance_criteria else 0
    7. name = "heavy" if score >= 3 else "light"
    8. return TIERS[name]

    Layer three cost optimizations on top of this—they usually make a bigger difference than switching models:

    1. Context trimming: feed only signatures plus relevant call sites, not the whole repository. Irrelevant context not only costs money but also induces hallucinations.
    2. Result caching: build hash caches for highly repetitive inputs like function signatures, field names, and rule tables, and reuse them directly when regenerating the same change request.
    3. Async separation: route offline batch jobs (backfilling tests, generating copy tables) through a queue plus a containerized task pool, isolated from interactive coding sessions at the resource level so they don’t slow each other down.

    Three-Layer Split: Keeping Open-Ended Gameplay Controllable

    Open-ended doesn’t mean disordered. The reliable approach is to split the system into three layers and enforce one-directional dependencies between them:

    1. Presentation layer (text rendering / branch display)
    2. ↓ read-only
    3. Rules layer (combat, reputation, economy, event triggers)
    4. ↓ read-only
    5. Data layer (world graph, NPC tables, item tables, save snapshots)

    Key constraints: the rules layer must not assemble narrative text directly, and the data layer must not contain logic decisions. Once you enforce this, changing gameplay doesn’t touch text and changing text doesn’t touch rules, the scope of model generation is naturally confined to a single file, and review cost drops sharply.

    For world data, a four-level tree of “region → town → village → point of interest” works well. Describe connectivity between nodes with an adjacency list, and attach NPCs to nodes along with reputation gates:

    1. {
    2. "nodes": [{
    3. "id": "east_village_01",
    4. "type": "village",
    5. "region": "East",
    6. "neighbors": ["east_town_01"],
    7. "npcs": [{
    8. "id": "village_chief",
    9. "reputation_gate": { "min": "Newcomer", "deny_text": "Outsiders, do not linger here." },
    10. "options": ["ask_about_junior", "coerce", "quest_board"]
    11. }]
    12. }]
    13. }

    Reputation System: Turning “Attitude” into a Computable Filter

    Reputation is most often implemented as a scattering of if statements. A better approach is to define the ranks as an ordered enum, declare the gate uniformly on each interaction option, and centralize the check in a single function:

    1. REP_ORDER = ["Unknown", "Newcomer", "Recognized", "LocalHero", "Renowned"]
    2. def option_available(option, player) -> bool:
    3. gate = option.get("reputation_gate")
    4. if not gate:
    5. return True
    6. return REP_ORDER.index(player.reputation) >= REP_ORDER.index(gate["min"])
    7. def visible_options(npc, player):
    8. return [o for o in npc.options
    9. if option_available(REGISTRY[o], player)]

    This way, reputation is both a “lock” and an “unlock condition”: the village chief posts a quest at the gate to raise reputation, and only by returning after reaching the required rank can the player trigger the main-story clue. The player’s positive feedback loop forms naturally, with no hard-coded story branches.

    Action-Point Economy: Making Stat Differences Actually Felt by Players

    The biggest risk in turn-based combat is “stats are defined but never used.” Here, a single mapping function converts attributes into a combat resource:

    1. def action_points(dexterity: float) -> int:
    2. return max(1, min(int(dexterity // 10), 4)) # floor keeps it dignified, cap prevents snowballing
    3. def resolve_turn(player, enemy, rng):
    4. queue = sorted(
    5. [(player, action_points(player.dex)), (enemy, action_points(enemy.dex))],
    6. key=lambda p: (p[1], rng.random()), reverse=True)
    7. for actor, ap in queue:
    8. for _ in range(ap):
    9. if actor.hp <= 0 or enemy.hp <= 0:
    10. return
    11. yield actor.strike(rng)

    Three design points matter:

    • A cap must exist. Without a cap, high-skill characters will one-shot their opponents and the combat system degenerates into a single opening check.
    • A floor must exist. Give the weaker side at least one move, and players will conclude “I lost because I’m bad” rather than “the game’s checks are rigged.”
    • Diminishing returns beat linear stacking. Gradually decay the damage coefficient of the 3rd and 4th actions—this preserves the speed advantage while keeping turn length readable.

    Engineering Guardrails for AI-Generated Code

    Open-ended projects iterate fast, so guardrails matter more than speed. The four most effective things:

    1. Contracts first. Have humans define the interface signatures, field names, and enum values first, then let the model fill in only the implementation. Once field names are left to the model’s discretion, cross-file refactoring spins out of control.
    2. Vertical-slice commits. Each change covers a single playable loop (e.g., “press the village chief → combat → reputation +1”) rather than sprawling across five systems at once.
    3. Automated gates. Put type checking, static analysis, unit tests, and save-compatibility checks all into the pipeline; model output that fails the gates isn’t allowed to merge.
    4. Templatize prompts, baking acceptance criteria into the template rather than adding them ad hoc:
    1. Role: Senior game systems engineer
    2. Input: domain model {schema}, change request {change}, acceptance criteria {acceptance}
    3. Constraints:
    4. 1. Only modify the bodies of functions marked TODO; do not change signatures or field names
    5. 2. Do not add external dependencies
    6. 3. Self-check type checking and unit tests before submitting
    7. 4. If information is insufficient, ask 3 clarifying questions first; never guess field names
    8. Output: diff + change notes + known risks

    For text games, add one more layer of branch snapshot tests: run the full main storyline with a fixed random seed and write each node’s text and state into a snapshot. Any change that causes snapshot drift requires manual confirmation, preventing “you edit one dialogue and accidentally break the ending check.”

    Content-Safety Boundaries and Compliance

    Role-playing gameplay naturally brushes against content boundaries. This isn’t a model-capability problem but a design constraint the product must build in. A layered strategy that works in practice:

    • Input side: intent recognition plus a hard blocklist; on a hit, refuse directly and offer an alternative.
    • Generation side: soft/hard tiering. Downgrade soft content to suggestive expression and block hard content.
    • Display side: age-rating toggles, parental controls, and highly sensitive categories off by default.
    • Observability side: retain full records of moderation logs, hit reasons, and actions taken, to support review and appeals.

    In the cloud, this is typically provided as general-purpose capabilities: text and image content-moderation services, a customizable sensitive-word library, log services and monitoring/alerting for observing hit trends, and object storage for archiving moderation records. Engineering-wise, it’s best to make moderation a standalone gateway decoupled from the generation logic, so that adjusting policies doesn’t require changing game code.

    Three Conclusions for Practitioners

    • The efficiency dividend lands on design and verification, not typing speed. Only by reinvesting the saved time into data structures, test coverage, and observability does the benefit actually stick.
    • Self-controlled model supply has matured. From public-cloud API calls to private deployment, platforms like Baidu AI Cloud offer multiple access options along with supporting data, compute, and content-safety capabilities; selection should be tiered according to compliance requirements and data sensitivity.
    • Small, fast iterations still hold, but the definition of “fast” has changed: the model handles output, while humans define boundaries, hold the gates, and stay accountable for results.

    Between “can generate” and “can be maintained long-term,” what stands in the gap isn’t model parameters—it’s this engineering discipline.