How to Use LLMs to Write Novels: Engineering Methods from Short Fiction to Million-Word Manuscripts

4 hr ago0 views

This article explains why a single prompt usually fails to produce a publishable novel and how to turn LLM-assisted fiction writing into an engineering workflow. It covers the core constraints of long-form generation, a four-layer pipeline from setting library to chapter text, structured prompt design, retrieval-augmented memory, rule-based consistency checks, measurable quality scoring, and practical differences between short stories and long novels. It also discusses cost control, compliance boundaries, and how individual writing workflows can be scaled into team-based content production systems.

1. Problem decomposition: why a casual prompt cannot write a novel

Many creators first try AI-assisted fiction writing by searching for a prompt and typing something like, “Please write a 10,000-word short story.” The result is often disappointing: the model drifts off topic, characters behave inconsistently, or the text reads like a compiled collection of facts rather than a story.

The problem is not the model itself, but the method. In many workflows, the model is treated as a one-shot text generator, while a novel is inherently a long-term engineering problem that requires maintaining state across chapters, scenes, characters, timelines, and plot devices.

Before starting a project, it helps to understand several hard constraints of LLM-based long-form writing:

The context window is limited.
A model can only “see” a limited number of tokens in a single request. One million Chinese characters correspond to roughly 1.5 million tokens, which exceeds the capacity of almost any single request.
Attention weakens over distance.
Even if the first several hundred thousand characters are placed into the context, the model’s attention to early setup tends to decline. This produces the commonly observed “middle forgetting” problem.
Generation is stochastic.
Sampling temperature and random seeds affect output. The same setting may lead to different conclusions in different chapters.
Long-range dependencies are hard to maintain.
Character relationships, timelines, unresolved foreshadowing, and the boundaries of a power system must remain consistent across dozens or even hundreds of chapters.

The conclusion is clear: to use a model for long-form fiction, the writer must move “human memory” outside the model and into an externally searchable, verifiable memory system. This is the core idea behind the methods described below.

2. Overall architecture: a four-layer creative pipeline

A practical approach is to split novel creation into four layers. Structured files should be used as interfaces between layers.

Layer	Problem solved	Main artifacts	Main consumers
L1 Setting layer	What is the world like? Who are the characters? What are the rules and boundaries?	Setting library: characters, locations, abilities, timeline, foreshadowing	Read-only reference across the pipeline
L2 Outline layer	How does the story develop? How are volumes and chapters divided?	Volume outline, chapter outline, plot point list	Scene layer and validation
L3 Scene layer	What happens in each chapter? What conflicts occur?	Scene cards: location, characters present, goal, conflict, turn, ending hook	Text generation
L4 Text layer	Turns scene cards into prose	Chapter text, chapter summary, state patch	Quality check and human polish

The value of this separation is that it converts “uncontrollable creation” into “inspectable intermediate artifacts.”

If you ask the model to write ten thousand words in one pass, logical errors may only become visible after the entire text is finished. With chapter outlines and scene cards, many problems can be caught before prose generation begins.

Creative request
  ↓
[L1 Setting library] ── Retrieve relevant settings ──┐
[L2 Outline / chapter outline] ────────────────────┤
[L3 Scene card] ───────────────────────────────────┤
                                                   ↓
                                       [L4 Text generation]
                                                   ↓
                                       Chapter summary + state patch
                                                   ↓
                                       [Consistency check] → Rewrite if failed
                                                   ↓
                                       [Human polish] → Final draft stored

3. Setting library design: use structured files instead of model memory

The setting library is the foundation of the entire system. The core principle is: all reusable information should be structured, assigned a unique ID, and searchable and verifiable by code.

A main file plus several supporting tables can be used:

{
  "world_id": "novel_001",
  "style_guide": {
    "point_of_view": "Third-person limited, following the protagonist",
    "tense": "Past tense",
    "language_style": "Short sentences preferred; avoid stacked adjectives; avoid summary-style lyricism",
    "length_baseline": "1,200 to 1,800 characters per scene"
  },
  "characters": [
    {
      "id": "c_001",
      "name": "Lin Jue",
      "aliases": ["Little Lin", "Jue"],
      "identity": "Apprentice watch repairman in a county town",
      "core_desire": "Find out the truth about his father's disappearance",
      "fatal_flaw": "Distrusts authority and tends to act impulsively",
      "catchphrase": "A watch does not lie",
      "relationships": [
        {
          "target": "c_002",
          "type": "Mentor-apprentice",
          "tension": "Surface respect, hidden suspicion"
        }
      ],
      "current_state": "Chapter 12: left arm fractured and not yet healed"
    }
  ],
  "locations": [
    {
      "id": "l_001",
      "name": "Old Clock Tower",
      "features": "Damp, large echo, only one door"
    }
  ],
  "rules": [
    {
      "id": "r_001",
      "name": "Cost of ability",
      "content": "Every future simulation consumes a real memory irreversibly"
    }
  ],
  "timeline": [
    {
      "id": "t_008",
      "chapter": 12,
      "event": "Clock Tower confrontation; Lin Jue is injured",
      "time": "D+31"
    }
  ],
  "foreshadows": [
    {
      "id": "f_003",
      "setup_chapter": 3,
      "content": "Half a gear left behind by the father",
      "status": "Unresolved",
      "planned_payoff_chapter": 58
    }
  ]
}

Several design points deserve attention:

Character profiles should include flaws and verbal signatures.
If a character is described only with labels such as “kind” or “brave,” the model will likely produce a flat character. Flaws create conflict, while catchphrases and speech habits provide recognizable linguistic fingerprints.
State fields should update as chapters progress.
Injuries, poisoning, item acquisition, relationship deterioration, and similar events are all state changes. Before generating the next chapter, the latest state must be loaded. Otherwise, obvious continuity errors may occur, such as a character who had a broken leg in the previous chapter walking normally in the next one.
Power systems must define costs and limits.
A cost-free ability destroys narrative tension, and models tend to let protagonists become increasingly overpowered.
Foreshadowing should be managed like an account book.
Each foreshadowing item should record its setup chapter, planned payoff chapter, and current status. This is key to preventing a long novel from becoming incoherent or failing to resolve its setups.

4. Prompt design: layered templates and parameter tuning

For novel writing, prompts can be organized into four sections and injected as template variables in code.

[Role and style constraints]
You are a novelist skilled in {genre}. Follow these style rules: {style_guide}
[World and characters: read-only reference]
Do not add new settings.
{Retrieved relevant setting fragments}
[Chapter task]
Chapter number: {chapter_no}
Chapter outline: {chapter_outline}
Scene card: {scene_card}
Hard requirements: {must_include / must_not_include}
[Output contract]
1) Chapter summary: no more than 200 words
2) Chapter text: {word range}
3) State patch: JSON containing character state changes, new foreshadowing, and timeline events

A simplified calling logic example:

def write_chapter(chapter_no, scene_card):
    # 1. Retrieve settings relevant to this chapter: characters, locations, foreshadowing, ability rules
    context = retrieve_settings(
        query=scene_card["query_text"],
        top_k=8,
        chapter_no=chapter_no
    )
    # 2. Load rolling summaries: whole-book summary plus the most recent three chapters
    memory = load_rolling_summary(chapter_no)
    prompt = build_prompt(
        style=STYLE_GUIDE,
        context=context,
        memory=memory,
        outline=get_outline(chapter_no),
        scene=scene_card
    )
    for attempt in range(3):
        result = llm.chat(
            system="You are a rigorous novelist. Follow the setting library and output contract strictly.",
            user=prompt,
            temperature=0.8,
            max_tokens=6000
        )
        issues = consistency_check(result, chapter_no)
        if not issues["blocking"]:
            return result
    return manual_review(result, issues)

Several parameter-level practices are useful:

Use temperature by task type.
Tasks requiring determinism, such as setting organization and outline extraction, can use lower temperatures, typically around 0.2 to 0.4. Prose generation can use higher temperatures, around 0.7 to 0.9, to produce more language variation. If multiple candidate openings are needed, several samples can be generated in parallel and the best one selected.
Negative constraints are often more effective than positive requirements.
Instructions such as “Do not introduce any new characters,” “Do not summarize the entire chapter,” and “Avoid words such as ‘involuntarily’ or ‘slowly’” can significantly reduce drift.
Require structured output.
The summary, main text, and state patch should be returned in the same response so that code can parse them and write them back into the setting library. Pure prose output is difficult to validate automatically.
Scene cards must include conflict.
Without conflict, a scene card will usually turn into a flat sequence of events. Each card should at least specify who wants what, who or what is blocking them, and what the cost is.

5. Consistency assurance for long-form novels: retrieval augmentation and rule validation

A short story can be proofread by reading it through. A million-character novel must rely on machine assistance. Two complementary mechanisms are useful.

5.1 Retrieval-augmented “on-demand memory”

The setting library can be split into entry-level text fragments and vectorized for retrieval. When writing a chapter, the chapter outline and scene description can be used as the query to retrieve the most relevant setting entries and inject them into the context. This keeps the context focused on information related to the current chapter, saving tokens and reducing interference from irrelevant material.

A layered rolling summary should also be maintained:

Whole-book summary: around 1,000 to 2,000 characters, covering the main plotline, core conflicts, and major character arcs.
Volume summary: around 500 to 1,000 characters per volume.
Recent chapter summaries: around 150 to 300 characters per chapter, keeping only the most recent three to five chapters.

When writing Chapter 80, the context can contain the whole-book summary, relevant volume summaries, recent chapter summaries, and retrieved setting entries. The total remains within a controllable range of a few thousand tokens.

5.2 Rule-based hard validation

Some errors should not be left to the model to judge. Code can check them faster and more accurately:

Character names and alias registry.
Maintain an alias table and prohibit unregistered variants.
Timeline monotonicity.
Except for explicitly marked flashbacks, event timestamps should increase monotonically. A character cannot be in two places at the same time.
State continuity.
An injured character who has not recovered should not act as if fully healed. A character who has exited the story should not reappear.
Foreshadowing payoff rate.
Track the proportion of planted foreshadowing items that are resolved within their planned chapters. If the ratio falls below a threshold, trigger a reminder.
Ability-rule compliance.
Scan the text for ability usage and compare it with the defined boundaries, costs, and limits.

Validation results can be divided into blocking issues and warnings. Blocking issues require rewriting the affected section, while warnings are left to human judgment.

def consistency_check(chapter_text, chapter_no):
    blocking = []
    warnings = []
    for name in extract_names(chapter_text):
        if name not in NAME_REGISTRY:
            blocking.append(f"Unregistered character name: {name}")
    for event in extract_events(chapter_text):
        if not is_timeline_consistent(event, chapter_no):
            blocking.append(f"Timeline conflict: {event}")
    if uses_dead_character(chapter_text):
        blocking.append("A character who has left the story appears again")
    if unresolved_ratio(chapter_no) > 0.6:
        warnings.append("Too many unresolved foreshadowing items; consider resolving one or two in this chapter")
    return {
        "blocking": blocking,
        "warnings": warnings
    }

6. Quality evaluation: turning “well written” into measurable standards

Subjective judgments about prose quality do not scale well. However, editorial judgment can be decomposed into scorable dimensions. Each chapter can be automatically or semi-automatically scored after generation.

Dimension	Check question	Suggested weight
Opening hook	Does the first 200 words establish suspense, conflict, or anomaly?	15%
Scene conflict	Does each scene have a clear goal and obstacle?	20%
Character consistency	Do actions and dialogue match the character profile, including flaws?	20%
Information progression	Does the chapter advance the main plot or reveal key information?	15%
Ending hook	Does the chapter ending create a reason to continue reading?	10%
Language quality	Does the text avoid stacked adjectives, summary-style lyricism, and repetitive sentence patterns?	10%
Setting compliance	Does the chapter pass hard validation?	10%

Chapters below a threshold should not be accepted directly. Instead of rewriting the entire chapter, it is more efficient to perform targeted rewrites only on the low-scoring dimensions. This greatly reduces rework cost.

Several common “AI-style” problems can also be added to negative prompts:

Excessive filler adverbs;
Summarizing what happened in a chapter instead of dramatizing scenes;
Directly telling emotion, such as “he was very angry,” instead of showing it through action and dialogue;
Highly uniform sentence patterns and paragraph lengths, lacking variation between short and long sentences.

7. Different approaches for short stories and long novels

Short fiction: approximately 1,000 to 10,000 characters

The goal is to complete a full emotional arc in a very limited length. Practical points:

Use a four-part structure: hook, escalation, reversal, and aftertaste.
Present the core conflict in the first section.
Keep only two to four main characters. The setting library can be lightweight, with emphasis on character flaws and relational tension.
Generate a complete first draft in one pass, then perform two or three rounds of targeted rewrites.
Genres with strong conflict, a single main scene, and a clear emotional landing point tend to have a higher success rate.
Before delivery, check the target platform’s word-count range, formatting requirements, and originality declaration rules.

Long fiction: more than 100,000 characters

Long-form projects require an engineering process:

First complete a volume-level outline, ideally with 20 to 40 chapters per volume, then expand it into chapter outlines and finally scene cards.
Use a tracking board for written chapters, pending chapters, foreshadowing status, character appearance distribution, and average chapter length.
Maintain a stable update rhythm and a buffer of finished chapters to avoid breaks in continuity.
At the end of each volume, conduct a full review of the setting library and merge any changes back into it.

8. Efficiency, cost, and compliance boundaries

Throughput reference

Once the workflow becomes familiar, a short story can often be taken from initial idea to finalized draft within a few hours. For long-form projects, once prompt templates and the setting library are mature, the bottleneck often shifts from generation speed to human review and polishing. What truly determines output is whether the quality-check process is smooth, not how fast the model generates.

Cost control

Token consumption in long-form writing mainly comes from two sources:

Repeated injection of settings and summaries on the input side;
Rewriting and rework on the output side.

Effective control methods include:

Retrieving settings on demand instead of injecting the entire setting library;
Compressing summaries into tiers;
Performing local rewrites on low-scoring chapters instead of rewriting the whole chapter;
Reusing cached system prompts and style constraints for batch tasks.

Compliance and copyright

Compliance must be taken seriously:

Avoid creating near-copies of existing works by changing character names or place names. This creates both infringement risk and homogeneous content.
Imitating the distinctive style of a living author may create legal risk. It is safer to constrain style by genre rather than by a specific named author.
If generated content involves real people or real events, factual accuracy must be verified.
Many content platforms have labeling or declaration requirements for AI-generated content. Check the current rules of the target platform before publishing.

9. Scaling up: building an enterprise-level creative pipeline

When fiction writing moves from an individual hobby to team-based production, such as a content studio or an IP incubation team, prompts alone are not enough. The workflow needs to become an operable system.

Key components include:

Model service layer: a unified inference access layer that abstracts differences between models and supports switching and gradual rollout.
Storage layer: object storage for manuscripts and versions, relational databases for chapter outlines, states, and validation results, and vector retrieval services for the setting library.
Task orchestration: asynchronous generation tasks driven by a message queue, supporting batch processing, retries, and failure isolation.
Observability: logging prompt versions, parameters, latency, token consumption, and quality scores for each generation to create traceable quality data.
Permissions and audit: manuscript access levels, modification trails, and publishing approvals.

These capabilities can be assembled from general cloud capabilities, such as model services, object storage, vector retrieval, message queues, and monitoring/alerting offered by Baidu AI Cloud. The focus is on standardizing the creative process and turning quality metrics into data, rather than relying on a single-point tool.

10. Implementation roadmap

Choose a genre and write a style constraint document.
The document should be around 200 to 500 characters and include point of view, sentence-pattern preferences, and prohibited words.
Build a lightweight setting library.
Start with main character profiles, core ability rules, and the timeline for the first three chapters.
Complete the volume-level outline and the first batch of chapter outlines.
Check whether the causal chain of the main plot is closed.
Write three sample chapters.
Run through the loop of generation, validation, scoring, and rewriting. Use the results to calibrate prompt templates.
Add setting retrieval and hard validation scripts.
Move into stable production after the basic loop is working.
Establish a weekly review mechanism.
Analyze the distribution of low-scoring dimensions and use the findings to improve the setting library and prompts.

11. Common pitfalls

Using the same prompt for the entire book and never updating the setting library, which leads to character-state drift.
Writing chapter outlines too coarsely, such as “the protagonist defeats the villain,” forcing the model to invent details and drift away from the main plot.
Planting foreshadowing without maintaining a ledger, resulting in a low payoff rate.
Ignoring summary layers, causing the context to overflow or lose key information.
Fully automated generation without human polish, producing highly templated language that is difficult for readers to distinguish.
Focusing on the word count of a single generation instead of scene granularity, which multiplies rework costs.

Conclusion

The core of AI-assisted novel writing is not finding one universal prompt, but building a creative pipeline that combines a setting library, layered outlines, retrieval augmentation, rule validation, and quantitative evaluation.

Short fiction depends on structure. Long fiction depends on systems. When the engineering method is applied correctly, the same workflow can scale from a short story of a few thousand characters to a million-character novel.