A Multi-Agent Paper Writing Pipeline: From Topic Selection to Final Manuscript

1. Why “Write a Whole Paper With One Prompt” Is Doomed to Fail

Many people who use large language models to assist with writing hit the same wall: the first round produces a stunning outline, the third round starts contradicting itself, and by the fifth round the reference list contains papers that do not exist. This is not a single-point capability gap. It is a structural flaw created by forcing a long-horizon, multi-role research task into a single conversation:

• Context degradation. A paper body runs into the tens of thousands of words, and once literature excerpts are added, the material easily exceeds the model’s effective attention range. Memory of early constraints — assumptions, variable definitions, inclusion and exclusion criteria — gets diluted.
• Mixed responsibilities. When the same role both “proposes ideas” and “evaluates ideas,” it will tend to confirm itself. Review credibility comes from role independence.
• No validation loop. Content generated during the writing stage has no fact-checking step, so hallucinated citations and incorrect statistical descriptions flow straight through to the final draft.
• Non-interruptible process. Researchers usually already have partial results — a rough draft, pilot data, reviewer comments — and a monolithic conversation cannot be entered from the middle.

The engineering answer is not to swap in a stronger model. It is to decompose the research process into orchestrable capability units — each with a single responsibility, an explicit input/output contract, and interception by checkpoints — and then chain them into a pipeline with an orchestrator. The sections below break down the design of four stages: research, writing, review, and finalization.

2. How Capability Units Are Organized: Declarative Skill Packages

Academic workflows suit pluggable skill packages because their methodology is highly stable while execution details vary widely. A skill package typically contains four kinds of assets:

research-paper-skills/
├── research/              # Research stage
│   ├── SKILL.md           # Responsibility boundaries, I/O contract, trigger conditions
│   ├── agents/
│   │   ├── citation-verifier.md   # Citation provenance checks
│   │   ├── socratic-mentor.md     # Socratic questioning
│   │   └── devil-advocate.md      # Adversarial reviewer
│   └── templates/
│       ├── search-protocol.md     # Search protocol template
│       └── prisma-flow.md         # Systematic review screening flow template
├── writing/               # Writing stage (outline / argument / draft / abstract / figures / citations)
├── review/                # Review stage (editor-in-chief + domain reviewers + devil's advocate)
└── pipeline/              # Orchestrator (state machine, checkpoints, recovery logic)

Metadata is described declaratively so the pipeline can discover and schedule units automatically:

name: research-stage
version: 1.3.0
inputs:  { topic: string, constraints: object, existing_notes?: string[] }
outputs: { research_question: doc, method_design: doc, evidence_table: table }
gates:
  - every_claim_has_source
  - reference_list_verified == 100%
sub_agents: [citation-verifier, socratic-mentor, devil-advocate]

The critical design element here is gates: when a stage’s output does not satisfy a hard condition, it is not allowed to flow into the next stage. No matter how fluent the prose is, if the citations have not all passed verification, the pipeline stops at that step. This turns “quality” from a hope expressed in a prompt into an enforceable structural constraint.

3. Research Stage: Forcing a Vague Topic Into an Executable Research Question

The most common mistakes at the research stage are a topic that is too broad and a claim that cannot be refuted. What works in practice is three sub-agents working together.

Citation verification agent. For every reference introduced, the model does not merely “claim” that the paper exists. It runs a three-step validation: issue a title-level search against a public academic literature metadata service; compare four fields — title, first author, year, and DOI; and check whether volume, issue, and page numbers are consistent with the year. Any entry that cannot be matched is flagged as unverified and is forbidden from entering the reference list. The executable checking logic looks roughly like this:

def verify(ref) -> dict:
    hits = search_by_title(ref.title)          # Metadata service lookup
    exact = [h for h in hits if norm(h.title) == norm(ref.title)]
    if not exact:
        return {"status": "unverified", "action": "manual_review_required"}
    h = exact[0]
    ok = (year_match(h.year, ref.year)
          and author_overlap(h.authors, ref.authors) > 0.5
          and (ref.doi is None or h.doi == ref.doi))
    return {"status": "verified" if ok else "conflict", "record": h}

Hallucinated citations have some typical fingerprints that make cheap batch pre-screening possible: a DOI prefix that does not match the claimed journal, volume or issue numbers that contradict the year, a citation count that deviates oddly for the stated era, and an author name combination with zero hits in the metadata database.

Socratic questioning agent. Using a constrained question set, it presses round after round to converge “I want to study X” into a testable proposition: what are the target population and the sampling frame? How is the dependent variable measured, and where is the reliability and validity evidence? What is the causal identification strategy, and are the identification assumptions defensible? What evidence would refute your hypothesis? Where do the boundary conditions fail? The final artifact is a research question card, not a vague paragraph of introduction.

Devil’s advocate agent. Its output is a “threat list”: alternative explanations, omitted variables, endogeneity, common method bias, measurement error, external validity, and insufficient statistical power. Each threat either gets a mitigating design or is explicitly written into the limitations section. Allowing limitations to be admitted is more valuable than making the model defend itself aggressively.

4. Writing Stage: Decompose Into Sub-Agents, and Build a Style Profile

The common practice at the writing stage is to assign each part of the task to an independent sub-agent rather than letting one mega-prompt handle everything:

Stage element	Sub-agent responsibility	Artifact
Structure	Argument skeleton, section weighting	Outline and claim–evidence mapping table
Argument	Bind each claim to evidence strength	Argument cards
Draft	Generate section by section, control terminology consistency	Per-section body text
Abstract	Bilingual, length-controlled	Structured abstract
Figures	Caption, axis, and significance-marking conventions	Figure list and captions
Citations	Render intermediate citation format into the target style	Reference list

Style profiling is the highest-return part of this pipeline. The approach is not to ask the model to “imitate someone’s tone.” Instead, you extract quantifiable features from the author’s own previously written texts that the author is entitled to use: average sentence length and sentence-length variance, subordinate clause nesting depth, passive voice ratio, paragraph length distribution, preference for connectives, first-person usage habits, and fixed spellings of terminology and abbreviations. These statistics are then written into the writing prompt as explicit constraints:

style_profile:
  avg_sentence_len: 24.6      # words
  passive_ratio: 0.18
  citation_density: 0.09      # citations per sentence
  forbidden_patterns: ["it is worth noting that", "in today's society"]
  paragraph_len_range: [90, 160]

This works far better than “write more academically,” because it converts a subjective requirement into checkable numbers. But there must be a red line: style calibration applies only to the expression layer and must not rewrite facts, data, or conclusions. Otherwise polishing becomes academic misconduct.

5. Review Stage: Independent Multi-Perspective Blind Review With Quantitative Thresholds

The value of review lies in independence. A configuration that closely mirrors a real journal process looks like this: one editor-in-chief who consolidates and adjudicates, three domain reviewers who score independently from different perspectives, and one devil’s advocate dedicated to finding systematic flaws. Review dimensions are best split into six: methodology, evidence sufficiency, fit with the disciplinary perspective, interdisciplinary value, reproducibility, and writing conventions — each scored independently, then weighted.

Thresholds can be fixed as interpretable bands so authors know what to do next:

Weighted total	Decision	Author action
≥ 80	Accept	Finalize according to formatting rules
65–79	Minor revision	Add clarifications, fix wording
50–64	Major revision	Add experiments/evidence, restructure the argument
< 50	Reject	Redesign the research question

Two engineering details deserve attention. First, scores anchor behavior. Once an author sees “78,” subsequent revisions circle around recovering those two points while substantive flaws go ignored, so the total score should always be output together with a list of must-fix items. Second, consistency must be checked. If the spread among the three reviewers exceeds a threshold (for example, a range greater than 20 points), an arbitration flow should be triggered, in which the editor-in-chief asks both sides of the disagreement to present their evidence, rather than simply averaging. An average hides a fundamental methodological dispute.

The most useful review artifact is actually not the score but a prioritized revision roadmap: P0 blockers (affect whether the conclusion holds), P1 necessary items (affect credibility), and P2 improvements (affect readability).

6. Orchestrator: A 10-Stage State Machine With Checkpoint Resume

The core of the orchestrator is not a prompt but an explicit state machine. Each stage defines an input contract, an artifact path, and an acceptance checklist, and the state lives in files, which makes insertion at any stage and resumption after interruption possible:

{
  "stage": "4-revision",
  "artifacts": {
    "1-research-question": "out/rq.md",
    "2-method-design": "out/method.md",
    "2.5-completeness": "out/gap-report.md",
    "3-draft": "out/draft.md",
    "4-review": "out/reviews/*.json"
  },
  "blockers": ["P0: missing power analysis", "P0: 3 references failed verification"],
  "resume_from": "allowed"
}

Three common insertion points are worth supporting explicitly in practice. Authors who already have a draft enter at the completeness check and run only the missing sections. Those who have received peer review comments go directly into the revision stage, with the comments parsed into structured P0/P1/P2 items. Those who only want polishing and formatting skip research and review and go straight to writing and export.

7. Multi-Format Output and Reproducibility

The right way to handle format conversion is a single source of truth with multi-target rendering, not a different prompt every time you generate. The intermediate layer uses a structured citation and document model — for example, citations as generic structured entries (type, author list, year, title, publication, volume/issue/pages, identifier). A style definition then renders these into either an author–date reference list or a numbered sequential one; the body is rendered from a structured section model into a rich-text document template or a scholarly typesetting markup source, which is finally compiled into submission-ready files.

For reproducibility, the pipeline should automatically persist to disk: the model and prompt versions used, the full query string and hit count for every retrieval round, screening counts (for the review flow diagram), a hash of each stage’s artifacts, and review scores plus arbitration records. This is both the basis for meeting journal reproducibility requirements and the evidence base for later accountability and self-review.

8. Cost, Latency, and Compliance Boundaries

Cost control comes from tiering, not from trimming prompts. High-frequency, low-difficulty tasks such as literature pre-screening, format checking, and batch abstract generation go to lightweight models; high-capability models are called only at critical nodes such as argument restructuring, method design, and review adjudication. Retrieval results for identical inputs can be cached and reused. To be clear, the total cost of a single paper varies widely with literature scale, review rounds, and retries. Any claim of a “fixed cost” should be treated with suspicion — run a real load test with your own corpus scale before estimating a budget.

Data boundaries are more easily overlooked than process design. Unpublished data, human subject information, and corporate partnership data should not enter public online environments. Workable options include choosing a large model service platform that provides data isolation and enterprise-grade controls (platforms such as Baidu AI Cloud generally document enterprise-side capabilities such as private deployment, private network access, and log auditing), de-identifying data locally before calling a remote model, and handling classified content entirely offline. Actual capabilities, availability, and sign-up procedures should be confirmed against the platform’s documentation for the current release.

The responsibility boundary cannot be outsourced. The pipeline can substantially reduce mechanical labor, but the value judgment about the research question, the interpretation of conclusions, the lawful use of data, and the AI usage disclosure at submission all remain the author’s responsibility. Most journals and publishers now require disclosure of generative AI usage and explicitly state that AI cannot be listed as an author. Treating this pipeline as a “team of assistants that speeds things up” is appropriate; treating it as a “ghostwriter that removes the need to think” will be exposed in peer review and later audits.

9. Summary

The engineering value of this pipeline does not lie in how well any single step writes. It lies in three structural choices: breaking long tasks into sub-agents with single responsibilities, so review escapes self-confirmation; turning quality into executable gate conditions, so hallucinated citations and evidence gaps cannot flow into the next stage; and externalizing state, so a human can take over judgment at any stage. Stacked together, AI-assisted writing stops being “generate a block of text” and becomes “an auditable, interruptible, reproducible production line.” A practical way to start: run the two lowest-risk steps — the completeness check and format export — on an existing draft, verify the output quality, and only then extend forward into the research and review stages.