Multi-Model Text Processing: Model Selection, Orchestration, and Engineering Practices

1. Treat “Overall Performance” as an Engineering Problem

Statements such as “model A is stronger than model B” are rarely useful in engineering practice for three reasons: the task is not defined, there is no baseline comparison, and the result cannot be reproduced. The same prompt may lead to completely reversed rankings under different context lengths, constraint counts, or temperature settings.

To make performance discussable, first turn “overall performance” from a scalar into a capability vector:

• Code generation and refactoring: single-file completion, cross-file edits, adherence to existing style, and whether the code compiles
• Logical reasoning and computation: multi-step derivation, numerical correctness, and satisfaction of constraints
• Long-document understanding and summarization: information retention, structural fidelity, and omission of key constraints
• Creative writing and concept discussion: novelty of ideas, style consistency, and human adoption rate
• Multilingual conversion: terminology accuracy, register compliance, and control of literal-translation phrasing
• Instruction following and format stability: parseable structured output rate and field completeness
• Engineering attributes: context window, concurrent throughput, failure retry behavior, and unit price

The key conclusion is that differences among models are differences in capability distributions, not just rankings. Any claim of “best overall” is only valid when bound to a specific task distribution.

2. Define Tasks Before Comparing Models: A Three-Layer Task Taxonomy

Comparing models without task layering is like comparing databases without workload profiles. Text tasks can be divided into three layers, each with a different evaluation focus:

Layer	Typical tasks	Core metrics	Main risks
Deterministic transformation layer	format conversion, field extraction, draft translation, template filling	format parse rate, automatic validation rate	broken structure
Reasoning layer	analysis, solution design, code generation, logic verification	accuracy, explainability, unit-test pass rate	hallucination, implicit errors
Generation layer	creative copy, dialogue, concept explanation	human adoption rate, edit distance	style drift, vacuous expressions

“Draft translation” and “translation polishing” actually belong to different layers: drafting is deterministic transformation plus language generation, while polishing is constrained local rewriting. The task space for the latter is much smaller than open-ended generation. This is the fundamental reason why “two models working in sequence can outperform one model doing everything from scratch”: not because the relay model is stronger, but because the second stage has been artificially reduced in dimensionality.

3. Replace Subjective Impressions with First-Pass Rate and Rework Cost

The engineering metric most worth introducing is rework cost. The same prompt may be delivered as usable in one pass by one model, while another model requires three rounds of correction. Unit-price differences can be completely offset by human time.

A practical offline evaluation set should contain 50–200 real tasks and include:

1. Task input and all constraints, including style prohibitions, glossary, and output format
2. Golden answers or reference translations, together with acceptable ranges
3. Automatic validation rules such as compilation, regular expressions, JSON Schema, and terminology matching

Core metrics:

# Evaluation skeleton (pseudocode, expressing measurement logic only)
def evaluate(model, cases):
    pass_cnt, total_rounds, total_cost = 0, 0, 0.0
    for c in cases:
        draft, rounds, cost = run_with_repair_loop(model, c, max_rounds=3)
        total_rounds += rounds
        total_cost += cost                       # includes input and output token cost
        if passes_gate(draft, c.rules):          # rule gate: format / terminology / unit tests
            pass_cnt += 1
    return {
        "first_pass_rate": pass_cnt / len(cases),
        "average_repair_rounds": total_rounds / len(cases),
        "cost_per_task": total_cost / len(cases),
    }
# True unit cost = model call cost + repair rounds × human correction cost per round
def true_cost(m):
    return m["cost_per_task"] + m["average_repair_rounds"] * HUMAN_FIX_COST

Version drift must also be recorded: model-side updates can invalidate historical conclusions. Therefore, both the evaluation set and runtime results should be versioned to form a regression baseline.

4. Two-Stage Pipeline: Draft Model + Polish Model

The most recommended structure is to split drafting and polishing into two independently replaceable stages. Its benefit comes not from model strength but from architecture: clear stage boundaries, and intermediate artifacts that can be validated, cached, and rolled back.

pipeline: text_production
stages:
  - id: parse
    task: extract_constraints          # hard requirements: length, terminology, register
  - id: glossary
    task: load_terminology             # load glossary and banned-expression list
  - id: draft
    model_tier: standard               # general tier is enough; task difficulty is low
    output: draft.txt
    prompt_rules:
      - must cover all constraint points
      - do not use em dashes for parenthetical insertions
  - id: lint
    type: rule_engine                  # hard constraints take precedence over the model
    checks: [glossary_hit, banned_patterns, structure_schema, length_band]
    on_fail: retry_draft | escalate_tier
  - id: polish
    model_tier: flagship               # restricted rewriting only; small task space
    instruction: adjust style and sentence patterns only; do not change facts or structure
    output: final.txt
  - id: review
    type: sampling_human               # sample final review by ratio; feed results back into eval set

The prompt for the polishing stage should be written as “restricted rewriting,” not “rewrite the whole thing,” and should include a positive example:

Role: Technical text editor
Task: Make language-level revisions only. Do not add or remove information points, and do not change technical conclusions.
Hard constraints:
1. Do not use an em dash (—) for parenthetical insertion. Use commas, parentheses, or split sentences instead.
2. Terminology must match the provided glossary, with exact capitalization and spelling.
3. No sentence may exceed 45 Chinese characters. Split overly long sentences.
4. Remove redundant fillers such as “carry out,” “implement,” “related,” and “to some extent.”
Negative example: This solution carries out a substantial optimization in performance, thereby achieving an overall improvement.
Positive example: This solution reduces the batching interval from 1 second to 100 milliseconds, cutting tail latency by about 40%.

5. Turn Style Rules into Hard Constraints, Not Prompt-Only Wishes

A requirement such as “do not overuse em dashes in the prompt” has an uncontrollable hit rate if it is only written in the prompt. A reliable approach is to split rules into two types:

• Rules that can be judged deterministically: em-dash frequency, terminology spelling, spacing between Chinese and English, sentence length, banned-word lists, and output structure. All of these should be handled by deterministic checkers.
• Rules that can only be judged semantically: style consistency, empty arguments, and concept misuse. These should be handled by the polishing model plus sampled human review.

BANNED = {
    "em_dash_per_1k": 2,          # max em dashes per 1,000 characters
    "filler": ["carry out", "implement", "related", "to some extent", "as is well known"],
}
def lint(text, glossary):
    hits = {w: text.count(w) for w in BANNED["filler"] if w in text}
    em = text.count("—") / max(len(text) / 1000, 1e-6)
    missing = [t for t in glossary.terms if t.canonical not in text and t.alias in text]
    return {"filler": hits, "em_dash_rate": em, "glossary": missing,
            "ok": not hits and em <= BANNED["em_dash_per_1k"] and not missing}

When a rule is violated, trigger a retry. If the failure exceeds the threshold, escalate to a higher-tier model. This turns “style issues” from random sampling into deterministic gates in the pipeline.

6. Tier Routing and Cost Curves

Not every task deserves the highest tier. Maintain three tiers: flagship, standard, and light, and route by a task complexity score:

def complexity(task):
    score = 0
    score += min(len(task.input) // 4000, 3)      # context length
    score += len(task.constraints)                # number of constraints
    score += 2 if task.needs_multi_step_reasoning else 0
    score += 2 if task.error_cost == "high" else 0
    return score
def route(task):
    s = complexity(task)
    if s >= 5:  return "flagship"
    if s >= 2:  return "standard"
    return "light"
# Cascade: run the light tier first; automatically escalate if the gate fails; avoid blanket high-cost routing
def run_cascade(task, tiers=("light", "standard", "flagship")):
    for i, tier in enumerate(tiers):
        out = call(tier, task)
        if passes_gate(out, task.rules) or i == len(tiers) - 1:
            return out

Two easily overlooked engineering details: content-hash caching, where identical inputs plus identical prompt versions are reused to avoid duplicate billing; and idempotent retries, where retries must use the same random seed or fixed temperature, otherwise the problem cannot be reproduced.

7. Reliability, Compliance, and Vendor Decoupling

• Availability and quotas: multi-channel access, rate-limit backoff, timeouts and circuit breakers, and fallback to backup channels. Account-level risks such as rate limiting, bans, and quota changes should be absorbed by architecture rather than concentrated on a single point.
• Data compliance: classify and grade text first, redact sensitive fields before entering model context, and retain logs covering input summaries and model versions for auditability. For workloads with residency requirements, evaluate cross-border transfer and regional availability, and prefer compliant regional model service providers where appropriate.
• Vendor decoupling: abstract a unified calling interface, and keep prompts, glossaries, evaluation sets, and rule engines as owned assets. Models become replaceable backends.

class LLMBackend(Protocol):
    def complete(self, prompt: Prompt, schema: Schema | None = None) -> Result: ...
    def tokens(self, r: Result) -> tuple[int, int]: ...   # input/output token counts

• Fact and hallucination control: for extraction tasks, require citations to source snippets and location references. For numerical calculation tasks, use external tools or code execution for validation. For creative tasks, use sampled human review fed back into the evaluation set.
• Prompt-injection protection: treat external documents and web content as untrusted data, strictly separate them from the instruction area, and prohibit them from overriding system-level constraints.

8. Five Common Pitfalls

1. Replacing your own evaluation set with public leaderboards, because leaderboard task distributions differ from business distributions.
2. Comparing only unit price while ignoring repair rounds and human correction costs.
3. Using model self-evaluation as the quality gate, because self-evaluation correlates weakly with actual usability.
4. Stacking negative prohibitions in prompts without positive examples, leaving the model to guess boundaries.
5. Ignoring version drift and regression testing, which causes online quality regression after silent model upgrades.

9. Implementation Checklist

• Build an evaluation set of at least 100 real tasks, including automatic validation rules, and version it
• Report three metrics: first-pass rate, average repair rounds, and true unit cost including human labor
• Split text production into a four-stage pipeline: draft → rule validation → polish → sampled final review
• Turn style and terminology rules into deterministic checkers, leaving only semantic judgments to the model
• Implement three-tier routing and cascade escalation, combined with content-hash caching
• Abstract a unified backend interface to isolate vendors, and prepare rate-limit backoff and fallback channels
• Classify and grade inputs and outputs, redact sensitive data, and include model versions in logs for traceability

The core idea can be summarized in one sentence: model capability differences are real, but engineering value comes from decomposing tasks, turning rules into verifiable gates, counting rework cost as part of total cost, and assigning the appropriate model to each role. With this approach, replacing any model generation will not overturn the overall architecture.