Division of Thoughts (DoT): How Small and Large Models Can Collaborate on Complex Tasks

    Division of Thoughts (DoT) is a system architecture that decomposes complex user requests into manageable subtasks and assigns each one to the most appropriate model. This article explains DoT’s architecture, task decomposition, model routing, dependency scheduling, structured intermediate results, validation, failure handling, and the engineering benefits of coordinating models with different capabilities.

    In real-world applications, user requests are rarely simple input-output tasks. They often involve several stages, including information understanding, planning, data processing, logical reasoning, and result synthesis. For example, “Plan an anniversary dinner and provide a budget” may require choosing dishes, confirming the number of guests, creating an ingredient list, checking prices, calculating costs, and writing the final proposal.

    Using the most capable large model for every step can produce good results, but it also increases inference costs and response times. On the other hand, assigning every task to a small model may lead to errors in complex reasoning, long-context understanding, and multi-step planning.

    Division of Thoughts, or DoT, can be understood as a division-of-labor architecture for AI systems. Instead of requiring a single model to complete the entire task, DoT breaks a complex request into manageable subtasks and assigns different model tiers according to the characteristics of each task. Its goal is not simply to call multiple models. It is to give the system capabilities similar to a project management workflow, including task decomposition, role assignment, process orchestration, and quality control.

    1. The Overall DoT Architecture

    A complete DoT system typically includes the following roles:

    1. Planner: Understands the user’s intent and breaks the overall goal into subtasks.
    2. Adapter or Router: Analyzes task difficulty and selects an appropriate model for each subtask.
    3. Executor: Performs the actual work. Different executors may be powered by models of different sizes or specializations.
    4. Validator: Checks whether the output is complete and correct, and triggers retries or escalation when necessary.
    5. Aggregator: Combines the outputs of individual subtasks into the final response.

    The Planner answers “What needs to be done?” The Adapter answers “Who should do it?” The Executor answers “How should it be done?” The Validator checks “Can the result be trusted?” Finally, the Aggregator determines “How should the result be delivered?”

    The workflow can be summarized as follows:

    1. User request
    2. ↓
    3. Task understanding and decomposition
    4. ↓
    5. Build a task dependency graph
    6. ↓
    7. Model selection and task assignment
    8. ↓
    9. Parallel or sequential execution
    10. ↓
    11. Result validation and error handling
    12. ↓
    13. Information aggregation and final generation

    2. Core Capability One: Task Decomposition

    2.1 Generate a Task Graph from a Natural-Language Goal

    The Planner should first identify the result the user actually wants rather than immediately generating an answer from the surface wording of the request. It needs to determine:

    • What the final deliverable should be;
    • Which intermediate results are required;
    • Which steps depend on earlier steps;
    • Which steps can run in parallel;
    • Which steps require external data or tools;
    • Which stages require complex reasoning.

    For example, “Design a family dinner within a fixed budget” can be decomposed as follows:

    1. [
    2. {
    3. "id": 1,
    4. "task": "Confirm the number of guests, dietary preferences, and budget limit",
    5. "deps": []
    6. },
    7. {
    8. "id": 2,
    9. "task": "Design a menu that matches the number of guests and their preferences",
    10. "deps": [1]
    11. },
    12. {
    13. "id": 3,
    14. "task": "List the required ingredients and quantities",
    15. "deps": [2]
    16. },
    17. {
    18. "id": 4,
    19. "task": "Estimate ingredient costs using price information",
    20. "deps": [3]
    21. },
    22. {
    23. "id": 5,
    24. "task": "Generate the dinner plan and budget explanation",
    25. "deps": [2, 4]
    26. }
    27. ]

    Here, deps represents dependencies. For example, the menu must be determined before the ingredient list can be created, and the ingredient list is required before costs can be estimated.

    2.2 Why Use a Directed Acyclic Graph?

    DoT commonly uses a directed acyclic graph, or DAG, to represent task dependencies. Each node represents a subtask, while each directed edge represents an execution dependency.

    A DAG provides three important benefits.

    First, it clarifies execution order.
    The system can determine which tasks must wait for earlier results. This prevents logical errors such as calculating ingredient costs before the dishes have been selected.

    Second, it supports parallel execution.
    Tasks without dependencies can run at the same time. For example, after the basic requirements are confirmed, menu design and dinner-flow planning may be performed independently and combined later. Parallel execution reduces the total waiting time.

    Third, it simplifies failure recovery.
    If one node fails, the system only needs to rerun that node and the downstream nodes that depend on it. It does not need to restart the entire workflow.

    2.3 Control the Granularity of Decomposition

    If tasks are too broad, the benefits of model specialization are limited. For example, “complete the entire dinner plan” is still a complex task and cannot be routed or optimized in a fine-grained way.

    If tasks are too small, the system incurs additional overhead, including:

    • More routing requests;
    • More complicated context transfer;
    • Higher costs for managing intermediate results;
    • More communication and aggregation time between nodes.

    A practical subtask should have a clear objective, explicit inputs, verifiable outputs, and enough independence to be completed by one model within a relatively isolated context.

    3. Core Capability Two: Model Assignment

    After the task graph is created, the system must decide which model should execute each task. This is the responsibility of the Adapter, also known as a model adapter or task router.

    3.1 Different Tasks Require Different Capabilities

    Not every task requires the strongest available model. Tasks can generally be classified by complexity.

    Low-complexity tasks:

    • Extracting keywords;
    • Identifying dates, amounts, and locations;
    • Classifying text;
    • Converting content into a specified format;
    • Performing basic field cleanup;
    • Generating short text from a fixed template.

    These tasks usually have clear rules, short reasoning chains, and easily verifiable outputs. They can often be assigned to smaller models with lower costs and faster response times.

    Medium-complexity tasks:

    • Summarizing multiple text segments;
    • Organizing and comparing information;
    • Generating a plan based on multiple conditions;
    • Structuring semi-structured content;
    • Performing simple analysis using existing data.

    These tasks require a moderate level of contextual understanding and are suitable for medium-capability models.

    High-complexity tasks:

    • Multi-step logical reasoning;
    • Designing solutions under complex constraints;
    • Synthesizing long-context information;
    • Analyzing software architectures;
    • Making reasonable inferences from incomplete information;
    • Making decisions across multiple candidate results.

    These tasks place higher demands on reasoning, context retention, and error correction. They should generally be assigned to more capable large models.

    3.2 Model Routing Should Not Depend Only on Task Names

    Judging difficulty by task name alone is unreliable. For example, “summarize a report” might mean extracting key points, or it might require cross-section comparisons, contradiction detection, and a final conclusion.

    The Adapter should also consider:

    • Input length;
    • The number of constraints;
    • Whether multiple reasoning steps are required;
    • Whether precise calculations are needed;
    • Whether the output format is strict;
    • The potential impact of errors;
    • Whether a rule-based validator is available;
    • Whether multiple intermediate results must be referenced.

    These factors can be combined into a simple scoring model:

    1. def estimate_complexity(task):
    2. score = 0
    3. if task.get("long_context"):
    4. score += 2
    5. if task.get("multi_step_reasoning"):
    6. score += 3
    7. if task.get("strict_format"):
    8. score += 1
    9. if task.get("requires_calculation"):
    10. score += 2
    11. if task.get("high_risk"):
    12. score += 3
    13. if score <= 2:
    14. return "small"
    15. elif score <= 5:
    16. return "medium"
    17. return "large"

    This code is only illustrative. In a production system, model selection may combine rules, historical execution results, and manually configured policies.

    3.3 Support Dynamic Escalation

    Model assignment should not always be a one-time decision. A more robust approach is to use a “low-cost first, escalate on failure” strategy:

    1. Let a small model execute the task first;
    2. Check the result with rules or a validation model;
    3. Accept it if it is complete and reliable;
    4. Escalate to a more capable model if the result contains formatting errors, missing fields, or insufficient confidence;
    5. Retry failed tasks a limited number of times.

    This approach can reduce unnecessary large-model calls while maintaining output quality.

    4. Core Capability Three: Execution and Dependency Scheduling

    Once the task graph has been built, the system must execute nodes according to their dependencies.

    4.1 Sequential Execution

    Tasks with clear ordering requirements should run sequentially:

    1. Requirement confirmation → Menu design → Ingredient planning → Cost calculation → Plan aggregation

    Each downstream node must wait for the previous node to finish and then consume its output.

    4.2 Parallel Execution

    Independent tasks can run in parallel:

    1. Requirement confirmation
    2. ↓
    3. ┌──────────────┐
    4. │ │
    5. Menu design Event planning
    6. │ │
    7. └──────┬───────┘
    8. ↓
    9. Final aggregation

    Parallel scheduling can be implemented with a task queue or asynchronous execution mechanism. The scheduler should track the state of each node, such as:

    • pending: waiting to execute;
    • running: currently executing;
    • success: completed successfully;
    • failed: execution failed;
    • blocked: waiting for dependencies;
    • skipped: skipped because a condition was not met.

    4.3 Intermediate Results Must Be Structured

    If every node returns only a natural-language paragraph, downstream models may have difficulty parsing the result reliably. A better approach is to define an input and output contract for every node:

    1. {
    2. "task_id": 4,
    3. "status": "success",
    4. "result": {
    5. "total_cost": 268,
    6. "currency": "CNY",
    7. "items": [
    8. {
    9. "name": "Ingredient A",
    10. "quantity": 2,
    11. "estimated_cost": 36
    12. }
    13. ]
    14. },
    15. "warnings": []
    16. }

    Structured results provide three main benefits:

    • They are easier for downstream tasks to consume;
    • They can be validated automatically;
    • They are easier to record, trace, and replay.

    5. Result Validation and Error Handling

    Multi-model collaboration does not make results automatically reliable. A DoT system needs a complete quality-control process.

    5.1 Common Validation Methods

    Format validation: Check whether the response follows the required JSON, table, or field structure.

    Completeness validation: Confirm that required fields are present and that all task requirements have been addressed.

    Consistency validation: Check for conflicts between nodes, such as mismatches among the number of guests, amounts, and quantities.

    Rule-based validation: For calculations, dates, and enumerated values, deterministic programmatic checks should be preferred over relying entirely on model judgment.

    Semantic validation: Ask another model to verify whether the result satisfies the original objective and constraints.

    5.2 Failure-Handling Strategies

    Different failure types require different responses:

    • Incomplete input: Ask the user for the missing information;
    • Invalid output format: Ask the original model to regenerate the result in the required format;
    • Insufficient reasoning quality: Escalate to a more capable model;
    • Temporarily unavailable external data: Delay execution or use a clearly labeled estimate;
    • Repeated failure: Return partial results and clearly explain the limitations.

    Infinite retries should be avoided. The system should define a maximum retry count, timeout period, and fallback strategy. Otherwise, resource usage may become excessive and response behavior unpredictable.

    6. Engineering Benefits of DoT

    6.1 Lower Overall Inference Cost

    Assigning simple tasks to smaller models reduces the number of large-model calls. The savings can be particularly significant in batch text processing, data cleaning, and information extraction.

    6.2 Faster Response Times

    Independent nodes can execute in parallel, shortening the critical path. Even if some complex tasks take longer, other nodes can finish in advance.

    6.3 Better Maintainability

    Once a complex workflow is divided into modules, developers can adjust the prompt, model type, or validation logic of an individual node without changing the entire application.

    6.4 Improved Observability

    The system can record each task’s inputs, outputs, execution time, selected model, retry count, and failure reason. These records make it easier to identify problems and continuously improve the workflow.

    7. Key Considerations in Practice

    First, avoid decomposing tasks excessively just to demonstrate multi-model collaboration. If the number of nodes becomes too large, scheduling overhead may offset the savings from using smaller models.

    Second, task boundaries must be clear. Each node should specify its input source, processing objective, output format, and error conditions.

    Third, distinguish generative tasks from deterministic tasks. Amount calculations, date arithmetic, and format validation should be handled by programs whenever possible. Models should focus on understanding and organization rather than taking responsibility for every calculation.

    Finally, model-routing policies should be evaluated continuously. Teams can track success rates, average execution time, escalation frequency, and manual correction rates for different task types, then use the results to refine routing rules.

    8. Conclusion

    The core idea behind Division of Thoughts is to transform a complex problem into an orchestratable task network and then complete the overall objective through effective cooperation between small and large models.

    Three principles are especially important:

    1. Plan before execution: Use task decomposition and dependency graphs to define the workflow;
    2. Assign models according to capability: Prefer lightweight models for simple tasks and stronger models for complex ones;
    3. Validate continuously after execution: Use structured outputs, rule-based checks, and dynamic escalation to maintain result quality.

    For developers, DoT is not a fixed API or a single model. It is a system design approach for complex intelligent applications. By clearly defining tasks, organizing dependencies, establishing model-routing policies, and implementing quality controls, teams can build more efficient and reliable multi-model systems for customer support, content generation, data analysis, intelligent office tools, and workflow automation.