Building a Chat Log Organizer with Open-Source LLMs: A Long-Horizon Coding Workflow

1. Background: Long-Horizon Coding as the Differentiator

Over the past two years, AI-assisted programming has evolved from “completion” to “generation” and now to “agents.” Early models excelled at single-point tasks: completing a function, generating a utility class, or explaining a code snippet. These tasks share common characteristics: short context, singular objectives, and low failure costs.

What truly sets models apart today is long-horizon coding capability: the ability to work on cross-file, cross-module complex projects for hours rather than minutes, maintaining goal stability, ensuring error recovery, and accumulating progress. Typical characteristics of complex projects include:

• Multiple States: Requirements, decisions, progress, and known issues must be continuously remembered.
• Broad Dependencies: A change in one module impacts builds, tests, and documentation.
• Frequent Iterations: The first version is almost guaranteed to have issues. The key is whether the model can iteratively fix them based on feedback rather than starting from scratch.

Recent public benchmarks have started scoring “continuous iteration on complex tasks” as an independent dimension. Open-source models have entered the top tier in this aspect, with the gap between them and closed-source models mainly lying in long-horizon stability rather than single-point capabilities. This article uses a real-world project—a mobile chat log organizer and knowledge base sync tool—to break down a reusable development workflow for open-source LLMs.

2. Case Study: A Chat Log Organizer and Sync Tool

2.1 Requirements and Compliance Boundaries

The goal is straightforward: organize mobile chat logs into structured data and sync them to personal notes and knowledge bases for retrieval and AI assistant integration. Before starting, we must define three compliance boundaries:

1. Process Personal Data Only: Limited to data generated on personal devices and accounts, obtained via system or app-provided backup/export channels. No reverse engineering or bypassing of third-party platforms.
2. Local-First: Parsing, cleaning, and storage occur entirely on the local machine. Sensitive fields are desensitized by default, and any upload action requires explicit confirmation.
3. Data Minimization: Retain only fields necessary for retrieval (summaries, timestamps, contact notes). Original content is encrypted and archived.

These boundaries dictate the subsequent architectural design.

2.2 Technical Pipeline

The pipeline consists of five layers:

Export Layer -> Parsing Layer -> Structuring Layer -> Storage Layer -> Sync Layer

• Export Layer: Retrieves raw data from system backups or app export files.
• Parsing Layer: Converts private or semi-structured formats into an intermediate representation.
• Structuring Layer: Extracts entities (messages, sessions, timestamps, participants) and outputs a unified Schema.
• Storage Layer: Writes to a local database with encryption and incremental indexing.
• Sync Layer: Interfaces with note-taking and knowledge base tools using standard formats (Markdown, JSON), supporting incremental updates and conflict resolution.

The unified Schema is the foundation of the project. Example:

{
  "msg_id": "sha1(session+timestamp+content_summary)",
  "session": "Unique session identifier",
  "ts": "Message time, ISO8601",
  "role": "Sender identifier",
  "kind": "text|image|voice|file",
  "summary": "Desensitized content summary",
  "tags": ["Auto-classification tags"]
}

3. Core Methodology: A Two-Step Workflow

For this project, I did not immediately let the model write code. Instead, I strictly executed two steps: let the model produce a plan, review it, and then let it execute autonomously. This yielded the greatest benefits in practice.

3.1 Why Not Write Code Directly?

Direct coding has three typical failure modes:

• Goal Drift: In long sessions, the model gradually “forgets” the initial requirements, making local optimizations that conflict with the overall architecture.
• Context Waste: Writing and modifying iteratively generates massive ineffective back-and-forth, exponentially increasing token consumption.
• Error Amplification: An early incorrect abstraction is inherited by all subsequent code. The later it is corrected, the higher the cost.

A plan front-loads “hundreds of small decisions” into a single, centralized decision and solidifies them in text, serving as an anchor that can be re-read anytime during long sessions.

3.2 Step One: Let the Model Write the Plan

A plan is not a vague roadmap; it must be executable and verifiable. It is recommended to have the model output using the following template:

# Project Plan: Chat Log Organizer and Sync Tool v0.1
## Milestones
M1 Parsing Layer: Input export file, output JSONL matching the Schema
M2 Storage and Indexing: Local DB write, support incremental import
M3 Sync Layer MVP: One-way sync to Markdown directory
## Task List (Sorted by Dependencies)
- [ ] T1 Define intermediate representation and error codes
- [ ] T2 Implement export file parser + unit tests
- [ ] ...
## Acceptance Criteria
- M1: Sample file parsing success rate >= 99%, anomalous lines logged
- M3: Repeated runs do not produce duplicate entries (idempotency)
## Risks and Contingencies
- Private format version differences -> Parser dispatches by version number

During manual review, focus on three aspects: whether task granularity fits a single session, whether acceptance criteria are objectively measurable, and whether risks cover data formats and compliance. The plan itself must be versioned and continuously revised as execution progresses.

3.3 Step Two: Autonomous Execution

Once the plan is confirmed, enter the autonomous execution phase. The core is a closed loop of “implement -> self-test -> backfill results -> fix”:

for task in plan.tasks:
    agent.implement(task)            # Execute based on task list
    result = run_tests(task.scope)   # Run corresponding tests
    while not result.passed and attempt < MAX_RETRY:
        agent.fix(result.errors)     # Backfill raw error logs for the model to fix
        result = run_tests(task.scope)
    log_progress(task, result)       # Write progress to file for subsequent session reading

Key details:

• Raw Error Backfilling: Do not manually paraphrase errors. Pass the raw output from the compiler or test framework directly to the model; the fix success rate is noticeably higher.
• Externalized Progress: Plans, decisions, and progress are saved in project files, rather than relying on the model to remember conversation history.
• Human Checkpoints: Pause at the end of each milestone; proceed only after manual review.

In this case, this workflow allowed the model to run continuously for over six hours, producing the first runnable mobile version. Subsequent parser robustness and sync idempotency issues were also gradually resolved through multiple iterations under the same mechanism.

4. Long-Session Engineering: Context, Quotas, and Checkpoints

4.1 Context Management

The biggest enemy of long sessions is not model capability, but context pollution: irrelevant errors, deprecated code, and intermediate trial products continuously dilute effective information. In practice, follow the rule “if it can be saved to disk, keep it out of the conversation”:

• Write architectural decisions into decision record files.
• Remove details of completed tasks from the conversation, retaining only task IDs and conclusions.
• Cold-start each new session from three files: “Plan + Progress + Current Task.”

4.2 Quotas and Costs

Long-horizon tasks consume quotas far more than daily Q&A, mainly for three reasons: long contexts are repeatedly read across multi-turn tool calls, failure retries multiply call volumes, and autonomous execution “never tires.” Corresponding countermeasures:

• Control task granularity; claim only one or two tasks from the plan per session.
• Prune context promptly; replace large files with summaries.
• Set checkpoints to ensure that after an interruption, execution resumes from the latest consistent state rather than starting over.
• Monitor usage curves. Anomalous growth usually means the model is stuck in a retry loop; intervene immediately.

If you notice obvious anomalies in billing or quotas, retain call logs and report to the service provider. In this practice, the provider confirmed a quota statistics bias and provided a fix and compensation, proving that maintaining a complete evidence chain is crucial in long-session scenarios.

5. How to Evaluate Open-Source Models for Your Project

Public benchmark scores are the starting point for screening, but they cannot replace validation tailored to your specific scenario. I recommend a “30-minute mini-evaluation”:

1. Pick a real, medium-difficulty task from your project, such as “add boundary cases for the parser and fix defects.”
2. Run it twice using both “direct coding” and “plan-first” approaches.
3. Observe four dimensions: Plan Quality (is the breakdown logical?), Error Recovery (can it converge after error backfilling?), Long-Session Stability (does goal drift occur after half an hour?), and Tool Call Accuracy (are commands and file operations precise?).

Furthermore, if your scenario involves 3D content generation or multimodal understanding, design separate test cases using the same methodology. Do not infer specific capabilities based solely on overall public benchmark scores.

6. Conclusion and Practical Checklist

Looking back at this project, open-source models are fully capable of playing the role of “autonomous executor with plan constraints” in long-horizon coding. Human value is concentrated in two areas: writing good plans and conducting reviews at key nodes. Here is a ready-to-implement checklist:

• Define compliance and data boundaries before implementation.
• Plan first; align task granularity with single sessions.
• Ensure acceptance criteria are objectively measurable; backfill raw error logs.
• Save progress and decisions to disk; sessions must support cold starts.
• Monitor quotas, set checkpoints, and intervene on anomalies.
• Run a mini-evaluation with your own tasks before large-scale adoption.

When “plan first, let it run autonomously” becomes a team’s default workflow, long-horizon coding capabilities truly translate into stable delivery efficiency.