Why LLMs Feel "Hard to Use" in AI Coding Tools: A Skill-Based Engineering Practice
This article breaks down why a newly integrated LLM often feels "hard to use" in an AI coding tool, tracing the problem across four layers — integration, loop, task, and expectation — instead of blaming the model. It then lays out standard integration steps (a thin adapter layer, a parameter-and-prompt tuning checklist, and observability), explains how to package domain knowledge as skills through progressive disclosure, and walks through a concrete web-novel generation skill built on externalized state, a staged generation pipeline, and independent consistency validation. It also argues for judging usability with metrics rather than feel, and closes with cloud-based engineering support. The methodology is model-agnostic and reusable across reasoning models and coding tools.
1. Start by Dissecting “Hard to Use”: It’s Rarely a Single-Point Problem
When a newly integrated model produces a poor experience in an AI coding tool, the blame usually falls on “the model just isn’t good enough.” In actual troubleshooting, the problems are distributed across layers. Confusing these layers is what leads to endlessly tuning prompts with no improvement.
Integration layer: mismatch between protocol and tool descriptions. AI coding tools drive models through function calling or equivalent tool-description protocols. Different models follow tool schemas to different degrees: some prefer to stuff all parameters into a single string field, some ignore required fields, and some degrade to plain-text answers when the tool description is verbose. This isn’t a capability problem — it’s a matter of how well the prompt matches the schema.
Loop layer: insufficient parsing fault tolerance. An agent’s execution is a loop of “model output → parse → execute tool → fill back result.” If the parser has zero tolerance for format deviations, a single extra piece of explanatory text can fail the entire round, which in practice feels like “it crashes at the slightest thing.”
Task layer: context organization and long-range consistency. The longer the context, the more early constraints get diluted. This shows up as changing character settings midway, forgetting coding conventions, or repeating already-completed work.
Expectation layer: interaction paradigm mismatch. Users accustomed to conversational Q&A tend to give vague instructions in agent mode, but agent mode requires clear goals, constraints, and acceptance criteria.
2. Standard Steps for Integrating an AI Coding Tool
Before judging whether a model is good, make sure the integration pipeline itself is clean.
1. Layered adaptation: isolate model differences in one place
Add a thin adapter layer between the model and the tools, handling message formatting, tool schema conversion, and output parsing centrally, rather than scattering model-specific quirks across business code.
# Pseudocode: converge model differences into the adapter layerdef build_request(task, tools, cfg):return {"model": cfg.model_id,"temperature": cfg.temperature, # for agent scenarios, 0~0.3 is recommended"max_output_tokens": cfg.max_tokens, # leave room for tool calls + prose"tools": [normalize_schema(t) for t in tools],"messages": trim_context(task.history, cfg.context_budget),}def parse_output(raw):# Parsing priority: structured call > fenced json > regex fallbackreturn extract_tool_call(raw) or extract_json_block(raw) or fallback(raw)
2. Tuning checklist for parameters and prompts
| Dimension | Common setting | Notes |
|---|---|---|
| Temperature | 0~0.3 | Agent and coding tasks need stable output |
| Max output | Covers the longest single output | Truncation produces incomplete tool-call JSON |
| Timeout & retry | Exponential backoff, 2~3 attempts | Distinguish rate limiting, timeouts, and parsing failures |
| System prompt | Clear role, constraints, acceptance criteria | Reasoning models are more sensitive to system prompts; keep them concise and free of redundancy |
| Tool description | One sentence stating purpose and side effects | Vague descriptions are the primary cause of wrong tool calls |
| Structured output | Prefer JSON Schema constraints | Reduces work at the parsing layer |
3. Establish observability
Without trace replay you can’t attribute failures. At minimum, log each round’s tool name and inputs, return codes, token consumption, total number of rounds, and termination reason. This data will be reused in the later evaluation step.
3. Packaging Domain Knowledge as “Skills”
The essence of a Skill is: use a conventional directory structure to hold an on-demand-loaded domain instruction document, with optional scripts and resource files. Its value is that it lets a general-purpose agent gain specialized capabilities without modifying the main program.
The key design is progressive disclosure:
- The resident part contains only metadata and a short description, at very low cost;
- The full instruction body is loaded only when the task semantically matches the description;
- The instruction body then references finer reference files, read on demand.
Therefore, whether a skill is triggered correctly depends almost entirely on how well the description is written — the description must state “what it does” and “when to use it,” not just list feature nouns. A bad example is “novel generation tool, supports multiple genres”; a good example is “use when the user asks to continue or plan a long-running serial, or needs to keep characters and settings consistent.”
Directory structure example:
skills/serial-novel/├── SKILL.md # name + trigger description + workflow overview├── references/│ ├── structure.md # volume and chapter structure spec│ ├── style.md # writing style and dialogue constraints│ └── checklist.md # pre-submission self-check items├── templates/│ └── state.yaml # project state template└── scripts/└── validate.py # consistency validation
4. Extracting a Common Structure from Public Writing Skills
When surveying existing writing skills in code-hosting communities, the right approach isn’t to copy any single implementation, but to sample several and run a structured comparison, extracting the four questions they all answer:
| Dimension | Differences across implementations | Capability to abstract |
|---|---|---|
| Input | Some take only a one-sentence synopsis, some take a full outline | Layered input: synopsis → outline → chapter entries |
| State management | Most rely on conversation history, a few use external files | Externalize state, decouple it from long context |
| Output constraints | Word count, scene count, person, and tense control vary in granularity | Chapter-level hard constraints + prose-level soft constraints |
| Validation | Generally missing | An independent consistency-validation step |
The conclusion is clear: most open-source implementations fall short not in “prose prompts” but in lacking externalized state and a validation loop. This is exactly the part a self-built skill is most worth adding.
5. Designing a Skill for Web Novel Generation
1. Externalized state: a state file that can be repeatedly read and written
The main cause of long-serial failure is memory decay. Put settings, characters, timeline, and foreshadowing into a version-controllable structured file, read before each chapter is generated and written back afterward.
project: Starfall Scavengersgenre: sci-fi / wasteland / team growthchapter_word_target: 2500characters:- id: c01name: Lu Zhaogoal: find the navigation logs of the missing sisterflaw: over-relies on intuition, ignores dataspeech_style: mostly short sentences, few adjectivestimeline:- { chapter: 3, event: hatch scratches discovered, stakes: oxygen-cycle failure countdown }foreshadow:- { id: f01, planted_in: 3, promise: the numbering rule of the scratches, pay_off_in: null }
2. Chapter generation pipeline
Break it into four stages, each independently verifiable, to avoid the uncontrollable quality that comes from generating a whole chapter in one shot:
synopsis → volume outline → single-chapter entry (goal / conflict / turn / hook) → body → self-check revision
Constraint block example for a single-chapter call:
[Input] state.yaml + the chapter 4 entry of this volume's outline + 200-word summaries of the last two chapters[Hard constraints]1. Word count 2400~2800, scene count ≤ 22. Pay off at least one planted foreshadow, or plant a new one, and write it back to the foreshadow field3. The chapter must end with a hook, limited to one of: suspense / twist / countdown4. Dialogue must match character speech_style; the protagonist must not lecture at length[Output] body + an end-of-chapter update block (listing the fields changed this time and the reasons)
3. Consistency validation as an independent step
Self-check should not be done by the generating model evaluating itself, but by an independent call or script:
- Structural validation: word count, scene count, presence of a hook (scriptable, zero cost);
- Fact validation: whether character ages, locations, props, and the timeline conflict with the state file;
- Foreshadow ledger: whether any promises are past their deadline; alert the author once a threshold is exceeded;
- Style drift: compare sentence-length distribution and catchphrase frequency against the baseline.
# Structural validation snippetdef validate_chapter(text, state):errs = []n = count_chars(text)if not (2400 <= n <= 2800):errs.append(f"word count out of range: {n}")if not has_hook_tail(text):errs.append("missing end-of-chapter hook")for f in state.foreshadow:if f.pay_off_in is None and current_ch - f.planted_in > 30:errs.append(f"foreshadow {f.id} past deadline with no payoff")return errs
When validation fails, feed only the error list back to the model for targeted revision, rather than regenerating the whole chapter — this is key to controlling cost and keeping the style stable.
6. Use Metrics Instead of Feel to Judge “Ease of Use”
“Feels hard to use” must be turned into measurable items, otherwise tuning loses direction.
| Metric | Definition | Optimization direction |
|---|---|---|
| First-pass rate | Share of output that passes without human rework | Increase |
| Human edit ratio | Human-edited words / total words | Decrease |
| Consistency errors | Average fact conflicts per chapter | Approach zero |
| Average interaction rounds | Number of model calls to complete a task | Decrease |
| Per-chapter token cost | Total consumption including generation and validation | Keep controllable |
Fix a set of 5~10 representative tasks as a regression suite, and rerun it after each change to prompts, parameters, or skill docs, using the data to judge whether things have actually gotten better.
7. Engineering and Cloud Support
When generation tasks move from a single-machine script to a service, capabilities can be organized by common cloud product categories: use object storage to save the material library and finished versions, use a message queue to make long-running generation tasks asynchronous and to smooth out peaks, use a container platform to host stateless decoding and validation services, use a logging service to retain complete call traces, and use monitoring and alerts to set thresholds on failure rate and latency. On the model side, you can rely on model services and compute resources from platforms such as Baidu AI Cloud, choosing different parameter sizes and inference configurations as the business requires.
It’s worth emphasizing: no matter which model you integrate, the three things — the adapter layer, externalized state, and a validation loop — must be built by yourself. They are the transferable core assets.
8. Summary
A new model being “hard to use” is, most of the time, the result of integration and engineering not keeping up with the model’s capability iterations. The effective order of response is: first isolate model differences, and fix parsing fault tolerance and parameter configuration; then distill domain knowledge into skills with trigger descriptions and on-demand loading structures; finally, use externalized state, pipeline decomposition, and independent validation to solve long-range consistency, and drive continuous iteration with quantitative metrics. This methodology is model-agnostic — switch to any reasoning model or AI coding tool, and it can be reused directly.
