A Practical Handbook for LLM Applications: Prompts, Caching, and Skills
A practical engineering guide to building LLM applications, organized around the parts that matter most in production: controlling cost through prompt caching, writing prompts as maintainable specifications, packaging domain knowledge as reusable Skills, generating diagrams and 3D scenes through code, running an AI-native development lifecycle, and handling content provenance. It explains how cache hits are computed and how to protect them through prompt layering, then covers structured output, evaluation sets, defense against prompt injection, and the skill set required for AI Agent roles. Checklists, troubleshooting tips, and a 30-day action plan give you steps you can apply directly.
Why developers need a practical LLM handbook
LLM capabilities keep evolving quickly, but the skills developers actually need have already stabilized: integrating the model into a system in a way that makes calls predictable, costs controllable, outputs verifiable, and experience reusable. In practice, many teams write only a few dozen lines of calling code yet run into four recurring problems:
- Unpredictable bills: long documents get stuffed into prompts repeatedly, and input token volume inflates quickly.
- Hard-to-maintain prompts: as rules pile up, changing one line can break another use case.
- Repeatedly paid-for experience: a pitfall solved in one project gets hit again in the next.
- No content-provenance awareness: model-generated content is published directly, with no way to explain where it came from.
This article follows a “practical handbook” structure across seven parts: cost, prompts, Skills, multimodal generation, AI-native development, content provenance, and growth path. Each part gives you structures, criteria, or checklists you can apply directly.
Cost: read the “cache hit” metric
Why input tokens have two prices
In most platforms’ billing rules, input tokens are split into two unit prices: “cache hit” and “cache miss,” with the cache-hit price clearly lower. To understand why, start from the inference process.
When a model handles a request, it usually first performs a full computation over all input tokens, a step commonly called prefill. Its compute cost scales roughly linearly with input length and is the main cost driver in long-prompt scenarios.
If consecutive requests share an identical opening — the same system instructions, the same long document — that repeated computation is unnecessary. So the platform caches the computed result for this shared prefix. When a later request hits the cache, that repeated computation is skipped. The saved compute shows up as a lower unit price, which is how the two input-token prices arise.
How cache hits work
Implementation details differ across platforms, but two ideas are common in the industry:
- Match by prefix: the platform splits the request prefix into fixed-length token blocks, hashes them, and compares against existing cache entries, matching block by block from the front.
- Longest common prefix hit: matching proceeds block by block from the start of the request and stops at the first block that differs. All preceding blocks hit; all following blocks miss.
From this, three conclusions follow:
- Position decides the outcome: the same content placed at the prefix versus at the end has completely different cost implications.
- The prefix must be identical: even a single extra space or line break turns everything after it into a cache miss.
- Caching is an optimization, not a guarantee: caches have a lifetime; when they expire, are evicted, or are unavailable on the platform side, the request automatically falls back to full computation. Caching can only make calls faster and cheaper; it cannot become a correctness dependency.
Layer the prompt: stable first, volatile last
Arranging each request’s content in descending order of “change frequency” is the most effective way to improve the hit rate:
| Layer | Content | Change frequency | Position |
|---|---|---|---|
| Stable layer | Role definition, output spec, tool descriptions, fixed examples, knowledge documents | Very low | Frontmost |
| Conversation layer | Multi-turn dialogue history | Appended each turn | Middle |
| Volatile layer | Current user input, real-time retrieval results, timestamps, task parameters | Different every time | Last |
The sample code below shows this organization. It only illustrates the calling side’s organizing principle and doesn’t represent any specific platform’s API.
# Sample code: only illustrates the "stable first, volatile last" organization principleSYSTEM_PROMPT = """You are an enterprise knowledge base assistant. Follow these rules:1. Answer only based on the content in [Reference Material]; when the materialdoesn't cover it, explicitly state "Not mentioned in the material";2. Answer conclusion-first and explain in points;3. After each conclusion, mark the source with [number], where the numbercorresponds to the order of the reference material."""def build_messages(question, retrieved_docs, history):docs = "\n".join(f"[{i}] {d}" for i, d in enumerate(retrieved_docs, 1))return [{"role": "system", "content": SYSTEM_PROMPT}, # Stable layer*history, # Conversation layer: append only, don't rewrite{"role": "user", # Volatile layer: different every time, placed last"content": f"[Reference Material]\n{docs}\n\n[Question]\n{question}"},]
A few details are worth noting:
- Append to multi-turn dialogue; never rewrite history. Summarizing, correcting, or trimming history changes the prefix and forces the entire history back into cache-miss.
- Try not to put the current date in system instructions. When time information is needed, put it in the volatile layer, or have code add it after the answer is generated.
- Put retrieval results in the volatile layer; this naturally isolates context and reduces the risk of retrieved external text being executed as instructions (see section six).
- For long documents, prefer the platform’s file-level caching capability. If you can only put them in the prompt, keep them at a fixed prefix position to avoid repeated transmission and billing on every request.
Practices that quietly lower your hit rate
- Variables at the start: putting a user ID, timestamp, or random number at the very front means every request misses.
- Dynamically assembled prompts: adding a temporary instruction today and deleting it tomorrow means the prefix is always changing.
- Using a conversation ID for cache isolation: caches are usually keyed on request content, and many platforms also let callers pass a custom key to distinguish different tenants’ contexts. If you design this key to be “a new value per turn,” you artificially lower the hit rate.
- Unstable formatting: extra spaces mixed into templates, inconsistent line breaks; the same prompt assembled through different code paths.
- Unverified prompt updates: overwriting the production system prompt directly. Such changes often affect both answer quality and cache behavior, yet no one knows in advance.
Turn the hit rate into an observable metric
Most platforms return a token-usage breakdown in the response, and the input side usually distinguishes cached from uncached portions (the exact fields follow the platform you use). It’s worth collecting at least these metrics:
- Cache hit rate: cached input tokens ÷ total input tokens.
- Uncached input token volume: directly determines most of the input cost.
- Cost per request: cache-hit tokens times their unit price, plus cache-miss tokens times their unit price.
- Time to first token: the metric users notice most. After a long prefix hits the cache, wait time typically drops noticeably.
On Baidu AI Cloud, you can pipe call logs into the log service and use monitoring and alerting to set thresholds on these metrics. For example, when the hit rate drops 30% below its 7-day average, an automatic alert often catches a changed prompt or a broken template before users complain. Before each prompt release, it’s also worth comparing the old and new versions on the same sample set for both hit rate and answer quality before deciding to ship.
Privacy and compliance boundaries behind caching
A cache hit means some content may briefly exist, in a reusable form, inside the platform’s cache system. Before connecting enterprise data, do three things:
- Confirm the service’s terms on data retention, cache isolation, and training use;
- Avoid putting sensitive information such as ID numbers, bank card numbers, or keys into requests;
- Desensitize business data that must be input.
Prompts: from stacked tricks to a maintainable spec
Organize prompts with a four-layer structure
- Task layer: state the goal with a leading verb, e.g., “Extract the party A/B names, amount, and payment terms from the contract.”
- Constraint layer: language, length, terminology, prohibitions. Constraints must be checkable: “no more than 200 words” beats “keep it brief.”
- Format layer: give field names, types, value ranges, or a complete output example.
- Boundary layer: what to do when input is missing, content conflicts, or coverage is incomplete. E.g., “If any field is missing, set it to null; don’t guess.”
Writing the boundary layer clearly beats repeatedly stacking “please be careful” and “don’t make mistakes,” because the model needs explicit behavioral rules, not attitude requirements.
Few-shot examples: fewer, but cover the key branches
The value of few-shot examples is in demonstrating the input-to-output mapping. Rather than ten similar samples, use three: a normal one, one with distractor information, and one that needs a refusal. The example’s format must strictly match the target output; internal contradictions in the example leave the model at a loss.
Structured output: make results consumable by programs
If results feed into a business system, rather than asking the model to “return JSON,” specify down to the field level:
{"contract": {"party_a": "Sample Company A","party_b": "Sample Company B","amount": 100000,"currency": "CNY","payment_days": 30},"missing_fields": [],"need_review": false}
Three key points:
- Field names match the business model;
- Dates and amounts use a uniform format and type;
- Reserve fallback fields like
missing_fieldsandneed_reviewso that “uncertain” becomes a processable structured result.
For stronger constraints, choose a platform capability that supports structured output or tool calling. But even with constraints, validate on the code side: are enum values legal, are numbers in a reasonable range, do required fields exist.
Manage prompts like code
- Versioning: prompts go into the repository, reviewed and rolled back together with business code.
- Evaluation set: prepare 50–100 cases covering typical and boundary scenarios, each annotated with expected output or scoring points.
- Automatic scoring: rule matching and field validation can be judged by programs; open-ended questions can be scored by a model against the scoring points.
- Release gate: a new version ships only if quality is no lower than the old version and the hit rate doesn’t drop noticeably.
- Canary: observe a small traffic slice for a while, then gradually expand.
Context isn’t better the more you stuff in
Piling retrieval passages, tool descriptions, and dialogue history together dilutes the truly relevant information, pushes up input cost, increases the prompt-injection attack surface, and can trigger truncation when it gets too long. The practical principle is “provide on demand”: include only fragments relevant to the current question, and explain in the prompt how the material is organized. For example: “arranged from most to least relevant; prioritize the earlier material.”
Capability packaging: turn domain experience into a Skill
What is a Skill
A Skill is a “capability package” that typically has four parts:
- Metadata: name, applicable scenario, description;
- Operating instructions: a step-by-step process for this kind of task;
- Reference material: templates, specs, background documents;
- Optional scripts: programs that can be included when precise computation or file processing is needed.
An analogy: a prompt is like a sticky note on your monitor, full of one-off reminders; a Skill is like a team’s working manual — accumulated over time, version-managed, and consulted on demand. The model’s main prompt is like an employee’s job description; a Skill is the operating manual for a specific task.
A common Skill design in the industry uses progressive loading:
- Normally, only “name + short description” goes into context, which is cheap;
- When a Skill is judged applicable, its full instructions are read;
- Templates, reference material, or scripts are loaded only when needed.
This avoids stuffing all knowledge into the main prompt at once and makes it easy to share and reuse across teams.
Skill vs. long prompt vs. hard-coded tool calls
| Dimension | All rules in main prompt | Skill | Hard-coded flow tool calls |
|---|---|---|---|
| Context cost | Always occupied | Loaded on demand | Small |
| Scope | General | Reused per task | Fixed task |
| Maintenance cost | A change ripples everywhere | Independent version, independent review | Requires code change |
| Flexibility | High | High, portable across models | Low, fixed flow |
The three aren’t mutually exclusive. Deterministic computation suits scripts; judgment and generation steps suit Skill descriptions; general behavior stays in the main prompt.
A Skill structure example
The example below demonstrates a Skill for “explaining a complex concept to a beginner.” This kind of explanatory Skill is common inside teams: whenever a technical person needs to explain a technical solution to product, operations, or a customer, they can reuse it.
---name: explain-simplydescription: Explain a technical concept to a non-technical reader; build intuition first, then go into detail---## When to useThe user asks about an unfamiliar technical term, or asks to "explain it in plain language."## Steps1. Give a definition in one sentence of no more than 30 words;2. Explain the core mechanism with an everyday analogy, and make clear what the analogy maps to;3. Illustrate the real effect with one concrete example;4. Finally, add common misconceptions and applicability boundaries.## Constraints- Build intuition first, then explain the principle; don't lead with jargon;- For a term that must be used, add a parenthetical explanation on first appearance;- The analogy must point out "where it is similar and where it is different," to avoid giving the user a wrong mental model.## ExampleInput: What is a message queue?Output:- One-sentence definition: A message queue is like a "task transfer station" where producers drop in tasks and consumers take them out at their own pace.- Everyday analogy: It is like the pickup counter at a restaurant. The chef finishes a dish and puts it on the counter; the server picks it up when free — neither waits for the other. The difference: food at the counter goes cold, but a message in the queue can be redelivered until it is processed successfully.- Real effect: During a flash-sale traffic peak, the order service drops tasks such as "deduct inventory" and "send notification" into the queue; the backend consumes them at its processing capacity, so the frontend does not have to wait.- Boundary: The queue solves "peak shaving" and "decoupling"; it does not make processing faster. When messages pile up, you also need backlog monitoring and failed-retry handling.
In this example, three things do the real work:
- Steps: define what comes first and what comes next, which determines the output structure;
- Constraints: require the analogy to state the differences, avoiding a wrong mental model;
- Example: provide an input-to-output reference.
Key points for writing a Skill
- Description decides the trigger rate: describe the applicable scenario in task-oriented language; list “not applicable” cases where needed to avoid false triggers in unrelated tasks.
- Steps must be executable: avoid unverifiable phrasing like “analyze carefully” or “handle properly.”
- Define inputs and outputs: what materials are needed, and what form the output takes.
- Draw boundaries: for high-risk judgments involving compliance, legal, or financial matters, note “provide reference information and recommend human review.”
- Stay concise: the Skill itself is context; verbose instructions crowd out attention.
- Manage at scale: when there are many Skills, build a catalog and retrieval via metadata, match on name and description, then load the corresponding content.
Skill governance
- Skills go under version control; during review, check both content correctness and prompt-injection risk.
- External material referenced by a Skill must be annotated with source and update time.
- Skills with bundled scripts should restrict file read/write scope and network access, and must not write plaintext keys.
- Each Skill gets a set of test cases; run a regression after updates.
Multimodal: let the model “write code to draw”
Generate an image directly, or generate code?
For graphics with a clear structure, having the model output executable code is often more practical than generating an image directly:
- Vector graphics: code can output vector formats like SVG, which scale without distortion and whose elements can be modified by later programs.
- Web and animation: code runs directly in a browser, making iteration and version management easy.
- 3D scenes: the model can write scripts that generate geometry, set materials and cameras in 3D software, then a human fine-tunes details.
Note that model-generated 3D scripts usually rely on the target software’s own scripting interface, which can vary across versions. Confirm the runtime environment before generating, and validate with a local render after.
Structured constraints: make output controllable
Take a web animation: specifying the following in the prompt significantly reduces rework:
- Technical boundaries: are external resources allowed, what technology implements the animation, is mobile compatibility required.
- Visual specs: resolution, color scheme, font size, corner radius, spacing.
- Interaction specs: hover/click feedback, animation duration and easing.
- Performance requirements: target frame rate, upper limit on first-screen resource size.
- Accessibility: text-to-background color contrast, keyboard support.
A typical 3D modeling flow
- Describe the object: the subject, size proportions, placement, material texture, lighting mood.
- Specify parameters: use uniform units and a coordinate convention. E.g., meters as the unit, Y-axis up, with explicit length/width/height values.
- Generate the script: define parameters centrally in the script, and write size and position as adjustable variables for easy fine-tuning.
- Validate with a local render: check proportions, occlusion, materials, and camera angle.
- Iterate with feedback: use a screenshot of the render as feedback and describe the specific problem, e.g., “the desktop is about twice as high as the chair” or “the window is blocked by the wall.”
3D content especially tends to have the “looks about right, but the proportions are off” problem. Rather than vague feedback like “make it nicer,” use quantifiable descriptions: “change the desktop height from 1.5 m to 0.75 m.”
Quality checklist for graphic generation
- Renders correctly: code has no syntax errors and runs directly in the target environment.
- Sound geometry: uniform units, coordinated proportions, no mesh clipping.
- Meets visual standards: color, contrast, and font size match the design intent.
- Controllable performance: file size, element count, and render frame rate are within acceptable ranges.
- Maintainable: key parameters managed centrally, clear naming, with comments.
Process: the AI-native software development lifecycle
What is AI-native development
AI-native development isn’t “using one more assistant while writing code.” It treats AI as a participant across the whole flow and redesigns how requirements, design, coding, review, testing, release, and operations collaborate. The core shift: human value moves from “writing line by line” to “defining the problem, setting standards, judging results.”
Human-AI division of labor across stages
| Stage | AI is suited to | Humans must own |
|---|---|---|
| Requirements | Clarifying questions, mapping boundaries, generating an acceptance checklist | Business goals, priorities, trade-offs |
| Design | Comparing alternatives, interface sketches | Architecture decisions, risk judgment |
| Coding | Boilerplate, unit tests, refactor suggestions | Critical logic, security-related code |
| Review | Initial review, spotting omissions | Final merge decision |
| Testing | Test-case design, boundary data generation | Acceptance criteria, classifying production issues |
| Release & ops | Log analysis, initial incident triage | Change approval, emergency decisions |
Six key practices
- Spec first: write requirements as a verifiable checklist before letting AI code. When requirements are vague, AI only amplifies the vagueness into lots of rework.
- Convention files: maintain documentation in the repository stating the tech stack, directory conventions, commit conventions, prohibitions, and how to run tests, so every collaboration shares a baseline.
- Verifiable loop: AI’s output must be verifiable automatically or semi-automatically — unit tests, type checks, static scans, screenshot comparison. AI coding without a verification step is essentially accumulating hidden debt.
- Small commits: let AI complete one independently verifiable small goal at a time; problems are easier to locate.
- Review AI-generated code too: license compliance, dependency introduction, and missing error handling are common problems.
- Metrics-driven improvement: watch standard engineering-efficiency metrics such as change lead time, deployment frequency, change failure rate, and recovery time — not “how many lines of code AI generated.”
Security baseline: prompt injection and least privilege
When a system connects search, web reading, file parsing, and similar capabilities, the model touches large amounts of external content. An attacker may hide instructions in web pages, documents, or code comments to induce the model to leak the system prompt or perform unauthorized operations. Defense should at least do:
- Content isolation: explicitly label external content as “data to be analyzed,” not instructions.
- Least privilege: whitelist-validate tool-call parameters and restrict the readable/writable directory scope.
- Human confirmation for high-risk operations: delete, transfer, publish, and similar actions must be confirmed by a human before execution.
- Output filtering: sensitive information doesn’t go into logs; links in external content aren’t handed to users to click directly.
Content: the principle, boundaries, and handling of invisible watermarking
Why content provenance is needed
Once model-generated content enters blogs, official documents, coursework, or code repositories, its origin is often hard to explain. The value of content provenance: a platform can prove where content was generated, an enterprise can distinguish human versus machine output and assign responsibility, and readers can judge information credibility accordingly.
The general idea of text watermarking
A representative idea in the industry is “embed at generation, verify at detection”:
- At generation: for each step’s candidate words, use a key held only by the model provider to perform a pseudo-random partition, e.g., split into “preferred words” and “alternate words,” and slightly raise the probability that a preferred word is chosen. A single word shows almost no difference, but the share of preferred words in the whole text is slightly higher.
- At detection: recompute the preferred-word share of this text using the same key, then use a statistical hypothesis test to judge whether the hypothesis “this was randomly generated” holds, and output a confidence level.
Its advantage is that it barely affects readability; its drawback is that the statistic is essentially a probabilistic judgment, not a definitive yes/no conclusion.
Addendum: finer-grained provenance mechanisms
It’s worth stressing that watermarking is just one class of technique. In practice, content provenance is usually split into two layers:
- Explicit provenance: label “this article was AI-assisted” on the page, or write the generation tool and time into file metadata. The advantage is that it’s direct; the drawback is that it’s lost once content is copied and pasted.
- Implicit provenance: i.e., the watermark, embedded in the content itself and independent of the carrier. The drawback is that detection needs the key, and it can be broken.
Only by combining the two layers can you cover both needs: “users can see it” and “the platform can verify it.” As for whether a watermark is public and how to appeal, policies vary widely across platforms; when using third-party content, follow the relevant platform’s documentation.
The limits of watermarking
- Short text is unreliable: statistical tests need enough text; a few dozen words often can’t reach a credible level.
- Rewriting breaks it: after translation, deep rewriting, or re-summarizing, the watermark signal is greatly diluted or fully destroyed.
- False positives are possible: statistical tests always have some false-positive probability; don’t make a determination about any individual based solely on a detection result.
- The key is the Achilles’ heel: once the key leaks, the watermark’s discriminating power drops sharply.
So a more sound understanding: a watermark is one link in the content-provenance chain, not a synonym for an “AI content detector.”
What enterprises and individuals should each do
Enterprise side:
- Add an “AI involvement” field to the content publishing flow, clearly marking the human reviewer responsible;
- Write labeling requirements clearly into external publishing norms, e.g., AI assistance must be disclosed;
- Bring model-generated code into regular license scanning and security scanning.
Individual side:
- In coursework, official documents, submissions, and similar scenarios, follow the disclosure rules of your organization or platform;
- Treat model output as a draft, not a final version; verify key facts and data;
- When you receive a “detected as AI-generated” conclusion, keep materials from the writing process (drafts, edit history) so you can explain.
Skills: what AI Agent developer roles need to fill in
Judging from hiring trends, demand for AI Agent development roles is growing clearly. When the tech stack changes, many people’s first reaction is “I don’t know where to start learning.” Rather than chasing frameworks, fill in skills along one main line:
- Base calling layer: message structure, streaming output, structured output, error retry, timeout control.
- Context layer: prompt layering and caching strategy, retrieval augmentation, long-document chunking and recall evaluation.
- Tool layer: tool definitions and parameter validation, parallel calls, failure rollback, permission control.
- Planning layer: task decomposition, separation of planning and execution, reflection and retry strategies, termination-condition design.
- Evaluation layer: offline eval sets, online metrics, failure-sample attribution. This is the part that best distinguishes candidate depth in interviews.
- Engineering layer: containerized deployment, logging and tracing, cost control, canary release.
For learning order, first solidify layers 1, 2, and 5, build a small project that “runs, can be evaluated, and whose cost is clear,” then extend to tools and multi-agent collaboration.
Common interview questions mostly revolve around “how to measure effectiveness” and “how to locate problems,” not “which framework you use.” For example: how to design an eval set? How to tell whether it’s a model-capability problem or a prompt problem? How to troubleshoot a drop in cache hit rate? How to retry a failed tool call without double-charging?
Quick-reference troubleshooting checklist
- Answer quality suddenly drops: check whether the prompt was changed, whether retrieval recall changed, whether the model version was updated.
- Cost suddenly rises: check input length, whether long documents were added, whether the hit rate dropped.
- Unstable output format: add field-level descriptions and one complete example, and enforce strong validation on the code side.
- Agent stuck in a loop: set a maximum step count and maximum cost cap, and record the decision basis at each step.
- Inaccurate retrieval results: check chunking strategy, recall count, and reranking separately; run controlled experiments with a fixed question set.
- Worried about content being misjudged as AI-generated: keep process materials, follow disclosure norms, and don’t rely on a single detection conclusion.
Implementation: a 30-day action list
- Week 1 — establish a baseline: map all calling points, record token usage and per-call cost for each scenario, and identify the three most costly scenarios.
- Week 2 — layer the prompts: refactor prompts into the stable/conversation/volatile three layers, launch a hit-rate dashboard, and configure alerts.
- Week 3 — accumulate Skills: turn the team’s repeated explanations and repeated operations into 2–3 Skills with test cases.
- Week 4 — process and provenance: introduce convention files and review rules for AI-generated content into the development process, and clarify content labeling requirements.
The gap in LLM applications often lies not in whether you can call the API, but in whether you can turn every call into a measurable, reusable, traceable engineering asset. Starting from the single cache-hit metric and gradually linking prompts, Skills, process, and provenance into a system is the sustainable path to adoption.
