Optimizing LLM Coding Costs: Using Free Tokens to Build a Mini Program

Why coding-assistant costs spiral

LLM coding tools often start cheap or free. Once you move from a few functions to a complete mini program, token use climbs quickly. Coding work is long-context, multi-turn, retry-heavy, and often fully generated. A single page may include requirements, API contracts, style rules, existing code, error logs, and review comments. After a weak generation, developers add more conversation, regenerate, or enlarge context, so the same background is billed again and again.

Calling a coding assistant is not fundamentally different from a general LLM API: you send messages or text, receive generated output, and pay for input and output tokens. Cost control is not a “magic prompt.” It is engineering: bound the task, unify the access layer, limit context, record usage, define fallbacks, and treat free quota as a development-time budget—not an infinite resource.

Treat a large free token grant as a development-period quota. While it is valid, rate-limited, and acceptable under the provider’s data terms, run non-critical, reviewable, rollback-friendly work on it. Production should still have a paid or self-hosted inference option so the product is not tied to a single quota. Quota rules, validity, rate limits, and billing must follow the provider’s official documentation.

Overall approach: a controlled development compute pool

To finish a mini program rather than scattered snippets, combine an abstract access layer, task grading, and consumption budgets.

Abstract access layer. Application code should not scatter raw requests. A single client should handle auth, timeouts, retries, logging, metering, and response parsing. Switching providers or model parameters then stays out of business logic. If you use Baidu AI Cloud LLM inference, follow official docs for auth, rate limits, billing, and data terms. The examples below use environment variables and generic HTTP so business code is not bound to one response shape.

Task grading. Split work into “high-value, must be human-reviewed” vs. “low-risk, batch-generatable.” Requirement breakdown, API-contract drafts, tests, comments, style drafts, and error explanations can use free quota first. Payment, permissions, privacy, core algorithms, and security boundaries should stay human-led with stricter review. Free tokens do not lower the quality bar.

Consumption budgets. Cap tokens per module—for example, max input length per page component, at most two retries per task, and a max output length per generation. This is an engineering guardrail against unbounded tasks, not a precise finance model.

Generic access: unified client and usage logs

Keys stay off the client; requests are traceable; failures retry when appropriate; usage is counted. Pseudocode below is provider-agnostic; adapt fields to the official response format.

API_ENDPOINT = env.LLM_ENDPOINT
API_KEY = env.LLM_API_KEY
MODEL_ID = env.LLM_MODEL_ID
function chat(messages, temperature=0.2, max_output=2048):
    payload = {
        "model": MODEL_ID,
        "messages": messages,
        "temperature": temperature,
        "max_tokens": max_output,
        "stream": false
    }
    headers = {
        "Authorization": "Bearer " + API_KEY,
        "Content-Type": "application/json"
    }
    response = http_client.post(API_ENDPOINT, json=payload, headers=headers, timeout=60)
    response.raise_for_status()
    data = response.json()
    text = data["choices"][0]["message"]["content"]
    usage = data.get("usage", {})
    log_usage(task_id, MODEL_ID, usage)
    return text

Keep long-lived keys in server-side environment variables; a mini-program frontend must not hold them. Inject model ID, endpoint, and key via env so development, test, and production stay isolated. Always log usage; if the API has no standard usage field, approximate from input/output length and keep task ID, time, module, retry count, and request character counts. Timeouts and max output prevent hung connections and runaway spend.

Handle errors in the client. Timeouts, rate limits, and transient errors get exponential backoff; parameter errors, auth failures, and content-safety refusals should not be retried blindly. Cap retries and log usage per attempt. Streaming fits interactive completion but complicates concatenation and interruption; batch jobs are easier to log and validate with non-streaming calls.

Cache by task type + hash of input summary + model parameters. Reuse repeated contract drafts, style templates, and error explanations. Hits skip the model. Never put privacy or secrets into a shared cache.

Mini-program workflow: from requirements to runnable code

Do not generate an entire project in one shot. Split into reviewable increments.

1. Requirements and data model. Have the model turn user stories into a page list, entities, API contracts, and acceptance criteria in a structured format (JSON or tables). Humans check business boundaries, roles, and exception flows. Output is short and high-value—a good fit for free quota.

2. Pages and components. Generate one page or component at a time. Include directory layout, interface types, style rules, and files that must not change. Ask for “change notes + full files + key test points,” not the whole repo. Static-check first, then preview locally.

3. Business logic and pure functions. Extract testable logic into pure functions and generate boundary cases. Short context and cheap tests help both cost and quality.

4. Adapter layer. Generate adapters that map backend payloads to view models, and use mocks for empty, loading, and error states. Humans confirm contracts; the model must not invent fields.

5. Tests and review. The model can draft unit tests, edge cases, and error copy, but tests must actually run. Before release, check security, privacy, accessibility, performance, and recovery. Review all generated code, especially authz, input filtering, uploads, payments, and log redaction.

Token-control checklist

1. Layer context. Separate system rules, project summary, and the current task. Do not attach the full codebase every turn. Periodically compress the project summary to directory notes, key APIs, and coding standards.

2. Incremental patches. Prefer diffs or edit hunks over repeating whole files. Humans verify patch scope before apply.

3. Cap output and temperature. Code generation usually wants low temperature and a hard max output. Open-ended design talk can be looser but should have its own budget.

4. Batch vs. split. Merge small tasks into one structured call when useful, but do not stuff unrelated context just to save round-trips.

5. Cache and templates. Reuse prompt, contract, and test templates; hash similar requests.

6. Monitor and alert. Per module, track daily requests, input/output tokens, retry rate, and cache hit rate. Spikes often mean unbounded context.

7. Degrade. When quota, rate limits, or the service fail, fall back to local templates, human writing, or cached results. Production must not depend only on free quota.

Risks and compliance

Free quota is not unconstrained. Confirm validity, rate and concurrency limits, data-use terms, and content-safety rules. Redact user privacy, secrets, and internal endpoints before sending data off-box. Generated code may have vulnerabilities, license issues, or outdated APIs—run dependency checks, static analysis, and human review. Do not embed long-lived keys in the mini-program client; proxy all model calls through a server with auth, rate limiting, and audit logs.

Pre-release checklist

• Keys only in server env vars, with rotation supported
• Timeouts, max output, retry caps, and fallbacks
• Logs for task ID, module, usage, retries, and error types
• Input privacy redaction and output safety checks
• Human review, tests, and dependency checks on generated code
• Per-module token budgets and anomaly monitoring
• Swappable inference config so quota policy changes do not take down the product

Free large token grants are a development accelerator. What actually finishes the project is unified access, task decomposition, usage monitoring, and quality review. Treat model calls as part of the engineering system, and you can ship a runnable, maintainable mini program with costs under control.