Practical Ways to Reduce LLM Token Consumption

    This article explains how to cut token usage when working with large language models. It covers constraining model output, trimming natural-language input, and converting repetitive structured data into a compact table-style format before it reaches the model. It also addresses guardrails, strategy selection by data shape, and monitoring.

    When you use large language models day to day, token consumption comes mainly from two sources. The first is prompts from users or systems that are too long, repetitive, or vague. The second is structured data from your program, such as object arrays, configuration tables, and log fragments, that gets serialized as-is and placed into the context. With default JSON formatting, this second source often contains many curly braces, quotation marks, and repeated key names. These take up tokens and make the model’s parsing harder, which can encourage verbose answers or format drift. To reduce filler text and token usage at the same time, you need to optimize both prompt engineering and the way data is expressed.

    Constraining Output Behavior

    Start by constraining the model’s output. Defining the role, task boundaries, and prohibited actions in the system prompt works better than complaining afterward that the answer is too wordy. You can require the model to answer only the question asked, avoid repeating the user’s wording, skip boilerplate phrases and disclaimers, state directly when the answer is unknown, and default to short sentences and lists.

    For structured results, specify a single output format, such as key-value pairs only or table rows only, and provide positive and negative examples. Use parameters such as temperature and maximum output length together. Lower randomness when you need deterministic output, and set a reasonable cap so the model does not expand endlessly to appear complete. If the task allows, ask for the conclusion first, followed by a very brief justification, rather than building up to the answer.

    These constraints also consume a few tokens, but usually far fewer than the redundant text the model generates without them. They also improve first-attempt success and reduce the number of follow-up rounds.

    Compressing Natural-Language Input

    Next, compress the natural-language part of the input. Remove background, chat history, and repeated explanations that are unrelated to the current task. Condense facts confirmed across multiple turns into a short status summary instead of pasting whole passages.

    For code, logs, and documents, keep only the fragments directly relevant to the question, and label line numbers or field meanings instead of feeding in the entire file. Redundant adjectives, polite filler, and marketing language add almost no information in technical contexts, so remove them.

    If you must provide examples, follow the minimal runnable principle. One positive example and one boundary example are usually enough. There is no need to pile up similar samples.

    Representing Structured Data Compactly

    The most effective way to reduce tokens systematically is often how structured data is expressed. Data exchanged between programs should still use standard object serialization to preserve types and ecosystem compatibility. Before sending data to the model, however, convert it into an intermediate format optimized for tokens.

    The core idea is to treat an object array whose elements repeat the same keys as a table. Declare the table name, row count, and column names first. Then write the field values row by row, separated by delimiters. This removes the repeated keys and most braces and quotation marks from each row.

    For example, take the same batch of user records. The nested-object format repeats keys such as ID, name, and role for every record. The tabular format declares the columns once, and each row then needs only three values. For arrays with consistent length and identical field structure, this representation usually cuts symbol tokens noticeably. It also helps the model align column meanings, which lowers the chance of missing or misplaced fields.

    Guardrails for the Intermediate Format

    When designing this intermediate format, embed lightweight guardrails. State the array length explicitly so the model can check whether it read everything. Declare column names together in one place to reduce ambiguity. Where needed, give a short range of allowed values for enumerations. This metadata uses very few tokens, but it significantly reduces the risk of hallucination and truncation.

    The intermediate format serves only the prompt layer. Backend parsing, storage, and interface contracts should keep using conventional structured data, so business systems do not adopt a second serialization standard. The conversion should be reversible, or at least verifiable, so that no fields are lost when the model’s output is written back to the program.

    Choosing a Strategy by Data Shape

    Choose a strategy based on the shape of the data. Highly irregular, deeply nested trees are better kept hierarchical, but null values, default values, and annotation-only fields should be removed. Lists that are close to tables should be flattened first.

    Very long lists can be paginated or filtered by query conditions before being sent. In that case, the prompt should state that the data covers only the current page or the current filtered result. For values and identifiers, use short encodings whenever possible instead of long strings. Use one consistent precision for timestamps.

    If the same batch of data is referenced several times in one session, provide the complete compact table the first time. In later turns, refer to it only by row numbers or primary keys instead of pasting it again.

    Keeping the Format Readable for the Model

    Also consider the comprehension cost on the model side. Obscure abbreviations or delimiters without explanation may save tokens on the surface, but they increase the chance of misreading and lead to more rounds of back-and-forth. Compact formats should therefore remain easy to scan. Use common short English or Chinese words for column names, keep delimiters fixed, and add a one-line comment explaining the convention when needed.

    When comparing approaches, measure both token count and task accuracy, not only the length of the shortest string. For large arrays with consistent structure, compact tables often outperform loosely nested objects in both input length and extraction accuracy. For small objects or deep trees, the gains are smaller, so there is no need to force a conversion.

    Engineering Integration and Monitoring

    For engineering rollout, build the conversion as a separate layer. Business services keep producing standard objects. A gateway or prompt assembly module performs compression and guardrail injection before calling the model, then parses the response according to the agreed convention.

    You can also ask the model to fill in its output using the same table convention, so the program can validate row and column counts with simple rules. Caching identical prompt prefixes and reusing system-level constraints can further spread out fixed overhead.

    For monitoring, record input tokens, output tokens, retry counts, and format validation failure rates for each request. Use this data, not intuition, to decide whether to keep compressing.

    Conclusion

    Reducing LLM token consumption takes several measures working together: writing less irrelevant natural language, constraining the form of the output, and converting homogeneous arrays into tables. At the prompt layer, rewriting structured data as compact notation that declares column names, fills values row by row, and includes validation information such as length reduces cost and improves context utilization without losing information. The backend keeps using conventional data exchange, and the conversion happens only at the model boundary, so existing system contracts stay intact. In day-to-day work, keep comparing length and accuracy on real tasks, and avoid compressing for its own sake. That is how the gains stay stable.