Concise LLM Outputs: Practical Ways to Lower Token Costs

1. Why verbose output becomes a cost problem

Large language model services are typically billed by token, and both input and output count. On most services, output tokens cost more per unit than input tokens, so every extra paragraph adds directly to the bill. Longer output also takes longer to generate, which increases user wait time.

Verbose answers usually come from four sources:

• Greetings and filler, such as “Sure, let me take a look at this.”
• Restating the question and describing again what the user already said.
• Over-explaining background the user does not need.
• Repeating the conclusion by summarizing the same points at the end.

These parts add almost no information, but they consume a large number of tokens.

2. Understand what the cost is made of

Before optimizing, be clear about what a single request costs.

1. Input tokens: the system prompt, conversation history, content returned by external tools, and the user’s current question.
2. Visible output tokens: the answer text the user can read directly.
3. Hidden reasoning tokens: tokens some models generate while reasoning internally. These are usually billed, but the user cannot see them, and asking for a “short answer” does not compress them directly.

Concise visible output therefore optimizes only part of the cost. If the task itself needs a long reasoning process, shortening the visible answer will not significantly reduce the total charge.

3. Core principles of concise output

Concision is not deleting information. It is removing text that carries no information. In practice, follow four principles.

Lead with the conclusion. The first sentence should state the conclusion or the solution. Put the reasoning and details after it so the reader can tell in one sentence whether the answer solves the problem.

Remove restatement and filler. Do not repeat the user’s question, do not write openings or closings, and do not summarize again at the end of a section.

Keep necessary information. Reasons, key steps, preconditions, risk warnings, and runnable code must stay. Dropping risk warnings is the most common over-compression mistake.

Use structure instead of long sentences. Lists, numbered steps, and code blocks use fewer tokens than long paragraphs and are easier to read. Keep symbols and abbreviations within what the reader can understand.

Take a front-end performance issue. A verbose answer spends a full paragraph explaining the rendering mechanism. A concise version can look like this:

Every render creates a new object reference. When that object is passed to a child as a prop, the child’s shallow comparison treats it as changed and re-renders. Cache the object with memoization to fix it.

The two versions convey the same information. The second uses noticeably fewer tokens and still keeps the cause-and-effect chain intact.

4. Practical methods

4.1 Constrain style in the system prompt

Writing output rules in the system prompt is more stable than repeating them in every turn. For example:

CONCISE_STYLE = """
Answer requirements:
1. Give the conclusion first, then only the necessary basis.
2. Do not restate the user's question, and do not write an opening or a closing.
3. Code must be complete and runnable; explanations must not exceed three sentences.
4. If information is insufficient, ask only one clarifying question.
5. Risks involving safety, data loss, or cost must be stated explicitly.
"""
def build_messages(user_input: str, history: list[dict]) -> list[dict]:
    messages = [{"role": "system", "content": CONCISE_STYLE}]
    messages.extend(history)
    messages.append({"role": "user", "content": user_input})
    return messages

The system prompt itself is also input tokens. It is sent on every request, so keep it short and retain only rules that actually constrain the output. Some services cache a fixed prefix and can reduce this repeated cost. Follow the provider’s official documentation for the exact rules.

4.2 Control length by task type

Different tasks need very different output lengths. Set a separate output policy for each type:

• Simple Q&A: one or two sentences.
• Troubleshooting: cause, localization steps, and fix.
• Code generation: complete code plus necessary comments.
• Solution design: the recommended option first, then the trade-offs.

You can apply these levels by attaching a task-type label to the request, instead of using one rule set for every request.

4.3 Use output length limits carefully

A maximum output token count is a hard cap, but answers that exceed it are cut off. Truncated code will not run, and a truncated conclusion may lose critical conditions. If you set a cap, also tell the model in the prompt to prefer completeness, and detect truncated results.

4.4 Let the user decide when to expand

Keep the default answer concise, and offer an expansion path at the end—for example, the user types “explain in more detail” to receive the full process. Most requests then save tokens, while a complete answer remains available when detail is needed.

5. Cost estimates and quality checks

5.1 Calculate net savings

A concise system prompt adds input tokens, while concise output reduces output tokens. The two effects must be combined. Estimate cost as follows:

def estimate_request_cost(
    input_tokens: int,
    output_tokens: int,
    price_in: float,
    price_out: float,
) -> float:
    """Return the cost of one request. Unit prices are per million tokens."""
    return (input_tokens * price_in + output_tokens * price_out) / 1_000_000

Suppose, in one scenario, the system prompt adds 150 input tokens and each answer saves 400 output tokens on average, and the output unit price is four times the input unit price. This is an example assumption only; use the provider’s actual prices. With the input unit price set to 1, the net gain per request is:

400 × 4 − 150 × 1 = 1450

If the output savings are large enough, the fixed prompt overhead is offset. If answers are already short, the benefit of concision rules is limited.

5.2 Evaluate answer quality

Token count alone gives a one-sided result. Also track:

• Average output tokens and average cost per request.
• Task completion rate: whether the user can solve the problem without a follow-up.
• How often users edit the answer or ask again.
• Missing critical information, such as whether risk warnings were kept.

Run a controlled comparison on a set of real tasks, with and without the rules, and compare how these metrics change.

6. Boundaries and risks

Concision rules have several risks worth watching.

• Over-compression creates ambiguity. Dropping the subject or using vague abbreviations to save space can leave the reader unsure what is being referred to.
• Critical preconditions are removed. In database migrations, permission changes, and cost-related scenarios, preconditions and risk warnings cannot be omitted.
• Readability suffers. Readers who lack context must work harder to interpret heavily compressed text, which raises communication cost.
• The reasoning trail is hidden. When the reasoning chain must be reviewed, such as in code audits or data analysis, an explanation that is too short is harder to verify.

The floor for concision is that the reader can understand and act accurately, not that the text is as short as possible.

7. How to roll it out

1. Measure the output-length distribution of current requests and find the most verbose task types.
2. Write concise output rules for those tasks, and keep requirements for necessary risk warnings.
3. Run a controlled test on real samples and record cost, completion rate, and follow-up count.
4. Adjust the rules from the results and remove items that have no real effect.
5. Keep an expansion path for cases that need a detailed explanation, and review periodically whether the rules still apply.

Concise output is engineering work that needs ongoing adjustment. A rule is valuable only if it lowers cost while keeping answers accurate and readable.