How to Raise Gross Margin in the MaaS Model for LLMs: Engineering and Business Practices

    This article explains why revenue growth alone does not guarantee profit for large language model MaaS (Model as a Service) platforms, and which levers can move gross margin. It first covers the shift from project-based delivery to API-based, usage-driven revenue, and why ARR (Annual Recurring Revenue) is the best indicator of MaaS business health. It then analyzes the root cause of the "revenue without profit" problem: revenue and cost are tied to the same variable, token volume. On this basis, it presents two levers for improving gross margin: engineering techniques that lower inference cost per token, and customer-mix strategies that increase the share of high-value customers. It also shows how to build a simple unit economics model to quantify gross margin and make it a manageable metric. The goal is to help engineering teams and platform operators move from "feeling" margins to "measuring and managing" them.

    1. From Project-Based Delivery to MaaS: A Shift in How LLM Companies Make Money

    For a long time, many large language model companies operated more like project-based software vendors. Their revenue was the sum of one custom delivery engagement after another. These deals were large, ran on long timelines, and were hard to replicate. This model has one major weakness: it is not sustainable. Once a project is completed, the revenue curve falls, and growth depends on constantly landing the next big contract.

    The MaaS (Model as a Service) model changes the equation. Its core product is an API that packages model capabilities into a standardized, measurable service. Enterprise customers and developers pay for it as they consume tokens. As a result, revenue comes not from delivery but from ongoing usage. As long as customers call the model reliably in real production environments, revenue is generated continuously and automatically over time.

    One leading LLM company reported the following in its annual results after going public:

    • Full-year revenue grew by about 130% year over year, accelerating for several consecutive years. Its revenue now ranks among the highest of LLM companies in China.
    • Annual recurring revenue (ARR) on its MaaS API platform rose about 60-fold year over year, reaching more than RMB 1 billion.

    Together, these two figures signal one thing: the growth engine has shifted from winning new projects to renewing ongoing subscriptions.

    2. Why ARR Is the Health Indicator for a MaaS Business

    ARR (Annual Recurring Revenue) originated in the SaaS industry as the core metric for subscription revenue. It answers one key question: How much of this company’s revenue will come back automatically tomorrow?

    For MaaS platforms, ARR is even more important. It measures whether customers are continuously calling models and steadily consuming tokens, rather than relying on one-off projects to pad the numbers. A high-ARR MaaS platform with low volatility is already embedded in customers’ real business processes and has become part of their day-to-day operations.

    When evaluating ARR, focus on the following dimensions:

    • Retention and repeat usage: Whether calls continue consistently, rather than spiking in the first month and then falling off quickly.
    • Customer concentration: Whether revenue depends too heavily on a few very large customers, and how well the business withstands risk.
    • Depth of usage: Whether individual customers expand from trial scenarios into deep use across multiple business lines and use cases.
    • Token consumption mix: Whether usage is concentrated in a few lightweight tasks or covers heavy workloads with high frequency and long contexts.

    ARR is stable and reliable when millions of enterprise customers and developers use a platform continuously in production across more than 200 countries and regions.

    3. The Root Cause of the “More Revenue, Not More Profit” Problem

    The MaaS model has long faced a typical question from the market: Will it fall into a trap where the more you do, the thinner your margins become?

    The root cause is that in MaaS, revenue and cost are tied to the same chain:

    1. revenue = tokens called × price per token
    2. cost ≈ tokens called × unit compute cost + fixed costs

    Token volume appears on both sides. When a model provider cuts prices to win market share while compute costs remain high, every doubling of scale compresses the gross margin once more. If unit compute cost does not fall as scale grows, expansion actually dilutes profit. This is the mathematical essence of the “revenue without profit” problem.

    There are therefore only two ways out: push the cost per token low enough, or increase the share of revenue contributed by high-value customers. These two paths correspond exactly to the two levers for improving gross margin.

    4. Lever 1: Use Engineering to Drive Inference Costs Down

    Improving inference efficiency is the first-order engineering challenge for MaaS gross margins. The core goal is to use as little compute as possible per million tokens while maintaining the same service quality (latency, throughput, and accuracy).

    Common engineering optimization directions include:

    1. Continuous (In-Flight) Batching
      Traditional batching waits until a full batch of requests has arrived before running inference, leaving large amounts of GPU capacity idle. Continuous batching dynamically inserts new requests during the decode phase and removes requests that have finished. This significantly improves GPU utilization. The scheduling logic can be sketched as follows:

      1. # Pseudocode: scheduling logic for continuous batching
      2. def scheduler(running_queue, waiting_queue):
      3. while True:
      4. # Re-pack the batch at each step to fill available capacity
      5. batch = pack_to_capacity(running_queue)
      6. outputs = model.generate(batch) # one forward pass covers multiple sequences
      7. evict_finished(batch) # release slots immediately when sequences finish
      8. admit_from(waiting_queue) # fill freed slots with new requests right away
      9. yield outputs
    2. KV Cache Reuse and Prefix Caching
      Many business requests share the same system prompt or long context prefix. Caching the KV Cache of these prefixes and reusing it avoids repeated computation. This directly reduces the overhead of the first-token (prefill) phase.

    3. Model Compression and Distillation
      Techniques such as quantization (e.g., INT8/INT4), knowledge distillation, and sparsification reduce the compute required per inference while keeping accuracy loss under control.

    4. Elastic Scheduling and Resource Pooling
      Use a container platform with auto-scaling to schedule inference workloads onto an elastic compute pool according to priority. This avoids reserving too many idle resources for peak load. Pair it with monitoring and alerting to track utilization in real time and adjust the number of instances dynamically.

    5. Tiered Storage
      Store model weights, caches, and call logs on storage tiers with different performance levels, such as high-performance block storage for hot weights and object storage for cold logs. This balances cost against latency.

    The right metrics for measuring optimization results are cost per token (RMB per million tokens) and GPU utilization. The former should keep falling as scale grows, while the latter should remain consistently high.

    5. Lever 2: Optimize Customer Mix and Increase the Share of High-Value Customers

    Cost optimization alone sets a ceiling on gross margin. Customer value segmentation is what truly separates the leaders.

    Low-value customers tend to concentrate in trial use, scraping, and heavy-discount promotion scenarios. They consume large numbers of tokens, but at low prices, with high volatility and poor retention. High-value customers, by contrast, embed model capabilities into their core businesses. Their usage is stable and prices are reasonable, so they are more likely to renew. Continuously increasing the share of high-value, top-tier customers can significantly improve overall gross margin.

    The data supports this approach. At leading LLM companies, the gross margin of their MaaS API platforms has risen from about 3.3% to about 18.9%, while overall gross margin has stayed above 50% for a long time. This improvement is not a coincidence. It results from two engines working together: falling inference costs and a rising share of high-value customers.

    Engineering can support this goal in the following ways:

    • Tiered metering and quota management: Apply differentiated rate limits and quotas for customers at different value tiers to protect the SLAs of high-value customers.
    • Usage observability: Use logging services and monitoring and alerting to identify healthy customers with high growth and high consumption. Reach out to them promptly with technical integration support.
    • Cost attribution: Accurately attribute compute costs to individual customers and business lines to quantify each customer’s true gross margin contribution.

    6. Build a Unit Economics Model

    To turn gross margin from a “gut feeling” into a manageable metric, you need a quantifiable unit economics model:

    1. def unit_economics(tokens_m, price_per_mtoken, cost_per_mtoken):
    2. revenue = tokens_m * price_per_mtoken
    3. cost = tokens_m * cost_per_mtoken
    4. gross_margin = (revenue - cost) / revenue
    5. return {
    6. "revenue": revenue,
    7. "cost": cost,
    8. "gross_margin": round(gross_margin, 4),
    9. }
    10. # Example: 1,000 million tokens, unit price of 10, unit cost of 8
    11. result = unit_economics(1000, 10, 8)
    12. print(result) # gross_margin ≈ 0.20, i.e., 20%

    This model points to four levers for improving gross margin:

    • Raise the price per token: Achieve this by serving high-value customers.
    • Lower the unit cost: Achieve this through inference optimization.
    • Optimize the usage mix: Reduce wasted consumption.
    • Spread fixed costs: Improve the scale utilization of resources.

    Every gross margin improvement can ultimately be traced to one of these four variables.

    7. Conclusion

    The core value of the MaaS model lies in turning LLM capabilities into sustainable, measurable service revenue. Whether it can overcome the “revenue without profit” challenge depends on two things: using engineering to push the cost per token to an industry-leading level, and continuously shifting the revenue mix toward high-value customers. When ARR climbs steadily, unit costs keep falling, and the share of top-tier customers continues to grow, gross margin improvement follows naturally. The LLM company with the largest revenue in China offers the clearest example of this.