MaaS Gross Margins: Why Revenue Growth Alone Doesn't Lift Profitability, and How to Fix It

    This article explains why Model-as-a-Service (MaaS) businesses often see gross margins trail revenue growth. It treats the problem as one of unit economics and reviews the levers that can improve margins: inference efficiency, customer structure, and day-to-day operational metrics.

    1. Why MaaS Revenue Growth Doesn’t Automatically Lift Margins

    MaaS (Model as a Service) works like this: instead of selling one-time software licenses or custom projects, a company packages model capabilities behind standardized APIs, and customers pay for the Tokens they consume. The appeal of this model is that revenue is recurring. As long as a customer keeps running its business on the service, calls keep coming, which makes ARR (annual recurring revenue) a core measure of business health. The first annual report published by a China-based model service provider after its IPO shows this pattern. Full-year revenue more than doubled year over year, MaaS platform ARR grew dozens of times year over year, and total revenue reached the top tier of China’s model service vendors.

    The same report contains a figure that engineering teams should look at more closely. The company’s API gross margin rose from the low single digits to nearly 20%, while its overall gross margin has stayed above 50% for a long time. This curve of low, rising, and still-low margins reflects a structural feature of the MaaS model, not a management failure.

    The cause is that revenue and cost are tied to the same chain:

    • In traditional SaaS, the marginal cost is close to zero, so user growth almost directly equals profit growth.
    • Every MaaS dollar of revenue corresponds to a real amount of compute. As revenue grows, inference cost grows with it.
    • If the price per Token falls faster than the cost per Token, the business ends up in a situation where the larger it gets, the less it earns.

    So the gross margin problem in MaaS is not fundamentally a sales problem. It is a unit economics problem:

    1. Gross margin = 1 − (Σ inference compute cost + storage, network, and operations allocation) / Σ Token-based revenue

    There are three ways to improve this ratio: raise the revenue per Token (through pricing and customer mix), lower the cost per Token (through inference efficiency), and change the usage mix (through more high-value, predictable calls). The sections below cover each one.

    2. Measure First: You Cannot Optimize What You Do Not Meter

    Many teams see little return from inference optimization because they lack cost attribution by request, customer, or model. We recommend capturing structured metering at the gateway layer, writing billing attributes and technical attributes into the same log record:

    1. # Metering instrumentation at the inference gateway (illustrative, vendor-neutral)
    2. record = {
    3. "tenant_id": tenant, # Customer or project ID, used to attribute revenue and cost
    4. "model_route": route_name, # Model tier actually hit (large/medium/small)
    5. "prompt_tokens": in_tokens,
    6. "completion_tokens": out_tokens,
    7. "cached_tokens": cache_hit_tokens, # Input tokens served from the prefix cache
    8. "batch_size_at_prefill": b, # Size of the batch this request was in at prefill
    9. "gpu_seconds": gpu_elapsed * gpu_share, # GPU-seconds after allocation
    10. "ttft_ms": first_token_latency,
    11. "queue_wait_ms": wait_time,
    12. "outcome": "ok", # ok / truncated / rate_limited
    13. }

    With this data chain in place, you can calculate the decision metrics that matter: marginal cost per million tokens (cost per 1M tokens) and tokens produced per GPU-hour (tokens per GPU-hour). The second metric is the core currency for capacity planning and pricing negotiations.

    3. Technical Levers: Five Efficiency Levers on the Inference Side

    3.1 Continuous Batching: Keep the GPU Busy Instead of Queuing Requests Serially

    Generative inference has two phases: prefill (processing the input) and decode (generating one Token at a time). The decode phase is memory-bound, so a single request rarely saturates the GPU. A naive implementation aligns requests into fixed batches and waits for them to finish together. Short requests then sit idle while long requests hold the batch.

    Continuous batching (also called in-flight batching) schedules work at the iteration (step) level instead of the request level. When a sequence emits its EOS token, it leaves the batch immediately and frees its slot for a queued request. New requests can also join the running batch at the next step.

    1. # Scheduler main loop (pseudocode describing scheduling semantics, not a specific framework)
    2. while True:
    3. # 1) Reclaim slots of finished sequences and free their KV cache blocks
    4. finished = [s for s in running if s.done]
    5. for s in finished:
    6. free_kv_blocks(s)
    7. running = [s for s in running if not s.done]
    8. # 2) Fill free slots from the waiting queue by priority (priority = SLA tier × remaining budget)
    9. for req in scheduler.pick_next(free_slots := capacity - len(running)):
    10. if not can_allocate_kv(req.expected_len):
    11. break # Stop admitting when memory is insufficient to avoid OOM thrashing
    12. running.append(req)
    13. # 3) Run one forward pass for the whole batch in this step
    14. step_forward(running)
    15. # 4) Mark sequences that hit the output length or a stop condition as done, and stream results back
    16. stream_emit(running)

    Two engineering points matter here. First, set an admission watermark. When memory usage exceeds the threshold, stop accepting new requests and queue them instead of crashing. Second, schedule prefill and decode in separate pools or segments. This keeps long-input prefill from interrupting batches that are already decoding, which would otherwise cause large swings in time to first token (TTFT).

    3.2 KV Cache: GPU Memory Is the Largest Cost Item

    For long contexts, the biggest cost is not compute but the GPU memory used by the KV cache. Three common approaches help:

    • Paged memory management: Split the KV cache into fixed-size blocks, allocate them on demand, and map them to sequences through a block table. This reduces fragmentation and reserved-but-unused memory, so one GPU can serve more concurrent requests.
    • Prefix cache reuse: System prompts, few-shot examples, long documents, and tool definitions are often highly repetitive within a customer or business. Caching and reusing these prefixes avoids recomputing repeated input. The cache key must include the model version, sampling-related settings, and a tenant isolation identifier. Without it, answers can leak across tenants and expose data.
    • KV cache compression and quantization: Storing the cache at lower bit widths, or evicting entries based on attention importance, can increase concurrent capacity several times over. The trade-off is that you must validate quality against a business evaluation set to detect regressions.

    A simple way to check hit rates:

    1. -- Cache effectiveness diagnostic: a low hit rate points to problems with the prefix strategy or cache key design
    2. SELECT tenant_id,
    3. SUM(cached_tokens) * 1.0 / NULLIF(SUM(prompt_tokens), 0) AS cache_hit_ratio,
    4. AVG(prompt_tokens) AS avg_prompt_len,
    5. COUNT(*) AS calls
    6. FROM inference_log
    7. WHERE ts >= now() - interval '1 day'
    8. GROUP BY tenant_id
    9. ORDER BY cache_hit_ratio ASC
    10. LIMIT 20;

    3.3 Quantization and Speculative Decoding: Trade Precision for Throughput, but Keep a Rollback Path

    Weight quantization, activation quantization, and KV cache quantization directly increase throughput per GPU. They are among the most direct ways to lower marginal cost. Speculative decoding uses a small model or draft head to propose several Tokens, which the large model then verifies in one pass. It suits the decode phase at low concurrency and can noticeably reduce per-request latency. Under high concurrency and full load, however, its benefit shrinks and can turn negative.

    The key constraint is to build a regression evaluation set. Every change to quantization settings or decoding strategy should be compared on a fixed evaluation set. A change should go live only if quality regression stays within an acceptable threshold, and you should keep the ability to switch back to the full-precision path with one action.

    3.4 Model Routing: Not Every Request Deserves the Strongest Model

    Much real traffic consists of light tasks such as classification, extraction, rewriting, and formatting. Sending all of it to the strongest model spends the most expensive capability on the cheapest work. A better approach is a cascade router. A small model or lightweight classifier first estimates difficulty. Simple requests are answered by the small model, while low-confidence or high-risk requests escalate to the large model.

    1. def route(prompt, task_meta):
    2. if task_meta["risk_level"] == "high" or task_meta["json_schema_strict"]:
    3. return "large" # High-risk or strictly structured: send directly to the large model
    4. pred, conf = small_model.predict(prompt)
    5. if conf < CONFIDENCE_THRESHOLD:
    6. return "large" # If uncertain, escalate to guarantee a quality floor
    7. return "small"

    Routing lowers cost, and it also improves capacity structure. Cheap traffic is absorbed by the small-model pool, and the large-model pool can focus on high-value requests. Overall queuing latency becomes easier to control.

    3.5 Output Governance: Tokens Are a Cost, Not a Perk

    Oversized max_tokens values, missing stop conditions, redundant explanations, and entire long documents stuffed into the context all make bills look high to customers and make costs spiral out of control for the provider. Practical fixes include setting output length limits for each task type, using structured output to cut wasted characters, retrieving or summarizing long documents before they reach the model, and merging obvious duplicate calls into a single batched request. This area rarely requires an algorithmic breakthrough, but it is often the fastest way to improve gross margin.

    4. Commercial Levers: How High-Value Customers Improve Margins

    Technical optimization addresses unit cost. Customer structure addresses revenue quality. High-value anchor customers improve gross margin through four mechanisms:

    1. Predictable usage: Steady, continuous calls shift capacity planning from provisioning for peak demand to provisioning for average demand plus a buffer. This directly raises average utilization. Idle compute reserved for sudden peaks is one of the most overlooked cost sinks in MaaS.
    2. Well-defined workload profiles: Enterprise customers often use fixed system prompts and templates. Their prefix cache hit rates are high, and their input and output length distributions are concentrated. Batching is therefore much more efficient than with scattered long-tail developer traffic.
    3. Tiered pricing customers can accept: Customers who need SLAs, concurrency guarantees, or private deployment are willing to pay for reliability. They compare vendors on whether the service can sustain their peak load, not only on price per million Tokens.
    4. High switching costs: Once a customer’s prompt system, evaluation sets, toolchain, and business logic are built around one API design, migration costs rise over time. Renewal rates and net revenue retention (NRR) tend to increase as a result.

    This also explains why raising prices without losing volume is possible in practice. Whether a price increase holds depends on whether the model capabilities are differentiated and on the real cost to customers of switching. Marketing claims do not determine it.

    5. Operability: Turn Gross Margin into a Dashboard

    Gross margin should not be discovered only in quarterly financial reports. It should be an operational metric you can observe every day. At minimum, build the following views:

    Metric Meaning Optimization Direction
    Unit Token cost Compute and operations cost allocated per million Tokens Quantization, batching, routing
    Daily Tokens per GPU Core measure of hardware efficiency Memory management, concurrency scheduling
    Prefix cache hit rate How much repeated input is reused Cache key design, template governance
    Average batch size / batch fill rate Whether the GPUs are fully loaded Scheduling policy, admission watermark
    TTFT and P99 latency User experience and SLA compliance Pool-based scheduling, capacity buffers
    Customer-level gross margin and NRR Revenue quality Customer tiering, tiered pricing
    Rate-limiting and truncation rate Signal of capacity bottlenecks Scaling or degradation policies

    The data pipeline can be built entirely on general-purpose infrastructure. Call logs can land in object storage for offline analysis, metrics can feed a monitoring and alerting system with thresholds, and a container platform can host an elastic inference pool with a message queue for smoothing traffic spikes. The priority is not the choice of tools. It is to put technical metrics and financial metrics on the same wide table.

    6. Common Pitfalls and Risk Boundaries

    • Chasing GPU utilization alone: Pushing batch sizes to the maximum degrades TTFT and P99 latency. This triggers customer timeouts and retries, which can increase total call volume. Utilization must be constrained together with the SLA.
    • Ignoring cache correctness: If a cache key omits the tenant, model version, or changes to tool definitions, answers can leak between customers. This is a serious incident, not just a UX problem.
    • Quantizing without evaluation: Tuning quantization without a regression set is flying blind. Quality regressions may only surface in production.
    • The memory cliff with long contexts: Concurrency degrades quadratically as context length grows. Very long inputs need their own quotas and queuing policies.
    • Treating optimization as a one-time project: Model version upgrades and shifts in customer workloads erode the gains from optimization. Efficiency work needs ongoing processes and named owners.

    7. Conclusion: From Selling Tokens to Selling Predictability

    Improving MaaS gross margin does not come from a single clever technique. It is the product of inference efficiency engineering and customer structure quality. Efficiency engineering determines how many Tokens one GPU can produce. Customer structure determines how stable and how valuable those Tokens are when customers consume them. Dozens-fold ARR growth shows that demand is real. A gross margin that rose from the low single digits to about 20% shows that supply-side economics are starting to improve. Reaching the 50%-plus gross margins of traditional software will still require sustained work on memory management, batch scheduling, model routing, and tiered pricing.

    For developers and enterprise users, the practical takeaway is direct. When choosing a provider, do not compare only unit prices. Evaluate whether the provider’s unit economics are sound, including whether its price changes and cost reductions are sustainable. When building your own inference service, establish metering and cost attribution first, then optimize. Costs that are not measured can never be optimized.