Local Deployment vs. Cloud APIs: The Gross Margin Logic of LLM Delivery
The way a large language model is delivered—through a cloud API or a private deployment—determines its cost structure, scalability, and profit profile. This article compares the two models, explains how to analyze segment-level gross margins, provides a framework for estimating inference and project costs, and outlines key considerations for enterprise adoption.
1. Why the Same Model Can Produce Very Different Financial Results
When evaluating an LLM business, many people focus directly on total revenue or the net loss ratio. This overlooks a critical factor: the same model can be monetized through different delivery models, each with a very different cost structure and gross margin profile.
Two delivery models are most common.
The first is cloud-based service, primarily delivered through usage-based API calls and subscription plans. Customers access a centrally deployed model over the network.
The second is private deployment, in which the model and supporting systems are delivered to infrastructure owned or designated by the customer, such as an on-premises data center or private cloud. This model often includes customization, tuning, and on-site support.
The two models differ substantially in revenue recognition, cost sources, and economies of scale. Understanding these differences is essential for interpreting the financial performance of LLM companies.
2. Key Differences Between the Two Delivery Models
2.1 Revenue and Cost Characteristics of Cloud Services
The marginal cost of a cloud service primarily comes from inference compute, GPU memory usage, and network bandwidth. Each additional request consumes additional computing resources.
Typical characteristics include:
- Strong economies of scale: Higher utilization of each accelerator lowers the cost per request.
- Intense price competition: API prices are relatively transparent, making it easy for customers to compare providers. Price cuts directly compress gross margins.
- Volatile revenue: Usage can be affected by application popularity, seasonality, and competitors’ strategies.
Cloud-service gross margins depend heavily on engineering efficiency. With the same hardware, the choice of inference framework, batching strategy, and cache reuse can produce cost differences of several times.
2.2 Revenue and Cost Characteristics of Private Deployment
Revenue from private deployment is usually recognized on a project basis. A project may include software licensing, environment adaptation, model deployment, data integration, custom training or fine-tuning, and ongoing support.
Typical characteristics include:
- Concentrated customer base: Customers with strict data-security and compliance requirements—such as internet platforms, telecommunications operators, financial institutions, and large manufacturers—are common buyers.
- High contract value per project: A single project may be worth several million yuan or more, but the delivery cycle is usually longer.
- Gross margins depend on standardization: If every project requires customization from scratch, labor costs rise quickly. If reusable components can be accumulated, margins can improve significantly.
2.3 How Revenue Mix Affects Financial Statements
Assume that a company generates total revenue of CNY 100 million during a six-month period. This is an illustrative example. Private-deployment revenue accounts for 85% of the total and has a 60% gross margin, while cloud-service revenue accounts for 15% and has a 0% gross margin.
The blended gross margin is calculated using revenue weighting:
Blended gross margin = 85% × 60% + 15% × 0% = 51%
Illustrative code:
function blended_margin(segments):revenue = sum(s.revenue for s in segments)gross_profit = sum(s.revenue * s.margin for s in segments)return gross_profit / revenuesegments = [{name: "Private deployment", revenue: 8500, margin: 0.60},{name: "Cloud service", revenue: 1500, margin: 0.00},]print(blended_margin(segments)) # Output: 0.51
This example illustrates two points.
First, the consolidated gross margin can conceal significant differences between business segments. Second, as long as private-deployment margins remain high, the consolidated result may not deteriorate substantially even when the cloud business is loss-making.
When reviewing financial statements, it is therefore important to examine segment revenue, segment gross margin, and changes in the revenue mix together.
3. Why Segment-Level Gross Margins Should Be Analyzed Separately
3.1 The Limitations of Consolidated Metrics
The consolidated gross margin answers the question, “Is the company profitable overall?” It does not answer, “Which business can generate sustainable profits?”
If cloud services have negative gross margins over an extended period while private deployment maintains stable and relatively high margins, changes in the overall gross margin may simply reflect a shift in revenue mix. They do not necessarily indicate that every business segment has become more efficient.
3.2 Three Questions for Segment Analysis
When analyzing segment data, consider the following questions in order:
- Is the gross margin of each segment positive or negative? Is the main cause pricing, cost, or scale?
- Is the revenue mix changing? Is a segment gaining share while its gross margin improves at the same time?
- Is a loss-making segment still in an investment phase, or does it have a structural problem?
The third question is particularly important. Cloud services may operate at a loss during the early stages because infrastructure investment is high and usage is insufficient. This is characteristic of an investment phase.
However, if unit costs continue to decline slowly after the business scales, the problem may be inadequate engineering efficiency rather than temporary investment.
4. Estimating the Unit Cost of Cloud Services
4.1 Basic Unit-Cost Formula
A key metric for cloud services is the cost per million output tokens. It can be calculated as follows:
Unit cost = Cost per accelerator hour ÷ (Effective output tokens per accelerator hour ÷ 1,000,000)
For example, if the cost per accelerator hour is CNY 40 and effective throughput is 2,000,000 tokens per hour, the unit cost is:
40 ÷ (2,000,000 ÷ 1,000,000) = CNY 20 per million tokens
4.2 Main Factors Affecting Unit Cost
The following factors influence effective throughput:
- Batching: Processing multiple requests together improves utilization of computing resources.
- Quantization: Reducing weight precision within an acceptable accuracy-loss range lowers memory usage and increases concurrency.
- KV-cache reuse: Reusing intermediate results for requests with the same prefix reduces repeated computation.
- Scheduling strategies: Allocating resources according to request length and latency requirements helps prevent long requests from blocking shorter ones.
These are common inference-optimization techniques. Their effectiveness depends on the specific model, hardware, and traffic pattern. They must be validated through load testing rather than copied directly from another provider’s results.
4.3 Why Price Competition Compresses Gross Margins
Assume the market price falls to CNY 15 per million tokens while the unit cost remains CNY 20. Every unit sold would then generate a loss.
Only when the unit cost falls below CNY 15 can the service achieve a positive gross margin. This means that competition among cloud services is fundamentally a competition in engineering efficiency, not simply a competition over pricing.
5. Cost Structure and Optimization for Private-Deployment Projects
5.1 Main Project Cost Components
A typical private-deployment project may include the following cost categories:
- Environment adaptation: Assessing the customer’s compute, storage, and network conditions and determining the deployment plan.
- Model deployment and tuning: Loading the model, configuring the inference service, conducting performance tests, and validating stability.
- Data governance and compliance assessment: Managing data integration, masking sensitive information, defining access permissions, and completing compliance reviews.
- Custom development: Adapting the model to domain-specific data, conducting fine-tuning, and developing business interfaces.
- Operations and on-site support: Responding to incidents, upgrading versions, and performing routine inspections.
Custom development and on-site support are the most difficult areas to standardize and are therefore the most likely to reduce gross margins.
5.2 Three Ways to Improve Gross Margins
The first approach is to turn recurring capabilities into reusable components, such as deployment scripts, monitoring templates, and access-control modules.
The second is to establish project tiers. Requirements can be divided into standard features, configurable features, and custom features, with custom work priced separately.
The third is to automate operations and maintenance. Automated inspections and alerts can reduce the need for manual intervention.
All three approaches pursue the same goal: reducing the marginal delivery cost as the number of projects increases.
6. A Framework for Enterprise Selection
Enterprises can compare cloud APIs and private deployment across the following dimensions:
| Dimension | Cloud API | Private Deployment |
|---|---|---|
| Data sensitivity | Suitable for general business data | Suitable for highly sensitive or heavily regulated data |
| Usage pattern | Suitable for volatile demand and small initial volumes | Suitable for stable and predictable demand |
| Customization needs | Primarily provides general-purpose capabilities | Supports deeper customization |
| Time to launch | Usually shorter | Usually longer because the environment must be prepared |
| Cost structure | Usage-based fees; costs grow roughly with usage | Higher initial investment and lower marginal cost |
| Operations responsibility | Mainly handled by the service provider | Shared by the customer and service provider |
A practical decision process can follow this order:
- First determine whether the data is allowed to leave the local environment. This is a hard constraint.
- Then estimate usage and peak demand to assess whether fixed investment is economically justified.
- Finally evaluate the organization’s operations capabilities and determine whether it can support a self-managed deployment.
7. Practical Considerations for Implementation
7.1 Choosing the Right Metrics
The effectiveness of an LLM business should not be measured only by request volume. More informative metrics include effective request volume, customer retention, renewal rates, and changes in unit cost.
For private-deployment projects, organizations should also track delivery time, acceptance-test pass rates, and the proportion of secondary requirements.
7.2 Data Security
Regardless of the delivery model, organizations should follow the principle of minimizing data transfers.
Specific measures include:
- Encryption in transit
- Masking of sensitive fields
- Tiered access controls
- Retention of operational logs
When data crosses national borders, the organization must also verify the requirements of applicable regulations.
7.3 Monitoring and Alerting
After launch, organizations should set thresholds for latency, error rates, throughput, and token consumption.
For example, an alert can be triggered when first-token latency exceeds a defined threshold continuously for five minutes. A notification can also be sent when daily consumption exceeds a specified proportion of the budget.
Monitoring data can then be used to guide capacity planning and cost optimization.
8. Conclusion
The delivery model of an LLM determines its cost structure and path to profitability.
Cloud services rely on economies of scale and engineering optimization to generate profits. Private deployment relies on project-delivery capabilities and standardization.
When evaluating an LLM company or planning an enterprise project, revenue mix, segment-level gross margins, and cost composition should be analyzed together. Looking at a single metric is rarely enough to understand the underlying business economics.
