Building a General-Purpose LLM MaaS Platform: Capability Layers and a Practical Path
This article explains how to build an integrated LLM MaaS offering: how independent vendors and conglomerates differ in reporting, a four-layer capability model with observable metrics, code as the first stress test, a checklist for adapting to domestic AI chips, a six-step fine-tuning process, and delivery patterns for private enterprise instances versus on-device plus cloud collaboration.
Independent vendors versus conglomerates
Once general-purpose large models move from the lab to scaled service, competition shifts from “having a model” to delivering capability stably and repeatably to institutional customers and end devices. Two kinds of players exist. Independent foundation-model developers center on research and platformization. Inside technology groups, LLM businesses are usually embedded in cloud, search, productivity, or hardware ecosystems.
Revenue definitions differ. Independent vendors’ revenue is more concentrated, which makes pure model commercialization easier to track. Conglomerate share often includes internal synergy and can be inflated by the surrounding ecosystem. Before comparing market share, confirm the statistical scope. Rankings have limited value; how products are organized matters more.
Integrated model-as-a-service (MaaS) closes the loop across training, fine-tuning, deployment, and agent development. Platforms can bill by calls, tokens, tool executions, and fine-tuning job duration, with SLAs covering availability, time-to-first-token, and error rate. Clear billing dimensions also make operational ownership easier to assign.
Four layers of integrated MaaS and observable metrics
Integrated MaaS is typically organized by capability layers, each mapped to a class of models, with defined inputs, outputs, and metrics.
- Language model layer: natural-language instructions and context in; text or structured data out. Core skills include instruction following, long-context reasoning, and structured output. A key metric is the validity rate of structured output (example threshold: 99%+).
- Multimodal model layer: images, audio, video in; text-aligned semantic representations or descriptions out. This layer is how the system perceives the world. Key metrics include cross-modal retrieval recall and image–text alignment error.
- Agent layer: task goals and a tool inventory in; plan steps, tool-call sequences, and a final result out. Key metrics include multi-step task completion rate and tool-call success rate.
- Code model layer: repositories, tickets, or test cases in; patches, tests, and explanations out. Key metrics include unit-test pass rate and first-pass patch rate.
The platform also needs fine-tuning pipelines, an inference gateway, permissions and quotas, evaluation-set management, and agent orchestration.
Externally, models present three capability classes. Each should be broken into observable metrics rather than a single benchmark score.
- Deep thinking: multi-step reasoning and self-checks. Observables include average reasoning-trace length and answer consistency across multiple samples of the same question (example: consistency ≥ 90%, tuned to business risk).
- World understanding: multimodal comprehension and grounding. Observables include cross-modal alignment error and human-reviewed accuracy on image–text QA.
- Tool use: APIs, retrieval, and code executors. Observables include tool-call success rate (example ≥ 98%), timeout share, and rollback time after failure (example within 5 minutes).
Code is the first place gaps show
Coding workloads expose platform weaknesses fastest. Repo-level dependency resolution, cross-file refactoring, test coverage, and security scanning quickly reveal wasted context windows, hallucinated tool calls, and unstable tool protocols.
In practice, split code work into four stages: retrieval augmentation, patch generation, static checks, and sandbox execution. Rate-limit and time out each stage independently. Failures from an earlier stage must not be passed downstream as if they were valid.
Domestic-chip adaptation checklist
Use the following as an acceptance list.
- Operator coverage matrix: list every operator the model uses and mark support or fallbacks per hardware.
- Quantization strategy: compare output quality and latency at FP16, INT8, and similar, and record accuracy loss on the eval set.
- Memory slicing: budget weights, KV cache, and activations; define max concurrency and max sequence length.
- Communication topology: for multi-card setups, match tensor-parallel and pipeline-parallel bandwidth needs to the real interconnect.
- Reproducible performance protocol: fix batch size, input length, concurrency, and warmup; report P50/P95/P99 latency and throughput so hardware results are comparable.
- Monitoring and alerting: feed slow calls, OOM, and accuracy drift into a unified log/alert path and on-call process.
The same graph compiler and kernel library should produce reproducible curves across hardware. If they do not, document the gap before going live.
Container platforms handle elastic scale-out, object storage holds weights and datasets, and message queues decouple async evaluation and fine-tuning so training does not steal online-inference quota.
A standard fine-tuning flow
“Fine-tuning” is often used loosely. Operations break into six steps.
- Data cleaning and de-identification: remove duplicates, low-quality and unauthorized samples; mask PII and trade secrets.
- Evaluation-set construction: sample real tasks, including edge cases and known failures. Lock the set before training; it must not enter training.
- Method choice: LoRA and other parameter-efficient methods fit small data, fast iteration, and limited compute. Full-parameter fine-tuning fits large domain corpora, ample compute, and higher overfitting risk.
- Training config: set learning rate, epochs, and early stopping. Record data version, hyperparameters, and random seed for every run.
- Regression evaluation: test both general-capability and business sets so domain gains do not collapse general ability.
- Canary and rollback: validate on a small share or shadow traffic, then ramp. Keep the previous weights for a fast cutover.
Deployment: private enterprise instances versus device–cloud collaboration
Institutions and end devices are different delivery surfaces.
| Dimension | Private or dedicated instance | Device–cloud collaboration |
|---|---|---|
| Data residency | Data stays in an isolated network for training and inference | Sensitive data stays on-device when possible; hard tasks go to the cloud |
| Model scale | Typically a larger base model | On-device models are usually small or distilled |
| SLA focus | Availability, throughput, and audit completeness | Low latency |
