LLM Selection Guide: Product Tiers and the Open-Source Boundary
Choosing a large language model requires balancing capability, openness, and cost—not just benchmark scores. This article explains how some leading vendors structure flagship, lightweight, and multimodal or agent-specialized models; what characterizes the Chinese model ecosystem and how to verify its advantages; and why open weights are not the same as open source. It also provides a practical checklist covering hard constraints, licensing, TCO, and task-specific evaluation.
1. Why Selection Must Consider Capability, Openness, and Cost
LLM competition has moved from single-point capability benchmarks to system-level competition. For developers and enterprises, a launch-day score only shows how a model performs on certain test sets. What really affects deployment is whether model capability meets business needs and whether you retain control over deployment, invocation, and migration.
This article focuses on three questions:
- How leading vendors’ products are tiered;
- What characterizes the Chinese model ecosystem and which advantages can be verified;
- The difference between open weights and open source, and how it affects engineering and cost.
Most evaluation numbers cited here come from vendor self-reports or third-party publications. Test settings, versions, and dates differ, so they cannot be compared directly. The conclusions are for selection reference only and cannot replace testing on your own workloads.
2. Global Landscape: Some Leading Vendors Use a Three-Tier Product Structure
In the past year or two, some leading vendors’ product lines have shown a similar tiering approach. Consider several public product lines: OpenAI’s GPT series includes larger flagship models and smaller, cheaper mini versions; Anthropic’s Claude family is divided into Opus, Sonnet, and Haiku tiers by capability and price; Google’s Gemini also distinguishes Pro, Flash, and other positions. Specific model names change over time, so selection should be based on official documentation. Not all vendors use this matrix; some offer only a single main model.
The first tier is the flagship model. Flagship versions usually carry the strongest reasoning, coding, and agent capabilities of the period. Most are first opened to high-tier subscribers and enterprise customers while also offering APIs. Their role is to define the capability ceiling and serve complex analysis, long-chain code modification, and tasks requiring multi-step planning. If daily call volume is high and tasks are simple, flagship pricing will quickly raise costs, so it is not suitable for handling every request.
The second tier is the lightweight model. Lightweight versions are usually released some time after the flagship, aiming for faster response and lower unit prices. They can handle most routine Q&A, classification, information extraction, and summarization tasks. Their boundary is also clear: for tasks requiring long-chain reasoning or high-precision judgment, lightweight versions usually have higher error rates and need to be confirmed through evaluation.
The third tier is multimodal and agent-specialized versions. These models can understand images, operate interfaces, and execute tasks; some products also support multiple agents dividing different sub-steps of the same task. Typical scenarios include interface automation and visual document understanding. If your business handles only plain text, the value of this tier drops noticeably.
Engineering implication: match the tier to the business shape. Route high-frequency, low-complexity requests to the lightweight tier, and call the flagship tier for complex reasoning and agent orchestration. A routing strategy that combines tiers is usually more economical than a single model. However, routing itself adds development and maintenance costs, and may not be worth it at low request volumes.
3. The Chinese Model Ecosystem: Characteristics and Verification Methods
Several vendors pursuing the open-source route in China are widely discussed. From public information, Chinese models have three clear characteristics.
- Fast iteration. Major vendors’ release cycles have shortened noticeably. Judgments should be based on release logs and model cards.
- Open-weight models have high usage in developer communities. Several medium-sized open-weight models are often used as bases for fine-tuning and secondary development. To measure this, you can refer to download counts on Hugging Face, the number of derivative models, and community discussion.
- Deep integration with cloud services. Chinese vendors commonly connect models with object storage, message queues, container platforms, logging, and monitoring, lowering the engineering barrier from model to production.
There is no unified standard for capability rankings. Chinese-language understanding, code generation, and long-text processing are relatively strong areas for Chinese models. On some public Chinese benchmarks and code tasks, some models’ results are close to mainstream levels. But this closeness is conditional: test sets, evaluation versions, prompt templates, and whether tool calls are allowed all affect results. Selection should be based on your own task set.
For enterprise users, the Chinese ecosystem’s main value lies in three points: better fit for Chinese-language scenarios, clearer data-residency and compliance paths, and one-stop cloud offerings that reduce the work of building infrastructure yourself.
4. Unpacking “Open Source”: Open Weights vs. True Open Source
Many people equate “model files can be downloaded for free” with open source. The difference directly affects deployment methods, auditability, and licensing risk.
Open weights usually means: weight files are provided for download and local inference, but training data, training code, and data-cleaning pipelines are not necessarily public. Licenses may limit commercial scale, restrict using model outputs to train competing models, or require authorization for specific uses.
True open source has stricter criteria. According to the Open Source AI Definition published by the Open Source Initiative (OSI) in 2024, an open-source AI system must be provided under an OSI-approved license that grants the freedom to use, study, modify, and distribute it. It must also make weights and code available and provide sufficiently detailed information about training data so that skilled people can build a substantially similar system. The requirement is detailed information, not necessarily full disclosure of all training data. Common OSI-approved licenses include Apache 2.0 and MIT. If a model license includes user-count caps or usage policies, it is usually not a standard open-source license and must be evaluated separately.
The table below compares key dimensions of three approaches:
| Dimension | Open weights | True open source | Closed-source API |
|---|---|---|---|
| Weights | Public download | Public download | Not provided |
| Training code | Usually not public | Public | Not public |
| Training data information | Usually not detailed | Sufficiently detailed information required | Not public |
| License | Often includes restrictions | OSI-approved license | Terms of service |
| Cost structure | GPU and operations investment | GPU and operations investment | Billed by usage |
| Audit and reproducibility | Limited | Relatively feasible | Mainly depends on vendor |
| Exit risk | Depends on license and version | Lower | Depends on interface and price changes |
The practical impact on selection is mainly threefold:
- Controllability: Having weights means you can deploy and fine-tune, but not necessarily reproduce or audit the model. If the business has hard requirements for explainability and compliance auditing, this is a gap that must be assessed.
- Licensing risk: Check license terms line by line before launch. Large-scale commercial use, model distillation, and SaaS resale may trigger restrictions; consult legal counsel when necessary.
- Self-hosting cost: Open weights save API call fees but incur GPU, operations, and version-upgrade costs. The following formulas can serve as a starting point for estimation:
Self-hosted monthly cost ≈ GPU-hour unit price × number of GPUs × monthly running hours + operations labor + storage and networkAPI monthly cost ≈ price per million tokens × monthly call volume (millions of tokens)
The GPU type, unit price, average utilization, ratio of peak to average, inference framework throughput, and operations labor all change the conclusion. When average utilization is low, idle GPUs significantly raise self-hosting costs. Therefore there is no directly applicable call-volume threshold; compare based on your own measured TCO.
5. Selection Checklist
The judgment items and outputs for the steps are as follows.
Step 1: Define hard constraints first
- Judgment items: Can data leave your domain? Is private or local deployment required? Are there compliance audit or explainability requirements?
- Output: A one-page constraint list identifying which conditions are non-negotiable. Constraints will directly eliminate some candidate models.
Step 2: Check licensing and cost
- Judgment items: Does the license permit your use, including commercial scale, SaaS resale, distillation, and training other models on outputs? Also estimate self-hosted and API options by TCO.
- Output: A license-term summary (with original clause numbers) and a monthly cost table for both options.
Step 3: Evaluate on your own task set
- Judgment items: Collect real samples covering common requests and edge cases. A few hundred samples can be a starting point.
