LLM Selection in Practice: Closed APIs, Open Weights, and True Open Source

    This article explains how to choose between closed hosted LLM APIs, open-weight models, and truly open-source models. It covers the licensing spectrum, data governance, cost trade-offs, fine-tuning needs, operational burden, and practical engineering patterns such as abstraction layers for avoiding vendor lock-in.

    1. Why “Open Source or Closed Source” Has Become a Key Question

    As of 2026, the supply of large language model capabilities has developed into a clear two-track pattern. On one side are closed-source flagship models offered through cloud APIs, sold on the basis of top-tier performance and the latest capabilities, updated monthly and priced by usage. On the other side are open models whose weights can be downloaded and deployed independently, with both quantity and quality improving rapidly.

    For any team that needs to integrate model capabilities into real business workflows, model selection is no longer simply a matter of choosing the highest benchmark score. It requires trade-offs across capability, cost, data boundaries, and operational control.

    One of the most confusing terms in this discussion is “open source” itself. Many people assume that “being able to download weights” is equivalent to “open source,” but these two concepts differ significantly in legal licensing, engineering boundaries, and long-term risk. Understanding this spectrum is the prerequisite for making a sound decision.

    2. A Spectrum: Closed, Open Weights, and True Open Source

    It is more useful to view the current LLM supply landscape as a continuous spectrum—from fully managed to fully open—rather than a binary choice.

    1. Closed / Managed Services

    In this model, vendors expose only API endpoints. Model weights, training data, and training details are not made public. Users pay based on usage, usually measured by the number of input and output tokens, and do not need to manage underlying infrastructure.

    • Advantages: Fastest integration, nearly zero operational overhead, and capabilities that are often in the top tier. These services typically support advanced features such as multimodality, tool calling, and agent workflows.
    • Trade-offs: Data must leave the organization’s own environment; ongoing usage creates continuous cost; deep dependence can lead to high migration costs, or vendor lock-in.

    2. Open Weights

    This is the most mainstream category today, and also the most easily misunderstood. Vendors publicly release model weight files, which can be downloaded and used for local or private-cloud inference, and sometimes for fine-tuning. However, “open weights” does not necessarily mean “open training process.” Training code, data mixtures, and data-cleaning pipelines are usually not released alongside the weights.

    The key factor here is the license. Licenses for open-weight models vary widely and can generally be grouped into three categories:

    • Permissive licenses, such as Apache 2.0 or MIT-like licenses: Commercial use is generally permitted. Modification, redistribution, and integration into closed-source products are usually relatively free, making these licenses the first choice for enterprise self-hosting.
    • Restricted “responsible use” licenses: These allow download and use, but impose conditions related to commercial scale, number of users, use cases, or prohibitions in certain high-risk domains. Such “open” models require careful review before commercial deployment.
    • Research-only / non-commercial licenses: These explicitly prohibit commercial use and are not suitable for production systems.

    Therefore, when seeing that a model “can be downloaded,” the first step is not deployment, but reading the license.

    3. True Open Source

    Strictly speaking, “open source” should follow recognized open definitions, such as those established by the Open Source Initiative. It should provide not only model weights, but also reproducible elements such as training code and data construction methods, under a permissive and commercially usable license. Its value is that you do not merely receive a finished product; you can also understand, reproduce, and audit how it was built.

    In reality, models that fully meet strict open-source definitions while also reaching top-tier capability are still rare. Most enterprises face a middle ground: open weights with a permissive license. This category is already sufficient for most self-hosting and fine-tuning needs.

    Dimension Closed Managed Service Open Weights True Open Source
    Can weights be downloaded? No Yes Yes
    Can it be privately deployed? No Yes Yes
    Can it be fine-tuned? Depends on the vendor Usually yes Yes
    Does data leave your environment? Yes No No
    Cost structure Pay-as-you-go Mainly fixed resource investment Mainly fixed resource investment
    Operational burden Nearly zero High High
    License flexibility Governed by service terms Depends on license Usually the most permissive

    3. A Neutral View of the Current Supply Landscape

    From a general technical perspective, leading capability providers are advancing in two directions. First, they are improving core model capabilities, including stronger reasoning, multimodal understanding, tool calling, and agent abilities for operating complex tasks. Second, they are diversifying usage forms, moving from pure text conversation toward interactive, executable, multimodal outputs.

    At the same time, the open ecosystem is rapidly narrowing the gap with closed flagship models. Many open-weight models are now “good enough and highly cost-effective” for general tasks. This leads to an important conclusion: the difference between closed and open models is no longer an absolute capability gap, but a division of use cases. The real question is not which side is better in the abstract, but where your own business fits most cost-effectively on the spectrum.

    4. A Decision Framework for Developers

    Rather than debating “which one is better,” it is more practical to score your requirements across the following dimensions.

    1. Data Sensitivity and Compliance Boundaries

    This should be the first gate. In highly regulated scenarios involving user privacy, finance, healthcare, or government workloads, data often cannot leave its original environment. In such cases, private or self-hosted open-weight models are almost mandatory. For general, non-sensitive use cases such as content generation, marketing copy, or internal knowledge Q&A, a managed API is usually sufficient.

    2. Usage Volume and Cost Structure

    • Low volume, high volatility, and the need for fast launch: A managed API with pay-as-you-go pricing is the most economical option, because you do not pay for idle compute.
    • Stable, high-concurrency usage: After self-hosting, the marginal inference cost drops significantly, and the amortization effect of fixed resource investment becomes obvious. A rough break-even comparison can guide the decision:
    1. # Example: compare hosted API cost vs. self-hosted inference cost
    2. def cost_comparison(monthly_calls, tokens_per_call,
    3. api_price_per_1k, gpu_monthly_cost,
    4. throughput_per_gpu):
    5. # Hosted API monthly cost
    6. hosted = (monthly_calls * tokens_per_call / 1000) * api_price_per_1k
    7. # Self-hosted: estimate required GPU count and monthly cost based on throughput
    8. monthly_tokens = monthly_calls * tokens_per_call
    9. required_gpus = monthly_tokens / throughput_per_gpu
    10. self_host = required_gpus * gpu_monthly_cost
    11. return hosted, self_host

    The above code is only for decision illustration. Actual pricing and throughput should be based on real load tests, and parameters should not be copied directly.

    3. Customization and Fine-Tuning Needs

    If you need deep alignment with a vertical domain, injection of proprietary knowledge, or changes to model behavior and style, fine-tuning support in open-weight models is a major advantage. Most closed services either do not offer fine-tuning or provide it only in a limited form.

    4. Team Engineering Capability

    Self-hosting means you must handle inference services, GPU scheduling, elasticity, monitoring, and fallback yourself. This requires some level of machine learning operations, or MLOps, capability. If the team does not yet have this experience, the safer path is to validate the business with a managed API first and migrate later when appropriate.

    5. Use a Unified Abstraction Layer to Avoid Vendor Lock-In

    Regardless of which route you ultimately choose, one engineering practice worth adopting from day one is to hide differences between underlying models behind a unified abstraction layer, so that business code depends only on your own interfaces rather than a specific backend.

    1. from abc import ABC, abstractmethod
    2. class ModelBackend(ABC):
    3. """Unified backend interface that hides differences between hosted services and self-hosted models."""
    4. @abstractmethod
    5. def generate(self, prompt: str, **kwargs) -> str:
    6. ...
    7. class HostedAPIBackend(ModelBackend):
    8. """Closed / managed service: called through an API."""
    9. def __init__(self, endpoint: str, api_key: str):
    10. self.endpoint = endpoint
    11. # API keys should come from a key management service (KMS) or environment variables, never hard-coded
    12. self.api_key = api_key
    13. def generate(self, prompt: str, **kwargs) -> str:
    14. # Pseudocode: build request, authenticate, send, and parse response
    15. ...
    16. class SelfHostedBackend(ModelBackend):
    17. """Self-hosted open-weight model: local inference."""
    18. def __init__(self, model_path: str):
    19. self.model_path = model_path
    20. def generate(self, prompt: str, **kwargs) -> str:
    21. # Pseudocode: load local weights and run inference
    22. ...

    The benefit of this approach is that you can quickly validate requirements with a managed API today, and smoothly switch to self-hosting later when compliance requirements increase or traffic grows significantly—with little or no change at the business layer. When combined with configuration-based backend routing and gradual traffic shifting, migration risk can be kept to a minimum.

    6. Engineering Essentials for Production

    Choosing a model is only the beginning. Productionization requires surrounding capabilities such as inference services, containerization, autoscaling, observability, security, and fault tolerance. These capabilities are commonly provided as general infrastructure categories across mainstream cloud platforms, including object storage, message queues, container platforms, logging services, monitoring and alerting, and GPU compute instances. Platforms such as Baidu AI Cloud also provide similar general infrastructure and can serve as a foundation for self-hosted inference.

    • Inference services and containerization: Package the model as a standardized service to support scaling and version management.
    • Elastic scaling: Automatically add or remove instances based on request volume, balancing cost and peak-time experience.
    • Observability: Monitor and alert on latency, throughput, failure rates, and token consumption to detect anomalies quickly.
    • Data and key security: Encrypt sensitive data, manage credentials centrally, and strictly control access permissions.
    • Fallback and fault tolerance: Automatically fall back to a backup backend when the primary backend is unavailable, ensuring business continuity.

    7. Conclusion

    Returning to the original question: there is no standard answer to “open source or closed source”—only fit or mismatch. Closed managed services win on simplicity and frontier capabilities. Open-weight models are strong in data control, customization, and long-term cost advantages. True open source represents the highest level of transparency and freedom.

    What you need is not to chase the top-ranked model on a leaderboard, but first to clarify your data boundaries, cost scale, customization depth, and team capabilities. Then place the model at the right position on the spectrum. Use a unified abstraction layer to preserve room for switching, and keep both the choice—and the initiative—firmly in your own hands.