Free Access to Chinese LLMs: Side-by-Side Comparison and Selection Practice

    This guide explains how developers can evaluate free access to Chinese LLMs without treating zero-cost trials as production capacity. It covers the differences between open inference platforms and integrated model service platforms, the real limits of free quotas, API key and endpoint setup, GUI-based comparison, standardized experiment design, weighted scoring, multi-model automation scripts, task-based selection, and migration practices.

    Free Access to Chinese LLMs: Side-by-Side Comparison and Selection Practice

    One developer saw a notice online: an open inference platform had made two Chinese LLMs freely available, with access tied to the February release period. The topic was hot, but he was not fully convinced. He registered anyway, and three days later delivered his own result: a hands-on tutorial.

    Many developers have had a similar experience: moving from “is this real?” to “configured and running.” A related pattern is integrated LLM service platforms that offer multiple model families in one place, such as DeepSeek, ERNIE, Kimi, GLM, and MiniMax. These platforms can generally be divided into two categories:

    • Open inference platforms: expose online access to selected models. Quotas are limited, and queues may occur during peak hours. They are suitable for trial use and evaluation.
    • Integrated LLM service platforms: connect models from multiple vendors through a unified interface, covering invocation, fine-tuning, application orchestration, and hosting. They are more suitable for production environments.

    Another notable trend is that cloud vendors and chip vendors are actively adding Chinese models to their offerings. The reasons are easy to understand: for chip vendors, model invocation can drive hardware demand; for cloud vendors, models are an entry point for driving compute consumption. Whatever the motivation, this is good for developers: the threshold for trying models has dropped to nearly zero.

    1. The Real Boundaries of Free Quotas

    Before using free access, understand three things.

    • Billing: generative models are usually billed separately for input and output tokens. Free quotas are useful for learning the API and validating results, but before production adoption, you should estimate cost based on realistic usage.
    • Rate limits: individual access is usually limited by account or API key. Shared resource pools may queue or slow down during peak hours. This is the main cost of “free.”
    • Stability: model versions on a platform may not follow the vendor’s release schedule. Models may be updated, retired, or have quotas adjusted. Free quotas are suitable for trials, but should not be used for critical paths.

    The conclusion is simple: free quotas are useful for selection and evaluation. Production launch requires separate capacity planning.

    2. Preparation: Account, API Key, and Endpoint

    Using an open inference platform as an example, the setup process usually includes the following steps.

    1. Register an account: sign up with an email address and complete verification. Some platforms also require phone verification before enabling access. This step determines whether you are eligible to call the API.
    2. Create an API key: generate it in the console’s key management page and add a purpose note so you can distinguish environments and owners.
    3. Set validity: the default validity may be only a few days. After expiration, requests may suddenly return 401. You can extend validity for test environments, but production keys should be rotated regularly.
    4. Record the endpoint: open the model page and confirm the Base URL and model identifier. API calls usually combine the model ID with a path, so in code you only need to provide the Base URL.
    5. Store the key: many platforms display the full key only once at creation time. Put it in a password manager or a team secret management tool.

    One point worth noting: keys on generative model platforms often have access to all models on that platform, while billing usually applies to the platform account. This is fine for personal trials. In team environments, issue separate keys by purpose and by member, combined with usage statistics and accountability tracking.

    3. Configuration: Running Multiple Models Side by Side

    A GUI client is suitable for quick comparison. Using a desktop AI client as an example, the process usually has three steps.

    1. Add a model provider: choose “Custom Provider” or the “OpenAI-compatible” type, then enter the API key and Base URL.
    2. Add models: enter the model IDs. Versioned models often have lighter aliases, such as “mini” or “instruct” variants; add them as needed.
    3. Start comparison: in a new conversation, select multiple models at the bottom and set the display mode to “horizontal layout.” The same prompt can be sent to the selected models in parallel, and answers can be shown side by side.

    Many OpenAI-compatible model services also support Anthropic-style message interfaces. OpenAI-compatible APIs distinguish roles as system, user, and assistant, while Anthropic-style APIs place system instructions in a separate system parameter. Do not mix the two protocols. If the protocol is selected incorrectly, the client may put the entire message array into the system field, and the model will see only a block of JSON rather than a real instruction.

    Tools are only tools. The real challenge is how to ask and how to compare.

    4. Comparison: Control the Experiment First

    Before comparing models, align the experiment conditions.

    • Questions should come from real tasks: build a question set from prompts used in your business, for example 30 to 50 cases covering common scenarios and edge cases. Do not use “tell me a joke” as a benchmark.
    • Align parameters: set temperature to 0 or as low as possible, disable thinking mode, and limit the maximum output length to reduce noise caused by randomness.
    • Request order does not matter: shared resource pools may slow down during peak hours, but output quality should not change because of that. There is no need to deliberately wait for an “idle period.”
    • Sample multiple times: run the same question 3 to 5 times per model. Focus on stability rather than a single lucky response.

    A common mistake is treating the output from an inference platform as the model’s full capability. Sampling parameters, context window, inference mode, and the model version used during the call can all significantly affect the result.

    5. Scoring: Turning “It Feels Better” into a Reproducible Conclusion

    It is recommended to score models across six dimensions, each on a scale from 1 to 5, and then apply weights:

    • Instruction following (30%): whether the model strictly follows the requested format, fields, and length.
    • Factual accuracy (25%): whether it fabricates information, and whether it can clearly handle uncertain boundaries.
    • Reasoning depth (20%): whether multi-step problems are decomposed properly and whether the conclusion is consistent.
    • Chinese expression (15%): whether terminology is standard and whether the wording is natural when Chinese output is evaluated.
    • Response speed (5%): first-token latency and total response time.
    • Cost (5%): calculated from each model’s public price combined with average input and output length.

    Example Comparison Task: Extracting Structured Information from Product Reviews

    System prompt:

    1. You are a rigorous information extraction assistant. Extract information only from the original text. Do not infer or add content that is not present. Output JSON with keys: sentiment, praise, complaint, feature.

    Input:

    1. Customer support was very responsive, and delivery was on time, but the packaging was slightly crushed, and QR-code binding kept failing.

    Output format:

    1. {"sentiment":"positive/negative/neutral","praise":["…"],"complaint":["…"],"feature":["…"]}

    Example scoring record:

    Model Instruction following Factual accuracy Reasoning depth Chinese expression Response speed Cost Weighted score Main issue
    A 5 5 4 4 4 3 4.60 Medium speed
    B 4 5 5 5 3 4 4.50 Occasional extra explanation
    C 5 4 4 5 5 5 4.50 Praise mixed into complaint

    Example output, abridged:

    1. {
    2. "sentiment": "positive",
    3. "praise": [
    4. "customer support was responsive",
    5. "delivery was on time"
    6. ],
    7. "complaint": [
    8. "packaging was slightly crushed",
    9. "QR-code binding kept failing"
    10. ],
    11. "feature": [
    12. "customer support",
    13. "delivery",
    14. "packaging",
    15. "QR-code binding"
    16. ]
    17. }

    The behavior of the other two models can be summarized as follows:

    • B: placed “QR-code binding kept failing” under feature, which reminds us that the schema design needs to clearly distinguish “specific problems” from “affected features.”
    • C: even when temperature was set to 0, the order of JSON fields could still change. Systems that only parse by key can handle this, but regression tests based on text comparison need to parse the output before comparing.

    It is best to put the scoring table in a spreadsheet tool and use formulas to calculate the total score automatically, so that “which model is better” is driven by data.

    The general formula is:

    1. weighted_score = instruction_following * 0.30
    2. + factual_accuracy * 0.25
    3. + reasoning_depth * 0.20
    4. + chinese_expression * 0.15
    5. + response_speed * 0.05
    6. + cost * 0.05

    Replace the values with actual scores when applying it.

    6. Script: Run All Models with One Command

    On OpenAI-compatible services, the model is usually the only difference in the request body, specifically the model field. A short script can compare multiple models and roughly estimate cost:

    1. """Multi-model comparison script for Chat Completions-style APIs."""
    2. import json
    3. import os
    4. import time
    5. import urllib.error
    6. import urllib.request
    7. BASE_URL = os.environ["LLM_BASE_URL"].rstrip("/") # Example: https://<inference-platform-domain>/v1
    8. API_KEY = os.environ["LLM_API_KEY"]
    9. MODELS = os.environ["LLM_MODELS"].split(",") # Example: "model-a,model-b,model-c"
    10. SYSTEM_PROMPT = (
    11. "You are a rigorous information extraction assistant. "
    12. "Extract information only from the original text. "
    13. "Do not infer or add content that is not present. "
    14. "Output JSON with keys: sentiment, praise, complaint, feature."
    15. )
    16. USER_INPUT = (
    17. "Customer support was very responsive, and delivery was on time, "
    18. "but the packaging was slightly crushed, and QR-code binding kept failing."
    19. )
    20. # Prices are in currency per 1 million tokens. Check each model’s official pricing page.
    21. # These values are placeholders only.
    22. PRICES = {
    23. "model-a": {"in": 4.0, "out": 16.0},
    24. "model-b": {"in": 2.0, "out": 8.0},
    25. "model-c": {"in": 0.0, "out": 0.0},
    26. }
    27. def call_model(model: str, retries: int = 3) -> dict:
    28. payload = {
    29. "model": model,
    30. "messages": [
    31. {"role": "system", "content": SYSTEM_PROMPT},
    32. {"role": "user", "content": USER_INPUT},
    33. ],
    34. "temperature": 0, # Reduce randomness as much as possible
    35. "max_tokens": 512,
    36. }
    37. req = urllib.request.Request(
    38. f"{BASE_URL}/chat/completions",
    39. data=json.dumps(payload, ensure_ascii=False).encode("utf-8"),
    40. headers={
    41. "Authorization": f"Bearer {API_KEY}",
    42. "Content-Type": "application/json",
    43. },
    44. method="POST",
    45. )
    46. for attempt in range(retries):
    47. start = time.perf_counter()
    48. try:
    49. with urllib.request.urlopen(req, timeout=60) as resp:
    50. data = json.load(resp)
    51. usage = data.get("usage", {})
    52. return {
    53. "text": data["choices"][0]["message"].get("content", ""),
    54. "latency": round(time.perf_counter() - start, 2),
    55. "usage": usage,
    56. "cost": round(
    57. usage.get("prompt_tokens", 0) / 1e6 * PRICES[model]["in"]
    58. + usage.get("completion_tokens", 0) / 1e6 * PRICES[model]["out"],
    59. 6,
    60. ),
    61. }
    62. except urllib.error.HTTPError as e:
    63. if e.code in (429, 500, 502, 503, 504) and attempt < retries - 1:
    64. time.sleep(2 ** attempt) # Exponential backoff: 1s, 2s, 4s...
    65. continue
    66. raise
    67. raise RuntimeError(f"{model} call failed")
    68. for name in [m.strip() for m in MODELS if m.strip()]:
    69. try:
    70. result = call_model(name)
    71. print(f"=== {name} | {result['latency']}s | cost ≈ {result['cost']} ===")
    72. print(result["text"])
    73. except Exception as e:
    74. print(f"=== {name} === call failed: {e}")

    Before running, configure environment variables. Keep keys out of the code:

    1. export LLM_BASE_URL="https://<inference-platform-domain>/v1"
    2. export LLM_API_KEY="<your API key>"
    3. export LLM_MODELS="model-a,model-b,model-c"
    4. python compare.py

    To extend this into batch evaluation, two steps are enough: replace USER_INPUT with a loop over the question set, and append outputs as JSONL for analysis with pandas:

    1. import json
    2. import pandas as pd
    3. records = []
    4. for model in MODELS:
    5. for case in CASES: # CASES: [{id, system, user}, ...]
    6. try:
    7. r = call_model(model)
    8. records.append({
    9. "model": model,
    10. "case_id": case["id"],
    11. "text": r["text"],
    12. "cost": r["cost"],
    13. "latency": r["latency"],
    14. })
    15. except Exception:
    16. records.append({
    17. "model": model,
    18. "case_id": case["id"],
    19. "text": None,
    20. "cost": None,
    21. "latency": None,
    22. })
    23. with open("results.jsonl", "w", encoding="utf-8") as f:
    24. for r in records:
    25. f.write(json.dumps(r, ensure_ascii=False) + "\n")
    26. df = pd.read_json("results.jsonl", lines=True)
    27. print(df.groupby("model")[["cost", "latency"]].agg(["mean", "count"]))

    If writing results to a database, use parameterized statements rather than string concatenation. This point is also covered below.

    7. Selection: No Universal Model, Only the Best Fit

    After evaluation, map models to scenarios:

    • Creative writing and marketing copy: prioritize Chinese expression and divergent generation.
    • Long-document summarization and contract review: prioritize long context and faithful extraction of key points.
    • Structured extraction and data cleaning: prioritize stable output formats such as JSON.
    • Code generation and refactoring: prioritize cross-file reasoning, comment conventions, and test coverage.
    • Agent tool calling: prioritize correct function-call formatting and multi-turn state retention.
    • Batch offline tasks: prioritize unit price and throughput.

    Hybrid routing is often more cost-effective than “choosing one strongest model”:

    1. def route(prompt: str) -> str:
    2. if "```" in prompt or "def " in prompt:
    3. return "coding-model" # Code-related tasks
    4. if len(prompt) > 30000:
    5. return "long-context-model"
    6. return "general-model"

    If you also use integrated LLM service platforms such as Baidu AI Cloud Qianfan, a unified protocol and similar model parameters can further reduce switching costs.

    8. Before You Start, Revisit These Questions

    • Is the model and platform really more important than “free”? The real value of free quotas is lowering the trial threshold, not providing long-term zero cost. Many tutorials online focus entirely on how to register, but they do not explain how these models are billed in production or how to plan capacity.
    • Are the free quotas enough for your scenario? If every request requires tens of thousands of tokens of context, only a few dozen calls may exhaust the quota. Calculate the token volume per call, then estimate daily request volume, and only then discuss “free.”
    • Can you accept slower responses or model retirement? Personal experiments can tolerate this, but production environments need fallback models and degradation strategies.
    • Is the data compliant? Do not send raw data containing personal information or core business secrets to third-party platforms. Perform desensitization and compliance assessment first. When code is involved, also confirm the platform’s retention and training-use policies.
    • Is the platform reliable? Free platforms may also stop service or change policies, so always maintain portability.
    • Are you being misled by leaderboards? Leaderboards can only narrow the candidate list. Real business distribution is the final standard. It is recommended to run the same question set for regression evaluation every quarter. After model iterations, the answer may change.

    9. A Portable Integration Layer

    Tools can change, but interface habits should remain stable. In business code, do not hard-code one specific provider. Put an adapter layer in between, so future migration only requires changing configuration:

    1. from dataclasses import dataclass
    2. @dataclass(frozen=True)
    3. class Provider:
    4. base_url: str # Example: https://<inference-platform-domain>/v1
    5. api_key: str # Injected by a secret manager, not written into code
    6. model: str # Model identifier
    7. protocol: str = "openai" # "openai" or "anthropic"; do not mix them
    8. class ChatModel:
    9. """Unified chat entry point; switching platforms only requires a new Provider."""
    10. def __init__(self, provider: Provider):
    11. self.p = provider
    12. def chat(self, messages: list[dict]) -> str:
    13. # This can connect to services that support compatible protocols.
    14. # The core model field comes from provider.model.
    15. ...

    Migration checklist:

    • Protocol matching: OpenAI-compatible services use chat/completions; Anthropic-compatible services use messages. Do not mix system prompts into the message array.
    • Independent keys: issue one key per environment and per application so it can be revoked individually if leaked.
    • Error handling: handle 401 (invalid key), 429 (rate limiting), and 5xx (server errors) separately. For rate limiting, use exponential backoff plus random jitter.
    • Cost observability: record token usage and cost for each call, and set budget alerts.
    • Data compliance: define which data can leave the internal network. If necessary, deploy open-source models in a private environment.

    Closing Thoughts

    Once you complete these steps, you will have a platform-independent method: a question set, a scoring sheet, and an adapter layer. Which model is free and which model tops a leaderboard is only current information. This method is what allows you to avoid starting from zero the next time you see a “new model now available for free” announcement.