Guide to Selecting Chinese LLMs: Free API Integration and Multi-Model Horizontal Evaluation

    As Chinese large language models (LLMs) proliferate, selecting the right one for specific business scenarios becomes a daunting technical challenge. This article introduces a low-cost, high-efficiency evaluation strategy. By leveraging free model inference microservices provided by leading computing hardware vendors, developers can quickly access mainstream Chinese LLMs through a unified API. Combined with universal desktop clients for multi-model parallel testing, this approach allows for comprehensive horizontal capability evaluations without incurring high commercial API procurement costs.

    With the rapid development of AI technology, Chinese large language models (LLMs) have experienced explosive growth, with various flagship models emerging continuously. For developers and enterprise users, choosing the most suitable model for their business scenarios has become a daunting technical challenge. Directly purchasing commercial APIs for comprehensive testing is not only costly but also introduces technical overhead due to differing API standards and calling methods across providers. This article introduces a low-cost, high-efficiency model selection practice: quickly accessing mainstream Chinese LLMs through free model inference microservices (NIM) provided by leading computing hardware vendors, and conducting horizontal capability evaluations using universal desktop clients.

    1. Inference Microservices and LLM API Status

    In recent years, some top-tier computing hardware vendors have launched “model inference microservices” (NIM) to foster their ecosystems. The core value of these services lies in acting as an “LLM supermarket”: they uniformly package mainstream open-source and commercial LLMs and provide standardized API interfaces.

    In the past, these platforms mostly hosted international open-source models; today, an increasing number of excellent flagship Chinese LLMs are being listed. This means developers no longer need to register accounts and apply for quotas with each model vendor individually. By obtaining a single API Key on one platform, developers can access multiple LLMs, including those in the first tier of Chinese models. This aggregated service significantly lowers the time and financial barriers for developers to trial and evaluate new models.

    2. Account Registration and API Key Management

    To use these inference microservices, the first step is to register and authenticate an account on the platform.

    1. Account Registration and Identity Verification:
      Visit the developer center of the respective computing platform and register with an email. Note that to ensure the reasonable allocation of service resources, platforms usually require additional identity verification. After registration, developers must locate the “Verify” or similar identity verification portal in account settings and complete phone number verification. In practice, standard local phone numbers typically pass verification smoothly.

    2. Generating API Key:
      After logging into the platform console, generate a new calling credential in the API or key management page. When configuring key attributes, it is recommended to set the expiration time to “Never” or the maximum available period to prevent service interruptions during prolonged development or evaluation processes. The generated key usually has a specific prefix format (e.g., a random string starting with xxx-), granting permissions to call all free models available on the platform. In actual development, this key should be securely stored in environment variables or a secure key management service, avoiding hardcoding in codebases.

    3. Configuration and Integration of Universal Desktop Clients

    After obtaining the API key, developers can write their own code to call the API. However, if the goal is rapid testing and manual comparison, using desktop client tools compatible with industry-standard API formats (such as the mainstream Chat Completions format) is a better choice. These tools typically offer intuitive graphical interfaces and support custom third-party API integration.

    The configuration process generally follows these steps:

    1. Download and Install the Client: Choose a desktop AI assistant tool that supports custom model service providers.
    2. Configure Model Service: In the client’s settings interface, add a new custom model service. The provider name can match the computing platform (e.g., “NVIDIA”).
    3. Fill in Core Parameters:
      • API Key: Paste the key string generated earlier.
      • API Address: Enter the unified inference interface domain provided by the computing platform (usually in the format https://integrate.api.xxx.com).
      • Model ID: Enter the corresponding model identifier based on the model list available on the platform. For example, you can enter the flagship version of Model A or the latest version of Model B. Note that the latest versions released by some vendors might not be synced to the aggregation platform immediately; in such cases, you can conduct basic capability tests on previous versions and iterate once the new versions are online.

    After saving the configuration, the client can establish a connection with the remote LLM and start interactive conversations.

    4. LLM Horizontal Evaluation Methodology and Comparison Techniques

    Simple single-model conversations make it difficult to intuitively perceive capability differences between models. Excellent desktop clients usually offer a “multi-model parallel conversation” feature, which is a powerful tool for horizontal evaluation.

    1. Multi-Model Parallel Evaluation Setup:
      In the model selection bar at the bottom of the client, you can simultaneously check multiple configured target models. When you input a prompt and trigger a response, the client sends requests to different model APIs in parallel and displays their respective outputs in horizontally arranged windows. This UI design allows developers to see differences in response speed, content quality, and formatting at a glance.

    2. Constructing a Standard Evaluation Benchmark:
      To ensure objectivity, developers should prepare a set of test cases covering different dimensions in advance:

      • Basic Logic and Math Capabilities: Test the model’s accuracy and stability through complex logic reasoning or math calculation problems.
      • Code Generation and Web Development Capabilities: Referencing well-known WebDev leaderboard testing logic, require the model to generate front-end component code for specific functionalities, evaluating code executability and standardization.
      • Long Text Understanding and Instruction Following: Input lengthy documents with complex formatting requirements to test if the model can precisely follow instructions to extract key information.
    3. Resource Scheduling and Error Handling:
      Since free inference microservices usually have concurrency or resource quota limits, you may occasionally experience response latency or request queuing during multi-model parallel evaluations. For individual developers conducting routine selection tests, this delay is usually acceptable. However, when writing automated evaluation scripts, reasonable retry mechanisms and timeout handling should be implemented to ensure the robustness of the testing process.

    5. Conclusion

    In an era where Chinese LLMs are flourishing, model selection should not be blind trend-following but rather rigorous testing based on real business scenarios. By utilizing the free inference microservices provided by computing platforms, developers can cross the API integration threshold at minimal cost and gain access to a host of top-tier Chinese LLMs. Coupled with efficient desktop multi-model parallel comparison tools, developers can quickly establish their own evaluation benchmarks, thereby screening out the optimal solution that truly fits their business needs among numerous candidate models. This low-cost, high-efficiency evaluation practice is worth adopting and referencing by every developer in their daily work.