LLM Core Capabilities Explained: A Practical Guide to Building AI Agent Applications

    Large language models have evolved from passive question-answering systems into systems that can reason, generate code, plan tasks, and call external tools. This guide walks through a reusable methodology for evaluating, selecting, integrating, and deploying LLMs in production. It covers how to assess a model's capabilities beyond public leaderboards, the reasoning behind the Mixture of Experts (MoE) architecture, how to think about inference throughput, latency, and token-based billing, and how tool calling and the ReAct paradigm turn a chatbot into a task executor. It closes with engineering guidance on API compatibility, open-source licensing and compliance, cloud versus private deployment, and a hands-on example of building a photo-upload and poster-generation application.

    1. Introduction: LLMs Are Moving from Answering Questions to Doing Work

    Over the past two years, the evolution of large language models (LLMs) has followed a clear trajectory: they are no longer just passive “question-answering machines” but are progressively acquiring integrated capabilities for logical reasoning, code generation, task planning, and invoking external tools. This leap from “understanding language” to “executing tasks” is the core driver behind the current large-scale adoption of generative AI.

    For developers and enterprise users, a practical question emerges: faced with a constant stream of new models and tools, how do you scientifically assess a model’s capability boundaries and integrate it into your business systems in a way that is stable, low-cost, and maintainable? This article follows an “evaluate–select–integrate–apply” thread, aiming to provide a reusable methodology.

    2. How to Scientifically Assess an LLM’s Core Capabilities

    2.1 The Value and Limitations of Comprehensive Leaderboards

    A common industry practice is to have models compete in a series of standardized benchmarks covering mathematical reasoning, code generation, logical reasoning, general knowledge Q&A, and more, then compare them side by side using a unified score. The value of such comprehensive benchmarks is that they provide a reproducible, comparable reference point that helps developers form an initial impression quickly.

    However, leaderboard scores have clear limitations. First, benchmark questions may overlap with training data, creating a “test-taking” bias. Second, benchmark scenarios often differ substantially from real business use, so a high score does not necessarily mean good performance in your specific scenario. Third, leaderboards struggle to reflect engineering metrics such as latency, stability, and context length. Therefore, leaderboards can only serve as a first filter, not a final basis for decisions.

    2.2 Building a Multi-Dimensional Evaluation Set Tailored to Your Business

    A more rigorous approach is to build a private evaluation set that closely reflects your own business:

    • Task stratification: Break the business into typical tasks such as intent recognition, information extraction, copywriting, code completion, and multi-turn dialogue, and design test cases for each.
    • Scoring criteria: For open-ended generation, use human scoring or “model-as-judge” (using a stronger model to score outputs); for structured tasks (such as JSON output or classification), use hard metrics like accuracy and field completeness.
    • Robustness testing: Add adversarial inputs, extremely long text, and questions with ambiguous or false premises to observe the model’s stability and refusal boundaries.
    • Regression mechanism: After every model swap or upgrade, rerun the evaluation set to avoid hidden problems like “a version upgrade causing performance regression.”

    2.3 Capability Layering: Three Levels from Understanding to Execution

    When assessing, it helps to break capability into three progressive levels: the foundational understanding layer (can it accurately understand instructions and context?), the reasoning and generation layer (can it perform multi-step logical deduction and write runnable code?), and the execution layer (can it call tools, access external data, and complete end-to-end tasks?). The value of a truly “all-round” model lies in its ability to perform reliably at the third level as well, not just to “answer questions.”

    3. Architecture Explained: Why Mixture of Experts (MoE) Became Mainstream

    3.1 The Basic Principles of MoE

    In recent years, “Mixture of Experts” (MoE) has become one of the mainstream architectural choices for large models. Its core idea is sparse activation: the model contains multiple “expert” subnetworks internally, and during each inference a lightweight “router network” dynamically selects a small number of the most relevant experts based on the input to participate in computation, rather than having all parameters participate in every forward pass.

    An intuitive analogy: it is like a team of experts working in shifts, with different tasks assigned to the most capable members. This preserves breadth of capability while controlling the amount of computation per pass. The advantages of MoE include:

    • Large parameter capacity: The total parameter count can be made very large, giving broader theoretical knowledge coverage.
    • Relatively controllable inference cost: Because only some experts are activated at each step, the actual computation (activated parameters) is far smaller than the total parameter count, balancing capability and efficiency.

    3.2 Layered Selection Between Flagship and Lightweight Versions

    Around the same technical approach, vendors typically offer models of different sizes, forming a tiered product matrix:

    • Flagship version: Larger parameter scale and stronger reasoning, suited to scenarios with extremely high demands on quality, such as complex reasoning, high-quality code generation, and long-chain task planning.
    • Lightweight version: Smaller scale, lower latency, and lower cost, suited to high-frequency, cost-sensitive scenarios such as intent recognition, classification, summarization, and simple dialogue.

    The engineering best practice is often to “use them in combination”: use the lightweight version for front-end intent routing and simple tasks, then route complex tasks to the flagship version, striking a balance between quality and cost.

    4. Inference Performance and Cost: Throughput, Latency, and Billing Models

    4.1 Throughput and Time to First Token

    Two key metrics for evaluating the inference experience are:

    • Throughput (tokens/second): The number of tokens generated per unit of time, which directly determines whether output feels “as fast as typing” or “drip by drip.” Higher values yield a better experience for long-text generation.
    • Time to first token (TTFT): The time from issuing a request to receiving the first token. For interactive scenarios like real-time chat and code completion, TTFT often affects the experience more than total throughput.

    4.2 Understanding Token-Based Billing and Estimating Costs

    Most mainstream LLM APIs use a token-based billing model and typically price “input tokens” and “output tokens” separately (output is often more expensive because the generation stage is more compute-intensive). Understanding this lets you optimize costs through engineering means:

    • Trim the system prompt and context to reduce input tokens;
    • Use summarization and retrieval-augmented generation (RAG) instead of stuffing entire long documents into the context;
    • Enable prompt caching for cache-friendly, highly repetitive requests (if the service supports it).

    Below is a sample code snippet for estimating call costs (treat the price parameters as illustrative—use your actual official pricing; this only demonstrates the calculation method):

    1. def estimate_cost(input_tokens: int,
    2. output_tokens: int,
    3. price_in_per_1m: float, # Price per million input tokens
    4. price_out_per_1m: float # Price per million output tokens
    5. ) -> float:
    6. """Estimate the cost of a single call, priced per million tokens"""
    7. return (input_tokens / 1_000_000) * price_in_per_1m \
    8. + (output_tokens / 1_000_000) * price_out_per_1m
    9. # Example: assume a single request with 3000 input tokens and 500 output tokens
    10. # Replace price_in_per_1m / price_out_per_1m with the real unit prices of your chosen service
    11. cost = estimate_cost(3000, 500, price_in_per_1m=1.0, price_out_per_1m=2.0)
    12. print(f"Estimated cost per call: {cost:.6f}")

    By wiring this kind of estimation function into load testing and telemetry, you can forecast the total cost for a given call volume before launch, avoiding the situation where “quality meets the bar but cost spirals out of control.”

    5. From Conversation to Execution: Agents and Tool Calling

    5.1 Tool / Function Calling

    One key capability behind “doing work” is tool calling. The mechanism works like this: in your request you declare a set of available tools with their names, purposes, and parameter structures (usually a JSON Schema). When needed, the model returns a structured call intent (containing the tool name and arguments). Your program actually executes the tool, then feeds the result back to the model to enter the next round of reasoning, repeating until the task is complete.

    Below is a sample code snippet declaring a tool and initiating a call:

    1. # 1) Define the available tools and their parameter structures
    2. tools = [
    3. {
    4. "type": "function",
    5. "function": {
    6. "name": "generate_poster",
    7. "description": "Generate a promotional poster from a user-uploaded image and copy",
    8. "parameters": {
    9. "type": "object",
    10. "properties": {
    11. "image_url": {"type": "string", "description": "URL of the user's image"},
    12. "slogan": {"type": "string", "description": "Main slogan on the poster"}
    13. },
    14. "required": ["image_url", "slogan"]
    15. }
    16. }
    17. }
    18. ]
    19. # 2) Start a conversation with tools (client is a generic LLM client; fields depend on the actual service)
    20. resp = client.chat(
    21. model="<your-model-id>",
    22. messages=[{"role": "user", "content": "Use this avatar to generate a promotional poster with the slogan 'Now Launched'"}],
    23. tools=tools,
    24. )
    25. # 3) If the model returns tool_calls, execute the corresponding function and feed the result back before the next round
    26. # This step is handled by your business code; illustration:
    27. # result = generate_poster(**args)
    28. # messages.append({"role": "tool", "content": result})

    5.2 Multi-Step Planning and the ReAct Paradigm

    Complex tasks usually require multi-step reasoning. The widely adopted ReAct (Reasoning + Acting) paradigm has the model alternate between “thinking—acting—observing”: first analyze what to do next, call a tool to obtain external information, then decide the next step based on the observation, and so on until convergence. Combined with sensible limits on maximum steps and timeouts, this upgrades an LLM from “chat” into a reliable “task executor.”

    6. Engineering Implementation: Connecting LLMs to Your Application

    6.1 API Compatibility Lowers Migration Costs

    Today many model services offer an interface style consistent with widely adopted industry conventions (for example, a unified Chat Completions style). This means that when switching between different models in your code, you often only need to replace the model identifier and endpoint, without rewriting business logic. This compatibility significantly reduces vendor lock-in risk and makes multi-model failover and canary rollouts convenient.

    6.2 Open-Source Licensing and Compliance Considerations

    Some models are released as open source, with weights hosted on mainstream open-source communities and model repository platforms under permissive licenses such as MIT and Apache 2.0. Before commercial use, be sure to verify three things: whether the license type permits commercial use and redistribution, whether it requires attribution or open-sourcing of derivative works, and whether the training data sources and privacy compliance meet your industry’s regulatory requirements. The greatest value of open source is the ability to deploy privately, satisfying needs such as keeping data within your own domain, low latency, and deep customization.

    6.3 Weighing Cloud APIs Against Private Deployment

    • Cloud API: Fast to integrate, maintenance-free, and elastic, suited to rapid validation and fluctuating traffic; the concerns are data-egress compliance and long-term cost.
    • Private deployment: Full data control and deep fine-tuning, suited to strongly regulated, stable high-load scenarios; the cost is the need for GPU resources and operational capability.

    Cloud platforms, including providers such as Baidu AI Cloud, typically offer large-model services that can be paired with general-purpose cloud products such as object storage, content delivery, logging, and monitoring and alerting, forming a complete engineering loop from inference to storage and from observability to elastic scaling. When selecting, it is advisable to run a weighted evaluation across four dimensions: compliance, cost, latency, and ecosystem.

    7. Hands-On Example: Building a “Photo Upload + Poster Generation” Application

    Here is a typical lightweight application that ties the capabilities above together:

    Requirement: A user uploads an avatar photo and fills in a slogan; the system automatically calls the LLM to generate a downloadable promotional poster.

    Architecture design:

    1. The front end provides image upload and copy input;
    2. The back end writes the image to object storage to obtain an accessible image URL;
    3. It calls the LLM, triggering the “poster generation” tool through the tool-calling mechanism (inputting the image URL and slogan);
    4. The result is written to object storage and a download link is returned;
    5. Logging and monitoring and alerting record call latency, token consumption, and failure rates.

    Key flow illustration:

    1. def handle_generate(user_image_bytes: bytes, slogan: str) -> str:
    2. # 1. Upload the original image to object storage to get its URL
    3. image_url = object_storage.put(user_image_bytes)
    4. # 2. Let the model decide to call the poster generation tool (reuse the tools definition from 5.1)
    5. resp = client.chat(
    6. model="<your-model-id>",
    7. messages=[{"role": "user",
    8. "content": f"Generate a poster from this image: {image_url}, slogan: {slogan}"}],
    9. tools=tools,
    10. )
    11. # 3. Execute the tool and feed back the result
    12. for call in resp.tool_calls or []:
    13. args = json.loads(call.function.arguments)
    14. poster = generate_poster(args["image_url"], args["slogan"])
    15. poster_url = object_storage.put(poster)
    16. # 4. Return the download URL
    17. return poster_url

    This example shows that the LLM plays the role of an “orchestrator,” responsible for understanding intent and deciding on tool calls, while general-purpose cloud capabilities such as object storage, logging, and monitoring ensure stability and observability. Developers should focus on business orchestration and quality control rather than reinventing the wheel.

    8. Selection Recommendations and Best Practices

    Bringing the analysis together, here is an actionable checklist:

    • Evaluate first, then select: Replace a “leaderboard-only” mindset with a private evaluation set tailored to your business, covering all three layers of understanding, reasoning, and execution.
    • Tier your model usage: Use the lightweight version for high-frequency simple tasks and the flagship version for complex reasoning, balancing quality and cost.
    • Do the cost math: Factor token-based billing, prompt caching, and RAG into your design, and run cost estimation and load testing before launch.
    • Embrace interface standardization: Prefer services compatible with mainstream conventions, preserving the ability to switch between models and fail over.
    • Take compliance and open-source licensing seriously: Verify licenses and data compliance before commercial use, and prioritize private deployment for sensitive scenarios.
    • Back it up with engineering: Use general-purpose infrastructure such as object storage, logging, and monitoring to build stable, rollback-capable systems with observability.

    The value of a large language model does not lie in how impressive it is on any single metric, but in whether it can thread “understanding—reasoning—execution” into a complete chain and embed it reliably into your business workflows. Mastering evaluation methodology, understanding the essence of architecture and cost, and backing it with solid engineering capabilities is the key to making AI truly “do the work.”