From Chat to AI Agents: Protocol Selection, Token Cost Modeling, and Acceptance Checklists

    Summary: The center of gravity in LLM applications is shifting from chat windows to agents embedded in concrete workflows, and that shift can be reasoned about with engineering tools rather than product rankings. This article walks through three lines of analysis: where token spend actually lands once you break cost down to the task level, how to choose between open agent protocols such as MCP and A2A and vendor-proprietary interfaces, and how to select image models against measurable, scenario-specific criteria. It also covers how to read product traffic data responsibly, the telemetry fields needed for token accounting and cost attribution, and five common mistakes that quietly inflate agent bills. Wherever a claim depends on external data that cannot be verified, it is flagged as an unverified assumption with a stated verification method.

    Introduction: The Center of Gravity Is Moving from Chat Windows to Workflows

    For the past two years, the dominant interaction form for large models has been the chat box. As reasoning quality, context length, and tool calling matured, however, the center of application has shifted from conversational Q&A toward agents embedded in specific workflows.

    This is not a claim that depends on the ranking of any single product. It rests on three things an engineering team can verify on its own: which stages consume compute, which protocol is used to connect tools, and what criteria model capability is accepted against. This article follows those three lines, and every conclusion that relies on external data is labeled with its definition and its verification method. Anything that cannot be verified is treated as an unverified assumption.

    1. Where the Compute Actually Goes: Restoring Cost to Task Granularity

    An earlier draft cited “empirical observations from large-scale inference traffic” without naming a report, a time range, or a measurement basis, which makes it unverifiable. It is rewritten here as a self-verifiable hypothesis plus a method.

    Unverified hypothesis H1: Per-task token consumption comes mainly from three sources — code generation and completion (long context plus frequent tool output fed back into the context), complex reasoning (chains of thought produce a large number of intermediate tokens), and agent task orchestration (multiple tool-call rounds plus retrieval and regeneration). H1 should be validated with your own instrumentation rather than quoted from secondary sources: a measurement window of at least four weeks, a sample basis of “complete traces,” and a metric of “total tokens per successful task.”

    If H1 holds, three engineering consequences are predictable. Code requests are sensitive to time-to-first-token latency and to context window size. Reasoning requests consume far more in practice than the user-visible amount of Q&A suggests. And an agent task can consume tens of times the tokens of a single conversation, with a substantial share of that spend being wasted tokens from retries and failures.

    A minimal cost model (all values below are hypothetical and only illustrate the difference in magnitude):

    Form Requests/day Input per request Output per request Task multiplier Tokens/day
    Single-turn Q&A 10,000 800 300 1 11M
    Single-turn reasoning 10,000 800 1,200 (incl. chain of thought) 1 20M
    Agent task 1,000 3,000 1,500 8 (multi-round + tool feedback) 48M

    The conclusion is clear: request volume drops by an order of magnitude while tokens go up. Writing “average daily requests” into a budget approval is therefore a useless metric. The budget line has to read “daily tokens × unit price × task multiplier,” with the retry share broken out separately.

    2. Agent Protocols: Choosing Between Open Standards and Proprietary APIs

    The earlier draft used de-branded phrasing such as “agent context protocol” and “agent configuration description format,” which left readers unable to act. The concrete objects being referred to are:

    • MCP (Model Context Protocol): open-sourced by Anthropic in November 2024, used to connect models to external tools and data sources through a unified schema. It is already supported by several model providers and IDE clients.
    • A2A (Agent2Agent): proposed by Google in 2025 and donated to the Linux Foundation. It addresses interoperability between agents (task delegation, capability discovery) and sits at a different layer than MCP, which addresses tool access.
    • AGENTS.md: an open convention for agent configuration files that puts build, test, and lint commands into the repository, replacing proprietary agent configuration.

    Governance ownership, release cadence, and stewardship for all three can change. Before adopting, confirm the current status and latest version in each project’s official repository, and do not treat this article as an authoritative statement of governance today.

    Selection boundary: tool and data access should go through MCP-style protocols, cross-system agent collaboration through A2A-style protocols, and proprietary interfaces only when performance or permission constraints require them — behind an adapter layer that isolates them.

    Dimension Open protocol Vendor-proprietary API
    Tool description Structured schema, reusable across models Tightly bound to the SDK, migration requires a rewrite
    Ecosystem Supported by multiple clients Single platform
    Lock-in risk Low, buffered by the adapter layer High, replacement cost grows linearly with the number of tools
    Special capabilities Constrained by the pace of protocol evolution Available ahead of the standard
    1. # Standardized tool invocation: dependency injection + error isolation + version negotiation
    2. class ToolVersionError(RuntimeError):
    3. """Protocol version mismatch; the caller must degrade, not retry."""
    4. def call_tool(tool_spec: dict, arguments: dict,
    5. resolver, registry, max_retries: int = 2) -> dict:
    6. if tool_spec["protocol_version"] not in registry.supported:
    7. raise ToolVersionError(tool_spec["protocol_version"])
    8. handler = resolver(tool_spec["name"]) # injected from the registry, so implementations are swappable
    9. for attempt in range(max_retries + 1):
    10. try:
    11. return handler(**arguments) # the tool must be idempotent, otherwise retrying is unsafe
    12. except TransientError:
    13. if attempt == max_retries:
    14. raise

    Boundaries worth stating explicitly: tools must be idempotent, or a retry will charge twice or place a duplicate order. A version mismatch is a contract error and should fail fast and trigger degradation, not be treated as a transient fault and retried.

    3. Choosing Image Models by Scenario: Four Measurable Axes and Common Mistakes

    Different image models have different strengths and none leads on every axis, so selection has to be tied to a scenario and a measurable criterion rather than to leaderboard position.

    Scenario Primary metric (measurable) Test method Suggested passing bar Common mistake
    Creative generation Prompt adherence rate, usable output rate 50 fixed prompts × 3 samples Usable rate ≥ 70% Judging on the single best image
    Retouching / editing Source fidelity (SSIM/PSNR), local-edit hit rate 20 source images, one edit each time Fidelity takes priority over appearance Evaluating edits with text-to-image metrics
    Text rendering Character-level accuracy for short / medium / long text 20 examples at each of three lengths Medium and long text ≥ 90% Testing only short English words
    Character consistency Cross-scene consistency (human rating or CLIP similarity) 5 images of the same character in different scenes Human rating ≥ 4/5 Testing the same pose and lighting only

    Evaluation discipline: fix the random seed and the number of samples, record the model version and the call date, reuse the same evaluation set across models, and do not accept a single demo image as a basis for selection.

    4. Reading Product Traffic Data: Define the Basis First, Then Draw Conclusions

    The earlier claim that “leading general-purpose chat products are slowing down or even declining quarter over quarter” is an unverifiable absolute trend and has been removed. A verifiable reading requires three things:

    1. Data source: name the platform (for example Similarweb or Sensor Tower) and its sample basis — whether web and app are counted separately, what the deduplication rule is, and whether third-party distribution builds are included.
    2. Measurement window: at least 12 months of monthly series, which is what makes seasonality distinguishable. A single quarter-over-quarter movement is not enough to support a conclusion.
    3. Product subject: name the specific product and version rather than referring vaguely to “leading chat products.”

    Under those conditions, conditional statements are more reliable. If a product declines quarter over quarter for two or three consecutive months on a consistent measurement basis while comparable competitors rise in the same period, a structural competitive shift is a reasonable reading. If the whole category moves down together, it is more likely a change in measurement basis or a seasonal effect. If only one platform’s data moves, start by investigating sampling variation on that platform.

    5. A Checklist for Engineering Teams: Instrumentation Fields, Cost Attribution, Common Mistakes

    Token accounting and cost attribution (define the field semantics first, then instrument):

    1. # Instrumentation fields (append one record per reasoning step and per tool call)
    2. # trace_id: unique identifier for one complete task, shared across an agent's rounds
    3. # step_id: sequence number of the step within the task
    4. # model: model name + version + call date
    5. # input_tokens / output_tokens: prompt and generated tokens
    6. # tool_call_tokens: tokens from the tool response fed back into context
    7. # retries: retry count for this step, used for cost attribution
    8. # latency_ms / status: end-to-end duration and success/failure
    9. def task_cost(steps: list[dict]) -> dict:
    10. ok = [s for s in steps if s["status"] == "ok"]
    11. if not ok:
    12. raise ValueError("Task has no successful step; attribution is impossible")
    13. return {
    14. "input": sum(s["input_tokens"] for s in steps),
    15. "output": sum(s["output_tokens"] for s in steps),
    16. "tool": sum(s["tool_call_tokens"] for s in steps),
    17. "retries": sum(s["retries"] for s in steps),
    18. "trace_id": ok[0]["trace_id"],
    19. }

    The key metric is total tokens / successful tasks. Tokens spent on retries and failures count toward cost but produce no business value, and looking at averages alone hides a retry storm. The retry share therefore has to be alerted on the same screen as cost.

    Five common mistakes

    1. Evaluating cost by request count only, which severely underestimates the budget for agent workloads.
    2. Binding the tool layer directly to proprietary APIs, forcing a full rewrite at migration time.
    3. Shipping a new model to all traffic at once, with no canary rollout and no rollback switch.
    4. Writing tool descriptions into prompts instead of a structured schema, which leaves versions unmanageable.
    5. Missing idempotency in tools, so a single retry turns into a duplicate order or a duplicate charge.

    General cloud capability reference (examples only, not tied to any particular vendor): agent services can run on a container platform so they scale; message queues decouple long-running tasks for asynchronous orchestration; log- and metric-based alerting supports cost governance, with the token instrumentation above turned into a dashboard; business data lives in object storage, decoupled from the model provider so switching stays easy. At selection time, treat these as a “minimum viable foundation” checklist and confirm each item one by one.

    FAQ

    Q: How do I start migrating agent protocols? Introduce the open protocol at the tool layer only and add an adapter layer so business logic stays unaware of it. Roll out canaries per tool rather than switching everything at once.

    Q: How can I estimate token cost quickly? Use tasks per day × steps per task × (input + output per step) × unit price, then add a 20%–30% retry buffer and calibrate against real instrumentation after four weeks.

    Q: How should I accept an image model? Use the four-axis table in Section 3, fix the test set and the number of samples, record the model version, prioritize fidelity for retouching, and prompt adherence for creative generation.

    Q: How do I avoid vendor lock-in? Keep data and business logic in your own storage, adapt tools and models through a protocol layer, and keep availability verification for one alternative vendor in both contracts and budget.

    Conclusion

    Competition in AI has moved from “whose conversation is smarter” to “who is embedded more deeply in the workflow.” The migration of compute spend, the standardization of protocols, and the scenario-based selection of image models together sketch the technical trajectory from chat window to infrastructure. For developers, building your own measurement basis and instrumentation is worth more than chasing any single leaderboard.