The Decline of Vertical AI Foundations: How Office AI Moves into the Deep End

1. The pattern first: why vertical capability foundations retreat

Over the past two years, the narrative of “specialized models replacing general-purpose models” has reversed quickly. This is not the product of a single operational failure but of a fairly stable technical-and-commercial pattern. If you extract what the absorbed vertical foundation products have in common, it comes down to three things.

1. A narrow user pool, and the hardest audience to please. Take vertical code foundations as an example. Their early value came from aiming language modeling at repositories, function signatures and test cases, serving developers. That group has an extremely low tolerance for inaccuracy (the cost of debugging a single wrong completion can exceed the time saved), and it is simultaneously sensitive to context scope, integration with local development environments, code-exfiltration compliance and per-call cost. Fail on any one of the four and switching costs are effectively zero — migrate by changing a line of configuration.

2. It delivers a capability, not a workflow. This is the most fatal of the three. Vertical foundations typically provide only model services or interfaces, pushing adaptation, orchestration, retries, error handling and result validation entirely onto the user. That means they cannot reach non-technical audiences, and therefore structurally forfeit lateral diffusion inside an organization. A capability that can only circulate within the engineering population has a ceiling set by headcount, not by business value.

3. A single scenario, and commoditized as soon as it becomes standard. Once general-purpose models quickly catch up on code understanding, repository-level retrieval and cross-file reasoning, what remains of a vertical model’s differentiation is usually a few percentage points on a benchmark, not a “can’t do the job without it” advantage. The commercialization structure amplifies the consequence: if pricing is anchored to call volume, commoditization translates directly into call-volume migration and a steep revenue drop. Add churn from model version churn and changes to billing definitions, and the retreat is only a matter of time.

In one sentence: for a product that delivers an underlying capability, the moat equals the length of the capability scarcity window; when that window closes, usage and revenue collapse together.

2. Applying the same logic to office work: two mistakes that are easy to make

Many people follow the reasoning above straight to the conclusion that “office AI tools for white-collar workers are doomed too.” The direction isn’t wrong, but the argument is, because it conflates two entirely different payment logics while badly underestimating how fast commoditization happens.

Mistake one: treating “everyday assistance” as a scenario in itself

Everyday office assistance has fully entered the era of zero marginal cost. Mainstream collaboration suites ship writing, summarizing, translation and spreadsheet-formula generation natively; general-purpose chat entry points are free; lightweight document-processing scripts run in a few dozen lines of code. Both enterprises and employees have already grown used to getting “polish this paragraph, extract the key points from these meeting notes, generate a first draft” for free.

Technically, these tasks fall exactly into the “short input, short output, errors correctable by eye” band — where model capability is in surplus and scenario value is too thin. The benefit users perceive is “a few minutes saved each time,” and those minutes can be neither booked to an account nor written into any performance report. A nice-to-have capability almost inevitably lands at the bottom of the budget sequence in enterprise purchasing decisions.

Pilots look lively on sign-ups and active usage; at renewal, someone looks at per-seat cost and realizes the same output is obtainable from any substitute. So why pay separately for something replaceable? That is the mechanism behind “great registration numbers, dismal usage,” and it is the same curve as the retreat of vertical foundations.

Mistake two: counting only “time saved,” not “workflow changed”

Real enterprise-grade AI value does not come from saving time. It comes from compressing an action that used to require crossing systems, roles and tickets into a single auditable closed loop. The two payment logics are entirely different:

Dimension	General-purpose assistance	Process closed loop
Source of value	Quality of a single generation	End-to-end task completion
Data dependency	Manually pasted by the user	Proprietary enterprise data and system state
Output form	A piece of text	Write operations / state changes on business systems
Switching cost	Very low	Requires rebuilding integrations and the permission model
Billing anchor	Seats / characters	Task volume / automation rate

The conclusion: everyday office work is only an entry point, used to accomplish three things — identity verification, data authorization and habit formation. If the product’s endpoint also stays at “writes a little better,” the ending is essentially already written.

3. Three hard criteria for judging whether an office scenario is worth building

Before locking an architecture, filter out pseudo-requirements with three questions:

1. Does it carry proprietary enterprise context? If the answer involves no internal documents, system state, org structure or historical records, a general-purpose model can answer directly and building a product is a pure loss.
2. Is there a writable action? A read-only Q&A capability tops out at “assisting decisions”; a read-write loop tops out at “replacing a human step.” The former is hard to charge for; the latter can be aligned directly against labor cost.
3. Is it measurable, reconcilable and auditable? Enterprises pay only for outcomes that both the audit function and the finance function will accept.

For example: contract first-pass review (a proprietary clause library + write-back into the approval flow + clause-level citations of the original text), pre-screening expense claims (integration with receipt and budget systems + structured determinations + a traceable record), converting R&D requirements into tickets (read the repository and the requirements backlog + write to the task system). All three are closed loops; “help me write a weekly report” is not.

4. A layered architecture for enterprise office agents

Turning those three hard criteria into engineering requires five layers, none of which can be omitted:

Access layer      Identity / org structure / data permissions / multi-client sessions
   ↓
Orchestration     Task planning / multi-step execution / tool routing / human checkpoints
   ↓
Action layer      Function calls / business system APIs / writes and rollbacks
   ↓
Knowledge layer   Document parsing / hybrid retrieval / vector + keyword / citation spans
   ↓
Governance layer  Audit logs / sensitive-data masking / quotas and cost attribution

On the infrastructure side, general cloud service categories are entirely sufficient: object storage for raw documents and intermediate parsing artifacts, a message queue for asynchronous decoupling and peak smoothing of long-running tasks, a container platform for elastic scaling of model and orchestration services, and log and monitoring services for call-chain tracing and cost attribution. There is no proprietary technique at this layer; the point is making every inference traceable to a specific person, a specific document and a specific cost.

Key implementation: tool calls must carry permissions and side-effect declarations

The most common production incident in enterprise settings is a model calling an interface the user had no right to access, or executing an irreversible operation twice. The tool layer should declare permissions and side-effect levels explicitly:

from dataclasses import dataclass
from enum import IntEnum
from typing import Callable
class SideEffect(IntEnum):
    READ_ONLY = 0      # safe to execute automatically
    REVERSIBLE = 1     # requires a second confirmation
    IRREVERSIBLE = 2   # requires human approval
@dataclass
class Tool:
    name: str
    handler: Callable
    scope: str            # required permission, e.g. "expense:read"
    side_effect: SideEffect
    idempotent: bool      # safe to retry
TOOLS: dict[str, Tool] = {}
def tool(name: str, scope: str, side_effect: SideEffect, idempotent: bool = False):
    def deco(fn):
        TOOLS[name] = Tool(name, fn, scope, side_effect, idempotent)
        return fn
    return deco
@tool("expense.precheck", scope="expense:read",
      side_effect=SideEffect.READ_ONLY, idempotent=True)
def expense_precheck(context, ticket_id: str) -> dict:
    """Run a compliance pre-check on an expense claim and return a
    structured determination plus clause citations."""
    ...
def dispatch(user, ctx, name: str, args: dict, confirmed: bool = False) -> dict:
    t = TOOLS[name]
    if not user.has_permission(t.scope):          # enforced up front, never left to the model
        raise PermissionError(f"{user.id} has no access to {t.scope}")
    if t.side_effect >= SideEffect.REVERSIBLE and not confirmed:
        return {"status": "need_confirmation", "tool": name, "args": args}
    # writes always carry an idempotency key so a retry cannot double-submit
    return t.handler(ctx, **args)

Three design points: permission checks must be enforced in the dispatcher and cannot be written into a prompt on the assumption that the model will comply; the side-effect level determines whether a human checkpoint is required; and every write operation must carry an idempotency key, otherwise a timeout-and-retry becomes a business incident.

Knowledge layer: traceable citations matter more than “sounding right”

The typical failure mode in office scenarios is not “wrong answer” but “an answer that sounds convincingly real.” The mitigation is to require structured citations: every conclusion in an answer must be traceable to a specific span of a specific document.

def answer_with_citation(query: str, top_k: int = 8) -> dict:
    dense = vector_search(query, top_k)          # semantic recall
    sparse = keyword_search(query, top_k)        # keyword recall, catches terms and IDs
    chunks = rerank(deduplicate(dense + sparse))[:top_k]
    prompt = (
        "Answer strictly from the material below; tag every conclusion with its source number; "
        "if the material is insufficient, state what is missing and do not speculate.\n"
        + "\n".join(f"[{i+1}] {c.text}" for i, c in enumerate(chunks))
    )
    out = llm.generate(prompt)
    return {"answer": out.text,
            "citations": [{"id": i+1, "doc": c.doc_id, "span": c.span}
                          for i, c in enumerate(chunks)]}

Pure vector retrieval sees recall drop noticeably on strongly terminological material such as policy numbers, contract clauses and product specifications. Hybrid retrieval plus a reranking stage is the comparatively stable industry practice today.

Cost governance: context budgets must be managed explicitly

Long documents + multi-turn conversation + multi-step tool calls can send the cost of a single task out of control. Workable controls:

• Tiered caching: cache document-level parsing results, retrieval results, and results for semantically near-duplicate questions;
• Context budgets: set a token ceiling per request; on overflow, compress retrieved spans first, trim conversation history second, and only then downgrade the model;
• Routing and downgrading: send tasks with clear intent and fixed output formats to a smaller model, escalating only for complex reasoning;
• Cost attribution: tag along three dimensions — tenant, department, scenario — so that “which scenario actually saves labor” becomes a calculable fact rather than an opinion.

5. Use an ROI model to set priorities, not intuition

Run a uniform estimate for every candidate scenario:

Annual net benefit = beneficiaries × frequency per person × minutes saved per run ÷ 60 × cost per person-hour × adoption rate
                   − (inference cost + integration cost + operations cost) × 12

The “adoption rate” is the variable most proposals miss. A feature that theoretically saves 40% of person-hours delivers far too little to cover integration cost if adoption is only 15%. So the rollout order should be: start with high-frequency processes that have clear pass/fail criteria and a low cost of error (expense compliance pre-screening, contract clause extraction, converting meeting notes into to-dos, ticket classification and routing), turn them into auditable closed loops, and only then replicate them laterally across similar processes.

6. Evaluation: no measurement, no iteration

Prepare three classes of metrics before launch; missing any one makes it impossible to tell afterward whether a change helped:

• Offline evaluation set: 200–500 real samples labeled by the business side, covering edge cases and adversarial cases, with pass rate and citation accuracy compared before and after every change;
• Online task metrics: task completion rate, first-pass rate, human-takeover rate, average handling time, cost per task;
• Governance metrics: unauthorized-access blocks, sensitive-data-exfiltration blocks, write-operation rollbacks.

The one most worth watching is the human-takeover rate. It directly reflects whether the product has actually taken over a step of the process: if it does not decline, the product is still sitting at the assistance layer, and there is no defensible number to put on the table in a renewal negotiation.

7. Conclusion

The retreat of vertical capability foundations was not caused by poor execution. Their value happened to be exactly equal to the capability scarcity window. To avoid repeating that path, office AI must move its positioning up from “help me write faster” to “help me get this done,” and on the engineering side make five things solid: permissions, writable actions, citation traceability, auditing and cost attribution. Using general cloud capabilities to carry storage, asynchrony, elasticity and observability, while concentrating effort on process closed loops and governance, is the only path that takes an office AI scenario from “great registration numbers” to “renewable.”