# How to Validate AI Agent MAU Numbers: A Framework for Judging Technical Moats and Paid Capacity

Note: All product references in this article (“a certain AI workbench,” “a major platform vendor,” “a certain open-source agent project”) are illustrative placeholders. Every figure is either a commonly observed industry range or a stated assumption, used only to demonstrate the analysis framework. None of them represents the actual data of any specific product.

1. The framework matters more than any single number

A growth narrative like “a certain AI workbench passed 20 million monthly active users a few months after launch” raises three questions that are fully independent of each other:

1. Measurement definition. How were those 20 million counted? Accounts, devices, or sessions? Was the number inflated by incentive traffic from growth campaigns?
2. Technical exclusivity. With generic model supply becoming abundant, what actually prevents competitors from substituting an equivalent product?
3. Total paid capacity. How wide is the “paid square” behind the free wall? After deducting inference cost, how much confirmable revenue is left?

Data, moat, and paid capacity cannot substitute for one another. A product can simultaneously have high MAU, no moat, and a very thin confirmable revenue line. The rest of this article breaks down how to assess each one.

2. Prior constraint: commoditized model supply compresses product-layer exclusivity

Before discussing moats, look at the supply structure of this category. An agent product’s capabilities can be roughly split into four layers, with exclusivity difficulty increasing at each step:

Capability layer	Typical content	Difficulty of exclusivity
Model	General-purpose language models, reasoning, long context	Low: mainstream vendors’ capability gaps converge on a scale of months
Engineering	Task planning, tool calling, sandbox execution, permission control	Medium: depends on engineering investment, but architectures are public and easy to replicate
Skills	Domain workflow templates, Skill/plugin ecosystems	Low-to-medium: open-source communities already provide large amounts of reusable implementations
Data and trust	Private data connectivity, permission and compliance, audit trails, enterprise integration	High: deeply bound to an organization’s existing systems

One judgment is worth stating plainly: most of an agent product’s “impressive” capabilities do not come from a technical breakthrough of its own. They come from the natural overflow of underlying model capability, plus ready-made building blocks from open-source skill libraries. This is the root cause of a pattern seen repeatedly — an open-source agent project attracting large traffic the moment it launches, then being ported into managed versions across multiple cloud platforms. The differentiation window at the product layer is extremely short. Who ships first matters little; who sustains better distribution and integration matters a lot.

This does not mean the product layer has no value. The value of engineering lies in “stable, safe, observable,” not in “an algorithm nobody else can build.” The first is a long race; the second is a sprint.

3. Test 1: how to verify a 20 million MAU figure

3.1 Four common ways activity definitions get inflated

MAU is one of the easiest metrics to redefine. The first thing to do when auditing an MAU number is pin down the definition of the active event:

• Definition drift. Counting “opened the app” as active, then also counting “completed a task,” then also counting “brought back by a push notification.” The three definitions can differ by several multiples.
• Confused deduplication granularity. Deduplicating by account, by device, and by device fingerprint produces entirely different results. Multiple accounts on one device, or one person on multiple devices, makes the three definitions contradict each other.
• Incentive traffic contamination. Accounts created by cash-back signups, check-in tasks, and referral rewards count toward MAU in the short term but are almost invisible on the retention curve.
• Preinstallation and default entry points. Products that count as active simply by being preinstalled typically show DAU/MAU ratios below 0.05. These figures carry the least technical substance.

So the first move in judging authenticity is not looking at the size of the number, but demanding the active event definition, the deduplication dimension, and the measurement window. A number whose owners cannot state those three things is not comparable to anything.

from datetime import datetime, timedelta
# An auditable activity check: distinguish a "launch" from a "completed task"
def is_active(user, window_start, window_end,
              task_events=("task_started", "task_completed")):
    """
    The activity check should satisfy:
    1) Deduplication: by stable account ID (not device ID)
    2) Event granularity: count "a task started and completed"; exclude pure launches
    3) Time window: at least once within a calendar month
    4) Exclude test and internal accounts
    """
    if user.is_internal or user.is_bot:
        return False
    events = [e for e in user.events
              if window_start <= e.ts < window_end
              and e.name in task_events]
    # require at least one successfully completed task, to avoid counting "just opened it"
    return any(e.name == "task_completed" and e.duration_ms > 3000
               for e in events)

3.2 Triangulation: cross-check with three independent data sets

A single number cannot validate itself. It has to be cross-checked against three independent sets of data:

• Activity structure. DAU/MAU stickiness, average usage days per user, next-month retention. A healthy work tool usually shows a “low DAU/MAU, high average days” shape — used monthly, but only once or twice a week. If MAU is very high while next-month retention falls off a cliff, those 20 million were almost certainly bought.
• Call depth. Average daily tasks per user, average input/output tokens per user, distribution of context lengths. Marketing-driven activity looks like “open and leave,” with an extremely short median task duration. Real production use shows a clear long tail of long tasks and long-context sessions.
• Resource-side bill. Inference token consumption has to reconcile with MAU. If MAU triples while inference cost barely moves, activity is not converting into real calls and the number does not hold. Conversely, if cost grows far faster than MAU, the share of heavy users is anomalous and the retention curve needs to explain it.

3.3 A quantitative decision rule

Put the three together and you get a rough but effective discriminator:

• If average daily tasks per user < 0.2 and next-month retention < 15%, MAU is mainly a paid-acquisition result and does not constitute a user asset.
• Only if average daily tasks per user > 2 and next-month retention > 35% is there a basis for discussing moats and paid conversion.
• If inference cost growth persistently outpaces MAU growth, the business model needs recalculation even if the data is genuine (see Section 6).

The right answer to “is 20 million MAU real?” is this: without cross-validation across active event definition, retention curves, and resource bills, any affirmation or denial is a guess. And even if it is real, that does not make it valuable — a real number only proves people showed up. It does not prove anyone stayed, and it does not prove anyone paid.

4. Test 2: are usage habits a moat

A common defense is that once users form a usage habit, they will not migrate. This does not hold up technically, for three reasons:

1. Migration cost is overestimated. Migration cost for agent products is mostly “reconfiguration” cost: skill lists, permission settings, workflow templates. Those are exactly the standardized artifacts that can be exported and imported in one click. Because replicating the software is easy, a competitor can ship a compatible importer within weeks.
2. Switching cost is diluted by organizational decisions. Individual habits determine “which one do I use today.” Procurement determines “which one do we renew next year.” In B2B settings, habit is not a renewal argument; price, compliance, and integration capability are.
3. The conditions for habit formation do not hold. Habits come from high-frequency, stable, low-friction use. If a product’s capability mainly comes from the natural overflow of a general model, the experience fluctuates with each model version. “A stable expectation” does not exist, so habit cannot consolidate.

The assets that genuinely carry structural barrier properties are four: private data and workflow embedding, permission and audit compliance systems, ecosystem integration (deep coupling with an organization’s existing systems), and cost structure (whether unit inference cost can be an order of magnitude below a competitor’s). None of these can be replaced by “the users find it convenient.”

5. Test 3: total paid capacity and the “paid square” paradox

5.1 How to segment users correctly

Segmenting by usage volume produces wrong conclusions. The correct segmentation is by depth of value realization:

• L1, trial tier (the large majority): a handful of uses per month, fully covered by free quota. This tier produces almost no revenue — only word of mouth and data.
• L2, habit tier: reasonably stable monthly usage, but low task value (formatting, summarizing). Willing to pay a small subscription fee for convenience.
• L3, production tier (a small high-value population): tasks are embedded in production workflows, failure has a cost, and they will pay for stability, permissions, and audit.
• L4, organization tier: purchased per seat or per usage, with compliance and integration as the decision basis, not novelty of features.

Where the paywall sits determines how thick L2 and L3 are. Putting the paywall on usage volume (for example, N tasks per month) angers L2 (who exceed it by a hair) and L3 (whose production tasks are inherently high-frequency) at the same time. This is the easiest design to fail.

5.2 The paradox: homogeneous capability sources make “differentiated features” a weak basis for a paywall

The most important claim in the original argument is this: if you cannot monetize incremental usage, you have to monetize differentiated features. But an agent’s capabilities mostly come from the model itself and from open-source skills, so building a genuinely exclusive feature is very hard.

The paradox can be formalized:

• A paywall requires “exclusive supply” — users pay for a feature because they cannot get it elsewhere.
• If the feature is assembled from a general model plus open-source skills, any team with engineering capability can replicate it within weeks, so exclusivity approaches zero.
• When exclusivity approaches zero, the paywall can only sit on a usage quota, and quota is unfriendly to L2 users and insufficient for L3 users.

Therefore an agent product’s real paywall is usually not about features but about “responsibility”: who is accountable when it fails, where the data lives, whether it can be audited, whether it can integrate with existing systems. This also explains why enterprise pricing logic is typically based on seats and compliance tiers rather than feature points.

5.3 A reference point for total capacity: the ceiling set by collaborative office software

A useful empirical reference: the leading collaborative office suites have been operating for years, and their combined annualized revenue is still in the low single-digit billions of dollars. That line characterizes what small and midsize businesses actually pay for SaaS, not what the product is capable of.

This yields a counterintuitive conclusion: when collaborative office tools evolve into intelligent agents, their revenue will not grow exponentially. Enterprise software spend is anchored to total effective workload. If an agent merely performs existing work faster and does not create new deliverable output, it substitutes existing budget rather than creating new budget. Buyers will treat it as an “efficiency tool” and negotiate the price down, not treat it as a “growth engine” and pay a premium.

The counterexamples are scenarios that genuinely create new workload: data analysis, compliance review, and cross-system reconciliation that were never done at all because the cost was too high. Those are the entry points that open new budget.

6. Test 4: cost rigidity and unit economics of model-API-driven products

This is the most frequently overlooked and most lethal section.

6.1 The agent unit economics model

An agent task is not one API call. It is a “multi-step loop with accumulating context”:

# Token consumption estimate for a single task (illustrative assumptions)
TASK_STEPS = 8               # average tool calls / reasoning turns per task
SYSTEM_PROMPT = 3000         # system prompt and tool descriptions
TOOL_OUTPUT_PER_STEP = 1200  # tool output returned per step
HISTORY_ACC = 2500           # accumulated history context
OUT_PER_STEP = 500           # model output per step
input_tokens  = TASK_STEPS * (SYSTEM_PROMPT + TOOL_OUTPUT_PER_STEP + HISTORY_ACC)
output_tokens = TASK_STEPS * OUT_PER_STEP
# Typical industry magnitude: input is roughly 5-20x output; cost is dominated by input tokens
print(input_tokens, output_tokens)
# ~54000 input / 4000 output -- the hidden cost of one "simple task" is far above intuition

This example illustrates a counterintuitive fact: the cost of a single agent task is typically ten to several dozen times that of a single-turn Q&A on the same question. Context accumulates, tool returns are verbose, and failed retries double the bill.

6.2 What cost rigidity means

• Incompressibility. Inference cost is determined by token volume and model price. Vendors can optimize engineering (cache reuse, context trimming, routing to smaller models), but the gains are limited — typically a few multiples, and never down to zero.
• Very high marginal cost. Every additional active user creates real inference spend. This is economically different from traditional SaaS, where replication has zero marginal cost.
• The toxicity of a free square. The mechanism by which chatbot products get “killed by free-riding” is not that there are too few users. It is that the free-user share is too high, which turns inference cost into a fixed expense while the revenue side receives nothing corresponding.

6.3 Sensitivity analysis: how much revenue is needed to avoid looking like a rounding error

# Unit economics sensitivity analysis (all values are assumptions; method demonstration only)
MAU            = 20_000_000
FREE_RATIO     = 0.85      # share of free users
TASKS_PER_FREE = 6         # average monthly tasks per free user
TASKS_PER_PAID = 60        # average monthly tasks per paid user
ARPU_PAID      = 15        # average monthly revenue per paid user (USD)
COST_PER_TASK  = 0.02      # inference cost per task (USD)
paid  = MAU * (1 - FREE_RATIO)
free  = MAU * FREE_RATIO
tasks = free * TASKS_PER_FREE + paid * TASKS_PER_PAID
revenue = paid * ARPU_PAID
cost    = tasks * COST_PER_TASK
print(f"Paid users {paid:,.0f}, monthly revenue ${revenue:,.0f}, "
      f"monthly inference cost ${cost:,.0f}, gross margin {1 - cost/revenue:.0%}")
# ~$45M monthly revenue, ~$3.84M monthly inference cost

Raising COST_PER_TASK to 0.06 (a common magnitude for long tasks, multi-turn tool calls, and complex context):

for c in (0.02, 0.04, 0.06, 0.10):
    cost = tasks * c
    print(f"${c:.2f}/task -> monthly cost ${cost:,.0f}, "
          f"gross margin {1 - cost/revenue:.0%}")

The conclusion is straightforward: when per-task cost triples, gross margin drops from roughly 92% to roughly 75%; if the free-user share rises to 95% and paid conversion is only 1%, margin gets compressed further. In other words, the larger the MAU number, the greater the financial pressure from cost rigidity, while revenue-side elasticity is far smaller than cost-side elasticity. This is the mathematical root of the “high activity, low revenue” structure.

To make this kind of product’s revenue stop looking like a rounding error, total annual revenue needs to reach a magnitude where fixed costs can be amortized. Roughly reverse-engineering from the structure above, annualized total revenue needs to land in the $20 billion to $50 billion range before it can be compared against the single-digit-billions SaaS reference. That range looks exaggerated, but it is the joint result of cost rigidity and paid-capacity constraints, not a wish.

7. Scorecard: five questions to grade any agent product

Question	Signals to observe	Danger signals
Is the activity real?	Active definition, deduplication granularity, DAU/MAU, next-month retention	Only MAU is quoted, never the definition; retention falls off a cliff
Is there exclusive technology?	Depth of private data connectivity, compliance capability, unit cost	Features can be replicated quickly from “model + open-source skills”
Where is the paywall?	Billing dimension (quota / seat / outcome)	Quota-based billing only; L2 and L3 both lose
Is unit economics gross-margin positive?	Cost per task, free-user share, gross margin	Free share above 90% combined with high per-task cost
Is new workload created?	Whether tasks that “would never have happened” are now happening	Only substitutes existing processes; budget does not grow

8. Three engineering recommendations for practitioners and builders

For data and product teams. Write the active event definition into the metric dictionary first, and disclose it externally. Metric credibility comes from being auditable, not from the size of the number. Also wire inference cost into dashboards as a first-class metric, alongside activity and retention, so growth metrics and cost metrics do not drift apart.

For engineering teams. Make cost optimization a routine engineering item. Context compression, prompt caching, small-model routing, and tool-output trimming can meaningfully improve gross margin without changing the experience. Also set budget guardrails for long tasks (per-task token caps, timeout interruption) so that runaway loops cannot blow through the cost envelope.

For enterprises and buyers. When evaluating these products, shift the question from “how strong are the features” to “how much effective workload that would not otherwise have happened does it create.” The first determines the experience; the second determines the budget. With only the first, the purchase becomes a cost line item. With both, a long-term contract is worth signing.

9. Conclusion

Back to the original question: a certain AI workbench passed 20 million MAU. How reliable is that number?

The technical answer: without cross-validation across active event definition, retention curves, and resource bills, authenticity cannot be determined. And even if it is real, it only proves distribution capability, not product competitiveness.

The business answer: usage habits are not a moat; homogeneous model supply pushes feature exclusivity toward zero; and agent cost rigidity makes a free square expensive. The ceiling on paid capacity is set by incremental effective workload at the buyer, not by MAU.

The only stable method is to keep “the number, the moat, and unit economics” as three separate calculations, then look at them together.