Why Expense Tracking Fails: Recording Is Easy, Attribution Is Hard

1. Why Expense Tracking Fails: Recording Is Easy, Attribution Is Hard

Users resist expense tracking, but the barrier is rarely the lack of an entry point. Three costs get in the way:

• Data entry cost — every transaction requires manually choosing a category.
• Attribution and recall cost — months later, nobody remembers what a given charge was for.
• Interpretation cost — statements are hard to read, and figures reported under different conventions look contradictory.

Once an AI assistant is connected to account data, all three costs collapse at the same time: automatic sync removes entry work, a classification model handles attribution, and natural language answers the interpretation question.

The real engineering difficulty, however, is turning “the model can talk” into “the numbers are right.” That is what forces the system to separate calculation from expression.

2. Architecture: Four Layers

A usable smart expense-tracking capability is usually built from four layers:

1. Access layer — user identity verification and account authorization, obtaining read-only data access credentials.
2. Data layer — transaction pulling, incremental sync, normalization, and idempotent persistence.
3. Analytics layer — categorization and aggregation, period detection, anomaly detection, interest and balance calculations.
4. Interaction layer — deterministic calculation results are handed to a language model for organization and explanation, producing the final answer.

The key design principle: every amount, ratio, and period figure must come from the deterministic calculations in layer 3. The language model is responsible only for explanation and phrasing and is never allowed to “derive” a number. This is the first line of defense against hallucinated financial conclusions.

3. Onboarding: Credentials Never Land in Your System

The mainstream approach is to integrate a third-party financial data aggregation service — commonly referred to as open banking data aggregation or simply account aggregation. The flow is:

• The client creates an authorization session and redirects the user to a page hosted by the aggregation provider.
• The user signs in to their bank on the provider’s side, completes multi-factor authentication, and grants consent there.
• The application receives only an access token representing that consent, and then reads account and transaction data through the provider’s APIs.

This produces three hard constraints that belong in the product design:

• Never collect, store, or transmit bank usernames or passwords. Credentials never pass through the application server.
• Request read-only permissions only. No transfers, payouts, investments, limit changes, or any other write operations.
• Follow the principle of least privilege. Request only the data scopes you need, such as balances and transaction details, and let users revoke authorization at any time.

For token management, access tokens should be encrypted at rest and isolated per tenant, refresh tokens should be stored separately, and logs must never print a token or any fragment of an account number.

4. Data Normalization: The Foundation of Every Statistic

Transaction fields returned by different institutions vary widely: timestamp precision, amount sign convention, currency, raw merchant strings, and status enumerations. All of it must first be collapsed into a unified model:

{
  "txn_id": "hash(account_id, raw_txid)",
  "account_id": "acc_xxx",
  "posted_at": "2026-08-19T10:22:00Z",
  "amount_minor": -2399,
  "currency": "USD",
  "direction": "debit",
  "raw_merchant": "XX*MONTHLY-PRO-VIP",
  "merchant_key": "merchant_streaming_a",
  "category": "subscription",
  "status": "posted",
  "is_recurring_candidate": true
}

Engineering requirements:

• Store amounts as integers in the smallest currency unit. Never use floating point, which introduces accumulation errors.
• Use a stable hash for txn_id to achieve idempotent deduplication, because incremental sync inevitably produces duplicate deliveries and backfilled replays.
• Distinguish three states — authorized, posted, and reversed — and count only posted transactions in reporting.
• Build every downstream statistic on this normalized table, so each analytics task does not have to clean data on its own.

5. Categorization and Merchant Normalization

The accuracy ceiling of categorization is set by merchant normalization, not by model size. A three-tier strategy works well:

1. Rule tier — whitelist matching for known high-frequency merchants (subscriptions, telecom, utilities, transport). Highest priority.
2. Feature tier — normalize raw_merchant (strip noise suffixes, remove transaction reference numbers, unify case, restore common masking), extract keywords, and map them to the category taxonomy.
3. Model tier — for records that cannot be normalized, use text embedding vectors for nearest-neighbor retrieval, vote on a category from historical labeled transactions, and fall back to user confirmation when confidence is below a threshold, writing the result back as a training sample.

Keep the taxonomy shallow: no more than about ten top-level categories, with second-level categories added only as needed. Excessive depth makes statistics unreadable and raises the cost of user corrections. Manual corrections must be persisted at the merchant level, not the individual transaction level, so similar transactions are then classified correctly automatically.

6. Period-over-Period Comparison and Subscription Detection

A single month’s statement reveals nothing. Cross-month comparison is what exposes structural spending. Detecting subscription spending is fundamentally periodicity detection:

def detect_recurring(txns, amount_tol=0.02, period_range=(26, 35)):
    groups = group_by(txns, key=lambda t: t.merchant_key)
    result = []
    for merchant, items in groups.items():
        items = sorted(items, key=lambda t: t.posted_at)
        for a, b in zip(items, items[1:]):
            gap = (b.posted_at - a.posted_at).days
            amount_close = abs(b.amount_minor - a.amount_minor) \
                           <= abs(a.amount_minor) * amount_tol + 100
            if period_range[0] <= gap <= period_range[1] and amount_close:
                result.append({"merchant": merchant,
                               "interval_days": round(gap),
                               "amount_minor": a.amount_minor})
    return result

Keep an amount tolerance, because price increases and tax fluctuations move the figure slightly. Widen the period windows as well: roughly 26–35 days covers monthly billing, 7±1 days covers weekly, and 350–380 days covers annual.

Detection results then feed the calculation of monthly surplus: annualize recurring spending and spread it across months, subtract fixed expenses, and compare the result against average monthly income. That conclusion is far more reliable than looking at any single month’s balance.

7. Anomaly Detection: Three High-Frequency Problems

• Duplicate charges — two or more posted transactions from the same merchant, at the same amount, within the same billing cycle, with an interval shorter than half the minimum period, should be flagged as suspected duplicates, with an evidence chain of merchant, amount, and date.
• Unexpected fees — track fee-type merchants on a separate channel, build a baseline from average amount per transaction and frequency, and raise a prompt when a charge exceeds the baseline or a fee type appears for the first time (cross-border, cash advance, and currency conversion fees are common examples).
• Bill creep (fee drift) — run time-series trend detection over the same subscription or bill type, and use a cumulative-deviation approach to judge whether “several small increases” have become a systematic rise, rather than checking only whether a single charge doubled.

All three detectors should follow rules first, statistics as a supplement: rules are explainable and produce controllable false-positive rates, while statistical methods catch the slow changes rules do not cover. Every alert must point back to the original transactions, and users should be able to mark each one as normal or anomalous in a single click, with that feedback routed directly into threshold tuning.

8. Explaining Financial Concepts and Interest Calculations

Users will ask whether they are spending more than they earn, what their annual percentage rate is, and how much interest they paid over a year. Each question has to be reduced to a calculation:

• Spending more than you earn — total income and total spending within the selected interval, while separating recurring flows from one-off large expenses so that a single home purchase does not distort the conclusion.
• Annual percentage rate (APR) — the industry convention is to convert a per-period rate into an annualized figure, and in some markets related fees are folded in as well. As a result, the same credit line can show different annualized numbers under different conventions. The assistant’s correct behavior is to state which convention it used and point to the contract terms, rather than quoting a number stripped of context.
• Interest over one year — revolving credit card interest is commonly calculated daily on the average daily balance:

def daily_interest(balance_daily, daily_rate):
    """balance_daily: {date: interest-bearing balance}"""
    return sum(bal * daily_rate for bal in balance_daily.values())

In implementation, the daily rate must be derived from the rate disclosed on the user’s actual statement or contract; the assistant must not assume it. If the statement does not disclose it, reply explicitly that the data is insufficient instead of estimating.

9. Privacy, Security, and Compliance Boundaries

• Data minimization — pull only the necessary time window (for example, the last 12–24 months) and do not re-archive older data.
• Encryption in transit and at rest — transport encryption is the default, and stored data is encrypted with keys managed by a dedicated key management service.
• Redaction and minimal context — mask or truncate merchant names and account number suffixes before they enter the model context, so complete statements are never written into the inference path.
• Tenant isolation and revocability — authorization can be revoked at any time, after which synchronization stops and caches are purged.
• Observability — archive raw snapshots in object storage, carry sync jobs on a message queue, record the audit trail in a logging service, and cover sync failure rate and latency anomalies with monitoring and alerting.
• Clear permission boundaries — read data only; do not initiate any fund movement and do not give investment advice.

10. Pre-Launch Checklist

1. Does the authorization flow avoid touching bank credentials end to end, and can it be revoked in one click?
2. Is the transaction model unified, are amounts stored as integers, and is deduplication idempotent?
3. Is merchant normalization layered, and are manual corrections written back?
4. Does subscription detection cover monthly, quarterly, and annual billing as well as amount drift?
5. Can every alert be traced back to the original transaction evidence?
6. Do all figures come from deterministic calculation, with the model limited to explanation?
7. Are the interest calculation convention and the degraded response for missing data both stated explicitly?
8. Do sync, reconciliation, and alerting form a complete monitoring loop?

Get these eight items right, and smart expense tracking moves from “can chat” to “a financial analysis tool you can rely on.”