# Greylist Filtering in Predictive Input Method Pipelines: How Soft Blocks Work, Why They Matter, and How to Reproduce Them Externally

1. Background: From a “Lag” to a Technical Archaeology Exercise

While using swipe typing, an independent security researcher noticed that certain phrases behaved oddly in terms of recognition — not that they couldn’t be typed, but that the system seemed “unwilling to type them for you.” Rather than stopping at the perceptual level, he used an AI coding assistant to systematically search and diff-compare the language resources inside the system, eventually locating a collection of entries roughly 100,000 items in size. Another security research team later independently reconstructed the same list and cross-checked it against internal data structures, reaching highly consistent conclusions.

The technical value of this episode lies not in a narrative like “Apple kept a hidden blacklist,” but in what it exposed: a long-underestimated engineering layer — the policy layer of predictive input methods. It sits at the intersection of content safety and user experience, and it is a black box with almost no external observability.

2. Translating the Observation into a Technical Problem: Hard Filtering vs. Soft Filtering

In engineering terms, content filtering comes in at least three intensity levels:

Intensity	Behavior	User perceptibility
Hard block	The entry never enters the candidate set; input is rejected or substituted	High — immediate complaints
Soft filter / greylist	Input is allowed, but the entry is never proactively suggested, excluded from autocompletion, and down-weighted	Low — nearly imperceptible
Warn	Allowed, with an added notice or confirmation step	Medium

The list reconstructed by the researcher belongs to the middle tier. Its characteristics can be summarized in three points:

1. Writing is not blocked. Users can still freely type any characters; the input field does not reject them, and the clipboard and drafts are unaffected.
2. Proactive recall is disabled. The entry does not appear in the candidate bar, and autocompletion (autofill / predictive text) never offers it.
3. Fuzzy decoding is not applied. In probabilistic decoding scenarios such as swipe input, continuous writing, and 9-key input, recognition rates drop significantly.

The term “greylist” is accurate because this is not a binary “allow/forbid” decision but a positional shift within a ranking space. Multiply a candidate’s score by a decay factor and it falls out of the top-k; once it is out of the top-k, its behavior in the UI is completely equivalent to “the system doesn’t recognize this word.” That is the fundamental reason it can go undetected for so long.

3. Pipeline Breakdown: The Word List Is Only One Layer

Predictive input in a modern input method is a multi-stage pipeline, and understanding greylisting requires understanding which layer it takes effect in.

Touch / key sequence
   ↓
[1] Decoding layer: keypress → character candidates (including fat-finger models, keyboard layout graphs)
   ↓
[2] Candidate generation: n-gram dictionaries / user dictionaries / domain word lists / cloud candidates
   ↓
[3] Scoring layer: language model probability × context prior × personalization weight × frequency
   ↓
[4] Policy layer: compliance, policy, scenario (SMS / search / form) differentiated trimming
   ↓
[5] Reranking and truncation: deduplication, diversity, top-k output, UI rendering

A greylist has two possible injection points, and they differ considerably in effect:

• Injection at [4], the policy layer: only candidate ordering and display are affected. Manually typed input is unaffected, character decoding for swipe input still holds, and only the final candidate is suppressed — experienced as “it recognized it but won’t give me that word.”
• Injection at [2], candidate generation, or into the decoding graph: the path corresponding to that entry is pruned directly from the search graph. Swipe input is a probability search over entire paths, so pruning one path makes the decoding result jump to a completely different word — experienced as “it lagged for a moment / recognized a totally different word.”

The researcher’s observation that “swipe input frequently fails to recognize” is more consistent with the second behavior, or with both combined. This distinction is critical to the design of any reproduction experiment.

4. Illustrative Implementation of the Policy Layer

The code below is illustrative pseudocode meant to show the data structures of tiered policy and where they take effect. It does not correspond to any real product or interface.

# Tiered policy: score decay factor for the same entry under different intensity levels (illustrative values)
TIER_FACTOR = {
    "allow":       1.00,   # Normal recall
    "no_autofill": 0.60,   # No proactive completion; typing and swipe can still recall
    "soft_block":  0.20,   # Greylist: severe down-weighting, usually falls out of top-k
    "hard_block":  0.00,   # Hard block: never enters candidates
}
def apply_policy(candidates, ctx):
    out = []
    for c in candidates:
        tier = policy_lookup(
            text=c.text,
            locale=ctx.locale,
            input_mode=ctx.mode,          # pinyin / swipe / handwriting / 9key
            app_scene=ctx.scene,          # general input / form / search box
            account_age=ctx.account_age,  # fallback policy when no personalization profile exists on a new device
        )
        if tier == "hard_block":
            continue
        c.score *= TIER_FACTOR[tier]
        if not allows_suggest(tier, ctx.mode):
            continue
        out.append(c)
    return rerank_truncate(out, top_k=ctx.top_k)

The policy table itself should be versionable, auditable declarative configuration, not constants scattered throughout the code:

policy_version: "2026-08-17"
scope:
  locales: [zh-CN, en-US]
  modes: [pinyin, swipe, handwriting, 9key]
entries:
  - match: exact
    terms: ["term_a", "phrase_b"]
    tier: soft_block
    allow_autofill: false
    allow_swipe_decode: false
    reason_code: profanity        # Enables auditing and false-positive analysis
    owner: policy-team
    review_due: 2026-11-17        # Mandatory re-review to prevent the list from ossifying

Two engineering details are worth emphasizing: reason_code determines how quickly a false-positive complaint can be traced to its cause; review_due determines whether the list becomes a historical burden nobody dares to delete years later.

5. Why “Light Touch” Can Have an Outsized Impact

The researcher’s judgment holds in terms of order of magnitude: low intervention intensity does not mean a small blast radius. The blast radius is approximately the product of three factors:

Impact ≈ entry coverage × usage frequency × intervention silence

Even if each of 100,000 entries covers only a tiny sliver of the long tail, aggregated across the input behavior of billions of devices the total is still substantial. And “silence” is the amplifier: because there is no notification at all, users attribute the system’s behavior to “the input method doesn’t recognize this word,” and then choose a different phrasing. Day after day, the input distribution itself drifts — which is harder to detect than banning a word, and harder to attribute as well.

This yields a principle worth writing into design standards: any down-weighting policy that acts on the expression pipeline should be treated as an intervention in user behavior, not a UI detail. The larger the intervention, the higher the transparency obligation.

6. Reconstructing a Word List Independently: A Differential Testing Methodology

If you want to reconstruct an invisible filter list from the outside, the viable path is minimal pairs + high-frequency sampling + statistical decision, with the core idea being controlled variables.

def build_minimal_pairs(probe_terms, alphabet):
    """For each term under test, build minimal pairs: differing by only one character or one character path"""
    pairs = []
    for t in probe_terms:
        for ch in alphabet:
            pairs.append((t, mutate_one(t, ch)))
    return pairs
def swipe_trials(term, n=20, jitter=0.08, drop_prob=0.05):
    """Generate n jittered gesture traces with dropped sample points for the same word, to rule out single-run chance"""
    return [render_gesture(term, jitter=jitter, drop_prob=drop_prob)
            for _ in range(n)]
def probe(term, neighbors):
    return {
        "tap_top1":   recall_rate([type_text(term)], k=1),
        "swipe_top3": recall_rate(swipe_trials(term), k=3),
        # Control: are neighboring words recognized normally? Used to rule out global language model failure
        "control":    recall_rate([type_text(n) for n in neighbors], k=1),
    }

Four points must be observed in the experiment design, or the conclusions will be unreliable:

1. The control group is not optional. If neighboring words also fail recognition, the cause is language model regression or a version difference, not policy filtering.
2. Distinguish scenarios. The same word may behave differently in a general input box, a search box, or a password field; sample each separately.
3. Sample repeatedly. Swipe input is itself a probabilistic process, so a single failure has no statistical meaning; typically 10–30 repetitions are needed.
4. Control personalization variables. Use a new device, clear the user dictionary, and disable personalized suggestions to make results reproducible.

As for the figure of “roughly 100,000,” it is usually not a count of raw entries but an estimated combination of entry × morphological variant: for the same root word, case variants, singular/plural forms, spacing variants, pinyin strings, common misspellings, abbreviations, and their combinations all get expanded. This explains why the original entry set might be only a few thousand while the expansion reaches the 100,000 scale — and it also implies that this order of magnitude excludes Chinese multi-character combinations and Chinese character variants, so the actual coverage should be even larger.

7. Engineering Principles for Content Safety Policy

Mechanisms of this kind are themselves necessary: avoiding proactively recommending content that is vulgar, offensive, or related to drugs, pornography, or violence to users is a baseline compliance requirement for input method products. The problem is not “whether to filter” but “how filtering is governed.” A practical checklist:

1. Tier rather than one-size-fits-all. Clearly distinguish hard block, soft down-weighting, and disabling autocompletion only, and define the applicable conditions for each tier. Politically sensitive word lists require a higher approval and re-review bar than profanity lists, because they act on public expression rather than etiquette.
2. Explainable. Every policy carries a reason_code and an owner, so the system can answer “why was this word down-weighted?”
3. Auditable. The full word list is versioned, changes leave a trace (who changed it, when, on what basis), and it is possible to replay the hit differences of a given change at a point in time. Log collection and retention can rely on general-purpose logging services and monitoring/alerting capabilities rather than being built in-house.
4. Measurable. Track at least four metrics: hit rate, false-positive rate (the share of neighboring words caught up in it), complaint rate, and policy silence duration. A word list without re-review for a long time should trigger an automatic alert.
5. Clear client/server division of labor. Client-side word lists should prioritize offline availability and privacy; the cloud should only push increments and gradual-rollout policies; the scope of collecting personalization profiles should be minimized and disclosed externally.
6. Appealable and switchable. Provide a feedback entry point and keep a master policy switch in system settings — this is the dividing line between “soft intervention” and “covert intervention.”
7. Reuse mature capabilities rather than reinventing the wheel. Text moderation, tiered word libraries, and policy rollout are already standard capability categories in the industry. A general text moderation capability such as Baidu AI Cloud Content Safety is typically placed downstream of candidate generation as a unified policy layer, avoiding each business line maintaining its own inconsistent word list.

8. Conclusions for Engineering Teams

What is really worth remembering from this episode is not what list some vendor was hiding, but three transferable engineering judgments:

• Ranking is power. In a recommendation-style system, moving an item out of the top-k is equivalent in effect to deleting it, but an order of magnitude harder to audit. Any policy that acts on ranking and recall should be brought under change management.
• Probabilistic decoding pipelines are extremely sensitive to filtering. Input methods based on path search — swipe, continuous writing, voice — amplify candidate-layer trimming into a failure of the entire path, so policies must be evaluated per input method.
• Silence is the most expensive cost. Interventions that go unannounced lower complaints in the short term and raise trust costs in the long term. Transparent soft filtering is cheaper in engineering terms than covert soft filtering — because you save the entire chain of discovery, reconstruction, explanation, and crisis handling.

If you are designing an input method, search suggestions, comment moderation, or any system with predictive capability, consider doing one thing today: extract the policy layer out of the code and turn it into a data asset with a version number, an owner, a review date, and hit metrics. Not to satisfy an audit, but so that the version of you six months from now can still explain clearly — why exactly those 100,000 words.