Multi-Agent Continuous Refactoring: Failure Risks and Governance After Three Weeks

Letting multiple AI coding agents refactor a codebase continuously sounds like handing repetitive engineering work to tireless assistants. But without acceptance gates, environment isolation, and a human review cadence, agents can easily produce results that run yet are wrong: the architecture drifts from the target, tests connect to production, costs inflate quickly, and so-called skill modules are only placeholder implementations.

A practitioner with a development and AI research background once had two coding agents work in turns for about three weeks, with almost no careful code review. The outcome was an architecture that was not the agent runtime they wanted, unit tests connected to a production inference endpoint, and roughly 30 runs consuming nearly five times the token budget. The project had to be rebuilt from scratch. The value of this case is not in comparing tools. It is in exposing the common engineering risks and governance methods of multi-agent continuous refactoring.

1. The Efficiency Illusion: Why Continuous Refactoring Goes Off the Rails

The core problem with multi-agent continuous refactoring is that agents are good at local completion and rapid trial and error, but they do not automatically align with long-term architectural intent. Once a human hands a specification to an agent, the agent tends to choose the implementation path that most easily passes the current tests if there are no executable acceptance criteria. It can write modules, interfaces, and tests that look complete, but the code may only resemble the target: the directory structure looks like an agent runtime while the actual scheduling logic remains hard-coded; a skill registry appears to exist but has no contract validation or version management; unit tests pass, but default configuration values or shared environment variables cause them to call real production APIs.

Review lag is even more insidious. Not looking at the code for three consecutive weeks means giving up the correction window for architectural decisions. Agents generate a large number of changes every day. Without review by change set, architecture guard tests, and cost gates, problems compound. By the time a human actually reads the code, the rework cost is far higher than it would have been with small, frequent reviews.

2. Key Risks and Governance

2.1 Architecture Drift: A Specification Is Not an Executable Constraint

A specification usually describes what is wanted, but agents need to know what must not be done and how to verify that something was done correctly. To govern architecture drift, intent specifications need to be broken down into architecture specifications, interface contracts, and acceptance cases. Architecture specifications should define module boundaries, dependency directions, and allowed cross-layer calls. Interface contracts should define inputs and outputs, error codes, timeout and retry policies. Acceptance cases should cover critical paths and exception paths. Architecture tests can also be added, for example to forbid business modules from directly depending on production clients and to forbid test code from importing production credential modules.

def test_module_boundaries():
    assert not imports("business_logic", "infra.production_client")
    assert not imports("tests", "infra.production_credentials")

At the same time, each important change should produce an architecture decision record explaining why a module was added, why dependencies were adjusted, and what alternatives were considered. An agent can draft it, but a human must confirm it. Continuous refactoring without decision records makes it hard to answer why the code ended up the way it did.

2.2 Tests Hitting Production: Environment Isolation Must Default to Deny

Tests connecting to production are often not an agent deliberately crossing a boundary. They usually result from default configuration values, shared environment variables, unisolated network egress, and missing test doubles. The governance principle is default deny: when a test process starts, check the environment identifier and fail immediately if production configuration or production credentials are detected; production credentials must not enter development and test environments; network egress should use an allowlist, and the test environment should only access mock services or sandbox dependencies; CI should add static scanning to prohibit test code from referencing production endpoints.

def load_config(env: str):
    if env == "production" and os.getenv("ALLOW_PRODUCTION_TEST") != "false":
        raise RuntimeError("Test processes are not allowed to connect to production")
    return build_config(env)

In addition, unit tests should use contract-consistent test doubles instead of directly calling real inference endpoints. For external dependencies, recording and replay or local mock services can simulate timeouts, rate limiting, and exception responses. Only by making “tests must not touch production” a runtime-enforced constraint rather than a documentation convention can similar incidents be avoided.

2.3 Skill Definitions That Are Not Real: Registries Need Contract Tests

Fake skill modules usually look like this: the registry contains only metadata placeholders, and skill calls have no version, permissions, timeout, or observability; the agent claims to have implemented skill discovery, but it is actually just prompt concatenation or hard-coded branches. The governance approach is to treat skills as formal interfaces: each skill has a unique identifier, version, input/output contract, permission scope, failure fallback strategy, and call-chain tracing. The registry validates metadata completeness at startup, CI runs contract tests, and callers use contracts rather than concrete implementations.

task:
  goal: implement_agent_runtime
  allowed_paths:
    - src/runtime
    - src/registry
  forbidden_deps:
    - infra.production_client
  acceptance:
    - architecture_boundary
    - skill_contract_tests
    - unit_tests_use_mocks
    - production_network_blocked
  budget:
    max_tokens: 200000
    max_rounds: 20

Whether a skill is real cannot rely on the agent’s own claim. It must be proven by contract tests, integration tests, and call-chain tracing together.

2.4 Token Budget Blowups: Cost Needs Gates and Observability

Consuming nearly five times the token budget usually comes from repeatedly injecting full context, exploration without stop conditions, multiple agents overwriting each other, and missing round limits. Cost governance should be layered: set token budgets and maximum rounds at the task level; record per-round input and output summaries at the orchestration layer to avoid repeatedly feeding full history; use incremental context and stage summaries for long tasks; stop automatically when several consecutive rounds produce no effective code changes; and send cost metrics to monitoring and alerts, freezing the task and routing it to a human when thresholds are exceeded.

Cost gates are not about limiting efficiency. They keep agent trial and error within controllable boundaries. Continuous refactoring without budget constraints easily turns exploration into burning money.

2.5 Two Agents Taking Turns: Without a Single Source of Truth, They Overwrite Each Other

When two agents take turns, the biggest risk is the lack of a single source of truth and clear role separation. One agent may refactor an interface while the other adds tests against the old interface; one chooses to add an abstraction while the other inlines the implementation. The governance approach is to define orchestration roles explicitly: a planning agent breaks down tasks and updates decision records, an implementation agent modifies only allowed paths, and a review agent checks against the acceptance checklist. A shared task board records the current goal, change set, open questions, and acceptance status. Each change set must be submitted in small steps, with feature flags and branch isolation to avoid broad overwrites.

3. A Practical Workflow

Use a closed loop of layered specifications, small changes, automated gates, and human review. Layered specifications include intent specifications, architecture specifications, interface contracts, and acceptance cases. Small changes require each task to modify only one module or one contract. Automated gates include architecture tests, contract tests, environment isolation checks, cost budget checks, and static scanning. Human review happens daily or per change set, focusing on architecture decisions, dependency directions, test doubles, and production credential isolation. Going three weeks without looking at code is an anti-pattern. The right cadence is to review change summaries daily, review key diffs every two days, and hold an architecture retrospective weekly.

4. Review Checklist

1. Do the specifications include executable acceptance cases and prohibited items?
2. Does the test environment deny production configuration and production credentials by default?
3. Do skill modules have contract tests, versions, and call-chain tracing?
4. Are token budgets, round limits, and stop conditions configured?
5. Do multiple agents have a single source of truth and clear role separation?
6. Are changes reviewed by change set, with architecture decisions recorded?
7. Are architecture tests used to guard module boundaries and dependency directions?

Multi-agent continuous refactoring can significantly improve development efficiency, but it is not a turnkey solution. Only when architecture constraints, environment isolation, contract tests, cost gates, and human review are built into the engineering system can agents continuously produce maintainable code within controllable boundaries. Otherwise, three weeks of “automated refactoring” may simply package technical debt as progress—and still end in a rebuild from scratch.