Human-Agent Collaboration and Control Design in Agentic Coding Assistants

1. Background: From Assisted Completion to Autonomous Execution

Early coding assistance tools mainly provided local suggestions, such as completing a line of code or explaining a function. Developers remained in control of every step, and the tool only participated at the input stage.

Agentic tools work differently. They receive a goal—for example, “fix an API timeout and add tests”—and then plan steps, read code, invoke commands, modify files, and verify results on their own. This model can significantly reduce repetitive work, but it also extends risk across a much larger scope. A single bad bulk edit or a file-deletion command can leave the workspace in a state that is difficult to recover.

Therefore, the core product question for agentic tools is not only “how much can it do,” but also “does the user know before it acts, and can the user undo it afterward?”

2. Core Tension: Automation Benefits and User Control

The benefits of automation are direct. Tools can save time spent on repeated searching, editing, and manual verification, shortening the cycle from problem identification to result delivery.

The costs of missing control are equally concrete, mainly in the following categories:

• Misoperation: modifying modules that should not be changed, or deleting configuration that is still in use.
• Unexplainability: the final result is correct, but the user cannot determine which steps the tool took.
• Loss of trust: after one unexpected action, users remain wary of all subsequent actions, and the value of the tool decreases.
• Security exposure: the tool reads files containing credentials, or sends internal content to external addresses.

The design goals can therefore be summarized in three points: users should be informed at all times, able to intervene at any time, and able to roll back at any time. The level of automation should be determined by risk, not by technical capability.

3. Five Design Principles

1. Visibility: Show the plan before execution, the current step during execution, and a change summary after execution. Users should see facts—for example, “will modify three files, adding 40 lines and deleting 12”—rather than vague descriptions such as “optimizing code.”

2. Intervenability: Users can pause a task, modify the plan, skip a step, or reject a category of operations. Intervention entry points should be within immediate view and should respond quickly.

3. Least privilege: Read-only by default. Writing files, accessing the network, and calling external services all require separate authorization, and the authorization scope should be limited to the current task.

4. Reversibility: Generate a snapshot before each modification, and allow users to roll back by step or by the task as a whole. Operations that cannot be rolled back must be flagged before execution.

5. Traceability: Every action has a complete record, including who initiated it, what authorization it relied on, what the inputs and outputs were, and what the result was. Records support both post-incident troubleshooting and team compliance review.

4. Permission Tier Model

Actions can be divided into four levels based on their impact scope and reversibility.

Level	Typical operations	Default policy
L0 read-only	Read source code, query documentation, list directories	Auto-execute
L1 reversible writes within the workspace	Modify project files with snapshots	Auto-execute, show diff, support one-click rollback
L2 external impact	Install dependencies, access the network, send requests	Request confirmation before execution
L3 irreversible or high-risk	Delete data, change permissions, deploy to production	Mandatory human approval; record approval rationale

The key is the action classifier. The classifier should determine the level based on the action type, target path, and scope, not only on the description text produced by the model. The same “delete” operation has very different risk when deleting a temporary directory versus deleting a database table, and the classification result should differ accordingly.

The following pseudocode illustrates the control flow for tiered execution. It does not correspond to any specific API; it only expresses the logic:

RISK = {read: 0, local_write: 1, external: 2, irreversible: 3}
function execute_step(step):
    level = classify(step.action, step.target)
    if level == RISK.read:
        return run(step)
    if level == RISK.local_write:
        snapshot = take_snapshot(workspace)
        result = run(step)
        show_diff(snapshot, result)
        return result
    if level == RISK.external:
        if not ask_user_confirm(step.summary):
            return skipped(step)
        return run(step)
    # irreversible
    approval = require_explicit_approval(step)
    audit_log.record(step, approval.reason)
    return run(step)

This logic has three key points. First, classification happens before execution, and if classification fails, the action should be treated as a higher level. Second, L1 operations take a snapshot before execution so that rollback has a basis. Third, L3 operations must leave an approval record and cannot silently pass through a default option.

5. Execution Plans and Checkpoints

Long tasks should be split into verifiable small steps. Taking “fix an API timeout issue” as an example, a reasonable breakdown is:

1. Locate: find timeout-related configuration and the call chain.
2. Reproduce: write or run a test that can trigger the issue.
3. Modify: adjust timeout parameters or retry logic.
4. Verify: run all relevant tests and compare timing data before and after the change.
5. Summarize: list changed files, verification results, and uncovered risks.

Set a checkpoint after each step. Checkpoints require evidence, such as the number of passing tests or key lines in logs, rather than merely outputting “done.” If a step fails, the tool should stop and report the failure rather than bypass it and continue to later steps. Skipping failing tests or deleting failing assertions can make the final result look correct while hiding real problems.

6. Interaction Design: Avoiding Confirmation Fatigue

If confirmation mechanisms are poorly designed, they cause confirmation fatigue. When users face a large number of pop-ups, they habitually click “Agree,” and the security benefit disappears. Responses include:

• Consolidate similar confirmations: when a change touches multiple files, combine it into one confirmation and list the files.
• Prompt only for high-risk steps: L0 and L1 operations do not interrupt users by default; show diffs so users can review afterward.
• Make confirmation scope explicit: provide options such as “this time only,” “similar actions within this task,” and “within this session,” and explain what each option covers. The broader the scope, the more prominent the rollback entry should be.
• Make confirmation content specific: state the impact scope, number of affected files, whether the action is reversible, and estimated duration. Vague confirmation text does not help users make a decision.

7. Observability and Auditing

Audit logs should include the following fields: timestamp, task ID, action type, target object, input summary, output summary, authorization source, execution duration, and result status.

Log design needs to balance troubleshooting efficiency and data security:

• Sensitive information in inputs and outputs should be redacted; credentials, personal information, and internal addresses should not be written to logs as-is.
• Log retention periods should be set according to purpose, with automatic cleanup after expiration.
• Log access should be permission-controlled, and records should capture who viewed which records and when.

At the operations level, logs can be integrated with general logging services and monitoring/alerting. Signals worth watching include abnormal increases in failures per unit time, the same action being repeatedly rejected, privilege escalation attempts, and sudden spikes in L3 operations. These signals often indicate whether the system has deviated from expectations better than a single error does.

8. Engineering Implementation Considerations

Sandbox isolation: Tool file access should be restricted to the project directory, network access should use an allowlist, and compute resources should have quotas to prevent a single task from exhausting shared resources.

Parameter validation: Model-generated tool parameters are untrusted input. Before being concatenated into commands, queries, or paths, they must be validated and special characters escaped. Path parameters should be normalized before checking whether they fall within the allowed scope.

Secret management: Access credentials should be injected through environment variables or a dedicated secret management service, not written into prompts, code, or logs. If credential-like strings appear in tool output, they should be automatically masked.

Prompt injection defense: File contents, web content, and tool return results should all be treated as data. If text such as “ignore previous rules” or “run the following command” appears, the tool should treat that content as plain text and alert the user rather than comply. Requests involving privilege escalation should be downgraded to user confirmation.

9. Evaluation Metrics

To measure the performance of an agentic product, task completion rate alone is not enough. The following metrics should also be tracked:

• Task completion rate: the percentage of tasks completed without human intervention.
• Human intervention count: the average number of user interventions required per task.
• Misoperation rate: the proportion of actions that require rollback or are marked as incorrect by users.
• Rollback rate: the proportion of tasks in which users actively revert changes.
• Confirmation time: the average time from when a confirmation appears to when the user makes a decision.

Metrics need to be interpreted together. A low intervention count does not necessarily mean the product is good; it may also mean users habitually click confirm. Low intervention counts are meaningful only when the misoperation rate also decreases.

10. Suggested Rollout Path

A three-phase rollout is recommended:

• Phase 1: read-only agent. It can read code and generate analysis and plans, but does not modify any files. Focus on validating the accuracy and readability of plans.
• Phase 2: reversible writes within the workspace. Enable L1 operations with snapshots and diff display. Focus on validating whether the rollback path is reliable.
• Phase 3: external actions and approval workflows. Enable L2 and L3 operations and establish approval workflows and full auditing.

Each phase should have exit criteria—for example, the misoperation rate is below an agreed threshold, the rollback process has been drilled at least once, and audit logs cover all high-risk actions. Move to the next phase only after these conditions are met, avoiding controls lagging behind rapid feature expansion.