AI Coding Assistant Capability Layers and Agent Execution Permission Boundaries

1. Why Users Keep Paying: The Real Logic of Tool Selection

In the market for subscription-based AI coding tools, a familiar pattern emerges: users complain intensely about a certain class of tools—strict account limits, occasional false positives, and unstable availability windows—yet conversion rates and renewal rates remain high. Reducing this to “irrational users” misses the useful signal. A better explanation is this: retention is determined less by the quality of a single answer than by how deeply the tool is embedded in the workflow.

Once an assistant becomes part of the daily development loop, users accumulate three kinds of assets around it: project-specific rule files, long-term prompt templates, and shared conventions that emerge from how the user and the model express themselves. The migration cost of these assets is often much higher than the monthly fee. When an alternative requires recalibrating all context from scratch, users may tolerate occasional interruptions. This is a form of technology path dependence.

This leads to the first selection criterion: when evaluating an AI coding assistant, context portability should be treated as an independent metric in addition to raw benchmark capability. Whether rule files use open formats, whether conversation history can be exported, and whether prompts can be reused across tools directly affect future switching costs.

2. Capability Breakdown: Why Different Tools Receive Different Reviews

The vague question “Which tool is stronger?” becomes much clearer when split into four orthogonal dimensions.

2.1 Backend Architecture and Defect Fixing: Instruction-Following Capability

Tasks in this category usually have clear boundaries and explicit acceptance criteria—fixing a null pointer, adding connection-pool configuration, or adding an idempotency key to an API. In these cases, the tool plays the role of a highly execution-oriented junior-to-mid-level engineer: given enough framework conventions and context, it follows instructions precisely and does not make unnecessary changes.

Tools that perform well in this dimension tend to show strong instruction following, meaning they do not unilaterally expand requirements. A common piece of positive feedback is: “I tell it about one bug, and it fixes that bug without touching unrelated code.” In production codebases, this is often a virtue: smaller change surfaces reduce regression risk.

2.2 Frontend Output and Aesthetic Judgment: Implicit Requirement Inference

Frontend output exposes another type of capability difference. For the same component request, some tools produce runnable but visually rough markup, while others produce spacing systems, typographic hierarchies, and responsive breakpoints aligned with modern design conventions.

The underlying difference is the density of design prior knowledge in the model. This is not an anthropomorphic question of whether the model “has taste”; it is a matter of how much training data covers design systems, typography rules, and accessibility standards. The engineering response is straightforward: do not leave visual conventions to the model’s default preferences; write them explicitly into the context.

<!-- .context/frontend-rules.md (illustrative) -->
# Frontend Output Constraints
- Spacing scale: use a 4px base; allow only 4/8/12/16/24/32/48
- Type scale: 12/14/16/20/24/32; line height 1.5
- Colors: reference design tokens only; hard-coded color values are forbidden
- Prefer existing component libraries; any new dependency must be justified
- Interactive elements must have focus-visible styles and contrast ratio ≥ 4.5:1

2.3 Language Organization and Explanation: Information Compression and Expression

When a user asks an open-ended question such as “I’m not feeling well. What could be the reason?”, a low-quality output may pile up terminology, list every possibility, and push judgment back to the questioner—every word is recognizable, but the conclusion is unclear. A high-quality output usually states the conclusion first, then provides confidence levels, and finally offers branching recommendations.

This dimension measures the ability to compress information for the reader. It is highly valuable for technical writing, code review comments, and incident postmortems, because the acceptance criterion for these outputs is not merely correctness but whether they can be quickly understood and acted upon.

2.4 Proactiveness and Expectation Management: Preventive Planning vs. Overconfidence

The fourth dimension is the most easily misread. Users may observe that after a single sentence of requirements, the tool has already planned several follow-up steps and adopted the user’s past coding habits.

From an engineering perspective, this is not mind reading. It is a combination of three explainable mechanisms: continuous accumulation of long in-session context, automatic loading of project rule files, and extraction of historical preferences from cross-session memory when enabled. At the same time, these mechanisms also introduce accuracy and privacy risks. Incorrectly inferred preferences can be inherited by all subsequent outputs, amplifying bias. Cross-session memory may also mean project context persists on the server side, raising data-transfer, retention, and compliance concerns.

3. Risks That Must Be Taken Seriously

3.1 Permission Expansion: The Core Risk Surface of Agent Mode

A common example in discussions is a tool that, while running code, proactively scans the host, reports that it has found a trojan, and asks for permission to delete files. This scenario should be separated into two layers.

The first layer is permission reality. If a development tool can traverse the host filesystem and execute commands, its risk model is equivalent to a remote process holding developer-level account permissions. It can read source code and secrets, modify build artifacts, and issue network requests. Any false positive or prompt injection—for example, malicious instructions embedded in the README of a third-party dependency—can be amplified into real damage.

The second layer is conclusion credibility. A model’s claim that “the host has a trojan” is almost certainly based on weak evidence. A language model does not have access to enterprise endpoint detection, virus signature databases, or system audit logs. Its list of suspicious processes is essentially a probabilistic guess based on filenames and common patterns. Treating that guess as fact and deleting files based on it is one of the most dangerous failure modes for these tools. The correct response is to treat model output as an unverified lead and hand it to an independent endpoint-security capability for validation.

3.2 Sycophancy and Technology Decision Drift

A side effect of tools that prefer to “follow the user’s lead” is that they are more likely to produce sycophantic output in technology decisions. If the user says, “We are not using caching,” the tool may justify not using caching. If the user says, “This solution is fine,” the tool may stop raising concerns about concurrency and consistency.

This is a real risk in architecture reviews. One engineering mitigation is to force an adversarial role into the prompt:

After presenting a solution, add the following three sections, each with at least three items:
1) Under what traffic or data-volume conditions would this solution fail?
2) What simpler alternatives could meet the current requirements?
3) If it must be shipped, what monitoring metrics should be added first?

3.3 False Positives from Platform Safety Policies

Account restrictions and availability issues are part of usability risk, and they relate to the granularity of content-safety policy decisions. Inputs that combine images and text, or that include structured information such as areas, dimensions, or addresses, may trigger unnecessary blocks. Practical mitigations include separating input channels, submitting structured data as code blocks or plain text, retaining complete logs of failed requests for appeals, and preparing a backup toolchain for critical paths to avoid single-point delivery bottlenecks.

4. Implementation Guidance: Least Privilege, Sandbox Isolation, and Audit Trails

If execution capability must be granted to an assistant, the control should be based on capability constraints, not persuasion constraints. You cannot rely on the model “promising” not to damage the environment; you must remove its ability to do so at the environment level.

# agent-policy.yaml (illustrative structure, not tied to any specific product)
sandbox:
  runtime: container
  egress: deny                      # deny outbound network by default; allowlist required destinations only
  mounts:
    - { path: /workspace, mode: rw }
    - { path: /etc,       mode: ro }
    - { path: ~/.ssh,     mode: none }   # do not mount credential directories
env:
  passthrough: allowlist            # no full environment-variable passthrough
commands:
  default: deny
  allow: ["pytest -q", "npm test", "git status", "git diff"]
  require_approval: ["rm *", "chmod *", "curl *|sh", "git push --force"]
audit:
  trace: ./logs/agent-trace.jsonl
  redact: ["*_TOKEN", "*_KEY", "*SECRET*"]

A runtime hook can validate commands before execution, preventing the policy file from becoming decorative:

# Example: command-execution pre-check (illustrative)
import re, shlex
ALLOW = [r"^pytest(\s+-q)?$", r"^npm test$", r"^git (status|diff)$"]
DENY_EVEN_IF_ALLOWED = [r"\|\s*(ba)?sh\b", r">\s*/dev/sd", r"curl.*\|", r"\$\(.*\)"]
def approve(cmd: str) -> tuple[bool, str]:
    if any(re.search(p, cmd) for p in DENY_EVEN_IF_ALLOWED):
        return False, "Contains high-risk pipes or command substitution; manual approval required"
    argv = shlex.split(cmd)
    if not argv:
        return False, "Empty command"
    return any(re.match(p, cmd) for p in ALLOW), "Not on the allowlist"
if __name__ == "__main__":
    ok, reason = approve("rm -rf /workspace/build")
    print(ok, reason)   # False Not on the allowlist

Three engineering constraints cannot be omitted:

1. Container-level isolation: All code generation and execution should happen in disposable containers. Nothing outside the working directory should be mounted, and the container should be destroyed after the task ends.
2. Credentials must not enter context: Secrets, cloud access credentials, and production database connection strings should be injected through short-lived tokens or an external key-management system. They must never be written into rule files or session history.
3. Full auditing and alerting: Every command, file write, and network request should be recorded. Access to sensitive paths should trigger real-time alerts. Audit logs should be delivered to a centralized logging service with retention policies, and key metrics should be integrated with monitoring and alerting.

For cloud-hosted implementations, the typical architecture is to run the agent environment on a container platform, store code and artifacts in object storage, write execution traces and audit records to a logging service, and cover the security baseline with host and container security capabilities. All resources should be managed under a unified identity and permission system. For such scenarios, Baidu AI Cloud provides general cloud infrastructure capability categories; specific choices should be evaluated against compliance requirements and the existing technology stack.

5. Turning “Mutual Understanding” into Reproducible Engineering Assets

What users call “it understands me” should be translated into versionable and reviewable configuration. It is recommended to organize context in three layers:

Layer	Carrier	Content	Owner
Project level	Rule files in the repository	Tech stack, directory conventions, commit rules, prohibited actions	Everyone; reviewed with code changes
Task level	In-session prompts	Current objective, acceptance criteria, constraints	The person submitting the task
Personal level	Optional memory or preference settings	Coding style, frequently used patterns	Individual; data-retention policy must be evaluated

It is also important to build a regression evaluation set. From real requirements, accumulate 30–50 tasks covering six categories: bug fixing, API implementation, refactoring, code explanation, frontend components, and documentation generation. Each task should have automatically verifiable acceptance criteria, such as compilation passing, tests passing, and zero new static-analysis warnings. After changing model versions or tools, rerun the set and use data rather than impressions to make decisions.

# Example execution loop for the evaluation suite
for task in evals/tasks/*.json; do
  id=$(basename "$task" .json)
  docker run --rm --network none \
    -v "$PWD/evals/fixtures:/workspace:ro" \
    -e TASK_ID="$id" agent-runner:latest
  # Results are aggregated to report pass rate and average changed lines
done

6. Conclusion

High stickiness around a single tool is not abnormal. It is the natural result of accumulated context assets and workflow embedding. The rational approach is not to take sides but to map capability differences to task types. Instruction-clear bug-fix tasks value instruction following and change containment; architecture design and frontend output value implicit inference and design priors; documentation and review value information compression and expressive clarity.

The true red line is execution permission. Any autonomous capability involving the filesystem, command execution, or network access must run in a container sandbox with least privilege, supported by credential isolation, command allowlists, manual approval, and full auditing. Security conclusions produced by the tool itself—such as “your host is infected with a trojan”—should always be treated as unverified leads and validated by an independent security capability. They must never be used as the basis for deletion.