A Practical Guide to Connecting Coding Agents to CLI and CI

4 hr ago0 views

This guide explains how to plug coding agents into terminal and CI workflows with clear inputs, inspectable outputs, and auditable records. It covers CLI versus CI boundaries, capability layers and contracts, task YAML and command examples, gates and rollback checklists, and acceptance metrics.

The prerequisite for putting an agent into an engineering workflow is that every run has a defined input, inspectable output, and a traceable record. This article covers five parts: the boundary between the terminal and CI, capability layers and interface contracts, task templates and command examples, gates and checklists, and metrics and acceptance. The command name agent-cli is an example; replace it with the tool and flags your team actually uses.

1. Terminal vs. CI boundaries

A GUI is useful for exploring code and reviewing diffs, but it is hard to embed in existing pipelines. A terminal form fits these cases:

Overnight repository scans for similar issues, such as deprecated API calls.
Batch generation of patches or change notes written to files for human review.
Pre-commit static checks and unit tests, with failures emitted as reports.

Do not hand unattended batch agents: production config changes, database migrations, permission or access-policy changes, or secrets handling. Those need step-by-step human confirmation.

Unify the runtime before integration:

Version lock: record the exact agent CLI version and check it with agent-cli --version as the first pipeline step.
Image consistency: runners use a container image pinned by digest, for example image@sha256:..., so the same task does not diverge across machines.
Dependency lock: the repo must commit lockfiles; the agent must not upgrade dependencies on its own.
Auth: tokens are injected from the CI secret store as environment variables, scoped only to the target repo.
Repo mount: source is mounted or checked out read-only; the agent may write only an output directory. Code push is a later independent job with separate credentials.

Exploratory work can run on a developer machine. Batch jobs must first dry-run in read-only mode and only then allow patch writes.

2. Capability layers and interface contracts

Mixed terminology makes tool choice harder. The table below layers agent capabilities and states input, artifacts, permissions, and failure handling for each layer.

Layer	Role	Input constraints	Artifacts	Permission boundary	Failure handling	Audit
Prompt	One-shot human–agent chat	Free text; never paste secrets	Conversation result	No repo write	Human re-asks	Not in the pipeline
Command	One run triggered by a script or a person	Task file with validated parameters	Exit code, report, patch	Only output dir writable	Handle by exit code	One record per run
Skill	Versioned reusable procedure pack	Fixed steps and parameters	Artifacts defined by the skill	Only tools declared by the skill	Stop and report	Skill name and version
Tool call	Runtime actions such as read file or run tests	Tool name and args must be allowlisted	Tool return value	Per-tool authorization	Reject and count overreach	Tool name and arg summary

A command is one execution, a skill is a reusable procedure, and a tool call is a concrete action the agent requests. The agent may call only allowlisted tools and cannot expand its own permissions.

The contract requires a structured result on every run. Example JSON Schema for the result file:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "AgentTaskResult",
  "type": "object",
  "required": ["task_id", "status", "exit_code", "artifacts", "audit"],
  "properties": {
    "task_id": { "type": "string" },
    "status": { "enum": ["success", "gate_failed", "limit_exceeded", "denied", "error", "rolled_back"] },
    "exit_code": { "type": "integer", "minimum": 0, "maximum": 5 },
    "artifacts": { "type": "array", "items": { "type": "string" } },
    "audit": {
      "type": "object",
      "required": ["started_at", "finished_at", "model_version", "operator", "tool_calls"],
      "properties": {
        "started_at": { "type": "string", "format": "date-time" },
        "finished_at": { "type": "string", "format": "date-time" },
        "model_version": { "type": "string" },
        "operator": { "type": "string" },
        "tool_calls": { "type": "integer", "minimum": 0 }
      }
    }
  }
}

Downstream jobs read only status and exit_code; they do not parse natural-language reports.

3. Task templates and command examples

Each task is described by a YAML file:

task_id: fix-deprecated-api-2026-11-001
scope:
  repo_path: /workspace/repo          # read-only mount
  paths: ["src/**"]                   # paths allowed to read and modify
  deny_paths: ["infra/**", "secrets/**", ".github/**", "deploy/**"]
mode: patch_only                      # read_only | patch_only | commit_branch
tools_allowlist: [read_file, search, run_lint, run_unit_tests]
limits:
  max_files_changed: 30
  max_runtime_minutes: 20
gate:
  required_checks: [lint, unit_tests]
  on_fail: stop_and_report            # do not retry on gate failure
output:
  dir: out/
  patch_file: out/patch.diff
  report_file: out/report.json
  audit_file: out/audit.jsonl

Field constraints:

task_id must be globally unique; include a date and sequence for tracing.
deny_paths wins over paths. A change that hits a denied path fails the task.
mode defaults to read_only. Switch to patch_only only after a successful dry run. commit_branch needs separate approval and should not be enabled in the first phase.
max_files_changed and max_runtime_minutes are hard limits; exceeding them stops the run with exit code 3.

Run example. Secrets come only from the environment, never from the task file:

# Secrets injected from the CI secret store; local example uses a placeholder
export AGENT_API_KEY="${CI_SECRET_AGENT_KEY}"
agent-cli --version
agent-cli run \
  --task task.yaml \
  --output-dir out/ \
  --log-level info
echo "exit=$?"

CI gate script skeleton:

#!/usr/bin/env bash
set -euo pipefail
TASK_FILE="${1:?usage: gate.sh <task.yaml>}"
mkdir -p out
set +e
agent-cli run --task "$TASK_FILE" --output-dir out/
code=$?
set -e
case "$code" in
  0) echo "Patch written: out/patch.diff" ;;
  2) echo "Gate failed; see out/report.json"; exit 2 ;;
  3) echo "Resource limit exceeded"; exit 3 ;;
  4) echo "Unauthorized request rejected; check tools_allowlist and deny_paths"; exit 4 ;;
  5) echo "Rolled back; verify the baseline"; exit 5 ;;
  *) echo "Agent run error"; exit 1 ;;
esac

Define exit codes as a team and keep the table and script aligned. Downstream jobs continue or stop based only on the exit code.

4. Gates, least privilege, log redaction, and rollback

Check every item before enabling batch processing. If any item fails, do not open unattended runs.

Least privilege

Agent token covers only the target repo and is read-only.
Patch writes are limited to the output directory; push rights belong to a later independent job.
deny_paths covers infra, secrets, CI config, and deploy directories.
The agent runtime cannot reach production databases or hosts.
Tokens have an expiry and are rotated on a schedule.

Secret injection

Secrets are injected from the CI secret store as environment variables.
Secrets are not written to task files, logs, reports, or image layers.
Logs do not dump full environment-variable values.

Log redaction

Before writing the audit file, replace these with [REDACTED]: Authorization headers, api_key, token, password, cookie, private-key blocks, and internal network addresses. Apply redaction once at the agent output layer, not in ad-hoc scripts.

Each audit line is one tool call with: task_id, timestamp, operator, model_version, tool_name, args_summary, result_code. args_summary keeps argument types and lengths only, not raw content.

Gate policy

lint and unit_tests must both pass; any failure returns exit code 2.
Do not auto-retry more than once after a gate failure.
Failed tasks must emit report.json and keep the patch for human judgment.

Rollback triggers

Stop and roll back immediately if any of the following occur:

A commit is produced after a gate failure.
Files changed exceed max_files_changed.
A file under deny_paths is modified.
Runtime exceeds max_runtime_minutes.
Audit log write fails.

Rollback steps

Reset the workspace to the baseline commit recorded at task start.
List temporary branches the agent created; a human owner confirms before delete. Do not auto-delete.
Re-run unit tests and lint.
Compare the baseline commit hash with the workspace and confirm they match.
Append a rolled_back record to the audit file with the trigger and operator.

If post-rollback verification fails, stop all batch tasks and notify the platform owner.

5. Metrics and acceptance

Use one definition so teams can compare:

Task success rate = tasks that passed the gate and were not rolled back ÷ total tasks. Window: last 30 days, deduped by task_id, excluding infrastructure failures.
Human review rate = patches sent to human review ÷ successfully generated patches.
Rollback rate = tasks that triggered rollback ÷ tasks that produced a patch.
Median time to a reviewable patch, in minutes, from task start to patch, excluding queue time.

Acceptance has two stages:

Read-only dry-run stage

Two consecutive weeks with no overreach events (zero exit code 4).
Sample 100 audit records at random; redacted fields must not contain full secrets, tokens, or internal addresses.
Human-sample gate results and record false positives and false negatives.

Patch-write stage

Open patch writes only after success rate and rollback rate meet the team’s agreed thresholds.
Complete at least one rollback drill and keep the record.
After acceptance, commit_branch still needs separate approval and is not the default.