Building an AI Coding Agent and End-to-End Automation Workflow from Scratch

1. Start with the Core Principle: Automation Depends on What the System Can Observe

Before building an AI coding agent, establish one basic fact: a model does not automatically have access to the outside world. It cannot browse websites, operate devices, or retrieve external information without appropriate tools. The capabilities of an automation system ultimately depend on the data it can access, the tools it can call, and the reliability of its execution feedback.

From an engineering perspective, a complete automation loop usually contains three layers:

Layer	Main capabilities	Common implementation methods	Impact when missing
Information	Search for information, fetch web pages, read documents	Search APIs, web scraping services, knowledge bases	Cannot process real-time information or external materials
Interface	Open pages, click controls, fill out forms, verify screenshots	Browser automation protocols, page-operation services	Cannot operate web systems without public APIs
Device	Install applications, launch programs, simulate interactions, read logs	Device debugging bridges, mobile automation services	Cannot verify results on real devices

When these three layers are combined, the system can complete a full chain:

Retrieve information → create a plan → operate a web page or device → read the result → determine whether it succeeded → revise the approach → provide a conclusion

A model without tools is primarily a conversational consultant. With tools, permissions, and feedback, it can become an execution-oriented assistant.

2. Prepare the Environment Before Installing the AI Coding Agent

AI coding agents commonly run as command-line programs, editor plugins, or local services. Regardless of the form you choose, prepare a clean and reproducible development environment first.

2.1 Basic Environment

The following components are recommended:

1. A mainstream operating system, preferably a desktop or server system with full terminal support.
2. A version control system for saving changes, rolling back incorrect modifications, and reviewing diffs.
3. A runtime environment, such as an interpreter for running scripts or a platform for building front-end projects.
4. A package manager for installing project dependencies.
5. A browser and its automation driver.
6. A mobile device debugging environment, with a physical device or emulator when necessary.
7. A dedicated working directory to prevent the agent from modifying system files or unrelated projects.

After installation, verify that versions and executable paths are correct. Commands vary by operating system and programming language, but the validation approach is the same:

runtime --version
package-manager --version
version-control --version

These commands represent a general verification pattern. Replace the command names with those used by your language runtime and operating system.

2.2 Permissions and Security Boundaries

Do not grant the agent unrestricted system access on its first run. A safer approach is to:

• Start with a test project.
• Keep code, temporary files, and credential directories separate.
• Require human confirmation for high-risk operations such as deletion, overwriting, publishing, and payments.
• Keep secrets, access tokens, and private files out of the project context.
• Review scripts generated by the agent before running them.
• Use version control to record every important change.

The more capable the agent is, the more clearly its boundaries must be defined. Permission management is not a restriction on productivity; it prevents a faulty plan from causing irreversible damage.

3. Connect a Domestic or International Model by Understanding the Request Path

Connecting a model to an AI coding agent means allowing the agent to access the target model through a consistent model-calling protocol. The typical request path is:

Coding agent → model adapter → model service API → text, tool call, or structured result

The model adapter may be a local configuration file, environment variables, a gateway service, or an intermediate layer responsible for authentication and protocol conversion.

3.1 Core Configuration Information

You will usually need:

• Service endpoint;
• Model identifier;
• Access credentials;
• Request timeout;
• Maximum context length;
• Whether streaming output is enabled;
• Compatibility settings for tool calls and structured output.

Field names differ between model services, so do not copy a configuration template without verification. Follow the relevant API documentation and begin with a minimal request to confirm connectivity.

3.2 Three Integration Patterns

Direct Connection

In direct mode, the agent accesses a model service directly. The architecture is simple and the request path is short, making it suitable for individual development and single-model use cases.

Advantages include:

• Fewer configuration steps;
• Easier troubleshooting;
• More predictable latency;
• Easier reproduction of model behavior.

The main limitation is reduced flexibility when switching models. If a service becomes unavailable or a task requires a different model, the configuration must be changed manually.

Routing

Routing adds a unified scheduling layer between the agent and multiple models. Routing rules determine which model receives each request.

For example:

• Use a faster model for code completion;
• Use a model with stronger context handling for complex architecture analysis;
• Use a lower-cost model for document summarization;
• Send sensitive-data tasks only to a local or controlled environment.

The key to routing is not the number of models, but task classification. Start with a small number of stable rules instead of designing an overly complex scheduling system from the beginning.

Aggregation

An aggregation layer manages multiple model services behind a unified interface. The agent connects to a single entry point, while the aggregation layer handles protocol conversion, authentication, rate limiting, logging, and failover.

This approach is suitable for team environments and multiple projects, but it also creates additional maintenance responsibilities:

• Protect aggregation-layer credentials;
• Record request chains and error details;
• Prevent differences in model parameters from affecting results unexpectedly;
• Clearly separate development, testing, and production configurations.

4. Build a Productive Workspace with Context, Skills, and Rules

Connecting a model to an agent does not automatically provide stable productivity. The user experience is often determined by project context, reusable skills, and behavioral rules.

4.1 Project Rule Files

Create a rule document in the project root that describes:

• The project purpose and directory structure;
• Coding style and naming conventions;
• How to test, build, and start the project;
• Directories that must not be modified;
• Checks required before commits;
• Requirements for error handling, logging, and security;
• How the agent should ask questions when requirements are unclear.

This document functions as a project-level operating manual. It should provide executable decision criteria rather than abstract slogans. For example:

Before modifying an interface, inspect its existing callers.
After completing the change, run unit tests and static checks.
For data deletion, show the plan and request confirmation first.
When the business meaning is unclear, do not assume critical field definitions.

4.2 Skill Encapsulation

Encapsulating repeated processes as skills can significantly reduce the amount of explanation required in each conversation. A skill typically includes:

• Applicable scenarios;
• Required input;
• Execution steps;
• Output format;
• Failure handling;
• Whether human confirmation is required.

Examples include code review, API testing, log analysis, change-note generation, and web regression testing. The value of a skill lies in standardization, not in making prompts longer.

4.3 Extension Protocols and Tool Calls

When the agent needs access to search, browsers, file systems, or devices, expose those capabilities through a consistent tool protocol. Every tool should have:

• A clear name and purpose;
• Explicit input parameters;
• A predictable return structure;
• Permission restrictions;
• Error information;
• Timeout and retry policies.

The more accurate the tool description, the more reliably the agent can select it. Destructive operations should provide a preview or dry-run mode whenever possible.

5. Common Daily Operations and Long-Running Tasks

The everyday work of an AI coding agent is not limited to code generation. It also includes understanding projects, locating problems, running tests, and organizing results.

Daily tasks can be divided into four categories:

1. Exploration: Read directories, trace call relationships, and analyze configuration.
2. Modification: Add features, fix defects, and refactor code.
3. Verification: Run tests, inspect logs, and build the project.
4. Summarization: Generate change notes, identify risks, and list follow-up tasks.

For each task, use a “goal–constraints–verification criteria” format:

Goal: Fix the error message shown after a failed login.
Constraints: Do not change the API contract or alter the existing successful flow.
Verification: Add tests for failure scenarios and explain which files were modified.

For long-running tasks, use scheduled triggers or a persistent process. Examples include periodically running tests, scanning logs, or checking a task queue. Persistent execution must include:

• A maximum execution time;
• A maximum retry count;
• Concurrency limits;
• Log rotation;
• Error notifications;
• A shutdown and human takeover mechanism.

Without these controls, automated tasks can run repeatedly, exhaust resources, or spread errors.

6. Web Automation: From Element Location to Result Verification

Web automation should not focus only on whether a click succeeded. It should verify the complete business outcome. A recommended process is:

1. Open the target page.
2. Check whether the page has finished loading.
3. Locate the control and read its current state.
4. Perform the click, input, or selection.
5. Wait for asynchronous requests to finish.
6. Check notifications, page content, or network results.
7. Capture a screenshot and record key evidence.
8. If the operation fails, save the page state for later analysis.

Prefer stable attributes and semantic information when locating elements. Avoid relying too heavily on hierarchical paths that may change. Login flows, CAPTCHAs, and permission dialogs should retain a human takeover option.

Web automation must also account for:

• Network latency;
• Page redirects;
• Asynchronous loading;
• Temporarily invisible elements;
• Switching between multiple tabs;
• Expired sessions;
• Duplicate submissions.

7. Device Automation: Connect, Execute, Observe, and Recover

The key to mobile device automation is not sending tap commands, but creating an observable verification process. Each complete step should include:

1. Check whether the device is online.
2. Confirm the current application and page.
3. Install, launch, or interact with the application.
4. Read the UI state, system logs, or application logs.
5. Determine whether the expected result was achieved.
6. Save screenshots and logs if the operation fails.
7. Clean up test data and restore the environment.

Device operations vary across screen resolutions, operating system versions, permission states, and network environments. Do not rely only on fixed coordinates. Prefer recognizable text, control identifiers, or UI hierarchy information. When coordinate-based actions are unavoidable, combine them with screenshots and state checks.

8. Create a Combined Web-and-Device Feedback Loop

A more advanced automation scenario allows the agent to operate both a web interface and a mobile device. For example:

Create test data in a web console → trigger mobile synchronization → launch the application → verify the displayed page → return to the web interface and check the status → generate a test report

This type of task should not be implemented as a sequence of mechanical commands. The agent should work through an “observe–plan–execute–verify” loop.

Recommended design principles include:

• Encapsulate web and device operations as separate tools;
• Define a success condition for every step;
• Read the state after each operation instead of continuing blindly;
• Save screenshots, logs, and timestamps at important checkpoints;
• Set a maximum retry count;
• Stop immediately and request human intervention for unrecoverable errors.

The model should interpret intent and plan the workflow, tools should execute actions, and verification modules should determine the result. Separating these responsibilities makes the system easier to maintain.

9. Best Practices for Production Adoption

First, begin with low-risk tasks such as reading a project, generating tests, and organizing logs. Gradually expand permissions for writing files and controlling devices.

Second, every automation workflow should be replayable. Save the input, tool calls, important outputs, and final state so that failures can be investigated.

Third, prioritize verification capabilities. Automation without tests, logs, and screenshots may appear successful while merely proving that some actions were executed.

Fourth, a model is not a replacement for business rules. Critical workflows should still be constrained by code, validators, and permission systems. Do not delegate every decision to a natural-language model.

Fifth, periodically clean up context and skill configurations. Outdated rules can mislead the agent, redundant tools can make selection harder, and inactive interfaces can cause repeated failed requests.

Conclusion: Test the Agent Inside a Real Workflow

The value of an AI coding agent is not determined by how much code it generates in a single session. It comes from whether the agent can reliably complete a task from information retrieval to result verification.

Connecting a domestic or international model is only the starting point. Real engineering capability comes from appropriate routing strategies, clear project rules, controlled tool permissions, and feedback loops across web and device layers.

The most effective approach is to select a real but low-risk task, define its goal and acceptance criteria, and let the agent participate in searching, analysis, modification, execution, and verification. By continuously testing the boundaries of the workflow and observing where the agent is reliable or requires human takeover, you can gradually build an automation workspace that fits individual and team needs.