Engineering Path for Deploying LLM Agents in Financial Core Business

1. Why Finance Is One of the Highest-Priority Industries to Be Reshaped

The daily output of financial institutions is not simple question answering. It is knowledge work built on top of data, documents, rules, models, and human responsibility.

An investment banking proposal may need to combine industry data, comparable companies, regulatory interpretations, and internal templates. A KYC screening may need to connect business registration information, watchlists, equity penetration analysis, and internal policies. A portfolio analysis may need to align holdings, market data, risk models, and client constraint terms.

These tasks share several characteristics:

• Highly heterogeneous inputs: structured market and accounting data, semi-structured reports, and unstructured contracts and emails are mixed together.
• Dense and versioned rules: internal policies, regulatory requirements, and product terms are continuously revised.
• Accountability cannot be outsourced: outputs need to be attributable, traceable, and explainable.
• Asymmetric cost of errors: a single misstatement or unauthorized access event can cost far more than the labor saved.

Therefore, the path for frontier large language models entering finance is not simply to add a chat box. It is to embed model capabilities into existing systems and workflows, while designing compliance controls, data connections, and human review directly into the process.

This is also why insurance underwriting and asset management portfolio analysis can be viewed as entering critical production environments: they have explicit input and output contracts, verifiable results, and defined human review positions.

2. Reference Architecture: Six Layers

An auditable financial agent system can usually be divided into six layers. Each layer has a clear responsibility boundary and can be replaced independently.

1. Access layer
   Connects to existing entry points such as office suites, CRM, data terminals, and financial planning tools. It appears as plugins, sidebars, or batch jobs without changing users’ existing habits.

2. Context construction layer
   Handles two paths: retrieval-augmented generation and structured queries. Unstructured documents go through vector retrieval and reranking. Structured data goes through a semantic layer plus controlled SQL generation, rather than allowing the model to connect directly to production databases.

3. Orchestration layer
   Handles task decomposition, tool selection, loop control, and termination conditions. It is recommended to use a directed acyclic graph to declare steps explicitly, instead of leaving planning entirely to the model.

4. Execution layer
   A set of whitelisted deterministic tools for calculation, comparison, formatting, and internal service calls. Any action involving numbers or compliance judgments must be executed in code, not generated directly by the model.

5. Governance layer
   Covers identity and permissions, data classification, masking, audit trails, rate limiting, and quotas.

6. Evaluation and operations layer
   Includes offline evaluation sets, online canary releases, monitoring alerts, and regression gates.

User / system entry
    ↓
Access layer: office suite / CRM / batch jobs
    ↓
Orchestration layer: explicit DAG flow + termination conditions
    ├──→ Context construction: vector retrieval | semantic layer → controlled SQL
    ├──→ Execution tools: calculator / comparator / internal API / report renderer
    ↓
Governance layer: inherited permissions · masking · audit · quota
    ↓
Evaluation and operations: offline evaluation · canary · monitoring alerts

3. Data Connectivity: From “Can It Read?” to “What Is Allowed to Read?”

In financial scenarios, the most underestimated engineering effort is data connectivity. The most dangerous issue is permission inheritance.

If an agent uses a single high-privilege service account to retrieve data, any user may be able to ask questions and gain unauthorized visibility into clients or portfolios that they should not access.

The correct approach is to perform data access under the caller’s identity. The retrieval layer should inject data-domain visibility labels into filtering conditions. SQL generation should go through views or controlled query interfaces, and direct access to raw tables should be prohibited.

Example:

def build_context(user, question, data_scope):
    # 1) Push permission filters into retrieval and query layers
    #    instead of filtering after the fact in the application layer.
    filters = resolve_scope(user)
    # Example: {"region": ["East China"], "client_tier": [">=A"]}
    docs = vector_search(question, filter=filters, top_k=8)
    rows = metric_query(
        question,
        view="cust_portfolio_v",
        scope=filters
    )  # Read-only view only
    return assemble(docs, rows, policy=load_policy(user.business_line))

Three practical constraints apply:

• Both retrieval and queries must enforce permissions. Writing “do not output unauthorized information” in the prompt is not a reliable control.
• Computation should be pushed down. Metrics such as return, duration, VaR, and reserves should be calculated by existing model services. The language model should only explain and write.
• Data classification and masking must be applied. Sensitive personal information should be masked before entering the model context. Keys and credentials should be managed by a key management service and should not enter prompts.

4. Compliance Controls: Put Rules in Code, Not in Prompts

“Prebuilt compliance controls” in engineering terms can be mapped to four types of guardrails.

• Pre-guard
  On the input side, detect prompt injection, identify unauthorized intent, and recognize sensitive data. Content from external documents should always be treated as untrusted data and must not override system instructions.

• Post-guard
  On the output side, perform schema validation, numerical consistency checks, prohibited-expression detection, and mandatory disclaimer checks. Examples of prohibited expressions include promises of returns and absolute statements.

• Citation traceability
  Every conclusion must carry source snippet IDs and data version numbers. The rule is: no citation, no output.

• Full-chain audit
  Record input summaries, retrieval hits, tool calls, model and prompt versions, and human review conclusions. Logs should be sent to a centralized logging service with a defined retention period.

SCHEMA = {
    "type": "object",
    "required": ["conclusion", "citations", "confidence", "needs_review"],
    "properties": {
        "conclusion": {"type": "string"},
        "citations": {
            "type": "array",
            "items": {"type": "string"},
            "minItems": 1
        },
        "confidence": {
            "type": "number",
            "minimum": 0,
            "maximum": 1
        },
        "needs_review": {"type": "boolean"}
    }
}
def guarded_run(user, task):
    assert not contains_injection(task)  # Pre-guard
    result = orchestrator.run(
        task,
        schema=SCHEMA,
        tools=WHITELIST[user.role]
    )
    verify_numbers(
        result,
        source_of_truth="risk_service"
    )  # Numbers must be reproducible
    if result.confidence < THRESHOLD[task.type] or HIGH_RISK[task.type]:
        result.needs_review = True  # Force a human review checkpoint
    audit_log.write(user, task, result)  # Tamper-resistant audit trail
    return result

Human-AI collaboration should be tiered rather than treated the same way for every task. A practical classification is based on reversibility × impact amount × customer visibility.

• Draft-oriented tasks may directly generate a draft pending confirmation.
• External sending, transaction instructions, underwriting conclusions, and tax or estate explanations should be placed in the highest tier. They should only take effect after dual confirmation within the system.

5. Eight High-Frequency Scenarios

Scenario	Agent Responsibility	Required Human Fallback
Investment banking proposals / performance materials	Document collection, draft writing, consistency checks on terminology and metrics	Judgment on conclusions, pricing, and external disclosure
KYC and watchlist screening	Information completion, equity penetration analysis, summary of suspicious points	Final determination of risk rating
Financial analysis and reconciliation	Difference localization, hypothesis generation, chart generation	Confirmation of adjusting entries
Client meeting minutes	Transcription, action item extraction, CRM update	Confirmation of commitments made to clients
Portfolio rebalancing assistance	Constraint validation, scenario analysis, explanatory documents	Trading decisions and compliance approval
Estate and tax explanations	Clause interpretation, document checklist generation	Legal and tax opinions
Insurance underwriting	Completeness check of submitted materials, explanation of matched rules	Underwriting decision and exception approval
Asset management portfolio analysis	Attribution analysis, report automation	Sign-off on external reports

Insurance underwriting is a good example of a critical production environment because it has a clear rules engine and verifiable results. The agent is responsible for “reading materials and explaining why rules were matched,” the rules engine is responsible for “making the determination,” and humans are responsible for “exceptions.”

This three-part division — the model handles semantics, the engine handles determination, and humans handle responsibility — is currently one of the most reliable operating patterns.

6. Evaluation and Gating: No Measurement, No Production

A three-layer evaluation approach is recommended.

• Offline gold-standard set
  Build 100 to 500 examples per scenario, each with standard answers and citations. Measure field-level accuracy, citation verifiability, hallucination rate, unauthorized access rate, and format compliance rate.

• Regression gate
  Whenever the model version, prompt, or retrieval configuration changes, run the full offline set. If metrics degrade beyond a threshold, block the release.

• Online observation
  Monitor metrics such as uncited answer rate, human rewrite rate, review overturn rate, P95 latency, and token cost per task. These data points should in turn drive prompt and retrieval strategy iteration.

For cost control, common practices include semantic caching, batch processing of offline tasks, model routing based on task complexity, limiting retrieval context length, and limiting the maximum number of tool-call rounds.

Model routing means using smaller models for extraction and formatting, while reserving larger models for complex reasoning.

7. Cloud Capability Mapping and Rollout Cadence

Deployment does not require building all infrastructure from scratch. General cloud capability categories can cover most needs:

• Object storage for documents and audit archives.
• Data warehouse / lakehouse for semantic-layer support.
• Message queue for asynchronous long-running tasks.
• Container platform for stateless orchestration services.
• Vector retrieval service for the retrieval layer.
• Logging service and monitoring alerts for observability.
• Key management for credential storage.

Cloud platforms such as Baidu AI Cloud can provide this type of general capability combination. Actual selection should be based on compliance requirements, data residency, and the existing technology stack.

A phased rollout is recommended, with clear gates at each stage.

1. Copilot phase
   Read-only access, draft generation, and mandatory human confirmation.
   Gate: citation verifiability and human rewrite rate meet the target.

2. Automation with human review
   Write into internal systems and generate documents pending approval.
   Gate: zero unauthorized access events and 100% audit completeness.

3. Limited automation on critical paths
   Enable automatic execution only for low-risk, reversible, and clearly rule-based subprocesses. Keep one-click rollback and sampling-based review.

8. Five Common Pitfalls

• Replacing permission control with prompt instructions.
• Allowing the model to generate SQL directly against production databases.
• Letting the model perform arithmetic and metric calculations.
• Missing citation traceability, making conclusions impossible to verify.
• Failing to fix versions, so historical conclusions cannot be reproduced after model or retrieval upgrades.

In one sentence: the success of LLM deployment in finance depends less on model capability than on an engineering foundation that is connectable, verifiable, and accountable. Give semantic understanding to the model, give determination and calculation to code, and return responsibility to humans. The position of this division determines whether a project can truly enter core business.