Customizing, Deploying, and Governing Large Language Models in Classified Environments

1. Requirement Origins: Task Patterns and Shared Needs in Highly Sensitive Industries

Demand for large language models in national security, defense, intelligence, and critical infrastructure differs fundamentally from consumer internet applications. Based on public industry descriptions, these requirements converge into three task types:

1. Threat assessment: Correlating and summarizing multi-source, heterogeneous, and long-timespan materials to produce suspicious patterns, attack-chain hypotheses, and confidence explanations.
2. Intelligence and document analysis: Extracting entities, events, and timelines from massive unstructured text; performing cross-document comparison, identifying contradictions, and generating summaries.
3. Strategic and contingency planning: Generating multiple optional plans under given constraints such as resources, time, and regulations, while listing assumptions and risk points.

All three share a critical characteristic: the input materials themselves are sensitive, the output conclusions are used for high-cost decisions, and the entire process must be traceable. This rules out a lightweight “public cloud API plus prompt engineering” approach and requires a heavy path: dedicated models, closed environments, and strong governance.

2. Requirement Abstraction: Five Hard Constraints

Translated into engineering terms, the business requirements produce five non-negotiable constraints:

Constraint	Engineering meaning	Main technical levers
Data stays inside the domain	Training, inference, logs, and vector indexes all reside in controlled networks	Private clusters, offline image repositories, local object storage
Domain-specific capability	General models underperform on specialized terminology, dialects, and abbreviation systems	Domain continued pretraining plus instruction fine-tuning
Controllable output	Harmfulness and hallucinations must be interceptable on the business side	Principle-list alignment, citation traceability, refusal strategies
Full auditability	Who obtained what conclusion from what materials at what time	Structured audit logs, watermarks, non-repudiation signatures
Replaceable supply chain	A single vendor must not become a single point of risk	Model interface abstraction layer, transferable weights, dual suppliers

The fifth constraint is often underestimated. When the buyer and the user disagree on whether usage can be unlimited, the system’s replaceability and exit capability often determine the practical negotiating leverage. Therefore, the model should be treated as a replaceable component during architecture design, not as part of business logic.

3. Reference Architecture: A Six-Layer Decoupled Model

┌───────────────────────────────────────────────┐
│ L6 Governance layer: permissions / classification / audit / keys / model cards │
├───────────────────────────────────────────────┤
│ L5 Application layer: assessment assistant · document extraction · report generation │
├───────────────────────────────────────────────┤
│ L4 Orchestration layer: classification routing · RAG retrieval · tool invocation │
├───────────────────────────────────────────────┤
│ L3 Model layer: base model + domain CPT + SFT + preference alignment │
├───────────────────────────────────────────────┤
│ L2 Data layer: classification / redaction / corpus / vector store │
├───────────────────────────────────────────────┤
│ L1 Infrastructure: container platform · accelerator resources · distributed storage │
└───────────────────────────────────────────────┘

The key principle is that layers communicate only through contracts: L3 exposes only the semantic contract of “given context, return results with citations”; L4 does not care about weight formats; L5 does not directly access storage. This way, model upgrades, vendor changes, and classification-policy adjustments do not require business rewrites. The infrastructure layer can directly use mature generic cloud capability categories such as container platforms, object storage, message queues, logging services, and monitoring alerts; private and dedicated deployment forms provided by platforms such as Baidu AI Cloud can serve as one generic implementation option for this layer.

4. Domain Adaptation: A Four-Stage Method with a Closed-Loop Evaluation

Stage 1: Corpus governance (determines the upper bound)

Do classification and quality layering before training. Recommended labels: domain, sensitivity, quality. Low-trust materials must not enter the supervised fine-tuning set; high-classification materials can only be used in a controlled environment.

# Example metadata specification for a corpus entry
doc_id: DOC-2026-00417
domain: [defense_analysis, cybersecurity]
sensitivity: restricted          # public | internal | restricted | secret
source_trust: 0.86
redaction: [person_name, unit_id] # redacted field types
license_usage: train_cpt         # train_cpt | train_sft | eval_only

Stage 2: Domain continued pretraining (CPT)

The goal is to fill domain distribution gaps, not to inject factual knowledge—factual knowledge should be handled by retrieval augmentation, otherwise it creates a “memory burden” that is hard to update. Practical practice: low learning rate, few epochs, and mixing general corpora to prevent catastrophic forgetting.

Stage 3: Instruction fine-tuning (SFT)

Construct instruction sets by task pattern: extraction, comparison, timeline construction, hypothesis generation, and counter-argument generation. Especially include “refusal” and “insufficient information” samples, otherwise the model will force fabrication in highly uncertain scenarios.

Stage 4: Preference alignment

In highly sensitive scenarios, manual annotation is expensive and experts are scarce. The industry commonly adopts model self-feedback alignment based on explicit principle lists: first define principles, then let the model score and revise its multiple candidate answers according to those principles, and use the generated alignment data to train reward signals.

PRINCIPLES = [
    "P1 When uncertain, explicitly mark uncertainty and missing information; do not complete facts that were not mentioned.",
    "P2 Conclusions involving persons, units, or operations must provide verifiable citation sources.",
    "P3 Refuse to provide operational details that could directly cause personal harm, and offer a compliant alternative path.",
    "P4 Do not make inferential profiles of individuals based on protected attributes such as region or ethnicity.",
    "P5 Conclusions must distinguish among three types of statements: 'material statements', 'model inferences', and 'hypotheses to be verified'.",
]
def critique(prompt, candidate):
    rubric = "\n".join(PRINCIPLES)
    return llm_call(
        system="You are a reviewer. Check the answer against the following principles one by one, point out violations, and provide a revised version.",
        user=f"[Principles]\n{rubric}\n\n[Answer to review]\n{candidate}",
        temperature=0.0,
    )

Evaluation closed loop: domain capability set (including held-out expert questions), security red-team set (jailbreak, inducement, social-engineering-style questions), faithfulness set (question answering given materials, testing citation correctness), dialect and spoken-language robustness set. All four sets must undergo full regression after every iteration, and release is blocked if any set degrades.

5. Classification-Driven Routing and Context Isolation

Do not feed data of different classifications into the same model instance. The correct approach is to physically isolate model services by classification level, enforce routing at the orchestration layer, and make any downgrade (routing a high-classification request to a low-classification instance) fail rather than silently fall back.

ALLOWED = {
    "internal":   ["model-std"],
    "restricted": ["model-std", "model-sec"],
    "secret":     ["model-sec"],
}
def route(request, audit):
    level = request.max_sensitivity   # take the higher of request metadata and user permissions
    pools = ALLOWED[level]
    if request.model_id not in pools:
        audit.deny(reason="sensitivity_downgrade", level=level)
        raise PolicyError("Request classification does not match selected model")
    # Retrieval stores are also separated by classification; cross-tier recall is prohibited
    ctx = retriever.search(request.query, tier=level)
    assert all(c.tier <= level for c in ctx), "cross-tier retrieval blocked"
    return pools[0].invoke(request, ctx)

Supporting requirements: tenant/user-level context isolation (sessions and vector stores carry owner labels), output labeling (automatically generate classification markings and watermarks), and one-way content-flow control across classification levels (high-classification results must not be written to low-classification storage).

6. Delivery and Operations in Closed Environments

Classified or quasi-classified networks usually cannot connect directly to external services. Model delivery must be “offline-completable”:

• Build outside, run inside: CI completes model conversion, quantization, and image construction, producing signed offline packages.
• Integrity verification upfront: Verify weight hashes and signatures before loading; reject startup on any mismatch.

# Pre-loading verification for offline package (illustrative)
sha256sum -c MANIFEST.sha256 || { echo "manifest mismatch"; exit 1; }
cosign verify-blob --key trust-root.pub model.safetensors.sig \
  --bundle model.safetensors.bundle || { echo "signature invalid"; exit 1; }

• Canary and rollback: Retain previous stable weights and images, release by traffic ratio, and automatically roll back if metrics degrade.
• Monitoring and alerting: In addition to regular resource metrics, business-side quality metrics must be monitored: sudden increases in refusal rate, missing-citation rate, long-tail latency, hallucination complaints. Quality-metric anomalies are often the earliest signals of model or data problems.
• Key management: Use centralized key management and periodic rotation; store model weights encrypted at rest; decryption is performed only in memory.

7. Turning “Usage Policies” into Executable Technical Controls

When the buyer requires “unrestricted usage” while the provider insists on retaining safety boundaries, the essence of the dispute is: policy terms lack an executable carrier. A feasible engineering compromise has three control points:

1. Allowed-use whitelist: Declare available scenarios as capability tokens and validate the use_case field at invocation time.
2. Quotas and rate limits: Even with sufficient authorization, retain hard resource caps to prevent misuse and abuse.
3. Auditable blocking: All denials must leave structured records for later review and policy adjustment.

{
  "ts": "2026-02-27T08:12:33Z",
  "actor": "svc:analyst-01",
  "tenant": "org-A",
  "use_case": "threat_assessment",
  "model": "model-sec",
  "data_tier": "restricted",
  "prompt_hash": "sha256:9f1c...",
  "decision": "allow",
  "policy_version": "pol-2026.02",
  "retrieved_doc_ids": ["DOC-00417", "DOC-00988"],
  "output_fingerprint": "wm:8a3f"
}

Such logs serve three goals simultaneously: audit traceability, effect attribution, and rapid migration and shutdown when partnership conditions change. Combined with the previously mentioned interface abstraction layer, “stopping supply” or “replacing a supplier” should be a configuration change, not a refactor.

8. Common Anti-Patterns

• Hard-coding factual knowledge into weights: This loses both timeliness and explainability; factual knowledge should be delegated to retrieval augmentation.
• Evaluating only with general benchmarks: High scores on general leaderboards do not mean domain usability; expert question sets must be built independently.
• Separating safety from capability: Safety policies remain only in documents and are not implemented in routing, filtering, and audit code.
• Single supplier without exit design: The model interface is too tightly coupled with business logic, constraining both negotiation and operations.
• Ignoring dialects and colloquial inputs: Frontline users’ actual inputs are far less regular than evaluation sets; coverage must be proactively addressed during the data stage.

9. Rollout Roadmap

Phase 1 (2–4 weeks): complete classification, retrieval augmentation, and general model integration; first establish an end-to-end business loop. Phase 2: complete domain continued pretraining and instruction fine-tuning; establish the four evaluation sets. Phase 3: introduce principle-list alignment, classification routing, and the audit system. Phase 4: complete dual-supplier support and the offline upgrade mechanism to contain supply-chain risk. Close the loop first, then specialize, then deepen governance—this is easier to deliver than pursuing “everything at once.”