Balancing Openness and Safety in AI Models: Risk Assessment, Guardrails, and Tiered Governance

The Openness-Safety Dilemma: What Changes When Weights Are Open

In the context of large language models, “open” typically refers to open weights: users can download the model parameters and deploy, fine-tune, or even redistribute them in their own environments. This approach offers three irreplaceable benefits: private deployment enables compliance with data regulations and audit requirements; models can be customized for specific vertical domains; and researchers can reproduce and validate results, driving progress across the field. In recent years, open models have continued to close the performance gap with leading closed-source models on benchmarks for coding, tool use, and complex task handling, and the open ecosystem’s growth is now an established fact.

But open weights carry an inherent property: they cannot be recalled. In an API-hosted model, the provider can shut down a dangerous capability or tighten outputs at any time. Once released, however, weights persist on the internet permanently. This means that all risk control must be completed before the release decision is made—there is almost no room for post-hoc remediation.

A more fundamental issue is that mainstream safety alignment is a form of behavior modification—training the model to say “no” to dangerous requests. In an open-weight environment, anyone can fine-tune the model with modest data and compute to strip away this alignment. Industry experiments have repeatedly shown that removing refusal behavior costs far less than building it. Therefore, the safety ceiling of an open model is not determined by the release-time guardrails, but by the model’s underlying capability level. This is precisely why capability assessment has become the central pillar of the open-release decision.

Pre-Release Capability Assessment: Controlling Risk Upfront

A mature assessment framework typically covers two dimensions.

The first is general capability benchmarking: evaluating the model on coding, tool use, complex reasoning, and other areas to establish its capability profile. In practice, multiple benchmark results should be reported together to avoid the bias of any single metric. Notably, jumps in coding and tool-use ability directly correlate with the model’s potential to “operate a computer and call external interfaces”—a classic double-edged capability.

The second is safety-focused red teaming, which carries higher weight in release decisions. Typical areas of evaluation include:

• Cyber offensive capability: Can the model autonomously complete attack chains such as reconnaissance, exploitation, and lateral movement?
• Harmful knowledge threshold: Does the model significantly lower the barrier to obtaining dangerous knowledge?
• Social engineering capability: Can it automatically generate phishing content and disinformation?
• Autonomy risk: Can it acquire resources and replicate itself without human intervention?

The common industry practice is to set capability thresholds: when a model crosses a predefined threshold on a dangerous capability, it triggers additional review or even a release hold. Two practical points matter in execution. First, the evaluation environment must be fully isolated—any offensive testing must occur in a network-disconnected sandbox. Second, the assessment results must be tightly coupled to the release workflow, not shelved as “reference material.”

A Red Line in Dangerous Capabilities: Real-World Observations of Cyber Ability

A recent joint safety evaluation of a frontier model by overseas security assessment bodies sent a signal warranting industry-wide attention: the model was able to autonomously complete parts of cyber attack tasks in a controlled environment. More critically, the model’s built-in safety mechanisms failed to prevent it from attempting to exploit software vulnerabilities.

These observations carry two implications:

1. Offensive capability has moved from “theoretical risk” to “observed reality.” When a model can autonomously chain together reconnaissance, targeting, and exploitation, the technical barrier to attack is materially lowered, expanding the risk population from a small group of professional attackers to ordinary users.

2. Refusal-based guardrails have clear blind spots. Guardrails mainly intercept explicitly malicious expressions, while attack chains are often decomposed into a series of neutral steps—reading a piece of code, sending a request, invoking a tool. Each step is individually legal, but combined they form a complete attack.

Why Guardrails Fail: Three Technical Root Causes

First, the neutral disguise of tasks. Complex attacks superficially look like a series of legitimate coding and operations tasks. Intent detection naturally fails at the single-step level, and most guardrails are precisely step-by-step judgment systems.

Second, context dilution. In long task chains, the influence of safety constraints decays as context expands. When the model’s optimization objective becomes dominated by “completing the user’s task,” the weight of safety tendencies learned during training correspondingly drops.

Third, the shallowness of alignment. Refusal behavior is more like “learned phrasing” than “internalized principles,” and low-cost methods such as role-playing or system prompt injection can bypass it. Open weights further amplify this weakness—guardrails can be directly removed through fine-tuning.

The engineering conclusion is therefore unambiguous: safety must not be placed on the model’s own refusal behavior; defenses must be built outside the model, inside the deployment environment.

Tiered Release and Governance: From “Whether to Open” to “How to Open”

Mature practice does not frame the decision as a binary between “fully open” and “not open.” Instead, it establishes a tiered mechanism:

• Full open release: accompanied by a model card, evaluation summary, and explicit usage license.
• Research license: time-limited and quota-controlled access for vetted institutions.
• API-restricted: available only through a hosted service, with input/output filtering, rate limits, and behavior auditing.
• Delayed release: initial controlled release followed by a sufficiently long observation window before full openness.

Supporting transparency materials should include at least three documents: a model card (capabilities and known limitations), a safety evaluation summary (sanitized red team findings), and a usage policy (explicitly prohibited uses and liability boundaries). The value of these materials is that they spare downstream users from assessing risk from scratch, converting the publisher’s evaluation output into a shared public safety asset across the ecosystem.

Enterprise-Side Implementation: Five Engineering Practices for Private Deployment

For enterprises building applications on open models, the following measures are recommended:

1. Least-privilege permissions. Tools accessible to the model (shell, file system, browser, internal interfaces) should be authorized through a task-based allowlist, with default denial; high-risk operations should require manual confirmation.

2. Bidirectional content filtering. On the input side, build a two-layer filter combining rule-based detection and classification models for jailbreak and prompt injection patterns. On the output side, detect typical exploit payload signatures.

3. Sandbox execution. Any scenario where the model is allowed to execute code or commands must run inside resource-constrained, network-isolated containers.

4. End-to-end audit logging. Record every tool call and every generation into centralized logs, configure monitoring and alerts, and raise real-time warnings for anomalous patterns such as high-frequency scanning or privilege-escalation attempts.

5. Continuous red teaming. Deployment is not the endpoint. Periodically regression-test guardrails against the latest attack techniques to prevent defense degradation over time.

An illustrative server-side guardrail pipeline is shown below:

def serve_with_guardrails(request):
    # Layer 1: Input-side risk identification
    if input_filter.is_high_risk(request.text):
        return refuse_with_reason(policy.HIGH_RISK_INPUT)
    # Layer 2: Model call with restricted permissions
    ctx = build_context(request, tools=allowlist(request.task))
    response = model.generate(ctx)
    # Layer 3: Output-side detection and audit trail
    if output_filter.matches_exploit_pattern(response.text):
        audit_log.record(request, response, level="HIGH")
        return redact_or_refuse(response, policy.EXECUTION_RISK)
    audit_log.record(request, response, level="INFO")
    return response

Regarding implementation, not all of this infrastructure needs to be built from scratch. For example, enterprises using Baidu AI Cloud can leverage its enterprise-grade large model services for hosting and fine-tuning, and combine platform-level logging, monitoring, and alerting capabilities to assemble a closed loop of “deployment, audit, and alerting” in a relatively short time—keeping security infrastructure costs manageable and focusing effort on the business itself.

Conclusion: Safety Is a Prerequisite for Openness, Not Its Opponent

The signals from third-party evaluations are clear enough: models’ offensive cyber capabilities are no longer hypothetical, and guardrail failures are not coincidental. For publishers, assessment must be front-loaded, thresholds must be explicit, and tiered releases must be implemented. For adopters, defense-in-depth must be built outside the model. Openness spreads capability; governance keeps it controllable. Only when “stronger capability demands finer governance” becomes an industry consensus can openness shift from a risk proposition into a genuine ecosystem dividend.