Anthropic Large-Model Research Survey: Scientific AI Adoption in Data Analysis, Writing, and Enterprise Practice

1. Key Survey Findings: Data Analysis and Writing Dominate

A recent large-model vendor released a survey report on how researchers use large models. The main findings are as follows:

• Task concentration. Researchers mainly use large models for two types of tasks: data analysis and processing, and text writing and revision.
• Uneven disciplinary use. Large-model applications are more common in computer science, mathematics, and statistics than in health sciences, experimental physics, and related fields.

The report also notes that whether AI can accelerate scientific discovery depends on two factors: the capability boundaries of the model itself, and how scientists use it. In other words, tool capability sets an upper bound, but the actual outcome is determined by the way it is used.

This conclusion is consistent with recent industry observations: when large models enter scientific research, they first gain a foothold in two high-frequency, general-purpose, and iterative areas: data and text.

2. Why Data Analysis and Writing Are Adopted First

Combining technical characteristics with real-world usage, there are three main reasons.

1. Data formats naturally fit model capabilities

Research data are often presented in tabular form, such as experiment records, survey results, measurement sequences, and simulation outputs. Column headers are field names, each row is a record, and each cell is a value. This structure can be converted into both natural language and executable code. Large models can infer data meaning from column names, value distributions, and a small number of examples, then generate corresponding processing logic. As a result, the barrier to “understanding data” is significantly reduced. Scientific texts such as papers, reports, and literature reviews are also mainly natural language, and language modeling is a core capability of large models.

2. Fast feedback and visible errors

Whether an analysis conclusion holds can be checked through recalculation, cross-validation, or reproduction. Writing quality also has clear evaluation dimensions: whether the logic is consistent, whether the expression is standardized, and whether citations can be verified. The faster and clearer the feedback, the more efficient the collaboration between humans and models. By contrast, wet-lab validation is often measured in weeks or months, making it difficult to form an efficient iteration loop.

3. General needs and low startup cost

From outlier screening to statistical testing, and from figure captions to paper polishing, these needs are highly similar across disciplines and do not require a dedicated system for every laboratory. Researchers can use conversational interaction directly or embed tools into daily analysis workflows, quickly obtaining benefits. This also explains why these use cases spread the fastest.

It should be emphasized that early adoption does not mean maturity. In data analysis, problems such as hallucination, non-reproducibility, and incorrect implicit assumptions still exist. In writing, one must also be alert to content that appears fluent but is factually incorrect. The key is therefore to turn “usable” into “reliably usable.”

3. Data Analysis: From “Can Calculate” to “Can Be Trusted”

When introducing large models into data analysis, the recommended principles are “structured input, executable code, and verifiable output,” supported by necessary validation mechanisms. A complete workflow usually includes five steps:

1. Understand the data: clarify units, dimensions, the meaning of missing values, grouping variables, and sample size;
2. Clean and preprocess: handle missing values and outliers, unify units, and construct derived variables;
3. Statistical testing: select appropriate tests based on data distribution and research hypotheses;
4. Interpret results: explain effect sizes, confidence intervals, and limitations;
5. Deliver reproducibly: output code and intermediate results so that others can recalculate them.

1. Illustrative Code: Constrained, Traceable Analysis Workflow

The following is illustrative pseudocode in a Python-like syntax style. The design idea is to require the model not to directly provide “conclusions,” but to output executable analysis logic with precondition checks.

import statistics as stats
def run_analysis(df, plan):
    # 1. Pre-check assumptions: sample size, grouping, and key fields required by the method
    check = validate_assumptions(df, plan.method)  # e.g., independence, normality, homogeneity of variance
    if not check.ok:
        return {"status": "blocked", "reason": check.reason}
    # 2. Execute analysis with a fixed random seed to ensure reproducibility
    result = plan.method(df, group_col=plan.group_col,
                         value_col=plan.value_col,
                         random_seed=42)
    # 3. Robustness check: see whether the conclusion remains consistent after removing outliers or using another method
    alt = plan.alternative_method(df, group_col=plan.group_col,
                                  value_col=plan.value_col)
    result["robust"] = agree(result.effect_direction, alt.effect_direction)
    # 4. Generate conclusions: every conclusion must point to a specific intermediate result field
    conclusions = []
    for c in plan.claims:
        if c.result_field not in result:
            raise KeyError(f"Conclusion lacks supporting data: {c.result_field}")
        conclusions.append(render(c.template, **result[c.result_field]))
    return {"status": "ok",
            "statistics": result,
            "conclusions": conclusions,
            "code": plan.to_source()}  # Include executable code for recalculation
# Companion example: summarize the mean difference between two independent samples (illustrative)
def summarize_difference(a, b):
    diff = stats.mean(a) - stats.mean(b)
    # In real projects, use dedicated hypothesis tests and correct significance thresholds for multiple comparisons
    return {"mean_a": stats.mean(a), "mean_b": stats.mean(b),
            "effect_direction": "up" if diff > 0 else "down",
            "absolute_difference": abs(diff)}

This code reflects three engineering principles:

• Traceable conclusions. Every textual conclusion maps to a specific statistic, preventing the model from making unsupported statements.
• Reproducible results. A fixed random seed and source code output make it easier for others to recalculate results.
• Robust conclusions. If the conclusion direction changes after switching to another test method, the system should prompt manual review rather than directly output the conclusion.

2. Five Risk Types to Guard Against

Risk Type	Typical Manifestation	Mitigation
Numerical hallucination	Fabricated sample sizes, means, or p-values	All numbers are calculated by code; the model only interprets them
Missing assumptions	Directly applying parametric tests to non-independent or non-normal data	Validate method assumptions in advance; automatically switch methods when assumptions are not met
Causal misreading	Describing correlation as causation	Require conclusions to explicitly state evidence level and study limitations
Multiple comparisons	Many hypothesis tests without correction, increasing false positives	Apply correction methods uniformly and enforce them in the workflow
Non-reproducible results	Random data splits or missing-value imputation methods not recorded	Fix random seeds and fully record preprocessing steps

4. Writing and Literature: Verification as a Prerequisite

Writing is one of the most frequent scenarios for researchers. Common needs include language polishing of papers, abstract refinement, responses to reviewer comments, grant proposal writing, and figure caption preparation.

1. Build on Retrieval-Augmented Generation

For pain points in literature review, background surveys, and related-work sections—such as “cannot find the right sources, cannot see all relevant work, and cannot remember details”—Retrieval-Augmented Generation (RAG) can be used:

1. Split, vectorize, and write documents such as papers, patents, and internal reports into a vector store;
2. When a user asks a question, first retrieve relevant passages, then let the model organize the answer;
3. Mark sources in the generated output to facilitate item-by-item verification.

It is worth noting that retrieval quality often determines answer quality more than the model itself. Poor chunking granularity, missing metadata, and weak recall or ranking can directly lead to incorrect citations or missing information.

2. Three Writing Norms

• Separate facts from expression. The model handles structure, expression, and logical organization, while facts, data, and conclusions must be confirmed by the author. After the first draft is completed, the author should perform “source-by-source comparison” for every citation to confirm that the reference exists and that the viewpoint is accurately represented.
• Transparent disclosure. Journals and research management organizations have increasingly clear disclosure requirements for generative AI tools. Whether such tools are used, and in which parts of the work, should be truthfully explained according to the rules of the target journal or institution.
• Clear responsibility. Using a tool does not transfer the responsibility of the named author for content authenticity, originality, and ethical compliance. Practices such as copying entire passages from existing literature or generating fake references should be avoided.

5. Discipline Differences: Why Health Sciences and Experimental Physics Use Less

Differences in adoption depth across disciplines do not mean that models “cannot do the work.” They are more often caused by differences in data, workflows, and risk structures.

1. Sensitive Data and High Compliance Barriers

Health and life sciences involve personal health information, and some data may qualify as sensitive personal information. Processing such data must follow requirements such as informed consent, ethics review, and de-identification. General cloud services often cannot directly satisfy these constraints, so relevant teams tend to prefer local deployment or controlled research data environments.

2. Instrument Coupling and Inconsistent Standards

Data in experimental physics and many other experimental sciences come from various instruments and acquisition devices. Formats are diverse, dimensions vary, and metadata—such as sampling conditions, instrument parameters, and environmental conditions—is often missing or incomplete. Analysis workflows rely heavily on domain experience and specialized software. General-purpose models have difficulty directly understanding raw data and require additional data governance and domain adaptation.

3. Expensive Validation and Limited Fault Tolerance

In these disciplines, hypothesis validation depends on experiments or high-cost simulations. A single feedback cycle can be long, and the cost of errors is high. Building trust in models takes longer, and researchers place more emphasis on explainability and reproducibility.

4. Directions for Improvement

• For sensitive data, use dedicated deployment and data masking to ensure raw data does not leave the controlled environment;
• For instrument data, build unified metadata standards and data catalogs to provide structured context for models;
• For high-cost validation, let the model first handle “front-end exploration” tasks such as literature screening, experiment design, parameter scanning, and simulation code generation, while humans make the final judgment.

6. Evaluating the “Acceleration Effect”: Building Measurable Metrics

The report emphasizes that AI’s acceleration effect on scientific research depends on both model capability and usage patterns. This requires an evaluation system that can be quantified. Metrics can be established across four dimensions:

• Factual accuracy: citation verifiability rate, numerical recalculation consistency rate, hallucination rate;
• Workflow efficiency: time change per step, data-to-draft cycle time, manual rework rate;
• Result quality: proposal adoption rate, reproduction success rate, number of technical errors in peer-review comments;
• Knowledge and capability: how well researchers understand methods, and whether excessive reliance occurs when results are adopted without validation.

For evaluation methods, it is recommended to build dedicated evaluation sets for each research direction, including real tasks and gold-standard answers or expert ratings. Blind evaluation should be used to reduce subjective bias, and statistics should be calculated separately by task type to avoid using an overall average that hides weak areas.

7. Reference Architecture for an Enterprise Scientific AI Platform

For institutions that want to accumulate scientific AI capabilities, a “layered decoupling” architecture can balance flexibility, security, and maintainability. One possible reference architecture, illustrated with general capabilities from Baidu AI Cloud, is as follows:

┌──────────────────────────────────────────────────────────┐
│  Application Layer    Data Analysis Assistant | Literature Q&A | Writing Polishing | Experiment Record Retrieval │
├──────────────────────────────────────────────────────────┤
│  Orchestration Layer  Task Orchestration | Tool Invocation | Prompt Template Management | Human Review Nodes │
├──────────────────────────────────────────────────────────┤
│  Model Layer          General Large Model | Domain-Fine-Tuned Model | Embedding Model │
├──────────────────────────────────────────────────────────┤
│  Knowledge & Data Layer    Vector Database | Object Storage | Structured Data Storage | Metadata Catalog │
├──────────────────────────────────────────────────────────┤
│  Security & Governance Layer   Identity & Access Control | Data Masking | Logging | Monitoring & Alerting | Cost Metering │
├──────────────────────────────────────────────────────────┤
│  Infrastructure Layer    GPU Cloud Servers | Container Engine | Elastic Bare Metal Servers │
└──────────────────────────────────────────────────────────┘

Key points for each layer are as follows:

• Infrastructure layer: Training and fine-tuning can leverage Baidu AI Cloud’s Baige AI Heterogeneous Computing Platform, using large-scale cluster capability and multiple acceleration options to improve resource utilization. Inference services can be deployed on a container engine with elastic scaling to handle the tidal usage pattern of scientific research: frequent interaction during the day and batch analysis or batch polishing at night. For scenarios with high compliance requirements, elastic bare metal servers can be used to support dedicated deployments where data stays within the required boundary.
• Model layer: General large-model capabilities can be used directly, or models can be fine-tuned with domain data through the Qianfan large-model platform so that they better understand domain terminology and analysis standards. Non-structured content such as literature and experiment records should be vectorized, and the vector database should support hybrid retrieval and permission filtering.
• Data and knowledge layer: Raw experiment data and literature repositories can be stored in object storage, with lifecycle policies and version management to control costs; structured analysis results can be stored in a cloud database. For example, the columnar storage product GaiaDB-X is suitable for carrying large-scale scientific analysis data.
• Orchestration layer: Agent development tools can be used to orchestrate retrieval, computation, plotting, and other tools into reusable workflows. Human review nodes should be set at key points to form a closed loop of “model proposes, human confirms, system executes.” Prompt templates should be centrally managed and versioned for easier evaluation and optimization.
• Security and governance layer: Logging can be used to audit query and operation records, meeting traceability requirements for scientific data; cloud monitoring can track call volume, latency, and failure rate, and trigger alerts when necessary.

Three principles should be followed during implementation:

1. Data classification: classify data into four levels—public, internal, sensitive, and controlled—and match each level with appropriate deployment modes and access controls;
2. Least privilege: grant data and tool access by project, and mask sensitive data by default;
3. Full traceability: retain model inputs, retrieval sources, generated results, and human modification records to support later audit and reproduction.

8. Phased Implementation Roadmap

1. Pilot phase (1–2 months): choose low-risk, high-frequency scenarios such as paper polishing, code assistance, and preliminary tabular data analysis. Build an evaluation set and baseline, and clarify “red lines” that must not be crossed.
2. Expansion phase (3–6 months): embed high-frequency functions into researchers’ daily workflows, such as electronic lab notebooks, data platforms, and literature management systems. Accumulate domain knowledge bases and prompt templates, and organize training sessions.
3. Deepening phase (6–12 months): standardize the analysis workflow into a pipeline of “data validation → computation → robustness check → result delivery.” Build dedicated environments and domain models for sensitive or specialized fields such as health sciences and experimental physics.
4. Governance phase (continuous): establish spot-check mechanisms for model outputs, monitoring for abnormal usage, and regular reviews of effectiveness, and update metrics and standards based on evaluation results.

9. Risk Boundaries and Compliance Bottom Lines

• Prevent hallucinations from entering critical decision chains. Key decisions such as changes to experimental design, clinically relevant conclusions, and patent applications must be reviewed by professionals. Model outputs should be treated only as reference material.
• Protect data and privacy. Sensitive data must not be input into unauthorized external services. Ownership and scope of use of research data should be clearly constrained at the platform level.
• Maintain academic integrity. Comply with generative AI usage rules of journals, funding agencies, and affiliated institutions, and disclose usage truthfully.
• Maintain reproducibility. Analysis results should include code, parameters, random seeds, and data versions so that third parties can reproduce them.
• Avoid capability hollowing. The core of scientific training is developing judgment. Tools should help researchers validate hypotheses faster, not replace their thinking.

Conclusion

The core signal from this survey is that the role of large models in scientific research has moved from “being able to chat” to “being able to do work.” Data analysis and writing are adopted first because they combine generality, fast feedback, and relatively low compliance barriers. Lower usage in health sciences and experimental physics reflects practical constraints such as sensitive data, specialized workflows, and expensive validation.

For research institutions and enterprise technology teams, real competitiveness does not lie in “which model is connected,” but in three capabilities: whether they can build a high-quality data and knowledge foundation, whether they can design “verifiable, traceable, and reproducible” human-AI collaboration workflows, and whether they can continuously measure AI’s actual contribution with quantitative metrics. As model capabilities continue to improve, these underlying capabilities will determine how much AI can increase research efficiency and whether the results can withstand review.