How Researchers Use LLMs for Research: Data Analysis, Writing, and Implementation Paths

I. The Current Landscape: AI Use in Research Is Already Stratified

Over the past two years, discussions about how AI enters scientific research have shifted from “whether it can be used” to “where it is used and how deeply.” Based on public research reports and extensive team practices, researchers’ use of general-purpose large models is highly concentrated in two types of tasks: data processing and analysis, and text writing and editing. The former includes data cleaning, statistical testing, generation of visualization scripts, and interpretation of results. The latter includes first drafts of literature reviews, organization of experimental records, polishing of paper paragraphs, responses to peer-review comments, and drafting project proposals.

It is worth noting that these two types of tasks share a clear property: their input and output boundaries are well defined, and results can be verified quickly. Conclusions from data analysis can be checked with statistical tests and replication experiments. The quality of text can be judged quickly by peer reading. By contrast, AI use in hypothesis generation, experimental design, and instrument parameter calibration is clearly lower, because errors in these areas are costly, verification cycles are long, and context dependencies are strong.

II. Disciplinary Differences: Why Computational Fields Have Higher Adoption

A stable pattern across reports is that use is more widespread in computer science, mathematics, and statistics, while adoption lags in fields dominated by wet-lab work or clinical workflows, such as health sciences and experimental physics. This difference does not stem from tool availability, but from the combined effect of several structural factors:

Factor	Computational fields	Health sciences and experimental physics
Data form	Code, structured tables, public datasets	Patient data, instrument-specific formats, images, and spectra
Whether data can leave the environment	Often can be processed after de-identification	Strongly constrained by privacy and ethics approval
Verification cost	Unit tests and reproducible scripts; seconds	Requires bench work, reagents, and ethics cycles; weeks to months
Corpus density	Sufficient training corpus; consistent terminology	Long-tail domain terminology; ambiguous abbreviations
Cost of failure	Rollback possible; impact controllable	Involves subject safety and experimental cost

The conclusion is clear: the speedup AI provides for research depends on the product of task verifiability, data availability, and workflow embedding depth—not on model parameter size. If any one factor approaches zero, the overall benefit collapses. This is also why a seemingly advanced usage, such as “letting AI read a paper and then give conclusions,” produces far less than “letting AI rewrite this R code snippet into a reproducible plotting function.”

III. How to Objectively Evaluate AI Use Within a Team

Many institutions evaluate AI use in research only by asking whether it has been used. This indicator carries almost no information. A more useful approach is to define a set of tiered metrics, provided that instrumentation is first added along the call chain.

# Example: logging fields for a single call to a research AI assistant
event = {
    "session_id": "s-20260901-0421",
    "user_role": "phd_student",            # Role: undergraduate student / graduate student / PI / engineer
    "discipline": "statistics",            # Discipline tag
    "task_type": "data_analysis",          # data_analysis / writing / code / literature / hypothesis
    "data_sensitivity": "L2_internal",     # Data sensitivity level
    "input_tokens": 1820,
    "output_tokens": 640,
    "accepted": True,                      # Whether the output was accepted
    "edit_ratio": 0.23,                    # Proportion of human edits after acceptance
    "verified_by": "unit_test_passed",     # Verification method: unit test / manual review / unverified
    "model_version": "general-v3",         # Model version, used for attribution
    "latency_ms": 4200,
    "cost_units": 0.031,
}

On this basis, define the following metrics:

1. Penetration rate: The proportion of researchers with AI interaction records within a given period. It answers the question of breadth.
2. Task depth: The proportion of multi-turn sessions that span more than three steps. It distinguishes “looking up information” from “completing an analysis subtask.”
3. Acceptance rate and edit rate: Acceptance rate reflects usefulness, while edit rate reflects “deliverable readiness.” If edit rate remains above 50% for a long time, the prompting method or model choice is likely mismatched with the task.
4. Verification pass rate: The proportion of AI outputs that pass unit tests, reproducible scripts, or comparison with human baselines. This is the metric that should be assessed most seriously.
5. Reproduction rate: Whether others can reproduce the same conclusion from archived artifacts without additional verbal explanation.

A practical judgment criterion: if a team has a high acceptance rate but a low verification pass rate, AI is likely generating things that look plausible. This is exactly the highest-risk state.

IV. High-Value Scenario 1: Reliable Implementation in Data Analysis

Data analysis is the area most prone to problems because models can generate code that is syntactically correct, logically plausible, but statistically wrong. The reliable approach is to put the model in the position of generator, while giving verification authority to an engineered validation pipeline.

Recommended four-stage workflow: clarify data contracts → model generates analysis script → automated validation → human interpretation and sign-off.

# Example: analysis scripts produced by a model must pass four validation checks
def validate_analysis(script_path: str, snapshot_id: str):
    assert_path_whitelist(script_path, allow_prefixes=["analysis/"])
    assert_no_network_calls(script_path)      # Static scan: no external calls or implicit data fetching
    assert_deterministic(script_path, snapshot_id, runs=3)   # Results must be identical across three runs
    assert_stats_match_reference(script_path, snapshot_id)   # Compare key statistics with a human baseline
    return True

Several practical rules:

• Establish data contracts first: Ask the model to output a “data dictionary + field semantics + missing-value assumptions.” After a person confirms it, write the analysis code. This step removes most definitional ambiguities.
• Do not modify raw data directly: Scripts generated by the model may only read snapshots and write new artifacts; raw data remains immutable.
• Statistical methods must be declared explicitly: Require the model to state the assumptions of each test it uses (normality, independence, multiple-comparison correction) and provide executable assumption-checking code, rather than only conclusions.
• Raise an error when uncertain: In prompts, require “when data does not satisfy the assumptions, stop and output diagnostic information,” instead of forcing a p-value.

A practical prompt skeleton:

Role: Senior statistical analyst.
Inputs: Data dictionary (below), analysis objective (below).
Constraints:
1. Use only declared fields; do not introduce external data.
2. Every statistical conclusion must include its assumptions and verification code.
3. If assumptions are not met, output diagnostics and possible alternatives; do not give a conclusion.
4. Output a directly executable script with paragraph-by-paragraph comments.
Data dictionary: <paste>
Analysis objective: <paste>

V. High-Value Scenario 2: Writing and Text Editing

The benefit of writing tasks comes mainly from lowering startup cost and enabling cross-language conversion, not from replacing judgment. The effective division is: give structural, formatting, and language-level work to the model; conclusions, data, and attribution of contributions must remain human responsibilities.

Available sub-scenarios include: organizing scattered experimental result tables into a first draft of a methods description; adjusting chapter structure and figure/table numbering according to a target journal template; breaking down and restructuring long sentences written in Chinese-influenced English; mapping reviewer comments one by one into a response checklist; and splitting project task descriptions into milestones and acceptance criteria.

Scenarios that must be prohibited or require mandatory review include: literature citations (models are highly likely to fabricate nonexistent references), clinical or safety-related conclusions, text involving subject information, and any case where model output is directly submitted as a final conclusion. Citations must be traced back to the source text one by one; there is no compromise on this point.

VI. Risk and Governance Checklist

1. Hallucination governance: All factual outputs must carry traceable sources. Conclusions without sources should be marked as “to be verified.” Citation tasks require mandatory manual verification.
2. Reproducibility: Archive the “model version + prompt template version + data snapshot ID + random seed + artifacts,” forming a reproducibility manifest that can be published with the paper appendix or code repository.
3. Data compliance: Route by sensitivity level. When personal information or controlled data is involved, use private deployment or dedicated instances, prohibit entry into public inference pipelines, and enable encryption in transit and at rest, key management, and least-privilege access.
4. Academic norms: Follow the AI-use disclosure requirements of the institution and publisher, and state the scope of use in methods or acknowledgments. AI cannot be an author.
5. Quality gatekeeping: High-impact outputs require dual review, with review records entered into audit logs.

VII. General Technical Architecture

AI implementation in research scenarios does not require a complex architecture. A general set of cloud components is sufficient, and equivalent capabilities from vendors map to the same concepts:

• Access layer: A unified inference gateway responsible for identity authentication, quota and rate limiting, model routing, and project-based cost aggregation.
• Capability layer: Document parsing (PDF/tables/charts), retrieval augmentation (local paper libraries and experimental records), a code interpreter sandbox with restricted network and file-system permissions, and task orchestration.
• Data layer: Object storage for raw data snapshots and archived artifacts; relational databases for metadata and experimental data ledgers; message queues to decouple long-running tasks (batch analysis, batch polishing); and vector retrieval to support semantic recall of literature and experimental records.
• Governance layer: Log services for centralized collection of call and audit records; monitoring and alerts covering latency, failure rate, cost anomalies, and outbound transmission of sensitive data; and centralized configuration and key management.

On Baidu AI Cloud, these general components can be assembled in the same layered way, while sensitive-data scenarios can use dedicated resources and internal network access policies to keep data within a controlled environment.

VIII. Maturity Evolution Path

• L0 Observation: Individual users adopt tools in an ad hoc manner, with no instrumentation and no norms.
• L1 Point-level efficiency gains: Habits form around low-risk tasks such as writing polish and code snippet generation. Benefits are visible but not measurable.
• L2 Pipeline embedding: Data analysis enters a “generation–validation–archiving” pipeline, metrics can be collected, and verification pass rate becomes an assessment item.
• L3 Research process restructuring: AI takes on an orchestration role in literature screening, structuring experimental records, and batch parameter scanning, while personnel shift toward hypothesis design and result interpretation. At this stage, reproducibility and verification pass rates remain high simultaneously.

Most teams are currently between L1 and L2. The key to crossing this threshold is not switching to a stronger model, but making verification infrastructure. Executable tests, immutable data snapshots, and traceable artifact archiving are the three essentials. Once these are done well, every improvement in model capability will be directly converted into research output. Otherwise, even a stronger model will only amplify unverifiable outputs.