LLM Use in Research: Data Analysis and Writing Are the Main Use Cases

1. Research Background and Method: Characterizing Research Use Cases with Real Conversation Data

The most reliable way to understand AI’s actual role in research is not a questionnaire survey, but analysis of large-scale real usage data. A recent study from an AI research organization was based on hundreds of thousands of research-related conversations.

1.1 Data Scale and Privacy Protection

The study analyzed more than 200,000 research conversations over roughly two months, covering both general chat assistant interactions and coding assistant conversations for programming scenarios. Its privacy design is particularly noteworthy:

• The system retained only structured labels such as discipline category, research subfield, and task type for each conversation;
• Specific research data, hypotheses, experimental methods, preliminary findings, and unpublished papers were not extracted or stored;
• All analysis results were presented in aggregated form.

This approach offers a useful model for the industry: in LLM usage analysis, “metadata available, content invisible” is a feasible paradigm that balances insight and compliance. Teams planning to deploy LLM capabilities in research or enterprise settings can learn from this minimized data collection approach.

1.2 Two-Dimensional Validation: Behavioral Data Plus Survey

The study not only analyzed real conversations but also surveyed more than 500 scientists who actively use AI, mainly from the life sciences and health sciences. The survey results were broadly consistent with the conversation data. This “behavior plus attitude” validation significantly improves confidence in the conclusions.

2. Core Findings: Task Structure Is Heavily Skewed Toward Analysis and Expression

2.1 Data Analysis: The Clear Usage Leader

Data analysis accounted for the following shares in the two types of conversations:

Task Type	Share of General Chat Sessions	Share of Coding Assistant Sessions	Share of Researchers’ Actual Work Hours
Data analysis	33.7%	48.1%	27.6%
Writing and communication	32%	—	16%
Data collection	11.5%	11.9%	28.6%
Ideation and hypothesis generation	4.2%	2.1%	7.1%

Two clear cases of “overuse” and two cases of “underuse” emerge:

• Overuse: data analysis (33.7% vs. 27.6%) and writing (32% vs. 16%). Writing has the highest overuse ratio, indicating that researchers already recognize the value of LLMs in text generation, polishing, and structured expression.
• Underuse: data collection accounts for nearly 30% of researchers’ work hours but only about 10% of AI sessions. Hypothesis generation is used at a rate far below its share of work hours.

2.2 Why Is Writing the Most Popular Scenario?

From a technical perspective, this result matches the current capability boundaries of LLMs:

1. Text generation is a native strength of LLMs. Tasks such as rewriting paper abstracts, responding to reviewer comments, and drafting methodology descriptions are essentially “input context → output structured text,” which aligns closely with model training objectives.
2. Data analysis has a clear feedback loop. Researchers can immediately verify whether a piece of analysis code or a statistical result is correct, and the cost of errors is manageable. For example, a model can help write a data cleaning script:

import pandas as pd
def clean_experiment_data(df: pd.DataFrame) -> pd.DataFrame:
    # LLMs can help generate template code like this; researchers verify the logic
    df = df.dropna(subset=["subject_id", "reaction_time"])
    df = df[df["reaction_time"].between(100, 3000)]  # Filter outliers
    return df.groupby("condition")["reaction_time"].agg(["mean", "std"])

Researchers only need to review the filtering thresholds and statistical definitions. The verification cost is far lower than validating a scientific hypothesis.

3. Discipline Differences: Quantitative Fields Show Significantly Higher Penetration

3.1 Computer Science, Mathematics, and Statistics Lead

Breaking down conversation data by discipline reveals clear differences:

• Computer science had the highest share of general research conversations (11.9%), followed by engineering (8.4%) and medicine (7.7%);
• More importantly, the normalized results show that computer science researchers account for only 4.3% of the global researcher population but contributed 32.7% of coding assistant research conversations, a penetration rate nearly 8 times higher;
• Researchers in AI and machine learning account for only 1.2% of the total but represent 5% of general conversations and 15.2% of coding assistant conversations.

3.2 Three Explanations for the Differences

The researchers argue that discipline-level usage differences are not only related to population size but are closely tied to whether research tasks are suitable for AI. This can be summarized in three points:

1. The degree of task digitalization. Computer science inputs and outputs are naturally code and text, which fit seamlessly with model capabilities. In contrast, physical sciences require laboratory work, and much of the work happens in the physical world, where AI is difficult to involve.
2. Researchers’ tool proficiency. Researchers in quantitative fields are more familiar with programming and automated workflows, and can more quickly integrate models into existing toolchains, such as building reproducible analysis pipelines with container platforms, logging services, monitoring, and alerting infrastructure.
3. Verification cost. Outputs that can be verified quickly, such as code, statistical results, and draft text, are more likely to be trusted. For hypotheses that require experimental validation, the payoff period for AI involvement is longer.

By contrast, usage in art, humanities, and some social sciences is noticeably lower, consistent with the unstructured and qualitative nature of their research materials.

4. Implications for Research and Engineering Practice

4.1 Opportunity: Data Collection Is the Next Breakthrough

Data collection accounts for 28.6% of researchers’ work hours but only about 11% of AI sessions. This large gap represents a potential opportunity. Possible directions include:

• Using general information extraction capabilities to help build structured data tables from literature and public datasets;
• Combining cloud infrastructure such as object storage and message queues to build automated data aggregation pipelines;
• Lowering the development barrier for data organization scripts through natural language interaction.

4.2 Boundary: Hypothesis Generation Should Not Be Over-Relied On

Ideation and hypothesis generation account for less than 5% of usage, which is not necessarily a bad thing. Scientific hypotheses need to be built on domain intuition, literature accumulation, and experimental constraints. Current models are better at helping design validation plans and simulate statistical power after a hypothesis is given, rather than replacing researchers in posing original questions.

4.3 Recommendations for Research Teams

• Start with tasks that have high verification efficiency: prioritize code assistance, data analysis, and writing polishing scenarios and establish usage guidelines;
• Establish human review mechanisms: AI-generated analysis code must undergo definition review and sample validation;
• Pay attention to privacy and compliance: when unpublished data is involved, adopt de-identification and minimized collection strategies similar to those in this study.

5. Conclusion

This large-scale conversation analysis sends a clear signal: the current value of LLMs in research is mainly concentrated at the “accelerating execution” level—analyzing data and expressing results—rather than “replacing thinking.” For research institutions, the rational approach is to treat AI as a highly efficient execution component in the research pipeline, while building adequate supporting infrastructure for data, review processes, and privacy compliance to truly amplify its value.