Research AI Usage Profile: Data Analysis and Writing Dominate

I. What Is Being Measured: Define the Scope First

For engineering teams, the most valuable part of this study is not any single finding but the design of its measurement scope. The scope sets the credibility limits of the conclusions. It also determines whether the numbers can be recalculated, reused and analyzed again.

The study covers about two months and 207,000 research sessions, split into two categories:

• Conversational and collaborative workbench sessions: 129,000 sessions, about 62%. Interaction is mostly natural-language Q&A, long-document reading and writing, and multi-turn refinement.
• Command-line (CLI) agent sessions: 78,000 sessions, about 38%. Interaction is mostly repository-level context, multi-step tool calls, and direct edits to executable files.

Throughout this article, “session share” means the share of a given task or discipline within all sessions of the corresponding interaction type, unless stated otherwise:

• When discussing the conversational assistant, the denominator is all conversational and collaborative workbench sessions.
• When discussing CLI agents, the denominator is all CLI-agent sessions.

The 62% and 38% figures in the original report are rounded for display. To recalculate them precisely, go back to the raw counts:

• 129,000 / 207,000 ≈ 62.3%
• 78,000 / 207,000 ≈ 37.7%

Counting the two session types together in a single statistic would badly distort the picture:

• Conversational sessions are closer to a cognitive assistance need.
• CLI-agent sessions are closer to an engineering execution need.

Only by keeping them apart can you see how task distribution differs across interaction paradigms.

For privacy measurement, the study keeps only three categorical labels: discipline, research subfield and task type. It does not extract or store substantive content such as research data, hypotheses, experimental methods, preliminary findings or unpublished papers. This is a typical metadata allowlist approach: protect the content layer first, and analyze only behavioral structure.

In engineering terms, this scope has to be enforced in the schema and the gateway layer, not just stated in a privacy policy. A minimal event structure can be abstracted as:

from dataclasses import dataclass
from typing import Literal
@dataclass(frozen=True)
class ResearchInteractionEvent:
    session_type: Literal["chat", "agent_coding"]
    discipline: str
    subfield: str
    task_label: str
    turn_index: int
    # Explicitly excluded: prompt text, code content, data samples, file paths,
    # dataset names, institution identifiers, device identifiers, and similar

When platforms reuse these findings, they should add three verifiable details:

• How the session sample was selected.
• Where the task label taxonomy came from.
• How the exclusion list was audited.

If only aggregate results are kept, at minimum record the denominator, the time window and the rounding rule. Otherwise, the numbers are easy to misread.

II. Don’t Be Misled by Session Shares: Normalization Is the Starting Point

Raw shares are only a directional signal. What actually explains usage is use concentration, which is a domain’s share of sessions divided by its share of researchers:

concentration = domain session share / domain researcher headcount share

The domain session share comes from the publicly reported aggregate data. The domain researcher headcount share uses the researcher headcount basis given in the report.

• The observation window is about two months.
• Percentages are computed as decimals, then rounded to one decimal place for display.

To keep the figures consistent, this article uses 11.7% throughout as the computer science session share within conversational and collaborative workbench sessions. No other full-population aggregate value is mixed in.

Here is the runnable code:

from dataclasses import dataclass
@dataclass(frozen=True)
class DomainShare:
    name: str
    session_share: float
    headcount_share: float
def concentration(s: DomainShare) -> float:
    if s.headcount_share <= 0:
        raise ValueError("headcount_share must be positive")
    return round(s.session_share / s.headcount_share, 1)
samples = [
    DomainShare("Computer Science (chat/collaborative)", 0.117, 0.043),
    DomainShare("AI and Machine Learning (chat/collaborative)", 0.050, 0.012),
    DomainShare("AI and Machine Learning (CLI agents)", 0.152, 0.012),
    DomainShare("Computer Science (CLI agents)", 0.327, 0.043),
]
for s in samples:
    print(s.name, concentration(s))

The corresponding results are:

1. Computer Science (conversational/collaborative): about 2.7x (11.7% / 4.3% ≈ 2.7). The numerator is the computer science session share within conversational and collaborative workbench sessions. The denominator is the share of computer science researchers among all researchers worldwide.
2. AI and Machine Learning (conversational/collaborative): about 4.2x (5.0% / 1.2% ≈ 4.2).
3. AI and Machine Learning (CLI agents): about 12.7x (15.2% / 1.2% ≈ 12.7).
4. Computer Science (CLI agents): about 7.6x (32.7% / 4.3% ≈ 7.6).

The most informative takeaway is not that “AI researchers like to use AI.” It is this: the more researchers work with code, experiment scripts and data processing as their main output, the more likely they are to move a tool from “answering questions” to “editing repositories, running pipelines and producing executable results.”

On the other side, usage shares are lower in the following fields:

• Healthcare and medicine.
• Physics that depends on experimental apparatus.
• Arts and humanities.
• Some social sciences.

A low share does not necessarily mean low value. These tasks are not easily carried end to end by text-and-code tools:

• Lab procedures.
• Ethics review.
• Field interviews.
• Peer review of materials or specimens.

If platforms look only at shares, they are likely to misjudge demand. They should also examine task fit, access to entry points and verification mechanisms.

III. Task Structure: Data Analysis and Writing Consume Most Calls

Task distribution is even more concentrated than discipline distribution. The publicly reported aggregate data shows:

Task type	Share of conversational/collaborative sessions	Share of CLI-agent sessions
Data analysis	33.7%	48.1%
Writing and communication	32.0%	No exact figure in the original report; qualitatively lower than in conversational sessions
Other tasks (literature, methods, figures, etc.)	About 36.3%, back-calculated from the two disclosed figures	Not broken down with precise values in the original report

At the same time, researchers report spending about 27.6% of their working time on data analysis. Put the 27.6% time investment next to the 33.7%–48.1% share of tool calls, and you can draw a fairly solid conclusion: data analysis is currently one of the AI entry points with the highest return on investment and the relatively clearest validation path.

Why do data analysis and writing tasks come out ahead? There are three structural reasons:

1. Outputs are verifiable. Analysis scripts can be run, and results can be checked against known statistics. Code can run unit tests. By contrast, whether a generated research hypothesis is correct often cannot be known until the experiment is finished, which takes months of feedback time.
2. Context boundaries are clear. Dataset schemas, code repositories and manuscript sections usually have clear boundaries. That makes them easy to slice, search and rebuild context from. Tacit domain intuition and unspoken experimental constraints are much harder to encode.
3. The cost of errors is relatively manageable. A wrong analysis definition can be recalculated, and a wrong wording polish can be revised. By contrast, a mistake in experimental design or a core hypothesis can contaminate the entire research chain.

This also explains why “data collection and hypothesis generation” tasks have low usage. It is not that they are unneeded, but that they lack reliable validation and constraint mechanisms. This is a capability gap rather than purely a demand gap.

IV. Task-to-Tool Fit Matrix: Turning Findings into Engineering Decisions

Crossing the discipline side with the task side gives you a more actionable decision framework.

Task type	Best-fitting interaction mode	Key constraints	Engineering focus	Suggested acceptance criteria
Statistics and data cleaning	Either; agents preferred	Consistent metric definitions, reproducibility	Sandboxed execution, fixed environment, locked dependencies	Outputs can be recalculated; result snapshots are consistent
Modeling and hyperparameter tuning	CLI agents	Data must not leave the environment	Private inference, field allowlist, masked inputs	No sensitive field leakage; parameter traces are auditable
Paper/report writing	Conversational	Factual consistency, citation standards	Retrieval augmentation, citation traceability, version control	Citations can be verified; key facts are sourced
Code implementation and refactoring	CLI agents	Regression risk	Unit test gates, diff review, branch protection	Tests pass; regression defect rate is under control
Literature review	Conversational	Coverage and timeliness	Retrieval quality, freshness, duplicate-rate control	Coverage and timeliness can be assessed
Hypothesis generation / experimental design	Weak in both modes today	High verification cost	Structured prompt templates, human review, low-coverage eval slices	Not aimed at automatic pass; emphasize traceability and reviewability

Hypothesis generation and experimental design should not be treated simply as high-value automated capabilities. A more reliable approach is to use structured templates to collect research objectives, boundary conditions, existing evidence and risk assumptions, then have domain experts review them manually. Platforms can offer this as an observational feature rather than a tool that produces final research conclusions.

V. Practical Advice for Platforms and Engineering Teams

What platforms really need is not a better single-point model score, but a system that connects task labels, evaluation, privacy, capacity and execution. The table below is organized by action, input, output and acceptance metric.

Action	Inputs	Outputs	Acceptance metrics
1. Task routing and model tiering	Task labels, session type, context length, tool requirements	Routing policy and model specs	Latency, success rate, cost per task
2. Verifiability-first capability building	Analysis scripts, code changes, result snapshots	Execution results, test reports, change explanations	Run-through rate, regression rate, reproduction rate
3. Eval sets weighted by task mix	Real task label distribution	Task-slice eval sets	Consistent with real distribution; low-coverage slices are observable
4. Privacy compliance moved upstream to the gateway	Telemetry fields, prompts, attachments, regional requirements	Allowlisted events, audit logs, interception records	No unauthorized field collection; audits are traceable
5. Capacity and cost designed for peaks	Historical windows, submission deadlines, academic calendars	Quota and elastic scaling policy	No degradation at peak; cost under control
6. Feeding usage data back into the product	Low-usage disciplines, task entry points, failure samples	New entry points, templates and tools	Entry point usage up; task completion rate up

The details:

1. Route by task label, not by user identity. Short Q&A can go to a lightweight model. Long-context analysis, code changes and repository-level understanding should go to high-spec models, with context compression, retrieval augmentation and result summarization. Routing should be based on task type, context size, tool requirements and risk level.

2. Make verifiability a delivery gate. For code and analysis tasks, the following should be standard steps:

   • Sandboxed execution.
   • Unit tests.
   • Dependency locking.
   • Result snapshot comparison.

   A model output counts as delivered only when it runs and can be reproduced. Generating a piece of text does not count.

3. Design eval sets around the real task distribution. If data analysis and writing tasks make up a large share, give them more weight in eval sets. At the same time, set up separate low-coverage slices for tasks such as hypothesis generation, experimental design and literature coverage. Use them to observe trends and control risk, not to boost the overall score.

4. Move privacy compliance upstream to the access layer. Implement the following uniformly at the gateway:

   • Telemetry field allowlists.
   • Sensitive-information interception for prompts and attachments.
   • Regionalized data residency.
   • Auditable operation logs.

   In a generic cloud environment, for example:

   • Object storage can retain audit records.
   • A logging service can collect and search logs.
   • Monitoring and alerting can cover abnormal call volumes.
   • Key management can protect credentials.

   These are generic capability examples only, not an official certification. Actual compliance must be assessed against your own enterprise architecture.

5. Plan capacity and cost around peak usage within the observation window. Research usage has clear seasonality, such as submission deadlines, grant application periods and academic calendars. Quotas and elastic scaling should not be based on daily averages alone. Design for peak usage within the window so that the experience does not degrade during busy periods.

6. Low-usage disciplines do not mean low demand. In many cases, low usage comes from missing entry points rather than insufficient real demand. For lab scenarios, adding non-text entry points is more effective than piling on more text-task features, for example:

   • Adapters for instrument data formats.
   • Structured experiment record templates.
   • Experimental plan review templates.

VI. Risk Boundaries: Capabilities That Should Not Be Overestimated

Even if data analysis and writing dominate usage today, platforms should not ignore these risks:

• Hallucination and factuality risk: Writing and communication make up a large share of usage. Once generated content goes into manuscripts, reports or grant applications, citation accuracy must be manually verified.
• Data leakage risk: Research data often contains unpublished results, restricted samples, commercially partnered data or ethics approval materials. Uploading raw sensitive data should be strictly prohibited. Only masked structural descriptions or field descriptions should be allowed.
• Reproducibility risk: If generated analysis scripts depend on unpinned environment versions, random seeds or package versions, the results may not be reproducible. Make environment locking and result snapshots part of the delivery requirements.
• Publication ethics and authorship risk: AI involvement disclosure requirements are tightening. Tools should support exporting a “usage summary,” so researchers can use it for methods statements, acknowledgments or compliance disclosure.
• Capability misjudgment risk: High-concentration fields are prone to a “the more capable the tool, the more it should be used” feedback loop. This can lead to overestimating AI’s contribution to methodological innovation, hypothesis generation and experimental design.

VII. Conclusion and Common Questions

The value of this study is that it turns “how AI is used in research” from an impression-based judgment into a quantifiable distribution problem. It yields three transferable engineering conclusions:

1. Normalize first, then look at shares. Session shares alone lead to misjudgment. Concentration better reflects real usage intensity.
2. Break down task distribution by interaction type. Conversational assistants and CLI agents have different task structures, so they should not be lumped together.
3. Use verifiability to explain task priorities. Data analysis, code implementation and writing polish dominate because they are easier to verify, easier to keep errors under control and easier to close the loop.

Common questions can be answered as follows:

• How much of research AI usage is data analysis? In this sample, data analysis accounts for 33.7% of conversational/collaborative sessions and 48.1% of CLI-agent sessions.
• What is the difference between CLI agents and conversational assistants? CLI agents lean more toward repository context, multi-step tool calls and edits to executable files. Conversational assistants lean more toward natural-language Q&A, document reading and writing, and multi-turn refinement.
• Does low usage in a discipline mean low demand? Not necessarily. Low usage can come from missing entry points, a mismatch between the task type and the tool, data sensitivity or high verification costs.

For platforms, the real competition is no longer a single model score. It is infrastructure such as the task label system, eval set design, the privacy compliance pipeline and execution sandboxes. The figures in this article come from the original report. Engineering conclusions should be rechecked against your own telemetry.