Deploying Medical Large Language Models: Data Foundations, RAG, and Closing the Loop

    This article provides a practical framework for moving medical large language models (LLMs) from concept to production. It outlines the essential components of a domain-specific data foundation, explains the role of retrieval-augmented generation (RAG) in grounding model outputs, and details four representative deployment scenarios across drug discovery, Traditional Chinese Medicine, clinical knowledge retrieval, and pharmaceutical quality management. The article also addresses key engineering challenges, evaluation methods, and a phased implementation roadmap for building reliable, auditable, and compliant medical AI systems.

    The healthcare and pharmaceutical industry is moving beyond proof-of-concept with large language models (LLMs) toward real-world business deployment. Unlike general internet domains, medical data is highly fragmented, semantically complex, and requires deep domain expertise. It also involves strict constraints around patient privacy, intellectual property, research compliance, and clinical safety. Even a model with strong language comprehension can produce factual errors, inaccurate citations, concept confusion, or entirely fabricated research conclusions when lacks access to reliable professional data.

    Therefore, the core of building a medical LLM is not simply procuring a model. It requires constructing a complete engineering pipeline that covers data governance, knowledge organization, retrieval augmentation, model reasoning, business validation, and continuous feedback. In this pipeline, the model handles comprehension and generation; the data foundation provides factual grounding; RAG connects questions to credible knowledge; and business systems integrate outputs into an executable workflow.

    Why Medical LLMs Depend on a Professional Data Foundation

    Medical data generally exhibits four defining characteristics.

    First, data comes from diverse sources. Research and development (R&D) data may reside in laboratory notebook systems, sample management platforms, analytical testing systems, patent databases, publication repositories, and project management tools. Clinical data may be scattered across electronic health records, laboratory results, imaging reports, follow-up notes, and medical knowledge bases. Field naming conventions, coding schemas, and data granularity vary significantly across these systems.

    Second, domain terminology is dense. A single drug may have multiple names, including a generic name, brand name, R&D code, and chemical name. Complex hierarchical relationships exist between diseases, targets, genes, proteins, pathways, and adverse reactions. Without term normalization, models may incorrectly treat distinct entities as the same object.

    Third, data freshness is critical. Drug development progress, clinical guidelines, regulatory policies, and safety information undergo continuous change. Once model training is complete, knowledge embedded in parameters does not automatically update. Relying solely on a pre-trained model to answer the latest questions is insufficient.

    Fourth, results must be explainable and traceable. In medical scenarios, conclusions require more than a generated sentence. Systems should indicate which literature, experimental records, or guideline versions support the answer, and specify whether the conclusion has undergone human review.

    In practice, the foundational engineering for medical LLMs typically includes data ingestion, cleaning and deduplication, entity recognition, relation extraction, access control, version management, quality assessment, and knowledge updates. The more solid the data foundation, the more reliable downstream components—RAG, AI agents, and model fine-tuning—will perform.

    A General Architecture for Medical Data Foundations

    A common technical architecture can be organized into five layers.

    1. Data Ingestion Layer

    This layer handles structured, semi-structured, and unstructured medical data. Structured data includes sample information, experimental indicators, test results, and project status. Unstructured data includes research papers, patents, experiment reports, clinical records, meeting materials, and policy documents.

    During ingestion, the system should preserve metadata such as source system, creation time, responsible person, data version, and access scope. For document-type data, page numbers, chapters, table positions, and attachment relationships should also be recorded to support later citation and audit trails.

    2. Data Governance Layer

    The governance layer addresses data usability through several key activities:

    • Field standardization: Unify names, units, formats, and enumerated values.
    • Entity alignment: Map drugs, diseases, targets, genes, experimental methods, and other objects to a unified entity standard.
    • Deduplication: Identify duplicate documents, repeated experiments, and multi-version materials.
    • Quality validation: Check for missing values, outliers, logical conflicts, and temporal inconsistencies.
    • Desensitization: Process patient names, contact details, identification numbers, and other sensitive fields.
    • Permission tagging: Control access based on department, project, role, and data classification level.

    Governance is not a one-time task. New projects, terminologies, and data sources are continuously added, so organizations must establish an ongoing maintenance mechanism.

    3. Knowledge Organization Layer

    This layer typically combines a document store, a structured database, and a knowledge graph.

    The document store is suitable for preserving complete content such as papers, patents, guidelines, and experiment reports. The structured database supports queries on samples, indicators, batches, and project status. The knowledge graph expresses multi-hop relationships such as “drug–target–pathway–disease” and “syndrome–formula–herb–compound”.

    For example, R&D knowledge can be structured as follows:

    1. Compound A
    2. ├── Acts on: Target B
    3. ├── Affects: Signaling pathway C
    4. ├── Associated disease: Disease D
    5. ├── Experimental evidence: Report E
    6. └── Data version: 2025-version X

    This approach helps the model recover context from scattered text and supports relationship queries, evidence tracking, and result explanation.

    4. Retrieval-Augmented Layer

    The basic RAG workflow proceeds as follows: after a user asks a question, the system converts the input into a search query, retrieves relevant content from the professional knowledge base, and applies ranking, filtering, and permission checks before passing the evidence segments to the model for answer generation.

    Relying solely on vector search is not recommended. A hybrid retrieval strategy is more robust:

    1. Keyword search for exact matches on drug codes, gene names, and experiment IDs.
    2. Semantic search for natural language expressions and synonymous questions.
    3. Structured queries for filtering by time, project, metric, and numeric range.
    4. Graph retrieval for entity relations and multi-hop reasoning.
    5. Re-ranking mechanisms to improve evidence relevance.

    Results should also undergo source, time, permission, and credibility filtering. High-authority resources such as clinical guidelines and regulatory documents should be prioritized. Unreviewed internal drafts should be clearly flagged with their reference level.

    5. Model and Application Layer

    The model layer can include general-purpose LLMs, domain-specific models, classification models, extraction models, and prediction models. Different tasks do not all need to be handled by one generative model.

    For instance, literature summarization can be handled by a generative model; drug name recognition is better suited to an entity extraction model; abnormal result detection can combine rules and statistical models; and risk alerts should involve both model output and human review.

    The application layer provides interfaces such as Q&A assistants, R&D agents, literature analysis tools, project management assistants, and clinical knowledge search portals, embedding these capabilities into daily workflows.

    Scenario 1: AI-Assisted Drug Discovery and Project Initiation

    In early-stage drug discovery, the core problem is rarely a lack of information. It is fragmented information, difficult correlation, and lengthy decision cycles. Researchers must simultaneously analyze papers, patents, experimental results, competitor progress, target information, and safety data.

    A complete R&D intelligence system can be organized into four capability modules:

    First, R&D intelligence aggregation. The system searches relevant materials by disease, target, mechanism of action, or compound characteristics, then generates structured summaries with citations.

    Second, patent and competitive landscape analysis. The system identifies compounds, claims, indications, and filing dates in patents to help researchers evaluate technical pathways and intellectual property risks. Original evidence must be preserved; models should not replace professional legal judgment.

    Third, candidate compound analysis. Based on existing experimental data, the system organizes information on activity, selectivity, solubility, stability, metabolism, and safety, while highlighting data gaps.

    Fourth, project decision support. Management can view project stages, key milestones, experiment completion status, failure reasons, and pending hypotheses, reducing information lag caused by manual reporting.

    In this scenario, the AI agent should be positioned as an information organization and analysis assistant—not an “automated decision-maker” for R&D direction. For critical conclusions, the system should display evidence sources, reasoning processes, confidence levels, and items requiring confirmation.

    Scenario 2: Multi-Modal Traditional Chinese Medicine Knowledge and Intelligent Formulation

    Traditional Chinese Medicine (TCM) data has unique multi-modal characteristics. Ancient texts and medical records are primarily text-based; tongue images, pulse patterns, and herb morphology involve visual information; while formula composition and chemical components require structured data correlation.

    To build such a system, organizations can develop a heterogeneous knowledge network around “syndrome–formula–herb–component–target–pathway–disease.” Text models parse ancient texts and medical cases; vision models recognize herb images or tongue features; structured modules manage dosages, processing methods, compatibility relationships, and contraindications.

    An intelligent formulation workflow can follow these steps:

    1. Input patient symptoms, syndrome characteristics, and baseline information.
    2. Retrieve similar cases, classic formulas, and modern research.
    3. Generate candidate formulas based on syndrome–herb property relationships.
    4. Check for contraindications, dosage limits, and compatibility risks.
    5. Output formulation rationale, reference sources, and items requiring human confirmation.

    It must be stressed that intelligent formulation is a support tool only. It cannot replace a licensed practitioner’s syndrome differentiation, review, and prescribing responsibility. System outputs should avoid absolute statements and clearly distinguish classical records, modern research, and model inference.

    Scenario 3: Clinical Knowledge Retrieval and Medical Document Assistance

    The main value in clinical settings is reducing the burden of information retrieval and document organization. For example, healthcare professionals may need to query treatment pathways for a disease, medication precautions, interpretation of test indicators, or follow-up recommendations.

    A qualified clinical Q&A system should support at least the following capabilities:

    • Identify disease, drug, patient characteristics, and time range from the question;
    • Prioritize retrieval from reviewed guidelines, institutional policies, and professional resources;
    • Sort content by version and time;
    • Provide sources and scope of applicability with answers;
    • Explicitly state “cannot confirm” for questions with insufficient evidence;
    • Route high-risk questions to human review.

    For medical records and documentation assistance, models can help with information summarization, formatting, terminology normalization, and preliminary summarization. However, models should not add facts not provided by the patient. The system should log generated content, modifications, and the final approver to create a complete audit trail.

    Scenario 4: Pharmaceutical Manufacturing and Quality Management

    Data in manufacturing quality scenarios typically includes batch records, test results, deviation investigations, change controls, equipment maintenance logs, and quality events. LLMs can be used for knowledge retrieval, deviation attribution support, document comparison, and drafting quality reports.

    For example, when a batch deviation occurs, the system can retrieve historical similar events based on material, equipment, personnel, environment, and process parameters. It can list potential contributing factors and generate an investigation checklist. However, final root cause determination must remain with quality personnel who review on-site records and experimental results.

    In document management, RAG helps compare different versions of standard operating procedures (SOPs), identifying changes in clauses, scope, and approval status. For regulatory and quality documents, version control must be strictly enforced to prevent mixing old and new content in a single response.

    Key Engineering Challenges in RAG Deployment

    Document Chunking Should Not Be Overly Mechanical

    If chunks are too large, retrieval results contain large amounts of irrelevant content. If chunks are too small, context is lost. Chunking should be based on heading structure, paragraph semantics, table structure, and chapter relationships, while preserving document ID, page number, and parent–child paragraph relationships.

    Retrieval Results Must Be Verifiable

    The system should not simply return plausible text. It must present cited sources. For values, dosages, times, and experimental conclusions, original references should be preserved.

    Distinguish Between “Not Retrieved” and “Does Not Exist”

    The absence of content in the knowledge base does not indicate that a fact does not exist in reality. The system should use cautious phrasing such as “the current knowledge base lacks sufficient evidence” to avoid incorrectly interpreting retrieval failure as a negative conclusion.

    High-Risk Answers Require Rule-Based Safeguards

    For questions involving medication contraindications, patient privacy, clinical diagnosis, experimental safety, and regulatory judgment, the system should include sensitive word recognition, access controls, answer templates, human review, and audit logging mechanisms.

    How to Evaluate Whether a Medical LLM Is Effective

    Evaluation should not rely only on response fluency. A multi-dimensional indicator system is needed.

    At the data level, assess completeness, accuracy, duplication rate, update frequency, and permission coverage. At the retrieval level, measure recall, ranking quality, citation accuracy, and evidence coverage. At the generation level, evaluate factual consistency, hallucination rate, terminology accuracy, and refusal appropriateness. At the business level, track retrieval time, document processing efficiency, R&D analysis cycle, and manual revision rate.

    Test sets should cover normal questions, complex questions, boundary cases, and adversarial input. Special attention should be paid to creating a test set where the answer is absent from the knowledge base, verifying that the system can safely refuse to answer rather than generate fabricated conclusions.

    Medical LLM adoption should not aim for full scenario coverage from the start. A phased approach is more practical.

    Phase 1: Select a scenario with clear boundaries and relatively complete data, such as internal literature search or quality document Q&A. Validate data governance and RAG effectiveness first.

    Phase 2: Add structured query capabilities, knowledge graphs, and permission systems to evolve from “can answer” to “answers with traceability.”

    Phase 3: Introduce AI agents and workflow orchestration so the system can handle retrieval, analysis, report generation, and task routing. Human confirmation should be retained at every high-risk checkpoint.

    Phase 4: Establish a continuous feedback loop. Collect user acceptance, modification, rejection, and correction records to optimize the knowledge base, retrieval strategies, and prompt templates.

    Conclusion

    Deploying medical LLMs is fundamentally a collaborative effort across data engineering, knowledge engineering, model engineering, and business process engineering. LLMs can improve comprehension, summarization, and collaboration efficiency, but they cannot replace specialized data, experimental validation, clinical judgment, or compliance review.

    A sustainable solution should be built on a high-quality data foundation, enhanced with RAG and knowledge graphs for factual reliability, connected to business processes through AI agents, and safeguarded by access controls, audit trails, and human review. Only when model outputs can be verified, traced, and integrated into real workflows does medical artificial intelligence truly move from demonstrating capability to creating practical value.