Making LLMs Read NMR Spectra: The Engineering Path for Chemistry Agents

1. First, Understand the Problem: Why Structure Elucidation Is an “Efficiency Black Hole” in the Lab

Anyone who has done organic synthesis has a muscle-memory fatigue about one thing: once a new spectrum arrives, the next two or three hours are basically not creative work.

The time is actually spent on three kinds of tasks: mapping each peak in a pile of instrument output to an atom in the structure; checking whether integration ratios, splitting patterns, and coupling constants are self-consistent; and writing those conclusions into a fixed-format assignment description to paste into a lab notebook or report. For a natural product of moderate complexity, having the whole process consume a full day is not an exaggeration.

The reason this is hard, fundamentally, is that it is an information-incomplete inverse problem. A spectrum gives the response of atomic nuclei in a specific chemical environment, while the structure determines that environment—inferring the structure from the response is mathematically underdetermined: different structures can give highly similar spectra, and noise, overlapping peaks, solvent residues, impurities, spinning sidebands, and carbon satellites further erode solvability. The human brain manages to solve it by relying on “empirical priors” accumulated over decades: which shift ranges correspond to which functional groups, and which cross-peak patterns a 2D correlation should show. Such priors are hard to write out explicitly and hard to learn from a small number of samples.

The conclusion is direct: this is not a general capability problem that a “bigger model, longer context” can quietly solve, but a scientific-reasoning problem that needs external tools and verifiable processes to backstop it.

2. Why the Past Few Rounds of “AI Spectrum-Solving” Failed Together

Reviewing the attempts of recent years, the failure modes are highly consistent and worth breaking down one by one, because each corresponds to a design constraint.

First, treating the inverse problem as a classification problem. The mainstream approach is to have the model memorize the “structure → spectrum” mapping and then emit it in reverse. On the skeletons covered by the training set it looks fine; the moment it meets a new molecule outside those skeletons, the model does not admit it doesn’t know—it produces an answer that “looks quite similar.” The cost of errors in a professional field is asymmetric: if a single atom is assigned wrongly, the entire structural conclusion is void.

Second, lacking an inspectable intermediate representation. Emitting the final SMILES or the final assignment table directly means that when something goes wrong, you cannot locate which step is at fault—was peak extraction wrong? Was candidate-structure generation wrong? Was 2D correlation matching wrong? Without intermediate artifacts, experts can only recheck from scratch, and the efficiency gain drops to zero.

Third, the toolchain is disconnected from the lab. Requiring users to export a specific format, install plugins, and learn a proprietary syntax yields a result that isn’t even faster than doing it by hand. In a real workflow, the input is the PDF and images exported by instrument software, or raw data files; the output requirement is a piece of text that can be pasted straight into the lab notebook, plus an assignment-annotated figure.

Fourth, always giving a single answer. A system that does not express uncertainty is almost unusable in professional settings, because it deprives the expert of the ability to “quickly judge whether this one deserves a closer look.”

3. The Viable Engineering Paradigm: Replace “Guess” with “Guess + Verify”

The core idea is a single sentence: don’t trust the model’s memory; trust the model’s ability to call tools and complete verification.

The forward problem (given a structure, predict the spectrum) is computable, with mature algorithms and toolchains; the inverse problem is unsolvable, but it can be transformed into a search process of “generate several candidates → verify each forward → rank by fit.” This transformation is the premise on which the whole solution stands.

A deployable closed loop looks like this (illustrative pseudocode; function names are placeholders representing general capabilities you need to wire in):

def elucidate(peak_table, sample_meta):
    candidates = generate_candidates(  # molecular formula, degree of unsaturation, functional group hints
        formula=peak_table.molecular_formula,
        dof=peak_table.degree_of_unsaturation,
        hints=peak_table.functional_group_hints,
    )
    candidates = [sanitize(c) for c in candidates]  # valence / aromaticity / charge legality
    candidates = dedupe_by_canonical_form(candidates)  # canonical deduplication to remove tautomer ambiguity
    scored = []
    for c in candidates:
        if not valence_legal(c) or not charge_balanced(c):
            continue
        pred = {
            "1h":  predict_proton_shifts(c),
            "13c": predict_carbon_shifts(c),
            "hsqc": predict_heteronuclear_correlations(c),
            "hmbc": predict_long_range_correlations(c),
        }
        s = score_against_observed(pred, peak_table)  # chemical shift residual + cross-peak recall/precision
        scored.append((s, c, pred))
    scored.sort(key=lambda x: x[0], reverse=True)
    best, runner_up = scored[0], scored[1]
    if best.score < ACCEPT_THRESHOLD:
        return Report(status="insufficient_evidence", next_experiments=suggest(best))
    if best.score - runner_up.score < MARGIN_THRESHOLD:
        return Report(status="ambiguous", candidates=[best, runner_up],
                      discriminating_experiments=diff_experiments(best, runner_up))
    return Report(status="resolved", structure=best.canonical,
                  assignment=atom_level_assignment(best.pred, peak_table),
                  confidence=calibrate(best.score))

This skeleton contains a few details that are easy to overlook but decide success or failure:

• Legality checks must come before scoring. Candidates with illegal valence, unbalanced charge, or wrong aromaticity determination should be discarded outright, even when their spectral fit looks high. Chemical constraints are hard constraints, not soft preferences.
• Deduplication must use a canonical representation. Tautomers, and the same molecule written in different ways, if left unnormalized, will manufacture a lot of “false diversity” in the candidate pool and make the ranking results uninterpretable.
• Suggested follow-up experiments must be emitted. When evidence is insufficient, pointing out “recommend adding an HMBC or NOESY to distinguish regioisomers” is far more valuable than forcing a low-confidence answer. This is exactly how experts themselves work.

The model’s role then becomes clear: it is responsible for understanding unstructured natural-language and image inputs, proposing chemically meaningful candidate hypotheses, explaining conflicts, generating descriptions that follow lab conventions, and deciding when to call which tool and when to stop.

4. The Input-Parsing Layer: Turning “an Image” into Trustworthy Structured Facts

The dirtiest and most critical engineering link is extracting a machine-computable peak table from the PDF, image, or raw data exported by the instrument. Errors in this layer propagate to the entire downstream, so the design principle is traceability: each extracted peak must record which file, which page, and which coordinate region it came from, so that any downstream conclusion can be traced back to the original pixels.

The suggested intermediate contract is roughly as follows:

{
  "spectrum": {"nucleus": "1H", "solvent": "CDCl3", "field_mhz": 400,
               "reference_shift": 7.26},
  "peaks": [
    {"ppm": 7.31, "multiplicity": "d", "j_hz": 8.2, "integration": 1.0,
     "provenance": {"file": "sample42.pdf", "page": 1, "bbox": [312, 88, 690, 140]},
     "flags": ["solvent_residual_candidate"]}
  ],
  "quality": {"overlap_ratio": 0.12, "baseline_drift": "mild",
              "s_over_n": 48.0, "unassigned_regions": ["1.2-1.5"]}
}

The two fields flags and quality are often omitted, but they are the key inputs that teach the model to doubt itself. Solvent residual peaks, water peaks, spinning sidebands, and carbon satellites—signals that are not the analyte—must be flagged at the structured stage, or the downstream will strain to find an assignment for a group that doesn’t actually exist.

5. The Dividing Line of Reliability: Teach the Model to Abstain

In professional settings, the value of being able to abstain is higher than the value of being able to answer. A system that can accurately say “current evidence is insufficient to distinguish these two regioisomers; additional measurement is required” is more useful than a system that always answers at 80% accuracy, because the latter forces the expert to stay skeptical of every conclusion.

Uncertainty usually comes from four places: the spectral quality itself is inadequate (overlap, low signal-to-noise); the forward prediction tool is inaccurate in this chemical environment (e.g., strong hydrogen bonding, metal coordination, dynamic conformations); the candidate space is generated incompletely (the real answer simply isn’t in the pool); or multiple candidates have near-tied scores that cannot be separated.

The corresponding control is layered calibration:

• Judge fit using the chemical shift residual distribution rather than absolute values; the residual should be normalized against the prediction tool’s own error level.
• Constrain 2D correlation using both cross-peak recall and precision; raising recall alone will reward the wrong kind of structure—one that is “over-connected.”
• Set an abstention threshold and calibrate it on a holdout set so that the manual-review cost on abstained samples is minimized.

The evaluation metrics should change accordingly; a single “structure exact-match rate” is nowhere near enough. At minimum you need a set: atom-level assignment accuracy, structure exact-match rate, premature-convergence rate (the share of cases that give a definite answer when evidence is insufficient), hallucinated functional-group rate, abstention rate, and the proportion of genuinely difficult cases among abstained samples. Only by reading the last two together can you tell whether the model is “cleverly uncertain” or “indiscriminately shirking.”

6. The Evaluation Set and Continuous Regression

Without an evaluation set, all optimization is self-persuasion. It is worth building the set in difficulty layers:

1. Baseline layer: small molecules with simple structures and clean spectra, used to verify basic correctness.
2. Complex layer: polycyclic natural products and heteroatom-containing skeletons, used to test candidate-generation ability.
3. Stereochemistry layer: diastereomers and systems containing chiral centers, which usually require NOE-type data.
4. Adversarial layer: artificially injected solvent peaks, water peaks, baseline drift, partially overlapping peaks, and low-SNR samples, used specifically to test robustness and abstention behavior.

Annotation requires experts to perform double-blind independent assignment and to measure agreement; samples with low agreement should be removed or marked “unsolvable even by humans,” otherwise the evaluation ceiling itself is blurry. For the online side, spot-checking replaces full-volume review, and the spot-check results are fed back as new regression samples to form a closed loop. At the same time, version drift must be monitored: changing instruments, changing solvent batches, or updating the prediction-tool version can all quietly degrade metrics that were once stable.

7. Platform Implementation and Data Compliance

From a delivery standpoint, this kind of system is naturally a “heavy-tool, heavy-storage, heavy-audit” shape, well suited to being built on general cloud capabilities:

• Container platform hosts the inference services and tool sandboxes; tool execution must be isolated with restricted outbound network access, so that the structure parser cannot affect the main service when it hits malformed input.
• GPU compute pool powers multimodal input parsing and model inference, with elastic scaling to absorb the concentrated arrival of batch spectrum-solving tasks after the day’s experiments end.
• Object storage holds the original spectral files and exported data, managed with lifecycle policies for hot and cold data; the metadata database stores molecular formulas, candidates, scores, and assignment results, supporting retrospection.
• Message queue decouples “task submission” from “spectrum-solving computation,” so large batch jobs can be queued, retried, and observed.
• Log service + monitoring alerts + audit trails record each call’s input version, tool version, model version, conclusion, and confidence. In research and R&D scenarios, reproducibility is the compliance bottom line.

The data-asset nature needs particular emphasis: molecular structures and spectra are often among a company’s most core R&D assets. For that reason, private deployment, least-privilege access, tenant- and project-level data isolation, transport and at-rest encryption, and a “raw data never leaves the domain, only conclusions flow back” policy should be fixed at the architecture-design stage, not patched in before launch. If you use public-cloud capabilities, you also need to be explicit about where the data is stored, how it is encrypted, and its retention period, and confirm all of this at the contract level. On cost and latency, a tiered strategy works: use a lightweight model for legality checks, format parsing, and routine-sample screening, and escalate only difficult or high-disagreement samples to a flagship model for review.

8. The Rollout Roadmap and Common Anti-Patterns

The pragmatic pace is a three-phase one: the first phase is the copilot, which outputs assignment suggestions that must be human-confirmed before being written into the lab notebook; the second phase is pre-fill, which automatically fills in routine-sample conclusions in the lab-notebook system and marks confidence, with humans only spot-checking and correcting; the third phase, once both confidence and drift monitoring are stable, opens automatic release for high-confidence, low-risk sample types, while everything else continues through human review.

Finally, here are several recurring anti-patterns for self-checking: trying to make the model memorize a spectral library without connecting a prediction tool; using a single accuracy metric to drive decisions; allowing the model to directly rewrite raw instrument data; mixing the evaluation set with training data so that metrics look inflated; and ignoring the expert workflow, delivering a tool that is “technically correct but nobody wants to use.” Hit any one of these five and the project will basically stall at the demo stage.

Summary: Getting an LLM to process NMR spectra, the real workload lies not in the model itself, but in replacing unverifiable intuition with a verifiable computational closed loop, replacing uninspectable output with a traceable structured intermediate representation, and replacing “always has an answer” with “knows when it should not answer.” This paradigm is not limited to chemistry; any professional scenario where “the forward direction is computable and the inverse is hard to solve”—materials characterization, failure analysis, spectral diagnosis—can follow the same skeleton.