Preprint
Article

This version is not peer-reviewed.

Paired Experimental 1H/13C NMR Fingerprint Prediction Under Molecule- and Scaffold-Disjoint Evaluation

A peer-reviewed version of this preprint was published in:
Chemistry 2026, 8(10), 140. https://doi.org/10.3390/chemistry8100140

Submitted:

21 September 2026

Posted:

22 September 2026

You are already at the latest version

Abstract
Machine-learning approaches for small-molecule nuclear magnetic resonance (NMR) often rely on predicted spectra or evaluation schemes that insufficiently control structural overlap. Here, we curated 555 paired experimental 1H/13C NMR observations representing 514 canonical structures from nmrshiftdb2 to establish an auditable benchmark for NMR-to-molecular-fingerprint prediction. Spectra were encoded as fixed-length peak-count vectors and molecular structures as 2,048-bit radius-2 Morgan fingerprints. Ridge models were evaluated across ten molecule-disjoint and ten Bemis–Murcko scaffold-disjoint partitions, with replicate spectra grouped by structure and model selection restricted to validation data. For molecule-disjoint evaluation, fused 1H+13C data achieved a mean Tanimoto similarity of 0.327 ± 0.019 and Top-10 retrieval of 39.0%, outperforming 13C (0.316 ± 0.021; 33.5%) and 1H (0.268 ± 0.013; 20.5%) alone. Under scaffold-disjoint evaluation, fused performance decreased to 0.265 ± 0.036 and 23.7% Top-10. Moreover, adding 410 provenance-confirmed simulated structures did not improve experimental-test performance. These results establish a rigorously curated experimental benchmark, demonstrate reproducible complementarity between 1H and 13C peak information, and identify scaffold novelty and spectral provenance as key challenges for robust NMR-based molecular fingerprint prediction.
Keywords: 
;  ;  ;  ;  

1. Introduction

Nuclear magnetic resonance (NMR) spectroscopy is a cornerstone of molecular structure elucidation in organic chemistry. Nevertheless, interpretation of NMR spectra remains labor-intensive and often requires substantial expert judgment, particularly for structurally complex or closely related compounds. Machine-learning (ML) approaches offer opportunities to assist this process by predicting structural features, molecular representations, or candidate structures directly from spectral observations. The inverse problem, however, remains intrinsically challenging: distinct molecular structures can produce similar one-dimensional spectra, experimental conditions introduce variability in chemical shifts and peak patterns, and the number of plausible molecular structures increases rapidly with molecular size and complexity.
Recent studies have demonstrated substantial progress toward automated NMR interpretation. Huang et al. combined experimental one-dimensional NMR spectra and molecular formulae with substructure prediction and molecular graph generation [1]. NMR2Struct employed multitask neural learning to infer molecular structures from 1H and 13C NMR data, although its principal large-scale training regime relied on simulated spectra and performance decreased when evaluated on experimental data [2]. Liu and Cole investigated neural prediction of molecular substructures using experimental records from nmrshiftdb2 [3]. More recently, SPECTRE extended spectrum-to-fingerprint retrieval to substantially larger chemical spaces [4], whereas NMRTrans employed literature-mined experimental peak sets together with set-transformer architectures [5]. Collectively, these studies establish NMR-to-fingerprint prediction and candidate retrieval as active areas of research; consequently, methodological novelty can no longer rest solely on mapping NMR spectra to molecular fingerprints or retrieving candidate structures.
An important unresolved issue is whether reported predictive performance reflects genuine generalization to previously unseen chemical space. Random or row-level data partitioning can inadvertently place replicate spectra, or observations corresponding to the same molecular structure, across training and evaluation sets, resulting in information leakage [6,7]. Even structure-disjoint partitions may retain closely related molecular scaffolds and therefore provide optimistic estimates of performance on genuinely novel chemotypes [8]. A further complication concerns data provenance: dataset annotations do not always clearly distinguish experimentally measured spectra from predicted or simulated records. These limitations become particularly consequential for relatively small experimental datasets, where duplicated observations or clusters of structurally related molecules can disproportionately influence performance estimates.
Here, we address these issues through a data-centered and leakage-controlled benchmark based exclusively on provenance-defined experimental 1H/13C NMR observations for primary model evaluation. Specifically, we ask whether paired 1H and 13C peak lists provide complementary information for molecular fingerprint prediction, quantify the loss in predictive and retrieval performance when entire Bemis–Murcko scaffolds are withheld, and determine whether augmentation with provenance-confirmed simulated data improves generalization to experimental spectra. The novelty of this study therefore lies in the combination of paired experimental 1H/13C data, replicate-aware molecule disjoint evaluation, scaffold-disjoint evaluation across repeated partitions, record-level spectra provenance auditing, and controlled assessment of simulated-data augmentation within a fully reproducible benchmark.

2. Materials and Methods

The overall workflow used for experimental 1H/13C NMR data curation, molecular representation, leakage-controlled partitioning, model training, performance evaluation, and provenance-audited simulated-data augmentation is summarized in Figure 1. Experimental records from nmrshiftdb2 were curated, paired at the molecular-structure level, canonicalized, and encoded as fixed-length 1H and 13C peak-count vectors. Molecular targets were represented as 2048-bit radius-2 Morgan fingerprints. Model performance was evaluated using repeated molecule-disjoint and Bemis–Murcko scaffold-disjoint train/validation/test partitions, followed by multi-output Ridge regression and candidate retrieval against the closed Gold reference library. A separate NP-MRD provenance-audit branch was used to identify provenance-confirmed simulated records and assess their contribution as a Bronze auxiliary training dataset.

2.1. Data Source and Curation

The primary data source was an nmrshiftdb2 SDF export containing molecular structures, assigned one-dimensional NMR spectra, and associated metadata. nmrshiftdb2 is an open community database for molecular structures and NMR data [9,10]. Parsing was restricted to properties corresponding to records with assigned one-dimensional 1H or 13C NMR spectra. HSQC and other two-dimensional or unrelated spectral properties were excluded. Records explicitly marked as predicted or calculated were excluded from the experimental Gold dataset. The original source file was preserved unchanged, and the corresponding derived dataset was documented using a SHA-256 cryptographic digest to enable integrity verification.
Following curation and 1H/13C pairing, the Gold dataset comprised 555 paired observations corresponding to 516 source molecule identifiers and 514 unique canonical isomeric structures. Two pairs of source identifiers canonicalized to identical structures; consequently, all data-leakage controls and partitioning procedures were based on canonical SMILES rather than source database identifiers. The final dataset contained 10,968 assigned 1H chemical shifts and 9,328 assigned 13C chemical shifts. Thirty-three canonical structures were represented by multiple paired observations, accounting for 74 dataset rows. Among these structures, six showed variation in the recorded solvent, and eleven showed variation in the recorded magnetic field. We treated these metadata as descriptors of acquisition conditions and did not interpret them as evidence of independent samples or laboratories.

2.2. Spectral and Molecular Representations

Each 1H NMR peak list was transformed into a count vector spanning -1 to 15 ppm using fixed 0.05-ppm bins, resulting in 320 spectral features. Each 13C NMR peak list was analogously encoded over -15 to 235 ppm using 0.5-ppm bins, yielding 500 features. The combined 1H + 13C representation was obtained by concatenating both vectors, producing an 820-feature input vector. These chemical-shift ranges retained all 10,968 1H and 9,328 13C chemical shifts in the curated Gold dataset. Multiplicity, coupling constants, integrals, line-shape information, and raw signal intensities were not included. Molecular targets were represented as 2,048-bit Morgan fingerprints with radius 2 generated using RDKit. Models produced continuous prediction scores for each fingerprint bit [11], whereas binary decision thresholds were determined exclusively from the corresponding validation partition. Molecular similarity was quantified using the Tanimoto coefficient [12]. For candidate-retrieval evaluation, continuous predicted fingerprints were ranked against the closed reference library containing the 514 unique canonical structures represented in the Gold dataset. For structures represented by multiple spectral observations, predictions were averaged at the canonical-molecule level before calculation of molecule-level metrics.

2.3. Leakage-Controlled Data Partitioning

Ten independently seeded train/validation/test partitions were generated under two evaluation regimes: molecule-disjoint and scaffold-disjoint partitioning. Each partition used a two-stage group-aware random split implemented with scikit-learn GroupShuffleSplit. For each seed (20260824-20260833), 15% of groups were first assigned to the test set. Of the remaining groups, 15/85 were assigned to validation and the rest to training, yielding target group proportions of 70/15/15. Because groups were kept intact and differed in size, the realized observation-level proportions could deviate from these targets. The second-stage random state was the partition seed plus 10,000. For molecule-disjoint evaluation, the grouping key was canonical SMILES, so all observations corresponding to the same canonical molecular structure remained in one partition. For scaffold-disjoint evaluation, structures were grouped by their RDKit Bemis-Murcko scaffold [13]. Acyclic molecules, for which the Bemis-Murcko scaffold is empty, were conservatively assigned to one <ACYCLIC> group; this group contained 43 canonical structures and 46 observations. No molecule or scaffold group was divided across partitions. Canonical-structure overlap between training, validation, and test sets was verified to be zero for every run, and scaffold overlap was also verified to be zero for scaffold-disjoint partitions. The curated dataset contained 277 scaffold groups, including 218 singleton scaffolds. A separate frozen audit partition contained 386/85/84 observations in the training/validation/test sets under scaffold grouping and 387/84/84 observations under structure grouping. This partition was retained for auditing and reproducibility; the repeated performance estimates reported here were obtained from the complete ten-partition protocol rather than from a selected individual split.

2.4. Baseline Model, Training, and Model Selection

The primary baseline model was multi-output Ridge regression applied to standardized binned spectral representations. Input scaling parameters were estimated exclusively from the training partition and subsequently applied unchanged to the corresponding validation and test data. Model fitting was likewise restricted to the training partition. The Ridge regularization coefficient (α) and fingerprint decision threshold were selected using only the corresponding experimental validation partition, without access to test-set information. The repeated benchmark used α = 1,000. Identical partition seeds, target representations, model-selection procedures, evaluation procedures, and candidate-retrieval libraries were used for the 1H-only, 13C-only, and fused 1H + 13C comparisons, allowing direct paired comparison among spectral modalities.
All computational analyses were performed in Python 3.12.13 using RDKit 2026.03.5, NumPy 2.5.2, pandas 3.0.5, SciPy, and scikit-learn 1.9.0. Matplotlib was used for data visualization. The complete computational environment, including dependency specifications, is provided in the public repository described in the Data and Code Availability section.

2.5. Statistical Analysis

For each partitioning regime and spectral modality, performance was summarized as the arithmetic mean and sample standard deviation across the ten seeded partitions. Differences between spectral modalities were evaluated using paired bootstrap resampling of the ten matched partition-level results with 10,000 resamples. The resulting bootstrap intervals quantify variability associated with the selected data partitions. They should not be interpreted as conventional population-level confidence intervals derived from independent observations because training and test observations can recur across different seeded partitions. Per-partition metric values and paired modality differences for all ten seeded partitions are provided in the archived research package. Additional analyses stratified by molecular size, training-set similarity, scaffold frequency, solvent, and magnetic field were considered exploratory and were interpreted descriptively because several strata contained small numbers of observations and the corresponding factors were potentially confounded.

2.6. NP-MRD Provenance Audit and Simulated-Data Augmentation

The relationship between the experimental Gold dataset, the NP-MRD provenance-audit workflow, and the Bronze simulated auxiliary dataset used for controlled augmentation is summarized in Figure 2.
The official NP-MRD 2025-08-01 assignment-table package and corresponding molecular structure files were inventoried, and their integrity was documented using cryptographic hashes. NP-MRD is a public resource containing natural-product structures and experimental, simulated, and predicted NMR records [14]. Assignment tables were parsed, molecular structures were canonicalized, duplicates relative to the Gold dataset were identified, and NP-MRD identifiers were cross-linked to their corresponding public compound pages. Candidate records were required to contain nonempty 1H and 13C assignments and to exhibit 13C assignment coverage between 0.70 and 1.00 relative to the annotated molecular structure. Spectral provenance was assigned from the spectrum description associated with the linked page-level record and was not inferred from the organization or name of the bulk archive. Only structures directly linked to records explicitly described as simulated were admitted to the Bronze auxiliary experiment. Bronze entries corresponding to structures present in a given test partition were excluded. Under scaffold-disjoint evaluation, Bronze entries sharing a Bemis–Murcko scaffold with any validation or test structure were additionally removed. Bronze sample weights of 0, 0.1, 0.25, 0.5, and 1.0; Ridge α values of 100 and 1,000; and fingerprint decision thresholds from 0.10 to 0.50 in increments of 0.05 were evaluated. The combination with the highest mean Tanimoto similarity on the experimental validation set was selected without access to the test set. All final test observations remained Gold and experimental. Page-level screening additionally identified potential experimental 1H/13C pairs. These records were excluded from the principal experiment because matching solvent and compatible magnetic-field metadata were considered insufficient to establish acquisition from a common physical sample, and most records would require processing and independent verification of the corresponding raw FID data.

3. Results

3.1. Paired 1H and 13C NMR Data Improve Fingerprint Prediction

The fused 1H + 13C model showed the highest performance under both evaluation regimes (Table 1 and Figure 1). Under molecule-disjoint evaluation, the mean Tanimoto similarity was 0.327 ± 0.019 for the fused model, compared with 0.316 ± 0.021 for 13C and 0.268 ± 0.013 for 1H. Mean Top-10 retrieval was 39.0%, 33.5%, and 20.5%, respectively. The paired mean improvement of the fused model over 13C alone was 0.0117 in Tanimoto similarity (paired-bootstrap interval: 0.0077–0.0156) and 5.5 percentage points in Top-10 retrieval (4.2–7.2 percentage points). Thus, 13C provided the largest individual contribution to predictive performance, whereas the addition of 1H produced a smaller but consistent improvement.
A schematic representation of the complementary information captured by 1H and 13C NMR and its integration into the fused fingerprint-prediction workflow is shown in Figure 3.
The distribution of performance across the repeated partitions is shown in Figure 4. The fused model consistently outperformed the individual 1H and 13C representations, whereas scaffold-disjoint evaluation produced lower and more variable performance than molecule-disjoint evaluation.

3.2. Scaffold-Disjoint Evaluation Reveals a Generalization Gap

When molecular scaffolds rather than individual structures were withheld, fused-model performance decreased substantially (Table 1 and Figure 1). Mean Tanimoto similarity decreased from 0.327 under molecule-disjoint evaluation to 0.265 under scaffold-disjoint evaluation, while mean Top-10 retrieval decreased from 39.0% to 23.7%. The standard deviation of fused-model Tanimoto similarity simultaneously increased from 0.019 to 0.036, indicating greater performance variability among scaffold-disjoint partitions. Despite this overall reduction, fusion retained a paired advantage over 13C alone. The mean improvement was 0.0142 in Tanimoto similarity (paired-bootstrap interval: 0.0097–0.0186) and 4.0 percentage points in Top-10 retrieval (2.1–6.0 percentage points). The dataset contained a high proportion of singleton scaffolds (218 of 277 scaffold groups), and all acyclic structures were conservatively assigned to a common scaffold group, making the scaffold-disjoint benchmark a stringent assessment of transfer to structurally distinct chemical families. Exploratory analysis of the frozen partition showed the same general trend. Molecules with low maximum structural similarity to the training set exhibited lower predictive performance than molecules with closer training-set analogues. Performance estimates for smaller strata were comparatively unstable. Because molecular size, structural similarity, and scaffold frequency covary in this dataset, these exploratory associations were not interpreted as causal.

3.3. Record-Level Provenance Audit of NP-MRD

The official NP-MRD package contained 1,685 assignment tables (Figure 2). Of these, 836 contained assignments for both 1H and 13C, and 796 satisfied the Gold-like 13C coverage criterion. Cross-linking of assignment tables with public NP-MRD records associated 706 tables directly with page-level spectrum records. All 706 directly traceable records were described as simulated: 377 were linked through chemical-shift submissions, whereas 329 were located through the experimental-spectra section but carried descriptions identifying them as simulated. Consequently, archive placement alone was not considered sufficient evidence for classification of an assignment table as experimental.
A separate page-level screening identified 23 NP-MRD compound identifiers with potential experimental 1H/13C pairings after application of solvent and magnetic-field filters. Subsequent file recovery produced a provisional assigned-shift dataset comprising 14 pairs from 11 structures. These records were excluded from the principal evaluation because their labels were inherited from source pages, their assigned peaks depended on the corresponding source structural annotations, and their raw-FID provenance had not been independently validated. Accordingly, this provisional dataset was retained as a potential resource for future external validation rather than used as evidence of external generalization in the present study.

3.4. Simulated-Data Augmentation Does Not Improve Experimental-Test Performance

The Bronze auxiliary dataset contained 410 unique structures associated with provenance-confirmed simulated NMR records (Figure 5). Validation selected a Bronze weight of zero for fused molecule-disjoint prediction and 0.1 for fused scaffold-disjoint prediction. On the corresponding frozen test sets, fused-model Tanimoto similarity remained unchanged at 0.347 under molecule-disjoint evaluation because Bronze augmentation was rejected during validation. Under scaffold-disjoint evaluation, Tanimoto similarity changed only from 0.1950 to 0.1947 after augmentation. Although some candidate-retrieval ranks changed modestly, the primary fingerprint-similarity outcome did not improve after inclusion of the simulated Bronze data. Thus, under the present spectral representation, validation procedure, and experimental test conditions, augmentation with the provenance-confirmed simulated assigned shifts did not improve fingerprint prediction.

4. Discussion

The present study establishes a leakage-controlled benchmark for molecular fingerprint prediction from paired experimental 1H and 13C NMR peak lists. The principal findings are that (i) combining both nuclei consistently improves fingerprint prediction and candidate retrieval relative to either nucleus alone; (ii) most of the predictive performance under the present representation is contributed by 13C, with 1H providing a smaller but reproducible complementary contribution; (iii) performance decreases substantially when generalization is evaluated across unseen molecular scaffolds rather than merely unseen structures; and (iv) augmentation with provenance-confirmed simulated NP-MRD records does not improve fingerprint similarity on the experimental test data under the conditions examined. The complementary performance of 1H and 13C is consistent with the broader premise underlying multimodal NMR approaches such as NMR2Struct, SPECTRE, and NMRTrans, in which different spectral modalities provide complementary structural information [2,4,5]. However, the comparatively modest contribution of 1H observed here should be interpreted in the context of the deliberately simplified spectral representation. The present 1H vectors encode only peak positions and counts and omit multiplicity, integration, scalar coupling, and line-shape information, all of which provide substantial structural information in conventional 1H NMR interpretation. Consequently, the weaker 1H-only performance should not be interpreted as evidence that 1H NMR is intrinsically less informative than 13C NMR. Rather, it indicates that simple peak-count binning captures only a limited fraction of the information available from proton spectra.
This distinction is relevant when comparing the present results with more complex NMR-based machine-learning systems. Huang et al. combined experimental one-dimensional NMR spectra with molecular formula information for substructure prediction and graph generation [1], whereas NMR2Struct used multitask neural learning for structure elucidation and demonstrated the potential of large-scale computational approaches [2]. SPECTRE subsequently implemented multimodal spectrum-to-fingerprint retrieval at substantially larger scale [4], and NMRTrans introduced set-transformer architectures for experimental NMR peak sets [5]. Accordingly, the principal contribution of the present study is not the introduction of NMR-to-fingerprint prediction itself, but the establishment of a compact and auditable experimental benchmark in which paired spectra, replicate structures, evaluation partitions, and spectral provenance are explicitly controlled.
A particularly important finding is the performance decrease observed when the evaluation criterion changes from molecule-disjoint to scaffold-disjoint partitioning. Holding out exact structures is necessary because replicate spectra occur in the curated dataset, but molecule-level separation alone does not eliminate the advantage conferred by structurally related analogues. Under scaffold-disjoint evaluation, fused mean Tanimoto similarity decreased from 0.327 to 0.265 and Top-10 retrieval from 39.0% to 23.7%, while between-partition variability increased. These results indicate that performance estimated from structurally less restrictive partitions can overstate transfer to unfamiliar chemical families, consistent with the rationale for scaffold-based evaluation in molecular machine learning benchmarks [12]. The exploratory observation that molecules with lower maximum similarity to the training set were more difficult to predict supports this interpretation, although molecular size, scaffold frequency, and structural similarity are confounded and therefore prevent a causal attribution.
These findings have broader implications for evaluation of NMR machine-learning systems. Recent models have demonstrated increasingly sophisticated architectures and substantially larger training resources [2,4,5], but architectural performance and chemical generalization are distinct questions. A model can successfully interpolate among structurally related compounds while remaining substantially less reliable for previously unseen scaffold classes. Repeated scaffold-disjoint evaluation therefore provides a useful complement to conventional structure-disjoint testing, particularly when the intended application involves compounds outside well-represented chemical families. In small experimental datasets such as the present one, repeated partitions are also preferable to reliance on a single favorable split because the large number of singleton scaffolds produces appreciable partition-to-partition variability.
The NP-MRD provenance audit identifies a second issue that is relevant beyond the specific models evaluated here. Database-level or archive-level labels do not necessarily establish whether individual NMR records are experimental, simulated, or predicted. In the audited NP-MRD release, all 706 assignment tables that could be directly connected to page-level spectrum descriptions were classified as simulated, including records located within an experimental-spectra section. This observation demonstrates the importance of record-level provenance verification when constructing machine-learning datasets. It does not imply that simulated NMR data are intrinsically unsuitable for model development; rather, simulated and experimental observations should be explicitly distinguished so that their respective contributions to training and evaluation remain auditable.
Consistent with this distinction, adding 410 provenance-confirmed simulated structures to the training workflow did not improve the primary fused fingerprint-similarity outcome on experimental test data. This result is relevant to the broader problem of transferring models trained with synthetic or predicted spectra to experimental measurements. NMR2Struct, for example, reported reduced performance when moving from its principal simulated-data setting to experimental spectra [2]. The present findings provide complementary evidence from a simpler model and an explicitly provenance-controlled dataset: merely increasing the number of simulated assigned peak lists is not sufficient to guarantee improved performance on experimental observations. More realistic spectral simulation, explicit domain adaptation, richer spectral representations, or greater experimental chemical diversity may therefore be required to reduce this distribution gap.
The scope of these conclusions must nevertheless remain clearly defined. The present model predicts a non-invertible molecular fingerprint and ranks candidates within a closed library of 514 structures in which the correct structure is known to be present. Top-k retrieval therefore measures candidate prioritization rather than database-scale molecular identification or de novo structure elucidation. This distinction is especially important when comparing the present benchmark with methods designed to generate molecular structures or search substantially larger chemical spaces [1,2,4]. A practical predictor developed separately within the project incorporates molecular-formula filtering and direct 13C reference matching; its performance represents a dereplication-oriented application and should remain distinct from the leakage-controlled benchmark evaluated here.

4.1. Utility for the Scientific Community

The principal reusable contribution of this study is a controlled reference protocol rather than a new predictive architecture. Researchers developing neural networks, transformers, alternative fingerprints, or richer spectral representations can evaluate their methods using the same molecule- and scaffold-disjoint partitions and compare performance against a transparent Ridge baseline. Such comparisons can help distinguish genuine methodological improvements from apparent gains caused by replicate leakage, close structural analogues, favorable partition selection, or changes in the candidate library. The paired dataset additionally permits controlled comparison of 1H, 13C, and fused representations, while replicate spectra and available acquisition metadata provide a basis for future robustness studies.
The provenance workflow provides a complementary resource for spectroscopic data curation. File hashes, canonical structures, spectrum identifiers, page-level classifications, and inclusion decisions are retained explicitly, providing an auditable path from source records to model inputs. The negative Bronze augmentation result further demonstrates that increasing the amount of simulated spectral data does not necessarily improve transfer to experimental measurements. These resources may therefore support future external-validation studies, domain-adaptation strategies, and acquisition of experimental datasets with better documentation of raw FID data, solvent, magnetic field, temperature, pH, sample identity, and laboratory provenance.
Appropriate applications of the present system include methodological benchmarking, candidate prioritization within a predefined library, comparison of spectral modalities, and methodological training. The model should not be used independently to confirm molecular identity, claim database-scale identification, or infer structures absent from the candidate library. Definitive structural assignment continues to require complementary spectroscopic information and, where appropriate, independent chemical evidence.

4.2. Limitations

Several limitations define the scope of the conclusions. First, the 514 unique structures represented in the Gold dataset cover only a small fraction of organic chemical space, and 218 of the 277 scaffold groups are singletons. This limits both model training and the precision with which generalization to new chemical families can be estimated. Second, the spectral representation deliberately omits signal intensities and important sources of structural information from 1H NMR, including multiplicity, integration, coupling, and line shape, and does not incorporate multidimensional NMR experiments. Third, candidate retrieval is evaluated against a closed library of only 514 structures, substantially smaller than chemical spaces encountered in realistic identification problems. Fourth, acquisition metadata are incomplete and cannot establish whether apparently distinct spectra originate from independent samples or laboratories. Finally, paired-bootstrap intervals quantify variability across overlapping seeded partitions and should not be interpreted as population-level confidence intervals derived from independent samples.

4.3. Future Directions

The highest priority is independent evaluation using an external experimental dataset for which raw spectral data, acquisition conditions, structural assignments, and independence from nmrshiftdb2 are explicitly documented. The provisional experimental NP-MRD candidates identified during the provenance audit could contribute to such an evaluation, but their experimental status and assignments should first be verified against raw FID data or primary records. A second priority is evaluation against a substantially larger, deduplicated candidate library with explicit reporting of target coverage, which would provide a more realistic assessment of retrieval performance.
Methodologically, future studies should evaluate learning curves and robust nearest-neighbor controls to determine how strongly performance depends on experimental sample size and structural similarity to the training set. Richer 1H representations incorporating multiplicity, integration, coupling information, or raw spectral profiles may clarify whether the relatively small contribution of 1H observed here results primarily from information loss during binning. Architectures with modality-specific encoders could further determine whether 1H and 13C require different representations before fusion. Ultimately, these experiments should help distinguish whether future improvements are primarily constrained by experimental data availability, spectral representation, chemical-domain shift, or model architecture.

5. Conclusions

Paired experimental 1H and 13C NMR peak lists provide reproducible complementary information for Morgan-fingerprint prediction, with 13C accounting for most of the predictive performance under the simple binned representations evaluated here and 1H providing a smaller but consistent additional contribution. Performance decreased substantially when previously unseen scaffolds were withheld, indicating that, within the present benchmark, generalization to new chemical families is more challenging than generalization to unseen molecules within structurally related chemical space. Record-level provenance auditing further showed that all directly traceable assignment tables examined in the audited NP-MRD release were described as simulated, and controlled augmentation with these provenance-confirmed simulated data did not improve fingerprint similarity on the experimental test set. Collectively, these findings highlight rigorous data curation, leakage-controlled partitioning, repeated scaffold-disjoint evaluation, and explicit spectral provenance as important requirements for assessing machine-learning models for NMR-assisted molecular annotation. The resulting benchmark provides a reproducible reference for evaluating whether future methods improve chemical generalization while maintaining transparent control of data provenance and information leakage.

Author Contributions

The authors contributed as follows: Conceptualization, J.L.O.R.; methodology, J.L.O.R.; software, J.L.O.R.; validation, J.L.O.R.; formal analysis, J.L.O.R., N.C.-S. and C.A.L.-R.; investigation, J.L.O.R.; resources, J.L.O.R.; data curation, J.L.O.R., N.C.-S. and C.A.L.-R.; writing—original draft preparation, J.L.O.R.; writing—review and editing, J.L.O.R., N.C.-S. and C.A.L.-R.; visualization, J.L.O.R., N.C.-S. and C.A.L.-R.; supervision, J.L.O.R.; project administration, J.L.O.R.; funding acquisition, J.L.O.R. All authors.

Funding

This research was funded by Secretaría de Ciencias, Humanidades, Tecnología e Innovación (SECIHTI) through the Basic and Frontier Science Program, grant number CBF-I-2025-246.

Data Availability Statement

The analysis code, environment specifications, documentation, and scripts used to reconstruct the curated dataset, evaluation partitions, tables, and figures are openly available at https://github.com/olivaresjlo78/nmr-ml-paired-benchmark under the licenses specified in the repository. The versioned research package is archived in Zenodo at https://doi.org/10.5281/zenodo.22681993. The Zenodo record includes the curated derivative dataset, canonical structure identifiers, paired 1H/13C peak lists, fixed molecular and spectral representations, all molecule- and scaffold-disjoint split assignments, per-run metrics, model-selection records, provenance-audit tables, figures, and SHA-256 checksums.
Source archives are cited by database name, release or retrieval date, stable record identifiers, and source URL. Files are redistributed only when permitted by their applicable licenses; otherwise, the provided retrieval and processing scripts allow users to reconstruct the analyzed derivatives from the original repositories. nmrshiftdb2-derived material retains the database license and attribution notice, and NP-MRD-derived material retains its source attribution and applicable reuse terms. Public files are accessible without an individual data request.

Acknowledgments

During the preparation of this manuscript, the authors used GPT-5.6 Sol (OpenAI) as a technical assistance tool for code generation and debugging, as well as for language editing and improvement of textual organization. All AI-assisted code was reviewed, tested, and validated by the authors before incorporation into the computational workflow. The authors reviewed and edited all AI-assisted output and take full responsibility for the content of this publication.

Conflicts of Interest

The authors declare no conflicts of interest, and the funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Abbreviations

The following abbreviations are used in this manuscript:
FID Free Induction Decay
GenAI Generative artificial intelligence
HSQC Heteronuclear single quantum coherence
ML Machine Learning
NMR Nuclear magnetic resonance
NP-MRD Natural products magnetic resonance database
RDKit Open-source cheminformatics software toolkit
SDF Structure-data file
SHA-256 Secure hash algorithm 256-bit
SMILES Simplified molecular input line entry system

References

  1. Huang, Z.; Chen, M.S.; Woroch, C.P.; Markland, T.E.; Kanan, M.W. A framework for automated structure elucidation from routine NMR spectra. Chem. Sci. 2021, 12, 15329–15338. [Google Scholar] [CrossRef] [PubMed]
  2. Hu, F.; Chen, M.S.; Rotskoff, G.M.; Kanan, M.W.; Markland, T.E. Accurate and efficient structure elucidation from routine one-dimensional NMR spectra using multitask machine learning. ACS Cent. Sci. 2024, 10, 2162–2170. [Google Scholar] [CrossRef] [PubMed]
  3. Liu, S.; Cole, J.M. Automated determination of the molecular substructure from nuclear magnetic resonance spectra using neural networks. J. Chem. Inf. Model. 2025, 65, 8435–8447. [Google Scholar] [CrossRef] [PubMed]
  4. Xu, W.; Ryu, B.; Tong, A.; Mao, H.H.; Kim, H.; Zhao, J.J.; Zhang, C.; Xu, Y.; Wang, M.; Gerwick, W.H.; Cottrell, G.W. SPECTRE: A multimodal spectral transformer for small molecule annotation. J. Chem. Inf. Model. 2026, 66, 2501–2512. [Google Scholar] [CrossRef] [PubMed]
  5. Yang, L.; Yang, Z.; Xie, J.; Wang, Y.; Gao, B.; Wei, X.; Sun, J.; Wu, J.; He, C.; Li, Y.; Gu, Q. NMRTrans: Structure elucidation from experimental NMR spectra via set transformers. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’26), Jeju, Republic of Korea, 9–13 August 2026. [Google Scholar] [CrossRef]
  6. Kapoor, S.; Narayanan, A. Leakage and the reproducibility crisis in machine-learning-based science. Patterns 2023, 4, 100804. [Google Scholar] [CrossRef] [PubMed]
  7. Király, P.; Tóth, G. Being aware of data leakage and cross-validation scaling in chemometric model validation. J. Chemom. 2025, 39, e70026. [Google Scholar] [CrossRef]
  8. Willett, P.; Barnard, J. M.; Downs, G. M. Chemical Similarity Searching. J. Chem. Inf. Comput. Sci. 1998, 38, 983–996. [Google Scholar] [CrossRef]
  9. Steinbeck, C.; Krause, S.; Kuhn, S. NMRShiftDB—constructing a free chemical information system with open-source components. J. Chem. Inf. Comput. Sci. 2003, 43, 1733–1739. [Google Scholar] [CrossRef] [PubMed]
  10. Kuhn, S.; Kolshorn, H.; Steinbeck, C.; Schlörer, N. Twenty years of nmrshiftdb2: A case study of an open database for analytical chemistry. Magn. Reson. Chem. 2024, 62, 74–83. [Google Scholar] [CrossRef] [PubMed]
  11. Rogers, D.; Hahn, M. Extended-connectivity fingerprints. J. Chem. Inf. Model 2010, 50, 742–754. [Google Scholar] [CrossRef] [PubMed]
  12. Wu, Z.; Ramsundar, B.; Feinberg, E. N.; Gomes, J.; Geniesse, C.; Pappu, A. S.; Leswing, K.; Pande, V. MoleculeNet: a benchmark for molecular machine learning. Chem. Sci. 2018, 9, 513–530. [Google Scholar] [CrossRef] [PubMed]
  13. Bemis, G. W.; Murcko, M. A. The properties of known drugs. 1. Molecular frameworks. J. Med. Chem. 1996, 39, 2887–2893. [Google Scholar] [CrossRef] [PubMed]
  14. Wishart, D.S.; Sajed, T.; Pin, M.; et al. The Natural Products Magnetic Resonance Database (NP-MRD) for 2025. Nucleic Acids Res. 2025, 53, D700–D708. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Overview of the leakage-controlled workflow used for experimental 1H/13C NMR-to-fingerprint benchmarking.
Figure 1. Overview of the leakage-controlled workflow used for experimental 1H/13C NMR-to-fingerprint benchmarking.
Preprints 234447 g001
Figure 2. Overview of the Gold experimental benchmark and Bronze simulated-data augmentation design. The Gold dataset was constructed from experimental paired 1H/13C observations from nmrshiftdb2 and partitioned into training, validation, and test sets for the primary benchmark. A separate NP-MRD provenance-audit workflow was used to identify directly traceable records and classify them by provenance, yielding a Bronze auxiliary dataset composed of provenance-confirmed simulated structures. Bronze data were optionally added to the training partition only, whereas validation and test sets remained exclusively Gold experimental data. This design was used to assess whether simulated Bronze data improve performance on experimental Gold test data.
Figure 2. Overview of the Gold experimental benchmark and Bronze simulated-data augmentation design. The Gold dataset was constructed from experimental paired 1H/13C observations from nmrshiftdb2 and partitioned into training, validation, and test sets for the primary benchmark. A separate NP-MRD provenance-audit workflow was used to identify directly traceable records and classify them by provenance, yielding a Bronze auxiliary dataset composed of provenance-confirmed simulated structures. Bronze data were optionally added to the training partition only, whereas validation and test sets remained exclusively Gold experimental data. This design was used to assess whether simulated Bronze data improve performance on experimental Gold test data.
Preprints 234447 g002
Figure 3. Schematic representation of the complementary contribution of 1H and 13C NMR information to molecular fingerprint prediction. In the present benchmark, 1H and 13C peak-position information was encoded independently and subsequently combined into a fused spectral representation. The 1H input primarily captures proton-environment and local chemical-shift information, whereas 13C provides complementary information related to distinct carbon environments and the carbon skeleton. The fused representation was used to predict molecular fingerprints, which were evaluated using Tanimoto similarity and Top-10 candidate retrieval. Multiplicity, integration, coupling constants, and line-shape information were not included in the present 1H representation.
Figure 3. Schematic representation of the complementary contribution of 1H and 13C NMR information to molecular fingerprint prediction. In the present benchmark, 1H and 13C peak-position information was encoded independently and subsequently combined into a fused spectral representation. The 1H input primarily captures proton-environment and local chemical-shift information, whereas 13C provides complementary information related to distinct carbon environments and the carbon skeleton. The fused representation was used to predict molecular fingerprints, which were evaluated using Tanimoto similarity and Top-10 candidate retrieval. Multiplicity, integration, coupling constants, and line-shape information were not included in the present 1H representation.
Preprints 234447 g003
Figure 4. Performance of 1H, 13C, and fused 1H + 13C models across repeated molecule- and scaffold-disjoint partitions. Tanimoto similarity and Top-10 retrieval are shown for the ten seeded partitions under each evaluation regime. Scaffold-disjoint evaluation resulted in lower mean performance and greater between-partition variability than molecule-disjoint evaluation.
Figure 4. Performance of 1H, 13C, and fused 1H + 13C models across repeated molecule- and scaffold-disjoint partitions. Tanimoto similarity and Top-10 retrieval are shown for the ten seeded partitions under each evaluation regime. Scaffold-disjoint evaluation resulted in lower mean performance and greater between-partition variability than molecule-disjoint evaluation.
Preprints 234447 g004
Figure 5. NP-MRD provenance audit and controlled simulated-data augmentation. The provenance workflow identified assignment tables that could be directly linked to page-level spectrum records and classified them according to their associated descriptions. All directly traceable assignment-table records included in the audit were described as simulated. The resulting provenance-confirmed simulated structures constituted the Bronze auxiliary dataset. Controlled Bronze augmentation did not improve fused-model Tanimoto similarity on the experimental Gold test data.
Figure 5. NP-MRD provenance audit and controlled simulated-data augmentation. The provenance workflow identified assignment tables that could be directly linked to page-level spectrum records and classified them according to their associated descriptions. All directly traceable assignment-table records included in the audit were described as simulated. The resulting provenance-confirmed simulated structures constituted the Bronze auxiliary dataset. Controlled Bronze augmentation did not improve fused-model Tanimoto similarity on the experimental Gold test data.
Preprints 234447 g005
Table 1. Performance of the NMR-to-fingerprint models across ten molecule- and scaffold-disjoint partitions. Tanimoto similarity and Top-10 retrieval are reported as mean ± standard deviation across the ten partitions. Mean median rank represents the arithmetic mean of the median candidate ranks obtained independently for each partition. Candidate retrieval was performed against the closed library of 514 unique canonical structures in the Gold dataset.
Table 1. Performance of the NMR-to-fingerprint models across ten molecule- and scaffold-disjoint partitions. Tanimoto similarity and Top-10 retrieval are reported as mean ± standard deviation across the ten partitions. Mean median rank represents the arithmetic mean of the median candidate ranks obtained independently for each partition. Candidate retrieval was performed against the closed library of 514 unique canonical structures in the Gold dataset.
Split Input Mean Tanimoto Top-10 Mean median rank
Molecule-disjoint 1H 0.268 ± 0.013 20.5 ± 2.0% 66.8
Molecule-disjoint 13C 0.316 ± 0.021 33.5 ± 4.5% 32.3
Molecule-disjoint 1H+13C 0.327 ± 0.019 39.0 ± 2.5% 23.5
Scaffold-disjoint 1H 0.230 ± 0.027 8.2 ± 3.8% 115.5
Scaffold-disjoint 13C 0.251 ± 0.035 19.8 ± 7.2% 72.0
Scaffold-disjoint 1H+13C 0.265 ± 0.036 23.7 ± 8.0% 53.5
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.