Nine-species benchmark
benchmark · DDA · 5 versions · 34 papers
| Kind | benchmark |
| Acquisition | DDA |
| Organisms | Apis mellifera, Bacillus subtilis, Candidatus Thiodiazotropha endoloripes, Homo sapiens, Methanosarcina mazei, Mus musculus, Saccharomyces cerevisiae, Solanum lycopersicum, Vigna mungo |
| Home | https://massive.ucsd.edu/ProteoSAFe/dataset.jsp?accession=MSV000081382 |
The field’s most-cited evaluation set: tryptic DDA runs from nine taxonomically distant organisms, used leave-one-species-out so a model is tested on a proteome it never trained on. Assembled by DeepNovo from nine unrelated public submissions.
Versions
original (DeepNovo, 2017)
The MGF files DeepNovo curated and deposited. What a paper means by “the nine-species dataset” unless it says otherwise.
released 2017-07-18 · introduced by De novo peptide sequencing by deep learning
Where it lives:
- MassIVE · MSV000081382
Assembled from 9 third-party submissions:
revised (main)
Re-curated to remove peptide redundancy between species, which leaked test peptides into training in the original, and re-released with a fix for a bug that wrongly detected shared peptides between species. Introduced by “A multi-species benchmark for training and validating mass spectrometry proteomics machine learning models”. 2,844,842 spectra. The MassIVE record carries dated update folders, so that accession is not a single fixed object.
spectra 2,844,842 · released 2024-11-08 · introduced by A multi-species benchmark for training and validating mass spectrometry proteomics machine learning models
Where it lives:
- DOI · 10.5281/zenodo.12926326 construction code and archive, not the spectra
- GitHub · Noble-Lab/multi-species-benchmark construction code and archive, not the spectra
- MassIVE · MSV000090982
- Zenodo · 10.5281/zenodo.13685813 main variant, plus annotated spectra in mzSpecLib
InstaNovo split
A fixed train/validation/test split published as parquet with its own DOI, which makes it the only version of this benchmark that is reproducible by citation alone. Its 499,402 training spectra are the same count NovoBench retrains every architecture on.
spectra 639,286 · train 499,402 · validation 28,572 · test 111,312 · released 2024-12-01
Where it lives:
- DOI · 10.57967/hf/3821
- Hugging Face · InstaDeepAI/ms_ninespecies_benchmark
ProteoBench selection
The selection ProteoBench’s de novo DDA-HCD module scores submissions against.
spectra 779,879
Where it lives:
revised (balanced)
The same re-curation, then randomly thinned so the species are more evenly represented: smaller and balanced rather than larger and skewed. Shipped in the same Zenodo record as the main variant, so “the revised benchmark” is still two different things.
released 2024-11-08 · introduced by A multi-species benchmark for training and validating mass spectrometry proteomics machine learning models
Where it lives:
- Zenodo · 10.5281/zenodo.13685813 balanced variant
Deposited by (2)
More than one paper here means one piece of work published twice, a preprint and its version of record; a deposit itself happens once.
- A multi-species benchmark for training and validating mass spectrometry proteomics machine learning models (2024) peer-reviewed revised (main)
- A multi-species benchmark for training and validating mass spectrometry proteomics machine learning models (2024) preprint revised (main)
Used by (32)
- Learning from tandem mass spectra at scale with a self-supervised foundation model for proteomics (2026) preprint version not stated
- Spectra Fragment-Ion and Amino Acid Probability Prediction for Peptide Sequencing (2026) thesis version not stated
- False discovery rate control for trustworthy AI-based de novo peptide sequencing (2026) preprint version not stated
- Learning Fragmentation Physics or Exploiting Sequence Priors? Benchmarking Bias in Deep Learning Models for De Novo Peptide Sequencing (2026) preprint revised (main)
- π-MSNet: A billion-scale, AI-ready living proteomics data portal (2026) preprint original (DeepNovo, 2017)
- XA-Novo: an accurate and high-throughput mass spectrometry-based de novo sequencing technology for monoclonal antibodies and antibody mixtures (2026) peer-reviewed version not stated
- Accurate de novo sequencing of the modified proteome with OmniNovo (2025) preprint original (DeepNovo, 2017), revised (main)
- MassNet: billion-scale AI-friendly mass spectral corpus enables robust de novo peptide sequencing (2025) preprint original (DeepNovo, 2017)
- InstaNovo enables diffusion-powered de novo peptide sequencing in large-scale proteomics experiments (2025) peer-reviewed InstaNovo split, original (DeepNovo, 2017)
- Pairwise Attention: Leveraging Mass Differences to Enhance De Novo Sequencing of Mass Spectra (2025) preprint revised (main)
- Bridging the Gap between Database Search and De Novo Peptide Sequencing with SearchNovo (2025) ML conference original (DeepNovo, 2017), revised (main)
- π-PrimeNovo: an accurate and efficient non-autoregressive deep learning model for de novo peptide sequencing (2025) peer-reviewed original (DeepNovo, 2017), revised (main)
- Deep Learning Methods for De Novo Peptide Sequencing (2024) peer-reviewed version not stated
- Bridging the Gap between Database Search and De Novo Peptide Sequencing with SearchNovo (2024) preprint original (DeepNovo, 2017), revised (main)
- Transforming de novo peptide sequencing by explainable AI (2024) preprint version not stated
- Sequence-to-sequence translation from mass spectra to peptides with a transformer model (2024) peer-reviewed revised (main)
- NovoBench: Benchmarking Deep Learning-based De Novo Peptide Sequencing Methods in Proteomics (2024) ML conference original (DeepNovo, 2017), revised (main)
- Enhancing Peptide Mass Spectra Encoder through Pretraining using Contrastive Predictive Coding (2024) thesis version not stated
- π-PrimeNovo: an accurate and efficient non-autoregressive deep learning model for de novo peptide sequencing (2024) preprint original (DeepNovo, 2017), revised (main)
- Bidirectional de novo peptide sequencing using a transformer model (2024) peer-reviewed version not stated
- Deep learning-driven fragment ion series classification enables highly precise and sensitive de novo peptide sequencing (2024) peer-reviewed original (DeepNovo, 2017)
- De novo peptide sequencing with InstaNovo: Accurate, database-free peptide identification for large scale proteomics experiments (2023) preprint InstaNovo split, original (DeepNovo, 2017)
- DpNovo: A DEEP LEARNING MODEL COMBINED WITH DYNAMIC PROGRAMMING FOR DE NOVO PEPTIDE SEQUENCING (2023) thesis version not stated
- Denovo-GCN: De Novo Peptide Sequencing by Graph Convolutional Neural Networks (2023) peer-reviewed version not stated
- Sequence-to-sequence translation from mass spectra to peptides with a transformer model (2023) preprint version not stated
- DePS: An improved deep learning model for de novo peptide sequencing (2022) preprint original (DeepNovo, 2017)
- Computationally instrument-resolution-independent de novo peptide sequencing for high-resolution devices (2021) peer-reviewed original (DeepNovo, 2017)
- Peptide Sequencing with Deep Learning (2020) thesis original (DeepNovo, 2017)
- Uncovering Thousands of New Peptides with Sequence-Mask-Search Hybrid De Novo Peptide Sequencing Framework (2019) peer-reviewed version not stated
- DeepNovoV2: Better de novo peptide sequencing with deep learning (2019) preprint original (DeepNovo, 2017)
- Postnovo: Postprocessing Enables Accurate and FDR-Controlled de Novo Peptide Sequencing (2018) peer-reviewed version not stated
- De novo Peptide Sequencing (2016) peer-reviewed InstaNovo split, original (DeepNovo, 2017)
Checkpoints trained on this (2)
| Method | Checkpoint | Version of this dataset | Size | Get it |
|---|---|---|---|---|
| Casanovo | nine-species | not stated | 5.1 GB | Zenodo |
| PhysNovo | nine-species | not stated | 1.7 GB | Zenodo |
Each link rests on stated evidence rather than a text match:
- Casanovo nine-species: the Zenodo record is titled ‘Casanovo model weights on nine-species benchmark’; it does not say WHICH version of the benchmark, so none is asserted
- PhysNovo nine-species: the Zenodo record is titled ‘PhysNovo Model Weights Trained on the Nine-Species Benchmark Dataset’; it does not say which version
Methods on these papers (29)
- 9-species multi-species benchmark benchmark
- CPC spectrum encoder pretraining adjacent
- Casanovo algorithm
- De novo peptide sequencing (Proteome Informatics chapter) review
- DePS algorithm
- Deep Learning Methods for De Novo Peptide Sequencing review
- DeepNovo V2 algorithm
- Denovo-GCN algorithm
- DpNovo algorithm
- FDR control for AI-based de novo sequencing post-processor
- Fragment-ion and amino-acid probability models adjacent
- InstaNovo algorithm
- InstaNovo+ algorithm
- InstaNovo-FM algorithm
- NovoB algorithm
- NovoBench benchmark
- OmniNovo algorithm
- Pairwise algorithm
- PointNovo algorithm
- PostNovo post-processor
- Prior-vs-physics benchmark for DL de novo sequencing benchmark
- SMSNet algorithm
- SearchNovo adjacent
- Spectralis post-processor
- XA-Novo algorithm
- XuanjiNovo algorithm
- π-MSNet adjacent
- π-PrimeNovo algorithm
- π-xNovo algorithm
Taken from the describing links only, so a paper that merely ran a tool on this data does not make that tool a method of it.