A living proteomics benchmark for comprehensive evaluation of deep learning-based de novo peptide sequencing tools

preprint · Nature Methods (Registered Report) · 2026

preprint · Nature Methods (Registered Report) · 2026. Marina Pominova et al. Mass spectrometry-based proteomics is essential for understanding protein composition and function, yet…
Date 2026-03-12
Type preprint
Venue Nature Methods (Registered Report)
Publisher figshare
Contribution benchmark
DOI 10.6084/m9.figshare.31680883
Citations (OpenAlex) 0

Abstract

Mass spectrometry-based proteomics is essential for understanding protein composition and function, yet traditional sequence database-based methods face challenges in identifying novel peptides, post-translational modifications, and diverse proteomes. De novo peptide sequencing, which operates independently of sequence databases, offers a powerful approach for uncovering these unknown peptides. However, the lack of consistent evaluation frameworks for the growing array of deep learning-based de novo sequencing tools limits their adoption and effective application. Here, we introduce a comprehensive, community-driven benchmarking resource designed to assess the performance of various de novo sequencing tools across a broad range of experimental conditions and proteomic applications. Our benchmark employs heterogeneous datasets to establish a standardizedevaluation framework, providing a transparent, evolving resource accessible through an interactive online dashboard. This benchmark is anticipated to offer key insights into tool performance, aiding researchers in selecting suitable tools and identifying areas for future refinement and development in de novo peptide sequencing.

Authors

  1. Marina Pominova · University of Antwerp
  2. Jeroen Van Goey · InstaDeep Ltd
  3. Charlotte Adams · University of Antwerp
  4. Ceder Dens · University of Antwerp
  5. Shaorong Chen · Westlake University, Zhejiang University
  6. Guillaume Deflandre · UCLouvain
  7. William E. Fondrie · Talus Bioscience
  8. Cheng Ge · Jiangsu University of Technology, Ocean University of China
  9. Zhi Jin · Shanghai Artificial Intelligence Laboratory, Soochow University
  10. Daniela Klaproth-Andrade · Technical University of Munich
  11. Konstantinos Kalogeropoulos · Delft University of Technology, Kavli Institute of Nanoscience, Technical University of Denmark
  12. Joel Lapin · Technical University of Munich
  13. Tianze Ling · Beijing Institute of Lifeomics, State Key Laboratory of Medical Proteomics, Tsinghua University
  14. Kaiyuan Liu · Indiana University, Indiana University Bloomington
  15. Qixin Liu · Rapid Novor Inc.
  16. Zhongzhi Luan · Beihang University
  17. Alfred Nilsson · KTH Royal Institute of Technology
  18. Christian Nix · Technical University of Munich
  19. Yingying Sun · Westlake University
  20. Tim Van Den Bossche · Ghent University, VIB
  21. Ruitao Wu · Shandong University of Technology
  22. Jun Xia · The Hong Kong University of Science and Technology, The Hong Kong University of Science and Technology (Guangzhou), Westlake University
  23. Qingyang Xiao · Indiana University
  24. Shu Yang · Beihang University
  25. Tingpeng Yang · Peng Cheng Laboratory, Tsinghua Shenzhen International Graduate School, Tsinghua University
  26. Chenyu Yao · Rapid Novor Inc.
  27. Melih Yilmaz · University of Washington
  28. Di Zhang · Shandong University of Technology
  29. Xiang Zhang (Shanghai AI Lab) · Fudan University, Shanghai Artificial Intelligence Laboratory, University of British Columbia
  30. Jingbo Zhou · Westlake University, Zhejiang University
  31. Cheng Chang · Beijing Institute of Lifeomics, International Academy of Phronesis Medicine (Guangdong), National Center for Protein Sciences (Beijing), State Key Laboratory of Medical Proteomics
  32. Shan Chang · Jiangsu University of Technology, Ocean University of China
  33. Ekapol Chuangsuwanich · Chulalongkorn University
  34. Julien Gagneur · Helmholtz Center Munich, Technical University of Munich
  35. Laurent Gatto · UCLouvain
  36. Tiannan Guo · Westlake Institute for Advanced Study, Westlake University
  37. Lukas Käll · KTH Royal Institute of Technology
  38. Ming Li · Bioinformatics Solutions Inc., Central China Institute of Artificial Intelligence, Peng Cheng Laboratory, University of Waterloo, University of Western Ontario
  39. Stan Z. Li · Westlake University
  40. Benjamin A. Neely · National Institute of Standards and Technology
  41. Michael R. Shortreed · University of Wisconsin-Madison
  42. Sira Sriswasdi · Chulalongkorn University, King Chulalongkorn Memorial Hospital
  43. Baozhen Shan · Bioinformatics Solutions (Canada), Bioinformatics Solutions Inc., Western University
  44. Siqi Sun · Fudan University, Shanghai Artificial Intelligence Laboratory
  45. Haixu Tang · Indiana University, Indiana University Bloomington, University of California San Diego
  46. Haipeng Wang · Chinese Academy of Sciences, Shandong University of Technology, University of Chinese Academy of Sciences
  47. Yu Wang · Peng Cheng Laboratory
  48. Mathias Wilhelm · Technical University of Munich
  49. Lei Xin · Bioinformatics Solutions (Canada), Bioinformatics Solutions Inc., Western University
  50. Bin Ma · Rapid Novor Inc., University of Waterloo, University of Western Ontario, Western University
  51. William Stafford Noble · University of Washington
  52. Timothy P. Jenkins · Technical University of Denmark
  53. Wout Bittremieux · Biomedical Informatics Research Center Antwerp, Indiana University, University of Antwerp, University of California San Diego, University of Washington

Methods and tools

  • Living proteomics benchmark: Community-driven, continuously-updated benchmarking resource for deep-learning de novo peptide sequencing: evaluates tools across a broad span of experimental conditions and proteomic applications rather than one frozen test set. Nature Methods Registered Report co-authored by 53 researchers spanning most of the groups that build the tools being benchmarked.

Data used

  • A comprehensive LFQ benchmark dataset to validate data analysis pipelines on modern day acquisition strategies in proteo (as deposited) · PXD028735
  • ABRF iPRG2020 Metaproteomics Study Data (as deposited) · PXD034795
  • Anti-FLAG-M2 antibody sequencing (as deposited) · PXD023419
  • Benchmarking DIA data analysis workflows (as deposited) · PXD044981
  • Critical Assessment of Metaproteome Investigation (CAMPI): a Multi-Lab Comparison of Established Workflows (as deposited) · PXD023217
  • De novo peptide sequencing tools benchmark (as deposited) · MSV000096182
  • Effect of POR knockdown on the proteome of HepaRG cells (as deposited) · PXD028577
  • Files for: Improved Protein Inference from Multiple Protease Bottom-Up Mass Spectrometry Data (as deposited) · PXD012272
  • Fully automated sample processing and analysis workflow for low-input proteome profiling (as deposited) · PXD021882
  • IPRG2015 (as deposited) · MSV000079843
  • Monoclonal antibody de novo assembly (as deposited) · MSV000079801
  • Nine-species benchmark (version not stated)
  • ProteomeTools (21-PTM subset) · InstaDeepAI/PXD009449, PXD009449
  • ProteomeTools (Bruker timsTOF) · PXD019086
  • ProteomeTools (Parts I-III) · PXD004732, PXD010595, PXD021013
  • ProteomeTools (TMT 6-plex) · PXD023119, PXD023120
  • ProteomeTools (non-tryptic (Bruker)) · PXD043844
  • Rewiring of B cell receptor signaling by Epstein-Barr virus LMP2A (as deposited) · PXD018566
  • Ultra-fast label-free quantification and comprehensive proteome coverage with narrow-window data-independent acquisition (as deposited) · PXD046453
  • non-canonical HLA peptides in cancer (as deposited) · PXD013649

Seen in the charts

Back to the full map

Back to top