Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencing
preprint · ICML 2025 · 2025
| Date | 2025-06-16 |
| Type | preprint |
| Venue | ICML 2025 |
| Publisher | arXiv |
| Contribution | algorithm |
| DOI | 10.48550/arXiv.2506.13485 |
| Citations (OpenAlex) | 0 |
Abstract
Peptide sequencing-the process of identifying amino acid sequences from mass spectrometry data-is a fundamental task in proteomics. Non-Autoregressive Transformers (NATs) have proven highly effective for this task, outperforming traditional methods. Unlike autoregressive models, which generate tokens sequentially, NATs predict all positions simultaneously, leveraging bidirectional context through unmasked self-attention. However, existing NAT approaches often rely on Connectionist Temporal Classification (CTC) loss, which presents significant optimization challenges due to CTC’s complexity and increases the risk of training failures. To address these issues, we propose an improved non-autoregressive peptide sequencing model that incorporates a structured protein sequence curriculum learning strategy. This approach adjusts protein’s learning difficulty based on the model’s estimated protein generational capabilities through a sampling process, progressively learning peptide generation from simple to complex sequences. Additionally, we introduce a self-refining inference-time module that iteratively enhances predictions using learned NAT token embeddings, improving sequence accuracy at a fine-grained level. Our curriculum learning strategy reduces NAT training failures frequency by more than 90% based on sampled training over various data distributions. Evaluations on nine benchmark species demonstrate that our approach outperforms all previous methods across multiple metrics and species.
Methods and tools
- RefineNovo: Curriculum learning
Cites (17)
- Bridging the Gap between Database Search and De Novo Peptide Sequencing with SearchNovo (2024) semanticscholar
- Sequence-to-sequence translation from mass spectra to peptides with a transformer model (2024) semanticscholar
- NovoBench: Benchmarking Deep Learning-based De Novo Peptide Sequencing Methods in Proteomics (2024) semanticscholar
- π-PrimeNovo: an accurate and efficient non-autoregressive deep learning model for de novo peptide sequencing (2024) semanticscholar
- AdaNovo: Adaptive De Novo Peptide Sequencing with Conditional Mutual Information (2024) semanticscholar
- Accurate de novo peptide sequencing using fully convolutional neural networks (2023) semanticscholar
- Mitigating the missing-fragmentation problem in de novo peptide sequencing with a two-stage graph-based deep learning model (2023) semanticscholar
- De novo peptide sequencing with InstaNovo: Accurate, database-free peptide identification for large scale proteomics experiments (2023) semanticscholar
- Algorithms for de-novo sequencing of peptides by tandem mass spectrometry: A review (2023) semanticscholar
- Introducing PandaNovo for practical large-scale de novo peptide sequencing (2023) semanticscholar
- De novo mass spectrometry peptide sequencing with a transformer model (2022) semanticscholar
- Computationally instrument-resolution-independent de novo peptide sequencing for high-resolution devices (2021) semanticscholar
- Uncovering Thousands of New Peptides with Sequence-Mask-Search Hybrid De Novo Peptide Sequencing Framework (2019) semanticscholar
- pNovo 3: precise de novo peptide sequencing using a learning-to-rank framework (2019) semanticscholar
- A potential golden age to come—current tools, recent use cases, and future avenues for de novo sequencing in proteomics (2018) semanticscholar
- De novo peptide sequencing by deep learning (2017) semanticscholar
- PEAKS: powerful software for peptide de novo sequencing by tandem mass spectrometry (2003) semanticscholar
Cited by (1)
- Bidirectional Representations Augmented Autoregressive Biological Sequence Generation (2025) semanticscholar