Bidirectional Representations Augmented Autoregressive Biological Sequence Generation
preprint · NeurIPS 2025 · 2025
| Date | 2025-10-09 |
| Type | preprint |
| Venue | NeurIPS 2025 |
| Publisher | arXiv |
| Contribution | algorithm |
| DOI | 10.48550/arXiv.2510.08169 |
| Citations (OpenAlex) | 0 |
Abstract
Autoregressive (AR) models, common in sequence generation, are limited in many biological tasks such as de novo peptide sequencing and protein modeling by their unidirectional nature, failing to capture crucial global bidirectional token dependencies. Non-Autoregressive (NAR) models offer holistic, bidirectional representations but face challenges with generative coherence and scalability. To transcend this, we propose a hybrid framework enhancing AR generation by dynamically integrating rich contextual information from non-autoregressive mechanisms. Our approach couples a shared input encoder with two decoders: a non-autoregressive one learning latent bidirectional biological features, and an AR decoder synthesizing the biological sequence by leveraging these bidirectional features. A novel cross-decoder attention module enables the AR decoder to iteratively query and integrate these bidirectional features, enriching its predictions. This synergy is cultivated via a tailored training strategy with importance annealing for balanced objectives and cross-decoder gradient blocking for stable, focused learning. Evaluations on a demanding nine-species benchmark of de novo peptide sequencing show that our model substantially surpasses AR and NAR baselines. It uniquely harmonizes AR stability with NAR contextual awareness, delivering robust, superior performance on diverse downstream data. This research advances biological sequence modeling techniques and contributes a novel architectural paradigm for augmenting AR models with enhanced bidirectional understanding for complex sequence generation. Code is available at https://github.com/BEAM-Labs/denovo.
Methods and tools
- CrossNovo: AR + NAR hybrid
Cites (18)
- MassNet: billion-scale AI-friendly mass spectral corpus enables robust de novo peptide sequencing (2025) semanticscholar
- Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencing (2025) semanticscholar
- Bridging the Gap between Database Search and De Novo Peptide Sequencing with SearchNovo (2024) semanticscholar
- Sequence-to-sequence translation from mass spectra to peptides with a transformer model (2024) semanticscholar
- π-PrimeNovo: an accurate and efficient non-autoregressive deep learning model for de novo peptide sequencing (2024) semanticscholar
- AdaNovo: Adaptive De Novo Peptide Sequencing with Conditional Mutual Information (2024) semanticscholar
- De novo peptide sequencing with InstaNovo: Accurate, database-free peptide identification for large scale proteomics experiments (2023) semanticscholar
- Algorithms for de-novo sequencing of peptides by tandem mass spectrometry: A review (2023) semanticscholar
- Introducing PandaNovo for practical large-scale de novo peptide sequencing (2023) semanticscholar
- Comprehensive evaluation of peptide de novo sequencing tools for monoclonal antibody assembly (2023) semanticscholar
- De novo mass spectrometry peptide sequencing with a transformer model (2022) semanticscholar
- Computationally instrument-resolution-independent de novo peptide sequencing for high-resolution devices (2021) semanticscholar
- Deep Learning in Proteomics (2020) semanticscholar
- Uncovering Thousands of New Peptides with Sequence-Mask-Search Hybrid De Novo Peptide Sequencing Framework (2019) semanticscholar
- De novo peptide sequencing by deep learning (2017) semanticscholar
- PEAKS DB: De Novo Sequencing Assisted Database Search for Sensitive and Accurate Peptide Identification (2012) semanticscholar
- PepNovo: de novo peptide sequencing via probabilistic network modeling (2005) semanticscholar
- PEAKS: powerful software for peptide de novo sequencing by tandem mass spectrometry (2003) semanticscholar