Diffusion-based Foundation Model For Mass Spectrometry Via Self-Supervised Learning on Unlabelled Data
thesis · 2026
| Date | 2026-06-11 |
| Type | thesis |
| Publisher | MSc thesis |
| Contribution | adjacent |
| Supervisor | Konstantinos Kalogeropoulos, Jeroen Van Goey, Jemma Daniel, Divanisha Patel |
| Link | https://jeroen.vangoey.be/files/theses/diffusion-foundation-model-mass-spectrometry.pdf |
Abstract
The identification of mass spectra is key to proteomics, which is a technology used in a wide range of biological and medical applications. With the surge of machine learning applications in proteomics, this data needs to be processed into representations suitable for AI models to learn from. However, mass spectrum data contains noise from different sources and instrument configurations, which can reduce its utility in downstream machine learning models. Current approaches, such as the InstaNovo and Casanovo models, that use encoders to provide these representations in the form of embeddings are also limited, as they do not fully reduce the impact of noise in the data and originate from models optimized for the singular task of de novo peptide sequencing. In addition, some of these methods require supervised learning, which relies on labelled data, yet a large amount of mass spectrum data currently remains unlabelled. In this study, a diffusion-based encoder is trained using a self-supervised masked spectrum prediction task to learn robust representations of mass spectra from unlabelled data. The resulting embeddings should provide general-purpose spectrum representations that can be transferred to downstream proteomics tasks, reducing reliance on task-specific supervised models and enabling more effective use of large-scale mass spectrometry datasets. Overall, our model achieves an R² of 0.784 for precursor m/z at large-scale compared to the baseline (0.923), while learning compact and organised latent representations. This indicates a trade-off between representation compactness and predictive accuracy. For classification, the foundation model achieves a precursor charge accuracy of 0.647 on the full dataset, which is below the majority-class baseline of approximately 0.734 given the strong class imbalance (charge 2 dominates 73.4% of spectra), while the InstaDeep baseline achieves 0.942, indicating strong performance far above trivial baselines. For fragmentation type, the model achieves 0.624 compared to a majority-class baseline of 0.547 and a random baseline of 0.250, indicating marginal but non-trivial performance. Based on the small-scale ablations, the results suggest that the diffusion-based encoder using masking and the multi-loss objective supports learning organised and compact latent representations, however, additional mechanisms, such as parameter tuning at scale, are required to improve predictive performance of the current model, particularly under imbalanced classification settings and at a large dataset scale. In addition, our findings show that the centroid loss requires mechanisms, such as AdaLN modulation, to reduce its performance penalty on learned embeddings.
Methods and tools
- Diffusion spectrum foundation model: Self-supervised diffusion encoder for tandem mass spectra, trained by masked spectrum prediction on unlabelled data to give general-purpose spectrum embeddings transferable to downstream proteomics tasks instead of representations inherited from a model optimised only for de novo sequencing. Reaches an R² of 0.784 for precursor m/z at scale against a 0.923 baseline while learning compact, organised latent representations, a trade-off between compactness and predictive accuracy; precursor-charge and fragmentation-type classification stay near the majority-class baselines. Ablations indicate the masking and multi-loss objective drive the latent structure, and that the centroid loss needs modulation such as AdaLN to avoid penalising the learned embeddings.