ByNovo

algorithm · Transformer (AR)

ByNovo: algorithm · Transformer (AR). RankNovo’s in-house base model with an auxiliary b/y ion head

RankNovo’s in-house base model with an auxiliary b/y ion head

Kind algorithm
Deep learning yes
Acquisition DDA
Family Transformer (AR)

Reported comparisons (3)

The comparison tables this method’s own papers print, standardised: every value on a 0-1 scale, methods down the side, the measure and then the species across. These are numbers papers report about themselves and their baselines. They are not a leaderboard, and they do not compare across tables: each was produced by a different group, on the dataset named in its corner, with each baseline either retrained, run from released weights or quoted from another paper. Where the paper says which, it follows the method’s name (hover it for the sentence); most papers do not say. Bold is the best value in a column and underline the runner-up, our ranking rather than the paper’s own marks.

Table1

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing, page 7: Evaluation of RankNovo in comparison to baseline and base models on the 9-species-V1 and 9-species-V2 test sets. Bolded entries indicate the best-performing models. The symbol “†” indicates that the model serves as both a baseline and a base model. “⋆” signifies that the base models were developed and trained by us. Here, ByNovo is the best base model. So the performance of three self-trained base models (R-Casa, R-Contra, R-By) is only provided in Appendix D.1.

Nine-species benchmark Amino acid precision
Bacillus subtilis Candidatus Thiodiazotropha endoloripes Apis mellifera Homo sapiens Methanosarcina mazei Mus musculus Vigna mungo Solanum lycopersicum Saccharomyces cerevisiae Average
PEAKS 0.719 0.586 0.633 0.639 0.673 0.600 0.644 0.728 0.748 0.663
DeepNovo 0.742 0.602 0.630 0.610 0.694 0.623 0.679 0.731 0.750 0.673
PointNovo 0.768 0.589 0.644 0.606 0.712 0.626 0.730 0.733 0.779 0.687
Casanovo 0.749 0.603 0.629 0.586 0.679 0.689 0.668 0.721 0.684 0.667
Casanovo V2 · released 0.888 0.791 0.823 0.872 0.877 0.813 0.891 0.891 0.915 0.862
ContraNovo · released 0.901 0.807 0.848 0.920 0.896 0.839 0.913 0.898 0.919 0.882
ByNovo 0.920 0.823 0.876 0.917 0.914 0.841 0.932 0.912 0.934 0.897
RankNovo 0.926 0.838 0.885 0.929 0.920 0.860 0.938 0.918 0.938 0.906
Nine-species benchmark Peptide recall
Bacillus subtilis Candidatus Thiodiazotropha endoloripes Apis mellifera Homo sapiens Methanosarcina mazei Mus musculus Vigna mungo Solanum lycopersicum Saccharomyces cerevisiae Average
PEAKS 0.387 0.203 0.287 0.277 0.356 0.197 0.362 0.403 0.428 0.322
DeepNovo 0.449 0.253 0.330 0.293 0.422 0.286 0.436 0.454 0.462 0.376
PointNovo 0.518 0.298 0.396 0.351 0.478 0.355 0.511 0.513 0.534 0.439
Casanovo 0.537 0.330 0.406 0.341 0.478 0.426 0.506 0.521 0.490 0.448
Casanovo V2 · released 0.793 0.558 0.669 0.712 0.754 0.555 0.772 0.783 0.837 0.714
ContraNovo · released 0.815 0.575 0.711 0.820 0.780 0.616 0.799 0.794 0.854 0.752
ByNovo 0.833 0.582 0.731 0.789 0.799 0.596 0.814 0.807 0.871 0.758
RankNovo 0.851 0.620 0.752 0.820 0.813 0.629 0.836 0.822 0.885 0.781

RankNovo reranks the candidates of six base models. Casanovo V2 and ContraNovo are the released models; ByNovo is RankNovo's own in-house model, with an auxiliary b/y ion-annotation head (its Appendix B.4); R-Casanovo, R-ContraNovo and R-ByNovo are reverse-decoding versions of the three, trained by RankNovo's authors and recorded as the 'reverse decoding' variant. All six were trained on MassIVE-KB and evaluated zero-shot. '†' marks a model that is both baseline and base model, '⋆' one the authors trained. The plain Casanovo row repeats the values other papers print for Casanovo (0.749 and 0.537 on Bacillus) and the paper does not say where they come from. THE TEXT AND THE TABLE DISAGREE on V1: the prose gives RankNovo an amino-acid precision of 0.829 and a peptide recall '6.1%' above ByNovo; the table prints 0.824, and 0.660 against ByNovo's 0.623. The table's values are recorded.
In the paper: column amino acid precision, Average: the original table bolded RankNovo (0.824), RankNovo (0.906).
In the paper: column amino acid precision, Bacillus: the original table bolded RankNovo (0.874), RankNovo (0.926).
In the paper: column amino acid precision, C. bacteria: the original table bolded RankNovo (0.746), RankNovo (0.838).
In the paper: column amino acid precision, Honeybee: the original table bolded RankNovo (0.810), RankNovo (0.885).
In the paper: column amino acid precision, Human: the original table bolded RankNovo (0.802), RankNovo (0.929).
In the paper: column amino acid precision, M. mazei: the original table bolded RankNovo (0.840), RankNovo (0.920).
In the paper: column amino acid precision, Mouse: the original table bolded RankNovo (0.828), RankNovo (0.860).
In the paper: column amino acid precision, Ricebean: the original table bolded RankNovo (0.859), RankNovo (0.938).
In the paper: column amino acid precision, Tomato: the original table bolded RankNovo (0.844), RankNovo (0.918).
In the paper: column amino acid precision, Yeast: the original table bolded RankNovo (0.816), RankNovo (0.938).
In the paper: column peptide recall, Average: the original table bolded RankNovo (0.660), RankNovo (0.781).
In the paper: column peptide recall, Bacillus: the original table bolded RankNovo (0.738), RankNovo (0.851).
In the paper: column peptide recall, C. bacteria: the original table bolded RankNovo (0.539), RankNovo (0.620).
In the paper: column peptide recall, Honeybee: the original table bolded RankNovo (0.630), RankNovo (0.752).
In the paper: column peptide recall, Human: the original table bolded RankNovo (0.642), RankNovo (0.820).
In the paper: column peptide recall, M. mazei: the original table bolded RankNovo (0.672), RankNovo (0.813).
In the paper: column peptide recall, Mouse: the original table bolded RankNovo (0.583), RankNovo (0.629).
In the paper: column peptide recall, Ricebean: the original table bolded RankNovo (0.733), RankNovo (0.836).
In the paper: column peptide recall, Tomato: the original table bolded RankNovo (0.691), RankNovo (0.822).
In the paper: column peptide recall, Yeast: the original table bolded RankNovo (0.703), RankNovo (0.885).

Table6

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing, page 17: Evaluation of RankNovo in comparison to baseline and base methods on the 9-species-V1 test set. Bolded entries indicate the best-performing models. The symbol “†” indicates that the model serves as both a baseline and a base model. “⋆” signifies that the base models were developed and trained by us.

The same table is also printed in RankNovo: A Universal Reranking Approach for Robust De Novo Peptide Sequencing (Table1, page 8).

Nine-species benchmark
original (DeepNovo, 2017) 		Amino acid precision
Bacillus subtilis Candidatus Thiodiazotropha endoloripes Apis mellifera Homo sapiens Methanosarcina mazei Mus musculus Vigna mungo Solanum lycopersicum Saccharomyces cerevisiae Average
PEAKS 0.719 0.586 0.633 0.639 0.673 0.600 0.644 0.728 0.748 0.663
DeepNovo 0.742 0.602 0.630 0.610 0.694 0.623 0.679 0.731 0.750 0.673
PointNovo 0.768 0.589 0.644 0.606 0.712 0.626 0.730 0.733 0.779 0.687
Casanovo 0.749 0.603 0.629 0.586 0.679 0.689 0.668 0.721 0.684 0.667
Casanovo V2 · released 0.806 0.685 0.727 0.69 0.774 0.768 0.769 0.799 0.762 0.753
ContraNovo · released 0.828 0.706 0.761 0.771 0.798 0.799 0.804 0.808 0.782 0.784
ByNovo 0.858 0.723 0.791 0.767 0.823 0.803 0.836 0.828 0.804 0.804
Casanovo reverse decoding · retrained 0.804 0.699 0.728 0.719 0.769 0.776 0.782 0.795 0.762 0.759
ContraNovo reverse decoding · retrained 0.839 0.716 0.775 0.782 0.806 0.811 0.816 0.822 0.798 0.796
ByNovo reverse decoding 0.855 0.724 0.794 0.762 0.821 0.81 0.835 0.831 0.762 0.799
RankNovo 0.874 0.746 0.81 0.802 0.84 0.828 0.859 0.844 0.816 0.824
Nine-species benchmark
original (DeepNovo, 2017) 		Peptide recall
Bacillus subtilis Candidatus Thiodiazotropha endoloripes Apis mellifera Homo sapiens Methanosarcina mazei Mus musculus Vigna mungo Solanum lycopersicum Saccharomyces cerevisiae Average
PEAKS 0.387 0.203 0.287 0.277 0.356 0.197 0.362 0.403 0.428 0.322
DeepNovo 0.449 0.253 0.330 0.293 0.422 0.286 0.436 0.454 0.462 0.376
PointNovo 0.518 0.298 0.396 0.351 0.478 0.355 0.511 0.513 0.534 0.439
Casanovo 0.537 0.330 0.406 0.341 0.478 0.426 0.506 0.521 0.490 0.448
Casanovo V2 · released 0.646 0.46 0.527 0.492 0.592 0.493 0.628 0.637 0.629 0.567
ContraNovo · released 0.684 0.487 0.576 0.624 0.628 0.563 0.676 0.655 0.669 0.618
ByNovo 0.708 0.499 0.597 0.584 0.639 0.545 0.696 0.667 0.676 0.623
Casanovo reverse decoding · retrained 0.628 0.467 0.515 0.511 0.57 0.505 0.611 0.611 0.601 0.558
ContraNovo reverse decoding · retrained 0.682 0.499 0.583 0.606 0.621 0.566 0.673 0.654 0.664 0.616
ByNovo reverse decoding 0.703 0.493 0.59 0.554 0.637 0.543 0.685 0.659 0.629 0.610
RankNovo 0.738 0.539 0.63 0.642 0.672 0.583 0.733 0.691 0.703 0.660

RankNovo reranks the candidates of six base models. Casanovo V2 and ContraNovo are the released models; ByNovo is RankNovo's own in-house model, with an auxiliary b/y ion-annotation head (its Appendix B.4); R-Casanovo, R-ContraNovo and R-ByNovo are reverse-decoding versions of the three, trained by RankNovo's authors and recorded as the 'reverse decoding' variant. All six were trained on MassIVE-KB and evaluated zero-shot. '†' marks a model that is both baseline and base model, '⋆' one the authors trained. The plain Casanovo row repeats the values other papers print for Casanovo (0.749 and 0.537 on Bacillus) and the paper does not say where they come from. THE TEXT AND THE TABLE DISAGREE on V1: the prose gives RankNovo an amino-acid precision of 0.829 and a peptide recall '6.1%' above ByNovo; the table prints 0.824, and 0.660 against ByNovo's 0.623. The table's values are recorded.

Table7

Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencing, page 17: Evaluation of RankNovo in comparison to baseline and base methods on the 9-species-V2 test set. Bolded entries indicate the best-performing models. The symbol “†” indicates that the model serves as both a baseline and a base model. “⋆” signifies that the base models were developed and trained by us.

The same table is also printed in RankNovo: A Universal Reranking Approach for Robust De Novo Peptide Sequencing (Table2, page 8).

Nine-species benchmark
revised (main) 		Amino acid precision
Bacillus subtilis Candidatus Thiodiazotropha endoloripes Apis mellifera Homo sapiens Methanosarcina mazei Mus musculus Vigna mungo Solanum lycopersicum Saccharomyces cerevisiae Average
Casanovo V2 · released 0.888 0.791 0.823 0.872 0.877 0.813 0.891 0.891 0.915 0.862
ContraNovo · released 0.901 0.807 0.848 0.920 0.896 0.839 0.913 0.898 0.919 0.882
ByNovo 0.92 0.823 0.876 0.917 0.914 0.841 0.932 0.912 0.934 0.897
Casanovo reverse decoding · retrained 0.876 0.804 0.814 0.891 0.867 0.821 0.881 0.891 0.898 0.860
ContraNovo reverse decoding · retrained 0.909 0.815 0.865 0.923 0.901 0.849 0.919 0.907 0.925 0.890
ByNovo reverse decoding 0.919 0.822 0.879 0.912 0.912 0.843 0.932 0.913 0.936 0.897
RankNovo 0.926 0.838 0.885 0.929 0.920 0.860 0.938 0.918 0.938 0.906
Nine-species benchmark
revised (main) 		Peptide recall
Bacillus subtilis Candidatus Thiodiazotropha endoloripes Apis mellifera Homo sapiens Methanosarcina mazei Mus musculus Vigna mungo Solanum lycopersicum Saccharomyces cerevisiae Average
Casanovo V2 · released 0.793 0.558 0.669 0.712 0.754 0.555 0.772 0.783 0.837 0.714
ContraNovo · released 0.815 0.575 0.711 0.820 0.780 0.616 0.799 0.794 0.854 0.752
ByNovo 0.833 0.582 0.731 0.789 0.799 0.596 0.814 0.807 0.871 0.758
Casanovo reverse decoding · retrained 0.759 0.558 0.643 0.732 0.723 0.558 0.721 0.768 0.799 0.696
ContraNovo reverse decoding · retrained 0.821 0.581 0.719 0.815 0.779 0.620 0.804 0.798 0.861 0.755
ByNovo reverse decoding 0.831 0.585 0.729 0.781 0.794 0.589 0.815 0.803 0.873 0.756
RankNovo 0.851 0.620 0.752 0.820 0.813 0.629 0.836 0.822 0.885 0.781

RankNovo reranks the candidates of six base models. Casanovo V2 and ContraNovo are the released models; ByNovo is RankNovo's own in-house model, with an auxiliary b/y ion-annotation head (its Appendix B.4); R-Casanovo, R-ContraNovo and R-ByNovo are reverse-decoding versions of the three, trained by RankNovo's authors and recorded as the 'reverse decoding' variant. All six were trained on MassIVE-KB and evaluated zero-shot. '†' marks a model that is both baseline and base model, '⋆' one the authors trained. The plain Casanovo row repeats the values other papers print for Casanovo (0.749 and 0.537 on Bacillus) and the paper does not say where they come from.
In the paper: column peptide recall, Human: the original table only bolded RankNovo (0.820); ContraNovo (ContraNovo†) (0.820) ties with it and is bolded here too.

Papers describing it (3)

Authors (9)

Zijie Qiu, Jiaqi Wei, Xiang Zhang (Shanghai AI Lab), Sheng Xu, Kai Zou, Zhi Jin, Zhiqiang Gao, Nanqing Dong, Siqi Sun

Seen in the charts

Back to the full map

Back to top