CodonRoBERTa beats ModernBERT in mRNA optimization
openmed introduced codonroberta-large-v2, a transformer for mrna optimization that outperforms modernbert. trained across 25 species for $165 using 55 gpu-hours, the model achieves a perplexity of 4.10 and spearman cai correlation of 0.40.
OpenMed released an end-to-end protein AI pipeline featuring CodonRoBERTa-large-v2, a new transformer architecture optimized for mRNA sequences. The model achieved a perplexity of 4.10 and a Spearman CAI correlation of 0.40, surpassing ModernBERT in handling the statistical properties of 64-token codon alphabets. The team scaled training across 25 species for a total cost of $165, requiring only 55 GPU-hours.