Protein representations from deep language models have become the state-of-the-art across many tasks in computational protein engineering. In recent years, progress has focused primarily on model scale: exponentially growing amounts of compute, with later models reaching sizes larger even that the datasets they were trained on.  We propose an alternative direction considering the codons. We show that large language models trained on codons, instead of amino acid sequences, provide high-quality representations that outperform state-of-the-art representations across a variety of tasks. This reaffirms the importance of codons – they are predictive of properties beyond the amino acids they encode for. These results also suggest that, in addition to commonly studied scale and model complexity, the richness of biological data provides an orthogonal direction to improve the power of machine learning in biology.

Language of proteins is in the codons not the amino acids

Where thousands of chemistry professionals meet to share ideas and advance scientific and technical knowledge. Spring 2023 abstracts, posters, and presentations will launch at the end of March.

ACS Spring 2023

The official archive and repository for all ACS Meetings & Expositions abstracts and select posters or presentations.