ProteomeLM is a transformer-based language model trained on complete proteomes that produces contextualized protein embeddings and attention signals which recover protein–protein interactions unsupervised and support supervised PPI and gene essentiality prediction across diverse taxa.
0:00Welcome to Base by Base, the paper cast that brings genomics to you wherever you are. Thanks for listening, and don't forget to follow and rate us in your podcast app. So picture the inside of your body right now.
0:10Right, like down at the microscopic level. Exactly. If you zoom all the way down into a single one of your cells, you are not looking at some quiet, empty swimming pool. You are looking at a chaotic, overcrowded metropolis.
0:24Oh, absolutely. It's just packed. Yeah, you have 10s of 1000s of different proteins all packed together vibrating bumping into each other 1000000s of times a second. And yet somehow, in all that chaos, they know exactly who they need to partner up with to keep you alive.
0:40It really is the ultimate bustling city, you know? I mean, every metabolic process, every immune response, every breath you take relies on those specific proteins, finding their specific partners in the dark.
0:50Which is just mind blowing when you think about it. How could this change how we treat disease if we could actually map all of those interactions instantly? What really happens when a foreign virus enters that city and tries to find a partner?
1:02Well, that's the multi-billion dollar question, right? Because mapping that chaotic city has always been one of the hardest problems in biology. And that brings us to the core of our deep dive today. Today we celebrate the work of Mal Brank, Zolofi, and Bitball, who have advanced our understanding of the entire cellular dictionary.
1:21Yeah, their 2026 paper in the proceedings of the National Academy of Sciences, or PNAS, introduces something called Purdomel M. And I have to say, this isn't just an incremental step forward. This is a complete paradigm shift in how we understand the machinery of life.
1:37So the mission for our deep dive today is to explore how these researchers took artificial intelligence, specifically natural language processing, and applied it to the entire dictionary of a cell's proteins.
1:48We call that the proteium, right? Exactly, the protium. And by doing that, they created a tool that can map biological interactions up to a 1000000 times faster than we could before. And a 1000000 times faster.
1:59That is just staggering. It really is. To put that leap in perspective, you know, previous AI models in biology, focused primarily on single proteins or maybe small genomic neighborhoods, they were looking at life on a very microscopic scale.
2:13Okay, so let's unpack this. Are you listening? If those previous protein language models were like an AI learning the vocabulary and the grammar of, say, single isolated sentences. Yeah, like reading one line at a time.
2:26Exactly. Then Proteo melon M is like an AI reading an entire, massive novel, and understanding the complex hidden relationships between all the characters at once. I love that analogy. It's reading the whole book, not just a single page.
2:38But before we get into how this AI actually reads the book of life. We need to understand the agonizingly slow real world problem it was built to solve. Right. So the bottleneck we've been stuck with is the process of mapping protein, protein interactions or, you know, PPIs.
2:53As we mentioned, these PPIs are the fundamental mechanisms behind almost everything a cell does. So if a pathogen invades your body, The interaction between the viral protein and your cellular receptor is a PPI.
3:04Exactly. And if you want to develop a new therapeutic drug, you almost always need to target these interactions. Knowing the interactions is the whole ball game. But finding them experimentally in a lab is painstakingly slow, right?
3:16And incredibly expensive. Oh, painfully slow. I mean, we do have curated databases of these interactions, like the string database, but they are vastly incomplete. They are mostly biased toward well studied, simple organisms.
3:31So we turn to computers to fill in those massive gaps. We use computational methods to predict who is interacting with who. Yeah, and we have incredible tools now for predicting 3D protein structures, you know, things like alpha fold Multimer, which is brilliant at showing how 2 proteins physically fit together.
3:47But there's a catch with that, isn't there? A huge one. Structural modeling is computationally heavy. You can't just throw an entire organism's worth of proteins at it and ask it to find every possible pair.
3:58Right, it would take a lifetime. So what do researchers do instead? Well, they typically use sequence-based methods as a 1st filter. The gold standard for this has been something called direct coupling analysis or DCA?
4:09DCA. Okay, how does that work? DCA looks for evolutionary patterns, specifically co-evolution between the amino acids in different proteins. The idea is that if 2 proteins interact, the specific amino acids at their contact points have to maintain a sort of chemical lock and key fit.
4:25Right, so if one side of that lock mutates over 1000000s of years of evolution, the other has to mutate in a complementary way, or the organism dies. Exactly. So DCA looks for proteins that evolve together.
4:37That makes sense on paper, right? If they are adapting together across 1000000s of years, they are probably working together. Yeah, that makes logical sense. So where does the problem arise? The sheer scale of the math.
4:48DCA is a highly specific family by family model. It requires building a specific statistical model for every single candidate pair of proteins you want to test. Oh, wow. It's like trying to figure out which employees in a massive global corporation are collaborating on secret projects, but instead of looking at the whole network, you have to track and analyze every single email between every single pair of employees one by one.
5:13That is exactly it. You have to pull the complete evolutionary alignment for employee A, and the complete alignment for employee B, stitch them together, and build a custom model just for that one potential pairing.
5:25And doing that for every potential pair must be devastating computationally. Let's run the numbers. A simple bacteria like E. coli has about 4000 proteins. That means there are roughly 8000000 possible pairs to test.
5:39Just for a simple bacteria. Right. And humans have around 20,000 proteins. That creates about 200 million possible interacting pairs. Building 200 million separate evolutionary models is something that brings even modern systems to their knees.
5:53Wait, but with today's supercomputers and massive data centers. Is it really that big of a bottleneck? I mean, can't we just throw a massive server farm at it and brute force the math? You'd think so, but no.
6:03It restricts large scale interaction screens mostly to simple, well-studied organisms like E. coli or yeast. The old pipelines simply cannot scale effectively to give us a real-time global view of complex life.
6:16Which leaves us with huge blind spots. Massive blind spots when it comes to human biology or complex eukaryotes, or even poorly sampled pathogens that might be causing diseases right now. So because tracking those individual emails is mathematically crushing, The researchers decided to train an AI to just read the entire inbox at once.
6:36Yes, exactly. Tell us about the actual architecture of prodium LLM and how it bypassed this entire problem. Well, podiumenolm is a transformer-based language model. It's built on the same foundational architecture as the large language models that power modern AI chat bots.
6:52But instead of English text, it was trained on nearly 32,000 complete protiums. 32,000 complete dictionaries of cellular life. That is a massive data set. It is, spanning the entire tree of life from ancient Archaea to bacteria, human beings, and even viruses.
7:07Okay, so how does it actually learn from all that? Here is how it works. The researchers take individual protein representations using an existing sequence model called ESMC. That model already understands the basic spelling of individual proteins.
7:20Okay, so it knows the words. Right. They feed these protein embeddings into proteomel M. Then they do something called masked language modeling. Oh, this is the massive game of fill in the blank we talked about.
7:32Exactly. They take a proteo, and let's say a list of all 4000 proteins in a bacteria, and they randomly mask or hide 50% of them. Then they force the AI to reconstruct those masked proteins using only the unmasked ones as context.
7:48So the AI has to look around at the 50% of the proteins it can see and deduce, okay, based on who is present in this cell, I predict the missing proteins must be X, Y, and Z. You got it. And by forcing the model to repeatedly guess the missing pieces across 32,000 different species, it is forced to learn the deep underlying dependencies between the proteins.
8:10Ask to learn why certain proteins always show up together across the tree of life. Exactly. But there is a massive innovation here that we need to unpack. It has to do with how the AI actually organizes this information before it makes those guesses.
8:21Right. The difference between positional encoding and functional encoding. Yes, and this is perhaps the most elegant part of the paper. Standard language models, like the ones that process English, use what is called positional encoding.
8:33Because the AI needs to know exactly where a word sits in a sentence. Right. The order of words dictates the grammar. Exactly. The dog that the man is very different from the man with the dog, location matters.
8:45And early genome models use positional encoding too, because in bacteria, functionally related genes are often physically grouped together on the DNA. In those structures called operon. So reading them left to right makes sense.
8:57Right, but eukaryotes, like humans, don't work that way. Our genomes are much messier. Our genes are not neatly ordered at all. So their physical location on a chromosome might not have any correlation with their function.
9:09Often absolutely no correlation with who they interact with. So if you try to train an AI on human proteins based on their physical order on the genome. The AI will just get confused. The signal is lost in the noise.
9:22Wow. So how do they fix a messy human genome for the AI? They threw out positional encoding entirely. Instead, they used functional encoding. They utilized a database called orthodb, which groups genes into orthologous groups.
9:38Okay, what does that mean in plain English? It basically traces the evolutionary family tree of the genes, showing which proteins share a common ancestor and generally have the same function across different species?
9:50The AI is fed this evolutionary hierarchy instead of physical genome locations. So instead of organizing a massive library by alphabetical order on the physical shelf, they organized it by genre and historical theme.
10:03That's a perfect way to put it. But wait, if the AI doesn't know where the proteins physically sit in the genome, how can it reliably guess how they interact without just taking shortcuts? Ah, the researchers anticipated that?
10:14They knew the air might get lazy. If it knows 2 proteins are in the same functional family based on that database, it might just copy that family label and guess they interact without actually learning the underlying biological rules.
10:25Okay, so how do you stop an AI from cheating? To prevent this. They designed a custom mathematical rule called a polar loss function. A polar loss function. What does that actually do? Think of it like a mathematical blindfold combined with a strict scoring system.
10:41It essentially penalizes the AI during training, if it relies too heavily on those broad family categories, and forces it to look for contrasting signals. So by removing the physical location data and mathematically preventing the AI from cheating with the functional categories, the model had to learn the actual biological logic of the entire system.
11:01Exactly, which brings us directly to the aha discovery of this paper. When the researchers finally looked under the hood to see how the AI was making its guesses, they found something incredible. Right.
11:13They looked at the model's attention coefficients. Yes. In a transformer model, attention heads are basically the mathematical weights that tell the AI, which parts of the context are the most important when it's trying to fill in a blank.
11:25It's like the AI showing its work. highlighting the specific clues it used. And what did the AI's math reveal? When they mapped out these attention coefficients? They discovered the model had naturally learned protein interactions.
11:39Keep in mind, this model was trained on pure, unlabeled data. So it was never explicitly taught what an interaction was. Never. It was never fed a database of interactions during training. You're saying this AI basically taught itself cellular biology just by playing a massive game of filling the blank with 32,000 prodiums.
11:58Yes. And the accuracy is astounding. When they looked at E. coli, they found that just one single attention head, specifically head 7 of layer 3 in their network, could predict known protein interactions with an area under the curve, or AUC, score of .92.
12:13Wow. For context for you listening, an AUC of one. is absolute perfection, and .5 is basically coin toss. So .92 without even being taught what an interaction is, that is highly predictive. Highly predictive.
12:26And they found that the middle layers of the neural network held the richest data. Why the middle layers? Well, in AI language models, early layers often capture simple local features, while middle layers capture complex, higher order relationships.
12:38The fact that the interaction signal was strongest in the middle layers proves the model was truly capturing system level biological logic. But proteins interact in different ways, right? Sometimes they physically touch and bind together, but sometimes they just operate in the same neighborhood or belong to the same pathway.
12:56So could the AI tell the difference between 2 proteins actually touching and 2 proteins just working in the same factory? It could perfectly disentangle them. Proteomalm didn't just find broad genetic associations, like proteins that happen to be turned on at the same time.
13:12Okay. It could distinguish between those broad associations and direct physical binding. It could even identify same complex interactions. Like parts of a machine physically interlocking together. Exactly.
13:23They tested it on the E. coli ribosome, which is a massive complex molecular machine made of many different proteins. Prodeomilms, attention heads, could isolate the proteins belonging to that specific complex from the 1000s of other proteins in the cell with an AUC of over .99.
13:42So it mapped the parts list of a microscopic machine just by reading the evolutionary context. That's wild. It separated the specific parts of that machine from the noise of the wholesale almost perfectly.
13:54Okay, so if the AI is reading the entire cellular dictionary at once, instead of looking at individual pairs, what does that actually mean for the researcher sitting at the computer? How much time does this save when unleashed on the real world?
14:06This is where the theoretical brilliance translates into massive practical application? Remember the direct coupling analysis, the DCA method we talked about earlier? The one that requires a custom model for every single pair of proteins.
14:17Right. Recently, a major study used that DCA pipeline to screen the entire human interactome. That's about 1900000 protein pairs. To do that, it took them over 30 days, running constantly on a cluster of 50 to 100 high powered GPUs.
14:35A month of solid supercomputing just to get a baseline screen for human biology. Now, compare that to ProteomeLM, because Proteo LM processes the entire dictionary at once and calculates the attention weights for all possible parasimultaneously.
14:48It processed the entire human protium on a single GPU in under 10 minutes. Wait, under 10 minutes, one GPU, 10 minutes versus 100 GPUs for 30 days. Exactly. It represents up to a 6 order of magnitude reduction in inference compute time.
15:04A 10000 times faster. Yes. I mean, imagine you're a virologist staring down a novel pathogen that just emerged, waiting 30 days for a supercomputer to map its interactions could mean a global crisis. Right, time is everything.
15:16Doing it in 10 minutes on a single GPU changes the entire timeline of human response. Finding a drug target for a rare disease can now happen at a speed that was physically impossible just a few years ago.
15:27And the best part is, it's not sacrificing accuracy for that speed either. It's actually more accurate. For human proteins, the proteomillum, unsupervised model, hit an AUC of .83 compared to DCA's .73.
15:41So it's better at it too. Much better. When they looked at the top 10000000 scored pairs, Proteom LM recovered 50% of the experimentally validated known interactions. DCA only recovered 20%. It's a 1000000 times faster and more than twice as effective at finding the needle in the haystack.
15:58They also tested it on 19 different human bacterial pathogens, over 102 million protein pairs in total. Porteomilla maintained high accuracy scores across the board, ranging from .87 to .92 for these pathogens.
16:12It generalizes incredibly well. As a raw unsupervised screening tool, it's a total game changer. But the researchers wanted to push the boundaries of what this whole cell awareness could achieve, didn't they?
16:23They did. They wanted to move beyond just mapping interactions to predicting survival itself. Right. The unsupervised model is brilliant for rabbit screening. But to get state of the art predictive power, the researchers built supervised models on top of the base AI.
16:36Yes. The 1st was called Proteomel MPPI. They took the data about the individual proteins themselves, what data scientists call node features, and they mathematically combined it with the AI's attention weights.
16:48Which represent the connections between them or edge features. Exactly. By training a specialized classifier on this combined data, they achieved state-of-the-art interaction predictions across multiple species.
16:59But the 2nd model they built is the one that really caught my eye, ProteomeLMS. Ah, yes. This model is designed to predict gene essentiality. For anyone listening who is in steep in biology, what exactly does it mean for a gene to be essential?
17:14Gene essentiality asks a literal life or death question. If we remove or mutate this specific gene, will the organism die? Is this protein essential for survival or is it just a nice to have accessory?
17:28Like removing the steering wheel of a car versus removing the radio. Great comparison. And predicting this is incredibly difficult because essentiality isn't just about the sequence of a single gene. It depends heavily on the entire cellular context.
17:40Because you have to ask, do other proteins compensate for it? Does removing it break a vital metabolic network? Exactly. And because proteomelm naturally understands the whole network, it is uniquely suited for this kind of prediction.
17:55So how did they test it? They tested proteom LMS on experimental data from diverse species. The embeddings from the deeper layers of the model produced highly accurate predictions for organisms like yeast and E. coli that were completely hidden from the AI during training.
18:11But the ultimate test of its generalization power came when they fed it, the genomes of synthetic, lab engineered minimal cells, right? Yes. You're talking about organisms like JCVISIN, one.0, and JCVISIN 3A.
18:24Right. These are organisms whose genomes were literally designed and synthesized by humans in a laboratory, specifically at the J. Craig Venter Institute. Exactly. The researchers there wanted to know what the absolute minimum requirements for life were.
18:36So they started with a natural bacteria and systematically stripped away every single gene they could to create the simplest possible living cell. It is a biological house of cards. Almost every remaining gene is vital.
18:47But how does proteomelm handle a synthetic cell? I mean, predicting what a natural bacteria needs to survive makes sense, it has 1000000000s of years of evolutionary history that the AI can read. Yeah. But a lab made minimal cell didn't evolve naturally as a redundancies.
19:05It's an alien landscape. Doesn't an engineered cell break the evolutionary rules the model learned? You would think so. But the AI's performance here is the most profound revelation of the entire paper.
19:16If the AI was simply memorizing evolutionary trivia, it would fail completely on a synthetic cell. Because that cell lacks the normal evolutionary contact. Exactly. But proteomel MS was able to predict essentiality for these synthetic cells with incredibly high accuracy.
19:31Wait, so it's successfully predicted which genes were life or death in an organism created by humans? It did. That success proves that the model has internalized the deep foundational rules of cellular machinery.
19:44That is wild. It understands the fundamental physics of the protium so well that it can look at an unnatural human engineered network and correctly deduce which parts are load bearing. It proves that even synthetic biology must obey the hidden biological grammars that proteo MLM has decoded.
20:03Let's take a breath, because we have covered massive ground today. We started with the sheer chaos of the cellular metropolis and the immense month-long computational bottlenecks we use to face, just trying to map a single species interactions.
20:17We've seen how treating the entire protium as a single cohesive language completely bypasses that bottleneck. Yeah, by organizing data functionally through evolutionary history rather than physically along the messy human genome.
20:28Proteomelm cracked the code. It naturally discovered protein networks without being told what they were. It sped up the analysis by up to a 1000000 times, and it successfully generalized the fundamental rules of survival to the entire tree of life, even artificial life in a Petri dish.
20:44It's an absolute triumph of AI in biology. But if we pull all these threads together, what does this mean for the future? It leaves us with an incredibly provocative question. It does. Right now, we are using this AI to read the book of life.
20:57It looks at an existing protium and infers the rules, what interacts, what is essential, but if a language model truly understands the fundamental grammar of life so deeply. Well, what happens when we start using these systems to write?
21:11Could we eventually prompt an AI to authorship? Could we ask it to generate entirely novel synthetic protium from scratch? Like a complete organism designed on a computer that perfectly obeys these hidden biological grammars?
21:24A completely alien, fully functional cell, dreamed up by an AI? That is a thought that will stick with you long after you take your headphones off. The immense chaotic city inside your cells is finally being mapped, and the tools doing the mapping might soon be capable of building new cities entirely.
21:40This episode was based on an open access article under the CCBY 4.0 license. You can find a direct link to the paper and the license in our episode description. If you enjoyed this, follow or subscribe in your podcast app and leave a 5 star rating.
21:55If you'd like to support our work, use the donation link in the description. Now stay with us for an original track created, especially for this episode, and inspired by the article you've just heard about.
22:05Thanks for listening, and join us next time as we explore more science, base by base.
