IndeLLM uses protein language models (ESM2) to score in-frame indels and a compact Siamese transfer-learning model that achieves state-of-the-art pathogenicity prediction with MCC = 0.77.
0:00Welcome to Base by Base, the papercast that brings genomics to you wherever you are. Thanks for listening, and don't forget to follow and rate us in your podcast app. I want you to start by imagining you're sitting in a comfortable chair, reading a classic novel.
0:13You're deep in the story, the rhythm of the words is perfect, the plot is moving, and then suddenly you turn the page, and someone has ripped out a random paragraph. Or maybe even weirder. They've glued in a page from a completely different book right in the middle of a sentence.
0:30You would be totally jarring. You'd lose your place completely. Exactly. Now, if there was just a typo, a single letter changed, you'd probably just skip over it. Your brain autocorrects you get the gist.
0:40Sure. But when you change the length of the text, when you add or delete big chunks, the whole rhythm breaks. The page numbers shift. The chapter might suddenly just turn into nonsense. And that is actually the perfect setup for the problem we're tackling today.
0:54In genetics, we've become incredibly efficient at spotting those single letter typos. We call them SMPS single nucleotide polymorphisms. We can find them, we can read them. We understand them pretty well.
1:06We do, but the problem of the ripped out paragraph, that's a whole other story. We call those indels insertions and deletions. And for a long time, our computer models, well, they looked at these indels and just sort of panicked.
1:18Panicked. Why? Because most of our AI models, they expect a fixed structure, a grid. When you stretch or shrink the DNA sequence, the computer really struggles to compare the before and after. So the central question for this deep dive is, how do we teach artificial intelligence to read a sentence that keeps changing size?
1:37And maybe more importantly, what happens to our ability to, you know, cure disease when we finally figure it out? It's a huge question. And the answer suggests that solving this isn't just about reading the code better.
1:49It's really about understanding the hidden grammar of protein stability. It opens the door to predicting diseases that, until now, we're basically invisible to our best tools. Okay, let's unpack this, but 1st we need to give credit where it's due.
2:04The paper we're covering today comes from a powerhouse international team. It really does We're celebrating the work of Oriole, Gracia Carmona, Vilda LA part, and their colleagues at University College London, King's College London, and the Norwegian University of Life Sciences.
2:19And we should also mention the contributions of Grovi MDM, Christina Rango and Franco Fertrinali. They've really pushed the boundaries of what these protein language models can do. So let's set this scene on the biology side before we get into the AI.
2:34We always hear about mutations, but usually people are talking about those single letter swaps. How big of a deal are these indels, really? Are they rare? No, not at all. They aren't rare at all. We often overlook them because they're mechanically trickier to analyze, but Indels actually make up about 18% of all human genetic variation.
2:5318%. Wow. Almost one in five. is definitely not a rounding error. No. And we need to make a distinction here because not all indels are created equal. You have frame shift indoles and you have in frame indoles.
3:04Okay, walk us through the difference. So a frame shift in Dell is usually catastrophic. Remember, DNA is red in groups of 3 letters, right? Codons. If you delete one letter, just one, every single word after that point gets shifted over.
3:20It turns the rest of the book into gibberish. Total gibberish, the machine just stops working. The protein probably doesn't even fold correctly. Right. Exactly. those are usually obvious. But today we're interested in the in frame in Dells.
3:33This is where you insert or delete a multiple of 3 letters. So you might lose exactly one whole word or add exactly 2 whole words. So the reading frame is preserved. The rest of the sentence structure stays intact.
3:44It's like taking the sentence the cat sat on the mat and just removing cat. You get sat on the mat. It's grammatically broken, but you can still ring the words. That's a great way to put it. The protein is still made.
3:54It still folds, but it might be missing a critical gear or have an extra lever that just gets in the way. These are the sneaky ones. They're very sneaky. They're linked to cystic fibrosis, childhood cataracts, various cancers, but because they don't break the whole code.
4:08There effects are incredibly hard to predict. And historically, how have we even tried to guess what these mutations do? I mean, if the protein still looks mostly right, how do you know it's broken? Poorly, to be honest.
4:22The tools were very human centric. They relied on manual annotations. Basically notes that scientists had already written in the margins of the human genome. So if you had a mutation in a species, we hadn't studied as much, you were just out of luck.
4:36Pretty much. Or they'd look for conservation, you know, is this part of the protein usually the same in every animal? But that's all based on what we already know. We want to find the stuff we don't know.
4:44Precisely. And that's where the game changer comes in. Protein language models or PLMs. Think of these like chat GPT, but for biology. Models like ESM 2 have been trained on 1000000000s of protein sequences from bacteria to humans.
4:59They've learned the language of biology without us having to teach them the explicit rules. So the model just knows that, I don't know, subject usually implies verb or in protein terms, this shape usually implies that function.
5:11Correct. But here is the problem, and it goes right back to your book analogy. PLM's work by calculating probabilities. They look at a sequence and say, how likely is this sentence? But if you have a wild type protein that is 100 amino acids long and a mutant that is 95 amino acids long.
5:28You can't compare the scores. You really can't. It's mathematically invalid. It's like trying to compare the weight of a single apple to the weight of a fruit salad. Of course, the fruit salad is heavier, it has more stuff in it.
5:39The raw probability score of the shorter sequence will almost always look different just because of the length, not because of the biology. So this is where the new paper comes in. They introduced a solution called Indel Elm.
5:50And they had to invent a new way to score these changes to make the Apple, you know, comparable to the fruit salad. They did. They call it their 0 shot scoring approach. And 0 shot just means using the AI model as is, without training it specifically on disease data.
6:08So how did they fix the length issue? Do they just chop the fruit salad in half? It's a bit more elegant than that. Instead of scoring the whole protein, they ask the model to calculate the probability only for the overlapping regions.
6:20Ah, the parts that are in both versions of the book. Exactly. If you insert a paragraph, they ignore that new paragraph for the score calculation. They just look at the text around the insertion. Did that insertion make the surrounding text feel awkward to the model?
6:35Did it disrupt the flow of the story? Right. Did it disrupt the flow? By doing this, they make the math comparable again. That makes a lot of sense. It's like focusing on the seams. But I noticed in the paper, they also change the way they do the math itself.
6:48They moved away from something called log sums. Now, I haven't done calculus in a while, so help me out here. Why does that matter? It matters a lot for accuracy. Usually a scientists use the sum of log probabilities.
7:02It's a standard statistical trick because these probability numbers get very, very small, and logs make them easier to handle. But the researchers found that log sums tend to overemphasize mistakes in noisy regions.
7:14Easy regions. Yeah, parts of the protein that aren't very important, evolutionarily speaking, regions that change a lot between species, sort of like junk DNA or flexible loops on the outside of the protein.
7:26So the model is naturally unsure about those regions anyway. It is. They vary so much. If you use a log some calculation, that uncertainty, it just accumulates, it screams louder than it should. So if you have a long protein with a lot of these junk sections, The log sum makes it look like a bad protein, even if the important parts are perfect.
7:46The noise drowns out the signal. That's exactly it. It's like trying to listen to a song on the radio, but the static is turned up to volume 10. So they switch to a simple sum of probabilities. It prevents the noise in the junk regions from overwhelming the signal in the important conserved regions.
8:00It sounds like a minor tweak, but it cleans up the signal significantly. So that's the scoring method, the math part. But they didn't stop there. They also built a brain to process this data. They call it the Siamese network.
8:13Now, whenever I hear Siamese, I think of twins. That is the right image, a Siamese network is a twin network. But before we get to the twins, we have to talk about transfer learning. Okay, let's unpack that first.
8:26Imagine you have a genius professor. That's the giant ESM 2 model. It knows everything there is to know about protein grammar. But it's too big and busy to look at every single patient case. So what you do is you take the thoughts, or technically the embeddings from that genius professor, and you feed them into a smaller, more specialized student.
8:46And that's the Siamese network, the student. Yes. They took the mathematical representations from the giant model and fed them into a really simple neural network with only one hidden layer. Why only one layer?
8:58If you're building a brain? Why not make it super complex? wouldn't a deep network be smarter? Not necessarily. In AI, there's a danger called overfitting. If you make the student too smart, they just memorize the answers to the test.
9:10They don't learn the actual concepts. By keeping the network simple, they force it to learn the general rules of what makes a protein go bad, not just memorize the specific mutations that were in its training data.
9:22So it's forcing the model to actually understand the logic. Okay, so why is it a Siamese network? Where do the twins come in? Because it processes 2 inputs at once. It looks at the healthy protein and the mutated protein in parallel like twins, and it compares them.
9:37But here is the critical design choice. In the paper, they call it Model 4. This was the one that really worked. It was. In previous attempts, people just shoved the whole sequence into the model and said, you know, good luck.
9:49But Model 4 explicitly separated the data. It said, okay, computer, here is the math for the overlapping parts, the context. And separately, here is the math for the actual insertion or deletion. It's like telling the computer.
10:01Here is the sentence structure, and separately, here is the new word that was added. Judge them both, but judge them as distinct pieces of evidence. Precisely. By splitting the input, the model can weigh the evidence differently.
10:15It can say, hmm, the context looks fine, but this inserted piece, this piece is trouble. That separation turned out to be the key to everything. So let's get to the scorecard. Did this new method actually work?
10:27Let's start with the 0 shot method, the one that just looks at the context and uses the new math. It did well. It achieved a Matthews correlation coefficient or MCC of .65. And just to orient us, MCC is basically a scorecard where one is a perfect prediction and 0 is just random guessing.
10:42Right. So .65 is quite solid. It beat the standard scoring method, which was sitting at .58. And importantly, it performed just as well as these complex models that were specifically trained on human data, even though the 0 shot method was just using general biological rules.
10:58But the real heavy hitter was the Siamese network, the twin student. Absolutely. The Siamese network achieved an MCC of .77. That sounds like a big jump. It is massive in this field. It beat the defending champions.
11:10There are standard tools scientists use like Provian and Muttpred Indel. Provian scored a mere .73. So Indel M outperformed the state of the art by about 5.5%. That's significant. But I want to talk about why it was better because looking at the data, there was a specific aha moment regarding false negatives.
11:31Yes, this is fascinating. So the 0 shot model, the one that ignored the inserted text, it was pretty bad at predicting pathogenic insertions. It had a 19% error rate there. It kept saying harmful insertions were benign.
11:44Why? Was it just missing the clue? Because the inserted amino acids themselves were statistically likely. The new paragraph wasn't gibberish. It was written in perfect English, so the model looked at the context, saw the new text look totally normal, and said, this is fine.
11:59But it wasn't fine. No. Even a perfectly written paragraph can ruin a book if it contradicts the plot. The inserted sequence looked biologically valid, but it was breaking the propane structure. And the Siamese network caught this.
12:11It did. Because Model 4 analyzed the inserted sequence separately from the context, it realized that even good looking insertions can be structurally damaging. It stopped getting fooled by the pretty grammar.
12:23Exactly. And the error rate, I mean, it just plummeted. It went from 19% all the way down to 6%. That really validates the decision to split the input. It's almost like the model developed in intuition for context.
12:35Exactly. Now, numbers are great, but we love real world examples. The paper dove into some specific diseases to show interpretability. Let's talk about FGFR1. Fiberblast growth factor receptor one. It's a crucial receptor for cell growth.
12:49The researchers looked at a specific mutation, a deletion of a single methynine at position 535. Just one amino acid gone, one beat off the string. Just one. But when they ran it through into LM, the model showed a massive drop in probability in the surrounding area.
13:05And when they mapped this to the 3D structure of the protein, they saw exactly why. What was he hitting? It destabilized the alpha C helix. In the world of Kine's enzymes, that helix acts like the on switch.
13:16If you break that helix, you break the switch. The enzyme can't function. So the model didn't just say bad. It pointed to the map and said the switch is broken. That's it. And they saw something similar with a protein called GLMN, or glomulin.
13:32There was a deletion of an asparagin at position 393. And what did the model see there? It predicted destabilization across the entire C terminal domain? That's the tail end of the protein. Now, biology tells us that this specific domain is essential for binding to a larger complex.
13:48So the model correctly predicted that this mutation destroys the protein's ability to, you know, hold hands with its partners. That interpretability is huge. It's the difference between a black box and a real diagnostic tool.
14:00If a doctor knows why a mutation is bad, they might be able to figure out how to bypass it. It is. And it brings us to the broader implications. Because this isn't just about human health. Right. You mentioned earlier that these models are trained on everything.
14:12That's the beauty of generalizability. Because PLMs are trained on bacteria, plants, animals, everything. Indilem isn't limited to human diseases. The paper actually mentions using this for agriculture, specifically honeybees.
14:26Honeybees. We're analyzing BDNA with this. Yes. Honeybee populations are collapsing, and we need to understand their genetic resilience. If you want to spot a harmful mutation in a bee queen, you can use indelum, you don't need a bee specific AI model.
14:40The language of proteins is universal. That's incredible. It really highlights how these language models are capturing the fundamental physics of biology, not just memorizing human symptoms. It also opens doors for synthetic biology.
14:53If we want to design new proteins to eat plastic or generate energy, we need to know how to edit them without breaking them. This tool helps us predict those edits. And to help us see that physics, they also focused on visualization.
15:06Yes. They created a way to paint the 3D structure of a protein with these probability scores. So if you're a researcher, you can look at a 3D model on your screen and it lights up. Exactly. If a region turns red, you know exactly where the Indel is causing mechanical stress.
15:22It makes the abstract math visible. It helps you form a hypothesis about why the disease is happening. We do have to be fair though. Are there limitations? Is this a magic bullet? Not quite yet. The 0 shot method, as we discussed?
15:36Still ignores the identity of the inserted text to some degree. It just looks at the context, so it might miss gain a function mutations. Gain a function, meaning the protein doesn't break. It actually starts doing something new and dangerous.
15:47Exactly. If the new piece of code adds a new capability, like a toxic activity, the context score might miss it because the structure looks fine. The Siamese network is better at this, but it's still a challenge.
15:58And what about really big changes? Yeah, very large Indels like deleting a whole chapter of the book are still tricky because there's just so little context left to analyze. So for a tool that you can run on a browser, it's impressive.
16:10That is a key point. They released a plug and play Google Collab notebook. You don't need a supercomputer. You can run this analysis on a laptop that really democratizes this kind of high-level research.
16:22So let's wrap this up. What is the main take home message here? We used to think changing the length of a protein sequence was just too complex for simple scoring. It was in the too hard pile. Indel M proves that if you ask the AI the right question, specifically by focusing on the overlapping context and treating the insertion separately, you can predict disease better than ever before.
16:43It's about asking the right question. It always is. And if I can leave you with one thought. If we can now accurately predict the structural impact of adding or removing entire paragraphs of genetic code, what does this mean for our ability to rewrite those paragraphs to cure disease, we're moving from spell checking to editing.
17:01That is a powerful thought to end on. This episode was based on an open access article under the CCBY 4.0 license. You can find a direct link to the paper and the license in our episode description. If you enjoyed this, follow or subscribe in your podcast app and leave a five-star rating.
17:17If you'd like to support our work, use the donation link in the description. Now, stay with us for an original track created especially for this episode, and inspired by the article you've just heard about.
17:27Thanks for listening and join us next time as we explore more science base by base.
