We summarize a psychometric validation of the EAGL measure using US adult online samples. The study produced a validated 17-item EAGL-short that captures three core genetic literacy constructs and can be used to assess and target genetic communication and education.
0:00Welcome to Base by Base, the paper cast that brings genomics to you wherever you are. Thanks for listening, and don't forget to follow and rate us in your podcast app. So uh, we live in this really fascinating era right now where getting your DNA sequence requires like almost 0 effort.
0:14Oh, absolutely. incredibly easy. You just spit in a plastic tube. You mail it off in this little prepaid box, and then, you know, a few weeks later, an app on your phone just lights up with this massive super colorful dashboard of your entire genetic code.
0:28Yeah, and it throws so much at you all at once. It really does. It tells you about your deep ancestry. Uh, your likelihood of having a unibrow. And then far more seriously, your inherited health risks for things like heart disease or cancer.
0:42Which is a lot to process for anybody. Exactly. And it feels incredibly empowering to hold that much data in the palm of your hand. But this introduces a highly relatable modern problem, I think. We are increasingly bombarded with this direct to consumer genomic data.
0:58And yet, do we actually understand what we are reading? Right. Do we really get what those risk scores mean? Exactly. What really happens when we think we know more about our DNA than we actually do. Because honestly, reading a DNA result without true genetic literacy is like trying to use a foreign language dictionary to perform surgery.
1:17That is a perfect way to frame it. Because the gap between simply possessing that data and actually comprehending how all those variables interact, I mean, it's vast. Right. You might know what the words mean technically.
1:29Yeah, we can have all the access in the world. But if the foundational understanding isn't there, sitting at your kitchen table looking at a complex health report can easily lead to confusion or false confidence or, you know, ultimately poor medical decisions.
1:44Which means before we can even begin to fix this gap in our collective understanding. We have to figure out how to accurately measure it in the first place. Exactly. You can't fix what you can't measure.
1:54So today, we celebrate the work of the engagement methods unit at the social and behavioral research branch of the National Human Genome Research Institute, along with our collaborators, who have advanced our understanding of genetic literacy.
2:07But, you know, something to say we need genetic literacy. How do you even begin to measure something so abstract and complex in the general public? To answer that, we really 1st need to define the target.
2:20I mean, genetic literacy isn't about handing someone a high school biology test and just seeing if they remember what mitochondria do. Right, it's not just trivia. No, not at all. It is strictly defined as having sufficient knowledge and understanding of genetic principles to actually make personal decisions about your well-being.
2:36And to effectively participate in social decisions regarding genetic issues. It's highly practical. Okay, let's unpack this. Because when you say the word literacy, my mind immediately goes to reading comprehension, or maybe like general health literacy, we hear a lot about health literacy and public health campaigns, how is genetic literacy different from just generally understanding health?
2:57So health literacy is essentially your ability to navigate the healthcare system itself. It's, you know, reading a prescription label to know how many pills to take or knowing the difference between an urgent care center and a primary care doctor.
3:09Yeah, that logistical stuff. Right. Then there is numeracy, which is how facile a person is with basic math and probability. Like, can you convert a percentage into a proportion? Can you understand a one in 4 chance?
3:21Which seems like it would be a huge deal in genetics. Oh, it is. Those are incredibly important foundational skills, and they interconnect deeply with genetics, but rigorous discriminate validity studies have shown that genetic literacy is its own completely distinct construct.
3:38Really, completely distinct. Yeah, you can be highly health literate, and you can be great at math, but still be completely lost when a genetic counselor starts talking about polygenic traits, where dozens of genes interact simultaneously with your environment.
3:52So if it's this highly specific super nuanced thing, how were doctors and researchers testing for it historically, were we just handing people a vocabulary test? Historically, yes, that is heavily what we relied upon, and it created a major blind spot in the field.
4:08Wow. Yeah. older measures, like the rapid estimate of adult literacy and genetics, which is known as real G, or the genetic knowledge index, the GKI. They had 2 critical flaws. First, they often lacked rigorous psychometric validation.
4:24Meaning what, exactly? Meaning the tests themselves weren't stress tested to ensure they consistently measured what they actually claim to measure. And second, they primarily measured subjective knowledge, which just means rating your familiarity with terms or basic objective knowledge, like simple, true, or false facts.
4:41They completely missed a third, crucial dimension called knowledge comprehension. Oh, interesting. So previous tests were like asking someone if they recognize a steering wheel rather than testing if they can actually drive the car in traffic.
4:53That is exactly the distinction. Knowledge comprehension is the ability to interpret and apply complex genetic information when it's presented in a real world context. Like actually using the information to make a choice.
5:05Right. For example, if a genetic counselor hands you an infographic, explaining how your genetics and your diet interact to reach a threshold for a complex condition, like diabetes. Can you read that infographic, synthesize the variables, and answer questions about your actual risk.
5:22The older tests simply didn't measure that applied logic. Right, because just knowing the word mutation tells the doctor absolutely nothing about whether you understand how a mutation operates in the real world to affect your health.
5:34Exactly. Or a better driving test, going back to your analogy. They created the EAGL measure, which stands for the education and assessment of genetic literacy. But to build a test that complex. I mean, you can't just write down a few questions and assume they work for everybody.
5:51How did they actually construct and validate this EAGL measure? Well, they employed a highly rigorous methodology. They used sequential sampling of 2,708 English speaking adults in the United States. That is a pretty massive sample size.
6:05It is, and they recruited these participants via an online research platform called prolific. The team started with a huge 46 questions survey known as the EAGL long. Okay. And to refine this unwieldy survey into a highly validated, efficient tool that a clinic could actually use on a daily basis.
6:24They applied 2 powerful statistical techniques, exploratory factor analysis and confirmatory factor analysis. I have to admit, those terms sound a bit intimidating. Exploratory and confirmatory factor analysis.
6:37What is the actual mechanism there? Think of it like organizing a massive, really messy closet. Okay, I can picture that. So exploratory factor analysis is the process of looking at the giant pile of items and letting the statistics reveal what naturally groups together.
6:51You throw all 46 questions at the 1st large group of participants. The math then reveals underlying patterns based on how people answer. Oh, so clusters things that are essentially asking the same thing.
7:02Exactly. It shows you that, say, questions 4, 12, and 19 are essentially measuring the exact same underlying trait. So you can then remove the redundant questions or discard the ones that just confuse people and don't group with anything.
7:16So if exploratory analysis is looking at that massive pile of clothes and deciding to group all the shirts together and all the pants together to find the natural categories, I assume the confirmatory part is taking that exact same closet organization system to a completely stranger's house to see if it holds up with entirely different clothes.
7:34That perfectly describes the mechanism. Yes. Once you establish your neat organized categories with the 1st group, you run a confirmatory factor analysis on a completely new group of people. To make sure it wasn't just a fluke.
7:46Right. It confirms whether the structural integrity you found actually holds up universally. And through this dual process, they whittled that heavy 46 question EGL long down to a highly validated lean 17 question test, which they call the EGL short.
8:02That is brilliant. And what's fascinating here is that through all of that rigorous statistical sorting. The analysis naturally validated 3 distinct core constructs of genetic literacy. So the data just cleanly separated into 3 buckets on its own.
8:16It did. The 1st construct is subjective knowledge. This is what you think you know. Participants self-rate their familiarity with words like mutation or genome or heredity on a scale of one to seven. Okay, so just self-reported confidence.
8:30Right. The 2nd construct is conceptual knowledge. This is what you actually know, like objective, true or false facts about DNA and inheritance. And the third, the critical missing piece from those older tests is knowledge comprehension.
8:43Ah, the driving and traffic part. How did they simulate that real world application in a 17 question survey? They uniquely tested it by embedding a practical exercise right in the survey. They had participants read an infographic about autism.
8:57Yeah, the infographic modeled autism as a complex condition, visually demonstrating how multiple genetic factors in environmental factors can combine to reach a threshold for a diagnosis. Participants had to review it and then answer a series of multiple choice questions to prove they could actually interpret the data being presented, rather than just reciting facts.
9:17Wait, why use autism as the specific test case for comprehension? Doesn't that risk biasing the results if a test taker already has a personal connection to it? Like, if my brother is autistic, I might already know a lot about the condition before I even look the infographic.
9:31That is a great point. And the research team anticipated that exact potential bias. They purposely chose autism as the model for a complex condition, because it involves both genetic and environmental factors, which makes it a really robust test for nuanced comprehension.
9:47So it was intentionally chosen for its complexity. Yes. But they explicitly tracked that bias by asking participants if they or their immediate family were autistic, and measuring that connection actually illuminated one of the most vital insights of this entire deep dive.
10:02Okay, so if testing this new model on a complex condition like autism is the ultimate stress test for comprehension, the results must reveal where people are actually crashing the car. Where did the data show the biggest blind spots when they unleash this on nearly 3000 people?
10:18Well, we can start with the most dominant factor across the entire data set. When they ran the regressions, they found that numeracy was undeniably king. Really? Just basic math. Yeah. A participant's basic math and probability skills were the absolute strongest predictor of their genetic literacy across all 3 subscales.
10:37The statistical significance showed a P value of less than .001. If you understand basic probability, your ability to understand genetics scales up dramatically. I mean, that structural link makes sense when you think about it.
10:50Genetics is fundamentally a system of odds, inheritance, risks and percentages. But what about the autism connection you mentioned? How did personal experience skew the results? This is where we see the subjective objective knowledge gap play out in stark numbers.
11:02Participants who reported a personal or familial connection to autism did score significantly higher on the subjective knowledge section. Okay, so they fell highly confident. They recognize the terminology on the page.
11:14Right. Their adjusted mean score was 5.97 out of 7 compared to 5.78 for those without a connection. They were deeply familiar with the vocabulary. However, when it came to actual conceptual knowledge, the objective true or false facts and knowledge comprehension, the ability to synthesize the threshold model, those with a personal connection to autism did not score significantly better than anyone else.
11:39Wow. So their confidence was elevated, but their actual mechanical competence was completely flat. Yes. It perfectly illustrates a pervasive psychological phenomenon in science literacy. It's often related to the Dunning Kruger effect in medical settings.
11:52Where people overestimate their own competence. Exactly. People who have personal experiences with a scientific topic, sit in waiting rooms, they read support forums, and they repeatedly hear terms like heritability or allele.
12:05Because they hear the jargon frequently, they feel fluent. But that mere exposure to vocabulary masks a deep gap in actual structural understanding. Precisely. High confidence does not equate to high competence.
12:19That is slightly terrifying when you think about patients making irreversible medical decisions based on that false confidence. But surely environmental factors like education or geography play a massive role in actual competence, right?
12:33If you live near a major research hub, your baseline knowledge has to be higher. You know, the researchers hypothesize exactly that. They assumed that living in a metropolitan area, surrounded by massive research hospitals, genetic counselors and advanced testing infrastructure, would naturally elevate your genetic literacy compared to living in a rural area.
12:52Yeah, that seems like a logical assumption. But the data showed the exact opposite. Geographic location had absolutely no significant main effect on overall genetic literacy levels. Here's where it gets really interesting.
13:03Does this mean that living right down the street from a world-class genetics clinic gives you absolutely 0 baseline advantage in understanding genetics over someone living in a rural area? The data suggests there is no advantage at all.
13:17And the reason comes down to the deep abstraction of genetic data. You do not absorb complex molecular biology through osmosis just because you live in a city with a hospital. Right. If a doctor speaks in dense medical jargon, an urban patient is just as lost as a rural patient.
13:34Exactly. The deficits in genetic literacy are universal. They completely transcend geography. That completely flips the script on how we usually think about healthcare access and education. It really does reframe the challenge.
13:46However, they did find a crucial interaction between formal education level and that personal connection to autism. Oh, how? For individuals with higher overall educational attainment. Having a personal connection to autism actually did help them score a bit higher on knowledge comprehension.
14:02Oh, I see Because higher education provides a broader analytical framework. It gives them the cognitive tools to actually process and organize the complex medical information they encounter in their personal lives, turning that exposure into actual comprehension.
14:17Right. But for those with lower educational attainment, that personal connection remained trapped as just vocabulary recognition that didn't translate into higher applied literacy. So knowing that these deficits are universal, and that they are often masked by this dangerous false confidence where people think they know more than they do, how does this change the real world?
14:37Like, what do doctors and policymakers actually do with this validated 17 question EEGL short? If we connect this to the bigger picture? Having a brief psychometrically validated tool like the EGL short, fundamentally changes clinical practice.
14:51Imagine a patient coming in for genetic counseling regarding a hereditary cancer risk. Before the consultation even begins, they can take this brief 17 questions survey on a tablet in the waiting room.
15:01Oh, wow. So the doctor instantly sees the results and realizes, oh, this patient has incredibly high subjective confidence, but very low conceptual knowledge. I need to be careful not to assume they understand what I'm saying, just because they nod along when I use technical terms.
15:17Exactly. It allows healthcare providers to actively tailor their communication. If a patient presents with a massive subjective objective gap, the doctor knows they need to slow down. They need to avoid jargon, draw diagrams, and explicitly check for comprehension by having the patient explain the risk back to them.
15:36That is incredibly practical. And furthermore, because the study proved that these deficits transcend geography, policymakers now understand that educational interventions must be implemented uniformly.
15:46You cannot simply target rural areas under the false assumption that urban populations already understand the science. Okay, I have a pushback on the intervention strategy, though. If numeracy is the absolute strongest predictor of genetic literacy across every single metric they tested, shouldn't public health initiatives just focus heavily on teaching basic math and probability?
16:07Like why fund complex genetic education campaigns at all if math is the foundational bottleneck? It's a fair question, but while numeracy and genetic literacy are deeply correlated, discriminate validity confirms they remain distinct constructs.
16:22Math and probability are the foundational scaffolding of a building. You absolutely need that scaffolding to construct the house. But the house itself, understanding how a specific genetic variant interacts with an environmental exposure to trigger a physical health outcome that still requires biological context.
16:39You can't just live in this scaffolding. Exactly. You can understand that a 25% chance means one in four. That is numeracy. But understanding why 2 parents who do not have a disease, have a one in 4 chance of passing a recessive genetic condition to their child, requires conceptual genetic knowledge.
16:55That makes total sense. The EAGL measure proves that true comprehension requires synthesizing both the mathematical probability and the biological reality. But every study has its boundaries. Yeah. More of the limitations here.
17:08A sample of 2,700 people online is robust, but it doesn't represent the entire human race. No, it doesn't. And the researchers were highly transparent about the statistical limitations. First, utilizing an online platform like prolific, inherently introduces a selection bias based on technology.
17:26You are automatically excluding individuals who lack reliable internet access or baseline digital literacy. Right. If you are actively taking online surveys. You already possess a certain level of technical comfort.
17:38Exactly. Which suggests that the baseline genetic literacy in the broader offline population might actually be lower than what was measured in this sample. Second, relying on self-reporting for the subjective knowledge portion always introduces implicit bias, tying back into that phenomenon where people inaccurately estimate their own familiarity.
17:57Third, this initial validation was strictly limited to participants in the United States who speak English as their primary language. And finally, using autism as the single model for a complex condition could introduce condition specific biases despite their rigorous efforts to measure and account for the personal connection effect.
18:16So what are the immediate next steps for this research? Where do they take the EGL measure to ensure it works for everyone? The essential next steps involve validating the EGL measure in multiple other languages and testing it across diverse cultural contexts?
18:30They need to conduct in-person or telephone surveys to capture the data of offline populations. That will be huge for getting a real baseline. Yeah. And they will also likely test the tool using different complex conditions like heart disease or Alzheimer's to verify if the knowledge comprehension skills remain stable, regardless of the specific disease model used in the infographic.
18:51Right, checking if it scales. But the vital takeaway is that the EEGL short is now a fully validated, highly efficient tool ready for widespread implementation in clinics and academic research. So, what does this all mean?
19:04If we distill everything down? The EAGL measure gives us the 1st rigorously validated tool to prove that truly understanding genetics requires more than just recognizing buzzwords. It revealed that high confidence does not equal high confidence, and that gaps in genetic comprehension are a universal challenge that transcend geography.
19:25It's a massive leap forward for how we approach patient care and education. Absolutely. What does this mean for the next time you or a loved one are handed a genetic test result by a doctor? Will you just recognize the terms, or will you actually comprehend the risk?
19:39It is a vital question we all need to evaluate when navigating our own healthcare. This episode was based on an open access article under the CCBY 4.0 license. You can find a direct link to the paper and the license in our episode description.
19:52If you enjoyed this, follow or subscribe in your podcast app and leave a 5 star rating. If you'd like to support our work, use the donation link in the description. Now stay with us for an original track created especially for this episode and inspired by the article you've just heard about.
20:07Thanks for listening and join us next time as we explore more science base by base.
