AI in Clinical Trials: How to Govern the Future of Research

Host 1: Artificial intelligence is moving so fast it feels almost impossible to keep up, doesn't it?

Host 2: It really does.

Host 1: It's completely transforming, well, pretty much every sector, and clinical research is right there at the forefront.

Host 2: Absolutely.

Host 1: And this isn't just about, you know, cool new technology, it's raising some really deep questions about how we innovate responsibly, how do we make sure the ethical safeguards are solid even when the tech is racing ahead like this?

Host 2: It's a huge balancing act. Um the pace of AI development often feels like it's outrunning our ability to even understand, let alone regulate, all the implications.

Host 1: Mhm. Yeah.

Host 2: So, for institutions running clinical trials, it's this constant, fascinating challenge.

Host 1: Exactly. And that's what we're diving into today. We want to cut straight to how major institutions are actually tackling this. We want to give you a clearer path to understanding the ethical stakes and uh the practical solutions for using AI in studies involving human subjects.

Host 2: Mhm. Right.

Host 1: Our focus is a really groundbreaking new framework. It's from the Multi-Regional Clinical Trials Center, the MRCT Center, and it's specifically for AI adoption and oversight in clinical research.

Host 2: That's the one. We're going to unpack the MRCT Center's, right, A Framework for AI Adoption and Oversight in Clinical Research.

Host 1: Okay.

Host 2: It was just presented uh June 24th, 2025.

Host 1: Okay.

Host 2: And it's quite a visionary document. Its explicit aim is to, and I'm quoting here, "improve the integrity, safety, and rigor of global clinical trials."

Host 1: Wow.

Host 2: Think of it as a much needed GPS for navigating this, well, this very new territory.

Host 1: A GPS. I like that. So, why this framework and why right now? What makes this moment so critical for something like this?

Host 2: Well, because AI isn't some future concept for clinical research anymore. It's happening now.

Host 1: Right.

Host 2: We're seeing it, you know, revolutionizing everything from diagnostics and drug discovery, all the way to providing critical clinical decision support.

Host 1: Okay.

Host 2: Its use in studies with human subjects is expanding incredibly fast.

Host 1: Yeah.

Host 2: So, the ethical and practical questions that it raises aren't theoretical anymore. They're immediate. They're right here.

Host 1: And that rapid expansion creates a real problem, I guess, for Institutional Review Boards, IRBs, who are, you know, the gatekeepers for ethical research.

Host 2: Right. Yeah.

Host 1: They're suddenly facing this whole new set of ethical and regulatory questions they've just never had to deal with quite like this before.

Host 2: Yeah, the complexity is just skyrocketing.

Host 1: We're talking about huge shifts around privacy, the transparency of these sort of black box algorithms.

Host 2: Right. How do you explain that?

Host 1: The potential for bias baked right in, and crucially, how do you maintain human autonomy when an AI is making really significant decisions?

Host 2: And what's really telling is that, you know, while agencies like the FDA, HHS, NIST, they've put out ethical principles,

Host 1: Mhm.

Host 2: IRBs have largely been left without practical tools for that specific protocol-level review.

Host 1: Ah, okay.

Host 2: And that often means research proposals involving AI arrive, frankly, without enough detail.

Host 1: Which leads to delays, I imagine.

Host 2: It leads to delays and just a general sense of uncertainty.

Host 1: Yeah.

Host 2: It's a clear gap in actionable guidance.

Host 1: So, this framework steps right into that gap. It's designed to give structured, actionable steps, allowing for a more consistent, thoughtful review of AI-driven research.

Host 2: Exactly. The goal is really for oversight to actually keep pace with the innovation, instead of always, you know, playing catch-up.

Host 1: That makes sense. Giving reviewers a common language, a systematic approach.

Host 2: Precisely. That's vital in such a fast-moving field.

Host 1: Now, putting together something this comprehensive sounds like a massive effort. How did it actually come together? Was it just like academics in a room, or?

Host 2: Oh, far from it. This was a really rigorous, collaborative effort. The framework was developed by a task force.

Host 1: Okay.

Host 2: It was convened by the MRCT Center and WCG back in spring 2024. And it wasn't just a few people, it brought together a really diverse group. You had experienced IRB chairs, leading ethicists, AI technologists, people from industry, all looking at it from different angles.

Host 1: And what I find really interesting is that it wasn't just theoretical brainstorming, right?

Host 2: Not at all.

Host 1: It was informed by actual clinical research using AI that was already under IRB review.

Host 2: Yes, that real-world input was key.

Host 1: Plus, it built on foundational documents like the Common Rule, SACHRP recommendations, NIST guidelines, so it feels really grounded.

Host 2: Absolutely. That grounding is indispensable. And its ethical foundations are incredibly solid. It explicitly builds on core US research ethics regulations, specifically the Common Rule, 45 CFR 46,

Host 1: Mhm.

Host 2: and even more deeply, it reaffirms those ethical principles from the Belmont Report: respect for persons, beneficence, justice. The challenge now is understanding how AI interacts with, and sometimes stretches, our usual understanding of those principles.

Host 1: That's a critical point. It's not trying to reinvent ethics,

Host 2: No.

Host 1: but show how existing principles apply to this new tech. It compliments current policies, doesn't replace them.

Host 2: Exactly right. The framework is clear that it uses existing definitions: research, human subjects, minimal risk.

Host 1: Okay.

Host 2: It actually encourages IRBs to operate within their established authority, even if the AI parts feel unfamiliar. It's about integrating AI oversight, not disrupting the existing system.

Host 1: Okay, let's get into the structure then. It's broken down into four core components. Can you walk us through the first one, Component A?

Host 2: Sure. Component A is titled, Initial Questions to Guide Oversight. Its main purpose is to quickly help IRBs get their bearings.

Host 1: Right.

Host 2: It starts with the fundamentals. Is this actually human subjects research under the Common Rule? What's the AI's intended use? And really importantly, who is the human subject in this specific AI research? Because that's often more complex than it first sounds.

Host 1: So, it's about cutting through the noise early on, determining the scope. What are some of those key filters?

Host 2: Precisely. Beyond defining the human subject, it asks things like, is there adequate evidence of risk considerations in the protocol? Does this research involve more than minimal risk? And what do we actually know about the AI algorithm itself?

Host 1: Hm.

Host 2: These questions aim to identify early whether the AI is part of the actual intervention being studied, or if it's just being used as, say, an administrative tool.

Host 1: Ah, okay.

Host 2: 'Cause that really changes the depth of review needed.

Host 1: That distinction seems vital. Okay, so once those initial things are established, what's the next layer the framework helps peel back?

Host 2: That takes us to Component B, Review Considerations by AI Development Stage. What's really insightful here is how the guidance is tailored based on the AI's maturity level.

Host 1: Okay.

Host 2: It assesses the contextual risks across three different phases: discovery, translation, and deployment. This approach, by the way, is adapted from some key preprint research in the field.

Host 1: That sounds smart. It recognizes that the questions and the risks, well, they change as an AI goes from just an idea to something widely used.

Host 2: Exactly.

Host 1: Are there specific things within these stages that IRBs should focus on?

Host 2: Absolutely. Across all those stages, there are crucial cross-cutting topics, things like the algorithm's stability, you know, how consistently does it actually perform.

Host 1: Right.

Host 2: Then, the identifiability of data, can individuals be re-identified? And a deep dive into the data sources and how that data was collected in the first place.

Host 1: Mhm.

Host 2: These are critical threads that run through the whole AI development lifecycle.

Host 1: That makes perfect sense. Okay, so how does the framework make sure those core ethical principles are truly woven into every stage? That sounds like Component C.

Host 2: It is. Component C directly addresses ethical considerations.

Host 1: Mhm.

Host 2: This is where IRBs assess how the AI's design and how it's being applied line up with those established ethical principles we talked about earlier.

Host 1: Like Belmont.

Host 2: Exactly. It's about translating those big ideas into practical oversight specifically for AI.

Host 1: So, what specific ethical areas are IRBs looking at here? Because things like informed consent or privacy must take on totally new dimensions with AI.

Host 2: They really do. The framework prompts them to delve into areas like human agency and oversight, making sure humans remain central to decision making, not just sidelined by an algorithm.

Host 1: Okay.

Host 2: Then, the complexities of privacy, confidentiality, and data governance, especially with sensitive health data.

Host 1: Sure.

Host 2: They also look at technical robustness and safety. Does the AI actually work reliably and safely?

Host 1: Yeah.

Host 2: Transparency, can we understand, at least to some degree, how it works?

Host 1: Not just a black box.

Host 2: Right. And crucially, representation and fairness to guard against bias, and, of course, how informed consent works or needs to work differently in an AI context.

Host 1: That's a really comprehensive list for some very tricky ethical ground. And finally, Component D, does that look at AI in a different way?

Host 2: It does. Component D focuses on Artificial Intelligence Deployed in the Administration of Research. This part specifically looks at AI used to support research operations, not as the actual intervention being studied.

Host 1: Ah, okay.

Host 2: Think of it as AI helping backstage maybe, rather than being the star of the show.

Host 1: So, things like using AI to help find the right participants for a trial maybe, rather than the AI being the treatment?

Host 2: Exactly that. Examples include AI assisting with participant recruitment or matching, maybe helping develop research materials, transcribing interviews, or even helping analyze qualitative data like interview transcripts.

Host 1: Right.

Host 2: This component pushes IRBs to consider the impact on participants even if it's indirect. It stresses institutional oversight and transparency even for these administrative uses. It recognizes these tools can still affect participant protection, so you can't have ethical blind spots even in what might seem like just operational tasks.

Host 1: This framework sounds incredibly thorough and very forward-thinking. So, how is it actually being used? Is it just sitting on a shelf, or is it more dynamic?

Host 2: Oh, it's absolutely designed as a living tool. It's publicly available on the MRCT Center website,

Host 1: Good.

Host 2: and they're actively developing practical case examples to show how to apply it. It's meant for IRBs, researchers, institutions,

Host 1: Mhm.

Host 2: and, crucially, they're welcoming feedback.

Host 1: Ah, that's important.

Host 2: Because everyone recognizes, as AI keeps evolving, our oversight has to adapt too.

Host 1: Mhm.

Host 2: It's definitely a journey, not, you know, a final destination.

Host 1: That's a really powerful way to put it. Okay, let's make this more concrete. Let's bring it to life with a real example.

Host 2: Great idea.

Host 1: This one was shared by Kevin Nellis from SUNY Downstate Health Sciences University. This should give you a really tangible sense of how the framework gets applied. So, give us the snapshot of this study.

Host 2: Absolutely. So, this case study involves a pilot study. It uses a wearable EEG device,

Host 1: Okay, brainwaves,

Host 2: paired with a machine learning, ML, tool. And the goal is to support women with alcohol use disorder, or AUD, in managing overeating behaviors.

Host 1: Interesting combination.

Host 2: Yeah. The participants are healthy women aged 18 to 45 who also have AUD, and they were recruited specifically from the Flatbush area of New York City.

Host 1: Okay.

Host 2: And the study design provides real-time, AI-driven feedback on their eating behavior based on the EEG data.

Host 1: Wow. Okay. That is a lot to unpack in one study. You've got an investigational device element, the AI/ML part, and a behavioral intervention all rolled together.

Host 2: Mhm.

Host 1: Why is this study such a perfect test case for the framework?

Host 2: Well, it's an excellent example because it hits on so many of the framework's core challenges. For instance, if you think about Component B, AI stage and device oversight,

Host 1: Mhm.

Host 2: this study falls squarely into that discovery/translation phase. It cleverly combines two devices that are already FDA-cleared with one investigational device.

Host 1: Ah, okay.

Host 2: So, that immediately required the IRB to evaluate its non-significant risk, or NSR, status. That's a critical early assessment the framework guides them through.

Host 1: Okay, so it helps navigate that complex device landscape.

Host 2: Yeah.

Host 1: What about the data? That seems highly sensitive here: medical records, brain activity.

Host 2: Absolutely. That hits on the framework's points about PHI use and data security. The study accesses medical records and questionnaire data.

Host 1: Right.

Host 2: So, the framework prompts a really thorough review of how that data is stored securely. There's a critical emphasis on making sure no protected health information, or PHI, is stored on portable devices.

Host 1: Got it.

Host 2: And crucially, the informed consent process must explicitly disclose any potential future use of the data and any known limitations of the algorithm itself.

Host 1: That transparency piece again.

Host 2: Exactly. That directly aligns with the framework's ethical considerations around privacy, consent, and transparency.

Host 1: And what about bias? You mentioned recruitment was localized in Flatbush. That's always a concern with AI development, isn't it?

Host 2: A huge concern, and that's where Component C's focus on bias and participant equity comes in.

Host 1: Mhm.

Host 2: Since it's a localized pilot in Flatbush, NYC, a key question the IRB has to ask is, is this recruited group really representative of all women with AUD?

Host 1: Right.

Host 2: The framework pushes for a careful look at equity in recruitment and how that might affect the fairness and future scalability of this intervention. It's about building in fairness proactively.

Host 1: This all sounds incredibly robust. And you mentioned something about ethical guardrails and "AI exceptionalism" within this case review. That sounds interesting. It raises that question of how we avoid our own biases about AI itself, right?

Host 2: That's such a profound point. Yeah.

Host 1: Either thinking it's automatically better or automatically flawed.

Host 2: Exactly. And the framework helps significantly here. It guides IRBs to make sure consent forms clearly state the AI's role and its limitations, not just trumpet its potential benefits,

Host 1: Hm.

Host 2: and it underscores the need for the investigators themselves to have the right AI expertise. It's all about grounding the review in objective, informed reality

Host 1: Right. Yeah.

Host 2: rather than getting swayed by either, you know, excessive enthusiasm or maybe excessive fear about the tech itself.

Host 1: That brings us neatly to this really thought-provoking concept mentioned in the source material, AI exceptionalism. It highlights this kind of split personality in how we often talk about AI, doesn't it?

Host 2: It absolutely does. The source describes these two contrasting views you often hear, and navigating between them is really central to what the framework tries to achieve.

Host 1: Yeah.

Host 2: On one side, you've got the tech halo effect, a very positive, almost reverent view of AI.

Host 1: Right. The tech halo view, as I understand it, sees AI as fundamentally new and just inherently superior.

Host 2: Mhm.

Host 1: It's the belief that AI will transform everything. Its learning is always advancing, always getting more accurate and faster, it's purely data-driven, it'll advance humanity, elevate communication. It sounds almost utopian.

Host 2: It can lean that way. But then, on the complete opposite side, you have the horns effect,

Host 1: Oh.

Host 2: which is a much more cautious, often quite negative perspective. This view also sees AI as fundamentally novel, but believes it's inherently unknowable, maybe untrustworthy, needing way more scrutiny.

Host 1: Hm.

Host 2: It highlights AI's potential for baseline bias, the risk of it replacing human judgment altogether, and the ever-present threat of privacy loss. It's a viewpoint often steeped in apprehension, maybe even fear.

Host 1: So, you have the halo and the horns. What does this all mean for you, the listener, and for the framework itself? How does this framework help us avoid getting trapped in either of those extremes?

Host 2: Well, that's really its core function in many ways. It provides a structured, um almost dispassionate lens to evaluate AI in research.

Host 1: Right.

Host 2: By grounding the assessment in those established ethical principles and very practical considerations, it helps cut through both the hype of the tech halo and the fear of the horns effect.

Host 1: Hm.

Host 2: It ensures we're making balanced, informed decisions about AI's role, especially in studies that involve people's lives and their data.

Host 1: And that ultimately feels like what we desperately need as this technology just keeps advancing at this incredible pace.

Host 2: Absolutely.

Host 1: We've just unpacked how the MRCT Center's framework offers these essential, actionable tools for navigating the really complex landscape of AI in clinical research,

Host 2: Mhm.

Host 1: ensuring these studies aren't just innovative, but also deeply ethical and rigorously sound.

Host 2: And this deep dive, it's so crucial because it literally touches on how future medical breakthroughs will be developed,

Host 1: Mhm.

Host 2: and more personally, how your data, if you ever participate in a study, will be handled in this fast-changing tech environment.

Host 1: Yeah.

Host 2: It's really about building and maintaining trust in an age of, well, incredible change.

Host 1: A powerful point to end on. And it leaves us with this final thought for you to consider: Given AI's relentless, continuous evolution, how might frameworks like this one need to keep adapting? How do we ensure that balance between groundbreaking innovation and robust protection for human subjects is always maintained? And maybe, what role do you think transparency really plays in building or maybe even eroding trust in these AI-driven studies? Something to mull over. Thanks for joining us for the deep dive.