AI at the Front Door: Does Clinical Decision Support Actually Work in Primary Care?

Sam: Everyone is rushing to buy AI for primary care clinics, but we might be moving way faster than the actual evidence of safety and effectiveness.

Maya: Today we are looking at a systematic review titled AI-Based Clinical Decision Support Systems in Primary Care: Systematic Review of Clinical Implementation, published in the Journal of medical Internet research. And as always, a quick reminder that our voices are AI-generated, and we are summarizing a public document.

Sam: And just to be totally clear upfront, this is a summary of research, not medical advice. But Maya, let us talk about this document. Primary care is essentially the front door to the entire health system. It is where everything starts.

Maya: It is. The authors point out right away that primary care is central to high-performing health systems. It contributes to improved population health outcomes, greater equity, and reduced avoidable hospitalizations. But it is also incredibly strained right now.

Sam: Right, they talk about rising case complexity, workforce shortages, and fragmented information systems placing increasing pressure on clinicians. Which is exactly why health system leaders are so desperate for a tech solution.

Maya: Exactly. They want tools that can help. Specifically, they are looking at AI-based clinical decision support systems, or AI-CDSSs. The formal definition the authors use is computer systems designed to impact clinician decision making about individual patients at the point in time that these decisions are made.

Sam: We already have traditional, non-AI decision support systems though, right? The pop-ups that tell you to order a test or prescribe a drug.

Maya: We do. But traditional systems have a key limitation. They have limited capacity to account for variability in patient characteristics and clinical contexts. Basically, they follow rigid rules. The hope is that adding AI methods into these systems will improve data-driven precision and enable more individualized recommendations.

Sam: So that is the promise. Precision, less burnout, better care. To figure out if it is actually working, the researchers set out to synthesize evidence on the effectiveness and safety of these AI systems compared with usual care or non-AI systems in primary care settings.

Maya: And their search was massive. An information specialist systematically reviewed major databases from inception to June 27, 2025. They identified 4074 records through database searches and 12 additional records through citation searching.

Sam: 4074 records. That sounds like a mountain of evidence.

Maya: You would think so. But after removing duplicates, 3034 records remained. Of those, only 126 were retained for full-text screening. And at the end of the entire process, they included just 14 records on 11 distinct studies.

Sam: Wait, really? Just 11 distinct studies made the cut? Out of all that hype and all those products on the market, we only have 11 robust studies testing this in real primary care settings?

Maya: That is exactly what this reveals. Most evidence is stuck in model development and validation. The authors are very clear that performance assessments conducted in closed contexts therefore poorly predict how AI-CDSSs will perform when deployed in clinical practice.

Sam: Because real clinical care is messy. It is not a clean, curated dataset. So let us look at those 11 studies. Who is actually running them and what are they testing?

Maya: Well, they were conducted across North America and Europe. Specifically, 7 of the studies, which is 63.6%, were in North America, and 4, or 36.4%, were in Europe. They only included high-income countries, which is a major limitation.

Sam: What about the funding behind these studies? Are these mostly academic projects or vendor-driven?

Maya: There is a strong commercial presence. Out of the 11 studies, 5 reported at least 1 conflict of interest, and 5 were funded by industry. In terms of the actual AI models, 6 of the 11 identified systems, or 54.5%, were commercial products. Another 2 were institutionally developed but not open source.

Sam: So we are largely looking at proprietary models. And how did they actually set up these studies? Are we talking about randomized controlled trials?

Maya: Only 4 were experimental studies, like randomized trials. The other 7 were quasi-experimental, meaning they used things like historical controls or before-after analyses. And the risk of bias was a major issue across the board.

Sam: How bad was the bias?

Maya: The reviewers evaluated the overall risk of bias as high for all experimental studies. For the quasi-experimental ones, 1 was at serious risk of bias, while all others were at critical risk of bias. Mostly due to uncontrolled confounding and issues with how outcomes were measured.

Sam: Critical risk of bias. That is tough. But let us look at the actual clinical conditions they tried to treat. Primary care covers everything from a sore throat to complex chronic diseases. What did these 11 studies focus on?

Maya: It was a mix. AI systems were primarily evaluated in musculoskeletal conditions, which was 3 out of 11, or 27.3%. Then urinary tract infections at 2 out of 11, or 18.2%, and diabetes, also at 2 out of 11, or 18.2%. Others looked at major depressive disorder, asthma, and low ejection fraction.

Sam: And who were the patients and clinicians testing this?

Maya: That is another gap. Only 6 of the 11 studies, or 54.5%, even reported clinician sample sizes, which ranged from 7 to 358. And of those 6 studies, 4 did not report any clinician characteristics at all. Age and sex for clinicians were only reported in 2 studies.

Sam: Wait, so we do not even know basic demographics about the doctors and nurses using the AI in most of these trials?

Maya: Exactly. They were slightly better with patient data. 8 of the 11 studies reported patient sample sizes. In the experimental groups, those ranged from 14 to 24,083 patients. But even then, sex and ethnicity were only reported in 5 studies.

Sam: It feels like we are missing a lot of basic context. But let us get into the actual results. Did these systems actually help? The paper breaks this down by how complex the clinical situation was, right?

Maya: Yes, this is a really interesting part of the review. Because there was so much heterogeneity across the studies, they could not just do a simple meta-analysis. So, they explored the differences by classifying the implementation contexts into simple, complicated, or complex.

Sam: How did they define those buckets?

Maya: They adapted an approach informed by the NASSS framework. They looked at 4 criteria: condition complexity, decision logic, system complexity, and preference sensitivity. They classified 4 out of 11 contexts as simple, 4 out of 11 as complicated, and 3 out of 11 as complex.

Sam: Give me an example of a simple context in primary care.

Maya: A simple context would be screening for low ejection fraction, or managing shingles vaccination alerts. The target is well-defined, the decision logic is protocolized, the system is operationally simple, and preference sensitivity is low. It is mostly guideline-driven.

Sam: And did the AI work well in those simple contexts?

Maya: Yes, this is where they saw the most consistent positive effects. Specifically on workflow-related outcomes and process-level outcomes. For example, 1 experimental study in a simple context found a positive effect on the detection rate of low ejection fraction.

Sam: That makes sense. If it is a clear rule, the AI can flag it easily. What about the complex contexts? What does a complex scenario look like?

Maya: Complex contexts involve clinically heterogeneous conditions, individualized decision logic, multicomponent adaptive systems, and high preference sensitivity. Good examples from the review are chronic musculoskeletal pain management and major depressive disorder.

Sam: Where the right answer depends heavily on the patient's goals, their tolerance for side effects, and how they respond over time.

Maya: Exactly. In these settings, the results were very different. Patient-level effects were limited and inconsistent. For instance, 2 studies looked at pain intensity in musculoskeletal conditions. 1 experimental study reported no effect, and 1 quasi-experimental study actually reported a negative effect.

Sam: Quick note before we carry on. This spot is open for a sponsor. If your company builds or sells AI for healthcare and wants to reach the clinicians, health-system leaders and industry teams who listen to this show, the link to our sponsorship page is in the show notes.

Maya: And now, back to the document.

Sam: Wait, a negative effect? The AI made it worse?

Maya: It means the control group did better on that specific metric in that specific study. Both studies also reported no effect on mental health outcomes. The review notes that in complex settings, AI-CDSSs might just be used differently. Clinicians might use them to validate their own reasoning rather than changing their management decisions.

Sam: So if the AI is just confirming what the doctor already thought, you are not going to see a massive shift in patient outcomes. You are just adding another step to the visit.

Maya: Exactly. The authors write that when decision tasks become more cognitively demanding, clinicians appear less likely to consult decision support systems and may shift toward situation-assessment tools rather than algorithmic recommendations.

Sam: Which brings us to the workflow issue. Adding another step to a time-constrained primary care visit is a big deal. How are these AI systems actually delivered to the clinicians? Are they built right into the electronic health record, the EHR?

Maya: Not mostly, no. Out of the 11 studies, 4, or 36.4%, evaluated stand-alone systems. Only 2, which is 18.2%, reported electronic health record-embedded systems. Another 3 used EHR data without specifying integration within the interface, and 3 did not report interoperability information.

Sam: So a huge chunk of these are just separate tools sitting outside the main workflow.

Maya: Yes, and that creates real friction. In 7 of the 11 studies, which is 63.6%, using the AI required additional clinician tasks. That included accessing and reviewing separate dashboards, manual data entry, or printing out reports before the consultation.

Sam: Manual data entry? In primary care? That is the exact opposite of what AI is supposed to do.

Maya: It is a massive barrier. The review states that clinical misalignments commonly lead to nonadoption, inappropriate use, and long-term abandonment of AI-CDSSs. If a system adds clinical or administrative steps into already time-constrained consultations, clinicians will just stop using it.

Sam: Which leads perfectly into the concept of alert fatigue. The paper mentions this specifically, right?

Maya: It does. It notes that alert fatigue represents a well-documented failure mode. They cite background literature showing override rates exceeding 50% to 73% for medication-related alerts in traditional systems. Clinicians become progressively less likely to accept alerts as their frequency increases.

Sam: So if the AI is just throwing more alerts into the mix, even if they are slightly smarter alerts, the doctor might just reflexively close them anyway.

Maya: Precisely. Although, 1 study in a simple context did use an AI specifically to reduce alert fatigue by suppressing low-value shingles vaccination reminders. In that case, they saw no effect on the actual order count, but 2 experimental studies overall did report reductions in alert burden and clinician workload.

Sam: Okay, so there is a glimmer of hope for using AI to silence the noise. But let us talk about the stakes here. If an AI gives a bad recommendation and a clinician follows it, a patient gets hurt. How did these 11 studies measure safety?

Maya: This is perhaps the most alarming finding in the entire document. The safety evaluation was rare and poorly specified. Only 2 studies prespecified safety outcomes.

Sam: Only 2 out of 11 studies actually set out to measure if the tool was safe before they started?

Maya: Yes. And even in those, the reported safety information mainly involved technical malfunctions or self-reported side effects. They largely relied on the absence of serious adverse events as a proxy for safety.

Sam: But a lack of immediate, glaring disasters does not mean a system is safe. Especially in primary care where a missed diagnosis might not show up for months.

Maya: Exactly. The authors warn that this approach risks missing harms that are subtle, cumulative, or delayed, such as missed follow-up, delayed diagnosis, or inappropriate treatment. Because these events require prospective monitoring, not just waiting to see if someone reports a severe event retrospectively.

Sam: Did any of the studies look at how often the doctors actually disagreed with the AI? Like, tracking when the human had to step in and say, 'No, the computer is wrong here'?

Maya: No. The review explicitly states that no study reported clinician interaction measures such as agreement with, override of, or reliance on AI outputs. We have no structured data on how the humans actually mitigated the AI's risks in practice.

Sam: That feels like a massive blind spot for anyone buying these systems right now. You are deploying software that advises on patient care, but you are not actively tracking when your doctors have to override it to keep patients safe.

Maya: It is a huge blind spot. Risk mitigation strategies were described in 4 of the 11 studies, or 36.4%. These included a human-in-the-loop oversight, internal quality checks, and automated safeguards like requiring patient validation before escalating severe alerts. But reports of errors were uncommon, found in just 3 studies, or 27.3%, and were mainly incidental things like image quality issues or software glitches.

Sam: Software glitches like what?

Maya: Like an incorrect questionnaire score comparison in an algorithm for depression, or broken links between pages. But again, these were incidental findings, not the result of systematic safety monitoring.

Sam: The authors use a very specific word in the discussion regarding safety monitoring. Algorithmovigilance. What does that actually mean in a primary care clinic?

Maya: It essentially means treating algorithms like we treat new drugs. Postdeployment monitoring. The authors point out that calls for systematic algorithmovigilance remain only partially operationalized in practice. We need structured adverse event reporting methodologies for AI systems in clinical care, tracking both the technical performance and the human-computer interaction over time.

Sam: Because right now, it sounds like we just plug it in and hope nobody complains.

Maya: That is what the evidence suggests. The authors conclude that there is an important gap between the rapid development of AI-CDSSs and the lack of robust, comparable evidence to support their reliable use in primary care.

Sam: Let us talk about patient-level outcomes. Did these AI tools actually make patients healthier? Because that is the ultimate goal, right?

Maya: It is, but patient-level outcomes were only evaluated in 5 of the 11 studies, which is 45.4%. And as we discussed with the complex contexts, the results were mixed.

Sam: What about system-level outcomes? Did they save the clinics money or make them more efficient?

Maya: System-level outcomes were reported in 6 of the studies, or 54.5%. 2 experimental studies evaluated economic outcomes with mixed effects. 1 study in a simple context involving low ejection fraction found the intervention was cost-effective. But another study, in a complicated context for childhood asthma, found no effect on health care costs.

Sam: It feels like the takeaway here is deeply fractured. If you are doing something highly structured, like screening an ECG for a specific anomaly, AI can help. If you are managing a patient with multimorbidity and competing priorities, the AI either gets ignored, adds to the workload, or just fails to move the needle.

Maya: That is an excellent summary. The review specifically states that AI-CDSSs in primary care may provide more consistent benefits for workflow-related and process-level outcomes in structured and protocolized decision contexts, while evidence on safety remains insufficient to support confident routine implementation.

Sam: So if you are a primary care director evaluating a commercial diagnostic algorithm right now, what is your most memorable takeaway from this document?

Maya: My memorable takeaway is that you cannot rely on vendor performance data from closed contexts. You have to demand evidence of workflow integration and prespecified safety monitoring. If the system requires your doctors to log into a separate dashboard, or if the vendor cannot tell you how often clinicians override their alerts, you are buying a problem, not a solution. The pace of deployment could exceed the availability of effectiveness and safety evidence, and you do not want your clinic to be the unmonitored test lab.

Sam: That is a powerful warning. We are deploying these systems way faster than we are proving they work or are safe. To recap, this systematic review found only 11 studies testing AI decision support in routine primary care. The benefits were largely confined to simple, structured tasks, while complex care saw inconsistent results, and rigorous safety monitoring was almost entirely absent.

Maya: A quick reminder that the source is linked in the show notes, and that this is not medical advice.