Will AI Find the Right Polyps? Decoding NICE Guidance on Colonoscopy Tools

Sam: Artificial intelligence is getting incredibly good at spotting polyps during colonoscopies, but there is a major catch. It might be finding the wrong ones.

Maya: Today we are looking at a document titled "Artificial intelligence (AI) technologies to help detect or characterise colorectal polyps", published by the National Institute for Health and Care Excellence.

Sam: Just a quick reminder before we dive in, our voices are AI-generated, and this podcast is a summary of that public document.

Maya: This document is a piece of HealthTech guidance that aims to set the clinical and economic bar for adenoma detection tools in the NHS. It looks at ten different AI technologies and figures out which ones are ready for use, which ones need more evidence, and what that evidence actually needs to look like.

Sam: This is a big deal for gastroenterologists on the ward and for medtech manufacturers in the boardroom. Let us start with the basics. Why is NICE looking at AI for colonoscopies right now?

Maya: Colonoscopies are procedures where healthcare professionals look inside the bowel using a flexible tube with a camera. The goal is to find polyps, which are small growths. Some polyps can develop into colorectal cancer if they are not treated. Standard colonoscopy can reduce the risk of bowel cancer by up to 90%.

Sam: So the current system already works really well. What is the unmet need that AI is supposed to solve?

Maya: The demand for colonoscopies is very high in the NHS. Missed polyps can lead to delayed cancer diagnoses. Some polyps are just harder to spot. For instance, smaller polyps or a specific type called sessile serrated lesions, which can also become cancerous. AI acts as a second observer to highlight areas of concern, which is known as computer-aided detection.

Sam: And some tools go a step further than just detecting, right?

Maya: Exactly. Some AI technologies offer computer-aided diagnosis, which helps the endoscopist assess the features of the polyp to decide if it needs to be removed. But out of the ten technologies evaluated, all ten had detection features, while only four had characterisation features.

Sam: Let us get right to the headline recommendations. Out of those ten tools, what did NICE decide?

Maya: NICE decided that 6 AI technologies can be used in the NHS, but specifically during an evidence generation period. Those six are CADDIE, CAD EYE, ENDO-AID, EndoScreener, GI Genius, and MAGENTIQ-COLO. They can be used as options to help detect polyps for people who do not have diagnosed inflammatory bowel disease or Lynch syndrome.

Sam: So a conditional green light. You can use them, but you have to collect data. How long is this evidence generation period?

Maya: It is set for 4 years. During this time, the technologies can be paid for using core NHS funding. The companies have to confirm agreements are in place to generate the required evidence, and NICE will check in annually. At the end of the 4 years, the companies submit their data for a final decision on routine adoption.

Sam: What about the other four technologies? Why did they not make the cut for this funded period?

Maya: The committee concluded that more research is needed for 4 AI technologies used to help detect polyps. Those are Argus, Discovery, ENDOANGEL, and the Endoscopic Multimedia Information System. For Argus, Discovery, and EMIS, the data was too limited to show they improved the detection of polyps. For ENDOANGEL, the issue was that they could not estimate its cost-effectiveness because of a lack of technology cost data.

Sam: Let us dig into that clinical evidence, because this is where the document gets really interesting. How do they actually measure if an AI tool is working during a colonoscopy?

Maya: The main outcome used in the studies is the adenoma detection rate. This is the percentage of screening colonoscopies where at least one adenomatous polyp is found. Adenomatous polyps are the precancerous ones. For the six recommended technologies, the pooled data showed a statistically significant improvement in the adenoma detection rate.

Sam: That sounds like a total win. If we find more precancerous polyps, we prevent more cancer. But earlier you hinted there was a catch.

Maya: The catch is polyp size and type. The clinical experts explained that polyps smaller than 5 mm are much less likely to become cancerous. One clinical expert actually stated that less than 0.1% of small polyps under 5 mm develop into colorectal cancer. The AI software mostly improves the detection of these smaller polyps.

Sam: Wait, really? So the AI is boosting the overall detection numbers by finding tiny polyps that probably would never harm the patient?

Maya: That is the concern. The document notes that AI technologies may improve detection of polyps 5 mm or less, and polyps 6 mm to 9 mm, compared with polyps of 10 mm or more. When they looked at the detection of advanced adenomas or sessile serrated lesions, the AI improved the rate, but it was not statistically significant for any technology.

Sam: If it is not statistically significant for the big, dangerous polyps, then does using AI actually prevent future cancers?

Maya: That is exactly the uncertainty NICE highlighted. The committee said it is uncertain in practice whether using AI increases the number of cancer cases detected. The historical link between improved detection rates and reduced cancer rates was established before AI existed, largely due to better human technique. AI does not improve an endoscopist's technique or visualisation. It just highlights what is on the screen.

Sam: Which leads perfectly to the problem of false positives. If the AI is highlighting everything, is it flagging things that are completely normal?

Maya: There was very little information on how many times the technologies incorrectly flagged something as a polyp. The committee noted that an increase in detection and removal of benign polyps may increase laboratory costs, affect surveillance intervals, and lead to an increase in future colonoscopies.

Sam: That has huge implications for health system leaders. You are not just paying for the AI software licence. You are potentially paying for more lab work and bringing patients back for more colonoscopies because you found a tiny, irrelevant polyp.

Maya: Exactly. The committee explicitly said more evidence is needed on the impact of these technologies on clinical management, especially regarding patient follow up, surveillance intervals, and additional excision and testing.

Sam: What about the endoscopists themselves? Does having a computer point out every little bump make them faster, slower, or maybe a bit lazy over time?

Maya: On the timing, the evidence suggests procedure length increases by 1 to 2 minutes with the software. The committee concluded that is not a substantive impact. Some endoscopists mentioned the technologies can be distracting, though. As for getting lazy, the committee discussed the potential for deskilling due to overreliance.

Sam: Is there any proof of deskilling?

Maya: Quick note before we carry on. This spot is open for a sponsor. If your company builds or sells AI for healthcare and wants to reach the clinicians, health-system leaders and industry teams who listen to this show, the link to our sponsorship page is in the show notes.

Sam: And now, back to the document.

Maya: They looked at a large UK real-world study called NAIAD, which had 3 phases, including a phase after the software was withdrawn. But the committee judged it was difficult to draw strong conclusions about deskilling from it, because it is hard to control for other factors like changes in case mix.

Sam: Let us talk about the boardroom side of this: the health economics. If the clinical benefit is mostly finding tiny polyps, how did the financial models shake out?

Maya: The external assessment group built a decision-tree model with 5 disease states: no significant pathology, IBD, low-risk adenoma present, advanced adenoma present, and colorectal cancer. They used a model called the MiMiC-Bowel individual patient simulation model to estimate lifetime costs and quality-adjusted life years.

Sam: And what did that simulation say? Are these AI tools cost-effective?

Maya: For the nine modelled technologies, yes, they were estimated to be cost-effective compared with standard colonoscopy. ENDOANGEL was excluded because of a lack of technology cost data. But the group strongly advised caution. They estimated the probability of each intervention being cost-effective was approximately 50% at NICE's usual willingness-to-pay range of £20,000 to £30,000.

Sam: A 50% probability is basically a coin toss. Why is the model so uncertain?

Maya: Because it relies heavily on the adenoma detection rate as a proxy for sensitivity. The model assumes finding more adenomas leads to fewer long-term cancer cases. But as we discussed, if the AI is mostly finding polyps that have less than a 0.1% chance of becoming cancer, the real-world health gains might be much smaller than the model predicts.

Sam: Did the model account for the downstream costs we just talked about, like the extra lab tests for the tiny polyps?

Maya: Not fully. The committee raised concerns that the model did not fully account for the downstream impact. Also, because of data limitations, a single detection rate estimate was applied for some technologies, which likely overstated their effect on higher-risk adenomas. Plus, the model assumed that all future colonoscopies for a patient would be done without AI, which does not reflect reality.

Sam: So that is exactly why NICE wants a 4 years evidence generation period. They need to see if the real-world costs match the simulation.

Maya: Precisely. The committee explicitly said NHS trusts should consider the costs, particularly upfront costs, when implementing these tools. They also warned buyers to take the temporary nature of this guidance into account when negotiating contract lengths and licence costs.

Sam: That is a huge takeaway for procurement teams. Do not lock yourself into a ten-year contract if the national guidance might change in four years.

Maya: Exactly right. Now, let us switch gears from detection to characterisation. We mentioned earlier that some AI tools try to diagnose the polyp without removing it. What did NICE say about that?

Sam: I would imagine that is a lot riskier than just pointing at a polyp.

Maya: It is. There are clinical strategies like resect-and-discard, where a small polyp is removed but not sent to the lab, and diagnose-and-leave, where a tiny polyp is left in place if the doctor is highly confident it is benign. AI could theoretically help with this by characterising the tissue.

Sam: But?

Maya: But NICE stated more research is needed on 4 AI technologies used to help characterise polyps: CADDIE, CAD EYE, GI Genius, and MAGENTIQ-COLO. The evidence was limited and inconsistent. One major issue was that studies only reported results for polyps diagnosed with high confidence, which biases the results and does not reflect real-world clinical uncertainty.

Sam: And the clinical experts had a specific issue with how AI handles those sneaky sessile serrated lesions, right?

Maya: Yes. Experts were concerned that some technologies classify sessile serrated lesions as non-significant polyps. Since those lesions have the potential to cause cancer, mischaracterising them is a significant clinical risk. So, AI technologies for characterising polyps should only be used in research for now.

Sam: There is another massive caveat in this document regarding who actually gets to use this technology. You mentioned earlier that the approval is only for people who do not have diagnosed inflammatory bowel disease or Lynch syndrome.

Maya: That is a vital point for equality and clinical practice. People with IBD or Lynch syndrome were excluded from many of the clinical trials. Their bowels may look different or they may have different types of polyps.

Sam: So the AI simply has not been trained on or proven to work for the patients who are often at the highest risk for colon cancer?

Maya: Correct. The clinical experts noted the technologies might not work as well if these conditions were not well represented in training datasets. Because of this, NICE recommends more research on all 10 AI technologies for people with diagnosed IBD or Lynch syndrome. Access for these patients should be through research or non-core funding.

Sam: Were there any other gaps in the training data mentioned?

Maya: Yes, equality considerations were raised regarding ethnicity. A clinical expert pointed out that people from ethnic minority backgrounds often present later for colonoscopy and have lower screening uptake. This could lead to underrepresentation in the datasets used to train the software.

Sam: Did the manufacturers have an answer to that?

Maya: Representatives from 2 of the companies confirmed their technologies had been trained on datasets derived from a broad range of providers and countries to ensure equal representation. Still, evidence generation over the next 4 years could help address equality by tracking how effective the AI is across different groups.

Sam: So if I am a clinical lead looking at this document, my takeaway is that AI is definitely an option, but it is not a magic bullet. I can buy one of the six approved systems, but I need to be prepared to collect data, watch my pathology costs, and know that we do not yet have the evidence to use it for our IBD patients.

Maya: And if you are a manufacturer, the message is clear. Do not just show that your tool increases the overall adenoma detection rate. You have to prove it detects advanced adenomas and sessile serrated lesions, and you have to prove it actually reduces post-colonoscopy colorectal cancer rates without bankrupting the system with false alarms.

Sam: That is the perfect memorable takeaway. AI is fantastic at spotting things, but the NHS needs proof that it is spotting the right things, the things that actually save lives.

Maya: To recap, NICE has conditionally recommended 6 AI tools to help detect colorectal polyps for a 4 years evidence generation period. The tools improve overall detection, but their impact on cancer prevention and cost-effectiveness remains uncertain.

Sam: We have linked the full NICE guidance in the show notes so you can read it for yourself. And remember, this podcast is for informational purposes only and is not medical advice.