Company Spotlight: Ultromics EchoGo Amyloidosis (1.0) - Finding Disease in the Echo

Sam: Imagine an algorithm that looks at a routine heart ultrasound and flags a rare, dangerous condition with an 84.5% sensitivity. But here is the catch, it completely refuses to make a guess on 14.0% of the scans.

Maya: Today we are looking at exactly that tool, the EchoGo Amyloidosis 1.0 by Ultromics Limited. And just to be clear right up front, Sam and I are AI-generated voices, and this is a summary of the official 510(k) summary document published by the U.S. Food and Drug Administration on November 15, 2024.

Sam: This is a Company Spotlight episode, but we want to emphasize that this is completely independent. We are not sponsored, we are only looking at the public FDA summary, not marketing materials. We are going to dig into exactly what this software does, how it was tested, and where the limitations are.

Maya: Let us start with what the EchoGo Amyloidosis 1.0 actually is. The FDA document classifies it as an adjunctive cardiovascular status indicator. Specifically, it is an automated machine learning-based decision support system.

Sam: And who is it actually for?

Maya: The indications for use are very specific. It is indicated as a screening tool for adult patients aged 65 years and over with heart failure who are undergoing cardiovascular assessment using echocardiography. When the interpreting physician uses it, the device provides information alerting the physician for referral to confirmatory investigations.

Sam: So this is not diagnosing the patient on its own.

Maya: Exactly. The document explicitly warns that patient management decisions should not be made solely on the results of the EchoGo Amyloidosis 1.0 analysis.

Sam: Okay, so how does it work under the hood? What is the input?

Maya: It requires a 2D echocardiogram of an apical four chamber view, which the document calls an A4C. That echo needs to be acquired without contrast and it has to contain at least one full cardiac cycle. You feed that DICOM file into the system, and the AI algorithm, which uses a convolutional neural network, gives you a binary output. It basically says the scan is suggestive of the presence or absence of cardiac amyloidosis.

Sam: And it does this automatically, right? The summary says it is fully automated without a graphical user interface.

Maya: That is right. It can be hosted on Ultromics platform or on third-party infrastructure. It just interfaces with external applications like a Picture Archiving and Communication System, or PACS.

Sam: Because this is a 510(k) clearance, the company had to prove substantial equivalence to a predicate device. What did they compare it to?

Maya: The primary predicate device is actually another product from the same company. It is called EchoGo Heart Failure, which was cleared under a previous 510(k). That predicate device also takes an apical four-chamber view, but it was designed as a diagnostic aid for detecting heart failure with preserved ejection fraction, or HFpEF.

Sam: Wait, really? They are using a heart failure algorithm as the predicate for an amyloidosis algorithm?

Maya: Yes, legally speaking, the intended use is the same. Both are software-only devices that act as adjunctive cardiovascular status indicators providing information on the cardiovascular state of patients undergoing routine echocardiography. The FDA accepted this reasoning to clear the device.

Sam: There are some differences though, right?

Maya: Definitely. The predicate device, EchoGo Heart Failure, requires the input echo to be assessed to have an ejection fraction of 50% or greater. The subject device, EchoGo Amyloidosis 1.0, does not have that ejection fraction requirement. Also, the predicate was indicated for adults over 25 years of age, whereas the new device is strictly for adults over 65.

Sam: Let us get to the part everyone cares about. The performance numbers. How well does it actually find amyloidosis?

Maya: To prove performance, they ran a retrospective case-control study using data from multiple sites spanning nine states in the USA. For their testing data cohort of patients over 65 with heart failure, they had a total of 1,164 patients.

Sam: How did that break down between positive cases and negative controls?

Maya: They had 749 controls and 415 cases. On that group, the device achieved a sensitivity of 84.5% and a specificity of 89.7%.

Sam: Those are solid numbers for a software tool looking at a standard echo. But we have to talk about positive predictive value, because the document breaks this down in a really interesting way based on prevalence.

Maya: This is crucial for anyone buying this for their health system. In their testing dataset, the native disease prevalence was 36.7%. That is a very high concentration of sick patients, which is normal for a case-control study. At that 36.7% prevalence, the Positive Predictive Value was 82.7%.

Sam: Right, meaning if the AI said a patient had it, there was an 82.7% chance they actually did. But the real world does not look like a case-control study.

Maya: Exactly. The FDA summary includes an inferred estimate based on a more realistic disease prevalence of 2.2%. If you assume only 2.2% of the screened population actually has the disease, that Positive Predictive Value drops all the way down to 15.6%.

Sam: Wow, 15.6%. So in a real-world screening scenario, if the AI flags someone, the vast majority of those alerts would be false positives.

Maya: Yes, based on those prevalence estimates. However, the Negative Predictive Value at that 2.2% prevalence is 99.6%. So if the tool says the patient does not have it, you can be highly confident they do not.

Sam: That makes sense for a screening tool. You cast a wide net, accept some false positives to refer for confirmatory testing, but you do not want to miss the true positives. But I want to go back to that stat I mentioned at the very beginning. The no-classifications rate.

Maya: This is one of the most transparent parts of the document. The system has a predetermined uncertainty threshold. If an input study demonstrates a higher than acceptable level of uncertainty, the device just outputs a no classification report. In the clinical investigation, it did this on 14.0% of the data.

Sam: So for 14.0% of the scans, the AI basically says, I cannot tell you.

Maya: Right. And those primary performance stats we just talked about, the 84.5% sensitivity and 89.7% specificity, those exclude the uncertain predictions.

Sam: What happens if you include them? The document actually runs that math, right?

Maya: It does. To represent the worst-case scenario, they calculated what happens if you take all those no-classification reports and assume the device got them wrong. When you do that, the sensitivity drops to 72.8% and the specificity drops to 76.0%.

Sam: That really shows how much the algorithm relies on tossing out borderline or poor-quality data to maintain its high accuracy. As a clinician, how do you feel about a tool that just rejects 14.0% of cases?

Maya: I actually prefer it. I would much rather an algorithm tell me the image quality or the anatomical presentation is too ambiguous than just guess and give me a false sense of security. But it does mean you still need traditional workflows for those patients.

Sam: Let us dig into the demographics, because the summary provides a massive breakdown of how the tool performed across different patient sub-groups. This is where we see if the AI works equally well for everyone.

Maya: The testing dataset of 1,164 patients had a racial breakdown of 58.4 to 66.0% White, 24.6 to 26.8% Black, and 1.9 to 11.8% Asian, depending on the case or control split.

Sam: How did the sensitivity hold up across those groups?

Maya: For White patients, sensitivity was 84.8%. For Black patients, it was slightly higher at 85.9%. But there were some noticeable differences in other demographics. For instance, between sexes. The sensitivity for male patients was 87.5%, but for female patients, it dropped to 73.9%.

Sam: That is a significant drop for female patients. Does the document explain why that happens?

Maya: No, it just provides the raw numbers in the subgroup tables. It does not offer a clinical explanation for the performance gaps.

Sam: What about body mass index? We know BMI can affect ultrasound image quality in general.

Maya: We see exactly that in the data. For patients with a BMI between 18.5 and 25, the sensitivity was 88.7%. But for patients with a BMI over 35, the sensitivity fell sharply to 68.8%.

Sam: That makes total physical sense. Ultrasound waves have a harder time penetrating tissue in higher BMI patients, which degrades the image the AI has to work with.

Maya: Right. We also see variations based on the anatomy of the heart itself. The summary looks at Left Ventricular Posterior Wall Thickness. When the wall was thicker than 16 millimeters, sensitivity was fantastic at 94.2%. When the wall was only 10 to 12 millimeters, sensitivity fell to 80.0%.

Sam: So if the disease is more advanced and the heart wall is visibly thicker, the AI has a much easier time spotting it. That is logical.

Maya: The document also breaks down performance by the manufacturer of the ultrasound machine used to capture the image. Sensitivity on Philips machines was 81.9%, while on GE machines it was 88.8%. So there is some variability depending on the hardware in your clinic.

Sam: Are there other specific clinical conditions they tested for interference?

Maya: Yes, they broke it down by several comorbidities. For patients with a history of Atrial Fibrillation, sensitivity was 83.2%, compared to 83.5% for those in Sinus Rhythm. So very little difference there. For patients with Coronary Artery Disease, it was 84.7%. And for patients with Aortic Stenosis, it was 85.7%. So the presence of those specific conditions did not seem to drastically derail the algorithm.

Sam: That is reassuring. The tool seems pretty robust across different clinical presentations, even if it struggles a bit with higher BMIs or female patients. We should also touch on repeatability and reproducibility. Software shouldn't give you two different answers for the same image.

Maya: And it does not. The repeatability testing, which compared the device output from a single DICOM clip analyzed multiple times, demonstrated 100% positive and negative agreement. For reproducibility, which compared device outputs from different DICOM clips taken from the same individual, it showed an average positive agreement of 85.5% and a negative agreement of 79.9%.

Sam: So if you scan the same patient twice and feed both clips to the AI, it will agree with itself 85.5% of the time on positive cases. That is not perfect, but again, that is likely due to the slight variations in how the human sonographer captured the two different clips.

Maya: Exactly. That is the reality of ultrasound data. Before we wrap up, we need to highlight the limitations mentioned in the FDA summary. First, the clinical performance testing was retrospective. It was validated on historical cohorts rather than in a prospective, live clinical setting.

Sam: Right, which means we do not yet have data in this document showing how much it actually alters physician behavior or patient outcomes in real time.

Maya: And second, the summary explicitly states that the essential performance of EchoGo Amyloidosis 1.0 was not evaluated on patients with prior heart transplantation.

Sam: So if your patient has a transplanted heart, this tool is not cleared for them.

Maya: Correct. Ultromics also notes that to meet special controls for this type of device, they conducted formative and summative usability assessments with 18 clinicians who specialize in cardiology to ensure the risk of misinterpreting the status indicator was appropriately mitigated.

Sam: That is an important step. You can build the best algorithm in the world, but if the doctor misreads the output, it is useless.

Maya: Time for a quick recap. EchoGo Amyloidosis 1.0 is an automated AI tool cleared to screen for cardiac amyloidosis in patients 65 and older with heart failure. It achieved an 84.5% sensitivity and 89.7% specificity in retrospective testing, though it rejects 14.0% of images as too uncertain to classify.

Sam: And remember, at a real-world disease prevalence of 2.2%, the positive predictive value drops to 15.6%, reinforcing its role as a screening tool that requires confirmatory testing, not a standalone diagnostic.

Maya: You can find a link to the FDA document in the show notes. Please remember that this podcast is not medical advice, and is for informational purposes only.