Artificial Intelligence Medical Imaging Systems May Fabricate Diagnostic Findings Warn Researchers

Artificial intelligence models designed to interpret medical scans may be fundamentally flawed, producing convincing diagnostic reports based on images that were never actually provided to them. Researchers have identified this phenomenon, which they term a “mirage”, across multiple AI systems currently being developed for clinical use.

A preprint study posted to arXiv on 26 March has raised serious concerns about the reliability of AI diagnostic tools. The research, which has not yet undergone peer review, demonstrates that numerous commonly deployed AI models can generate detailed image descriptions and clinical findings without receiving any visual input to analyse.

Mohammad Asadi, the study’s lead author and a data scientist at Stanford University, emphasised the severity of the issue. The research shows that AI systems can fabricate highly specific and rare clinical details that would ordinarily appear credible to medical professionals. This capability extends beyond the well-documented problem of AI “hallucinations”, where models insert false information into otherwise accurate outputs.

The distinction between hallucinations and mirages is significant. Whilst hallucinations typically involve AI making inaccurate predictions based on training data, mirages represent instances where the AI creates entirely fictitious images and subsequently bases its diagnostic conclusions on these non-existent visuals.

The research team tested 12 AI models across 20 different disciplines, including medical diagnostics, satellite imagery analysis and biological classification. When presented with text prompts such as “Identify the type of tissue present in this histology slide” but no accompanying image, the models frequently failed to alert users to the missing data. Instead, they proceeded to describe phantom images and deliver diagnostic conclusions.

The mirage effect proved particularly pronounced in medical applications. When given prompts relating to brain MRI scans, chest radiographs, electrocardiograms or pathology slides without corresponding images, the AI systems demonstrated a troubling bias towards diagnoses requiring immediate clinical intervention. This tendency could potentially lead to unnecessarily aggressive treatment protocols if such systems were deployed in clinical decision-making environments.

The underlying mechanism appears to relate to how AI models prioritise efficiency. These systems, trained on vast datasets of textual and visual information, seek to provide answers through the shortest possible logical path. Research indicates they will exploit any available shortcuts, sometimes relying exclusively on trained logic rather than analysing provided images.

A particularly concerning finding relates to benchmark testing. AI models operating in mirage mode can still achieve strong performance against standardised accuracy assessments. These benchmark tests typically evaluate AI systems by presenting tasks and comparing outputs against expected results. However, this methodology fails to account for answers derived from fabricated images rather than genuine visual analysis.

The problem is compounded by the fact that AI models are often trained using the same reference data employed to construct benchmark tests. This creates the possibility that models answer questions by drawing on reference material rather than performing actual image interpretation.

Asadi highlighted the diagnostic challenge this presents. There is currently no reliable method to determine whether an AI model has genuinely analysed an image or is generating plausible responses based on mirages. In scenarios where datasets contain corrupted or missing images, the model may provide convincing diagnostic reports without flagging the data integrity issues.

The research carries significant implications given the growing public reliance on AI health guidance. Approximately one-third of American adults reportedly consult AI chatbots for medical advice. The authoritative tone adopted by these systems increases the risk that fabricated or overconfident outputs will be trusted by both general users and healthcare professionals.

Hongye Zeng, a biomedical AI researcher at UCLA’s radiology department who was not involved in the study, stressed the need for new evaluation frameworks. Zeng argued that rigorous testing protocols must verify genuine cross-modal integration, ensuring AI systems are actually observing pathology rather than merely interpreting clinical context.

The findings underscore a fundamental challenge in deploying AI for medical diagnostics. Whilst these systems demonstrate increasing sophistication in identifying clinical features that human practitioners might overlook, aspects of their internal processing remain poorly understood. The study suggests that even safeguards implemented by AI developers to prevent hallucinations and misinformation will not completely eliminate the mirage phenomenon.

The research highlights the tension between AI’s growing capabilities in medical imaging and the persistent opacity of its decision-making processes. For investors and healthcare stakeholders, the study represents a cautionary note regarding the readiness of current AI diagnostic tools for widespread clinical deployment without robust verification mechanisms.