An MIT-led team is proposing a framework to make medical AI systems less overconfident and more collaborative, arguing that today’s tools risk misleading clinicians by masking uncertainty. The approach, detailed in BMJ Health and Care Informatics, adds modules that gauge a model’s confidence, flag mismatches between evidence and certainty, and prompt clinicians to order specific tests or seek specialist input. A core element, the “Epistemic Virtue Score” from University of Melbourne researchers, acts as a self-check on diagnostic predictions. The group is working to implement the design in systems trained on the MIMIC database and to pilot it within Beth Israel Lahey Health, with potential uses in imaging and emergency care. The effort also tackles bias by broadening data inputs and convening cross-disciplinary stakeholders to question dataset completeness. The project was supported by the Boston-Korea Innovative Research Project through Korea’s health industry agency.
Related articles:
Ethics and governance of artificial intelligence for health
Clinical Decision Support Software: Guidance for Industry and FDA Staff
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Reporting guidelines for clinical trials involving artificial intelligence: CONSORT-AI and SPIRIT-AI
On Calibration of Modern Neural Networks
