The healthcare landscape is undergoing a revolutionary transformation as artificial intelligence technologies emerge across all medical domains. This technological revolution brings tremendous promise: AI possesses remarkable capabilities to enhance existing medical technologies, refine personalized treatment approaches, and leverage big data to address the needs of historically marginalized communities.
However, to realize these benefits, the healthcare ecosystem must guarantee that AI systems remain reliable and avoid reinforcing existing systemic biases. Experts at the MIT Abdul Latif Jameel Clinic for Machine Learning in Health (Jameel Clinic), an initiative dedicated to advancing AI research in healthcare, advocate for establishing a comprehensive framework to assist scientists and medical professionals in this crucial mission.
Advancing Fair AI in Medical Settings
The Jameel Clinic recently convened the AI for Health Care Equity Conference to evaluate cutting-edge developments in this field, featuring innovative machine learning methodologies that promote fairness, customization, and inclusion; pinpoint critical areas where healthcare delivery can be improved; and explore regulatory frameworks and policy implications.
Nearly 1,400 participants joined virtually to learn from pioneers across academia, industry, and government who are dedicated to enhancing healthcare equity and addressing the technical obstacles in this domain while identifying viable pathways forward.
During the event, Regina Barzilay, the School of Engineering Distinguished Professor of AI and Health and the AI faculty lead for Jameel Clinic, alongside Bilal Mateen, clinical technology lead at the Wellcome Trust, revealed the Wellcome Fund awarded to Jameel Clinic to develop a collaborative platform supporting equitable AI healthcare solutions.
The initiative's primary objective extends beyond solving theoretical questions or achieving specific research milestones—it aims to tangibly enhance patient wellbeing globally. Researchers at Jameel Clinic emphasize that AI healthcare tools must be designed with versatility in mind, serving diverse populations rather than being tailored to a single demographic. To accomplish this, each AI solution requires rigorous evaluation across various populations, typically spanning multiple cities and countries. Additionally, the project aims to provide open access to the broader scientific community while safeguarding patient confidentiality, thereby democratizing these efforts.
"What has become increasingly clear to us as funders is that the nature of scientific inquiry has fundamentally transformed in recent years, becoming inherently more computational than ever before," notes Mateen.
Practitioner Insights
This urgent call to action responds to the healthcare challenges of 2020. At the conference, Collin Stultz, a professor of electrical engineering and computer science and a practicing cardiologist at Massachusetts General Hospital, discussed how healthcare providers typically determine treatment approaches and why these methods frequently fall short.
In basic terms, physicians gather patient information and utilize it to formulate treatment strategies. "The decisions clinicians make can enhance patients' quality of life or extend their longevity, but these determinations don't occur in isolation," explains Stultz.
Instead, he highlights that a complex network of influences can shape how patients receive care. These forces range from highly specific to broadly applicable, extending from factors unique to individual patients to provider biases—such as knowledge derived from flawed clinical studies—to systemic issues like unequal healthcare access.
Data Considerations and Algorithm Development
A central theme at the conference focused on how racial representation is handled in datasets, particularly given that race is a fluid, self-reported variable often defined in imprecise terms.
"The disparities we're attempting to address are substantial, conspicuous, and persistent," states Sharrelle Barber, an assistant professor of epidemiology and biostatistics at Drexel University. "We must critically examine what this variable truly represents. Fundamentally, it serves as an indicator of structural racism," Barber asserts. "It's not biological or genetic. This is something we've emphasized repeatedly."
While certain health aspects are purely biological—such as hereditary conditions like cystic fibrosis—the majority of health conditions involve more complex factors. According to T. Salewa Oseni, an oncologist at Massachusetts General Hospital, research tends to overemphasize biological influences on patient health and outcomes, while socioeconomic factors warrant equal consideration.
Even as machine learning experts identify existing biases within healthcare systems, they must also confront algorithmic limitations, as various conference speakers highlighted. They must navigate critical questions throughout all development phases, from initially defining the technology's purpose to monitoring real-world implementation.
Irene Chen, a MIT PhD student specializing in machine learning, evaluates each development stage through an ethical framework. As a first-year doctoral student, Chen discovered that an "off-the-shelf" algorithm designed to predict patient mortality generated significantly different outcomes based on racial categories. Such algorithms can have tangible consequences, influencing how hospitals distribute resources among patients.
Chen endeavored to understand why this algorithm produced such inconsistent results. In subsequent research, she identified three distinct bias sources that could be isolated from any model. The first is "bias" in a statistical sense—perhaps the model doesn't align well with the research question. The second is variance, controlled by sample size. The final source is noise, unrelated to model adjustments or increased sample sizes. Instead, it indicates issues during data collection, occurring well before model development. Many systemic inequities, such as limited healthcare access or historical medical distrust within certain communities, become "consolidated" as noise.
"Once you identify which component is responsible, you can propose a solution," Chen explains.
Marzyeh Ghassemi, an assistant professor at the University of Toronto and soon-to-be professor at MIT, has investigated the balance between anonymizing sensitive health data and ensuring fair representation for all patients. In approaches like differential privacy—a machine learning technique providing uniform privacy protection for all data points—individuals who are too "distinct" within their cohort begin to lose predictive significance in the model. In health data, where studies often underrepresent certain groups, "minorities are those who appear unique," Ghassemi observes.
"We need to generate more data, and it must be diverse data," she states. "The robust, private, equitable, high-quality algorithms we're developing require large-scale datasets for research purposes."
Beyond Jameel Clinic, other organizations recognize the potential of leveraging diverse data to create more equitable healthcare. Anthony Philippakis, chief data officer at the Broad Institute of MIT and Harvard, presented on the All of Us research program—a groundbreaking National Institutes of Health initiative aiming to address gaps for historically underrepresented populations by collecting observational and longitudinal health data from over 1 million Americans. This database seeks to reveal how diseases manifest across different demographic groups.
One of the most significant questions at the conference—and in AI generally—concerns policy. Kadija Ferryman, a cultural anthropologist and bioethicist at New York University, notes that AI regulation remains in its early stages, which presents opportunities. "There's considerable potential to develop policies with fairness and justice principles in mind, rather than attempting to reform existing regulations," Ferryman suggests.
Even before formal policies emerge, developers should adhere to certain best practices. Najat Khan, chief data science officer at Janssen R&D, urges researchers to be "exceptionally systematic and meticulous from the outset" when selecting datasets and algorithms; thorough assessment of data sources, types, gaps, diversity, and other factors is essential. Even extensive, commonly used datasets contain inherent biases.
Even more fundamentally, we must welcome diverse future researchers into the field.
"We must ensure we're nurturing and investing in data science talent with diverse backgrounds and experiences, providing opportunities to address patient issues they genuinely care about," Khan emphasizes. "If we succeed in this, you'll observe—and we're already beginning to see—a fundamental shift in our talent landscape—a more multiskilled, diverse workforce."
The AI for Health Care Equity Conference was collaboratively organized by MIT's Jameel Clinic; Department of Electrical Engineering and Computer Science; Institute for Data, Systems, and Society; Institute for Medical Engineering and Science; and the MIT Schwarzman College of Computing.
