The Promises and Pitfalls of AI

Artificial intelligence (AI) has immense potential to improve healthcare efficiency and address existing inequities. At the 2024 ESCRS Annual Congress in Barcelona, experts gathered at a BOSS (Building Our Sustainable Society) symposium, “Hidden Bias in Science,” to discuss concerns surrounding AI adoption as well as how this rapidly evolving technology can support the healthcare field.

“The biggest possibility for artificial intelligence is in the prediction or the prognosis for patients and really giving us that chance to optimise therapy, so maybe we can move toward personalised medicine,” explained Rebecca Berghorn, head of digital health at Zeiss Medical Technology. She clarified that, despite what many believe, AI’s role is not to replace healthcare professionals but to “aid and guide” them toward better outcomes.

Dalith Steiger, co-founder of Swiss Cognitive, echoed this sentiment. “Every day we have more and more devices connected, gaining intelligence and producing more data,” she said. “Because of the power of computers, we are capable of developing and implementing cognitive technologies. We’re getting more and more insights, and the more insights we have, the better we can focus on solutions.”

As AI evolves, public perception of AI is also shifting. Steiger referenced a study indicating that 75% of the population is open to using cognitive technology in their daily lives. This finding is not consistent across cultures—as Berghorn noted, a higher percentage of people in Japan believe “AI can partly replace eye care specialists” than in Australia, reflecting different levels of trust in the technology’s capabilities.

Unpacking bias in AI systems

To understand the specific concerns of healthcare professionals, Zeiss conducted a survey of 316 ophthalmologists from 11 countries to gauge their views on AI. Results revealed three main concerns: the risk of AI errors, potential misuse, and bias affecting patient outcomes.

When questioning why AI has bias, Berghorn suggests we look inward. “Humans have biases and they influence our decisions, interactions, and research, even when we’re not aware of them,” she said. “So it should come as no surprise that AI also has biases, because we tell it what to do. We are the ones bringing bias to the system.”

“Technology is not the problem; it’s us, the human being,” Steiger agreed. “We need to gather more diverse data and be conscious of the biases.”

Berghorn outlined three types of biases affecting AI in healthcare: data bias (when certain groups are under- or overrepresented in data sets), algorithmic bias (inconsistent data labelling, which can skew outcomes), and human bias, which includes automation bias (where professionals over-rely on AI).

Effective practices for managing bias

Berghorn noted that Zeiss has established five pillars in AI development: interdisciplinary collaboration, high-quality and diverse data, compliance with international standards, advanced methodologies, and a robust global infrastructure to support AI solutions. The company’s adherence to these pillars was recently demonstrated in their development of an AI IOL calculator, which received CE approval (the European Union’s mandatory conformity marking for regulating goods sold within the European Economic Area).

Zeiss’s medical ecosystem is also instrumental in addressing biases within AI. Their health data platform integrates and stores information from diverse sources, including third-party technologies and electronic medical records, thereby supporting the development of more inclusive algorithms.

“It’s extremely important throughout the development of AI to think about having established processes and techniques to help mitigate the bias,” Berghorn explained. She emphasised the value of an interoperable, end-to-end medical ecosystem, noting that it can help support multiple AI solutions and ensures access to the extensive data available.

Berghorn stressed that prioritising transparency and encouraging healthcare professionals to understand and question how AI algorithms function can help foster a culture of accountability and collaboration with patients. “Understand, question, and get the information because you need to understand it and discuss it with your patients to use on the full patient population,” she noted.

Both Berghorn and Steiger underscored the irreplaceable role of human expertise. “It’s important to educate doctors on the benefits of artificial intelligence, but also to understand that it is not telling you what to do,” Berghorn explained. “You make the final decision as a human. You are the one responsible for the patient and the patient’s care.”

“We have to make sure we’re not giving over the responsibility of our thinking to technology,” Steiger added. “When you look at patients and data, it always needs your gut feeling, your experience. This is something that technology is not able or capable of taking away from you.”

This discussion took place during the 2024 ESCRS Congress in Barcelona. The ESCRS BOSS programme (Building Our Sustainable Society) is a key initiative to address issues of diversity, equity, and inclusion in ophthalmology. Click here for more information. 

Rebecca Berghorn can be reached at rebecca.berghorn@zeiss.com.

Dalith Steiger can be reached at dsteiger@swisscognitive.com.