The AI Conundrum: Unleashing Potential or Awakening the Terminator?

Dr Birch submitted this essay to the John Henahan Writing Prize essay contest, answering the prompt “What is the potential role for AI in ophthalmology, and what are the negative implications and caveats?”. It was rated in the top 5 of 41 essays submitted by the medical editorial board of EuroTimes.

By Yarrow Scantling Birch MD

The year is 2079. The artificially intelligent (AI) machines have won—the once sacred doctor-patient relationship has decayed under the weight of ruthless standardisation. Time, a once cherished human resource for building understanding, has been replaced by AI-driven algorithms that economise on human interaction. Empathy has become an antiquated notion, trampled under the boots of scientific progression. As an ophthalmologist of the future, you work virtually to moderate the activity of numerous clinical rooms, hearing synthetic conversations, and witnessing the replacement of human touch with the sterility of a robotic handshake.

The revolution

As we enter the Fourth Industrial Revolution, technologies capable of surpassing human intelligence are emerging. AI encompasses data-driven computer systems that use algorithms and machine learning to rapidly process large quantities of data and solve complex problems. Deep learning (DL), a variant of machine learning, is inspired by the cortical architecture of our own brains. DL employs deep neural networks (DNN) to analyse inputs through interconnected artificial neurons across multiple layers.

A wealth of pixelated information

AI in ophthalmology shows promising potential in detecting retinal disease and glaucoma, the leading causes of blindness in Western society. Optical computed tomography (OCT) scans use infrared light to capture detailed retinal structures and is becoming the gold standard ophthalmic imaging tool. DNNs can digest the wealth of pixelated data on OCT into lower-level inputs. These DNNs can outperform ophthalmic experts in the diagnosis of various retinal conditions and have been developed into autonomous commercial AI systems, such as IDx-DR (LumineticsCore). Similar advancements have been made in detecting early glaucoma. Integrating AI systems in front-of-house triage and screening services has the potential to improve accessibility and affordability of eye care, as well as alleviate work from busy eye clinics. Moving forward, AI needs to expand beyond ophthalmic imaging to other ocular biomarkers such as the oculome, which holds promise for the early detection of systemic disease. This would unlock an era of personalised and whole-system medicine.

Black box learning

The lack of transparency in AI poses a significant challenge. DNNs resemble enigmatic black boxes, making it difficult to unravel their inner workings. This obscurity compromises the principle of nonmaleficence, as AI models generate outputs without clear rationales and erode trust in their validity. An unexpected revelation occurred when Google researchers developed a DNN model to predict cardiovascular risk, only to discover that gender could be identified from fundus photographs alone. This surprised the researchers due to the seemingly implausible nature of such a hypothesis, but equally, there was no means to investigate the underlying reasoning behind this output.

Collaboration, regulation, and bias

Commercial interests are driving an unregulated arms race in AI innovation with no consideration for potential harm. In ophthalmology, there are numerous patented algorithms but few fully approved regulatory devices on the AI market. Given these AI models thrive off large quality data sets, it appears wasteful that data sharing and collaborations are not being forged amongst medical technology firms. This results in greater bias within individual AI models, less standardisation of diagnostic inputs, and less generalisability to larger populations. Efforts like the EU’s AI Act aim to establish legal legislation for AI products and address issues regarding safety and bias, but these are not keeping up with the progress of AI technology.

The issue of privacy is crucial as ophthalmic images used to train AI models can be reverse-engineered to reveal confidential information. Patient autonomy is maximised when patient-derived data is obtained with informed consent, lawfulness, and compliance with data regulations. However, the use of patient-derived data for commercial ventures remains a major challenge and may create future disputes.

Liability is another ethical challenge. AI engineers responsible for developing algorithms impacting clinical care should bear equal responsibility for adverse outcomes from AI errors, especially if the technology is claimed to be autonomous. This remains a major argument for why eye care professionals still need to oversee AI decision-making and will not be replaced anytime soon. AI models still lack the ability to contextualise information within the wider clinical picture and undertake nuanced decision-making.

Conclusions

Currently, AI has the potential to revolutionise the screening and diagnostic workflow within ophthalmology. However, as we navigate this new revolution, it is crucial for humanity to take an active role in steering the trajectory of AI research. AI has the potential to open Pandora’s box, both unleashing immense potential, but also raising ethical dilemmas regarding transparency, commercial bias, and ownership of confidential data. Taking a moment to pause and ensure AI legislation keeps up pace with technological advancements will allow us to establish ethical frameworks to safeguard humans. In the immortal words of the Terminator, we embark on a future where AI machines and ophthalmologists will stand side by side to deliver outstanding patient care.

Dr Birch is a first-year trainee at Whipps Cross Hospital, Barts Health NHS Trust, London, UK. yarrow.scantling-birch@nhs.net