You

User-centred design – made for you, the user – is a central feature of responsible, or ethical AI for healthcare. That’s because the best, most successful, algorithms designed for use in the healthcare system are those that are designed with and for healthcare professionals. Healthcare professionals are best positioned to know what are useful clinical problems to ‘solve’ using AI, what features (or variables) are most important to include in clinical algorithms, how to ensure the outputs of algorithms are helpful in a clinical situation, and how to embed clinical algorithms into clinical pathways. 

Healthcare professionals should be involved in all stages of design from data creation (i.e., recording information in an electronic health record that may be used to train an AI algorithm), ideation (i.e., coming up with the problem statement), to data selection, model selection, training, validation, evaluation, and implementation. They should also be heavily involved in user testing. It’s not good enough to have a highly accurate algorithm that no clinician wants to use when caring for their patients because the user interface is too complicated, or the data entry process is too cumbersome. 

Healthcare professionals can also play a crucial role in the use of AI in healthcare by understanding its use and being aware of its limitations. For example, healthcare professionals can prevent harm coming to patients from the use of AI algorithms by being aware of ‘automation bias.’ Automation bias is where humans, in this instance healthcare professionals, assume that computers are always right and so take any information provided by a computer at face value. Clinicians or whoever is using them should feel comfortable and confident enough to review the outputs of any AI algorithm used in their practice and to question them if they do not think they look correct or if they feel ‘off.’ 

Healthcare professionals can also help to continue to refine AI algorithms once they have gone live in clinical systems by providing regular feedback to the development team on what is working well and what is not. In this way, the AI algorithms and the healthcare professional should be seen as a team, not as two separate entities trying to compete against each other.

It is also worth developers of algorithms intended to be used in healthcare being aware of the factors that increase the willingness of healthcare professionals to adopt AI. 

Healthcare practitioners are willing to adopt AI when the model: 

• is of clear clinical value 
• is accompanied by a user-friendly and clinician-centric interface
• has a decision-making process that is sufficiently transparent/explainable 
• has been independently validated and both its potential and limitations have been made clear
• allows for contextualisation and clinical judgement 
• clearly improves patient outcomes
• has had adequate training on how to use it. 

Of course, it is not just the acceptance of AI by healthcare practitioners that matters. It is also important that patients and publics are willing for AI to be used in the planning and execution of their care. Less is known about patient and public attitudes. However, in general, there seems to be a belief that the potential benefits of AI outweigh the risks, when it is believed that AI is capable of improving the process of diagnosis and treatment management.

This is, however, neither a unilaterally held nor unconditional belief. Some patients, for example, worry about ‘uniqueness neglect’, fearing that AI models may ignore their unique characteristics and circumstances, resulting in poorer quality care and worse outcomes, while others will only accept the use of AI in a purely supportive function. It is, therefore, also important that healthcare practitioners are aware of these concerns and are willing to take a hands-on role in improving the public awareness and understanding of AI in healthcare, as this is likely to increase the overall social acceptability. ^.