Artificial intelligence in medicine

More and more medical devices are using artificial intelligence to diagnose patients more precisely and to treat them more effectively.

Although a lot of devices have already been approved (e.g., by the FDA), a lot of regulatory questions remain unanswered. With the AI Act, the EU has published a regulatory framework for AI, and not just in medical devices.

This article describes what manufacturers whose devices are based on artificial intelligence procedures should pay attention to.

Please also note the guidelines on the use of artificial intelligence (AI) in medical devices, which the Johner Institute has developed together with AI experts and notified bodies. Find out more in chapter 4.

1. Artificial intelligence: what is it?

The terms artificial intelligence (AI), machine learning, and deep learning are often used imprecisely or even synonymously.

General definitions

The term “artificial intelligence” (AI) itself leads to discussions about, for example, whether machines are actually intelligent.

We will use the definition below:

So it is about machines ability to take on tasks or make decisions in a way that simulates human intelligence and behavior.

A lot of artificial intelligence procedures use machine learning, which is defined as follows:

And deep learning is, in turn, part of machine learning and is based on neural networks (see Fig. 2).

This gives us the following taxonomy:

Definition of the EU

The EU defines the term “AI system” in the AI Act:

“An AI system is a machine-based system designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.”

AI Act, Article 2 (Definitions), Section 1

This definition reveals three conditions that make a “machine-based system” an “AI system“:

condition	comment
designed to operate with varying levels of autonomy	The word “varying” is vague, so that this partial condition hardly restricts what falls under this definition.
may exhibit adaptiveness after deployment	The word “may” means that this adaptability can exist but does not have to. This partial condition, therefore, does not restrict what falls under this definition.
infers, from the input it receives, how to generate outputs …	The system derives how it generates the outputs from the inputs. So, it is not a question of the system deriving the outputs from the inputs, but how (i.e., the way?) it does this.

Conclusion: It seems that only the third condition effectively defines what an AI system is. And this condition is difficult to understand.

b) Procedures

The assumption that artificial intelligence in medicine mainly uses neural networks is not correct. A study by Jiang et al. showed that support vector machines are used most frequently (see Fig. 2). Some medical devices use several methods at the same time.

The list of the most frequently and successfully used procedures is constantly changing. Procedures such as XGBoost, for example, have gained in popularity.

2. Applications of artificial intelligence in medicine

a) Overview

Manufacturers use artificial intelligence, especially machine learning, for tasks such as the following:

function	data
detecting a retinopathy	images of the eye fundus
counting and recognizing certain cell types	images of histological sections
diagnosis of heart infarctions, Alzheimer’s, cancer, etc.	radiology images, e.g., CT, MRI
detecting depression	speech, movement patterns
selection and dosage of medicines	diagnoses, gene data, etc.
diagnosis of heart diseases, degenerative brain diseases, etc.	ECG or EEG signals
detecting epidemics	internet searches
disease prognoses	laboratory values, environmental factors etc.
time-of-death prognosis for intensive care patients	vital signs, laboratory values and other data in the patient’s records

Further applications include:

• detection, analysis, and improvement of signals, e.g., weak and noisy signals
• extraction of structured data from unstructured text
• segmentation of tissues, e.g., for irradiation planning

The FDA has published an extensive list of AI-based medical devices that will be very helpful for manufacturers in order to: 

• create a clinical evaluation
• search for equivalent devices
• get ideas for new devices

It is interesting to note that the number of newly authorized AI-based devices is not increasing any further.

b) Tasks: classification and regression

The procedures are used for the purpose of classification or regression.

Examples of classification:

• decision as to whether there is a diagnosis
• deciding whether cells are cancer cells or not
• selecting a medicine

Examples of regression:

• determining the dose of a medicine
• time-of-death prognosis

3. AI from a regulatory perspective

a) Regulatory requirements

There are currently no laws or harmonized standards that specifically regulate the use of artificial intelligence in medical devices. However, many standards and best practices exist for the use of artificial intelligence procedures.

b) Possible solutions: explainability

General

Auditors should no longer be generally satisfied with the statement that machine learning procedures are black boxes.

“There are promising approaches in the current research literature on how the predictions of deep learning models can be made plausible. For example, when classifying images, it is possible to understand which input pixels are decisive for the classification (see below).

However, no standard methods have yet been established, as the current procedures have different strengths and weaknesses, and the current status quo is in a heuristic phase. However, it can be assumed that research in this area will make further progress towards explainability in the coming years.”

Many approaches are currently “only” aimed at explaining specific individual predictions based on the input data (local explainability).

Example

By using Layer Wise Relevance Propagation, it is possible to recognize which input data (“feature”) was decisive for the algorithm, e.g., for classification.

Fig. 5 shows, in the left picture, that the algorithm can rule out a number “6” primarily because of the pixels marked dark blue. This makes sense, because with a “6” this area typically does not contain any pixels. On the other hand, the right image shows in red the pixels that reinforce the algorithm’s assumption that the digit is a “1.”

The algorithm evaluates the pixels in the rising part of the digit as damaging for classification as “1.” This is because it was trained with images where the “1” is written as a simple vertical line, as is the case in the USA. This shows how important it is for the result that the training data is representative of the data that is to be classified later.

c) State of the art

Manufacturers are well-advised not to answer some auditors’ questions about the state of the art globally, but to differentiate between them:

1. Technical implementation: Relevant standards, such as those mentioned here, help prove that the development and verification or validation of the software and models conform to current best practices.

2. Performance parameters: Manufacturers should compare performance against classic procedures as well as other machine learning models and algorithms. This comparison should be based on all relevant attributes, such as sensitivity, specificity, robustness, performance, repeatability, explainability, and acceptance.

4. AI Guideline

The guideline for the use of artificial intelligence (AI) in medical devices is now available on Github at no cost.

We developed this guideline with notified bodies, manufacturers, and AI experts.

• It helps manufacturers to develop AI-based devices in compliance with the law and bring them to market quickly and safely.
• Internal and external auditors and notified bodies use the guideline to test the legal conformity of AI-based medical devices and the associated life-cycle process.

When we were writing it, it was important to us to give the manufacturers and notified bodies precise test criteria to provide for a clear and undisputed assessment. The process approach is also in the foreground. The requirements of the guideline are grouped along these processes.

5. Support

The Johner Institute supports manufacturers of medical devices that use artificial intelligence in

• developing devices in compliance with the law and placing them on the market,
• planning and carrying out corresponding verification and validation activities,
• evaluating the devices for benefit, performance, and safety,
• evaluating the suitability of the procedures (especially the models) and the training data,
• fulfilling the regulatory requirements for the post-market phase, and
• creating tailored standard operating procedures.

You can find a more complete overview here.

6. Conclusion, outlook

a) From hype to disillusion to actual practice

Artificial intelligence is currently receiving a lot of hype. A lot of “articles” praise it as either the solution to every medical problem or the start of a dystrophy in which machines will take over. We are facing a period of disillusionment. “Dr. Watson versagt” [“Dr. Watson fails”] was the title on article in issue 32/2018 of the German “Der Spiegel” on the use of AI in medicine.

It has to be expected that the media will write over-the-top and scandalized reports on cases where bad AI decisions have tragic consequences. But over time, the use of AI will become just as normal and indispensable as the use of electricity. We can no longer afford and no longer want to pay for medical staff to perform tasks that computers can do better and faster.

b) Regulatory uncertainty

The regulatory framework and best practices lag behind the use of AI. This leads to risks for patients (medical devices are less safe) and for manufacturers (audits and approval procedures seem to reach arbitrary conclusions).

The WHO feels compelled to focus more on the topic and develop a WHO guideline. The focus group “Artificial Intelligence for Healthcare” is evidence of this effort. Everyone is invited to participate.

This also applies to the guideline on the safe development and use of artificial intelligence in medical devices presented above. This guideline already formulates very specific requirements and thus helps manufacturers on the one hand and notified bodies and authorities on the other to achieve a uniform understanding of the state of the art and, thus, a common basis for product testing and audits.

---

Change history:

• 2024-01-26: Definition of “AI system” from the AI Act added to the definitions section
• 2023-11-03: FDA regulatory requirements moved to this article
• 2023-06-27: Outlook on AI regulation given in section 3.a)
• 2021-11-07: In section 2.a) evaluation of the FDA’s list added
• 2021-10-03: In section 2.a) list of the FDA added