Auditors and authorities regularly discuss with medical device manufacturers what constitutes the state of the art. These discussions are becoming more intense against the backdrop of the new EU regulations.
Manufacturers are facing delays in approvals, unnecessary product redesign, and costly clinical investigations or clinical performance studies for in vitro diagnostic devices (IVDs).
1. Why it is so difficult to determine the state of the art
Almost all manufacturers are willing to comply with regulatory requirements and develop products by the “state of the art”. Unfortunately, many obstacles are being placed in their way.
a) Lag of harmonised standards
Only a few standards have been harmonised for MDR and IVDR. More standards are to be expected in 2028 (see publications in the Official Journal). In the past, these harmonised standards usually represented a good consensus between manufacturers and authorities or notified bodies.
b) No definition of terms
Although the two regulations use the term “state of the art” more than 20 times and require compliance with this “state of the art,” they fail to define the term. Anyone hoping to find clarity in other sources is confronted with the additional question of the state of science and how it differs from the state of the art.
c) Inconsistent use of the term
Although the lack of a definition is bad enough, the EU regulations (MDR, IVDR) also use different variants of the term. There is not only
- the “state of the art,” but also
- the “generally acknowledged state of the art,”
- and the “current state of the art“.
d) Incorrect and inconsistent translations
Even those who believe they can differentiate between these variants will feel overwhelmed when comparing the German and English versions of the regulations. For example, the English version requires the plan for clinical evaluation to “take into account the state of the art.” The performance evaluation plan for IVDs must contain a “description of the state of the art.” In the German versions of the MDR and IVDR, the relevant passages state that the “latest state of the art” must be considered.
The German versions require the “latest state of knowledge” and the “latest medical knowledge,” while the English version always refers to the “state of the art.” These discrepancies are not isolated cases!
e) Notified bodies do not make a clear statement
Even upon explicit request, the German notified bodies were unable or not permitted to make a clear statement as to whether and, if so, which standards or versions of standards manufacturers should refer to when registering their products:
- Standards harmonized under the directives
- Standards intended for harmonization under the MDR/IVDR
- The latest version of the standards in each case
f) EU does not want definition by industry
In the Standardisation Request (M/565 of 15 May 2020), the EU Commission explicitly requires in Annex III, Part A, Chapter 1 that harmonised standards must not define relevant terms if these have not been described in the MDR or IVDR.
This opens the door to discussions during audits or reviews of technical documentation if the manufacturer refers to a definition in a (harmonised) standard. CEN and CENELEC consequently rejected the standardisation request.
Due to the poor usability of the EU website, you can also download the document directly here:
Thanks to Beat Keller for this valuable addition!
Conclusion
Manufacturers must develop their products according to the state of the art and comply with this standard. Unfortunately, manufacturers are left to their own devices when determining what constitutes the state of the art and how it is defined.
2. State of the art vs. state of science
a) Preliminary remark
There is no universally applicable, industry-independent definition of these two terms. The following explanations are, therefore, limited to the field of medical devices and IVDs.
b) Definitions of the ISO 14971:2019
The third edition of ISO 14971 has added the missing definition of “state of the art”:
Developed stage of technical capability at a given time as regards products, processes (3.14) and services, based on the relevant consolidated findings of science, technology and experience.
ISO 14971:2019
The standard supplements the definition with the following note:
The state of the art embodies what is currently and generally accepted as good practice in technology and medicine. The state of the art does not necessarily imply the most technologically advanced solution. The state of the art described here is sometimes referred to as the “generally acknowledged state of the art”.
ISO 14971:2019
This note is constructive for manufacturers:
- Manufacturers can equate the “state of the art” with the “generally recognized state of the art,” thus avoiding unnecessary discussions about the difference.
- The requirements do not necessarily consist of achieving the “most technically advanced solution.”
c) Product life cycle and differentiation
The development of products and technologies goes through several phases (Source):
|
Phase | Activity, objective | Typical participants | Example |
Basic research | Generating new knowledge without pursuing a specific objective in terms of a product | Universities, large-scale research | Image recognition with machine learning (ML) |
Applied research | Generating new knowledge to enable the development of improved or new components, products, or procedures | Universities, colleges | Transfer of these procedures to oncology |
| Experimental development | Combine the results of applied research with existing (proven) technologies to minimize development risks, i.e., find out how specific products and procedures can be procured and developed | Manufacturers, development service providers | Development of an MVP |
Product development | Develop or further develop a specific product in order to obtain approval and ensure its success on the market | Manufacturers, development service providers | Development of the specific product in accordance with MDR or IVDR |
These products are placed on the market, revised, and withdrawn from the market at the end of their product life cycle.
According to ISO 14971, state-of-the-art corresponds to generally accepted good practices in technology and medicine, not necessarily to the most technically advanced solution. The most technically advanced solution marks the boundary between the state of science and the state of the art. It is unclear whether MDR and IVDR mean exactly this boundary when they refer to the “state of the art.”
c) Example
The following example illustrates the differences:
Status | Encryption |
State of science | Quantum encryption |
State of the art | Recommendations and key lengths of the BSI (e.g. according to BSI TR-02102-1) |
Outdated status | WEP encryption of WLANs |
d) Problems
Unfortunately, many regulations do not precisely distinguish between the state of the art and the state of science. For example, MEDDEV 2.7/1 appears to equate “state of the art” and “current knowledge” in some cases. However, the latter corresponds more closely to the published state of science.
3. Regulatory requirements for the state of the art
a) Objectives of the regulations
The regulations ensure that products offer the highest possible benefits and safety, while newer products often offer greater benefits. However, their technologies are not as well established, so the benefit-risk ratio is not necessarily better.
Even with obsolete products, the risks – although low – are higher than the benefits of using them. Products that correspond to the state of the art offer an optimal benefit-risk ratio.
The regulations should also ensure that the requirements for products are not so high that manufacturers can no longer comply with them in an economically viable manner. After all, products that offer an excellent benefit-risk ratio but never make it to market are of no benefit.
Whether the authors of the regulations pursue this second aspect as an objective sometimes seems questionable.
b) Requirements of MDR and IVDR
MDR and IVDR require state-of-the-art technology in the following areas, among others:
- Definition of performance requirements (MDR Annex I.1. and IVD Annex I.9.1)
- Determination of the benefit-risk ratio and risk acceptance (Annex I.1.)
- Selection of risk-minimizing measures (Annex I.4)
- Software development (Annex I.17.2 of the MDR and Annex I.16.2 of the IVDR)
- Planning and conducting clinical evaluations or performance evaluations of IVDs (Annex IX.2.1 and Annex XIII.1)
- Conducting clinical investigations (Annex XV, Chapter II.3) or clinical performance studies for IVDs (Annex XIII.2.3.2)
The IVDR also requires state-of-the-art technology for:
- Planning post-market performance follow-up (PMPF) (Annex XIII.5.2)
- Scientific advice from EU reference laboratories (Article 100)
c) MEDDEV 2.7/1
The guideline for clinical evaluations, MEDDEV 2.7/1 rev. 4, formulates very similar requirements. It also obliges manufacturers to determine the state of the art and to take it into account when:
- determining the benefit-risk ratio
- updating the clinical evaluation
- determining the clinical performance and the clinical safety profile
- searching the literature
Benefits and risks should be specified, e.g. as to their nature, probability, extent, duration and frequency. Core issues are the proper determination of the benefit/risk profile in the intended target groups and medical indications, and demonstration of acceptability of that profile based on current knowledge/ the state of the art in the medical fields concerned.
When updating the clinical evaluation, the evaluators should verify: compatible with a high level of protection of health and safety and acceptable according to current knowledge/ the state of the art;
Sufficient detail of the clinical background is needed so that the state of the art can be accurately characterised in terms of clinical performance, and clinical safety profile.
Brief summary and justification of the literature search strategy applied for retrieval of information on current knowledge/ the state of the art, including sources used, search questions, search terms, selection criteria applied to the output of the search, quality control measures, results, number and type of literature found to be pertinent. Appraisal criteria used.
Applicable standards and guidance documents.MEDDEV 2.7/1 rev. 4
4. Determining the state of the art
a) Step 1: Consider preliminary issues and identify initial criteria
Manufacturers must determine the state of the art.
Please note: There is no such thing as “the” state of the art for a product! Instead, it is necessary to determine the state of the art in various aspects of clinical benefit, safety, and performance (see Fig. 3).
Manufacturers are required to document the results of this analysis in the clinical or performance evaluation for IVDs and describe how they conducted it.
b) Step 2: Develop key questions
Manufacturers should consider key questions to help them determine the state of the art. These key questions can be divided into two groups:
Alternatives
- What alternative products, e.g., predecessor and competing products, are available?
- For IVDs: Are reference methods available, and if so, which ones?
- Are there alternative product categories? (e.g., an MRI instead of a CT scan)
- Are there alternative procedures or alternative diagnostic options in the case of IVDs? (e.g., chemotherapy instead of radiation therapy, qPCR instead of pathogen culture, manual image evaluation instead of ML-supported evaluation)
- What alternative materials and technologies are available?
- What alternatives exist in terms of other product designs?
- Are there other production processes?
Comparison of alternatives
- How do these alternatives compare in terms of clinical benefit?
Possible aspects:- Sensitivity and specificity of a diagnosis
- Number of days in hospital
- Percentage of patients cured
- Reduction in pain level on a 10-point scale
- Possibility of diagnosing different clinical subpopulations Aspekte:
- How do these alternatives compare in terms of clinical benefit?
Possible aspects:- The lifespan of an implant
- Accuracy with which an RF pulse is generated
- Number of possible reprocessing cycles
- Resolution of an imaging procedure
- Detection limit of a diagnostic analysis method
- Time until the diagnosis is available
- How do these alternatives compare in terms of clinical benefit?
Possible aspects:- Mean time between failures
- Safety against incorrect entries and other user errors
- The probability of detecting air bubbles in an infusion tube
- Stability if the product is dropped
- Fewer side effects
- Lower risks
c) Step 3: Search sources
Guides such as MEDDEV 2.7/1 specify concrete sources of information. The following sources are recommended for manufacturers:
- Authority databases, e.g., SwissMedic, BfArM, FDA
- Clinical literature, e.g., via PubMed and Embase
- Registers
- Trade fair and product catalogs, instructions for use
- Technical databases on, e.g., material properties
- Standards, regulatory guidelines
- Medical guidelines from professional associations
- Information from manufacturers about errors, e.g., bug reports and release notes
- Own post-market data, e.g., customer complaints, service reports, post-market clinical follow-up or performance follow-up for IVDs, log files
- Laboratory tests
Standards often represent a minimum consensus reached by a standards body and, therefore, do not always reflect the state of the art. For example, a standard such as IEC 62304 will likely serve as an absolute minimum rather than meeting the requirement of describing the state of professional software development.
d) Step 4: Evaluate sources
First, manufacturers should examine each aspect addressed in the key questions individually. At the end of this examination, the following questions should be answered:
- Is there any information that allows conclusions to be drawn about this aspect?
- What alternative products, technologies, procedures, etc. exist?
- Which of these are superior, and which are inferior?
- Does this information represent the state of the art or science?
The development of standards and guides usually takes several years. Therefore, they represent the state of the art rather than the state of science.
This evaluation now serves as input, for example, for
- clinical evaluation or performance evaluation,
- risk management files,
- development,
- post-market surveillance plan.
Ensure the alternative procedures, products, and technologies are comparable. Comparability also applies to the patient population, indications and contraindications, intended users, and intended use environment.
e) Step 5: Repeat the analysis periodically
What we consider state-of-science today may soon be state-of-the-art and quickly become obsolete. In machine learning procedures, for example, this cycle sometimes lasts only a few months.
Regulations, therefore, require manufacturers to update their devices regularly. This is addressed, among other things, during post-market clinical follow-up (PMCF) or, in the case of IVDs, during post-market performance follow-up (PMPF). In most cases, this should not exceed an annual cycle.
5. Conclusion and summary
a) Challenges and solutions
The EU regulations make it unnecessarily difficult for manufacturers to comply with the central requirements of these regulations due to the lack of definitions, the inconsistent use of the term “state of the art,” and annoying translation errors.
The third edition of ISO 14971 provides a much-needed definition. The fact that this standard is not intended for harmonization and, therefore, does not represent the state of the art seems like an afterthought.
b) Aspects that constitute state of the art
Manufacturers should treat the following terms as equivalent:
- State of the art
- Generally acknowledged state of the art
- Latest state of the art
ISO 14971 allows these simplifications.
The search for alternative products, procedures, and technologies is essential. Manufacturers must compare and evaluate these safety, performance, and clinical benefit alternatives.
The statement that a product corresponds to the state of the art is justified if it applies to all relevant aspects such as benefits, performance (e.g., essential performance characteristics), and safety.
However, this also means that an alternative product superior to a manufacturer’s product in one aspect does not necessarily require the manufacturer to follow suit. This assessment must consider all elements that constitute the state of the art.
c) Role of standards
With older standards, manufacturers will find it increasingly difficult to argue that they represent the state of the art. This assumption can be made for the latest editions.
Even the most recent standards do not claim to reflect the state of the art.
Unfortunately, the notified bodies have not yet been persuaded to make a clear statement as to whether and, if so, which standards can or must be used to determine the state of the art.
d) A continuous process
The state of the art is not a constant. Manufacturers must determine this state continuously, typically at least once a year. Manufacturers can comply with this obligation within the framework of post-market surveillance and clinical or post-market performance follow-up (for IVDs).
If you are unsure whether your product complies with the state of the art and will pass the next audit without any problems, please contact us, e.g., via our free MicroConsulting service.
The Johner Institute supports you with its Post-Market Radar to automatically monitor the state of the art.
Change history:
- 2020-09-28: Chapter 1.f) added: EU also prevents standards by prohibiting relevant definitions.
