We frequently get asked, “Do you also offer Computerized Systems Validation?” One of the reasons for the interest is certainly: Authorities and notified bodies increasingly address the Computerized Systems Validation (CSV) in audits.
This article introduces regulatory requirements regarding “Computerized Systems Validation” and provides guidance on how you can best meet these requirements.
Computerized Systems Validation CSV: What’s that?
a) Definition and purpose of CSV
Computerized Systems Validation (CSV) is a documented process of assuring that a computerized system does exactly what it is designed to do. [1]
b) What does Validation mean in this context?
In general, validation is the “confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled” [ISO 9001:2015].
As for medical devices, this involves an “assessment by objective means of whether the specified users are enabled to achieve the specified goals (intended purpose) within the specified context of use”.
This may sound as if only the finished product, here the installed computerized system, must be validated.
However, many regulations go beyond such an understanding. Such as the FDA’s requirements: they require the whole development process to be validated; not just its last phase or final product. More on this later.
c) Computerized Systems Validation vs. DQ, IQ, OQ, and PQ
DQ, IQ, OQ, and PQ – we come across these acronyms in the context of CSV. They stand for:
- DQ: Design Qualification
- IQ: Installation Qualification
- OQ: Operation Qualification
- PQ: Performance Qualification
These qualifications are particularly required in the pharmaceutical environment, e.g., by GMP-directives, and rank among equipment validations, just as CSV. IQ, OQ, and PQ comprise certain aspects of validation / qualification:
- IQ: When installing, first inspections at the site of the customer shall ensure: the device was delivered, set up, and installed according to the specifications. The documentation is available.
- OQ: Checks ideally shortly after IQ shall confirm that the device operates according to specifications – even when reaching the specification limits.
- PQ: These tests address, inter alia, measurement accuracy (incl. calibration and adjustment).
d) Computerized Systems Validation vs. Software Validation
Computerized Systems Validation generally comprises both computer hardware as well as software running on the hardware. Particularly for standard hardware such as PC based systems, Computerized Systems Validation substantially equates to software validation. In case of specific hardware, it should for example be examined if
- the software can be installed on the respective hardware.
- the overall system operates at a satisfactory pace.
- the interplay with neighbor systems (other devices or IT systems) functions as specified.
CSV: Who requires what? Regulatory requirements
a) Overview
Requirements for validation of computerized systems can be found in:
- FDA 21 CFR part 820.70
- FDA 21 CFR part 11.10
- FDA 21 CFR part 820
- FDA guidance document regarding software validation (also addressing process software)
- FDA Computer Software Assurance for Production and Quality System Software
- ISO 13485, inter alia in Chapter 4.1.6, 7.5.2.1 and 8.2.3
- GMP directives
- GAMP 5, e.g., regarding the “risk-based approach of testing GxP systems”
b) FDA 21 CFR part 820.70
In 21 CFR part 820.70, the FDA writes:
“When computers or automated data processing systems are used as part of production or the quality system, the manufacturer shall validate computer software for its intended use according to an established protocol. All software changes shall be validated before approval and issuance. These validation activities and results shall be documented. “
c) FDA 21 CFR part 11
21 CFR stipulates in part 11.10:
“Persons who use systems to create, modify, maintain, or transmit electronic records shall employ procedures and controls designed to ensure the authenticity, integrity, and, when appropriate, the confidentiality of electronic records. Such procedures and controls shall include validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records.”
This demand for validation corresponds with what is required under 21 CFR part 820.70.
d) FDA Computer Software Assurance for production and Quality System Software
This guidance document is intended to replace the no longer up-to-date FDA guidance document on software validation in those parts that do not concern the software becoming part of the medical device or the medical device itself.
Requirements
The FDA emphasizes the risk-based approach in terms of critical thinking. Depending on the risk of the software/computerized system, specifically its critical functions, manufacturers must carry out “assurance activities” such as software tests.
Evaluation
On the positive side, the FDA’s document distinguishes the development of medical device software from the development of software used in QM systems.
On the negative side, the guidance document can be considered no big success:
Once again, it seems no software engineering experts wrote the guidance. No matter if it concerns definitions, terminology, or methods. The document uses its own concepts.
It is difficult to reduce software quality assurance to analytical quality assurance, especially software testing. Quality cannot be tested in the software. The approaches on classification of software and the proposed test techniques are inconsistent, the examples are not taking into account the concepts presented in full range.
The already large number of CSV specifications becomes even more extensive.
e) ISO 13485:2016
In its latest version, ISO 13485:2016 states the requirements for validation of computerized systems more clearly:
In Chapter 4.1.6, it is stipulated that manufacturers shall validate their computer software pursuant to documented procedures. This affects every software used in a process which controls the QM system. Validation shall not only take place before the first use but also after modifications to the software.
To put it even stronger: in Europe, CSV used to be mandatory only for manufacturing and service processes. Since ISO 13485:2016, this requirement also applies to all computerized systems being used in any of the processes regulated by the QM system. In the context of FDA, this has always been the case.
f) GAMP 5
GAMP 5 applies to medical device manufacturers as well. The authors write:
The scope has been widened [compared to Gamp 4] to include related industries and their suppliers, including biotechnology and systems used in medical device manufacturing (excluding software embedded within the medical devices).
Then, the Medical Device University is just right for you! Here, you will learn everything about Computerized Systems Validation and the corresponding regulations – and you will be able to carry them out independently.
Computerized Systems Validation CSV: How is this done?
At first, you should understand: which requirements for Computerized Systems Validation do you want to or must meet?
- Minimal requirements: Either way, you must validate your finished computer system.
- Additional requirements: If you develop the system yourself or have it developed, you must at least document the whole development process.
Both variants are addressed in the following.
The actual validation
If you intend to validate the “finished” (possibly already installed) computerized system, you must know the requirements for the computerized system. If these requirements are lacking, you must deduce them in retrospect.
Unfortunately, many companies and, in part, even authorities, lump different types of requirements together.
- Stakeholder requirements and purpose: Those are the truly important objectives and requirements different stakeholders such as users, operators, and legislation have to be able to reach their respective aims. If you have, for example, a laboratory information system, one requirement would be: The user must be able to recognize via the system whether the patient has hepatitis.
- System requirements specification: This requirement describes what the system must be able to do to fulfill the stakeholder requirements. Ideally, those system or software requirements are specified as black box requirements. In the case of laboratory information system, such a requirement would be that the system shell display all patients with at least one positive hepatitis value in red and bold font.
Strictly speaking, validation is an examination of whether the first type of requirements is met. However, it can be observed that in practice, the fulfillment of rather system or software requirements is examined during validations. However, technically speaking, this is actually a verification.
In the chapter on step-by-step instructions you can read about how to conduct this type of validation.
The complete development process
When “validating” according to the FDA software validation guidance documents, you will document the complete development process such as:
- Software requirements
- Verification of software requirements including traceability to stakeholder requirements
- Software architecture and detailed design
- Verification of software architecture and detailed design including traceability to software requirements
- Software code, code reviews, and unit tests including traceability to software architecture and detailed design respectively
- Integration and system tests including traceability to software architecture and software requirements respectively
IEC 62304 and IEC 82304 as well as the aforementioned FDA documents provide you with valuable information on how to define and implement such a development process.
Computerized Systems Validation: Step-by-step instructions
To validate a computerized system, proceed as follows:
1. Document requirements
If the requirements for your software and computerized system are not or not completely documented, catch up on this retroactively.
1. Graphical User Interfaces
- Display: Describe the appearance of your Graphical User Interface (GUI) and what must be displayed.
- Algorithms: If this display depends on calculations, then state them.
- Input: Describe which data the user can enter in which formats. Also, how the system shall react in the case of a wrong entry.
- Navigation: Specify the navigation through the system, i.e. which “screens” the system shall display when certain selections are made, e.g., by clicking.
- Access right: An aspect of the aforementioned requirement is the specification of role concepts, authorization, and authentication.
2. Data Interface
- System context: Describe with which adjacent systems your system will interact with.
- Interoperability: Specify how they are proceeded. Name the standard (e.g., TCP/IP, HTTP, bus systems), formats, classification systems, authorizations etc. Do not forget to determine how your system shall react if data is transmitted in a wrong format or volume, in an unexpected frequency etc.
- Authorizations: Describe how adjacent systems shall authenticate and authorize on your system.
- Performance: Specify how fast your system must be able to react to data transfers. This requirement will probably depend on the number of transactions, data volumes, or the like.
3. Runtime environment
- Hardware: Specify the minimum requirements for hardware such as CPU, RAM, hard drive, screen size, screen resolution etc.
- Operating system: Also determine the operating system which is required at least.
- Other software: Do you allow for other software such as anti-virus programs? Do you even suppose this, e.g., in form of a database? Do not forget to specify these requirements.
2. Specify test cases
The next step is to specify test cases, i.e. to determine
- Which data the user or the adjacent system shall enter or transfer.
- Which prerequisites must be met, e.g., regarding the software version or existing data.
- The procedure, e.g., the sequences in which to proceed i.e. to use the system, to input test data, to record results.
- Pass/fail criteria: Which values do you expect? Which outcomes meet the requirements?
- Test documentation is also part of the specification.
3. Execute and document tests
Finally, you must execute, document, and evaluate the tests according to the test specifications.
Further Tips
The following thoughts may help you with your Computerized Systems Validation:
- Systematic derivation of test data: You should not just come up with test data but systematically derive them using black box test methods such as testing methods based on equivalence classes, boundary values, decision tables, errors or conditions. Software testers can do that.
- Not just the happy path: Not only examine if your system reacts as specified but also if it deals properly with wrong or incomplete entries.
- Risk-based approach: If you are overwhelmed with the tests’ scope, focus on the most critical functionality sections, i.e. those leading to the highest risks in cases of errors.
- Regression: Computerized Systems Validation is not a one-time thing but has to be repeated each time you make changes to the system or software. Thus, automating the tests is recommended. There are tools with which you can semi automatically record and play back user interface tests as well.
Change history
- 2022-09-15: FDA Guidance Computer Software Assurance for Production and Quality System Software added
