CriTech Research, Logo Contact us for a Free Project Analysis - Email Button
 734-668-0005

We are a Veteran-Owned Firm.

Request IV&V

NSF-ISR


Software Verification & Validation

CriTech Research helps you ensure that all your software is implemented correctly and completely. Headquartered in Saline, Michigan, we offer software verification and validation for companies in the medical device industry.


Definitions

Validation (FDA)
Software validation is "establishing by objective evidence that all software requirements have been implemented correctly and completely and are traceable to system requirements" [Ref: NIST 500-234]. FDA considers software validation to be "confirmation by examination and provision of objective evidence that software specifications conform to user needs and intended uses, and that the particular requirements implemented through software can be consistently fulfilled." [Ref: FDA General Principles of Software Validation, Final Guidance, January 2002]. Software validation is essentially a design verification function as defined in FDA's Quality System Regulation (21 CFR 820.3 and 820.30) and includes all of the verification and testing activities conducted throughout the software life cycle. Design validation encompasses software validation, but goes further to check for proper operation of the software in its intended use environment.

Verification (FDA)
Verification is defined in FDA's Quality System Regulation (21 CFR 820.3) as "confirmation by examination and provision of objective evidence that specified requirements have been fulfilled." In FDA's General Principles of Software Validation, software verification provides objective evidence that the design outputs of a particular phase of the software development life cycle meet all of the specified requirements for that phase. Software verification looks for consistency, completeness, and correctness of the software and its supporting documentation as it is being developed, and provides support for a subsequent conclusion that the software is validated. Software testing is one of many verification activities intended to confirm that software development output meets its input requirements. Other verification activities include various static and dynamic analyses, code and document inspections, walkthroughs, and other techniques. In a software development environment, software verification is confirmation that the output of a particular phase of development meets all of the input requirements for that phase.

Validation (IEEE)
"Confirmation by examination and provisions of objective evidence that the particular requirements for a specific intended use are fulfilled."

Verification (IEEE)
"Confirmation by examination and provisions of objective evidence that specified requirements have been fulfilled."


 
Using the Above Definitions in Software Development

Validation, in its simplest terms, is the demonstration that the software implements each of the software requirements correctly and completely. In other words, it ensures that "the right product was built." Verification is the activity that ensures the work products of a given phase fully implement the inputs to that phase, or "the product was built right."

Levels of Verification

There are three levels of software verification:

Software Unit Testing — Testing conducted to verify the implementation of the design for one software element (e.g. unit, module, method, class) or a collection of software elements.
Software Integration Testing — An orderly progression of testing in which various meaningful subsets of software and/or hardware are integrated and tested separately from the rest of the system. This testing proceeds until the entire software system has been integrated.
Software System Testing — The process of testing a fully integrated software system to verify that the software system meets its specified requirements.

Ultrasound, Incubator

Types of Verification

There are four types of verification that can be applied to the various levels outlined above:

Inspection — Typical techniques include desk checking, walkthroughs, software reviews, technical reviews, and formal inspections (e.g., Fagan approach).
Testing — Also known as "white box" or "logic driven" testing. Given input values are traced through the test item to assure that they generate the expected output values as well as expected intermediate values along the way. Typical techniques include statement coverage, condition coverage, and decision coverage.
Analysis — Mathematical verification of the test item, which can include estimation of execution times and estimation of system resources.
Demonstration — Also known as "black box" or "input/output driven" testing. Given input values are entered and the resulting output values are compared against the expected output values. Typical techniques include error guessing, boundary-value analysis, and equivalence partitioning.


Explanation

The four types of verification can be used at any of the levels, although some work better than others for a given level of verification. As an example, the most effective way to find anomalies at the component level is inspection. On the other hand, inspection is not applicable at the system level (you don't look at the details of code when performing system level testing). A logical approach to testing is to utilize techniques and methods that are most effective at a given level.

Component level verification can easily get very expensive. Companies need to avoid making statements like "all paths and branches will be executed during component testing." These statements make for a very expensive test program, as all code developed is required to have one of the most labor-intensive type of testing performed on it. To minimize the costs of component verification, the V&V group develops rules for determining the type of verification method(s) needed for each of the software functions. As an example, very low complexity software function, which is not on the safety critical list, may only need informal inspection (walkthrough) performed. Other complicated functions typically require white box testing since the functions become difficult to determine how they work. We recommend performing inspections before doing the white box testing for a given module as it is less expensive to find the errors earlier in the development.

The FDA requires validation to prove your system meets its requirements and is useful in the real world. The resulting V&V effort has become a significant part of the software development effort for a medical device. One of the key pieces to demonstrate that the system is implemented completely is a Requirements Traceability Matrix (RTM), which documents each of the requirements traced to design items, code, unit, integration, and software system test. The RTM is an easy and effective way for documenting your implementation in a manner to which the FDA expects the medical device manufacturer to be able to answer:

• What are the requirements? • Where are they implemented? • How have you tested them?

We specialize in software verification and validation for medical devices.