Medical software verification

software verificationSoftware development is a complex and challenging task. Part of the work we do at TGMI involves creating software tools for use in genetic medicine. It is essential that these tools are reliable and accurate, as the data they produce may be used to inform clinical decision making. How then, do we ensure that our software is fit for purpose? Today I will talk about one of the ways we do this: verification.


What is software?

Software is the set of instructions that tell a computer what to do, and which makes it possible for human users to interact with the computer. Modern computers consist of hardware and software (there’s also firmware, but let’s leave that for now). Hardware is the the physical machinery of the computer e.g. the keyboard, monitor, hard disk etc, and software is the instruction set that controls these physical pieces. Without software a computer would be useless. Examples of software include the Windows and Linux operating systems, word-processing tools such as Word, and the web browser you are using to read this post.


What is verification?

Verification is the process of demonstrating that the logic of our software is correct, and that it does not contain critical bugs; this is sometimes summarised as ‘does it do what we think it does?’ Verification can take various forms, but usually involves a set of automated software tests (variously referred to as unit-tests, acceptance-tests and regression-tests) in addition to frequent ad-hoc tests by users. The automated tests are run whenever a change is made to the software, and new tests are added when new functionality is added. This helps to build confidence in the software, and also helps the developers to rapidly locate and fix any bugs that are reported by users.


What is a bug?

A bug is an error or flaw in a computer program, which causes the program to behave in a way that was not intended by the programmer. The exact origin of the term is unclear, but its use in computer programming was popularised by Grace Hopper, after the discovery of a moth trapped in one of the relays of an early computer. The moth was logged, and can still be seen at the Smithsonian National Museum of American History in Washington, D.C.


Does my software have bugs?

Almost certainly. A good rule of thumb is that all software has bugs, and the more complex the software is, the more bugs it has. Part of the job of a professional software developer is to employ a variety of tools in order to identify and fix critical bugs and to minimise the creation of new bugs.


Verification is critical to bug identification

Verification is a key part of the process to identify and fix bugs. Without a proper process of verification you simply will not know if your software is working as expected until somebody tries to use it. For certain research purposes this may be acceptable, for clinical tools it is not.


Verification is not enough

Software development is an interdisciplinary process that requires regular, effective communication between users and developers.

Verification does not ensure that the software actually does the right thing (this is covered by a related process called validation, which I discuss in a future blog), only that it does what the programmer intended it to do. It is quite possible to write the wrong program, but for that program to be entirely logically correct. This may happen, for example, if the requirements of the users are misunderstood by the developer, who then writes a perfectly well-functioning program that does not do what the users wanted. Software development is an interdisciplinary process and requires regular and effective communication between users and developers.


What is the TGMI doing?

At TGMI we are creating a large verification test-suite with hundreds (and eventually thousands) of automated tests, covering all the critical functionality of our software. We run these tests frequently, many times a day, adding new tests whenever bugs are detected. We also maintain close links between developers and users, who ensure that all our software is tested regularly on clinical data and any problems are fed back quickly. Developing software in this way means that we build, slowly but surely, on well-tested and reliable foundations, producing tools that can be used with confidence in the clinic.