Evaluating laboratory diagnostic tests

Abstract

International collaboration to set standards and methods is urgently needed

Recent technological developments have created a new generation of laboratory diagnostics, which promise to provide better ways of detecting diseases and monitoring response to treatment. These tests create the possibility of earlier and more accurate diagnosis, and of shifting health care from hospitals to the community—making it more effective, efficient, and accessible. But two recent reports—one of which is published this week—highlight the relatively low importance given by clinicians and policy makers to evaluating laboratory diagnostic tests.^{1 2} In an accompanying analysis, Melzer and colleagues outline the problems caused by this, specifically relating to the evaluation of genetic tests, and propose ways of overcoming them.³

So how should we evaluate tests? First, we must be clear about the purpose of the test—whether it is meant to diagnose, monitor, guide prognosis or treatment, or predict risk. Then the context in which it is used needs to be specified—for example, the disorder or disease, its prevalence in a particular population, and the care pathway that the test forms part of. Evaluating a test outside that care pathway is of limited use.

We can then use the ACCE framework⁴: the Analytic validity (to what extent the test measures what it claims); its Clinical validity (its ability to detect or predict the presence or absence of disease—its sensitivity, specificity, positive and negative predictive value); its Clinical usefulness (does the test lead to better patient outcomes?); and any Ethical, social, or legal implications (and perhaps economic implications in cost conscious health services).

Clinical usefulness should be the most important factor when deciding whether or not to adopt a test. But this is the least likely domain to be evaluated—to produce such primary evidence needs complex and expensive studies, often randomised controlled trials, with high quality of reporting to allow systematic review.⁵ Some high profile tests are currently being fully evaluated, such as testing for human papillomavirus, which is being evaluated by the ARTISTIC study.⁶ Such rigorous evaluation is not possible or necessary for all tests, but only for those that might lead to major changes in a care pathway and possibly substantial gain for patients. But the results of simpler forms of evaluation—at the very least of analytical and clinical validity—should be readily available, perhaps in a database of tests approved for use within health services. At present, even these results are often difficult to find.

A major reason for poor evaluation is that the regulatory framework for diagnostic tests is weak, with no international standards and no agreement on what evidence is required or by whom. Diagnostic tests are currently “CE” marked, which usually means that the manufacturer certifies that the product meets basic European Union safety and health requirements. In the UK, the Medicines and Healthcare Products Regulatory Agency expects the manufacturer to produce evidence for any clinical claims made for a test, but this lacks transparency and at best is limited to analytical and clinical validity. The NHS Centre for Evidence-based Purchasing reviews whatever evidence is available of clinical and cost effectiveness to advise NHS commissioners. But neither body can demand evidence of clinical usefulness, so manufacturers have little incentive to undertake such studies. Because diagnostics manufacturing companies are often small and lack experience in large scale trials or evaluations, they may need help with expertise and funding to produce better evidence.

Both reports agree on the need for more formal and systematic processes of evaluation and oversight of laboratory diagnostic tests.^{1 2} They suggest the need for a body to take on this responsibility, but they are open on whether this body should be professional (for example, led by the Royal College of Pathologists), regulatory (like the Medicines and Healthcare Products Regulatory Agency), or advisory (like the National Institute for Health and Clinical Excellence). Such a body would prioritise which tests need which level of evaluation and would have links to research funders, like the National Institute for Health Research health technologies assessment programme, to commission studies of clinical usefulness in the most important areas. It would also scan the horizon for developments that could greatly change patient care pathways and improve outcomes. The benefit of the advisory body taking responsibility would be that the tests would clearly be tied into a care pathway.

Exploration of the human genome seems to offer huge potential for genetic testing, and Melzer and colleagues describe how the problems outlined above all apply equally in this area. They call for harmonisation of regulatory standards internationally and for more transparency regarding the clinical evidence base for new tests. A realisation of how limited this is would lead to public and professional demand for better evidence and more formal evaluations, including trials. The valuable but voluntary UK Genetic Testing Network (www.ukgtn.nhs.uk) has evaluated over 89 tests, of which 70% were considered acceptable.

Genetic tests create particular concerns, but they may provide the stimulus to develop a better framework for evaluating and regulating all laboratory diagnostic testing in the public and the private sector. International collaboration to set standards and methods is essential. These reports have emphasised the need for such developments and have opened the debate on ways ahead.