WHO assessment of health systems performance

Sir

In a Viewpoint (May 24, p 1817),¹ one of the six principal authors of The World Health Report (WHR) 2000 launched an unprecedented attack on WHO. The publication of this piece almost 3 years after the WHR 2000 is surprising, since none of the issues raised are new, nor do they contribute to the ongoing scientific debate about how best to measure and improve the performance of health systems. Philip Musgrove's main addition is a series of subjective speculations, unsubstantiated references, and personal communications, which have not been validated or authorised. Examples include statements such as “Several WHO Representatives and Liaison Officers were taken by surprise. As some of them explained to me” or the amusing claim that “The imputed numbers are indicated by italicised numbers in the Annex tables, but the italic typeface is so close to vertically aligned that several people…could not see the difference” (italics added).

Since the departure of Musgrove from WHO, constructive scientific debate has evolved with the active contributions of many specialists. WHO has encouraged critical input on the concepts, methods, and data involved in the WHR 2000 through a series of technical and regional consultations, involving more than 170 scientists and policy-makers from more than 69 countries.² The organisation has engaged in an exhaustive scientific peer review, including the establishment of the Scientific Peer Review Group (SPRG), chaired by Professor Sudhir Anand of Oxford University and consisting of 13 external specialists, representing all geographic regions of WHO. The process of peer review was overseen by an external advisory group, which reported that the review process had been comprehensive, objective, transparent, and informative. The full report of the SPRG was presented in May, 2002, and WHO's response to its recommendations was accepted by WHO's Executive Board in January, 2003.³ The SPRG concluded that “the objectives of the health systems performance assessment (HSPA) initiative are valid, and that the provision of comparative data on health-system characteristics is a vital component of securing health-system improvements. […] WHR 2000 made an important breakthrough in seeking to provide an integrated quantitative assessment of health systems performance, and bringing the topic of health-system performance to the attention of policy makers worldwide.”⁴ In the interests of transparency, WHO has maintained a complete list of the published and unpublished debates and criticisms, as well as the full report of the SPRG, on its web site at http://www.who.int/health-systems-performance.

One of the questions widely debated related to the ethical and scientific basis of Musgrove's main thesis, that only primary data should be used and that all primary data should be used exactly as reported by countries. The recent example of the severe acute respiratory syndrome (SARS) epidemic, and experience in the early days of the HIV/AIDS epidemic, show clearly why WHO has an obligation to provide the best available evidence in a timely manner to Member States and the scientific community. By best available evidence, we mean a careful assessment of all sources of information, including judgments about the validity, reliability, and comparability of the information; this information in some cases includes model-derived figures. WHO also has a responsibility as the lead global public-health agency to develop the measurement methods and national capacity to use them in a continuous and evolving process, and to promote scientific integrity and high standards, but at the same time to emphasise areas that need more attention and intensified data collection efforts. That these factors are applied is equally important in the area of health-system performance, which affects the health of all population groups, as in the instance of a disease outbreak such as SARS.

The experience over the past 3 years provides convincing evidence that the original intent of the WHR 2000 to stimulate governments, civil society, and the research community to focus more attention on health systems has been achieved. It has also made a contribution to foster a global culture of accountability for outcomes. Stewards of health systems need to be accountable not only for budgets and processes but ultimately for the benefits delivered to populations. The conceptual framework for assessment of health systems has been strengthened enormously by the consultation process, especially leading to the new focus on coverage of effective interventions. Methods, including the development of the World Health Survey instrument, have been improved through widespread input from governments and academia. The empirical basis for assessment of inputs, processes, outputs, and outcomes of health systems has tripled or quadrupled through concerted action of national governments, including participation by 61 countries in the WHO Multi-Country Survey Study 2000–01 and 73 countries in the World Health Survey 2002. Finally, and most importantly, in a range of countries, such as Mexico, Iran, and China, results of the WHR 2000 have generated national policy debates and led to specific health-system reforms.