Exploiting routine data for international benchmarking of quality in mental health care

The paper by Kilbourne et al1 provides an extensive overview of the challenges in assessing quality of mental health care. Service users, informal carers, policy makers and the general public increasingly demand that mental health systems provide good “value for money”, and thus the need for validated, meaningful and purposeful data on quality of mental health care is growing. As outlined by the authors, many countries have taken actions to identify, define, collect and analyze such data.

In parallel with national activities, there is a growing interest for international benchmarking of mental health systems to inform national policies. The challenges in standardizing measurements become even larger when comparing mental health systems in different countries, due to differences in those systems and, in many cases, absence of common international definitions. Common indicator definitions and standardized data collection procedures are prerequisites for meaningful benchmarking between countries.

In spite of the above‐mentioned challenges, international benchmarking is an important moving force to foster development of mental health services in countries. For feasibility reasons, such benchmarking endeavours have to rely on routinely collected data, which tend to be dominated by hospital care data. So far, international comparisons of mental health systems have relied on existing administrative databases. Today, the increasing use of centralized repositories of electronic medical records presents a largely unchartered area of new possibilities for collecting and assessing quality of care data. It needs to be considered that data mining of electronic medical records will always require special attention to data security and confidentiality.

Electronic medical records can be extended to include patient‐reported outcomes and thus integrate the patients’ views. In the future, artificial intelligence may be applied to the analysis of data available from central repositories, contributing to the learning health care system. Kilbourne et al1 touch upon these possibilities, but a clear vision and detailed roadmap is needed to outline the research and policy actions that will enable us to use clinician‐ and patient‐derived information from repositories of electronic health records in order to assess and improve mental health care quality. A first step is to facilitate widespread adoption of patient‐reported outcome data collection into electronic health record systems2.

In the European Union, the need for comparable data and standardized definitions to enable comparisons of national mental health system performance has been identified, and processes to harmonize indicators and data collection have been initiated3, 4. Indeed, international comparisons based on disparate and non‐harmonized data may cause more confusion than clarification5. In an effort to foster comparability of data, most European countries have joined the Health Care Quality Indicators project, led by the Organisation for Economic Co‐operation and Development. The project's key areas include mental health care indicators but, due to variation in health care systems, so far only four quality indicators have been implemented and are reported annually; two of these relate to suicides during or after a hospitalization, and two are based on excess mortality in schizophrenia and bipolar disorder6.

Re‐hospitalization within 30 days after discharge from a psychiatric ward has been suggested as a further indicator for international comparisons of quality in mental health care, but uncertainties regarding the meanings of this indicator remain: is it an indicator of poor hospital care and premature discharge, or does it reflect insufficient community services and lack of continuity of care? Unplanned re‐hospitalizations are often disruptive for the patient and constitute a strain on limited health care resources. However, in some mental health systems, planned re‐hospitalizations are an integral part of individual treatment plans, making differences in readmission rates difficult to interpret.

The recent CEPHOS‐LINK (Comparative Effectiveness Research on Psychiatric Hospitalization by Record Linkage of Large Administrative Data Sets) project compared re‐hospitalization and its predictors in six different European countries, based on retrospective cohort studies with data from country‐specific large electronic health care registries. The study showed a clear interaction of case‐mix and country with readmission rates, even after harmonizing the national datasets7.

The European Commission‐funded BRIDGEHEALTH (BRidging Information and Data Generation for Evidence‐Based Health Policy and Research) project recently recommended to establish a European Research Infrastructure Consortium on health information for collection of comparable information on health system performance. In response to this recommendation, European Union member states are currently setting up a joint action for health information as a first step towards common governance, collection and analysis of health and health care information. Rendering large national health care databases and electronic health records repositories interoperable and thus comparable across countries is essential for comparing outcomes and processes in health service utilization in different countries.

The concluding recommendations by Kilbourne et al1 outline the way forward. However, like in all team sports, international rules are needed even for friendly tournaments. Involvement in the development of international standards for assessment of quality in care and adherence to internationally agreed standards will enable us to make meaningful comparisons of mental health systems. This is of special importance in times of electronic records, which open up new possibilities for assessment of quality of care.

It is not yet clear to which extent the above‐mentioned collaborative effort of the European Commission and the member states will cover harmonization and collection of relevant mental health systems data for comparison and benchmarking. From a European perspective, a joint mental health observatory is urgently needed to lead development and implementation of monitoring of mental health service provision. Such a centre would enable development and worldwide dissemination of indicators that reflect the European mental health care values of universality, access to good quality care, equity, and solidarity8.

Kristian Wahlbeck
Finnish Association for Mental Health, Helsinki, Finland