A Meta-Data Manifesto: The Need for Global Health Meta-Data

Abstract

Administrative health data recorded for individual health episodes (such as births, deaths, physician visits, and hospital stays) are being widely used to study policy-relevant scientific questions about population health, health services, and quality of care. An increasing number of international health comparisons are undertaken with these data. An essential pre-requisite to such international comparative work is a detailed characterization of existing international health data resources, so that they can be more readily used for comparisons across counties. A major challenge to such international comparative work is the variability across countries in the extent, content, and validity of existing administrative data holdings. Recognizing this, we have undertaken an international proof of concept pilot compiling detailed data about data – i.e., a “meta-data catalogue” – for existing international administrative health data holdings. We describe the methodological process for collecting these meta-data, along with some general descriptive results for selected countries included in the pilot.

“We are drowning in information, while starving for wisdom.” E.O. Wilson

“Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information?” T.S. Eliot

The world is increasingly connected and the production, collection, dissemination of data have never before been so widespread, rapid and increasingly complex. Health also has become more complex. Epidemics of infectious diseases arise episodically throughout the world, non-communicable diseases are increasingly prevalent globally, childhood malnutrition and obesity occur simultaneously and many countries struggle with increasing costs of healthcare delivery (1). Growing attention to the domain of health has led to demands for better health data to inform evidence-based policy (2,3). However, there are large disparities among countries in health data collection processes, holdings and capacity.(1,4—6).The growing focus on accurate health data and implementation of electronic health databases worldwide has created new challenges. As the magnitude of health data collected expands exponentially, the task of organizing it in a meaningful way such that is can be easily found, analyzed and subsequently used becomes progressively complex. As such, there is a pressing need to know where these data are, who holds them, how to use them and for what they are being collected. In short, the world needs data about health data, or stated another way - we need global health meta-data.

Statistical capacity building has been identified as a core need by many countries. Work on an international level to improve global statistics is ongoing through consortia such as the Partnership in Statistics for development in the 21st Century (PARIS21), the Organization for Economic Cooperation and Development (OECD), the World Health Organization (WHO), and the World Bank (3,7,8). The development and improvement of electronic health information systems provides increasing opportunities to compile and evaluate health data. Despite a rapidly growing quantity of health data globally, difficulties remain including great disparity among countries on the quality, quantity and validity of health data reported (4,6,9). Among low and middle-income countries great challenges exist with respect to the collection of health data, including accurate mortality data (1,4,9—12). Often, health data are collected by a variety of sources for different reasons, with differing agendas in a piecemeal fashion (4,12—14). For example, international donor agencies, frequently anxious to quantify results and promote accountability for aid, often support and implement their own data collection methods independent of the countries they are working in (14,15).

Developed countries are not immune to difficulties in providing accurate and timely health information. Major differences in recording practices exist among private and public organizations. Additional challenges include: inconsistencies between different social sectors recording similar health data, the shortage of trained personnel with competencies in coding health data, and the management and administration of national registries (6,9,13). The global state of health data reporting was summarized by Mathers et al. in a study showing that of the 115 countries that report mortality data to the WHO only 23 were considered to report high quality data (4). Fortunately, great opportunities to improve the state of global health data exist given technological advances in communication and a concerted international effort from many stakeholders and institutions (3,16).

The History of Administrative Health Data and their Contemporary Use:

Wide varieties of sources exist through which health data are collected. These include: national census, national surveys, vital registration systems, disease surveillance systems, vaccination programs, health finance accounting, and administrative health data (14,17). The latter are produced as a byproduct of healthcare delivery, whereby codified data are recorded for individual health episodes such as births, deaths, physician visits, hospital stays, medication dispensation, and health insurance claims among other sources (18). The origins of administrative health data can be traced to the First Statistical Congress in Brussels in 1853 when a multinational agreement on the need for consistent coding for causes of mortality was reached (19,20). This original document evolved into the International Classification of Disease (ICD) in 1948 and has since been under the auspices of the United Nations’ World Health Organization (21). The International Classification of Diseases underwent its 10th Revision in 1992 and is referred to as the ICD-10 (for which there are also many country level modifications). An 11th revision (ICD-11) is currently underway, with release expected in 2018 (21,22).

Today, a variety of different administrative health data coding systems exist that code for cause of death, classification of diagnoses and procedures that are employed by different countries including ICD-8, ICD-9-CM, ICD-10 and SNOMED among others (17). These data coding systems are primarily used for administrative purposes (i.e. physician billing and resource tracking), but have found secondary uses in population health research, health service performance analysis, quality of care and disease surveillance. From a more global perspective, in 2016 the Millennium Development Goals campaign ended and the Sustainable Development Goals were created (23). These goals served to highlight the potential use and great need for accurate health data to quantify progress and inform resource allocation (15,23—26). Establishing harmonized systems for coded health data collection related to the Sustainable Development Goals could help standardize data collection processes internationally.

Administrative Health Data in High-Income Countries for Comparative Research and Surveillance:

High-income countries apply administrative data in comparative research and surveillance by utilizing pre-existing and ongoing data collection methods that allow for health system performance comparisons both within and across nations (18,27,28). Indeed, these data are used to assess health care quality and to inform policy decisions. For example, in the United States, the Agency for Healthcare Research and Quality (AHRQ) among other groups have developed patient safety indicators that are used as quality measures of hospitalized care within the United States. These indicators are additionally used in other countries such as Canada and Germany and comparatively between countries, thus allowing for quality assessments of different healthcare systems (27,29—31).

Many challenges exist in spite of an increased focus on the potential of administrative data collection and reporting to serve as valid and valuable tools in healthcare quality assessment. These include: large variations across countries on the scope and depth of data holdings, and variability in data quality and coding practices. These challenges are further exacerbated by the use of different classification systems, versions of the ICD (ICD-9 vs. ICD-10), or the use of country specific coding manuals, and also differences in coders’ training and where the responsibility for coding ultimately lies (4,27). For example, Lozano et al. in the Global Burden of Disease 2010 study show that there is considerable heterogeneity across countries in the availability and quality of mortality data (1). Furthermore, a major challenge to international comparisons - in addition to the variability across countries in the extent, content, and validity of existing administrative data holdings - is simply finding and accessing administrative health data from different countries (1,26). An essential pre-requisite to such international comparative work is a detailed characterization of existing international health data resources, such that they are more readily available for comparison across counties.

Low and Middle-Income Countries that Lack Administrative Health Data: the Information Paradox and the Millennium Development Goals:

In less developed countries, far greater challenges exist with respect to the availability of accurate health data. The World Health Organization Family of International Classifications network (WHO-FIC), responsible for the production and maintenance of international classifications, identified an “information paradox” referring to current health data disparities. The paradox is that countries with the highest disease burdens often also have the weakest health information systems and least accurate health data available (32). For low-income countries, the United Nations Millennium Declaration in the year 2000 was a remarkable achievement in diplomacy that created the Millennium Development Goals (MDGs) and received commitment by the world’s heads of state (33). The premise of the MDGs was to end extreme poverty, hunger and disease with time-bound targets by 2015 (33). This International commitment to the MDGs and health was exemplified by a study by Ravishanker et al. which estimated that development assistance for health grew from $5.6 billion in 1990 to $21.8 billion in 2007 (34).

A direct consequence of the MDGs and this dramatic increase in aid for health is intensified international pressure to strengthen information systems to monitor the 48 MDG target indicators, 18 of which are health-related. As the 2015 MDG target date passed, there has been widespread recognition that many beneficiary countries of international aid aimed at achieving the MDG targets are simply not equipped to monitor progress towards these goals (10,11,26,35,36). In 2005, a WHO study showed that of the 192 officially listed countries, 75 either report no useable health data or data that is far outdated (4). As a result, a push from the WHO and the international development community to strengthen health information systems is ongoing. This impetus led to the development of groups such as the Health Metrics Network (HMN) (37). This network is a global partnership that works to build stronger health information systems in developing countries (37). Increased awareness about the need for accurate health information internationally from groups such as the HMN and the WHO are certainly welcome. However, large gaps in even the most fundamental health data (e.g., numbers and causes of death) still persist in much of the world as evidenced by the Global Burden of Disease study that found that only 130 of 187 countries studied even had vital registration systems in place from which mortality data could be gathered (1). A fundamental first step to achieving better health globally, reaching the development goals and subsequent sustainable development (23) is accurate information from which aid can be directed and sound policy formed.

The Data Deluge and the Need for Meta-data:

Technological advances such as the development of health information systems and implementation of electronic health databases worldwide, as a byproduct, now produce and collect enormous amounts of administrative health data. As countries continually increase their data holdings, the detail and complexity of the administrative health data they collect also increase. Moreover, a new post-MDG global movement towards universal health coverage has been growing globally. The WHO estimates that approximately half of the world’s countries are engaged in health reforms aimed at extending, deepening and improving coverage for needed health services (38). This global expansion of health insurance programs has the potential to produce great increases in administrative health data.

Despite this increasing availability of coded health data, the formal comparison of health delivery systems across countries is often difficult given large differences in existing content, quality and breadth of the health data. Additionally, with the boon of expanding administrative health data comes the new challenge of consolidating, sorting and organizing them such that they can be utilised in meaningful ways and provide useful information to a wide variety of stakeholders.

The discipline of library science has historically undertaken the study of the collection, organization, preservation and dissemination of information resources. This creation of ‘meta-data’, or ‘data about data’, is focused on generating structured information that describes, explains, locates, or otherwise makes it easier to find, utilize, or manage an information resource (39—42). Meta-data schemes arose to describe Internet resources such as the Dublin Core Meta-data Initiative (DCMI) that is widely utilized by the library community, governments, and researchers among others (39,41). Professional groups such as the Institute of Electrical and Electronics Engineers have published guidelines around the need and application of meta-data broadly in multiple disciplines (42).

There are organizations that aim to apply meta-data standards to health services research. One impressive national initiative is embodied in METeOR, an Australian repository for meta-data (43). The focus in this case, is on meta-data in the purest sense: dissemination of national data standards within Australia in order to achieve consistent definitions, structure, and quality of datasets that employ these standards. Similar national attempts around standardization of data quality can be found in the Canadian Institute of Health Information (44), and the American Agency for Healthcare Research and Quality, which has published quality indicators that rely on administrative data sources and have the potential for broad application in comparative health research (45). While individually promising advances in national-level health meta-data, international comparative efforts among Australian, Canadian and American health systems first require intimate knowledge of each of these organizations, how to contact them, their data holdings and specific meta-data elements that describe each individual database.

Meanwhile, other open access initiatives aim to catalogue existing health research databases. The Society of General Internal Medicine created a ‘Dataset Compendium’ to enable researchers to navigate publicly available North American datasets (46). It offers rich descriptions of datasets including their surrounding initiatives, points of contact for arranging data access, and offers a textured description of strengths and weaknesses, insights from experienced users, as well as general tips to working with secondary data sources. It does not focus on disseminating meta-data standards or definitions, nor does it provide meta-data that facilitates comparisons across data sources. Finally, although this Compendium lists some administrative health data sources, they are almost entirely national in scope (i.e., pertaining to the United States) and include data from surveys, trials, and disease specific registries not coded into international classification systems.

At an international level, the OECD has created an iLibrary that allows users to query and generate comparative statistics among 35-member countries across many different domains, including health. Links are given to the custodial agencies for the data at the national level for the countries involved. The OECD work is commendable for its open access, and user engagement through a tool that allows users to query their varied datasets to display health information at a population-based level (i.e. not at the level of an individual health encounter). The nature of the underlying national level dataset is varied, although presumably there are some sources that are administrative data in origin; however, meta-data standards from contributing nations are also not the focus (47). The iLibrary provides health data summaries of member nations in tables that facilitate comparisons of important health themes such as healthcare quality indicators; however, it is unclear the source of the national data listed, nor important meta-data elements that describe each nation’s data holdings. Also, despite an impressive collection of health-related data, by nature of the OECD membership, iLibrary only includes data for 35 countries and importantly excludes notable countries such as China, India and Brazil whose combined populations make up a significant proportion of the entire global population.

The Global Health Data Exchange, created and hosted by the Institute for Health Metrics and Evaluation represents another comprehensive catalogue of health data with a global scope. This catalogue includes a great many data types and sources including: data from census initiatives, demographic surveillance surveys, disease registries, modeled data, reports and articles from the published scientific literature as well as administrative data. While this very impressive compendium of health data does contain some meta-data that describes its administrative data sources, it does not include detailed descriptions of the coding practices or guidelines used to create them, nor the specific data elements or structure contained within each database. The limited meta-data descriptions included are thus likely inadequate for users to pursue international comparison studies themselves.

To our knowledge, no resource currently exists that is focused on administrative health meta-data holdings, with a truly comprehensive international scope. This is despite the fact that these meta-data are often made available by administrative agencies, either publicly or internally. The need for such a resource was described by an international group of health researchers known as the International Methodology Consortium for Coded Health Information (IMECCHI) in 2006 (48). This consortium has expertise in international comparisons of administrative health data and participates in ongoing comparative and methodological research (27,48,49). IMECCHI members revealed through their work that differences in clinical coding practices and standards between countries create a major barrier for international comparison studies. For example, Jetté et al. highlighted that use of country-specific ICD-10 clinical modifications pose significant challenges to the international comparability of hospital morbidity data, whereby identical codes have differing definitions across different countries (21). Further, Drösler and colleagues showed that while it was technically feasible to compare patient safety indicators between OECD countries, differing national coding practices resulted in highly variable and inaccurate indicator rate estimates, thus limiting interpretability (27). The characterization of each country’s administrative health data holdings is a fundamental first step to understanding the current state of international health data and assessment of the health information needs of individual countries.

The Global Health Meta-data Catalogue: A Pilot Initiative to Characterize Global Administrative Health Data Holdings

Ultimately, the optimal use of existing data resources to positively affect the health of populations and health systems performance requires increased awareness and access of existing data, including data from administrative sources and their associated meta-data standards. Given the need to characterize global administrative data holdings, we have undertaken an attempt to address the issue. We created a pilot open access online compendium of existing international health data resources or “Global Health Meta-data Catalogue” (GHMC). This collection is coordinated through IMECCHI, and it aims to characterize holdings of coded health information, starting with OECD nations, and eventually extending to all officially listed countries in the world. To create the GHMC we included databases that were national in scope, and contain codified administrative accounts of individual episodes of care using an international classification system (e.g., ICD 9-CM, ICD-10, SNOMED). An open access, online version of the meta-data catalogue has been created, accessible through a web-based tool at https://imecchi.com/ghmc/. It is a relational database organized by country in separate layers, which reflect the organization of national level health agencies (layer 1), the administrative health databases they administer (layer 2) and supporting documentation pertaining to meta-data standards, data definitions, validity documentation etc. for those particular databases (layer 3) (See Figure 1 for a screenshot depiction of the meta-data catalogue and its layers.)

Figure 1: Screenshot of Global Health Meta-data Catalogue.

Given the lack of national-level coded health data in many low and middle-income nations (the Information Paradox), and our goal to facilitate between country comparisons, we used broad inclusion criteria. We included any agencies with national-level data holdings, any health databases held by eligible agencies and any meta-data elements that described the databases included when available. We excluded agencies and databases with regional, state, or provincial-level scope that were not nationally representative. We excluded any health data that were generated from some type of study sample such as various types of surveys, census, trial, or cross-sectional study data. This directory provides a valuable tool for researchers and organizations interested in accessing administrative health data from a variety of countries. To date, initial steps of concept development have been completed. International collaboration has been initiated with ongoing interactions between IMECCHI and the World Health Organization Family of International Classifications (WHO-FIC) network. The catalogue has been partially populated with data from OECD countries in addition to some middle income and low-income countries.

Initial Insights and Challenges of the Global Health Meta-data Catalogue

The process of data collection thus far has highlighted a great variability in available data and quality of data across countries. Differences in coding practices have been identified, not only in the use of ICD-9 and ICD-10 but individual country’s amendments to these international classification schemes (i.e. Australia’s ICD-10-AM). Furthermore, variability in coding practices with respect to who is most responsible for the coding and which diagnoses take priority upon discharge from hospital have also been noted as previously identified by others (27,30). Our modest efforts through this catalogue pilot work also highlight the information paradox identified by the WHO, where there is indeed a paucity of administrative data in low-income countries with high burden of disease relative to OECD countries (32). Within individual countries, we have learned that researchers or administrators who use national level data often have intimate knowledge of these data that are difficult to access from abroad.

The GHMC and the Pressing Need for Global Health Meta-data

The GHMC represents an applied research tool that will help a global community to address the growing need for global health meta-data. Despite the challenges mentioned, it stands as a first step in understanding the state of global administrative health data holdings. Its utility depends upon the uptake of a broad user-community of researchers, policy makers, administrators, and information system developers. This community is essential to provide peer review, ensuring the completeness, accuracy and validity of such a compendium’s holdings. Through broad dissemination and use, the GHMC has the potential to help to harmonize international health data and improve international comparisons and collaboration. Moreover, in the low and middle-income countries as more resources are directed toward the development of efficient health information systems, the GHMC can serve as a system to monitor such processes as more data become available. Given its open access platform, it can also serve as a “roadmap” for these countries as they develop their health information system capacity to suit their individual needs by assessing what exists internationally.

Currently the GHMC represents only a proof of concept; however, given the explosion of information technologies many exciting future possibilities exist. The popularity of social networking can be extended to a community of users to improve communication. Mobile access, remote and distributive updating in addition to mobile applications are all other future possibilities. In addition, iterative improvements to the GHMC beyond the pilot phase can include improved functions such as key word searches that will help users to locate specific databases more efficiently or tools that will allow meta-data comparisons across specified countries. Ultimately, the future success of this project or others like it depend on future funding and active community participation by those who most stand to gain from its development.

In a time where a deluge of data continuously grows, it is only through systematic characterization and understanding of these data that we stand to gain the knowledge and wisdom to improve the decisions made, and ultimately the health of the populations these data stem from. We hope to highlight some of the issues faced today with respect to the state of administrative health data globally. We also present our early attempt to address some of these issues through an internet-based research tool. In a digital era, the global need and expectation for accurate, timely health data is heightened. Advancements in technology, the increasing global commitment to improved statistics and the international resolve to achieve the millennium development goals provides opportunity to make meaningful gains in understanding and improving the state of administrative health data globally. Without taking stock of what we hold, we will continue to move forward blindly, distributing resources inefficiently, with ongoing great health disparities globally and without the ability to adapt to the global health challenges we face. But with clarity around data, we will gain clarity and enhanced strategic vision to advance health.