Measuring the quality of haemophilia care across different settings: a set of performance indicators derived from demographics data

Abstract

Background

Haemophilia is a rare disease for which quality of care varies around the world. We propose data-driven indicators as surrogate measures for the provision of haemophilia care across countries and over time.

Materials and methods

The guiding criteria for selection of possible indicators were ease of calculation and direct applicability to a wide range of countries with basic data collection capacities. General population epidemiological data and haemophilia A population data from the World Federation of Hemophilia (WFH) Annual Global Survey (AGS) for the years 2013 and 2010 in a sample of 10 countries were used for this pilot exercise.

Results

Three indicators were identified: (i) the percentage difference between the observed and the expected haemophilia A incidence, which would be close to null when all of the people with haemophilia A (PWHA) theoretically expected in a country would be known and reported to the AGS; (ii) the percentage of the total number of PWHA with severe disease; and (iii) the ratio of adults to children among PWHA standardized to the ratio of adults to children for males in the general population, which would be close to one if the survival of PWHA is equal to that of the general population. Country-specific values have been calculated for the 10 countries.

Conclusions

We have identified and evaluated three promising indicators of quality of care in haemophilia. Further evaluation on a wider set of data from the AGS will be needed to confirm their value and further explore their measurement properties.

Background

Haemophilia is a rare genetic disease that results from mutations in the genes that code for proteins necessary for normal blood clotting, called coagulation factors. There is a wide variation in the reported prevalence of haemophilia across countries [1–4]. Such variation might be partly rooted in genetic causes, but it is likely that variability in the capacity for prompt and accurate diagnosis and for the provision of life-saving care [5–7], and the economic capacity of countries or individual citizens to afford treatment [8,9] play an important role.

As with many other chronic, rare diseases, haemophilia care is heavily dependent on the availability of costly resources [10] and a structured multi-professional care model [11–16] that is not provided homogeneously everywhere. Many patients are not identified and are untreated [9]. A few years ago, an estimate of the underserved haemophilia population was proposed at 80% of the total estimated number of haemophilia patients [16]. This number, though proposed as the gap to fill, was never formally assessed. The number likely reflected the current estimates of poverty rates. The earliest articles in Haemophilia that discussed the 80% number were Jones [17] and Jones and Robillard [18]. Jones [17] mentions that 80% of people with haemophilia A have no access to factor VIII therapy. Jones and Robillard [18] mention ‘in the 1990s it was estimated that 80% of the worldwide haemophilia population received little or no care, but no data were provided supporting this estimate’.

The World Federation of Hemophilia (WFH), an international not-for-profit organization, was established in 1963, as a global network of national member organizations (NMO, currently 134) of patients recognized from the World Health Organization. Each year since 1998 the WFH distributes the AGS questionnaire [19] to all of its NMOs. NMO staff members complete the survey (often in collaboration with clinicians and health ministries) by providing aggregate demographic, clinical and treatment data on people with bleeding disorders in their country. The Data and Demographics Committee (DDC) of the WFH, including epidemiologists, haematologists, academic researchers, data collectors and patient leaders, provides oversight of the data collection and insight into the interpretation and presentation of the AGS data.

With the growing interest in value-based healthcare where expenditure is linked to patient-centred outcome measures, and a concurrent reduction in available resources, it is relevant to identify and propose indicators allowing comparison of the provision of care to haemophilia patients, both across countries and within each country over time, with the ultimate goal of increasing access to care. The purpose of this paper is to describe the logical process followed to identify three indicators designed to provide metrics based on data collected for the AGS that can be used to assess (i) the completeness of identification of people with haemophilia (PWH) in a country, (ii) the capacity of a country to correctly classify the severity of PWH, and (iii) a measure of survival of PWH in a country.

Materials and methods

Due to differences in the epidemiology of haemophilia A and B, we based our proposed measures on haemophilia A, which is the most common of the haemophilias. For the analyses presented in this paper, we used data from the AGS on the total number of persons with haemophilia A (PWHA), the number of PWHA with severe disease and the number of PWHA by age group for the years 2013 (last available) and 2010 (most remote year since the last major change in the data collection process).

To provide examples of the calculation and interpretation of the indices, 10 countries were selected based on the following criteria: (i) data were provided in the AGS consistently over time; (ii) data were relatively complete; and (iii) the countries represented different geographical and economic sectors of the world. For each country, demographic data on the general population including the total number of males and the number of males by age group were obtained from the United States Census Bureau International Database, which collates data from national sources. All proposed indicators are percentages or ratios and, therefore, they are dimensionless.

Indicators: detailed description of the metrics and calculations

Indicator 1 Percentage difference between the observed and expected haemophilia A incidence

Ideally, this indicator would be calculated by comparing the observed incidence in a country to the expected (true) incidence of haemophilia, which is constant across all countries due to the genetic nature of haemophilia. However, the incidence of haemophilia is impossible to measure, primarily due to: (i) the lag between birth and diagnosis of haemophilia; (ii) the extreme variability in age of diagnosis for PWH with milder disease; and (iii) a further lag in reporting of new cases to a disease registry. Therefore, we used the prevalence of haemophilia A in 5–18-year-olds (the earliest age band in the AGS allowing sufficient time for diagnosis) as an estimate of the observed incidence. This has been verified in population-based studies of haemophilia occurrence, from which the value of expected (true) haemophilia A incidence (15.3/100 000 males) was also derived [20]. The indicator is calculated for each country using the following equation:

$$\%\text{Difference}=\left[\frac{O-E}{E}\right]\times 100$$

where O is the observed haemophilia A incidence (number of patients with haemophilia A in the age group 5–18), and E is the expected haemophilia A incidence (as cases × 100 000 males).

If a country has identified every case expected, the percentage difference will be 0. If a country has identified less than the expected number of patients the indicator will have a negative sign. Some of the possible interpretations for a negative value for this indicator are: suboptimal diagnosis of haemophilia, or diagnostic service not offered to the entire population at risk; incomplete coverage of the haemophilia care network in the country; incomplete capture of cases into the source used to provide data for the AGS. An increase in the percentage over time (i.e. a progressive smaller negative number) would indicate an improvement in the capacity for complete diagnosis or provision of care. It is also possible that the indicator may be greater than zero, i.e. more patients than expected are observed. The most likely explanation for this case would be a high number of mild haemophilia patients identified.

Indicator 2. The percentage of the total number of PWHA with severe disease

This indicator is calculated by dividing the number of patients with severe haemophilia A by the number of total patients with haemophilia A, multiplied by 100, using the following equation:

$$\%\text{Severe}=\frac{S}{T}\times 100$$

where S is the number of PWHA with severe haemophilia and T is the total number of PWHA.

Usually, severe haemophilia patients are more easily identified. The indicator will be higher when fewer mild and moderate than severe patients are identified and registered (e.g. 50%). The values will be lower (e.g. 50% vs. 20%), when more mild and moderate than severe patients are identified and contribute to the total number of patients. This indicator is expected to be associated with the ‘maturity’ of the health care system in a country or region. Interpretation may also vary depending on a combination of indicators. For example, if the percentage difference of observed from expected PWHA is small, a high percentage of severe to total may indicate that the system has recently improved its capacity to reach most of the expected patients in the catchment area, but still needs to improve diagnosis of milder patients. If instead, the percentage difference of observed from expected PWHA is high, but the ratio of severe to non-severe is appropriate, the case is more likely that a good diagnostic service has been in place for long enough to properly identify patients in some centres/areas of the country, but it has not been extended to the entire country. A decrease in the percentage of PWHA with severe disease over time would indicate an improvement in the provision of care. A potential confounder for this indicator is represented by suboptimal laboratory performance and consequent misclassification of patients, which could affect the ratio in either direction.

Indicator 3. The ratio of adults to children among PWHA standardized to the ratio of adults to children for males in the general population

The ratio of adults to children in haemophilia A is normalized to the ratio of adults to children in the male population.

$$\text{Ratio of adults to children}=\left[\frac{{\text{PWHA}}_{19-44}/{\text{PWHA}}_{5-13}}{{\text{POP}}_{19-44}/{\text{POP}}_{5-13}}\right]$$

where PWHA_19–44 is the number of people with haemophilia A in the 19–44 age group, PWHA_5–13 is the number of people with haemophilia A in the 5–13 age group, POP_19–44 is the number of males in the general population in the 19–44 age group and POP_5–13 is the number of males in the general population in the 5–13 age group.

The general assumption behind this indicator is that, in the presence of appropriate provision of care, the life expectancy in haemophilia should be similar to that of the general population. Therefore, when this indicator has a lower than expected number of patients in older age bands (a ratio lower than 1), we could hypothesize that either fewer individuals are retained in the system as they grow older (e.g. insufficient capacity of the system to care for older PWHA), or a lower than expected survival applies to that country (e.g. excess mortality). An increase in the ratio over time would indicate an improvement in the provision of care. The ratio can be higher than 1, pointing to a survival advantage for PWHA, which might be difficult to interpret. Possible explanations would be a differential emigration of healthy adults from the country (e.g. healthy adult males being able to leave the country more than adult PWHA), mild PWHA being identified by education and outreach programmes, or simply different upper limits for categorizing PWHA.

Extreme caution must be used when interpreting these indicators, particularly until a larger number of countries are evaluated.

Results

The following 10 countries were selected for the present exercise: Australia, Canada, Georgia, Ireland, Republic of Korea, Poland, South Africa, Turkey, UK and USA.

The source data for the general population of the 10 countries by age band (Table 1) and corresponding data for the haemophilia population from the AGS (Table 2) have then been used to calculate the three indicators. The theoretical incidence of haemophilia A was estimated as reported in methods.

Table 1.

Population of males in 10 countries by age in 2013.

Country	All ages	0 to 4	5 to 13	14 to 18	19 to 44	45+
Australia	11 160 085	691 918	1 232 279	717 053	4 184 706	4 334 129
Canada	17 151 406	903 447	1 653 672	1 044 888	6 065 948	7 483 451
Georgia	2 362 703	173 115	255 386	160 768	904 053	869 381
Ireland	2 385 371	192 548	298 008	147 753	931 777	815 285
Korea, Republic of	24 480 021	1 084 741	2 310 954	1 737 985	9 932 244	9 414 097
Poland	18 707 328	1 027 867	1 693 638	1 062 452	7 637 186	7 286 185
South Africa	25 944 849	2 680 280	4 401 693	2 336 977	11 556 518	4 969 381
Turkey	38 966 939	4 081 139	6 254 049	3 312 873	16 091 984	9 226 894
UK	31 509 359	1 978 880	3 289 619	1 909 490	11 240 282	13 091 088
USA	155 741 368	10 149 449	18 935 655	10 724 525	55 759 010	60 172 729

General population data from: United States Census Bureau International Programmes, International Database (census.gov).

Table 2.

Number of males with haemophilia A in 10 countries by age in 2013.

Country	All ages	All severe	0 to 4	5 to 13	14 to 18	19 to 44	45+
Australia	2071	713	98	274	139	764	796
Canada	3006	869	139	416	256	1226	969
Georgia	232	64	32	40	25	111	24
Ireland	575	197	56	98	46	221	154
Korea, Republic of	1602	1116	62	212	164	886	278
Poland	2280	1216	34	169	113	1202	760
South Africa	1741	1045	69	300	172	785	370
Turkey	4369	1476	264	943	605	1952	605
UK	5651	1291	355	740	475	2158	1920
USA	12 957	6841	1166	2980	1684	4924	2203

Data from the AGS, publication year 2014 (data year 2013).

Percentage difference of observed from expected incidence of haemophilia A

Percentage difference of observed from expected incidence of haemophilia A in the 10 countries considered in this pilot exercise are shown in Table 3. Estimates ranged from 54.2% less in South Africa to 111.1% more than expected incidence of haemophilia A in Ireland. Georgia and USA had an observed incidence within 3% of the expected.

Table 3.

Percentage difference between the observed and the expected incidence of haemophilia A in 10 countries, 2013 (Indicator 1).

Country	Estimated incidence	Percentage difference from expected incidence
Australia	21.19	38.5
Canada	24.90	62.8
Georgia	15.62	2.1
Ireland	32.30	111.1
Korea, Republic of	9.29	−39.3
Poland	10.23	−33.1
South Africa	7.00	−54.2
Turkey	16.18	5.8
UK	23.37	52.7
USA	15.72	2.8

This indicator assumes an expected incidence of people with haemophilia A in the population of 15.3 per 100 000 males, estimated from Soucie et al. [20].

Percentage of PWHA with severe haemophilia

Percentage of PWHA with severe haemophilia is shown in Table 4. Results ranged from 23% in the UK to 70% in the Republic of Korea. The mean and median values for the 11 countries were 41% and 34% respectively.

Table 4.

Severe haemophilia A patients as percentage of total haemophilia A patients in 10 countries, 2013 (Indicator 2).

Country	Total haemophilia A	Severe haemophilia A	% severe haemophilia A
Australia	2071	713	34
Canada	3006	869	29
Georgia	232	64	28
Ireland	575	197	34
Korea, Republic of	1602	1116	70
Poland	2280	1216	53
South Africa	1741	1045	60
Turkey	4369	1476	34
UK	5651	1291	23
USA	12 957	6841	53

Standardized ratios of adult to children PWHA to the ratios of adult to children in the general population

Standardized ratios of adult to children PWHA to the ratios of adult to children in the general population are shown in Table 5. The ratios ranged from 0.56 in USA to 1.58 in Poland. The ratios average was 0.82; the Republic of Korea and South Africa had ratios very close to 1.0.

Table 5.

Ratio of adult to children haemophilia A cases standardized to the ratio of adult to children in the general population of 10 countries, 2013 (Indicator 3).

	Ratio (19–44)/(5–13)		Rate ratio PWHA/Gen pop
	PWHA	General population
Australia	2.788	3.396	0.82
Canada	2.947	3.668	0.80
Georgia	2.775	3.540	0.78
Ireland	2.255	3.127	0.72
Korea, Republic of	4.179	4.298	0.97
Poland	7.112	4.509	1.58
South Africa	2.617	2.625	1.00
Turkey	2.070	2.573	0.80
UK	2.916	3.417	0.85
USA	1.652	2.945	0.56

Table 6 reports the same indicators calculated for 2010, presented side by side with the values for 2013 taken from Tables 3–5, and explores the responsiveness of the indicators over time.

Table 6.

Comparison of three indicators by country, years 2010 and 2013.

Country	Indicator 1 (% Difference observed and expected)		Indicator 2 (% Severe)		Indicator 3 (Adult to child ratio)
	2010	2013	2010	2013	2010	2013
Australia	30.7	38.5	29	34	0.94	0.82
Canada	47.0	62.8	27	29	0.81	0.80
Georgia	−10.4	2.1	56	28	0.90	0.78
Ireland	103.1	111.1	39	34	0.67	0.72
Korea, Republic of	−37.2	−39.3	67	70	0.88	0.97
Poland	−27.7	−33.1	21	53	1.58	1.58
South Africa	−59.7	−54.2	60	60	1.04	1.00
Turkey	−14.5	5.8	18	34	0.73	0.80
UK	45.6	52.7	35	23	0.86	0.85
USA	−12.3	2.8	56	53	0.51	0.56

Discussion

We have identified three indicators describing characteristics of the haemophilia A population of a country which should reflect the impact of haemophilia care. We have pilot-tested the proposed indicators on epidemiologic data from a defined set of countries that provide complete and consistent data to the AGS, and we have proposed some basic guidance for the interpretation of a range of possible values of the indicators.

The broader scope of this pilot exercise is to propose objective measures reflective of the provision of haemophilia care. The proposed indicators can be calculated directly from the AGS data and allow comparisons over time within the same country and cross-sectional comparisons among different countries. These indicators may represent ‘objective metrics to assess the impact of advocacy on the provision of care’, a goal identified as a valuable objective by the Medical Advisory Board of WFH. Moreover, the DDC recognized the value of the indicators as metrics to be used to verify the internal consistency of the AGS data. Finally, these indicators may inform health care development and support health care planning and decision-making.

The strengths and limitations of this approach are primarily determined by the quality and completeness of the data available in each country to compile the AGS questionnaire. Each WFH Report on the Annual Global Survey includes a discussion and list of caveats regarding the published data (see, e.g. pages 1–2 of the 2014 report, [21]). All of those issues apply to any use of the AGS data such as in this exercise.

The strengths of the indicators are that they rely on relatively basic data about the haemophilia population in each country, which in many instances will be readily available. A major limitation is that incompleteness of the available data can lead to incorrect conclusions about the level of care provision in the country. Therefore, it will be important that any interpretation of the indicators be tempered by knowledge of the capacity of each country to collect complete data.

For the sake of simplicity, we have decided to focus on haemophilia A only. Whereas the same analysis is feasible for haemophilia A and B, either separately or pooled, the former would have doubled the complexity of the exercise, and the latter would have increased the variability of the indicators, due to the different relative proportion of mild, moderate and severe persons with haemophilia B as compared to haemophilia A.

Two future improvements to the percentage difference between observed and expected haemophilia A incidence, can be collecting and analysing age bands broken down by severity and using Genome-Wide Association Studies [22] or registry data from countries with characteristics similar to the ones being assessed to estimate the true incidence of haemophilia. The value we have chosen is much lower than the incidence of haemophilia A observed in the UK, Australia, Canada and Ireland (+52.7%, +38.5%, +62.8% and 111.1%; Table 3), meaning that many more people than our theoretical benchmark are de facto diagnosed in those countries, possibly explained by higher number of mild and/or carriers diagnoses. The USA, Turkey and Georgia appear close to the theoretical incidence, while the Republic of Korea, Poland and South Africa present a gap ranging from −33.1% to −54.2%. We propose that after careful observation for a few years of the incidence observed over time in a group of sentinel countries, WFH would reassess which is the proper benchmark incidence to consider.

The percentage of severe haemophilia patients as percentage of total patients was lowest in UK (23%), Australia (34%) and Canada (29%; Table 4). Georgia presents a similar value, but a lower value of Indicator 1 (close to 0). We can anticipate for Georgia an increase in Indicator 1 and a decrease in Indicator 2 by expansion of a mature health care system to a larger population. In Poland and South Africa, the percentage of severe PWHA is around 50%, while there is a large negative difference between observed and expected (Table 3). A possible interpretation is that not all of the haemophilia population is reached, and those that are identified are more likely to have severe disease. Enhancing the diagnostic capacity in Poland and South Africa might improve both Indicators 1 and 2 (more patients and/or more mild are diagnosed), or improve Indicator 1 and temporarily worsen Indicator 2 (a larger number of severe PWHA, but not mild ones, diagnosed from a broader population base). Different attitudes toward genetic counselling and prenatal decision-making might also impact this indicator.

The standardized ratio of adult to children PWHA is the most sensitive to random variation in the AGS data, but is expected to be more robust than a similar indicator previously proposed by Evatt and Robillard [16], for it is normalized to the normal population. The Republic of Korea and South Africa have values closest to the theoretical value of 1. There is a lower than expected representation of older ages in many countries (UK, Australia, Ireland, Georgia, Turkey, Canada and USA). This may be the consequence of the high rate of blood-borne infections and consequent transient higher mortality and lower life expectancy in recent years in countries in which concentrates were largely available during the HIV/HCV epidemics. The lowest ratios are observed in USA and Ireland. However, Indicator 1 is very different between the two countries (higher in Ireland), and so is Indicator 2 (higher in USA), suggesting that different explanations have to be sought (e.g. immigration of young PWHA in Ireland), and stressing the importance of considering simultaneously the three indicators.

The overall performance of the proposed indicators over time (Table 6) shows an overall trend for the number to become more positive, but also some significant variability for all the three indicators, which makes is difficult and very likely not yet opportune attempting to provide guidance for interpretation.

In essence, we are suggesting a change in perspective, from pure ‘advocacy’ (the 80% statement), to ascertainment of the percentage (or number) of patients with haemophilia that may receive ‘adequate care’, to be compared across countries and over time. We say ‘may’, because diagnosis and registration do not suffice, adequacy of care also requires availability of factor concentrates. However, one can foresee that the indicators we are proposing, coupled with metrics derived from the units of factor concentrates used in a country, would provide a more efficient indicator than the traditional unit per capita measure. The task accomplished and discussed with the current paper is equivalent to creating a scaffold or reasonably robust numbers that can reflect quality of care, without going into discussing ‘adequacy of treatment’ in terms of units per capita or per patient.

Indeed, an interesting set of criticisms was raised in preliminary discussion and presentation of this paper to relevant stakeholders. Essentially, it is suggested that the indicators are sensitive to well-known phenomena like overestimation of proportion of PWHA as consequence of selective immigration of PWHA; underestimation of patients identified (diagnosed) and (well) treated in the developing world; and (systematic) underestimation of the number of patients affected by mild haemophilia in the developing world. As a consequence, the number of IU/capita estimated in the presence of the suggested over- or under-estimations will be as unreliable as the proposed indicators. Although one cannot deny the value units per capita has had on improving the care provided to haemophilia patients. Indeed, we are well aware that the proposed indicators (like IU/capita) are as good as the data they are calculated on. However, the AGS data are the best data that we have on a broad set of countries, and until they are benchmarked against better (and hopefully individual patient level) data from the same countries/areas, AGS data remain as the mainstay for planning and assessing haemophilia care and advocacy efforts. Any discrepancy (in plus or minus from the average value of each of the indicator) can be difficult to explain, but is notwithstanding pointing to some specific phenomenon worth investigation.

In this paper, we have provided a detailed explanation of the rationale for choosing the indicators, the formulas used to calculate them and examples of the indicators as calculated for a sample set of countries. It is not our aim to make any specific judgement on the specific countries involved or on the absolute appropriateness of the reference value proposed for the expected number of people with haemophilia [18]. We offer this as a proof-of-concept for one possible approach to a standardized assessment of some aspects of the provision of care to PWHA. The indicators here proposed need to be calculated on the AGS data for a few years, and interpreted until shown to reproducibly and meaningfully measure the quality of care for haemophilia. This would ideally lead to the ultimate goal to produce a standardized index or a set of metrics to complement, substantiate and ultimately replace the historical 80% definition.

In summary, we are confident that the proposed indicators were found to have some desirable properties, like discrimination (capacity to differentiate different levels in different settings) and responsiveness (capacity to measure changes over time) globally or within a set of countries. Face value relative to provision of care and widespread availability of the data needed for their calculation would be welcome additional characteristics. Finally, caution must be used when interpreting these indicators, particularly until a larger number of countries are evaluated.

Appendix

Members of the Data and Demographics Committee: Paula Bolton-Maggs (Serious Hazards of Transfusion, Manchester, UK); Magdy El-Ekiaby (Shabrawishi Hospital Blood Bank, Cairo, Egypt); Michael Makris (Department of Cardiovascular Science, The University of Sheffield, Sheffield, South Yorkshire, UK); Declan Noone (Irish Haemophilia Society, Dublin, Ireland); Alok Srivastava (Department of Haematology, Christian Medical College Vellore, Vellore, Tamil Nadu, India); Suely M. Rezende (Faculty of Medicine, Universidade Federal de Minas Gerais, Belo Minas Gerais, Brazil); Marijke van den Berg (Department of Laboratory Science, University Medical Center, Utrecht, The Netherlands); Jerzy Windyga (Institute of Haematology and Transfusion Medicine, Warsaw, Poland).