An analysis of the degree of concordance among international guidelines regarding alpha-1 antitrypsin deficiency

Abstract

Background

Practice guidelines (PGs) attempt to standardize practice to optimize care. For uncommon lung diseases like alpha-1 antitrypsin deficiency (AATD), a paucity of definitive studies and geographic variation in prevalence may hamper guideline generation. The current study assembled and assesses the degree of concordance among available PGs regarding AATD.

Methods

To assess concordance, 15 eligible guidelines focused on AATD were evaluated regarding recommendations surrounding 24 key clinical issues. A Delphi process achieved consensus on ratings for each statement among 3 reviewers. Agreement was quantified as the proportion of guideline comparisons with a matching rating.

Results

The overall level of agreement was 47% (1190/2520 comparisons). The overall “affirmative agreement percentage” (ie, when guidelines agreed in endorsing a practice), was 42% (501/1190 comparisons). The agreement for individual clinical statements ranged from 26% to 75%. A broad consensus was seen in the recommendation to test all patients with a history of fixed obstruction on pulmonary function testing (either from asthma or COPD). Given that AATD is an under-recognized disease and that diagnosis often occurs at a late stage, the authors are encouraged by this consensus. Where overall the guidelines were less explicit was when to refer to a specialist or AATD center. Deciding on a treatment strategy requires a thorough understanding of the alpha 1 serum level, genotype, pulmonary function testing, and imaging, and therefore the authors feel that all patients would benefit from a specialty referral if the diagnosis of AATD is being considered.

Conclusion

Available guidelines regarding AATD frequently disagreed in management recommendations. Possible explanations for discordance include differences in regional prevalence, availability of augmentation therapy, and insurance environments. Attempts to harmonize the various guidelines by empaneling a broadly representative international group of disease experts should be considered for AATD. Similar comparisons among guidelines for other diseases are recommended.

Introduction

Practice guidelines (PGs) represent an important strategy to harmonize and standardize clinical practice to optimize care. In this context, PGs have proliferated globally, often developed by disease experts and sometimes affected patients who are empaneled by professional societies.

In the specific case of alpha-1 antitrypsin deficiency (AATD), an uncommon autosomal codominant condition that predisposes to emphysema and cirrhosis, the first PG was issued in 1989 by the American Thoracic Society (ATS) with a subsequent 2003 PG co-sponsored by the European Respiratory Society (ERS), the American College of Chest Physicians, and the American Association for Respiratory Care.^1,2 The Canadian Thoracic Society (CTS) published its first iteration of an AATD guideline in 2001, later updated in 2012.^3,4 Since these earliest guidelines, multiple others have been published by a variety of international medical societies (Table 1).^5–14 More recently, the Alpha-1 Foundation empaneled a group of experts to prepare an updated PG in 2016.¹⁵ Notably, because AATD is uncommon and because clinicians, including pulmonologists, characteristically see few such patients in a career, reliance on published guidelines is commonplace.

Table 1.

Alpha-1 antitrypsin deficiency (AATD) guidelines analyzed in this study

Year	Guideline title	Reference
1989	Guidelines for the approach to the patient with severe hereditary alpha-1-antitrypsin deficiency - American Thoracic Society	1
1997	Alpha-1 antitrypsin deficiency: memorandum from a WHO meeting	5
2001	Alpha-1-antitrypsin deficiency: a position statement of the Canadian Thoracic Society*	3
2003	American Thoracic Society/European Respiratory Society statement: standards for the diagnosis and management of individuals with alpha-1 antitrypsin deficiency	2
2006	Guidelines for the Diagnosis and Management of Alpha-1 Antitrypsin Deficiency Recommendations of the Spanish Society of Pulmonology and Thoracic Surgery (SEPAR)*	6
2006	α1-Antitrypsin Deficiency: Situation in Spain and Development of a Screening Program*	7
2009	Belgian Guidelines for Diagnosis and Management of Patients with α1-Antitrypsin Deficiency	8
2012	Alpha-1 in the European Union Expert Recommendations	9
2012	Alpha-1 antitrypsin deficiency targeted testing and augmentation therapy: A Canadian Thoracic Society clinical practice guideline*	4
2014	Guidelines on Diagnosis and Treatment of Alpha-1 Antitrypsin Deficiency Argentina Association of Respiratory Medicine*	10
2015	Activity of the Alpha-1 Antitrypsin Deficiency Registry in Belgium	11
2015	Indications for Active Case Searches and Intravenous Alpha-1 Antitrypsin Treatment for Patients With Alpha-1 Antitrypsin Deficiency Chronic Pulmonary Obstructive Disease: An Update (SEPAR)*	12
2016	The Diagnosis and Management of Alpha-1 Antitrypsin Deficiency in the Adult. Journal of the COPD Foundation Clinical Practice Guidelines	15
2016	Standards for diagnosis and care of patients with inherited alpha-1 antitrypsin deficiency. Recommendations of the Polish Respiratory Society, Polish Society of Pediatric Pulmonology and Polish Society of Pediatric Gastroenterology	13
2017	European Respiratory Society statement: diagnosis and treatment of pulmonary disease in α1-antitrypsin deficiency	14

Note: *Denotes that the guideline is published in two languages with an English version available.

Naturally, the proliferation of guidelines regarding the same clinical condition begs the question: do all guidelines agree? In this context, the current study was undertaken to assess the degree of agreement among the available guidelines regarding the management of individuals with AATD.

Methods

To identify candidate guidelines, Medline and Embase were searched from 1974 to January 24, 2018, using the search terms “alpha-1 antitrypsin deficiency,” “COPD,” and “guidelines.” References in sourced documents were reviewed. Eligible guidelines were published in English and were issued by official respiratory organizations/medical societies and/or by national organizations. Guidelines issued by insurance organizations were excluded. Because initial review of more generic COPD guidelines indicated that few offered specific recommendations regarding AATD, such guidelines were not included. Furthermore, because the earliest report from the Canadian Thoracic Society (in 1992) was confined to a statement regarding augmentation therapy and two subsequent, more comprehensive guidelines were issued by the Canadian Thoracic Society (in 2001 and 2012), only the latter two guidelines were included.^3,4,16

To analyze the degree of concordance among the 15 eligible guidelines (Table 1), 24 statements regarding key management issues were formulated by two pulmonologists (JKS, AA) with deep experience with AATD (Table 2). The statements were developed after we compiled every recommendation by an AATD guideline and divided each recommendation into categories. Using the compilation of guideline questions as a model, we summarized these overall themes by the development of our summary statements.

Table 2.

Individual clinical statements and adjudicated guideline ratings

		Number of guidelines receiving each rating
	Clinical statement	Y	YC	Eq/NC	N
When to test
S1	Only patients with suggestive features of AAT deficiency should be tested (eg prominent basilar hyperlucency, <20 pack years smoking with emphysema, emphysema at a young age [eg, <40 years old])	2	0	3	10
S2	Patients with a diagnosis of COPD and/or fixed airflow obstruction (FEV1/FVC ratio <0.70) on PFTs should be tested.	11	0	2	2
S3	Patients with airflow obstruction that is fully reversible with bronchodilators should be tested (ie pure “asthma”).	2	1	3	9
S4	Patients with unexplained bronchiectasis should be tested.	5	3	6	1
S5	Patients with unexplained liver disease should be tested.	7	1	7	0
S6	Patients with panniculitis should be tested.	7	0	8	0
S7	Patients with c-ANCA vasculitis should be tested.	5	2	8	0
S8	All first-degree relatives (children, siblings, parents) of severely deficient homozygotes should be tested.	8	2	5	0
S9	All non-first-degree relatives, including partners, of homozygotes should be tested.	3	3	9	0
S10	Neonatal screening should be undertaken (ie all newborns tested).	0	3	10	2
How to test
S11	Initial testing should include a serum AAT level.	13	0	2	0
S12	Testing should always include both a level and genotype/phenotype. (ie, a genotype/phenotype should be performed irrespective of the AAT level)	2	2	2	9
How to Manage
S13	I should get a CT chest at baseline.	3	1	11	0
S14	I should get serial PFTs on my patients.	7	0	8	0
S15	I should get a liver ultrasound on my patients at baseline.	3	1	11	0
S16	I should administer hepatitis A and B vaccinations.	2	5	6	2
S17	I should get serial LFT testing on my patients.	3	3	9	0
S18	I should refer my patient to a pulmonologist and/or tertiary center.	2	1	12	0
S19	My patient should be encouraged to participate in an AAT registry.	10	0	5	0
S20	I should encourage smoking cessation.	12	0	3	0
When to treat/augment therapy
S21	I should only treat when the serum AAT level is below a protective threshold value (ie, <11 uM) and/or when there is a severe deficient genotype/phenotype.	12	0	3	0
S22	I should treat patients only when some degree of airflow obstruction is present (eg, FEV1 30–60%).	7	4	4	0
S23	I should treat patients with panniculitis as first-line therapy.	2	0	13	0
How to treat/augment therapy
S24	I should treat only with a dose of 60 mg/kg once weekly.	5	1	7	2

Abbreviations: Y, yes; YC, yes, conditional; Eq/NC, equivocal/no comment; N, no; AAT, alpha-1 antitrypsin; PFT = pulmonary function testing. LFT = liver function test.

To allow specific ratings for each guideline on each of the 24 statements, each statement endorsed a specific clinical action in managing a patient with AATD (eg, “get CT scan at baseline”). The 24 statements were bundled into 5 categories: When to initiate a diagnostic workup (“When to test”), how to test for AATD (“How to test”), how to manage individuals with AATD apart from augmentation therapy (“How to manage”), when to initiate augmentation therapy (“When to treat”), and how to treat with augmentation therapy (“How to treat”).

Two independent reviewers (AA and UM) read the 15 guidelines and initially coded each guideline on each of the 24 statements, assigning one of four ratings to summarize the guideline’s treatment of that statement (Table 3). The four possible ratings included: Yes (ie, the practice was endorsed); Yes, conditional (the practice was endorsed subject to qualifying conditions); Equivocal/no comment (the guideline was ambiguous or no comment was made); No (the practice was not endorsed).

Table 3.

Four rating categories applied to each of the 24 clinical statements for each guideline

Rating	Definition/meaning	Example
Yes (Y)	There is a clear recommendation for the action described in the clinical statement.	For clinical statement Question 22, augmentation therapy should be recommended for individuals with established fixed airflow obstruction, the 2015 Spanish guidelines “Indications for Active Case Searches and IV AAT Treatment for Patients with AAT COPD: An Update (Spain guidelines) 2015” clearly endorses intravenous augmentation therapy.
Yes, conditional (YC)	The recommendation for the action requires an additional decision or that an additional criterion be satisfied.	The 2003 American Thoracic Society/European Respiratory Society guideline states that testing adults with bronchiectasis for AATD should be undertaken only when an alternative cause for bronchiectasis has not been identified. The 2009 Belgian Thoracic Society guideline endorses “considering a baseline chest CT.” The European Union recommends population screening for AATD only when three conditions are met, ie, the prevalence of AATD in the population is high (≥1/1500 or more), smoking is prevalent, and adequate counselling services are available (7).
Equivocal/No Comment (Eq/NC)	The guideline does not mention or address the topic of the clinical statement or if the issue in the statement is addressed in the guideline, there is no definitive position stated.	Many guidelines do not make mention of whether patients with necrotizing panniculitis should be tested, and several guidelines do not make a specific recommendation to encourage smoking cessation. As another example, the 2006 Spanish guideline’s statement regarding treatment of panniculitis with augmentation therapy reads “The possible benefits … in the management of other less common manifestations of AAT deficiency, such as panniculitis, are not documented.” (8)
No (N)	The guideline specifically advises against the clinical practice.	Examples include Grade C or D recommendations in the American Thoracic Society/European Respiratory Society guidelines. Also, the Canadian Thoracic Society recommends against testing for AATD in individuals with unexplained bronchiectasis.

Abbreviations: AAT, alpha-1 antitrypsin; AATD, alpha-1 antitrypsin deficiency.

After independent ratings by the two reviewers for each statement on each guideline, all ratings were compared. In the case of discordant ratings, a third reviewer (JKS) helped to adjudicate and consensus was achieved for all ratings. Thus, a complete set of adjudicated ratings of the 24 statements was analyzed.

Notably, several guidelines cross-referenced each other. For example, some guidelines mimicked statements in the ATS, ERS, or CTS guideline. In tabulating agreement among PGs, the statements in each individual text were attributed to that specific society guideline even when derived or quoted from another guideline; for example, the recommendations of the Belgian Thoracic Society were ascribed to that guideline despite mirroring the ATS guideline.^2,8

Some texts were noted to have “internal discordance,” which was defined as conflicting recommendations within the individual guideline, usually found between the text and a table/graph (Table 4). In this situation, recommendations from the text were favored.

Table 4.

Parameters for quantifying guideline agreement

Terminology	Definition and example
Internal discordance	Definition: Conflicting recommendations within an individual guideline, usually found between the text and a table/graph. In this situation, we favored the recommendations from the text.
Example: The European Respiratory Society guideline stated in the text that, “The role of CT in the follow-up of patients in routine clinical practice requires further validation.” However, one figure in this guideline listed CT as part of an algorithm to treat patients with AATD. The recommendation from this guideline was ultimately rated as Eq/NC in stating whether a baseline CT chest should be ordered given the fact that the text does not explicitly endorse the practice.
Agreement percentage	Definition: For each clinical statement, 3 reviewers reviewed a guideline independently and then together reaching an adjudicated rating (Y, YC, Eq/NC, N). The proportion of guideline comparisons matching on the rating for the same clinical statement.
Example: The statement that clinicians should treat only with a dose of 60 mg/kg once weekly (S24: How Treat 1) resulted in the following ratings: Y =5, YC =1, Eq/NC =7, N=2 contributing 10, 0, 21, and 1 matching guideline comparisons, respectively, for a total of 32. Thus, the agreement percentage for this statement is 32/105 (30.5%) (Figure S1).
Affirmative agreement percentage	Definition: For each clinical statement, the proportion of guideline comparisons matching on a rating endorsing an action (with or without an added condition, Y + YC).
Example: The statement that clinicians should treat only with a dose of 60 mg/kg once weekly (S24: How Treat 1) resulted in the following ratings: Y=5, YC=1, Eq/NC =7, N=2 contributing 10, 0, 21, and 1 matching guideline comparisons, respectively, for a total of 10 Y + YC. Thus, the affirmative agreement percentage for this statement is 10/105 (9.5%) (Figures S1–S3).
Negative agreement percentage	Definition: For each clinical statement, the proportion of guideline comparisons matching on a rating disapproving an action (N).
Example: The statement that clinicians should treat only with a dose of 60 mg/kg once weekly (S24: How Treat 1) resulted in the following ratings: Y=5, YC =1, Eq/NC =7, N=2 contributing 10, 0, 21, and 1 matching guideline comparisons, respectively, for a total of 1 N. Thus, the negative agreement percentage for this statement is 1/105 (1%) (Figure S3).

Abbreviations: Y, yes; YC, yes, conditional; Eq/NC, equivocal/no comment; N, no; AATD, alpha-1 antitrypsin deficiency.

Whenever diametric discordance occurred (ie, one guideline rating was “No” and another guideline was rated “Yes”) on a statement, the three reviewers convened again to confirm consensus. For example, in contrast to other guidelines, the CTS guideline recommends targeted testing for AATD only when suggestive features are present (eg, emphysema of early-onset or basilar hyperlucency), whereas most others endorse testing all patients with COPD, whether or not suggestive features are present. In this instance, all three authors reviewed all guidelines again on this issue to assure accuracy in their ratings.

After final adjudicated ratings for each guideline were obtained, the number of guidelines receiving each particular rating (Y, YC, Eq/NC, N) was calculated for each clinical statement (Table 2). In addition to this guideline rating distribution – called “individual statement agreement” – we also aimed to quantify the degree of overall guideline concordance, which we call the “overall guideline agreement percentage.” Guideline concordance requires two guidelines to share the same rating (eg, “Yes,” “Yes, conditional,” etc.) for a specific clinical statement. Therefore, calculating the “overall guideline agreement percentage” involves comparing each guideline to every other guideline on each of the 24 clinical statements. In total, with 15 guidelines, there are 105 guideline comparisons for each clinical statement (ie, each guideline with each other one) and an overall total of 2520 comparisons (105 guideline comparisons for each statement × 24 clinical statements). While the number of guideline comparisons is rather straightforward, concordance also requires that the type of ratings for each statement (eg, Yes; Yes, conditional; Equivocal/No Comment; No) on each guideline match. The number of matching guideline comparisons is a direct result of the number of guidelines receiving a particular rating (Table 5, see legend for further explanation).

Table 5.

Number of matching guideline comparisons (“overall guideline agreement percentage”)

Number of guidelines receiving rating	Number of matching guideline comparisons
0	0
1	0
2	1
3	3
4	6
5	10
6	15
7	21
8	28
9	36
10	45
11	55
12	66
13	78
14	91
15	105

Notes: To better understand the concept of “overall guideline agreement percentage,” consider a clinical statement where none of the 15 guidelines were rated “Yes.” It would be impossible for a guideline comparison to match on a “Yes” rating for that statement and the number of matches is 0. Now consider a clinical statement where only 1 of the 15 guidelines was rated “Yes.” Again, it would be impossible for a guideline comparison to match on a “Yes” rating for that statement and the number of matches is 0. Finally, consider a clinical statement where 2 of the 15 guidelines were rated “Yes.” It would now be possible for a guideline comparison to match on a “Yes” rating, but there is only one such comparison for which this would occur and the number of matches is 1. This table lists the number of matching guideline comparisons as the number of guidelines receiving a particular rating increases from 0 to 15. Guideline concordance, therefore, is measured with a parameter called the “overall guideline agreement percentage,” which is the proportion of all guideline comparisons matching on the rating for the same clinical statement (example in Table 4).

Importantly, a guideline comparison match is possible when neither guideline addresses the specific clinical issue posed by one of the 24 statements, ie, both compared guidelines were rated “Equivocal/No Comment” on the statement. To distinguish topics on which PGs explicitly agreed in endorsing an action or agreed in disapproving an action, parameters called “affirmative agreement percentage” and “negative agreement percentage” were developed (Table 4).

Finally, in the context that the available guidelines have spanned several decades and that the emergence of new knowledge over time can affect recommendations in guidelines and could be the source of discordance, a “time-clustered” analysis of guideline concordance was conducted. Specifically, guidelines were stratified into three clusters by year of publication: 2013–2018, 2008–2012, and 2007 and earlier. The same analysis of concordance among guidelines as was conducted for the total group was conducted within each time cluster. If observed discordance is attributable to the emergence of new knowledge about AATD, then the degree of concordance between guidelines should be lower within time clusters than between guidelines published at largely separated time intervals.

Statistical analysis was performed using R software version 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria).

Results

As shown in Figure 1 (which depicts the guideline selection using the PRISMA method and format), the search strategy produced 936 unique records, of which 805 were eliminated based on the review of title and abstract. The 131 remaining articles were reviewed in full text; 116 were excluded, leaving 15 eligible guidelines that comprise the study sample (Table 1).¹⁷ Eligible guidelines emanated from 11 countries in North America, Europe, and South America, with authors from 53 countries. Several guidelines represented multiple iterations from the same medical society (eg, the ATS and ERS both issued two guidelines, one jointly; and the CTS issued two guidelines, one in 2001 and the later one in 2012).^2–4

Figure 1.

PRISMA flow diagram. Medline and Embase were searched using search terms “alpha-1 antitrypsin deficiency,” “COPD,” and “guidelines.” Eligible guidelines were published in English and were issued by official respiratory organizations/medical societies and/or by national organizations.

The distribution of adjudicated guideline ratings for each clinical statement shows substantial variability (Table 2). There is not a single clinical statement where all 15 guidelines receive the same rating. Also, notable is the large number of guidelines rated “Eq/NC” for many clinical statements.

The overall level of agreement among the 15 guidelines on the 24 clinical statements was 47% (1190/2520 comparisons [Table 6]). Affirmative agreement accounted for 42% of the matching comparisons (501/1190 comparisons), whereas 48% of the matching guideline comparisons were rated as “Equivocal/No Comment” on a specific statement (568/1190 comparisons). Higher agreement surrounded concepts of “When to treat” and “How to treat”; however, even these only agreed approximately half the time.

Table 6.

Overall guideline agreement percentage and by rating bundle (ie, when to test, etc.)

	Number of statements	Number of comparisons	Number of matching comparisons	Agreement percentage	Agreement breakdown by rating
Y	YC	Eq/NC	N
Overall	24	2520	1190	47%	470 (39%)	31 (3%)	568 (48%)	121 (10%)
When to test	10	1050	434	41%	150 (34%)	11 (3%)	190 (44%)	83 (19%)
How to test	2	210	118	56%	79 (67%)	1 (1%)	2 (2%)	36 (30%)
How to manage	8	840	425	51%	143 (34%)	13 (3%)	268 (63%)	1 (0%)
When to treat	3	315	181	57%	88 (49%)	6 (3%)	87 (48%)	0 (0%)
How to treat	1	105	32	30%	10 (31%)	0 (0%)	21 (66%)	1 (3%)

Abbreviations: Y, yes; YC, yes, conditional; Eq/NC, equivocal/no comment; N, no.

In considering individual clinical statements (Figure 2), the highest affirmative agreement percentage (guidelines agreeing that the following should be performed) was for the statements: “Initial testing should include a serum AAT level.” (74%), “I should encourage smoking cessation” (63%), and “I should only treat when the serum AAT level is below a protective threshold value and/or when there is a severe deficient genotype/phenotype” (63%). Conversely, the highest negative agreement percentage (guidelines agreeing that the following should not be performed) was for the statements: “Only patients with suggestive features of AAT deficiency should be tested” (43%), “Patients with airflow obstruction that is fully reversible with bronchodilators should be tested (ie, ‘pure’ asthma)” (34%), and “Testing should always include both a level and genotype/phenotype” (34%).

Figure 2.

Affirmative agreement among guidelines on individual clinical statements. For each clinical statement, the affirmative proportions of guideline comparisons endorsing an action (with or without an added condition, Y + YC) are plotted. Negative agreements are also plotted. The highest affirmative agreement percentage was for the statement: “Initial testing should include a serum AAT level” (74%). The highest negative agreement percentage was for the statement: “Only patients with suggestive features of AAT deficiency should be tested” (43%).

Abbreviations: Y, yes; YC, yes, conditional; AAT, alpha-1 antitrypsin.

Many guidelines were concordant in not addressing some of the 24 statements. For example, 74% of the guideline comparisons were rated “Eq/NC” regarding, “I should treat patients with panniculitis as first-line therapy.” Other statements rarely receiving definitive recommendations were “I should refer my patient to a pulmonologist and/or tertiary center” (63%), “I should get a CT chest at baseline” (52%) and “I should get a liver ultrasound on my patients at baseline” (52%).

Finally, to address the question of whether discordance between guidelines reflects changing knowledge over time, Table 7 presents the results of a time-clustered analysis. Specifically, the degree of discordance between guidelines published in 5-year windows (2013–2017, 2008–2012, 2007 and earlier) was similar (ie, for guidelines published between 2013 and 2017 [N=6 guidelines], overall discordance =51%; 2008–2012 [N=3], overall discordance =35%; 2007 and before [N=6 guidelines], overall discordance =49%), thereby suggesting that the passage of time and acquisition of new knowledge about AATD over time did not account for the observed discordance between guidelines.

Table 7.

Agreement percentage among guidelines stratified by time periods: 2013–2017, 2008–2012, and 2007 and earlier

	Number of statements	Number of comparisons	Number of matching comparisons	Agreement percentage	Agreement breakdown by rating
					Y	YC	Eq/NC	N
All AATD Guidelines 1996–2017 (N=15)
Overall	24	2520	1190	47%	470 (39%)	31 (3%)	568 (48%)	121 (10%)
When to test	10	1050	434	41%	150 (34%)	11 (3%)	190 (44%)	83 (19%)
How to test	2	210	118	56%	79 (67%)	1 (1%)	2 (2%)	36 (30%)
How to manage	8	840	425	51%	143 (34%)	13 (3%)	268 (63%)	1 (0%)
When to treat	3	315	181	57%	88 (49%)	6 (3%)	87 (48%)	0 (0%)
How to treat	1	105	32	30%	10 (31%)	0 (0%)	21 (66%)	1 (3%)
AATD Guidelines 2013–2017 (N=6)
Overall	24	360	183	51%	96 (52%)	2 (1%)	63 (34%)	22 (12%)
When to test	10	150	80	53%	46 (58%)	1 (1%)	17 (21%)	16 (20%)
How to test	2	30	22	73%	16 (73%)	0 (0%)	0 (0%)	6 (27%)
How to manage	8	120	50	42%	15 (30%)	0 (0%)	35 (70%)	0 (0%)
When to treat	3	45	29	64%	18 (62%)	1 (3%)	10 (34%)	0 (0%)
How to treat	1	15	2	13%	1 (50%)	0 (0%)	1 (50%)	0 (0%)
AATD Guidelines 2008–2012 (N=3)
Overall	24	72	25	35%	10 (40%)	0 (0%)	11 (44%)	4 (16%)
When to test	10	30	10	33%	4 (40%)	0 (0%)	2 (20%)	4 (40%)
How to test	2	6	1	17%	1 (100%)	0 (0%)	0 (0%)	0 (0%)
How to manage	8	24	9	38%	4 (44%)	0 (0%)	5 (56%)	0 (0%)
When to treat	3	9	4	44%	1 (25%)	0 (0%)	3 (75%)	0 (0%)
How to treat	1	3	1	33%	0 (0%)	0 (0%)	1 (100%)	0 (0%)
AATD Guidelines 2007 and earlier (N=6)
Overall	24	360	175	49%	46 (26%)	8 (5%)	111 (63%)	10 (6%)
When to test	10	150	60	40%	3 (5%)	1 (2%)	52 (87%)	4 (7%)
How to test	2	30	16	53%	10 (63%)	0 (0%)	0 (0%)	6 (37%)
How to manage	8	120	74	62%	23 (31%)	7 (9%)	44 (59%)	0 (0%)
When to treat	3	45	21	47%	9 (43%)	0 (0%)	12 (57%)	0 (0%)
How to treat	1	15	4	27%	1 (25%)	0 (0%)	3 (75%)	0 (0%)

Abbreviations: Y, yes; YC, yes, conditional; Eq/NC, equivocal/no comment; N, no; AATD, alpha-1 antitrypsin deficiency.

Discussion

The main finding in this comparison of 15 available AATD guidelines from multiple medical societies in multiple countries is that there is substantial variation in recommendations regarding how to manage routine clinical issues in AATD. Sources of disagreement among the available guidelines include the degree to which the various guidelines addressed the 24 statements that were posed (ie, many were rated equivocal/no comment) as well as disagreement regarding specific managerial decisions that were addressed in the guidelines. The time cluster analysis of concordance among guidelines published within 5-year intervals shows that the degree of discordance within the time clusters was also high, suggesting that the emergence of new knowledge over time did not account for the observed discordance between guidelines.

While our study could not examine specific reasons for discordance among the guidelines (which would require querying the guideline authors), several explanations are possible. First, disease prevalence and manifestations vary across countries, likely causing recommendations to differ.¹⁸ For example, the recommendation to test all patients with fixed airflow obstruction for AATD in a country where the prevalence of severe deficiency of AATD is high (eg, countries with substantial populations of North European descent) may be inappropriate in countries where the prevalence is lower (eg, countries in Asia or Africa).¹⁹ As evidence of this influence of disease prevalence on recommendations, some guidelines made diagnostic recommendations that are conditional on disease prevalence.² For example, the 2003 ATS/ERS guideline recommends testing all symptomatic adults with fixed airflow obstruction in populations where the prevalence of AATD resembles that in North America and Northern Europe (level A recommendation) but lowers the strength of the recommendation (to level B) in populations where the prevalence is lower.²

A second possible reason for discordance among guidelines from different countries regards the availability of specific therapy in the source country and how health care is funded there. For example, in countries where augmentation therapy for AATD has not yet been approved by regulatory bodies (eg, the United Kingdom), guidelines would not be expected to recommend augmentation therapy, whereas treatment with augmentation therapy is endorsed in some guidelines from countries where augmentation therapy has been approved (eg, the United States and some European countries).²⁰ Similarly, in countries with universal health care coverage, recommendations regarding the allocation of costly therapy (like augmentation therapy) or expensive testing strategies may be especially conditioned by the perceived incremental cost-effectiveness of the recommendation.²¹

Of course, a third potential reason for discordance is that physicians and regulatory bodies can interpret available data about the efficacy of treating AATD differently. For example, the United States Food and Drug Administration (FDA) and the European Medicines Agency (EMA) have interpreted the results of a recent randomized trial of augmentation therapy (eg, the RAPID trial) differently, resulting in different approvals.²² Specifically, the drug Respreeza (CSL Behring, King of Prussia, PA) was approved with text stating that the drug slows lung destruction on CT by the EMA but not by the FDA.^23,24

Finally, as available guidelines span the interval from 1989 to 2016, the evolution of knowledge over time could account for varying recommendations.^2,15 However, the time-clustered analysis which showed that the degree of discordance among guidelines published within 5-year clusters was similar to the overall rate of discordance suggests that the emergence of new knowledge was not a primary driver of discordance among available guidelines.

The current study extends available knowledge in several ways. First, to our knowledge, this is the only study to compare recommendations among available guidelines regarding AATD. Second, this is, to our knowledge, one of only a few studies to assess concordance among available guidelines for any specific disease.^25–28 In 2001, LaCasse et al compared the quality and recommendations from 15 extant guidelines regarding the management of COPD.²⁸ Though lacking a systematic comparison of guideline concordance on each management element, the earlier analysis reported substantial discordance among the available guidelines. Details regarding AATD management recommendations were not available in the report. Mortensen and Nordestgaard examined variation among 5 medical societies’ guidelines in recommending statins.²⁵ As with our findings, frequent discordance was observed, eg, the prevalence of eligible candidates for statin use among an index Danish population using each of the 5 guidelines varied more than two-fold, ie, from 15% to 42%. Similarly, the other two analyses – which addressed relatively common conditions like managing urinary incontinence and imaging for dental and maxillofacial indications – also demonstrated broad heterogeneity of recommendations.^26,27 Beyond the general degree of discordance, guideline generation for uncommon diseases like AATD is especially challenged by the relative paucity of large, definitive studies upon which to make management recommendations.

Several shortcomings of the current study warrant discussion. First, although a Delphi process to adjudicate discordant ratings from individual reviewers was used to derive the final adjudicated ratings, there was, not surprisingly, some discordance among the reviewers (AA and UM) in their initial reviews. Such initial discordance raises the possibility that the 24 statements used to assess the AATD guidelines (Table 2) were insufficiently clearly formulated. At the same time, our practice of involving three independent raters and serially reviewing the guidelines to assure rating accuracy (eg, when there was diametric discordance) affirms the possibility that some discordance resulted from ambiguity in some of the specific guideline recommendations. As a specific example of ambiguity, the Belgian Thoracic Society guideline states:

Augmentation therapy should be considered in non-or ex-smokers with an AAT serum level under 50 mg/dL and with moderate to severe obstruction (FEV1 between 30–65% predicted) or with a rapid decline in FEV1.

This specific text could easily be variously interpreted as allowing augmentation therapy for specific FEV1 parameters or for unspecified changes in FEV1 over time, leading to potential ambiguity in the recommendation, especially for patients whose FEV1 initially exceeds 65% but appears to be declining.

Another potential shortcoming of the analysis is that ascribing recommendations to a guideline that mimics another guideline could cause overestimation of the degree of concordance. For example, if the Belgian Thoracic Society guideline directly quoted the ATS/ERS guideline, the two guidelines would be deemed concordant.² Notably, because such direct quotation of one guideline from another was infrequent in this study, we submit that this potential bias toward overestimating concordance is minimal. Given that our study showssubstantial discordance idespite a bias that would inflate concordance, this emphasizes the variability among guidelines and the resultant confusion that both clinicians and patients may experience in trying to offer and receive optimal care for AATD.

Conclusion

To our knowledge, this is the first study to analyze the degree of concordance among various international guidelines regarding AATD, and one of very few comparisons of guidelines for any disease. The analysis recognizes population and treatment milieu differences as potential sources of guideline variation across diverse populations. Beyond the specific implications for AATD, which might include attempts to harmonize the various guidelines by empaneling a broadly representative international group of disease experts, our findings also encourage similar analyses of guideline concordance for other diseases.

Abbreviations

AATD, alpha-1 antitrypsin deficiency; PG, practice guidelines; ATS, American Thoracic Society; ERS, European Respiratory Society; CTS, Canadian Thoracic Society; COPD, chronic obstructive pulmonary disease; FDA, Food and Drug Administration; EMA, European Medicines Agency.

Disclosure

Dr Robert A Sandhaus served in Advisory Board for Grifols, CSL Behring, and Shire, outside the submitted work. The authors report no other conflicts of interest in this work.