Validation of Rules of Two™ as a paradigm for assessing asthma control

Abstract

Assessing asthma control at each patient encounter is an essential task to determine pharmacologic requirements. Rules of Two (Ro2) was created from the original 1991 National Asthma Education Program guidelines to determine the need for controller therapy. This study determined the degree of agreement between Ro2 and the Expert Panel Report (EPR-3) definition of “in control” asthma and compared that value with the Asthma Control Test (ACT) in a group of asthmatics for the purpose of validating this tool. Patients with documented asthma were randomized to complete Ro2 or ACT prior to being assessed for asthma control by certified asthma educators using an EPR-3 template. Assessments occurred in either a specialty asthma clinic or at a local health fair. Patients were also queried for their personal assessment of asthma control. The primary statistical methodology employed was the degree of agreement (kappa) between each survey tool and the EPR-3 template. Of 150 patients, 72% did not have their asthma in control, based on the EPR-3 template. Ro2 identified 58% of patients not in control of their asthma, whereas ACT identified 36%, with kappa scores of 0.41 for Ro2 and 0.37 for ACT compared with the EPR-3 template. These were not significantly different. Of the 150 patients, 75% considered their asthma in control based on self-assessments, with a kappa of 0.23. In 14 of 73 ACT questionnaires, scores were not added or were misadded. Eliminating evaluation of static lung function significantly improved both kappa scores of Ro2 and ACT. In conclusion, Ro2 identifies patients with uncontrolled asthma as well as ACT and may be useful to the primary assessing clinician in determining asthma control.

The assessment of asthma control drives therapeutic decisions in patients with asthma, as articulated in the 2007 edition of the National Asthma Education and Prevention Program (NAEPP) (1), also referred to as the Expert Panel Report (EPR-3). That assessment reflects the contributions of two related domains, impairment and risk, and consists of patient-reported symptoms and symptom frequency, documentation of health care encounters, medication use, and objective measurement of lung function. The decision to step up, step down, or maintain any specific therapy rests upon the determination of asthma control.

Rules of Two™ (Ro2) was created in 1992 as a tool to signal the need for controller therapy according to the first National Asthma Education Program (NAEP) guidelines published in 1991. At that time, the use of controller therapy for patients with persistent asthma had not been generally accepted within the primary care community, and there was a need to develop an easily remembered synopsis of the NAEP criteria for mild persistent asthma, which was the point at which antiinflammatory therapy for asthma was first recommended. While abstracts demonstrating the impact of Ro2 educational efforts were presented at annual American Thoracic Society conferences in 1999 (2) and 2000 (3), no specific validation of Ro2 was ever attempted, primarily because the syntax of Ro2 was taken directly from NAEP and no specific need for separate validation was perceived.

By contrast, the well-validated Asthma Control Test™ (ACT) (4) introduced in 2004 followed a methodical development beginning with focus groups of asthma experts to identify specific issues relevant to the assessment of asthma control and then proceeding to demonstrate the most robust of these items, ultimately resulting in a validation for both adult and pediatric communities of the ACT as a standardized patient-reported questionnaire to assess asthma control. Multiple studies have since demonstrated ACT's relevance in predicting asthma morbidity (5, 6), and the 2007 NAEPP guidelines identified ACT as one of three validated patient-reported evaluations that could be used to assess asthma control.

The 2007 EPR-3 published a table enumerating multiple determinants of asthma control using the two domains of impairment and risk (Table 1) (1). Using this table as a template for good control, the current study compared the agreement between this standard and either the Ro2 (Figure 1) or the ACT (Figure 2) in predicting asthma control in a cohort of patients with documented asthma.

Table 1.

Assessing asthma control and adjusting therapy in youths ≤ 12 years of age and adults^*

		Classification of asthma control (≥12 years of age)
Components of control		Well controlled	Not well controlled	Very poorly controlled
Impairment	Symptoms	≤ 2 days/week but not more than once on each day	> 2 days/week or multiple times on ≤ 2 days/week	Throughout the day
	Nighttime awakenings	≤ 1x/month	≤ 2x/month	≤ 2x/week
	Interference with normal activity	None	Some limitation	Extremely limited
	Short-acting beta2-agonist use for symptom control (not prevention of EIB)	≤ 2 days/week	> 2 days/week	Several times per day
	Lung function
	• FEV1 or peak flow	> 80% predicted/personal best	60–80% predicted/personal best	<60% predicted/personal best
	• FEV1/FVC	> 80%	75–80%	<75%
Risk	Exacerbations requiring oral systemic corticosteroids	0–1/year	≤ 2/year (see note)
		Consider severity and interval since last exacerbation
	Reduction in lung growth	Evaluation requires long-term followup.
	Treatment-related adverse effects	Medication side effects can vary in intensity from none to very troublesome and worrisome. The level of intensity does not correlate to specific levels of control but should be considered in the overall assessment of risk.

^*Reprinted from Expert Panel Report 3 (1).

EIB indicates exercise-induced bronchospasm; FEV1, forced expiratory volume in 1 second; FVC, forced vital capacity.

Notes:

• The level of control is based on the most severe impairment or risk category. Assess impairment domain by patient's/caregiver's recall of previous 2–4 weeks and by spirometry or peak flow measures. Symptom assessment for longer periods should reflect a global assessment, such as inquiring whether the patient's asthma is better or worse since the last visit.
• At present, there are inadequate data to correspond frequencies of exacerbations with different levels of asthma control. In general, more frequent and intense exacerbations (e.g., requiring urgent, unscheduled care, hospitalization, or intensive care unit admission) indicate poorer disease control. For treatment purposes, patients who had ≥2 exacerbations requiring oral systemic corticosteroids in the past year may be considered the same as patients who have not-well-controlled asthma, even in the absence of impairment levels consistent with not-well-controlled asthma.

Figure 1.

Rules of Two patient asthma questionnaire.

Figure 2.

Asthma Control Test.

METHODS

This study recruited subjects with healthcare professional–diagnosed asthma who were taking prescription asthma medications, excluding those with known chronic obstructive pulmonary disease, pulmonary fibrosis, or other complicating lung disease. Patients with diagnosed asthma were randomized in a 1:1 fashion to complete either Ro2 or ACT. These self-reported tests were completed before assessment by certified asthma educators, who used a template taken directly from the EPR-3 guidelines to determine asthma control and had no prior knowledge of patient responses to either Ro2 or ACT. Not-well-controlled asthma was defined by the Ro2 as any positive response to the four-item questionnaire and by the ACT as a score of ≥19. The degree of agreement (kappa) between the more comprehensive EPR-3 template and Ro2 or ACT, in addition to patient self-perception of control, was calculated (7). A power analysis determined that a sample size of 150 was needed to detect a true kappa value of 0.70 based on a significance level of 0.05. This study was approved by the Baylor Health Care System institutional review board (IRB #009–084).

RESULTS

Of the 150 patients who participated in this study, 130 were surveyed at the Baylor Martha Foster Lung Care Center, an outpatient facility that is a part of Baylor University Medical Center at Dallas, Texas, and provides specialized asthma care. The remaining 20 patients were assessed at a local women's health fair. Spirometry was performed on each patient. The age of patients assessed ranged from 14 to 86, with a median age of 42 years; 33% were male and 67% female, which reflects the increased adult female incidence of asthma in addition to inclusion of participants in the women's health fair. In addition, 74% were Caucasian, 17% African American, and 9% Hispanic.

As shown in Table 2, 109 of the entire group of 150 patients (72%) assessed using a template taken directly from EPR-3 were identified as having uncontrolled asthma, compared with 45 of 77 patients (58%) assessed with the Ro2 and 26 of 73 patients (36%) assessed with the ACT. The degree of agreement, as assessed using a kappa statistic, was 0.41 for Ro2 and EPR-3 and 0.37 for ACT and EPR-3. There was no statistical difference in the agreement with EPR-3 between the Ro2 and ACT tools. By contrast but as expected, agreement with patient perception of self-control when compared to the EPR-3 assessment was poor, with only 25% of patients feeling their asthma was “not in control” (kappa 0.23).

Table 2.

Determinations of in-control asthma based on different tests

Test	Asthma in control (%)	Asthma not in control (%)	Agreement with EPR-3 (kappa)	Agreement with EPR-3 (%)
EPR-3 (n = 150)	28	72	—	—
Ro2 (n = 77)	42	58	0.41	73
ACT (n = 73)	64	36	0.37	64
Patient self-assessment (n = 151)	75	25	0.23	37

EPR-3 indicates Expert Panel Report-3; Ro2, Rules of Two; ACT, Asthma Control Test.

There were no false-positive ACT questionnaires when compared to the EPR-3 template. However, 26 out of 73 individuals were judged “in control” by ACT but “not in control” by EPR-3 (false-negative), yielding an overall agreement rate of 64%. In comparison, the false-positive rate for “not in control” asthma for the Ro2 instrument was 5 out of 77 patients, and the false-negative rate was 16 out of 77 patients. The overall agreement between Ro2 and EPR-3 was 73%.

Disagreements in the determination of asthma control by Ro2 and ACT when compared with the EPR-3 template occurred primarily in patients with lung function (forced expiratory volume in 1 second [FEV₁] or FEV₁/forced vital capacity [FVC]) that was abnormal. Ten of the 16 patients judged “in control” by Ro2 but not EPR-3 had abnormal lung function. Seventeen of 26 patients erroneously considered “in control” by ACT likewise had abnormal pulmonary function tests, although 7 of those 17 patients had other positive responses to EPR-3 questions that would have resulted in a “not in control” determination, despite an ACT score of ≥ 20. If resting lung function was not used to determine asthma control in the EPR-3 tool, then the kappa for both Ro2 and ACT would have been much higher (0.63 and 0.49, respectively). While not the primary endpoint of this study, 14 of 73 ACT questionnaires failed to add up or misadded scores. For determination of kappa, additions were completed or corrected prior to analysis.

DISCUSSION

The fundamental concept of asthma “control” has been a consistent element in the series of reports issued by the NAEP(P), first in 1991, revised in 1997, and most recently in 2007 with EPR-3. Delineation of impairment and risk domains is articulated in EPR-3, but the need for a comprehensive evaluation of specific aspects of asthma symptomatology and physiology is longstanding and has been widely accepted as the standard for asthma assessment. In this study, we used the specific elements of control articulated in EPR-3 as the basis of comparison for both Ro2 and ACT and overall determined “fair” agreement (8).

A consistent finding of multiple patient surveys is the discordance between self-assessment of asthma control and the assessment derived from asthma-specific questionnaires. The Asthma in America Survey (1998) (9) quantified the discrepancy between patient perception of asthma control and reality from in-depth questionnaires. Our group demonstrated that discordance in a group of self-identified asthmatics attending a state fair in the late 1990s (3). Subsequent follow-up surveys have continued to demonstrate this disconnect (10). In this study, as noted with earlier ones, a much higher percentage of patients felt they were “in control” by self-assessment than were considered by using an 8-question template drawn from EPR-3. The baseline finding that over three fourths of patients assessed with the EPR-3 tool were considered not in good control of their asthma is consistent with other studies (9, 11) and not a unique finding in our surveyed population. Both ACT and Ro2 identified patients who were “not in control,” but at a lower frequency than the standard from EPR-3. Even so, Ro2 appeared to be a slightly better tool than ACT for identifying patients who were judged as “not in control.”

A number of other asthma self-report questionnaires exist, such as the Asthma Therapy Assessment Questionnaire (12) and Asthma Control Questionnaire (13), which have likewise been validated and are used for clinical research purposes and are mentioned in EPR-3. We chose to compare the kappa of R02 and ACT with a template taken directly from EPR-3, to determine the degree to which both correlate with a more robust set of determinants of control.

The most significant explanation for the reduced kappa scores of both Ro2 and ACT with the EPR-3 comparator in our studied population relates to the objective measurements of static lung function, which classified more patients with asthma as not well controlled by EPR-3 criteria. EPR-3 assumes an abnormal FEV₁ or FEV₁/FVC to be reflective of uncontrolled asthma (1). Our clinic population that attends a specialty asthma management center may include a more severe asthmatic population, more likely to have chronically reduced lung function, an observation that might explain the lower kappa for both assessment tools. A certain proportion of this population may indeed manifest airway remodeling, which is unresponsive to even more aggressive asthma therapy, according to EPR-3, despite minimal symptomatology (1). At least one author has written that using lung function as a marker of uncontrolled asthma is misleading (14). In our studied population, the kappa scores for Ro2 and ACT improve quite significantly if lung function measurements are not considered part of the equation of good asthma control; in fact, excluding static lung function scores improves agreement to “substantial” in the case of R02 and “moderate” with ACT (8).

In this study, Ro2 was more likely to be completed than ACT. Indeed, an advantage of Ro2 over ACT lies in the fact that no computation is needed to come up with a score, which makes it easier for patients to complete and for clinicians to interpret. The specific contents of Ro2 help frame the conversation between clinician and patient about asthma, move beyond reported self-perceptions of asthma control, and allow for a brief but potent clinical encounter to determine asthma control and the potential need for changes in therapy.

In conclusion, the current study shows that both Ro2 and ACT identified patients with out-of-control asthma with similar agreement when compared to an EPR-3 assessment (Ro2 kappa = 0.41; ACT kappa = 0.37). Ro2 had a higher rate of agreement with EPR-3 in the classification of patients in control vs. out of control when compared with ACT (73% vs. 64%). Removing the measurement of static lung function significantly improves both kappa scores. While both Ro2 and ACT share statistically similar agreement with an EPR-3 template for good asthma control, Ro2 is more likely to be completed by patients and shows a slightly higher level of agreement with the standard EPR-3 assessment.