Clinical diaries in COPD: compliance and utility in predicting acute exacerbations

Abstract

Background

Daily diaries are often used to collect data on disease activity, but are burdensome and compliance may be poor. Their use in chronic obstructive pulmonary disease (COPD) and impact on the prevention and treatment of exacerbations is poorly researched.

Methods

We investigated diary-keeping in COPD and ascertained items that best predicted emergency attendances for exacerbations. Participants in the active limb of a clinical trial in COPD kept daily diaries rating breathlessness, cough, sputum, physical activity, and use of reliever medication.

Results

Data on 55 participants, 67% of whom were female, showed that overall compliance with diary-keeping was 62%. Participants educated to primary school level only had lower compliance (P = 0.05). Twenty patients had at least one emergency attendance, in whom the relative risk of an acute exacerbation for an increase in item score rose from six days prior to hospitalization, most sharply in the last two days. Even for optimal combinations of items, the positive predictive value was poor, the best combination being cough, activity level, and inhaler use.

Conclusion

Good compliance can be achieved using daily diaries in COPD, although this is worse in those with a poor educational level. Diary-keeping is not accurate in predicting acute exacerbations, but could be substantially simplified without loss of efficiency.

Introduction

Prospective daily diaries are often used to collect data such as symptoms and medication use to provide information about disease activity or impacts.^1,2 Although this methodology decreases recall bias compared with retrospective reports,² it is frequently burdensome, and reduced compliance decreases validity of the data.^3,4 Electronic diaries are preferable to paper² because actual compliance can be ascertained.⁵ Reasons for incomplete entries have included varying levels of participant motivation^6,7 and anxiety.⁸ However, there is no general correlation between diary completion and symptom severity,⁹ demographic or clinical characteristics, treatment, pain, mood, stress, or activity.⁴ The complex issues around compliance, and the relationship between symptoms and behavior has led to calls for more research examining the parameters that contribute to diary compliance.^1,3,4

We are especially interested in these issues in the context of chronic obstructive pulmonary disease (COPD), where little analytical research has been done on diary-keeping, even though diaries have been widely used on an empiric basis in clinical trials. COPD is a chronic progressive disease characterized by airflow obstruction, that in Western countries is most commonly due to cigarette smoking.¹⁰ Key symptoms are breathlessness, cough, and sputum production, and COPD negatively affects well-being and the ability to perform daily tasks.¹¹ Many studies in COPD have required completion of daily diaries of symptoms,^12,13 and while baseline clinical or physiological characteristics do not seem to influence diary compliance,¹³ whether other factors are important has not been addressed.

How to use diaries most effectively to anticipate acute exacerbations in COPD (AECOPD) seems especially problematic.^13,14 Given that prevention of these is a major goal of COPD management, this is a serious difficulty, and has been little addressed although much talked about.¹¹ Diaries for COPD were originally adapted from asthma studies, though with more emphasis on cough and sputum.¹⁵ Little attention has been paid to subjective “feelings of wellness” or daily activities, despite these being very important to patients.¹⁶ In studies of AECOPD, symptom-based diaries have often been used to define as well as predict the event, although they are poorly validated in this context.^16,17 Indeed, the definition of AECOPD in research studies is very variable. The importance of this issue is underlined by the recent call on the part of the US Food and Drug Administration for better standardization of acute outcome assessment in COPD studies, with more emphasis on patient-reported outcome instruments.¹⁸ Thus, in this paper we asked:

• What are the personal and social factors related to variations in diary keeping in COPD?

• How accurate and timely are the data collected in diaries for giving warning of an impending AECOPD?

• Which diary items or combinations give best “value” and efficiency with respect to predicting AECOPD?

Materials and methods

Study design and participants

This trial is registered on the Australian New Zealand Clinical Trials Registry as ACTRN12605000006640. The Pathways Home for Respiratory Illness project was a 12-month, controlled clinical trial that aimed to assist COPD patients develop self-management skills, by building self-efficacy for patient-identified health goals. It did not involve specific interventions around AECOPD. This paper reports on diary-keeping in the active intervention group only, who were recruited while hospitalized with an AECOPD and linked with a community health nurse mentor on discharge. Briefly, participants had a diagnosis of COPD confirmed by spirometry, at least one exacerbation in the last 12 months, scored > 21 on cognitive screening using the Mini-Mental State Examination, had a home telephone service, no other active lung disease, were not receiving palliative care and gave written informed consent to participation.

Data collection

Daily diary entries were made in the evening for the previous 24 hours, and were initially paper-based, but participants were offered the option of using an electronic web-based diary subsequently.¹⁹ Paper diaries were collected weekly and entered electronically. For each of the six diary items, ie, breathlessness, cough, sputum, general wellness, physical activity, and “rescue” inhaler use, participants were asked to nominate their usual (“baseline”) levels. We used validated measures for breathlessness,²⁰ cough and sputum,²¹ and estimated physical activity level (average minutes per day spent on physical activity) and rescue medication use (number of inhalations per day of bronchodilator medication) over the four weeks prior to hospital admission. This diary-item baseline information was summarized and presented prominently in the diaries for reference purposes, following recommendations to record diary assessments as changes from an agreed baseline.²²

Each day participants were asked to rate items compared with their baseline using a seven-point rating scale, with the mid-point of the scale for any item as “no change from normal”. Participants received instructions on diary completion and completed at least one day under supervision.

Data management

Participants were recruited throughout the year, and for each patient day 1 was defined as the day on which the first diary entry was made. The study “year” was analyzed as 360 consecutive available days (12 “months” of 30 days’ duration). For each participant, the “available days” were 360 minus periods of hospitalization, terminating at the date of death or study completion. Three measures of diary compliance were defined, ie, overall compliance (percentage of available days that the diary was completed), duration of compliance (days elapsed from first to last diary entry, as a percentage of available days), and active compliance (percentage of days that the diary was actually completed between the first and last diary entries).

Participants with less than 75% overall compliance were excluded from the analyses of exacerbation prodromes. Missing entries were handled using the last observation carried forward approach, to provide an entry for every available day. For analysis, the item reports were assigned consecutive integer scores of −3 (much better/much more) to +3 (much worse/much less), so an increase in score represented a worsening of that item. An AECOPD was defined as a respiratory-related visit to the emergency department, whether admitted or not, ie, a “gold standard” event-based criterion. This allowed use of the diary entries to assess prediction of exacerbations, but not to define the exacerbation itself.

Statistical analyses

The three measures of diary compliance were compared across demographic and clinical variables using Student’s t-tests and one-way analysis of variance, after transformation to remove skewness where required. Associations among baseline and diary compliance measures were assessed using unadjusted Spearman’s rank correlations. The predictive value for AECOPD of diary-based scores was compared for single and combined item scores using log binomial regression to estimate relative risk. Relative risks were also estimated using item scores for single days prior to the acute exacerbations. Lagged scores for individual items were combined to form a series of multiday measures: two-day average, ie, the average for the two previous days, and adjusted for direction of change in item score over those two days; six-day weighted average, ie, the average of item scores for the previous six days giving greater weight to days closer to the acute exacerbation; and six-day change, ie, the average daily change over the six days prior to the exacerbation. For comparability, the multiday item scores were standardized to have a zero mean and unit standard deviation. The diagnostic accuracy of predictive models was assessed using the area under the receiver-operating characteristic curve and by calculating the sensitivity, specificity, positive predictive value, and negative predictive value.

Results

Demographic and clinical information

Fifty-five participants recruited into the active limb of the intervention kept daily symptom diaries. Twenty (36%) participants transferred to an electronic diary during the study, but this did not influence the overall outcomes or this analysis. Thirty-seven (67%) participants were female; 40 (73%) lived in an urban area and 28 (51%) were married or in a de facto relationship; 11 participants (20%) had completed education to year 10 or above, 33 (61%) to years 7–10, and 10 (19%) had a primary school education only.

Baseline forced expiratory volume in one second (FEV₁) measurements indicated that 19 participants had very severe COPD by Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria, and 26 had severe and nine had moderate disease (data not available for one participant). In total, there were 116 acute exacerbations involving 33 patients with a range of 1–17 exacerbations. Participants experiencing an AECOPD had on average 3.7 ± 3.83 episodes, and spent on average 26.3 ± 32.3 days in hospital over the 12-month follow-up period, with the maximum number of days hospitalized being 157.

Diary compliance

Overall compliance with diary-keeping was 62%, with a duration compliance of 73% (Table 1). Active compliance (daily compliance while diaries were actually being kept) was 85%. Overall and active compliance rates were similar across the demographic variables of gender, rurality, and marital status (Table 1). However, participants educated to primary school level only had a lower duration of compliance (P = 0.05) than those educated to years 7–10 or above year 10 (Table 1). There were no significant differences in measures of compliance across disease severity variables (Table 2). There were significant correlations between active compliance with greater breathlessness and Medical Research Council functional dyspnea score (Table 3). There were also significant correlations between duration compliance and greater cough and more activity.

Table 1.

Compliance with diary keeping by demographic variables in the Pathways Home study

Variable	N	Overall compliance*	P value	Duration compliance†	P value	N	Active compliance‡	P value
All subjects	55	62 (11,156/18,028)		73 (13,268/18,028)		49	85 (11,156/13,268)
Gender
Males	18	66 (3,764/5,657)	0.65	73 (4,153/5,657)	0.96	16	90 (3,764/4,153)	0.31
Females	37	60 (7,392/12,371)		74 (9,115/12,371)		33	82 (7,392/9,115)
Residential location
Rural	15	61 (2,967/5,002)	0.95	74 (3,622/5,002)	0.96	13	81 (2,967/3,622)	0.26
Urban	40	62 (8,189/13,026)		73 (9,646/13,026)		36	86 (8,189/9,646)
Marital status
Married/de facto	28	63 (5,490/8,699)	0.86	71 (6,226/8,699)	0.64	25	89 (5,490/6,226)	0.30
Other	27	61 (5,666/9,329)		76 (7,042/9,329)		24	80 (5,666/7,042)
Education level¶
Primary	10	42 (1,170/2,941)	0.11	48 (1,385/2,941)	0.05	8	87 (1,170/1,385)	0.67
Years 7–10	33	65 (7,258/11,200)		77 (8,679/11,200)		29	84 (7,258/8,679)
Above year 10	11	69 (2,369/3,527)		82 (2,844/3,527)		11	84 (2,369/2,844)

Notes: Compliances are expressed as mean percentage (number of days completed/total number of days available for completion).

^*Overall compliance: percentage of days of the entire study period (360 days) that the diary was completed;

^†Duration compliance: days elapsed from first diary entry to last, as a percentage of days available;

^‡Active compliance: percentage of available days that the diary was completed from the first to the last diary entry. Days after death and when in hospital were excluded from all measures;

^¶Education level not available for one participant.

Table 2.

Compliance with diary keeping by clinical variables in the Pathways Home study

	N	Overall compliance*	P value	Duration compliance†	P value	N	Active compliance‡	P value
Admissions to hospital
None	21	70 (5,284/7,560)	0.23	84 (6,319/7560)	0.20	20	83 (5,284/6,319)	0.25
1 to 3	22	62 (4,124/6,738)		71 (4,727/6738)		20	88 (4,124/4,727)
4 or more	12	49 (1,748/3,730)		61 (2,222/3730)		9	82 (1,748/2,222)
Days hospitalized
None	21	70 (5,284/7,560)	0.10	84 (6,319/7560)	0.13	20	83 (5,284/6,319)	0.44
1–14 days	17	68 (3,632/5,487)		75 (4,008/5487)		15	89 (3,632/4,008)
More than 2 weeks	17	47 (2,240/4,981)		60 (2,941/4981)		14	83 (2,240/2,941)
Visits to emergency department
None	46	64 (9,537/15,001)	0.21	77 (11,541/15,001)	0.10	43	83 (9,537/11,541)	0.16
1 or more	9	52 (1,619/3,027)		55 (1,727/3,027)		6	95 (1,619/1,727)
Severity (baseline FEV1 (% pred))
Mild (60%–80%)	2	73 (522/720)	0.65	96 (689/720)	0.46	2	75 (522/689)	0.64
Mod (40%–59%)	17	58 (3,418/5,590)		67 (3,991/5,590)		15	85 (3,418/3,991)
Severe (<40%)	35	65 (7,216/11,358)		77 (8,588/11,358)		32	85 (7,216/8,588)

Notes: Compliances are expressed as mean percentage (number of days completed/total number of days available for completion).

^*Overall compliance: percentage of days of the entire study period (360 days) that the diary was completed;

^†Duration compliance: days elapsed from first diary entry to last, as a percentage of days available;

^‡Active compliance: percentage of available days that the diary was completed from the first to the last diary entry. Days after death and days when patient in hospital were excluded from all measures.

Table 3.

Spearman’s rank correlations (r) of baseline measures with diary compliance measures

Baseline measure	Overall compliance	Duration compliance	Active compliance
Breathlessness	−0.02	−0.10	0.30*
MRC dyspnoea	0.06	−0.09	0.18*
Cough	−0.06	0.14*	−0.06
Sputum	−0.07	−0.02	−0.06
Activity	0.11	0.21*	0.01
Medication	−0.04	−0.00	0.01

Note:

^*P-value < 0.05.

Diary item scores as predictors of acute exacerbations

Thirty-four patients achieved ≥75% compliance for diary-keeping, of which 20 had at least one AECOPD. In this group, there were 59 AECOPD, (range 1–10 per patient). Figure 1 shows the relative risk of an acute exacerbation for a one-unit increase in single day scores (a one-category worsening of any item score) at each of the 10 days prior to an acute exacerbation. Generally, the relative risks increase from six days prior to the exacerbation, but most sharply in the last two days. For this reason, we chose to combine scores for each item over the six days prior and also the two days immediately before the exacerbation day itself. Table 4 shows the relative risks for the lagged multiday scores. There was little difference in relative risk of AECOPD between the two-day average and the six-day weighted average before the event, with breathlessness and wellness scores giving the greatest increase in risk. The variability of scores over the periods before AECOPD did not improve the predictive value of item scores.

Figure 1.

Relative risk of acute exacerbation for a one-unit reduction in single day symptom scores from the current day to 10 days prior to the exacerbation.

Table 4.

Relative risk (95% confidence intervals) of an acute exacerbation for a one standard deviation reduction in single-day and lagged multi-day symptom scores

Symptom	2-day average†	P value	6-day weighted average	P value	6-day change	P value
Breathlessness	2.15 (1.46–3.18)	0.000	2.32 (1.48–3.62)	0.000	0.96 (0.78–1.20)	0.752
Cough	1.94 (1.46–2.58)	0.000	1.88 (1.43–2.48)	0.000	1.21 (1.06–1.39)	0.005
Sputum	1.88 (1.33–2.64)	0.000	1.86 (1.31–2.66)	0.001	1.10 (0.98–1.23)	0.112
Wellness	2.17 (1.59–2.95)	0.000	2.26 (1.61–3.17)	0.000	1.12 (0.95–1.33)	0.174
Activity	1.93 (1.43–2.59)	0.000	1.93 (1.40–2.65)	0.000	1.25 (1.09–1.43)	0.001
Medication	1.59 (1.07–2.38)	0.023	1.63 (1.07–2.48)	0.022	1.04 (0.85–1.27)	0.699

Notes: Patients with less than 75% overall diary compliance were excluded. N = 36 (59 AEs for 20 patients, range 1–10).

^†Corrected for change.

Diagnostic value

Area under the receiver-operating characteristic curves for the two-day average and six-day weighted averages before AECOPD are shown in Table 5. The areas under the curve for the six-day weighted average were greatest for breathlessness (0.722) and wellness (0.717), but with little gain relative to the two-day pre-AECOPD average. Given this, we decided to focus on the two-day average for further analysis. There was no consistent benefit gained by combining two items, but minor improvements were seen across most three-item combinations. Nevertheless, even for optimal combinations, eg, cough, activity, and medication use, the positive predictive value was poor. While this gave a sensitivity of over 90% (ie, including the large majority of true positives), it gave a specificity of less than 25% (ie, most of these occurrences are false positives). The vast majority of these “diary events” did not lead to change in treatment or any other specific action, and did not meet our definition of an exacerbation (see Discussion section).

Table 5.

Area under the curve (AUC) values for single items and for combinations of two and three items for the 2-day average corrected for change

AUC	Cough	Activity	Breathlessness	Sputum	Wellness	Medication
Single item
Current day	0.705	0.725	0.725	0.670	0.724	0.646
2-day average†	0.688	0.705	0.700	0.666	0.708	0.615
6-day weighted average	0.672	0.707	0.722	0.656	0.717	0.636
Two items combined
2-day average†
Cough		0.727	0.709	0.696	0.718	0.704
Activity			0.721	0.720	0.717	0.722
Breathlessness				0.706	0.714	0.707
Sputum					0.710	0.690
Feel						0.718
Three items combined
Cough + activity			0.729	0.728	0.722	0.733
Cough + breathlessness				0.715	0.722	0.714
Cough + sputum					0.721	0.713
Cough + feel						0.720
Activity + breathlessness				0.723	0.724	0.725
Activity + sputum					0.718	0.727
Activity + feel						0.726
Breathlessness + sputum					0.717	0.708
Breathlessness + feel						0.721
Sputum + feel						0.717

Note:

^†Corrected for change.

Discussion

In this study, we have analyzed some core issues around compliance with diary-keeping in a population with severe COPD. We have assessed the content of the diary to test whether there is redundancy to allow for simplification, and focused on the utility of using changes in items to help predict AECOPDs. The AECOPD definition used was attendance at a hospital emergency department with worsening clinical symptoms. Can such data allow us to intervene early to avoid these major events, which are so distressing for patients?²³ For diaries to work as an instrument to help in management, they need to be kept and the items measured useful. There has been disappointingly little validation of these aspects in COPD for over 20 years since they were first introduced for empiric use.^15,17,18

The use of diaries to capture the reality of subjects’ lives in chronic disease is quite problematic. Although not much studied, this is also likely to be the case in COPD, where only 41% of participants achieved 80% compliance in an open observational study.²⁴ In the current study, our equivalent, ie, overall compliance, was higher at 53% over 12 months. The higher compliance achieved in our study may have been due to the mentoring process, with regular phone calls from the assigned community nurse. Patient participation in a controlled trial that supported the development of self-efficacy may also have played a part, because there is evidence that participants will tolerate the burden of diary-keeping if they feel it will help them.^25,26 Finally, there is evidence that participants in research studies will take on additional burdens for altruistic reasons unrelated to their chronic illness.²⁷

There is always the concern with nonobserved diary-keeping over how accurate and timely the entries may have been, but the fact that those who transferred to daily electronic diaries did not differ in compliance attributes to those who used paper only (data not shown) would suggest there was no substantial deviation. We may also have benefitted from an “end of day” reflective diary regimen, which has been found to have a compliance advantage.³ Our compliance was similar across severity of COPD by GOLD criteria,²⁸ and across demographic characteristics. Only the group with poorest educational attainment had a significantly worse compliance. The high compliance achieved allowed meaningful data analysis to assess how well the diaries served as a guide to the forewarning of exacerbations.

Given that AECOPD are major and traumatic life events for patients^13,29 and a source of major expense for health systems,³⁰ COPD guidelines focus on interventions to prevent them.^11,14 We found that, of our single item measures, breathlessness gave the best signal for an impending exacerbation, but this was not really evident until just two days before the exacerbation. For a combination of items, cough, activity and increase in symptomatic treatment use was best. We would suggest that focusing on these three items could make diary use in COPD a great deal simpler than currently, and because these are objectively measurable, the future may lie in more sophisticated monitoring systems that are independent of patient self-assessment. While there have been recent developments, such as a validated diary that give scores which are shown to change in response to exacerbations,³¹ these were not available at the time of our study design and require substantial investment in information technology for use.

In the recent review of the literature, Eff ing et al found that about 40% of AECOPD definitions have used “events”, including taking extra medication, but most often a presentation to hospital, as used in the current analysis.¹⁷ However, they found that the use of extra medications is dependent on the decision of patients to seek help and the vagaries in management by their primary care physicians.³² Thus, hospital attendance has generally been seen as the most objective outcome for defining an AECOPD. The alternative approach of symptom-based definitions of exacerbation has been rather arbitrary, sometimes using specific symptoms, eg, increased breathlessness and change in sputum, as in the original diary use by Anthonisen et al,¹⁵ but frequently nonspecific, such as an ill-defined “worsening”.³³ Authors using the criteria of Anthonisen et al¹⁵ have frequently added “cough”, even though this has not been specifically validated. Recent patient-centered qualitative research has led to the addition of “feeling” and daily activity status to the definition of an exacerbation.¹⁶

In spite of the volume of research using critical events inferred from diaries, and the importance of this issue,¹⁸ there has been very little attempt to validate these different approaches. Furthermore, the time course needed to qualify for an exacerbation has varied greatly, from immediate,³⁴ 24 hours or less,³⁵ or up to three days.³⁶ It is not surprising that there have been recent appeals to focus on the best use of symptom diaries to improve and standardize the definition of AECOPD. We have used the most objective “event” definition of an AECOPD, ie, hospital attendance, to avoid basing the definition on the same measures used to predict an exacerbation, thus avoiding circularity in study analysis. While this approach leads to fewer recognized exacerbations, they are unequivocal and highly relevant.²⁴ While our study was quite small by comparison with large multicenter studies done for drug certification reasons, it makes up for this by the detailed nature of the diary-keeping and its comprehensive analysis; it is unlikely that the conclusions and clinical relevance would be different with larger numbers.

One of the most difficult questions our study raises is the usefulness of diary-keeping for predicting even the most unequivocal and “severe” AECOPD. Even using a combination of items to give a high sensitivity, we found this also gave a low specificity, so most “predicted exacerbations” were false positives. While patients can be trained to respond to an increase in symptoms by alterations in medication use,^36,37 the clinical efficacy of this strategy has been shown to be small, with a large potential for overtreatment and unwanted adverse drug effects, as would be predicted from our study. Perhaps what is required is a change in strategy to using changes in symptom scores to prompt further urgent assessment focused on deciding whether hospitalization and/or escalation of therapy is required. Such an approach will require personal monitoring, preferably with objective measurements, and detailed case management.

In summary, we have provided substantial new data on compliance using diary-keeping in COPD and its utility for warning of an impending AECOPD. Good compliance can be obtained in a research setting, and while worse with poor educational level, is not affected by demographic factors or disease severity. Importantly, we have shown that diary keeping can be substantially simplified, but is not diagnostically accurate, and at best should trigger further rapid clinical assessment. Further studies of such a system are required to test assumptions already inherent in current clinical guidelines and identify if it is possible to mitigate the impact of AECOPD.

Table 6.

Measures of diagnostic accuracy for models with the highest area under the ROC curve

	AUC	Sensitivity (%) (n = 59)	Specificity (%) (n = 11,402)	PPV (%)	NPV (%)
2-day average†
Cough	0.688	89.8 (53)	32.4 (3676)	0.7	99.8
		11.9 (7)	94.0 (10,659)	1.0	99.5
Cough + activity	0.727	93.2 (55)	30.7 (3,476)	0.7	99.9
		23.7 (14)	88.7 (10,064)	1.1	99.6
Cough + activity + medication	0.733	93.2 (55)	26.9 (3,050)	0.7	99.9
		20.3 (12)	91.2 (10,344)	1.2	99.6
Cough + activity + breathlessness	0.729	91.5 (54)	31.8 (3,609)	0.7	99.9
		23.7 (14)	89.0 (10,097)	1.1	99.6

Notes: Cut-points selected were for probabilities of exacerbation equal to 0.0025 and 0.01.

^†Corrected for change.