Patient non-compliance with paper diaries

Doctors often ask patients to recall recent health experiences, such as pain, fatigue, and quality of life.¹ Research has shown, however, that recall is unreliable and rife with inaccuracies and biases.² Recognition of recall's shortcomings has led to the use of diaries, which are intended to capture experiences close to the time of occurrence, thus limiting recall bias and producing more accurate data.³

The rationale for using diaries would be undermined if patients failed to complete diaries according to protocol. In this study we used a newly developed paper diary that could objectively record when patients made diary entries in order to compare patients' reported and actual compliance with diary keeping. For comparison, we also used an electronic diary designed to enhance compliance in order to assess what compliance rates might be achieved.

Methods and results

We recruited 80 adults with chronic pain (pain for ⩾3 hours a day and rated ⩾4 on a 10 point scale) and assigned 40 to keeping a paper diary and 40 to an electronic diary. On satisfying the eligibility criteria, each patient was assigned to the next training session for which he or she was available, regardless of which diary it was for. We conducted one training session for each diary each week, with each training session for the paper diary matched by time and day of the week with an electronic diary training session. Participants were paid $150 and gave their informed consent; patients given the paper diary were not told that compliance would be recorded electronically.

The paper diary comprised diary cards bound into a DayRunner Organizer binder. The cards contained 20 questions drawn from several common pain instruments and included fields to record time and date of completion. The diary binders were unobtrusively fitted with photosensors that detected light and recorded when the binder was opened and closed; these were extensively tested and validated. The electronic diary was a Palm computer with software for data collection in clinical trials and presented identical pain questions via a touch screen and recorded time and date of entries. This system (invivodata) incorporated several features to maximise compliance, including auditory prompts, and has demonstrated good compliance.⁴

Patients were instructed to complete daily entries at 10 am, 4 pm, and 8 pm within 15 minutes of the target times. With the electronic diary, entries could not be initiated outside the designated 30 minute windows. We considered paper diary entries to be compliant if they were made within the 30 minute windows. A more liberal secondary outcome allowed a 90 minute window around the target times. Reported compliance was based on the time and date that patients recorded on their paper diary cards. Actual compliance was based on the electronic record (from the record of diary binder openings for paper diaries). Paper diary entries were deemed compliant if the binder was opened or closed at any point during the target time window. We also assessed “hoarding” with the paper diary, defined as days when the diary binder was not opened but for which diary cards were completed.

After three days' familiarisation, the participants began 21 days of diary keeping with weekly feedback. Participants completed an average of 20.5 days, and the table shows compliance rates. With the paper diary, reported compliance was 90%, but actual compliance was 11% (20% with the wider 90 minute window). With the electronic diary, actual compliance was 94%. Hoarding was common with the paper diary: 32% of days contained no diary openings, yet reported compliance (30 minute window) for these days was 92%. Most of the 40 patients (75%) had at least one day of hoarding.

Commentary

This study shows that concerns about compliance with paper diaries are justified.⁵ Although patients reported high compliance, actual compliance was low and hoarding was common. The excellent compliance achieved with the electronic diary indicates that low compliance was not due to this particular sample or to an overly burdensome protocol. Overall, these results call into question the validity of paper diary records.

Table.

Compliance rates for 80 patients' record keeping in paper and electronic diaries

	Paper diary (n=40)		Electronic diary (n=40)
30 minute window
Total No of episodes*	2445		2435
No of excluded episodes	126		7
Mean per cent compliance (95% CI)†:
Actual‡	11 (8 to 14)		94 (92 to 96)
Reported	90 (86 to 94)
90 minute window
Total No of episodes	2445
No of excluded episodes	134
Mean per cent compliance (95% CI)†:
Actual	20 (14 to 25)
Reported	95 (92 to 98)

^*Participants using paper diaries should have completed 2445 diary entries within the designated time windows. Of these, 114 were eliminated because the diary was open for more than 45 minutes, and 12 were eliminated because laboratory visits overlapped with time windows. Participants using electronic diaries should have completed 2435 entries, but 7 overlapped with laboratory visits. 

^†Compliance statistics were calculated separately for each participant and then averaged. 

^‡Compliance was significantly higher in the electronic diary group (t(73)=29.97, P<0.0001).