Skip to main content
The BMJ logoLink to The BMJ
. 2002 May 18;324(7347):1193–1194. doi: 10.1136/bmj.324.7347.1193

Patient non-compliance with paper diaries

Arthur A Stone a, Saul Shiffman b, Joseph E Schwartz a, Joan E Broderick a, Michael R Hufford b
PMCID: PMC111114  PMID: 12016186

Doctors often ask patients to recall recent health experiences, such as pain, fatigue, and quality of life.1 Research has shown, however, that recall is unreliable and rife with inaccuracies and biases.2 Recognition of recall's shortcomings has led to the use of diaries, which are intended to capture experiences close to the time of occurrence, thus limiting recall bias and producing more accurate data.3

The rationale for using diaries would be undermined if patients failed to complete diaries according to protocol. In this study we used a newly developed paper diary that could objectively record when patients made diary entries in order to compare patients' reported and actual compliance with diary keeping. For comparison, we also used an electronic diary designed to enhance compliance in order to assess what compliance rates might be achieved.

Methods and results

We recruited 80 adults with chronic pain (pain for ⩾3 hours a day and rated ⩾4 on a 10 point scale) and assigned 40 to keeping a paper diary and 40 to an electronic diary. On satisfying the eligibility criteria, each patient was assigned to the next training session for which he or she was available, regardless of which diary it was for. We conducted one training session for each diary each week, with each training session for the paper diary matched by time and day of the week with an electronic diary training session. Participants were paid $150 and gave their informed consent; patients given the paper diary were not told that compliance would be recorded electronically.

The paper diary comprised diary cards bound into a DayRunner Organizer binder. The cards contained 20 questions drawn from several common pain instruments and included fields to record time and date of completion. The diary binders were unobtrusively fitted with photosensors that detected light and recorded when the binder was opened and closed; these were extensively tested and validated. The electronic diary was a Palm computer with software for data collection in clinical trials and presented identical pain questions via a touch screen and recorded time and date of entries. This system (invivodata) incorporated several features to maximise compliance, including auditory prompts, and has demonstrated good compliance.4

Patients were instructed to complete daily entries at 10 am, 4 pm, and 8 pm within 15 minutes of the target times. With the electronic diary, entries could not be initiated outside the designated 30 minute windows. We considered paper diary entries to be compliant if they were made within the 30 minute windows. A more liberal secondary outcome allowed a 90 minute window around the target times. Reported compliance was based on the time and date that patients recorded on their paper diary cards. Actual compliance was based on the electronic record (from the record of diary binder openings for paper diaries). Paper diary entries were deemed compliant if the binder was opened or closed at any point during the target time window. We also assessed “hoarding” with the paper diary, defined as days when the diary binder was not opened but for which diary cards were completed.

After three days' familiarisation, the participants began 21 days of diary keeping with weekly feedback. Participants completed an average of 20.5 days, and the table shows compliance rates. With the paper diary, reported compliance was 90%, but actual compliance was 11% (20% with the wider 90 minute window). With the electronic diary, actual compliance was 94%. Hoarding was common with the paper diary: 32% of days contained no diary openings, yet reported compliance (30 minute window) for these days was 92%. Most of the 40 patients (75%) had at least one day of hoarding.

Commentary

This study shows that concerns about compliance with paper diaries are justified.5 Although patients reported high compliance, actual compliance was low and hoarding was common. The excellent compliance achieved with the electronic diary indicates that low compliance was not due to this particular sample or to an overly burdensome protocol. Overall, these results call into question the validity of paper diary records.

Table.

Compliance rates for 80 patients' record keeping in paper and electronic diaries

Paper diary (n=40) Electronic diary (n=40)
30 minute window
Total No of episodes* 2445 2435
No of excluded episodes  126    7
Mean per cent compliance (95% CI):
 Actual 11 (8 to 14) 94 (92 to 96)
 Reported 90 (86 to 94)
90 minute window
Total No of episodes 2445
No of excluded episodes  134
Mean per cent compliance (95% CI):
 Actual 20 (14 to 25)
 Reported 95 (92 to 98)
*

Participants using paper diaries should have completed 2445 diary entries within the designated time windows. Of these, 114 were eliminated because the diary was open for more than 45 minutes, and 12 were eliminated because laboratory visits overlapped with time windows. Participants using electronic diaries should have completed 2435 entries, but 7 overlapped with laboratory visits. 

Compliance statistics were calculated separately for each participant and then averaged. 

Compliance was significantly higher in the electronic diary group (t(73)=29.97, P<0.0001). 

Acknowledgments

We thank Jill Jackowsky, Dan Arnold, J Corey Harmon, Julie Grassell, and Doyle Carney for help with completing this study. We also thank Jackie Dunbar-Jacob, Jeffery Katz, Eva Krusinska, Dennis Turk, and Michael Weintraub for their comments on an early draft of the manuscript.

Footnotes

Funding: This study was supported by a grant from the National Cancer Institute (CA-85819; Arthur A. Stone, principal investigator) and by support in kind from invivodata.

Competing interests: AAS is vice-chair of the Scientific Advisory Board of invivodata, SS is a founder of invivodata, and MRH is director of scientific affairs at invivodata, which provides electronic diary support for clinical trials.

References

  • 1.Stone AA, Turkkan J, Jobe J, Bachrach C, Kurtzman H, Cain V. The science of self report. Mahwah, NJ: Erlbaum; 2000. [Google Scholar]
  • 2.Gorin AA, Stone AA. Recall biases and cognitive errors in retrospective self-reports: A call for momentary assessments. In: Baum A, Revenson T, Singer J, editors. Handbook of health psychology. Mahwah, NJ: Erlbaum; 2001. pp. 405–413. [Google Scholar]
  • 3.Shiffman S, Hufford M, Paty J. Subject experience diaries in clinical research. Part 1. The patient experience movement. Appl Clin Trials. 2001;10:46–56. [Google Scholar]
  • 4.Hufford MR, Shiffman S, Paty J, Stone AA. Ecological momentary assessment: Real world, real-time measurement of patient experience. In: Fahrenberg J, Myrtek M, editors. Progress in ambulatory assessment. Seattle, WA: Hogrefe and Huber; 2001. pp. 69–92. [Google Scholar]
  • 5.Hyland ME, Kenyon CA, Allen R, Howarth P. Diary keeping in asthma: comparison of written and electronic methods. BMJ. 1993;306:487–489. doi: 10.1136/bmj.306.6876.487. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Publishing Group

RESOURCES