Abstract
BACKGROUND
The purpose of this study was to determine if retrospective recall of oral contraceptive (OC) adherence provides data that are similar to data collected via daily diaries over the same time period. Factors associated with inconsistent agreement between prospective and retrospective measurements of adherence also were explored.
STUDY DESIGN
A total of 185 women participated in a longitudinal, prospective cohort of OC users and 113 of these women provided complete information on OC adherence during follow-ups. Concordance beyond chance was assessed using weighted kappa statistics and logistic regression was used to identify factors associated with inconsistent reporting of adherence.
RESULTS
There was substantial agreement between prospective and retrospective adherence information (weighted kappa=0.64; 95% CI: 0.52, 0.77). Participants with a high school education or less and those who had problems with feeling sad while using OCs had increased odds of inconsistent reporting of adherence (OR=4.38, 95% CI: 1.41, 13.61 and OR=3.52, 95% CI: 1.25, 9.94; respectively).
CONCLUSION
While prospective data collection via diaries may improve accuracy, the added expense and burden on study participants may not be necessary. However, the use of retrospective recall may not be appropriate for all study populations.
Keywords: contraceptives, oral, medication adherence, reproducibility of results, measurement of adherence
INTRODUCTION
Approximately 17% of US women ranging from 15 to 44 years old use oral contraceptives (OCs), making OCs the most popular form of contraception in the US [1]. Among women using OCs, an estimated one million unintended pregnancies occur annually because of poor adherence [2]. When evaluating OC adherence for research purposes it is important to focus on the validity of data collected. Objective methods of ascertainment, such as Medication Event Monitoring Systems (MEMs), offer many advantages, but constraints regarding expense, complexity of use, and appropriateness for all types of medications have yet to be fully resolved [3–5]. Thus, among the most commonly used methods to collect oral contraceptive adherence data are retrospective questionnaires and concurrent diaries [6, 7]. The utilization of a daily diary allows individuals to track and record their OC use concurrently and can enable researchers to establish temporal relationships between exposure and outcome variables [8]. Diaries have also shown improved accuracies in health research in populations with lower education levels or when collecting more sensitive data such as sexual behavior or substance abuse information [9, 10]. While daily diaries can reduce the risk of recall and interviewer bias, their validity is contingent on properly filling them out every day [10]. Another method of measuring adherence is the retrospective interview or survey, in which individuals recall their OC use over a previously determined time frame. Retrospective interviews are subject to memory lapses, and recall and self-disclosure bias [9, 11], thus decreasing the validity of data collected via this method.
While a few studies comparing the validity of data collected via daily diaries and retrospective interviews regarding sexual behavior have been conducted [9, 10], to our knowledge there have been no studies done to evaluate the agreement between OC adherence levels obtained from daily diary data and levels obtained from retrospective techniques. Rather, studies of OC users have primarily focused on the ability to accurately recall OC histories including factors such as duration of use and formulation type [12–19]. Participant burden and study costs are greater when using diaries for data collection, but the risk of recall bias is higher with retrospective methods. The purpose of this study was to determine if retrospective recall of OC adherence over a 3 month period provides data that are dissimilar to data collected via daily diaries over the same time period. In addition, factors associated with inconsistent agreement between prospective and retrospective measurements of adherence were explored.
MATERIALS AND METHODS
Study design and population
The Fertility and Oral Contraceptive Use Study was a longitudinal, prospective cohort study conducted over two years that was approved by the local institutional review board. A total of 185 female participants aged 18–40 years were recruited from June 2009 to January 2011 at family medicine, obstetrics and gynecology, and family planning clinics affiliated with a local hospital and local public health department in North Carolina. In addition to approaching women in the waiting rooms, flyers were posted in the examination rooms of the clinics and informational letters were mailed to a subset of women who had been seen at the clinics during the 2009 calendar year. Women were eligible to participate in the study if they were current OC users, spoke English or Spanish, did not plan to move out of the area, had no history of sterilization procedures for the woman or her partner, and were never told by a physician that she and/or her partner would have difficulty conceiving or carrying a pregnancy to term.
Upon providing written informed consent, participants completed baseline interviews in private rooms at the clinics with trained, bilingual interviewers. The baseline interviews lasted approximately 15–20 minutes and collected information on demographic, lifestyle, and reproductive characteristics. In addition, anthropometric measurements were taken using standardized methods. Participants were then asked to complete a daily calendar diary for 3 months in which they recorded the following information: date; whether they took their pill, a placebo pill, or missed their pill; and if they had sexual intercourse. Most diary items simply required women to mark a check box. Upon completion of each monthly diary, participants mailed it back to the research office using a stamped, addressed envelope provided with the diary packet.
Additional follow-ups occurred at 3-month intervals, and women could be involved with the study for up to 15 months. During these follow-ups, information collected during the baseline interviews was confirmed and updated (e.g. confirmed women were still using OCs and updated information on anthropometric measurements, dual method use, frequency of sexual intercourse, etc.). The 3-month follow-up was a “rip-and-tear” postcard that asked questions related to OC use and the number of pills missed in the past 3 months. Participants checked their responses, ripped off the pre-stamped postcard, and placed it in the mail. The postcards were designed in such a way that the information mailed back to the researchers did not have the original questions included on them and only contained checkmarks by set responses (e.g. yes or no). All participants were compensated for their participation with gift cards to a local grocery store.
Measurement of variables
During the baseline interview, self-reported information on the following variables was collected: age, marital status, education, income, number of people residing in household, race/ethnicity, alcohol consumption, smoking, parity, prior discontinuation of OCs, reasons for OC use (i.e. to prevent pregnancy, lessen menstrual cramps/pain, and/ or lessen period length/heaviness), possible problems with OC use (i.e. ever had problems with headaches, mood swings, feeling sad, and/or weight gain while using OCs), and dual method use. Body mass index was calculated from height and weight measurements, and waist-to-hip ratio was calculated from measurements of waist and hip circumference.
The main outcome of interest was inconsistent agreement between the prospective (i.e. diary) and retrospective (i.e. postcard) measurements of adherence. On the diary cards, women simply checked for each day if they had taken their pill, forgotten to take their pill, or not taken their pill because it was a placebo pill. The 3-month postcard asked women the following: “During the past 3 months, how many active pills did you forget to take in your pill packs?” Women were able to choose from the following set of responses: 0, 1, or 2 or more.
The number of missed active pills recorded in the diaries was added during the 3 month period and categorized into 0 missed pills, 1 missed pill, and 2 or more missed pills. The summarized diary data were then compared to the retrospective recall data obtained from the 3-month postcard. If a woman’s prospective and retrospective adherence information did not perfectly agree, she was considered to be inconsistent in her reporting of adherence to an OC regimen.
Analysis
A total of 113 women were included in the analysis (Figure 1). These women either provided complete diary and 3-month follow-up information (n=105), or had missed 2 or more pills in the subset of diaries they had returned (i.e. they were considered to be in the highest category of non-adherence without completing all 3 diaries) and completed the 3-month postcard (n=8). Frequencies and percentages were calculated to describe the study population. Concordance beyond chance was assessed using weighted kappa statistics. The minimum acceptable value of kappa was set at 0 and the maximum attainable kappa was determined to be 0.93 [20]. Logistic regression was used to assess how various demographic, lifestyle, and reproductive variables were associated with inconsistent reporting of adherence. Backward elimination was used to retain only those variables significant at the P<0.10 level.
Figure 1.
Flow diagram of study participants.
RESULTS
Most participants were single (56.6%), non-Hispanic black (49.6%), and highly educated with nearly 70% indicating they had completed some college or more (Table 1). With respect to retrospective recall of adherence, the weighted kappa was 0.64 (95% confidence interval [CI]: 0.52, 0.77) indicating substantial agreement between the prospective and retrospective adherence information (Table 2). Furthermore, the amount of unachieved agreement beyond chance within the constraints of the marginal totals for adherence was 0.29 (i.e. kappa-maximum attainable kappa). According to the diary information, 43.3% of women missed 0 pills, 11.5% missed 1 pill, and 45.1% missed 2 or more pills. The range of missed pills for the 2 or more category was 2 to 26 with 23.5% indicating they missed exactly 2 pills and 5.9% showing 3 missed pills over the 3-month interval. Approximately 75% of women were consistent between their prospective and retrospective reporting of adherence. Of those with nonconcurrent reports, 15.0% (n=17) of women underreported their adherence and 9.7% (n=11) of women overreported their adherence on the 3-month postcard. With respect to self-reported adherence on the diaries and consistency of reporting between the two data collection methods, 83.7% of women who indicated they had missed 0 pills on their diaries had consistency of reporting as did 74.5% of women who missed 2 or more pills on their diaries. However, only 46.2% of women who marked that they missed 1 pill on their diaries had consistency between the diaries and 3-month postcard.
Table 1.
Comparison of select characteristics of women who completed all follow-ups for the adherence analysis (responders) versus women who only completed the baseline interview (nonresponders)
| Responders (N=113) |
Nonresponders (N=72) |
|||
|---|---|---|---|---|
| Characteristic | Number | Percent | Number | Percent |
| Age | ||||
| <25 | 40 | 35.4% | 27 | 37.5% |
| 25–34 | 55 | 48.7% | 33 | 45.8% |
| ≥35 | 18 | 15.9% | 12 | 16.7% |
| Marital status | ||||
| Married | 23 | 20.4% | 14 | 19.4% |
| Living with partner | 22 | 19.5% | 15 | 20.8% |
| Separated, divorced, or widowed | 4 | 3.5% | 6 | 8.3% |
| Single | 64 | 56.6% | 37 | 51.4% |
| Educational level | ||||
| High school or less | 34 | 30.1% | 37 | 51.4% |
| Some college | 38 | 33.6% | 23 | 31.9% |
| College graduate or more | 41 | 36.3% | 12 | 16.7% |
| Income | ||||
| ≤$30,000 | 60 | 53.1% | 51 | 70.8% |
| $30,001–$45,000 | 23 | 20.4% | 8 | 11.1% |
| >$45,000 | 30 | 26.6% | 13 | 18.1% |
| Race/ethnicity | ||||
| Hispanic | 12 | 10.6% | 19 | 26.4% |
| Non-Hispanic white | 37 | 32.7% | 17 | 23.6% |
| Non-Hispanic black | 56 | 49.6% | 33 | 45.8% |
| Other | 8 | 7.1% | 3 | 4.2% |
| Alcohol consumption | ||||
| Yes | 53 | 46.9% | 31 | 43.1% |
| No | 60 | 53.1% | 41 | 56.9% |
| Smoking | ||||
| Yes | 18 | 15.9% | 14 | 19.4% |
| No | 95 | 84.1% | 57 | 79.2% |
| Missing | 0 | 0.0% | 1 | 1.4% |
| Parity | ||||
| 0 | 49 | 43.4% | 25 | 34.7% |
| 1 | 29 | 25.7% | 18 | 25.9% |
| ≥2 | 35 | 31.0% | 29 | 40.3% |
| Use to prevent pregnancy | ||||
| Yes | 102 | 90.3% | 64 | 88.9% |
| No | 11 | 9.7% | 8 | 11.1% |
| Ever had problems with feeling sad while using OCs | ||||
| Yes | 25 | 22.1% | 6 | 8.3% |
| No | 88 | 77.9% | 65 | 90.3% |
| Missing | 0 | 0.0% | 1 | 1.4% |
| BMI | ||||
| <25.0 | 42 | 39.8% | 24 | 33.3% |
| 25.0–29.9 | 26 | 23.0% | 21 | 29.2% |
| ≥30.0 | 45 | 37.2% | 27 | 37.5% |
| Consistent reporting of OC adherence | ||||
| Yes | 85 | 75.2% | N/A | N/A |
| No | 28 | 24.8% | ||
Body Mass Index (BMI; calculated as kg/m2); Oral Contraceptive (OC)
Percents may not total 100 due to rounding.
P-values derived from Chi-square and Fisher’s Exact tests comparing responders and nonresponders were <0.05 for education, race/ethnicity, and ever having problems with feeling sad. Other variables examined that did not differ between responders and nonresponders included: number of people in household; prior discontinuation of OCs; use of OCs to lessen cramps, pain, length, and/or heaviness; ever had problems with mood swings, headaches, and/or weight gain while using OCs; dual method use; and waist-to-hip ratio.
Table 2.
Number of women classified into oral contraceptive adherence categories using prospective and retrospective measures
| Retrospective recall from postcard | ||||
|---|---|---|---|---|
| Prospective diary | 0 missed pills | 1 missed pill | ≥2 missed pills | Total |
| 0 missed pills | 41 | 3 | 5 | 49 |
| 1 missed pill | 4 | 6 | 3 | 13 |
| ≥2 missed pills | 7 | 6 | 38 | 51 |
| Total | 52 | 15 | 46 | 113 |
| Weighted κ=0.64, 95% CI: 0.52, 0.77 | ||||
In univariate assessments, less education, Non-Hispanic black race/ethnicity, and ever having problems with feeling sad while using OCs were all associated with statistically significant increased odds of inconsistent reporting of OC adherence (data not shown). Ultimately, only the education and problems with feeling sad variables met the criteria for inclusion in the multivariate model (Table 3). Specifically, participants with a high school education or less had 4.38 times the odds of inconsistent reporting of adherence as compared to participants who were at least college graduates (95% CI: 1.41, 13.61) and women who said they had ever had problems with feeling sad while using OCs had over three times the odds of inconsistent reporting of adherence as compared to participants who did not have that problem (odds ratio [OR]=3.52, 95% CI: 1.25, 9.94).
Table 3.
Adjusted odds ratios and 95% confidence intervals of the association between select characteristics and inconsistent reporting of oral contraceptive adherence
| Characteristic | Adjusted OR | 95% CI |
|---|---|---|
| Educational level | ||
| High school or less | 4.38 | 1.41, 13.61 |
| Some college | 1.16 | 0.35, 3.80 |
| College graduate or more | 1.00 | Referent |
| Problems with feeling sad | ||
| Yes | 3.52 | 1.25, 9.94 |
| No | 1.00 | Referent |
Confidence interval (CI); odds ratio (OR)
Adjusted for all other variables in the model.
DISCUSSION
In this diverse sample of adult women we found that there was substantial agreement between prospective diary information and retrospective recall of OC adherence, with nearly 75% of participants having perfect agreement (weighted kappa=0.64). However, there were groups of participants who were more likely to be inconsistent in their reporting of adherence. Specifically, women with a high school education or less and women who indicated that they had ever felt sad while on OCs had statistically significant increased odds of inconsistent reporting of adherence.
While several studies have investigated women’s ability to accurately recall OC histories [12–19], there is limited information on the validity of concurrent diary information versus retrospective recall with respect to OC adherence. Nearly two decades ago, a study was conducted among women attending university health services and publicly funded family planning clinics (n=103) to assess the agreement between women’s self-report and electronic reports of OC adherence [6]. Participants in this study were asked to keep a daily diary of their pill taking habits and their pill dispenser was equipped with an electronic device that recorded the date and time when a pill was removed. Overall, the diary and electronic records agreed on the number of active pills missed for approximately 45% of study participants, with women overreporting adherence on their diaries.
Other studies have examined the validity of adherence measures in the treatment of other medical conditions [3, 4, 21]. For example, Garber et al. conducted a summary of 86 previously published studies to evaluate the concordance of self-report measures and nonself-report measures. The authors determined that diary and questionnaire methods were moderately or highly concordant with nonself-report measures, suggesting that self-report measurement methods employing daily diaries and questionnaires provide reliable results in regards to medication adherence [21].
This study had several limitations. First, misclassification is possible. Participants may not have completed their diaries on a daily basis and simply recorded their adherence at the end of each week or just before sending them back. Furthermore, the act of completing diaries may have served as a cue to remember to take a pill and may have actually improved adherence in some women. In addition, despite receiving detailed oral and written directions on completing the diaries, some participants may not have understood the difference between an active and placebo pill. Since nearly 45% of participants indicated they missed 2 or more pills on their diaries we examined the diaries in detail to see if there were logical patterns that would coincide with a woman taking placebo pills. However, we did not see any patterns and in fact, many of the women with the most missed pills had also included when they had taken placebo pills. In this context, the apparently high proportion of women who missed 2 or more pills likely is explained in part by the decision to include 8 participants who had missing diaries only if the diaries indicated they had already missed 2 or more pills. When recalling information for the postcards, participants may have had difficulty in remembering how many days they missed pills. Conversely, it is possible that women were able to consult their last diary while completing the postcard and provide more accurate information. Despite efforts to ensure that this scenario did not occur, mailings did overlap by a few days for nearly 27% (n=30) of women included in this analysis. Of these women, nearly 87% had agreement between the two data collection methods.
Second, selection bias may have occurred. Women who completed all aspects of the study may differ from women who did not participate in all study follow-ups. We compared women who completed the diaries and postcard to women who completed only the baseline interview. While these two groups of women did not differ with respect to a number of characteristics including body mass index, age, marital status, parity, or income, they did differ with respect to race/ethnicity, educational level, and ever having problems with feeling sad while using OCs. Specifically, women who did not complete the diaries and postcard were more likely to be Hispanic, have a high school education or less, and to not have ever had problems with feeling sad while using OCs. Given that two of these variables were found to be strongly associated with inconsistent reporting of adherence, we conducted a sensitivity analysis to further explore the potential impact of selection bias on study results. While the conclusions remained largely unchanged over a broad range of reasonable assumptions regarding selection probabilities, when more conservative assumptions were considered the associations tended to increase in magnitude. Thus, the possibility for potential bias related to nonresponse cannot be ruled out.
This study also had a number of strengths. As noted, studies on the validity of self-reported OC adherence are sparse, with a focus on adolescents and college-aged women. This study considered adult women who were diverse with respect to race/ethnicity and socioeconomic status. However, generalizability may be limited due to the small sample size and high education level of the studied population. The Fertility and Oral Contraceptive Use Study was a feasibility study designed to investigate methodological issues associated with conducting a large, prospective study to examine the possible association between obesity and OC failure including issues related to adherence to an OC regimen. As such, we estimated that obtaining baseline information on approximately 200 women would allow us to explore these issues. However, further evaluation of these concerns in larger populations is warranted.
This research has implications for studies that need to collect information on OC adherence either as a main exposure, outcome, or covariate. If self-reported measures are used to obtain OC adherence data, there will likely be trade-offs. While prospective data collection via diaries may improve accuracy, the added expense and burden on study participants may not be necessary—indeed, these factors may impair study response and, thus, the validity of OC adherence data. For example, in the Fertility and Oral Contraceptive Use Study the response rate for the first diary was nearly 80% but dropped to 63% by the third diary whereas the response rate for the 3-month postcard was approximately 80%. Investigators have also noted that participant fatigue in completing diaries may impact the quality of data since participants may become less thorough in their reporting over time [22, 23].
In conclusion, the findings from this study suggest that retrospective recall of OC adherence is a method of collecting data on this variable that may be very similar to data collected via diary methods. However, the use of retrospective recall may not be appropriate for all study populations and may be most reliable among more highly educated women and those who, at baseline, report essentially positive experiences with OC use given that certain subgroups of women were found to inconsistently self-report OC adherence. Additional studies are needed to confirm these findings and to investigate whether other behavioral or demographic variables are associated with inconsistent reporting of OC adherence.
ACKNOWLEDGMENTS
This work was supported by the National Institutes of Health Grant 1R21HD056173-01A2. We thank the members of the FOCUS Research Team for their assistance with this project.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
REFERENCES
- 1.Mosher WD, Jones J. Use of contraception in the United States: 1982–2008. Vital Health Stat. 2010;23:1–44. [PubMed] [Google Scholar]
- 2.Hughey AB, Neustadt AB, Mistretta SQ, Tilmon SJ, Gilliam ML. Daily context matters: predictors of missed oral contraceptive pills among college and graduate students. Am J Obstet Gynecol. 2010;203 doi: 10.1016/j.ajog.2010.05.039. 323.e1-7. [DOI] [PubMed] [Google Scholar]
- 3.Shi L, Liu J, Fonseca V, Walker P, Kalsekar A, Pawaskar M. Correlation between adherence rates measured by MEMS and self-reported questionnaires: a meta-analysis. Health Qual Life Outcomes. 2010;8:99. doi: 10.1186/1477-7525-8-99. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Shi L, Liu J, Koleva Y, Fonseca V, Kalsekar A, Pawaskar M. Concordance of adherence measurement using self-reported adherence questionnaires and medication monitoring devices. Pharmacoeconomics. 2010;28:1097–1107. doi: 10.2165/11537400-000000000-00000. [DOI] [PubMed] [Google Scholar]
- 5.Peterson LS, Oakley D, Potter LS, Darroch JE. Women’s efforts to prevent pregnancy: consistency of oral contraceptive use. Fam Plann Perspect. 1998;30:19–23. [PubMed] [Google Scholar]
- 6.Potter L, Oakley D, de Leon-Wong E, Canamar R. Measuring compliance among oral contraceptive users. Fam Plann Perspect. 1996;28:154–158. [PubMed] [Google Scholar]
- 7.Hall KS, White KO, Reame N, Westhoff C. Studying the use of oral contraception: a review of measurement approaches. J Womens Health. 2010;19:2203–2210. doi: 10.1089/jwh.2010.1963. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Leigh BC. Alcohol consumption and sexual activity as reported with a diary technique. J Abnorm Psychol. 1993;102:490–493. doi: 10.1037//0021-843x.102.3.490. [DOI] [PubMed] [Google Scholar]
- 9.Graham CA, Catania JA, Brand R, Duong T, Canchola JA. Recalling sexual behavior: a methodological analysis of memory recall bias via interview using the diary as the gold standard. J Sex Res. 2003;40:325–332. doi: 10.1080/00224490209552198. [DOI] [PubMed] [Google Scholar]
- 10.Minnis AM, Padian NS. Reliability of adolescents' self-reported sexual behavior: a comparison of two diary methodologies. J Adolesc Health. 2001;28:394–403. doi: 10.1016/s1054-139x(00)00218-4. [DOI] [PubMed] [Google Scholar]
- 11.Harvey SM, Beckman LJ. Alcohol consumption, female sexual behavior and contraceptive use. J Stud Alcohol. 1986;47:327–332. doi: 10.15288/jsa.1986.47.327. [DOI] [PubMed] [Google Scholar]
- 12.Glass R, Johnson B, Vessey M. Accuracy of recall of histories of oral contraceptive use. Br J Prev Soc Med. 1974;28:273–275. doi: 10.1136/jech.28.4.273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Stolley PD, Tonascia JA, Sartwell PE, et al. Agreement rates between oral contraceptive users and prescribers in relation to drug use histories. Am J Epidemiol. 1978;107:226–235. doi: 10.1093/oxfordjournals.aje.a112529. [DOI] [PubMed] [Google Scholar]
- 14.Rosenberg MJ, Layde PM, Ory HW, Strauss LT, Rooks JB, Rubin GL. Agreement between women’s histories of oral contraceptive use and physician records. Int J Epidemiol. 1983;12:84–87. doi: 10.1093/ije/12.1.84. [DOI] [PubMed] [Google Scholar]
- 15.Coulter A, Vessey M, McPherson K, Crossley B. The ability of women to recall their oral contraceptive histories. Contraception. 1986;33:127–137. doi: 10.1016/0010-7824(86)90079-x. [DOI] [PubMed] [Google Scholar]
- 16.van Leeuwen FE, van Duijn CM, Camps MH, et al. Agreement between oral contraceptive users and prescribers: implications for case-control studies. Contraception. 1992;45 doi: 10.1016/0010-7824(92)90154-l. 399-08. [DOI] [PubMed] [Google Scholar]
- 17.Nischan P, Ebeling K, Thomas DB, Hirsch U. Comparison of recalled and validated oral contraceptive histories. Am J Epidemiol. 1993;138:697–703. doi: 10.1093/oxfordjournals.aje.a116907. [DOI] [PubMed] [Google Scholar]
- 18.Hunter DJ, Manson JE, Colditz GA, et al. Reproducibility of oral contraceptive histories and validity of hormone composition reported in a cohort of US women. Contraception. 1997;56:373–378. doi: 10.1016/s0010-7824(97)00172-8. [DOI] [PubMed] [Google Scholar]
- 19.Norell SE, Boethius G, Persson I. Oral contraceptive use: interview data versus pharmacy records. Int J Epidemiol. 1998;27:1033–1037. doi: 10.1093/ije/27.6.1033. [DOI] [PubMed] [Google Scholar]
- 20.Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85:257–268. [PubMed] [Google Scholar]
- 21.Garber MC, Nau DP, Erickson SR, Aikens JE, Lawrence JB. The concordance of self-report with other measures of medication adherence: a summary of the literature. Med Care. 2004;42:649–652. doi: 10.1097/01.mlr.0000129496.05898.02. [DOI] [PubMed] [Google Scholar]
- 22.Verbrugge LM. Health diaries. Medical Care. 1980;18:73–95. doi: 10.1097/00005650-198001000-00006. [DOI] [PubMed] [Google Scholar]
- 23.Wiseman V, Conteh L, Matovui F. Using diaries to collect data in resource-poor settings: questions on design and implementation. Health Policy Plan. 2005;20:394–404. doi: 10.1093/heapol/czi042. [DOI] [PubMed] [Google Scholar]