Abstract
Objective
To evaluate the validity of self-reported lipid medication use in an epidemiological study.
Methods
We studied medication self-reports compared with inventoried lipid medication containers at the fifth visit of the Atherosclerosis Risk in Communities Study (ARIC) in 2011–2013 (n=6,370). To assess the validity of self-reports, we computed sensitivity, specificity, positive and negative predictive values. We used multiple logistic regression to determine whether validity varied by participant characteristics. Comparisons were made with visit 4 (n=11,531), to determine if there was a change in validity as the pattern and types of lipid medication used changed over time.
Results
The prevalence of lipid medication use, according to medication containers was higher at visit 5 (56%) than visit 4 (14.3%). Statins were increasingly used. The percentage of participants reporting use/ non-use accurately was 91.8% at visit 5, lower than visit 4 (97.3%). The unadjusted kappa coefficient of agreement was 0.83 (95% CI – 0.82 to 0.85) at visit 5 and 0.89 (95% CI – 0.88 to 0.90) at visit 4. Agreement was higher, compared with their counterparts, for women, younger and more educated participants, and those using fewer total medications.
Conclusion
In this population sample, self-reported lipid medication use was highly accurate and therefore likely would be for similar epidemiological studies or clinical settings collecting this information.
Keywords: Lipid medication, Validation study, Medication use, Statins
1. Introduction
Epidemiological research studies or clinical settings often require people to self-report their medications. Understanding the validity and quality of self-reported medication data therefore is vital.
Some previous studies have documented the accuracy of recall of medication histories in clinical care settings. Hulka et al [1] reported that only about 27% of study participants with congestive heart failure were able to accurately recollect their therapeutic regimes. Another study showed high accuracy of recall of duration of use and names of most recent oral contraceptive drug regimens [2]. Parental recall of children’s inhaled corticosteroid use was found to be highly accurate [3]. Yet, only 68% of the drug use information collected at home from patients of a specialized geriatric outpatient care clinic agreed with the clinic data [4]. Of various medication types assessed, self-report of beta-blockers and centrally acting hypertensive medication use was more likely to disagree with clinic data than others [4]. Also, the presence of co-morbidities and use of multiple drug regimens in this older population decreased recall accuracy [4]. The accuracy of patient recall is reported to vary by clinical care setting, type of medication under consideration, patient condition and disease severity, doctor-patient relationship, demographic characteristics, past or current medication recall, and clinical factors [1–11].
Few studies have reported the accuracy of cardiovascular medication recall data. The Cardiovascular Health Study showed modest levels of agreement between directed recall of use of beta-blockers and beta-agonists compared with a medication inventory [12]. In addition to assessing reliability, authors also used physiologic markers of drug effect to assess validity and found that the medication inventory was superior to the directed recall [12]. Consistent with findings from similar studies on other medication types, in the Rotterdam Elderly Study [13], agreement between self-report and pharmacy records varied from poor to perfect by class of cardiovascular medication used. Commonly prescribed antihypertensive agents were self-reported with high accuracy (kappa = 0.90 to 0.96), while kappas of agreement for organoheparinoids and nitroglycerin were low (kappa = 0.26 – 0.53).
Only a few studies have assessed the accuracy of recall of lipid medication use. Researchers found a generally high sensitivity of recall of statin medication used in the past 6 months among older female controls (Sensitivity = 0.93, Specificity = 0.98) and cases with breast cancer (Sensitivity = 0.85, Specificity = 0.98) [14]. This recall accuracy decreased slightly among both cases and controls for 2-year and 8-year recalls [14]. The study also noted that although accuracy of self-report was generally high for statins, it was lower than for antihypertensive drugs. Cardiovascular epidemiological studies often ask about the use of lipid lowering medications, but the accuracy of self-reported anti-hyperlipidemic medication is not well documented.
Our study therefore measured the validity of patient self-report of lipid medication use in a population-based cohort study, the Atherosclerosis Risk in Communities (ARIC) Study. We also examined whether there was a change in validity as patterns of lipid medication use evolved between 1996–98 and 2011–13.
2. Methods
The ARIC study is a prospective cohort study comprised of 15,792 men and women in 4 communities: Forsyth County, NC, Jackson, MS, suburban Minneapolis, MN and Washington County, MD. At baseline, in 1987–1989, the participants were between 45 and 64 years and were selected either by list or area probability sampling [15]. ARIC performed four examinations of the cohort between 1987 and 1998, and conducted annual telephone interviews. The present study is mainly based on the fifth visit in 2011–2013, but with some data from the fourth visit in 1996–1998. Among 10,036 ARIC original cohort members deemed alive, 6,538 participated in visit 5 resulting in an overall response rate of 65% [16]. The visit 4 response rate was 80%. The institutional review board at each site approved the ARIC study protocol, and each participant supplied informed consent.
Study participants were instructed to bring to visit 5, the containers of all medications used in the past four weeks. If a participant forgot to bring any medication containers, he/she was telephoned later for this information. The medication containers were used to create a detailed medication inventory which is being used in this analysis as the gold standard to evaluate the validity of self-reports. The inventory was created by scanning barcodes on medication containers, with the scanning system directly linked to a medication database. When a barcode was missing, a medication was not identified by the database, or unmarked containers/loose pills were to be inventoried, the interviewer manually transcribed the medication using a predetermined protocol. Medication inventories have been found to be a reasonably accurate method for ascertaining cardiovascular medication use in the elderly, even though this method may generate higher estimates of prevalence of medication use as compared to serum levels [11, 16]. Self-report of lipid medication use was ascertained from the question, "Were any of the medications you took during the last four weeks for high blood cholesterol?” which was recorded as “Yes”, “No” or “Unknown”. We excluded participants whose medication container record was missing (n = 26); self-report of medication use was missing (n = 39) or ‘Unknown’ (n = 89); or both medication records and self-reported were missing (n = 14). Our final analytic sample for Visit 5 included 6,370 participants (97% of all visit 5 participants, Figure 1).
Figure 1.
Participant inclusion for analysis;: ARIC Visit 5;, 2011–2013 and ARIC Visit 4, 1996–1998.
Comparisons were made with medications reported and recorded at ARIC visit 4 to determine whether there was a change in validity as the pattern and types of lipid medication used changed over time. At visit 4 staff entered medication names by hand and they were linked to a medication database. The fourth visit analytic sample, after excluding observations with missing or unknown medication information (n=125), included 11,531 participants (99% of all visit 4 participants).
2.1. Statistical Analysis
We computed the sensitivity, specificity, positive predictive value and negative predictive value to determine the validity of the self-reported use of lipid medication as compared to the gold standard, the medication container records. Sensitivity represents the proportion of those who brought in a lipid medication container who reported use. Specificity represents the proportion who had no lipid medication container who reported no use. The positive predicative value represents the proportion who reported use who actually had a lipid medication container. The negative predictive value represents the proportion who reported not using a lipid medication who had no lipid medication container. We also calculated unadjusted, and prevalence- and bias-adjusted, kappa coefficients [17] (measures of agreement) and their respective 95% confidence intervals. To study factors related to accurate self-reporting, we created a dichotomous variable of agreement (Yes/No) between self-report and medication containers. Using multiple logistic regression, we examined whether accuracy varied by the number of medications used, age, sex, race, education, combined variable of prevalent cardiovascular disease (CVD) and CVD risk, and study center. The combined variable of prevalent CVD and CVD risk was created as follows. Study participants were first categorized based on the presence of prevalent CVD by ARIC criteria. Those without prevalent CVD were further categorized by CVD risk (>= 7.5% or < 7.5%) based on American College of Cardiology and American Heart Associations’ combined guidelines on the assessment of CVD risk [18]. The three resultant categories of this combined variable were: 1) Prevalent CVD, 2) No prevalent CVD, but CVD risk >= 7.5% and 3) No prevalent CVD, and CVD risk < 7.5%. All analyses were performed using SAS (version 9.3).
3. Results
The mean (range) of participant ages was 76.7 (66 to 90) years at visit 5 and 62.8 (52 to 75) at visit 4, and the percentages of women were 56% and 59%, respectively. As shown in Table 1, according to the medication containers, 61.6% of visit 5 participants currently used some lipid medication, as compared with only 14.6% at visit 4. Statins were the most commonly used type of lipid medication at both visits (85.0% and 75.7% at visits 4 and 5 respectively), followed by fibrates, ezetimibe, niacin, bile sequestrants and other types during visit 5. Other lipid medications included omega-3 fatty acids, etc. The relative use of statins increased and that of fibrates, niacin and bile sequestrants decreased from visit 4 to visit 5.
Table 1.
Distribution of lipid medications used by ARIC participants according to medication containers, ARIC visit 5 (2011–2013, n = 6,370) and visit 4 (1996–1998, n = 11,835).
| Medication Type | Visit 5 - N (%) | Visit 4 - N (%) |
|---|---|---|
| Total (Any lipid lowering medication) | 3921 (61.6a) | 1730 (14.6a) |
| Statin | 3331 (85.0) | 1309 (75.7) |
| Fibrate | 233 (5.9) | 201 (11.6) |
| Niacin | 127 (3.2) | 148 (8.6) |
| Bile Sequestrant | 53 (1.4) | 72 (4.2) |
| Ezetimibe | 201 (5.1) | - |
| Others | 33 (0.8) | - |
Abbreviation: ARIC, Atherosclerosis Risk in Communities.
% of total analytic sample for respective visits.
At visit 5, 91.8% of the participants accurately self-reported the use or non-use of lipid medication (Table 2). On the other hand, 97.3% of participants had correctly self-reported lipid medication use at visit 4, fifteen years earlier. At visit 5, the sensitivity and specificity of self-reported lipid medication use when compared with the medication containers was 0.91 and 0.93 respectively (Table 3). These measures of agreement were slightly higher at visit 4. The kappa coefficient of agreement between self-report and medication containers at visit 5 was 0.83 (95% CI – 0.82 to 0.85) and that for visit 4 was 0.89 (95% CI – 0.88 to 0.90), which did not change after prevalence and bias adjustment. Restriction of the visit 4 sample to only those who also came to visit 5 yielded similar estimates of percent agreements and kappa coefficients.
Table 2.
Frequency of self-report compared to actual use of lipid medication per medication containers, ARIC Visit 5 (2011–2013) and Visit 4 (1996–1998).
| Lipid Medication (Actual Use) | Self-report | |
|---|---|---|
| Yes | No | |
| ARIC Visit 5 | ||
| Yes | 3235 (50.8%) | 327 (5.1%) |
| No | 197 (3.1%) | 2611 (41.0%) |
| ARIC Visit 4 | ||
| Yes | 1515 (13.1%) | 140 (1.2%) |
| No | 170 (1.5%) | 9706 (84.2%) |
Abbreviation: ARIC, Atherosclerosis Risk in Communities.
Table 3.
Validity of self-report of lipid medication use, ARIC visit 5 (2011–2013) and visit 4 (1996–1998).
| Measure of validity | Visit 5 | Visit 4 |
|---|---|---|
| Sensitivity | 0.91 | 0.92 |
| Specificity | 0.93 | 0.98 |
| Positive Predictive Value | 0.94 | 0.90 |
| Negative Predictive Value | 0.89 | 0.99 |
| Kappa Coefficient (95% Confidence Interval) | 0.83 (0.82 – 0.85) | 0.89 (0.88 – 0.90) |
Abbreviation: ARIC, Atherosclerosis Risk in Communities.
The mean number of medications of any type recorded in the medication record was 8.9 in visit 5, compared with 5.2 in visit 4. The accuracy of recall of lipid meds was inversely associated with the number of medications taken. For visit 5, lipid medication kappas were 0.85, 0.84, 0.81, and 0.76 for participants who took 0–6, 7–8, 9–12, or 13+ total medications respectively. For visit 4, the kappa coefficients for the same total medication categories were 0.90, 0.89, 0.85, and 0.82.
As shown in Table 4, at visit 5, men were 39% less likely to accurately report lipid medication use as compared to women. The validity of self-report was lower among older adults than their younger counterparts: the third and fourth quartiles of age were both 26% less likely to accurately self-report that the first (youngest) quartile of age. Individuals with basic education were significantly less likely to accurately report lipid medication use compared to those with advanced education. The total number of medications used was inversely associated with accuracy of self-report. Individuals using 13 or more total medications were 41% less likely accurately report compared to those using 0–6 medications. Race and study center were not independently associated with accuracy of self-reported lipid medication use. A combined variable reflecting the participant’s predicted cardiovascular risk or the prevalence of cardiovascular disease was not statistically significantly associated with the accuracy of recall of lipid medication use (data not shown).
Table 4.
Adjusted odds ratios for agreement between self-report and medication containers by demographic characteristics, ARIC visit 5 (2011–2013).
| Variable | Odds Ratioa | 95% Confidence Interval |
|
|---|---|---|---|
| Sex | |||
| Women (n = 3745) | 1 (reference) | - | |
| Men (n = 2625) | 0.61c | 0.50–0.73 | |
| Age | |||
| 66 to 71 (n = 1654) | 1 (reference) | - | |
| 72 to 75 (n = 1749) | 0.95 | 0.73–1.25 | |
| 76 to 79 (n = 1380) | 0.74c | 0.57–0.97 | |
| > 79 (n = 1587) | 0.74c | 0.57–0.95 | |
| Raceb | |||
| White (n = 4861) | 1 (reference) | - | |
| African-American (n = 1491) | 0.64 | 0.36–1.51 | |
| Education level | |||
| Advanced (n = 2771) | 1 (reference) | - | |
| Intermediate (n = 2649) | 0.85 | 0.69–1.04 | |
| Basic (n = 939) | 0.70c | 0.54–0.92 | |
| Number of medications used | |||
| 0 to 6 (n = 2056) | 1 (reference) | - | |
| 7 to 8 (n = 1169) | 0.82 | 0.62–1.07 | |
| 9 to 12 (n = 1859) | 0.72c | 0.56–0.91 | |
| 13 or more (n = 1286) | 0.59c | 0.46–0.77 | |
| Center | |||
| Minneapolis, MN (n = 1882) | 1 (reference) | - | |
| Washington County, MD (n = 1716) | 0.98 | 0.75–1.27 | |
| Jackson, MS (n = 1367) | 0.89 | 0.47–1.67 | |
| Forsyth County, NC (n = 1405) | 0.86 | 0.65–1.12 | |
Abbreviation: ARIC, Atherosclerosis Risk in Communities.
Adjusted for all variables shown.
Asian and American Indian participants were excluded due to very small sample sizes.
Odds ratio significantly different from 1 at p<0.05.
4. Discussion
Clinical trials have documented that the use of lipid-lowering medication, particularly statins, reduces the risk of incident and recurrent cardiovascular disease [19]. Therefore, it is important for clinical and research purposes to document whether patients accurately report lipid medication use. To the best of our knowledge, this is the first general population study in the US to validate that participant recall of lipid medication use has a high level of accuracy. We also found that as the cohort aged and the prevalence of antihyperlipidemic medications, particularly statins, increased over the 15 years between ARIC visit 4 and visit 5, the validity of self-report went down only slightly. Rising use of statins is consistent with the fact that evidence-based guidelines have increasingly recommended the use of lipid medications for cardiovascular disease prevention over time [19].
Existing literature suggests that medication self-reports vary across populations, settings, and medication types, and likely are affected by doctor-patient communications, patient compliance, and individuals’ health conditions. We further showed that validity of self-reported lipid medication use varied by several demographic characteristics and validity was less for patients taking more medications. Thus, the decline in accurate reporting of lipid medication use from visit 4 (97.3%) to visit 5 (91.8%) may be due to greater medication use or poorer cognitive function at the latter visit. Yet, the validity of self-report of lipid medication, overall, seems relatively high for epidemiological research. Moreover, controlling for commonly available variables like gender and age in analyses that use self-reported measures of lipid medication use could help adjust for inaccurate reporting. Hence, findings of our study show patterns consistent with those of previously cited studies measuring the accuracy of recall of other cardiovascular medications among older adults [11–14]. Results of our analysis provide useful information for various epidemiological studies that depend on self-reported data to determine lipid medication use. In addition, our study is relevant to national studies like NHANES, which record lipid medication use through a similar questionnaire as the one used in ARIC [20].
Results from this study should be interpreted keeping its limitations in mind. While considerable effort was taken to ensure that participants produced all medication containers, some lipid medications could have been missed or miscoded. Although ARIC is community-based, it is not representative of the entire U.S., and the response rate to ARIC visit 5 was only 65% of those still alive. Those who took part tended to have fewer risk factors and health conditions than those who did not [21], and validity may be different for the non-responders. In addition, by visit 5, ARIC participants had considerable experience in a research study, which may have increased their accuracy of self-reported medications.
5. Conclusion
In conclusion, our study established that self-reported lipid medication use is highly accurate among ARIC participants and therefore likely would be for similar epidemiological studies collecting this information.
Highlights.
In a US population study, we assessed validity of self-reported lipid medication use.
92% of participants reported use/non-use accurately, compared with pill bottles.
Agreement was higher for women, younger, and more educated participants.
Agreement was also higher in those using fewer total medications.
Self-reported lipid medication use seems quite accurate.
Acknowledgements
The authors thank the staff and participants of the ARIC study for their important contributions over many years.
Role of the funding sources
The Atherosclerosis Risk in Communities (ARIC) Study was funded by the National Heart, Lung, and Blood Institute contracts HHSN268201100005C, HHSN268201100006C, HHSN268201100007C, HHSN268201100008C, HHSN268201100009C, HHSN268201100010C, HHSN268201100011C, and HHSN268201100012C.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Disclosure statement
The authors declare that there are no conflicts of interest.
Contributor Information
Sahiti Bhaskara, Email: bhask007@umn.edu.
Eric A. Whitsel, Email: eric_whitsel@unc.edu.
Christie M. Ballantyne, Email: cmb@bcm.tmc.edu.
Aaron R. Folsom, Email: folso001@umn.edu.
References
- 1.Hulka BS, Kupper LL, Cassel JC, Efird RL. Medication use and misuse: Physician-patient discrepancies. J Chronic Dis. 1975;28(1):7–21. doi: 10.1016/0021-9681(75)90045-4. [DOI] [PubMed] [Google Scholar]
- 2.Glass R, Johnson B, Vessey M. Accuracy of recall of histories of oral contraceptive use. Br J Prev Soc Med. 1974;28(4):273–275. doi: 10.1136/jech.28.4.273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Koster ES, Wijga AH, Raaijmakers JA, Koppelman GH, Postma DS, Kerkhof M. High agreement between parental reported inhaled corticosteroid use and pharmacy prescription data. Pharmacoepidemiol Drug Saf. 2010;19(11):1199–1203. doi: 10.1002/pds.2015. [DOI] [PubMed] [Google Scholar]
- 4.Jackson JE, Ramsdell JW, Renvall M, Swart J, Ward H. Reliability of drug histories in a specialized geriatric outpatient clinic. J Gen Intern Med. 1989;4(1):39–43. doi: 10.1007/BF02596489. [DOI] [PubMed] [Google Scholar]
- 5.Brody DS. An analysis of patient recall of their therapeutic regimens. J Chronic Dis. 1980;33(1):57–63. doi: 10.1016/0021-9681(80)90086-7. [DOI] [PubMed] [Google Scholar]
- 6.Gordon LG, Patrao T, Hawkes AL. Can colorectal cancer survivors recall their medications and doctor visits reliably? BMC Health Serv Res. 2012;12:440. doi: 10.1186/1472-6963-12-440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Wesr SL, Savitz TA, Kok G, et al. Pharmacoepidemiol. West Sussex, UK: John Wiley and Sons; 2000. Validity of pharmacoepidemiologic drug and diagnosis data. [Google Scholar]
- 8.Curtis JR, Westfall AO, Allison J, Freeman A, Kovac SH, et al. Agreement and validity of pharmacy data versus self-report for use of osteoporosis medications among chronic glucocorticoid users. Pharmacoepidemiol Drug Saf. 2006;15:710–718. doi: 10.1002/pds.1226. [DOI] [PubMed] [Google Scholar]
- 9.Stolley PD, Tonascia JA, Sartwell PE, Tockman MS, Tonascia S, Rutledge A. Agreement rates between oral contraceptive users and prescribers in relation to drug use histories. Am J Epidemiol. 1978;107(3):226–235. doi: 10.1093/oxfordjournals.aje.a112529. [DOI] [PubMed] [Google Scholar]
- 10.Solomon DH, Stedman M, Licari A, Weinblatt ME, Maher N, Shadick N. Agreement between patient report and medical record review for medications used for rheumatoid arthritis: the accuracy of self-reported medication information in patient registries. Arthritis Rheum. 2007;57(2):234–239. doi: 10.1002/art.22549. [DOI] [PubMed] [Google Scholar]
- 11.Psaty BM, Lee M, Savage PJ, Rutan GH, German PS, Lyles M for the Cardiovascular Health Study Collaborative Research Group. Assessing the use of medications in the elderly: methods and initial experience in the Cardiovascular Health Study. J Clin Epidemiol. 1992;45(6):683–692. doi: 10.1016/0895-4356(92)90143-b. [DOI] [PubMed] [Google Scholar]
- 12.Landry JA, Smyer MA, Tubman JG, Lago DJ, Roberts J, Simonson W. Validation of two methods of data collection of self-reported medicine use among the elderly. Gerontologist. 1988;28(5):672–676. doi: 10.1093/geront/28.5.672. [DOI] [PubMed] [Google Scholar]
- 13.Sjahid SI, van der Linden PD, Stricker BH. Agreement between the pharmacy medication history and patient interview for cardiovascular drugs: the Rotterdam Elderly Study. Br J Clin Pharmacol. 1998;45(6):591–595. doi: 10.1046/j.1365-2125.1998.00716.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Boudreau DM, Daling JR, Malone KE, Gardner JS, Blough DK, Heckbert SR. A validation study of patient interview data and pharmacy records for antihypertensive, statin, and antidepressant medication use among older women. Am J Epidemiol. 2004;159(3):308–317. doi: 10.1093/aje/kwh038. [DOI] [PubMed] [Google Scholar]
- 15.The ARIC Investigators. The Atherosclerosis Risk in Communities (ARIC) Study: Design and objectives. Am J Epidemiol. 1989;129(4):687–702. [PubMed] [Google Scholar]
- 16.Smith NL, Psaty BM, Heckbert SR, Tracy RP, Cornell ES. The reliability of medication inventory methods compared to serum levels of cardiovascular drugs in the elderly. J Clin Epidemiol. 1999;52(2):143–146. doi: 10.1016/s0895-4356(98)00141-3. [DOI] [PubMed] [Google Scholar]
- 17.Cunningham M. More than just the kappa coefficient: a program to fully characterize inter-rater reliability between two raters. SAS Global Forum. 2009:242–2009. [Google Scholar]
- 18.Goff DC, Jr, Lloyd-Jones DM, Bennett G, et al. for the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guidelines on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25 PT B):2935–2959. doi: 10.1016/j.jacc.2013.11.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Stone NJ, Robinson JG, Lichtenstein AH, et al. for the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task force on Practice Guidelines. Circulation. 2014;129(25) Suppl 2:S1–S45. doi: 10.1161/01.cir.0000437738.63853.7a. [DOI] [PubMed] [Google Scholar]
- 20.Carroll MD, Lacher DA, Sorlie PD, Cleeman JI, Gordon DJ, Wolz M, Grundy SM, Johnson CL. Trends in serum lipids and lipoproteins of adults, 1960–2002. JAMA. 2005;294(14):1773–1781. doi: 10.1001/jama.294.14.1773. [DOI] [PubMed] [Google Scholar]
- 21.Description of ARIC Visit 5 NCS Stage 1. [accessed on 1st July, 2014]; https://www2.cscc.unc.edu/aric/Visit_5_NCS_Stage_1_Cohort_Description. [Google Scholar]