Abstract
Purpose
This study examined the congruence of self-reported medications with computerized pharmacy records.
Design and Methods
Pharmacy records and self-reported medications were obtained for 294 members of a state pharmaceutical assistance program who also participated in ACTIVE, a clinical trial on cognitive training in nondemented elderly persons. The average age of the sample participants was 74.5 years (range = 65–91); 87.8% were females.
Results
Congruence between self-report and pharmacy data was generally high. Self-reports omitted drug classes in the pharmacy records less often than the pharmacy records did not include self-reported drug classes. The percentage of individuals with perfect agreement between self-reports and pharmacy records varied from 49% for major drug classes to 81 % for specific cardiovascular and central nervous system drugs. Within a drug class, agreement tended to be higher for individuals without a prescription in that class. Poorer health was consistently related to poorer self-report of medications.
Implications
Self-reported medications are most likely to be congruent with pharmacy records for drugs prescribed for more serious conditions, for more specific classes of drugs, and for healthier individuals.
Keywords: Brown bag method, Drug use in elderly persons, Medication adherence or compliance
Knowing the number and types of medications that an individual takes can provide information about the individual’s mental and physical health status and comorbidities (Clark, Von Korff, Saunders, Baluch, & Simon, 1995) as well as indicate possible cognitive impairment caused by medication side effects (Gillin & Byerly, 1990). Obtaining such information is especially important in studies of older adults, because of their greater use of medications and greater likelihood of being treated for multiple conditions. Medication data are often collected in research studies by use of a self-report method known as the “brown bag” method (e.g., Jobe et al., 2001). With the brown bag method, participants are asked to collect containers of their current medications in a brown paper bag and provide them to research staff. Staff then record the prescription and nonprescription items. Alternatively, self-reports of medications have been collected by means of a written list, a verbal report, or a medication form (e.g., Gerbert, Stone, Stulbarg, Gullion, & Greenfield, 1988; Stuart & Grana, 1998).
Regardless of how they are obtained, the utility of self-reports of current medications depends on the willingness and ability of the individual to volunteer such information. A few studies have explored the validity of patient reports of current medications. Gurwich (1983) found that self-reported medication histories were more complete than the physician’s medical chart. In a sample of older adults, Opdycke, Ascione, Shimp, Boyd, and Malloch (1994) found an average of 81% agreement for medication name between self-reported medication histories and the prescriptions listed in the patients’ medical records. In contrast, Gerbert and colleagues (1988) concluded that patient reports of medication regimens were unreliable relative to chart audits, physician interviews, and videotaped observation of doctor visits.
Several influences on the agreement of self-reported medications with other sources have been identified. Self-reports tend to be better for medications taken on a long-term basis or for serious health conditions (Hulka, Kupper, Cassel, Efird, & Burdette, 1975; Kelly, Rosenberg, Kaufman, & Shapiro, 1990) and for individuals who were younger and had better functional status (Landry et al., 1988). Hulka, Cassel, Kupper, and Burdette (1976) also found that greater scheduling complexity was related to a greater number of omissions by physicians of medications in the self-reports.
The comparison of self-reported medications with more “objective” measures warrants the consideration of several methodological issues. Computerized pharmacy records only provide information about which prescriptions have been filled, not which medications are actually being taken (Choo et al., 1999; Christensen et al., 1997). Complex dosing schedules can also make it difficult to determine the correct days’ supply of a prescription (Christensen et al., 1997). Finally, many previous studies on the congruence of “self” and “other” reports of medications have either used a sample in which most or all participants were prescribed at least one medication or have focused on a limited set of condition-specific medications (e.g., Choo et al., 1999; Christensen et al., 1997). A study of over 3,000 rural elderly persons (Helling et al., 1987) found that 29% reported taking no prescription drugs, whereas other individuals were reported up to 13 prescribed medications. Clearly, the statistical methods used in prior studies to assess congruence have to be adapted to accommodate both older adults taking no medications and those taking a wide variety of medications. Moreover, when a broader range of medications is included, decisions must be made about which classes of drugs should be examined in greater depth and about the level of therapeutic classification that should be used.
The present study assessed the congruence between the brown bag method of obtaining self-reported medication data and the prescriptions that had been filled by a sample of older adults, according to computerized pharmacy records. Self-report medication data were collected by use of the brown-bag method as part of a multisite clinical trial on cognitive training, the Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE; Jobe et al., 2001). Most participants at the Pennsylvania State University (PSU) site of ACTIVE were recruited from a state pharmaceutical assistance program, the Pharmaceutical Assistance Contract for the Elderly (PACE). As a result, pharmacy claim data were available for those ACTIVE participants at the PSU site who were members of PACE. This allowed us to compare the self-report medication data collected by means of the brown bag method with prescription data in the PACE pharmacy records.
This study addressed four research questions in a sample of low-income older adults who were prescribed a variety of medications. First, what proportion of a set of drug classes had congruent information in the self-reports and pharmacy records? Second, are discrepancies more likely to be attributed to omissions by the self-reports or by the pharmacy records? Third, what individual characteristics predict congruence (or discrepancies) between the self-reported medications and the pharmacy records? Finally, does congruence between self-reports and the pharmacy data vary by drug class? These four questions were examined for 10 major therapeutic drug classes prescribed to this sample of older adults as well as for specific drug classes within 2 of these major drug classes (i.e., cardiovascular drugs and central nervous system, or CNS, agents). Cardiovascular drugs and CNS agents were examined in greater detail because they are frequently prescribed to elderly persons and because their possible cognitive side effects were of particular interest in a clinical trial on cognitive training.
Methods
Sample
The analysis sample for this study was 294 members of PACE who completed the ACTIVE group baseline assessment; the included participants had also been randomized to a treatment group in either the pilot study or the main study for ACTIVE. The average age of the sample was 74.5 years (SD = 5.7 years; range = 65–91). The sample was predominantly White (96%) and had a mean income of $11,335 (SD = $2,817; range = 3,027–19,024). Women comprised the majority of the sample (87.8%), and only 17% of the total sample was currently married. Eighteen participants in the analysis sample (6%) were enrolled in an extension of the PACE benefits program that entailed a slightly higher copayment and a $500 yearly deductible, for individuals with slightly higher, but still low, incomes (less than $17,000 for single persons; less than $20,200 for married couples).
Medication Data
Two types of medication data were obtained. Self-reported medication data were collected during the group baseline testing session for ACTIVE by use of the brown bag method. Computerized pharmacy claims for prescription fills and refills were obtained from PACE for participants for a window of time that included the date of the brown bag data collection. Each prescription medication in the PACE data had a code to indicate its therapeutic purpose; the American Hospital Formulary Service (AHFS) codes were used. Each medication had three levels of AHFS codes, depending on the level of therapeutic specificity. The first level of the AHFS code indicated the major therapeutic drug class (e.g., cardiovascular drugs), and lower levels Indicated minor or more specific therapeutic drug classifications (e.g., beta blockers). The brown bag medications were coded with the same AHFS coding system. For both the PACE pharmacy data and the self-report data, a participant was given a 0 or 1 for each AHFS drug class to indicate whether the participant had at least one medication in that class (i.e., score of 1) or no medication In that class (i.e., score of 0). Over-the-counter items (e.g., Tylenol, Aleve, and Mylanta) were excluded from the brown bag data. Medications in the PACE pharmacy data whose supply was estimated to be depleted more than 5 days before the brown bag assessment were also excluded. Sample medications may have been included in the brown bag assessment, but they were not specifically recorded as samples.
Seven of the 17 major AHFS classes with at least one prescribed medication were excluded from further analysis, because the class was prescribed to or self-reported by less than 5% of the sample. As shown in Table 1, the percentage of the sample with at least one medication in a given class was relatively low for most of the 10 major therapeutic drug classes examined. The three most prevalent major therapeutic drug classes in both data sets were as follows: (a) cardiovascular drugs (PACE, 59%; self-report, 67%), (b) hormones and synthetic substitutes (PACE, 32%; self-report, 37%), and (c) CNS agents (PACE, 28%; self-report, 34%); these classes have been found to be among the most commonly prescribed drugs for elderly persons in prior studies (Johnson & Moore, 1988; Opdycke et al., 1994). In the self-report data, the electrolytic, caloric, and water balance class was reported as frequently as CNS agents.
Table 1.
Prevalence of Examined Drag Classes in the Pharmacy Records and Self-Report Data (n = 294)
| Therapeutic Class (AHFS Code) | PACE | Self-Report |
|---|---|---|
| Major drug classes | ||
| Antihistamine drugs (04) | 3.7 | 6.1 |
| Anti-infective agents (08) | 5.8 | 3.4 |
| Autonomic drugs (12) | 9.2 | 12.9 |
| Blood formation and coagulation (20) | 8.8 | 11.6 |
| Cardiovascular drugs (24) | 59.2 | 66.7 |
| CNS agents (28) | 27.6 | 33.7 |
| Electrolytic, caloric, and water balance (40) | 22.4 | 33.7 |
| EENT preparations (52) | 7.8 | 11.2 |
| Gastrointestinal drugs (56) | 18.4 | 24.8 |
| Hormones and synthetic substitutes (68) | 32.3 | 36.7 |
| Specific cardiovascular drug classes | ||
| ACE inhibitors (240404) | 13.3 | 15.6 |
| Cardiac glycosides (240408) | 6.1 | 9.5 |
| Beta blockers (240416) | 15.3 | 20.1 |
| Calcium channel blockers (240420) | 25.9 | 28.2 |
| Antilipemic drugs (2406) | 21.1 | 22.8 |
| Hypotensive agents (2408) | 4.4 | 6.1 |
| Vasodilating agents (2412) | 7.5 | 12.2 |
| Specific CNS drug classes | ||
| NSAIDs (280804) | 8.8 | 13.6 |
| Opiate agonists (280808) | 3.7 | 3.7 |
| Benzodiazepines (282408) | 8.8 | 12.9 |
Notes: PACE = the Pharmaceutical Assistance Contract for the Elderly (here it pertains to pharmacy records); AHFS = American Hospital Formulary Service; CNS = Central nervous system; EENT = eye, ear, nose, and throat; NSAIDs = nonsteroidal anti-inflammatory drugs.
Cardiovascular drugs and CNS agents were examined in further detail in this study because of their possible side effects of cognitive impairment or confusion, which was of particular interest because the sample was participating in a study on cognitive training. The prevalence of the 7 specific classes of cardiovascular drugs and 3 specific classes of CNS agents included in this analysis are also presented in Table 1. The number of major drug classes present for each individual (in the 10 classes studied) in each database ranged from 0 to 7 (SD = 1.5 drug classes). For the 10 major drug classes studied, the pharmacy records showed that 20% of the sample had no current prescriptions at the time of the brown bag data collection, whereas only 10% of the sample self-reported no current prescriptions.
Measures
Everyday Problems Test
The Everyday Problems Test (EPT) assessed problem solving and reasoning in everyday situations encountered by older adults. Participants were presented with 14 everyday stimulus materials (e.g., medication labels, transportation schedules, and a Medicare benefit chart) and used the information in the materials to solve two problems per stimulus. The stimulus and problems represented seven instrumental activities of daily living (IADL) domains. The test was untimed. The EPT has been shown in prior research to have high internal consistency, test–retest reliability, and structural stability (Willis & Marsiske, 1993). The score was the number of correct answers (score range: 0–28).
Mini-Mental State Examination
Performance on the Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975) served as an indicator of general mental status. All ACTIVE participants scored greater than 22 on the MMSE as a condition of their inclusion in the study. Thus, all participants had MMSE scores between 23 and 30.
Short Form-36 General Health Score
The Short Form-36 (SF-36) is a self-report instrument of health and the impact of health on functioning and quality of life (Ware & Sherbourne, 1992). The general health subscale score tapped perceptions about current health and health changes. General health scores ranged from 15 to 100 in this sample, with an average of 65.24 (SD = 19.19). Two participants did not complete this scale.
Visual Acuity
An in-person visual acuity test was performed prior to baseline testing (Rubin & Salive, 1995). Participants who self-reported extreme difficulty with reading ordinary newspaper print or with a performance-tested vision score worse than 20/50 were excluded from participation in ACTIVE.
Measures of Congruence
Several measures of congruence between PACE pharmacy records and the self-report data were calculated in this study. Each measure addresses a somewhat different question regarding congruence. These measures are described in the paragraphs that follow, and scores for three hypothetical cases are presented in Table 2. Two measures of congruence (agreement and omission–commission) provided information at the level of the individual and thus permitted examination of predictors of congruence at the individual level. The remaining two measures (percent agreement and kappa coefficient) provided information about congruence at the aggregate level for a given drug class.
Table 2.
Three Sample Patterns of Reported Medications and Associated Agreement and Error Scores
| Person | Data | Drug Class Number
|
Score
|
|||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 04 | 08 | 12 | 20 | 24 | 28 | 40 | 52 | 56 | 68 | Agreement | Omission | Commission | ||
| 1 | Self-Report | N | N | Y | N | Y | N | Y | N | Y | N | 0.50 | 0.67 | 0.75 |
| PACE | N | Y | N | N | Y | N | N | N | N | Y | ||||
| 2 | Self-Report | N | N | N | N | Y | Y | N | Y | Y | Y | 0.80 | 0.00 | 0.40 |
| PACE | N | N | N | N | N | Y | N | N | Y | Y | ||||
| 3= | Self-Report | N | N | N | N | N | N | N | N | N | N | 1.00 | 0.00 | 0.00 |
| PACE | N | N | N | N | N | N | N | N | N | N | ||||
Notes: In the analyses, N was coded as 0 and Y was coded as 1. PACE = the Pharmaceutical Assistance Contract for the Elderly.
Agreement Score
This score provided the proportion of information that was identical in the PACE pharmacy records and the self-report data (Opdycke et al., 1994). Scores ranged from 0 (no agreement) to 1 (total agreement) and were calculated with the following formula:
Because we were interested in the pattern of congruence for either the absence or the presence of medications within a defined set of drug classes, the denominator in this equation (i.e., the number of drug classes compared) was a constant value for all individuals. This practice differed from previous studies in which the denominator varied for each person, depending on the total number of drug classes reported in either data source. For example, for all of the hypothetical cases shown in Table 2, our study used a denominator of 10 (i.e., the total number of drug classes compared), as opposed to denominator values of 6, 5, and 0, respectively, which would have represented only the (different) number of drug classes present in the data for that person. This practice also allowed agreement scores to be calculated for those with no current medications (e.g., the third drug pattern shown in Table 2).
One benefit of the agreement score is that the same overall measure of the agreement could be used for each person, despite differences in the medications being taken by individuals. Another benefit is that, unlike the percent agreement and the kappa coefficient described in the paragraphs that follow, an agreement score was generated for each person rather than as a sample statistic. However, because the agreement score does not define one data source as more accurate (i.e., a “gold standard”), it does not indicate which source is the cause of any disagreement between the two databases. Rather, the source of discrepancies is addressed with an analysis of omission and commission scores, described next.
Omission and Commission Error Scores
These scores provided information about the source of reporting errors that may be responsible for disagreements between the self-reports and the pharmacy records. Scores were calculated on a per person basis, with values of 0 indicating no error, that is, none of the drugs in one source were omitted by the other source, and a value of 1 representing 100% error, that is, all of the drugs in one source were omitted by the other source (Hulka et al., 1975, 1976; Opdycke et al., 1994). The three types of matches-mismatches between the two data sources that are considered in this method are described in Table 3. Because we were considering congruence in a set of drug classes that were relevant to this sample, individuals with no prescription drugs were given scores of 0 to avoid removing them from the analysis (e.g., see the third drug pattern in Table 2).
Table 3.
Omission-Commission Analysis
| Notation | Error Score | Description |
|---|---|---|
| A | Number of classes that were self-reported and were in the pharmacy records | |
| B | Number of classes the person did not report that were in the pharmacy records | |
| C | Number of classes the person reported that were not in the pharmacy records | |
| B/(A + B) | Omission | Proportion of error attributed to the self-report |
| C/(A + C) | Commission | Proportion of error attributed to the pharmacy records |
| (B + C)/(A + B + C) | Combined | Combined errors of omission-commission |
Percent Agreement
We calculated an overall percent agreement as the percentage of the total sample whose self-report data were congruent with the PACE data. For a particular drug class, this value included congruence either for the presence or the absence of a prescription within that drug class. This type of measure was used in a previous evaluation of self-reported medications with the PACE population (Landry et al., 1988). We also computed two additional, more specific percent agreement values. First, for cases that had a prescription In a given drug class in the PACE data, the proportion of individuals who also reported that drug class in the brown bag data was calculated as the “percent agreement for cases with a prescription,” Second, for cases that did not have a prescription in a given drug class in the PACE data, the proportion of individuals who also did not report that drug class in the brown bag data was calculated as the “percent agreement for cases without a prescription.”
Kappa Coefficient
The kappa coefficient is a measure of agreement between the classifications made by two independent data sources and takes into account the agreement expected by chance (Gerbert et al., 1988; Wickens, 1989). Values range from 0 to 1; higher values indicate greater agreement. The classifications examined from the two data sources were whether or not a drug class had been reported (yes or no) by that source for a particular individual.
Results
Four main research questions were investigated in this study. First, the proportion of a set of drug classes that were congruent for each individual between the self-report data and the pharmacy records was examined; agreement scores were used for this analysis. Second, the source of any discrepancies between self-reports and pharmacy records was investigated with an analysis of omission and commission error scores. Third, predictors of the agreement and discrepancies between the self-report data and the pharmacy data were considered. Finally, the congruence (using percent agreement and kappa) between the self-report data and the pharmacy records was explored within each of 10 major drug classes and additionally for 10 specific drug classes (i.e., 7 specific cardiovascular drug classes and 3 CNS drug classes).
Congruence Between Self-Report Data and Pharmacy Records: Across Drug Class
In this section, we examined the proportion of a set of drug classes with congruent information between the two databases. We calculated three agreement scores for each individual, comparing the following sets of drug classes separately: (a) 10 major drug classes, (b) 7 specific cardiovascular drug classes, and (c) 3 specific CNS drug classes.
Major Drug Classes
An average of 91% agreement (i.e., 9 of 10 classes) between the self-report and pharmacy records was found for the 10 major drug classes included (SD = 11.3%). Almost half (49%) of the sample had perfect (100%) agreement between their self-report data and the pharmacy records. Although agreement scores ranged from 50% to 100%, indicating that only half of the 10 classes were congruent for some individuals, 91.8% of the sample had congruent information for at least 8 of the 10 classes.
Specific Cardiovascular Drug Classes
An average of 96.4% agreement (i.e., 6.7 of 7 classes) was observed for the 7 specific cardiovascular drug classes (SD = 9.0%). Agreement scores ranged from 43% to 100%, with 81% of the sample having perfect (100%) agreement between the two databases. Ninety-seven percent of the sample had congruent information for 6 of the 7 specific cardiovascular drug classes.
Specific CNS Drug Classes
An average of 92.7% agreement (i.e., 2.8 of 3 classes) was found for the 3 specific CNS classes examined (SD = 16.1%). Agreement scores ranged from 33% to 100%; 81% of the sample had perfect agreement (100%) between the self-report and pharmacy data, and 97% had congruent information for at least 2 of the 3 specific CNS classes.
Source of Discrepancies Between Self-Reports and Pharmacy Records
To determine the source of the discrepancies between the self-reported medications and the pharmacy records, we next calculated an omission error score, a commission error score, and a combined error score for each individual in three domains: (a) 10 major drug classes, (b) 7 specific cardiovascular drug classes, and (c) 3 specific CNS drug classes.
Major Drug Classes
On average, study participants did not report 7% of the major drug classes contained in their pharmacy records. The pharmacy records did not include an average of 24% of the major drug classes that were self-reported. The average combined error rate was 28%, with the majority of the discrepancies resulting when information about current prescriptions taken from the pharmacy records did not include drug classes that the individual had self-reported. However, all classes found in the pharmacy records were self-reported by 84% of the sample, and pharmacy records had information about all classes in the self-report data for 58% of the sample.
Specific Cardiovascular Drug Classes
The average rate of omissions was 1% for self-reports of medications in the seven specific cardiovascular drug classes; only 5 individuals did not report all cardiovascular drug classes present for them in the pharmacy records. Although an average of 13% of self-reported specific cardiovascular drug classes was not included in the pharmacy records, pharmacy records had information on all self-reported cardiovascular drugs (i.e., 0% error rate) for 82% of the sample. The average combined error rate for cardiovascular drug classes was 13.5%, with most discrepancies occurring when the information drawn from the pharmacy records did not include a cardiovascular drug class that had been self-reported.
Specific CNS Drug Classes
Self-reports of medications in three specific CNS drug classes omitted an average of 5% of the specific CNS drug classes found in the pharmacy records, whereas pharmacy records reported an average of 13% fewer CNS classes than the self-report data. However, the percentages of the sample with no errors of omission (94%) and with no errors of commission (87%) were quite high. The average combined error rate for CNS drug classes was 17%, with the majority of the discrepancies occurring when the information drawn from the pharmacy records did not include a class that had been self-reported.
Predictors of Agreement-Disagreement Between the Self-Report and Pharmacy Data
Linear regression analyses examined predictors of three person-level outcomes: (a) agreement score (i.e., congruence between self-reports and pharmacy data for a set of drug classes), (b) omission error score (i.e., self-reports omitted a drug class included in the pharmacy records), and (c) commission error score (i.e., pharmacy records omitted a drug class included in the self-reports). Analyses were performed for these outcome variables in three domains: (a) 10 major drug classes, (b) 7 specific cardiovascular drug classes, and (c) 3 specific CNS drug classes. Predictors included demographic variables (gender, marital status, age, and income) as well as measures of cognitive and physical functioning. Cognitive variables included the EPT and the MMSE; physical functioning variables were the SF-36 General Health subscale and a vision score. Results of the regression analyses are presented in Table 4.
Table 4.
Regression Analysis Summary for Participant Characteristics Predicting Agreement and Omission-Commission Error Scores (n = 292)
| Predictor | Agreement
|
Omission Errors
|
Commission Errors
|
||||||
|---|---|---|---|---|---|---|---|---|---|
| Major Drug Classes β | Specific CV Classes β | Specific CNS Classes β | Major Drug Classes β | Specific CV Classes β| | Specific CNS Classes β | Major Drug Classes β | Specific CV Classes β | Specific CNS Classes β | |
| Gender | −.08 | .01 | −.14* | −.02 | .07 | .11 | .03 | .00 | .09 |
| Marital status | .19** | .04 | .04 | −.16* | .04 | −.01 | −.16* | −.03 | −.01 |
| Income ($1,000s) | −.16* | −.08 | .05 | .13 | .05 | −.01 | .11 | .04 | −.05 |
| Age | −.03 | −.02 | −.03 | .00 | .02 | −.05 | .08 | .06 | .05 |
| MMSE | .04 | −.01 | .01 | −.05 | .01 | −.01 | −.05 | −.01 | −.01 |
| EPT | −.09 | −.02 | .01 | −.07 | −.04 | −.23*** | .10 | .00 | .12 |
| General health | .28*** | .17** | .19** | −.12* | −.07 | −.10 | −.09 | −.13* | −.15* |
| Vision | −.06 | −.03 | .04 | −.01 | .00 | −.05 | .10 | .03 | −.06 |
| R2 | 0.12*** | 0.04 | 0.07** | 0.05 | 0.01 | 0.08** | 0.05 | 0.02 | 0.05* |
Notes: Gender was coded 0 = male, 1 = female, and marital status was coded 0 = not married, 1 = married. Here n = 292 for the regression analyses because two participants did not complete the SF-36 measure of general health. CV = cardiovascular; CNS = central nervous system; MMSE = Mini-Mental State Examination; EPT = Everyday Problems Test.
p < .05;
p < .01;
p < .001.
Major Drug Classes
For the 10 major drug classes, marital status, income, and general health significantly predicted agreement between the two medication databases. Participants who were married, had lower income, and better general health were more likely to have self-report data that agreed with the pharmacy records. Marital status and general health were also significant predictors of errors of omission. Participants who were married and in better health were less likely to omit drug classes included in the pharmacy records. Marital status was the only significant predictor of errors of commission. Married participants were less likely to report a drug class that was not included in the pharmacy records.
Cardiovascular Drug Classes
General health was the only significant predictor of agreement (p < .01) and errors of commission (p < .05) for the seven specific cardiovascular classes. Individuals with better health had better agreement between their self-report data and the PACE pharmacy records for cardiovascular drug classes and were less likely to report a drug class that was not found in the pharmacy records. No significant predictors of errors of omission were found.
CNS Drug Classes
Gender (p < .05) and general health (p < .01) were significant predictors of the amount of agreement for the three CNS drug classes. Male participants and those in better health had higher levels of agreement for CNS drugs. The EPT was the only significant predictor of errors of omission for CNS drug classes (p < .001). Individuals with better everyday problem-solving skills were less likely to omit CNS drug classes included in the pharmacy records. General health was the only significant predictor of errors of commission for CNS drug classes (p < .05). Having poorer general health was related to a greater proportion of self-reported drug classes that were not being represented in information taken from the pharmacy records.
Congruence Between Self-Report Data and Pharmacy Records: Within Drug Class
In this section of results, we examined congruence separately within (a) each of the 10 major drug classes, (b) each of the 7 specific cardiovascular drug classes, and (c) each of the 3 specific CNS drug classes. Congruence (i.e., percent agreement) was investigated in three ways.-First, what proportion of the total sample has congruent information (i.e., both PACE and the self-report data were coded 0 or both were coded 1)? Second, of cases with a prescription in a drug class in the pharmacy records (i.e., PACE = 1), what proportion also self-report that drug class? Third, of cases without a prescription in a drag class in the pharmacy records (i.e., PACE = 0), what proportion also did not self-report that drug class? These results are presented in Table 5.
Table 5.
Percent Agreement and Kappa Coefficients Within Each Drug Class (n = 294)
| Therapeutic Class (AHFS Code) | Percent Agreement
|
κ (95% CI) | ||
|---|---|---|---|---|
| All Cases | Cases with PACE Rx | Cases with no PACE Rx | ||
| Major drug classes | ||||
| Antihistamine drugs (04) | 94.9 | 63.6 | 96.1 | .46 (.23, .69) |
| Anti-infective agents (08) | 94.2 | 29.4 | 98.2 | .34 (.11, .58) |
| Autonomic drugs (12) | 94.2 | 88.9 | 94.8 | .71 (.58, .84) |
| Blood formation and coagulation (20) | 96.6 | 96.2 | 96.6 | .82 (.70, .93) |
| Cardiovascular drugs (24) | 91.8 | 99.4 | 80.8 | .83 (.76, .89) |
| CNS agents (28) | 83.0 | 80.3 | 84.0 | .60 (.50, .70) |
| Electrolytic, caloric, and water balance (40) | 87.4 | 97.0 | 84.7 | .69 (.60, .78) |
| EENT preparations (52) | 91.8 | 69.6 | 93.7 | .54 (.38, .71) |
| Gastrointestinal drugs (56) | 88.8 | 87.0 | 89.2 | .67 (57, .77) |
| Hormones and synthetic substitutes (68) | 90.8 | 92.6 | 90.0 | .80 (.73, .87) |
| Specific cardiovascular drug classes | ||||
| ACE inhibitors (240404) | 97.6 | 100.0 | 97.3 | .90 (.83, .97) |
| Cardiac glycosides (240408) | 96.6 | 100.0 | 96.4 | .77 (.63, .90) |
| Beta blockers (240416) | 95.2 | 100.0 | 94.4 | .84 (.76, .92) |
| Calcium channel blockers (240420) | 96.3 | 97.4 | 95.9 | .91 (.85, .96) |
| Antilipemic drugs (2406) | 95.6 | 93.6 | 96.1 | .87 (.80, .94) |
| Hypotensive agents (2408) | 98.3 | 100.0 | 98.2 | .83 (.68, .98) |
| Vasodilating agents (2412) | 95.2 | 100.0 | 94.9 | .73 (.60, .87) |
| Specific CNS drug classes | ||||
| NSAIDs (280804) | 92.5 | 84.6 | 93.3 | .63 (.49, .77) |
| Opiate agonists (280808) | 93.9 | 18.2 | 96.8 | .15 (−.07, .37) |
| Benzodiazepines (282408) | 91.8 | 76.9 | 93.3 | .58 (.43, .73) |
Notes: PACE = the Pharmaceutical Assistance Contract for the Elderly (here it pertains to pharmacy records); AHFS = American Hospital Formulary Service; CNS = central nervous system; EENT = eye, ear, nose, and throat; NSAIDs = nonsteroidal antiinflammatory drugs; CI = confidence interval.
Major Drug Classes
As shown in Table 5, the overall percent agreement (i.e., for the presence or absence of a particular drug class) was high for all 10 major therapeutic drug classes studied, ranging from 83.0% to 96.6%. For cases in which a prescription for a drug class was found in the pharmacy records, the percent agreement was also relatively high (63.6–99.4%), except for anti-infective agents (29.4%). For cases in which no prescriptions for a drug class were found in the pharmacy records, the agreement between the self-report and pharmacy data was generally high (80.8–98.2%). The kappa coefficients for all 10 classes were significant at a 95% level of confidence, indicating significantly more agreement between the two databases than would be expected by chance.
Cardiovascular Drug Classes
Table 5 shows that the overall percent agreement values for the seven specific cardiovascular classes were all high (95.2–983%). The percent agreement was also high both for cases with a medication in the specific class and for cases without a medication in that class in the pharmacy records. As shown in Table 5, all percent agreement values for cardiovascular classes were over 93%, and several had 100% agreement. The kappa coefficients for all of the specific cardiovascular classes showed significant agreement (p < .05) between the self-reported medications and the pharmacy records.
CNS Drug Classes
Table 5 shows that the overall percent agreement values for the three specific CNS drug classes were high (91.8–93.9%). Agreement was also high for cases without a medication in the specific class in the pharmacy records (93.3–96.8%). The percent agreement was lower for cases with a medication in these classes in the pharmacy records. For nonsteroidal anti-inflammatory drugs (NSAlDs) and benzodiazepines, the values were good, at 84.6% and 76.9%, respectively. However, only 18.2% of those with a prescription for opiate agonists in the PACE data self-reported that drag class. It must be noted, however, that the prevalence of opiate agonists was very low in the total sample (3.7%, or 11 participants), and thus this discrepancy actually represented only 9 participants who did not self-report an opiate agonist but had a prescription for this type of drug in the pharmacy data. The kappa coefficients for all specific CNS classes, except opiate agonists, showed significant agreement (p < .05) between the self-reported medications and the pharmacy records.
Discussion
This study investigated the congruence between self-reports of medications obtained by use of the brown bag method and information about current medications taken from pharmacy records (i.e., PACE data) for a sample of low-income, rural elderly persons. This sample was of particular interest given their high rate of health problems, limitations in access to health care, and often limited education. The results indicated generally high levels of agreement across 10 major drug classes and across the specific cardiovascular and CNS drug classes examined. We also found that the brown bag data omitted drug classes found in the pharmacy records less often than the pharmacy records did not include information about drug classes that had been self-reported. However, it is important to note that the pharmacy records included all self-reported major drug classes for more than half the sample and included all self-reported cardiovascular and CNS drugs for over 80% of the sample. An analysis of predictors of agreement and discrepancies between the self-report and pharmacy data indicated that general health status was an important factor to consider. Better health was related to higher levels of agreement across the 10 major drug classes, across the 7 specific cardiovascular drug classes, and across the 3 specific CNS drug classes. Better health also predicted that fewer major drug classes would be omitted by the self-report data and that fewer specific cardiovascular and CNS drug classes would be self-reported that were not found in the pharmacy records.
Finally, within each drug class, high congruence was found for the total sample and for the subsample of individuals who did not have a prescription filled for a particular drug class. For individuals with a prescription medication in a drug class, very high agreement was found for reporting specific cardiovascular drug classes, confirming previous research findings that drugs for serious conditions are more likely to be reported (Hulka et al., 1975; Kelly et al., 1990). Medications taken for less serious conditions or taken on an intermittent or short-term basis are less likely to be reported (Kelly et al, 1990). In this study, the percent agreement for antihistamines, anti-infective agents, eye, ear, nose, and throat preparations, and opiate agonists was lower than the percent agreement found within the other major and specific CNS drug classes for those who were prescribed a medication in one of those classes.
The congruence of pharmacy records and self-reported medication use is an important issue because self-reports of medications are frequently used as a proxy for health status or the presence of chronic diseases as well as to study topics such as medication compliance, polypharmacy, and drug interactions. Self-reports of medications are often used in population-based studies (e.g., the Iowa 65+ Rural Health Study), where pharmacy records are lacking or expensive to obtain. Although self-report measures are widely used, few studies have examined the congruence of prescription records and self-report measures, and the samples available for such studies have often been limited and unique.
One typically validates self-report data by comparing them to an objective, gold standard measure. However, to have a gold standard measure of current prescriptions, individuals would need to obtain all medications through a single source, and all medications received (including samples) would have to be included in the database. In the current medical climate, where individuals are likely to have multiple sources for prescription coverage and multiple physicians, obtaining a gold standard measure of prescription medications is not likely. Thus, our analysis has emphasized the comparison of two sources of information about current prescription medications, rather than the absolute accuracy of either source. For example, although we utilized a method termed “error analysis” to investigate differences between the two databases, we discussed the “errors” as discrepancies or disagreements between two potentially incomplete sources of information about the same domain.
We believe the current study makes several contributions to the prior literature with respect to the measurement of the congruence of self-report and pharmacy records. First, congruence was examined both at the level of the individual and at the aggregate level, across drug categories and within both major and specific drug categories. Measurement of congruence at the individual level led to a second contribution of the study—the examination of person characteristics as predictors of congruence between self-reports and prescription records. Information on predictors could be useful in studies that only use self-report measures by indicating subsamples whose self-report might be expected to differ from pharmacy records. A final contribution of the study was the examination of congruence not only for individuals in the sample who take medications, but also for the significant proportion of community-dwelling, young-old elderly who take no medications. Although prior studies have often focused on participants who take medications, it is well known that, in community-dwelling populations, a proportion of elderly persons take no medications. In the current study, pharmacy records indicated that 20% of the sample had no current prescriptions, whereas the self-report data indicated 10% with no current prescriptions. However, a comprehensive study of congruence requires examining congruence not only for participants who take medications but also for those who do not.
This study had several limitations. First, the sample was relatively small and of low socioeconomic status and consisted of primarily women. Second, all participants participated in a prescription coverage program, which may increase the likelihood that medical conditions would be treated with prescription medications (Stuart & Grana, 1998). Third, congruence was examined at the level of the drug class rather than at the level of the actual medication. It is possible that some discrepancies were not detected with our analysis. For example, multiple medications within a drug class would have been coded as a “yes” for that class, although only one of these medications may have been self-reported. Finally, using computerized pharmacy records as a proxy for current medications can be problematic (Choo et al., 1999; Christensen et al., 1997).
Specifically in this study, an individual’s current medications in the PACE pharmacy data were based on the prescribed days’ supply and the date of the prescription fill or refill. Individuals who did not use their medications as often as directed by their physician may account for some of the discrepancies observed between the self-report data and the pharmacy records. For example, if elderly persons take a lower dosage than prescribed and extend their supply of the drug, medications might be included in the brown bag beyond their prescribed dosing window. Such noncompliance may explain why 10% of the sample had no current pharmacy records for prescription refills but reported taking one or more current medications. Days’ supply can also be difficult to determine for medications taken on an “as needed” basis, such as opiate agonists. The use of a class-specific window may have improved congruence for such classes. The elderly individuals may also be obtaining medications that are not recorded in the prescription records. A survey of PACE recipients found that approximately 29% have other prescription coverage (Pennsylvania Department of Aging, 2002). In addition, health care professionals often provide patients, particularly low-income patients, with medication samples, suggesting that they try the medication before a prescription is written. Or, the elderly persons may use medications prescribed for friends or family members (Johnson & Moore, 1988). Medications not obtained through PACE would not be recorded in the pharmacy claims data and could have falsely inflated the discrepancies between the PACE pharmacy data and the self-reports.
In summary, the findings of this study have several implications for the use of the brown bag method to collect medication information. We believe the brown bag method provides a reasonable substitute for pharmacy records as a measure of current medications. One must consider several factors, however, when using the brown bag method. First, congruence of brown bag data and pharmacy records may vary by the type and prevalence of the medication being considered. Medications for more serious conditions or that are taken on a long-term basis are more likely to be reported than medications taken on an as needed basis, for a short duration, or for less serious conditions. Furthermore, medications taken in forms other than by mouth (e.g., by transcutaneous patch) may be less likely to be reported with the brown bag method. Second, whether a general drug class (e.g., CNS agents), a more specific drug class (e.g., opiate agonists), or a set of drug classes is being considered may influence congruence. Although congruence was generally high, the best agreement tended to be found within a specific drug class rather than within a major drug class or for the overall pattern of all drug classes for an individual. Third, the congruence of self-reports for prescribed medications was better for more prevalent classes (e.g., cardiovascular drugs). Finally, the investigation of predictors of agreement between the brown bag data and the pharmacy data revealed that those individuals with worse health consistently had poorer congruence. Future research on the congruence of brown bag data with pharmacy records should include an investigation of reasons why drugs are omitted from both self-reports and pharmacy records as well as an examination of the congruence and stability of brown bag data over time.
Acknowledgments
Funding for the ACTIVE study was provided by the National Institute on Aging and the National Institute of Nursing Research (Grant AG14263). We are grateful for the assistance of the Penn State project staff, including Rosie Ammerman, Jeannie McKenzie, and Pam Davis, We thank the Pharmaceutical Assistance Contract for the Elderly (PACE) program in the Pennsylvania Department of Aging for their support in providing the pharmacy data for this study. Grace I. L. Caskie was funded by grants from the National Institute on Aging to K. W. Schaie and S. L. Willis (Grants R37 AG08055 and AG 14263). A prior version of this article was presented at the 2001 Annual Meeting of the Gerontological Society of America, Chicago, IL.
References
- Choo PW, Rand CS, Inui TS, Lee MLT, Cain E, Cordeiro-Breault M, et al. Validation of patient reports, automated pharmacy records, and pill counts with electronic monitoring of adherence to antihypertensive therapy. Medical Care. 1999;37:846–857. doi: 10.1097/00005650-199909000-00002. [DOI] [PubMed] [Google Scholar]
- Christensen DB, Williams B, Goldberg HI, Martin DP, Engelberg R, LoGerfo JP. Assessing compliance to antihypertensive medications using computer-based pharmacy records. Medical Care. 1997;35:1164–1170. doi: 10.1097/00005650-199711000-00008. [DOI] [PubMed] [Google Scholar]
- Clark DO, Von Korff M, Saunders K, Baluch WM, Simon GE. A chronic disease score with empirically derived weights. Medical Care. 1995;33:783–795. doi: 10.1097/00005650-199508000-00004. [DOI] [PubMed] [Google Scholar]
- Folstein MF, Folstein SE, McHugh PR. Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
- Gerbert B, Stone G, Stulbarg M, Gullion DS, Greenfield S. Agreement among physician assessment methods: Searching for the truth among fallible methods. Medical Care. 1988;26:519–535. doi: 10.1097/00005650-198806000-00001. [DOI] [PubMed] [Google Scholar]
- Gillin JC, Byerley WF. The diagnosis and management of insomnia. New England Journal of Medicine. 1990;322:239–248. doi: 10.1056/NEJM199001253220406. [DOI] [PubMed] [Google Scholar]
- Gurwich EL. Comparison of medication histories acquired by pharmacists and physicians. American Journal of Hospital Pharmacy. 1983;40:1541–1542. [PubMed] [Google Scholar]
- Helling DK, Lemke JH, Semla TP, Wallace RB, Lipson DP, Cornoni-Huntley J. Medication use characteristics in the elderly: The Iowa 65+ Rural Health Study. Journal of the American Geriatrics Society. 1987;35:4–12. doi: 10.1111/j.1532-5415.1987.tb01312.x. [DOI] [PubMed] [Google Scholar]
- Hulka BS, Cassel JC, Kupper LL, Burdette JA. Communication, compliance, and concordance between physicians and patients with prescribed medications. American Journal of Public Health. 1976;66:847–853. doi: 10.2105/ajph.66.9.847. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hulka BS, Kupper LL, Cassel JC, Efird RL, Burdette JA. Medication use and misuse: Physician-patient discrepancies. Journal of Chronic Diseases. 1975;28:7–21. doi: 10.1016/0021-9681(75)90045-4. [DOI] [PubMed] [Google Scholar]
- Jobe JB, Smith DM, Ball K, Tennstedt SL, Marsiske M, Willis SL, et al. ACTIVE: A cognitive intervention trial to promote independence in older adults. Controlled Clinical Trials. 2001;22:453–479. doi: 10.1016/s0197-2456(01)00139-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson JE, Moore J. The drug-taking practices of the elderly. Applied Nursing Research. 1988;1:128–131. doi: 10.1016/s0897-1897(88)80023-5. [DOI] [PubMed] [Google Scholar]
- Kelly JP, Rosenberg L, Kaufman DW, Shapiro S. Reliability of personal interview data in a hospital-based case-control study. American Journal of Epidemiology. 1990;131:79–90. doi: 10.1093/oxfordjournals.aje.a115488. [DOI] [PubMed] [Google Scholar]
- Landry JA, Smyer MA, Tubman JG, Lago DJ, Roberts J, Simonson W. Validation of two methods of data collections of self-reported medicine use among the elderly. The Gerontologist. 1988;28:672–676. doi: 10.1093/geront/28.5.672. [DOI] [PubMed] [Google Scholar]
- Opdycke RAC, Ascione FJ, Shimp LA, Boyd EL, Malloch CK. Comparison of pharmacist-obtained comprehensive medication histories and medical records in geriatric patients. Journal of Geriatric Drug Therapy. 1994;9:19–37. [Google Scholar]
- Pennsylvania Department of Aging. PACE—Annual report to the Pennsylvania General Assembly; January 1-December 31, 2001; Harrisburg, PA: Author; 2002. Apr, [Google Scholar]
- Rubin G, Salive M. Vision and hearing. In: Guralnik J, Fried L, Kasper JD, Simonsick E, Lafferty M, editors. The women’s health and aging study; Health and social characteristics of older women with disability. Bethesda, MD: National Institute on Aging; 1995. pp. 152–161. [Google Scholar]
- Stuart B, Grana J. Ability to pay and the decision to medicate. Medical Care. 1998;37:202–211. doi: 10.1097/00005650-199802000-00009. [DOI] [PubMed] [Google Scholar]
- Ware JE, Jr, Sherbourne CD. The MOS 36-item short form health survey (SF-36). I. Conceptual framework and item selection. Medical Care. 1992;30:473–483. [PubMed] [Google Scholar]
- Wickens TD. Multiway contingency tables analysis for the social sciences. Hillsdale, NJ: Erlbaum; 1989. [Google Scholar]
- Willis SL, Marsiske M. Manual for the Everyday Problems Test. University Park, PA: Pennsylvania State University; 1993. [Google Scholar]