CONGRUENCE OF MEDICATION INFORMATION FROM A BROWN BAG DATA COLLECTION AND PHARMACY RECORDS: FINDINGS FROM THE SEATTLE LONGITUDINAL STUDY

Abstract

The validity of health information obtained through participants’ reports of current medications (e.g., the brown bag method) is an important, but under-studied, area. In the current study, we examined the congruence of medication reports from a brown bag data collection with the pharmacy prescription records for 1430 participants (ages 23 to 97 years) of the seventh wave of the Seattle Longitudinal Study. Overall, the congruence of the brown bag data and pharmacy records was high. Congruence was better for younger participants, healthier participants, and for medications taken for serious conditions or on a regular basis. When the focus is on assessing participants’ medications at a specific point in time (e.g., on the day of testing), brown bag data may provide more complete information than pharmacy records. Age and health status of the participants as well as the type of medications of interest should be considered when determining the validity of medication information reported by participants.

To obtain information about the health status and medical conditions of their study participants, researchers often collect participants’ reports of current medications. The self-report method for medications is commonly referred to as the “brown bag” method, because participants are typically given a brown paper bag as the means for them to bring their current medications back to the testing site for testers to record the prescription and nonprescription items (e.g., Bosworth & Schaie, 1997; Jobe et al., 2001). However, self-reports of current medications have also been collected over the telephone (Landry et al., 1998), as a verbal report (Gerbert, Stone, Stulbarg, Gullion, & Greenfield, 1988), during in-home interviews (Hulka, Kupper, Cassel, Efird, & Burdette, 1975), and with medication forms or written lists (Kelly, Rosenberg, Kaufman, & Shapiro, 1990; Landry et al., 1998; Opdycke, Ascione, Shimp, Boyd, & Malloch, 1994; Stewart, Moore, Marks, May, & Hale, 1993; Stuart & Grana, 1998).

Little is known about the validity of self-reports of current medications. For example, are particular types of drugs systematically over-or under-reported? What is the overall congruence for a particular set of drug classes? A few studies have investigated the validity of self-reported medications, but they have typically involved samples that were somewhat restricted. For example, all individuals in the sample may have been diagnosed with a particular medical condition, such as chronic pulmonary obstructive disease (Gerbert et al., 1988), hypertension (Choo et al., 1999; Christensen et al., 1997), congestive heart failure, or diabetes mellitus (Hulka, Cassel, Kupper, & Burdette, 1976; Hulka et al., 1975). Or, all participants were taking at least one current medication (e.g., Choo et al., 1999; Opdycke et al., 1994). Other studies have samples that included only older adults (Caskie & Willis, 2004; Landry et al., 1988; Opdycke et al., 1994).

The limitations of the samples in these studies may make it difficult for researchers to generalize the findings to their own samples, unless they share the same selection criteria. In particular, this issue may arise when one’s sample covers a wide age range. Young adults who may be included in wide age range samples are less likely to have been diagnosed with the medical conditions included in the previous studies or even to be taking any medications at all. Additionally, it is well known that in community-dwelling populations, a proportion of elderly take no medications (e.g., Caskie & Willis, 2004; Helling et al., 1987; Landry et al., 1988).

Congruence typically has not been examined in samples that include these individuals because previous studies (e.g., Ascione, Kirscht, & Shimp, 1986; Gerbert et al., 1988; Hulka et al., 1975; Opdycke et al., 1994) have defined agreement as the presence of the same drug in two sources for an individual. Specifically, the agreement score represented a ratio of the number of drugs reported in both data sources being compared (e.g., brown bag data and pharmacy data) to the total number of drugs for an individual, counting all that were reported by either source. By definition, this operationalization required that at least one drug be present for each individual in the analysis, either in the brown bag data or in the pharmacy data. The analysis of discrepancies (i.e., errors of omission and commission) conducted in these studies was also dependent on individuals having at least one drug present in one of the data sources being compared. Necessarily, individuals who were not taking medications were excluded from the congruence analysis. In many of these studies, including only individuals who were taking medications was appropriate, due to their focus on medication misuse or other medical purposes. However, for researchers whose focus is on reports of specific categories of drugs, such as those with cognitive side effects or those used to treat cardiovascular disease, it may be just as important to know the validity of reports that participants are not taking a particular type of drug as it is to know the validity of reports that participants are taking a particular type of drug. To address this issue, in a previous paper (i.e., Caskie & Willis, 2004), we adapted the traditional methods for assessing the congruence of self-reported medications and pharmacy records to include participants without any medications and to measure agreement in terms of both the absence and presence of a drug class for a pre-defined set of drug classes for all individuals.

However, the sample in Caskie and Willis (2004) was limited in terms of age (range = 65–91 years) and had a relatively small sample size (N = 294). The study reported here extends the results of Caskie and Willis by using a larger sample (N = 1430) that covers a wider age range (range = 23–97 years). The current study also includes an explicit comparison of congruence analyses performed with the “traditional” method (i.e., excluding anyone with 0 medications, examining only drug classes present for an individual, and defining agreement only in terms of the presence of a drug class) and congruence analyses calculated with the method used in Caskie and Willis (i.e., including all participants, examining all selected drug classes for an individual, and defining agreement as congruent information regarding the absence or the presence of a drug class). The self-report medication data in this study were collected using the brown bag method as part of the Seattle Longitudinal Study (SLS), an ongoing longitudinal study of adult cognitive development (Schaie, 1996). Because the majority of the SLS sample was recruited through their membership in a health maintenance organization (HMO), pharmacy prescription data were available for all SLS participants currently enrolled in the HMO at the time of the brown bag data collection.

Thus, this study had three main research questions. First, how congruent are brown bag medications and pharmacy prescription records, and are discrepancies attributed to the brown bag data or the pharmacy records? Our analysis compared results obtained including all participants with results obtained for the subsample who had one or more medications to demonstrate differences between the two methods. Second, what individual characteristics in each age group predict the degree of agreement and discrepancy between the brown bag data and the pharmacy prescription records? Third, does congruence vary by drug class? Sixteen drug classes were compared for the total sample and within age group.

METHOD

Sample

Participants for this study (N = 1430) were a subset of the 1846 individuals tested during the seventh wave (1997–1999) of the Seattle Longitudinal Study (SLS). Inclusion of a participant into the analysis sample was dependent on meeting two criteria: (1) brown bag medication data were collected on the participant during the seventh wave of SLS testing and (2) pharmacy records were available from Group Health Cooperative (GHC) of Puget Sound for the time that the participant’s brown bag data were collected. Approximately 89% (n = 1636) of the individuals tested during the seventh wave of the SLS participated in the brown bag data collection. Of the 1636 brown bag participants, GHC pharmacy data were available for 1430 participants (77% of seventh wave participants).

The analysis sample was generally well-educated (M = 15.4 years, SD = 2.7 years, range = 7–20 years), and almost all participants (94%) reported their ethnicity as white. Women comprised 55% of the analysis sample. The average age of the sample was 62.8 years (SD = 15.1 years, range = 23–97 years). Median income was $49,666, with 19% of the sample earning over $80,000. Sixty-six percent of the participants were married.

The sample was divided into four age groups: (1) < 50 years old (n = 315; range = 23–49 years), (2) 50–65 years old (n = 423), (3) 66–75 years old (n = 348), and (4) >75 years old (n = 344; range = 76–97 years). The upper bounds of the three older age groups correspond to the second quartile (65), third quartile (75), and maximum value (97) of the age distribution. The first quartile was 51 years; however, the upper bound of the first age group was defined as 49 years to form more meaningful groups for interpretation. The age distributions in the youngest and the oldest age groups were slightly skewed, with 75% of participants in the <50-year-old group being between the ages of 37 to 49 and 75% of participants in the >75-year-old group being between the ages of 76 to 84. Participants who were under age 50 and those who were 50 to 65 years old did not differ significantly in education level; however, both groups had a significantly higher education level than the two older groups (p < .001), with those aged 66 to 75 years also having a significantly higher education level than those over 75 years of age (p < .01). The percentages of men and women did not differ significantly across the four age groups (χ²(3) = 1.35, p > .05).

The analysis sample of 1430 participants did not differ significantly from the full sample of 1,846 participants tested in the seventh wave of SLS in terms of education, income, self-rated health status, chronic disease status, sex, or ethnicity. The youngest age group (<50 years) was less represented in the analysis sample (p < .05).

Procedure

Brown Bag Data Collection

The brown bag data were collected during seventh-wave SLS testing sessions. At the first testing session, participants were given a brown bag and instructions that asked them to bring their prescription drugs to the next testing session (held within a week of the first session). Participants were also instructed to bring only current medications that were ordered by a doctor and received from a druggist. The following information was recorded for each medication: drug name, dosage, instructions, auxiliary labels, purpose, and length of use. Testers also noted if a participant reported taking no medications or did not want to participate.

Pharmacy Database

Computerized pharmacy prescription records from GHC were obtained for participants for the four months prior to the participant’s date of testing. Almost all (>93%) of the medications were dispensed in amounts prescribed for 90 days or fewer: 44.9% had a 30-day supply, 9.4% had a 60-day supply, and 18.3% had a 90-day supply. Using the days supply information and the date that the prescription was dispensed, we were able to estimate the end-date for all prescriptions. Prescriptions estimated to be current on the date of the brown bag assessment (i.e., the estimated end-date was on or after the date of the brown bag assessment) and those with supplies that were estimated to have been depleted less than a week prior to the brown bag assessment were included in the analysis. Medications in the anti-infective class were included if they were current or estimated to be depleted within 2 days prior to the brown bag assessment. The shorter window for anti-infective medications was used because this class had a much shorter modal prescription length (i.e., 10 days) than the other drug classes.

Medication Data Coding

The therapeutic purpose for each prescription medication in the GHC pharmacy data was assigned a code in the system developed by GHC (Group Health Cooperative of Puget Sound, 2002). Each brown bag medication was also coded using the same GHC coding system. For both the pharmacy and brown bag data, a participant was assigned a 0 or 1 for each drug class to indicate that the participant had at least one medication in that class (i.e., score of 1) or that the participant had no medications in that class (i.e., score of 0).

Drug Class Selection

We first examined which GHC drug classes were sufficiently represented in the databases. Of the 40 possible GHC major drug classes (Group Health Cooperative of Puget Sound, 2002), 36 were prescribed to at least one participant. Of these 36 major drug classes, the current analysis excluded 22 major drug classes due to low prevalence and/or difficulty estimating a correct days supply for these medications. Thus, 14 major drug classes were included. However, because the therapeutic classifications of the GHC major drug classes varied widely in terms of their breadth, we next examined whether specific drug classes within each major class should be examined rather than the major drug class. In 7 of the 14 major drug classes included, one or more specific classifications within each broad major class were selected for the congruence analysis because: (1) the specific class accounted for a large part of the major class (e.g., angiotensin-converting enzyme [ACE] inhibitors accounted for 74% of the antihypertensive major class; antidepressants accounted for 77% of the psychotherapeutic major class), or (2) the major class included multiple drug types that were of greater interest when examined separately (e.g., the hormones class included both estrogens and progesterones). The remaining 7 classes were examined at the most general level of therapeutic classification (i.e., at the major drug class level).

Thus, a total of 16 GHC classes (codes) were selected as the focus for the congruence analysis. Seven were classified as major drug classes in the GHC coding system: anti-infectives (20), antilipemics (26), calcium channel blockers (35), cardiac agents (38), diabetic agents (46), diuretics (50), and gastrointestinal agents (56). Nine were classified as specific therapeutic classes in the GHC coding system: salicylates (020201), ACE inhibitors (180901), glucocorticoids (220201), nonsteroidal anti-inflammatory drugs (220401), beta blockers (322201), estrogens (602001), progesterones (603001), antidepressants (7057), and thyroid replacement agents (741001).

Predictor Domains and Measures

Three domains of predictor variables were included: chronic disease status, cognitive, and demographic. All measures were obtained during the seventh wave of the Seattle Longitudinal Study in 1997 to 1999.

Chronic Disease Status Measure

The values used to estimate participants’ health were provided to the seattle longitudinal study by the health maintenance organization (i.e., GHC). The chronic disease status of each participant in this study was indicated by the total health costs predicted by GHC for the next year for that individual. Scores were calculated by GHC using a set of empirically derived weights based on age, gender, and pharmacy utilization for chronic conditions (Clark, Von Korff, Saunders, Baluch, & Simon, 1995). These scores were rescaled for this analysis in terms of $1000 units.

Cognitive Measures

Four cognitive factor scores (Schaie, Dutta, & Willis, 1991) were included as predictors in the cognitive domain. Each factor is described below; details about the individual tests that comprise the factor scores can be found in Schaie (1996).

The inductive reasoning factor measured the ability to recognize and understand novel concepts or relationships and was a composite of four tests: PMA Reasoning (Thurstone & Thurstone, 1949), Letter Series (Blieszner, Willis, & Baltes, 1981), Word Series (Gonda, Quayhagen, & Schaie, 1981; Schaie, 1985), and Number Series (Thurstone, 1962). The perceptual speed factor is a measure of speed of processing, specifically the ability to find figures, make comparisons, and carry out other simple tasks involving visual perception with speed and accuracy. The three tests included on this factor are from the ETS kit of factor-referenced tests (Ekstrom, French, Harman & Derman, 1976): Identical Pictures, Finding A’s, and Number Comparison. Verbal comprehension, the ability to understand ideas expressed in words, was measured by two additional tests from the ETS kit of factor-referenced tests (ETS Vocabulary V-2 and ETS Vocabulary V-4) as well as the PMA Verbal Meaning test (Thurstone & Thurstone, 1949). The verbal memory ability factor measured the ability to memorize and recall a list of words and was a composite of Immediate Recall (Zelinski, Gilewski, & Schaie, 1993) and Delayed Recall (Zelinski et al., 1993).

Demographic Measures

Individual characteristics of age, education, gender, marital status, and ethnicity were obtained with the demographic portion of the Life Complexity Inventory (Gribbin, Schaie, & Parham, 1980).

Congruence Analysis Measures

Several measures of the congruence between the pharmacy data and the brown bag data were calculated: agreement score, omission score, commission score, percent agreement, sensitivity, specificity, and the kappa coefficient. The agreement score measured the degree of identical information provided by two sources for a set of drug classes. Neither data source was considered inherently more accurate (i.e., a “gold standard”). The omission and commission scores examined the source of any discrepancies between the two databases for the same set of drug classes. These first three scores were calculated as person-level variables and thus permitted examination of predictors of congruence for a set of drug classes at the individual level. In contrast, percent agreement, sensitivity, specificity, and the kappa coefficient provided information about congruence at the aggregate level (i.e., as a sample statistic) and were calculated for each drug class separately. Further detail about each of these congruence measures is provided in the following paragraphs.

Agreement Score

The agreement score was calculated as the number of congruent pairs divided by the number of potential pairs; possible scores ranged from 0 (no agreement) to 1 (perfect agreement). The agreement score was calculated in two ways to contrast the method used by Caskie and Willis (2004) with the method used previously in other congruence analyses. As shown in Table 1, the two methods of calculation differed in two ways: (1) how the components of the agreement score (i.e., the number of congruent pairs and the number of potential pairs) were operationalized and (2) whether individuals with no current medications were given a score that would allow them to be included in the analysis.

Table 1.

Comparison of the two methods of calculation for agreement, omission, and commission scores

(A) Notation
				BB data
				Yes	No
		GHC data	Yes	A	B
			No	C	D
(B) Formulae
	Method 1: All participants		Method 2: Only participants with medications
	Participants with medications	Participants without medications	Participants with medications	Participants without medications
Agreement score	$\frac{(A+D)}{(A+B+C+D)}$	$\frac{(A+D)}{(A+B+C+D)}=1$	$\frac{A}{(A+B+C)}$	Excluded
Omission score	$\frac{B}{(A+B)}$	0	$\frac{B}{(A+B)}$	Excluded
Commission score	$\frac{C}{(A+C)}$	0	$\frac{C}{(A+C)}$	Excluded

A = Number of drug classes that were in both the BB data and GHC data.

B = Number of drug classes in the GHC data that were not in the BB data.

C = Number of drug classes in the BB data that were not in the GHC data.

D = Number of drug classes that were not in either the BB data or the GHC data.

Note. BB = Medications reported in the brown bag data collection. GHC = Pharmacy prescription records from Group Health Cooperative of Puget Sound.

Specifically, for the first method of calculation of the agreement score (Caskie & Willis, 2004), two types of congruent pairs were included: (1) the presence of a drug class for an individual in both databases (i.e., “Yes” in the pharmacy data and “Yes” in the brown bag data) and (2) the absence of a drug class for an individual in both databases (i.e., “No” in the pharmacy data and “No” in the brown bag data). The number of potential pairs (i.e., the denominator of the agreement score) was the same for all individuals, based on the predefined set of 16 drug classes examined for each person. A score was calculated for each individual, regardless of medication status.

For the second method of calculation of the agreement score (Opdycke et al., 1994), a congruent pair was defined only in terms of the presence of a particular drug class in both databases (i.e., “Yes” in the pharmacy data and “Yes” in the brown bag data) for an individual. The number of potential pairs (i.e., the denominator of the agreement score) varied between individuals, because it is calculated as the total number of drug classes reported in either data source for an individual. Thus, by definition, this score can only be calculated for the subsample of participants with at least one medication in either the pharmacy data or the brown bag data.

Omission and Commission Scores

Omission scores and commission scores focus on the proportion of drug classes omitted by one data source but present in another data source. Omission scores are defined as the proportion of discrepancies attributed to the brown bag data, calculated as the ratio of the number of drug classes present in the pharmacy data that were not included in the brown bag data to the total number of drug classes present for that participant in the pharmacy data. Commission scores describe the proportion of discrepancies attributed to the pharmacy data, specifically calculated as the ratio of the number of drug classes present in the brown bag data that were not included in the pharmacy data to the total number of drug classes present in the brown bag data.

Two sets of omission and commission scores were calculated to contrast explicitly the method used by Caskie and Willis (2004) with the method used previously in other congruence analyses. As shown in Table 1, the scores differed only in their treatment of individuals with no current medications. Because the numerator and denominator values used in the calculation of these scores would be 0 for participants with no drugs in either data source, these individuals have mathematically undefined scores. Previous analyses of congruence (Hulka et al., 1975; Hulka et al., 1976; Opdycke et al., 1994) have simply excluded these individuals from the analysis. In contrast, Caskie and Willis gave these individuals with no current medications scores of 0 to indicate no omissions or commissions; thus, this method provided a value for all participants in the sample, rather than only the subsample with at least one current medication.

Percent Agreement

Percent agreement values were calculated for each drug class as the percentage of the sample with congruent pharmacy data (GHC) and brown bag data (BB) for a given drug class. Congruence was defined in terms of both the presence of the drug class (i.e., “Yes” in the pharmacy data and “Yes” in the brown bag data) and the absence of the drug class (i.e., “No” in the pharmacy data and “No” in the brown bag data).

Sensitivity and Specificity

Sensitivity and specificity are most commonly used to indicate the accuracy of a diagnostic test in relation to a gold standard (e.g., Fletcher, Fletcher, & Wagner, 1996). In this paper, we used sensitivity and specificity to indicate the accuracy of one source of medication information in relation to another source with regard to information about a particular drug class. Because we did not consider either source of medication information to be a true “gold standard,” sensitivity and specificity were calculated twice—once with the pharmacy data considered the “gold standard” and then with the brown bag data considered the “gold standard.” This practice allowed us to determine whether the accuracy of information varied for particular drug classes. Specifically, sensitivity can be interpreted here as the accuracy of the report (i.e., presence) of a drug class, and specificity can be interpreted as the accuracy of the non-report (i.e., absence) of a drug class.

Kappa Coefficient

The kappa coefficient is a measure of agreement between the classifications made by two independent data sources, which takes into account the amount of agreement expected by chance (Gerbert et al., 1988; Wickens, 1989). Values range from 0 to 1, with higher values indicating greater agreement. The classifications examined were whether or not a drug class had been reported (Yes/No) in each data source for each individual.

RESULTS

We first present descriptive information about the medication patterns of our sample, including the prevalence of the 16 selected drug classes and the percentage of participants without any medications. Then, the results for the three research questions are presented. First, the degree of agreement between the brown bag report and the pharmacy records and the source of any discrepancies between them were examined. Second, regression analyses identified individual characteristics that predicted the degree of agreement and discrepancies between the brown bag data and the pharmacy prescription records. Third, congruence was examined within each of the 16 selected drug classes.

Descriptive Information on Medications

Total Sample

As shown in Table 2, the prevalence of the 16 drug classes ranged from 4.3% to 17.4% in the total sample. In the brown bag data, diuretics were the most prevalent category (17.4%), followed by estrogens (15.2%) and thyroid replacement agents (11.8%). In the pharmacy data, diuretics were also the most prevalent category (15.1%), followed by gastrointestinal agents (11.4%) and estrogens (10.7%). As shown in Table 3, the number of drug classes present for each individual (of the 16 classes selected for the analysis) ranged from 0 to 8 in the brown bag data and from 0 to 9 in the pharmacy data. On average, individuals reported 1.50 classes (SD = 1.53) whereas the pharmacy data contained an average of 1.30 classes (SD = 1.47) for individuals. Twenty-nine percent of the total sample (n = 415) had no medications in either database for the 16 classes studied; percentages were higher when each database was considered separately.

Table 2.

Prevalence of drug classes in the pharmacy data and brown bag data

	Total sample		<50 years		50–65 years		66–75 years		>75 years
	(n = 1430)		(n = 315)		(n = 423)		(n = 348)		(n = 344)
Therapeutic class	BB	GHC	BB	GHC	BB	GHC	BB	GHC	BB	GHC
ACE inhibitors	10.1	8.4	2.2	2.2	10.2	9.2	13.8	10.3	11.0	13.7
Antidepressants	10.3	9.2	12.4	10.2	10.4	9.2	9.2	7.8	9.3	9.9
Anti-infectives	5.8	5.6	5.1	4.8	4.0	4.3	4.9	4.9	9.6	8.7
Antilipemics	8.2	6.8	2.9	2.2	6.4	6.1	12.9	9.5	10.5	9.0
Beta-andrenergic blockers	11.3	10.6	3.2	2.5	7.1	7.6	16.7	15.8	18.6	16.3
Calcium channel blockers	6.7	5.7	1.0	0.6	5.0	4.5	7.8	5.7	13.1	11.6
Cardiac agents	6.2	4.6	0.6	0.3	1.7	0.7	7.8	4.6	15.4	13.4
Diabetic agents	5.2	4.3	1.9	1.6	6.1	5.7	6.9	4.9	5.5	4.7
Diuretics	17.4	15.1	4.1	3.2	13.9	12.3	23.0	19.8	28.2	24.7
Estrogens	15.2	10.7	3.2	1.9	23.4	17.7	16.4	9.8	14.8	11.0
Gastrointestinal agents	11.2	11.4	7.0	7.0	8.0	8.3	14.4	13.8	15.7	16.9
Glucocorticoids	8.8	7.1	10.8	8.3	7.6	6.4	9.2	6.9	8.1	7.3
NSAIDs	8.3	6.4	4.1	5.7	8.5	5.9	11.8	8.3	8.1	5.8
Progesterones	6.6	8.0	2.5	2.5	9.9	11.6	7.8	10.1	4.9	6.7
Salicylates	6.6	6.4	0.6	0.3	4.7	2.8	8.0	9.2	15.4	11.0
Thyroid replacement	11.8	9.9	3.8	2.5	12.1	9.9	12.9	12.4	17.7	14.2

Note. BB = Medications reported in the brown bag data collection. GHC = Pharmacy prescription records from Group Health Cooperative of Puget Sound. NSAIDs = Nonsteroidal anti-inflammatory drugs.

Table 3.

Descriptive statistics for drug class usage

			BB drug classes			GHC drug classes
Group	N	% with no medications	M	Range	% with no medications	M	Range	% with no medications
Total	1430	29.0	1.5	0–8	32.6	1.3	0–9	39.9
Males	644	38.0	1.2	0–8	41.9	1.0	0–8	49.1
Females	786	21.6	1.7	0–8	24.9	1.5	0–9	32.3
<50 years	315	54.9	0.7	0–5	58.7	0.6	0–6	65.4
Males	145	63.4	0.5	0–4	66.9	0.3	0–3	75.9
Females	170	47.6	0.8	0–5	51.8	0.7	0–6	56.5
50–65 years	423	29.6	1.4	0–7	32.2	1.2	0–7	39.5
Males	186	43.5	1.0	0–5	47.3	0.9	0–5	51.6
Females	237	18.6	1.7	0–7	20.3	1.5	0–7	30.0
66–75 years	348	19.0	1.8	0–8	23.6	1.5	0–7	30.5
Males	164	26.2	1.6	0–8	30.5	1.3	0–7	37.2
Females	184	12.5	2.1	0–8	17.4	1.7	0–6	24.5
>75 years	344	14.8	2.1	0–8	18.3	1.8	0–9	26.5
Males	149	19.5	1.8	0–7	23.5	1.5	0–8	32.9
Females	195	11.3	2.3	0–8	14.4	2.1	0–9	21.5

Note. BB = Medications reported in the brown bag data collection. GHC = Pharmacy prescription records from Group Health Cooperative of Puget Sound.

By Age Group

Table 2 also presents the prevalence of the 16 drug classes in the four age groups. Chi-square analyses, 4 (age group) × 2 (presence/absence of drug class), for each of the 16 selected drug classes indicated that age group was significantly related to presence of a drug class for 14 drug classes. Age group was not significantly related to whether glucocorticoids and anti-depressants were reported in either the pharmacy data or the self-report data. Additionally, having a prescription for nonsteroidal anti-inflammatory agents (NSAIDs) was not related to age group in the pharmacy data although age group was significantly related to brown bag reports of NSAIDs (p < .01). For most drug classes, the proportion of each age group prescribed or who reported a drug class was greater for older age groups. However, for estrogens, usage was much higher in the group aged 50 to 65 years in both the pharmacy data and the brown bag data.

As shown in Table 3, the proportions of each group who had no prescriptions in the 16 selected drug classes were: (1) under 50 years: 54.9%; (2) 50–65 years: 29.6%; (3) 66–75 years: 19.0%; and (4) over 75 years of age: 14.8%. Significant age group differences were observed in the number of drug classes (F(3, 1426) = 62.55, p < .001 for brown bag data; F(3, 1426) = 48.67, p < .001 for pharmacy data). On average, older individuals had a greater number of drug classes than younger individuals, except that individuals aged 66 to 75 years and individuals over 75 did not differ significantly in the number of classes reported in the brown bag data.

Agreements and Discrepancies between Brown Bag Data and Pharmacy Data

This section of results provides information on agreement and discrepancies (omission scores and commission scores) for the total sample and four age groups. As described previously, each of the three scores was calculated twice to provide a direct comparison of values when all participants are included (Caskie & Willis, 2004) and values calculated only for the subsample of participants who had at least one medication (Opdycke et al., 1994). Also, recall that the agreement score for the sample of all participants examines congruence (i.e., YY or NN) in all 16 classes of drugs selected for this investigation, whereas the agreement score used by Opdycke and colleagues examined congruence (defined as YY) only for drug classes present in at least one of the two data sources.

Table 4 presents average agreement, omission, and commission scores for the total sample and the four age groups examined as well as the percentages of individuals who had: (a) perfect agreement (agreement score = 1); (b) no omissions (omission score = 0); and (c) no commissions (commission score = 0). All scores ranged from 0% to 100% in the sample, except for agreement scores calculated using the entire sample. In these cases, agreement scores had the following ranges: (a) total sample: 56.25–100%; (b) under 50 years: 81.25–100%; (c) 50–65 years: 62.50–100%; (d) 66–75 years: 56.25–100%; and (e) over 75 years: 68.75–100%.

Table 4.

Agreement, omission, and commission scores for the total sample and by age group

			Agreement		Omission		Commission
Group	Score calculation method	N	M	Perfect agreement	M	No omissions	M	No commissions
Total sample	All participants	1430	96	58	12	80	21	68
	Participants with medications	1015	60	41	17	72	30	55
<50 years	All participants	315	98	74	8	89	15	82
	Participants with medications	142	54	42	17	75	33	60
50–65 years	All participants	423	96	61	12	81	21	70
	Participants with medications	298	62	45	17	73	30	57
66–75 years	All participants	348	94	50	14	76	24	63
	Participants with medications	282	59	39	18	70	30	54
>75 years	All participants	344	94	48	14	76	25	59
	Participants with medications	293	61	39	16	71	29	52

Note. All tabled agreement, omission, and commission values were rounded to whole percentages.

Total Sample

Using the agreement score that included all participants, average agreement between the pharmacy records and brown bag data was high (96%), and more than half of the sample (58%) had agreement on all 16 of the drug classes. Average agreement using the score calculated only for participants with medications was lower (60%), with 41% of the sample having perfect agreement on the drug classes that were present for them. The average rate of discrepancies was lower in the brown bag data (12%) than in the pharmacy records (21%). Further, 80% of the sample had brown bag data that omitted none of the drug classes reported in the pharmacy records while only 68% of the sample’s pharmacy records omitted none of the drug classes reported in the brown bag data. A similar pattern was observed for the subsample of participants with at least one medication.

By Age Group

Average agreement was very high for all age groups (94% to 98%) when all participants, regardless of the number of current medications, were included and agreement was defined as congruent information about either the absence or presence of a drug class. For the subsample of individuals with at least one medication, average agreement was much lower (54% to 62%) across age groups. A one-way analysis of variance (ANOVA) found significant age group differences in agreement (F(3, 1426) = 22.27, p < .001) when all participants were included but not for individuals with at least one medication (F(3, 1011) = 1.38, p = .25). For the agreement score calculated using all individuals, Tukey HSD comparisons found that agreement was significantly higher for younger groups compared to older groups, with the exception that the difference between the 66- to 75-year-old group and the over 75-year-old group was not significant.

For all age groups, the average rate of commissions made by the pharmacy records was higher than the average rate of omissions made by the brown bag data, using both methods of calculation. Using the age group samples that included all individuals, significant age group differences were found for omissions (F(3, 1426) = 3.59, p < .05) and commissions (F(3, 1426) = 5.16, p < .01). Tukey HSD comparisons found that individuals less than 50 years old had significantly lower rates of omissions and commissions than individuals aged 66 to 75 years and over 75 years (omissions: p < .05; commissions: p < .01). No age group differences were found for omissions (F(3, 1011) = 0.15, p = .93) or commissions (F(3, 1011) = 0.37, p = .78) calculated for individuals with at least one medication.

Predictors of Congruence

Predictors of agreement, omissions, and commissions in the four age groups were investigated with hierarchical regression analyses. The predictors were entered in three blocks: (1) health status (i.e., chronic disease score), (2) cognitive ability (i.e., memory ability, verbal ability, inductive reasoning, and perceptual speed), and (3) demographics (i.e., gender, marital status, and education). Because very few regression estimates changed as a result of entering the blocks hierarchically, only the results from the final model including all variables are shown in Table 5; instances where the significance of a predictor changed as variables were added are discussed in the following paragraphs. An initial model included an everyday problem-solving measure (EPT; Willis & Marsiske, 1993) that was removed from this analysis; this measure was uncorrelated with the outcomes and caused spurious regression weights consistent with traditional suppression effects (Conger, 1974).

Table 5.

Standardized regression coefficients for the final step of regression analyses with health, cognitive, and demographic variables predicting agreement, omission, and commission scores within age group

	<50 yearsa			50–65 yearsb			66–75 yearsc			>75 yearsd
Predictor	A	O	C	A	O	C	A	O	C	A	O	C
Chronic disease status	−.32***	.13*	.05	−.18***	.06	−.04	−.12*	.05	−.13*	−.12*	−.01	−.14*
Memory ability	.01	−.13	.01	.02	.01	−.01	.13*	−.08	−.10	.05	−.08	.01
Verbal ability	.00	−.10	.07	−.12	.09	.10	−.01	.14	−.06	−.07	.03	.01
Inductive reasoning	−.04	.11	.00	−.02	−.05	.08	.03	−.03	−.01	.00	−.01	−.01
Perceptual speed	.10	.02	−.11	.03	.05	−.09	.05	−.09	−.01	.06	.00	−.07
Gender (0 = M, 1 = F)	−.03	.08	−.03	−.18***	.11*	.19***	−.26***	.20**	.20**	−.19**	.18**	.14*
Marital status	−.02	−.04	.00	−.06	.02	−.02	.01	−.02	.02	−.04	.09	.10
Education	.02	.04	−.04	.03	−.03	.00	.01	.03	.07	.10	.08	−.04
R2	.12***	.05	.02	.08***	.03	.05**	.10***	.06*	.05*	.06*	.03	.04

Note. A = agreement. O = omission. C = commission.

^an = 302;

^bn = 411;

^cn = 342;

^dn = 333;

^*p < .05;

^**p < .01;

^***p < .001.

Agreement

Chronic disease status was a significant predictor of agreement between the brown bag and pharmacy data for all age groups, even after controlling for cognitive and demographic variables. Lower levels of chronic disease status (i.e., lower estimated health expenditures) were related to higher levels of agreement. This finding had a higher level of significance in the two younger age groups (p < .001) than the two older age groups (p < .05), although the significance level for chronic disease status was p < .01 for those aged 66 to 75 when gender, marital status, and education were not included in the model. For individuals aged 66 to 75 years, memory ability was a significant predictor of agreement (p < .05), after gender, marital status, and education were included in the model. Higher memory ability was related to better agreement. Finally, for the three older age groups, gender was a significant predictor of agreement; men were predicted to have higher levels of agreement than women.

Omission Scores

As shown in Table 5, few of the predictors examined were significantly related to the proportion of pharmacy drugs omitted in the brown bag data. For individuals less than 50 years old, chronic disease status significantly predicted omissions (p < .05), even after controlling for gender, marital status, and education. Poorer health (i.e., greater health care costs) was related to a higher rate of omissions by the brown bag data in the youngest age group. Gender was a significant predictor of omissions for the three oldest age groups; women had higher rates of omissions than men. For individuals 66 to 75 years old, verbal ability was significantly related to the omission of medications from the brown bag data (p < .05); however, this relationship was no longer significant when demographic variables were entered into the model, as shown in Table 5.

Commission Scores

For the two oldest age groups, chronic disease status was a significant predictor (p < .05) of the proportion of brown bag drug classes omitted in the pharmacy data, even after controlling for gender, education, and marital status. Interestingly, higher levels of chronic disease (i.e., higher health care costs) were related to fewer drug classes being omitted by the pharmacy records of those that had been reported in the brown bag data. Finally, gender was a significant predictor of commissions for all age groups except those under age 50. Men had fewer self-reported drug classes omitted in their pharmacy record than women did.

Congruence between Brown Bag Data and Pharmacy Records Data: Within Drug Class

Finally, we examined congruence separately within each of the 16 selected drug classes. Within each of these drug classes, congruence was investigated with the following statistics: (1) kappa coefficient, (2) percent agreement (i.e., both the brown bag data and the pharmacy data were coded 0 or both were coded 1), and (3) sensitivity and specificity of the pharmacy records, and (4) sensitivity and specificity of the brown bag data. These statistics were examined for both the total sample and each of the four age groups.

For the total sample and the four age groups, all but two kappa coefficients (range = .34–.92) were significant at a 95% level of confidence, indicating significantly more agreement between the two databases than would be expected by chance. Non-significant kappa coefficients were found for salicylates (κ = .00) and cardiac drugs (κ = .00) for individuals less than 50 years of age. The prevalence for both salicylates and cardiac drugs in this age group was very small, with only three participants recorded as taking a drug in the class and none of these three participants had congruent information between the brown bag data and the pharmacy prescription data. Percent agreement was high for all drug classes in both the total sample and the four age groups (85–100%).

Specificity of the brown bag data and the pharmacy records was high for all drug classes for both the total sample and the four age groups (range = 89–100%). In contrast, sensitivity was much more variable and tended to be lower; these values are presented in Table 6. For both the total sample and the four age groups, the sensitivity of the brown bag data tended to be higher than the sensitivity of the pharmacy data. Thus, a trend was observed for the brown bag data to be more likely to include a drug class, given that the pharmacy data included that class, than for the pharmacy data to include a drug class, given that the brown bag data included the class. Sensitivity of both data sources for reports of the salicylate drug class was especially poor and typically had the lowest values. As noted previously, the lack of sensitivity (0%) in the youngest group for salicylates was due to only three individuals reporting a salicylate in this group and none having congruent information; the same situation was observed for cardiac drugs in this age group. In contrast to salicylates, a few classes had sensitivity values that were quite high (e.g., ACE inhibitors: 73–97%; diuretics: 69–96%), with some reaching 100% agreement within particular age groups (e.g., brown bag reports of diabetic agents for individuals less than 50 years old and 66 to 75 years old).

Table 6.

Sensitivity of the brown bag data and the pharmacy data

	Total		<50 years		50–65 years		66–75 years		>75 years
Therapeutic class	BB	GHC	BB	GHC	BB	GHC	BB	GHC	BB	GHC
ACE inhibitors	95	79	86	86	92	86	97	73	97	79
Antidepressants	86	77	91	74	87	77	89	75	76	81
Anti-infectives	63	60	47	44	56	59	53	53	80	73
Antilipemics	96	79	100	78	92	89	94	69	100	86
Beta blockers	89	83	75	60	84	90	91	86	93	81
Calcium channel blockers	94	79	100	67	90	81	95	70	95	84
Cardiac agents	86	64	0	0	100	43	88	52	87	75
Diabetic agents	97	80	100	83	96	88	100	71	94	79
Diuretics	94	82	90	69	94	83	96	83	93	81
Estrogens	82	58	83	50	83	63	85	51	76	57
Gastrointestinal agents	71	73	86	86	77	79	63	60	69	74
Glucocorticoids	73	59	81	62	70	59	75	56	64	57
NSAIDs	67	53	33	46	68	47	83	59	75	54
Progesterones	53	65	63	63	57	67	49	63	48	65
Salicylates	40	38	0	0	30	50	31	36	53	38
Thyroid replacement	94	79	100	67	100	82	86	82	96	77

Note. BB = Medications reported in the brown bag data collection. GHC = Pharmacy prescription records from Group Health Cooperative of Puget Sound. All tabled values were rounded to whole percentages. NSAIDs = Nonsteroidal anti-inflammatory drugs.

DISCUSSION

This study examined the congruence of two methodologies for obtaining medication information about study participants, specifically, a brown bag data collection and pharmacy records. The sample was diverse in age (23 to 97 years) and in the type and number of medications being taken. In contrast, previous studies of the congruence of medication information have examined only older adults or individuals prescribed medications for specific medical conditions. This study also examined the influence on congruence of including the portion of many samples who do not take medications and of defining agreement in terms of both the presence and absence of a class of medications.

Overall, medication reports obtained with the brown bag method appeared to provide a more complete picture of individuals’ current medication profiles than did pharmacy records. These results supported the findings obtained by Caskie and Willis (2004) with a sample of older adults. Thus, when the purpose of collecting medication information is to ascertain the medications that individuals are actually taking at a particular time (e.g., to identify possible influences on cognitive assessments), brown bag data collections may serve this purpose better than pharmacy records.

The reasons for the trend of the brown bag data to include drug classes that were not included in the pharmacy data need further exploration. One explanation may be that medication samples dispensed by physicians would not be recorded in a pharmacy database. Another alternative is that individuals may have reported taking medications that had been obtained from friends or family members (Johnson & Moore, 1988). A final possibility is nonadherence to prescription dosing instructions. Cooper, Love, and Raffoul (1982) found that most nonadherence in their sample of elderly persons was intentional and extended the length of a prescription. Under-dosing has also been found in other studies of compliance or adherence (e.g., Isaac, Tamblyn, & the McGill-Calgary Drug Research Team, 1993). Nonadherence to prescription dosing instructions may have resulted in less accurate estimated end-dates for pharmacy-filled prescriptions. Collecting information as part of the brown bag procedure about the prescription fill-date (as well as when the participant began using a refill), dosing instructions, to whom it was prescribed, and whether the medication is a sample could be helpful in determining the impact of noncompliance on estimates of health status or comorbidities based on self-reported medications. Alternatively, researchers who have access only to pharmacy records need to be aware that using automated information such as pharmacy records may underestimate the number and types of medications being used by participants on a particular day.

It is also important to note that some pharmacy prescriptions were excluded from individuals’ brown bag data. Previous research has found that participants may be more likely to omit medications that are prescribed for less serious conditions, that are not taken regularly, or that are taken for a short period of time (Caskie & Willis, 2004; Kelly et al., 1990). The current study also found that these types of medication classes (e.g., NSAIDs and salicylates) were less likely to be reported by both databases than medication classes prescribed for more serious conditions (e.g., diabetic agents and ACE inhibitors). Alternatively, the omissions in the brown bag data of medications in the pharmacy records may have represented situations that can be difficult to detect when using pharmacy records (Choo et al., 1999; Christensen et al., 1997). For example, a prescribed medication may have been discontinued prior to the expected end-date due to the participant experiencing adverse side effects. This medication would likely have been omitted correctly by the individual from the brown bag data despite its status in the pharmacy records as an active prescription. Medications prescribed on an “as needed” basis that are not being taken currently could also result in omissions of pharmacy record medications. At the same time, medications taken “as needed” may also result in the brown bag data including a medication whose supply was expected to be depleted by that date.

Variations in congruence by age group, health status, and gender must also be considered. Agreement between individuals’ pharmacy records and brown bag data tended to be poorer in older age groups, most likely occurring as a function of the greater number of medications typically taken by older adults and, thus, the greater possibility for discrepancies. Similarly, better health was related to greater agreement for all age groups, although the relationship was less strong for older age groups. Of the demographic variables considered, gender was a consistently significant predictor of agreement, omissions, and commissions in individuals over age 50. Because no men were taking estrogens or progesterones, the method of score calculation used to include these cases may have contributed to the trend for men to show greater agreement and fewer discrepancies between the brown bag data and the pharmacy records. Future analyses of medication data that include hormones may need to consider agreement separately for males and females.

This paper contributes an important extension of the validity of sources of medication information by using a relatively healthy, community-dwelling sample with a wide age range and examining differences in four age groups. We also provided a comparison of the method used by Caskie and Willis (2004) with the methods used in previous papers on congruence. With this new method, rates of agreement tends to be higher and rates of discrepancies tend to be fewer due to our shift to assessing congruence for all individuals and to defining agreement in terms of both the absence and presence of a drug class. This focus is appropriate for typical samples that include individuals who are not currently taking medications and for investigations where reports of the absence of medications need to be as accurate as reports of the presence of medications. Our sample was limited, however, in terms of ethnicity, socioeconomic status, and education level. Future research should expand these investigations to non-white samples and samples of varying income and educational level. Follow-up investigations of medications that were not congruent would also be useful and important.

In summary, information about current medications was generally congruent when brown bag reports were compared to pharmacy prescription records. Use of the brown bag method to determine current medication usage is at least equivalent to the use of pharmacy records and may even provide more information due to issues of non-compliance, use of medication samples, and other issues. Because younger and healthier individuals tended to have more congruent medication information, health status and age group need to be considered when determining the validity of medication information for a particular sample. In addition, the specific medication classes examined may also influence congruence.