Parents were accurate proxy reporters of urgent pediatric asthma health services—a retrospective agreement analysis

Abstract

Objective

To assess agreement between parents’ proxy reports of children’s respiratory-related health service use and administrative data.

Study Design and Setting

A retrospective analysis of statistical agreement between clinical and claims data for reports of physician visits, emergency department (ED) visits, and hospitalizations in 545 asthmatic children recruited from sites in the greater Toronto area was conducted. Health services use data were extracted from the Ontario Health Insurance Plan and Canadian Institute for Health Information databases for each child for the interval coinciding with the proxy report for each health service type.

Results

Agreement between administrative data and respondent reports (n =545) was substantial for hospitalizations in the past year (κ =0.80 [0.74, 0.86]), moderate for ED visits in the past year (κ =0.60 [0.53, 0.67]), and slight for physician visits (κ =0.13 [0.00, 0.27]) in the past 6 months. Income, parent’s education, and child quality-of-life symptom scores did not affect agreement. Agreement for ED visits was significantly higher (P <0.05) for children who had an asthma attack in the past 6 months (κ =0.61 [0.54, 0.68]) compared to children who did not (κ =0.25 [0.00, 0.59]).

Conclusion

Parents of asthmatic children are reliable reporters of their child’s respiratory-related urgent health services utilization.

1. Introduction

Accurate health services utilization data are important for epidemiologic studies [1], health services research [2], and economic evaluations [3], and these data are often collected via self-reported questionnaires or interviews [4,5]. Given the increased use and importance of these data, it is imperative that the validity of self-reports be assessed. Several studies have investigated the validity of patient reports of health service utilization for ambulatory, homecare, and emergency department (ED) visits and hospitalizations by comparing patient reports to a second source of data such as medical records or administrative databases. Many of these validation studies have focused on older populations [6,7] and adult populations with chronic diseases such as asthma and cystic fibrosis [5,8].

Epidemiologic studies of children’s use of health services are relevant and important. Researchers have historically examined hospital admission rates as a marker for disease morbidity and to explore changes in morbidity over time. Due to children’s cognitive and developmental limitations, studies investigating children’s use of health service often rely on parent proxy reports. Few studies have investigated the validity of parent reports for health services utilization by their children [9–12]. Moreover, no studies have investigated the validity of parent reports for children with a chronic childhood disease. The objective of this study was to assess agreement between parents’ reports of children’s respiratory-related health services utilization (outpatient physician visits, ED visits, and inpatient admissions) and administrative health insurance claims data for children with asthma.

2. Methods

2.1. Participants

Approval was granted by the Hospital for Sick Children and Brampton Memorial Hospital Research Ethics Boards. Written informed consent was provided by parents/guardians and participants 16 years or older. Assent was provided by younger participants.

Data were extracted from a completed study of the effect of drug plans on outcomes in asthmatic children. Participants/parents were recruited from clinics, EDs, and primary and specialist physicians’ practices. Residents of Ontario aged 1–18 years with a documented diagnosis of asthma, an asthma prescription in the past year, and fluency in English were eligible. A sample of 879 participants was recruited between December 2000 and March 2003. Of these, 545 (62%) provided written consent to release their health card numbers for linking interview data to administrative claims data for the present study.

2.2. Data collection

Three questionnaires were administered by face-to-face interview. A demographics questionnaire asked parents about family characteristics and socioeconomic status. A health questionnaire asked parent proxies or adolescents (all children 16–18 years of age and certain children aged 10–15 years who were able to respond) about asthma history, symptoms, and asthma attack frequency in the past 6 months. An attack was defined as the sudden worsening of symptoms resulting in difficulty breathing that often required taking additional asthma medicine with or without an unscheduled ED or doctor visit. The health questionnaire also queried about medication and spacer use, number of respiratory-related ED visits and inpatient admissions in the past year, and number of outpatient physician visits (general/family practitioners and specialist visits) in the past 6 months. The Paediatric Asthma Quality of Life Questionnaire (PAQLQ) [13–15] assessed the child’s quality of life according to three domains: symptoms, activity limitations, and emotions. Domain scores ranged from 1 (worst) to 7 (best). Children 7 years of age and older were interviewed directly. For younger children, PAQLQ responses were obtained from parent proxies. Questionnaires that were incomplete after the interview were completed by telephone whenever possible.

Health card numbers were used to extract from the Ontario Health Insurance Plan (OHIP) database the date and type of service (ED, general/family practitioner, or specialist visit) with a diagnostic code for asthma. Data on admissions with an ICD-9CM/10 discharge code for asthma were extracted from the Canadian Institute for Health Information Discharge Abstract Database using all diagnosis fields. Because some physicians may submit a diagnostic code other than asthma when a respondent reports a visit as asthma related, an alternative approach that included services with an ICD9-CM/10 code for asthma or common respiratory conditions (cold, pharyngitis/laryngitis, upper respiratory infection, bronchitis, allergic rhinitis, pneumonia, influenza, other respiratory diseases, and respiratory signs/symptoms not yet diagnosed) was also examined. Health services utilization information was not available for one adolescent respondent, and this case was excluded from the analysis of agreement.

Agreement between reports and administrative data for each health service variable was examined when all respiratory diagnostic codes were used and when only asthma diagnostic codes in the administrative data were used, and the results were compared. The kappa (κ) statistic measuring agreement was higher for the asthma diagnostic code than for all respiratory diagnostic codes for outpatient physician visits (0.13 vs. 0.05) and lower for the asthma diagnostic code than for all respiratory diagnostic codes for ED visits (0.47 vs. 0.60) and inpatient admissions (0.76 vs. 0.80). Therefore, all analyses of agreement utilized the asthma diagnostic code for outpatient physician visits, and all respiratory diagnostic codes for ED visits and for inpatient admissions.

2.3. Statistical analysis

Statistical analysis was conducted using SAS 7.0. The proportion of agreement between parent/self-report and administrative data for each dichotomous health service use (e.g., any visit to ED in last year—yes/no) was assessed in two ways, by calculating observed agreement and a kappa statistic. Observed agreement, which is not chance-corrected, was calculated by summing the number of cases in which there was agreement between the self-report and OHIP data (yes/yes and no/no) and dividing by the total number of cases. The kappa statistic adjusts for the magnitude of agreement expected by chance [16]. Landis and Koch [17] suggested that kappa values of 0–0.2 represent slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.80 substantial agreement, and 0.81–1.00 almost perfect agreement. The difference in kappa values between the following subgroups was assessed using a z-test [16]: respondent group (adolescent vs. parent), annual household income (<CAD$45,000 vs. ≥CAD$45,000 [CAD, Canadian dollars]), education (completed secondary school or less vs. some college or university/received degree), PAQLQ symptom control (<5.5 vs. ≥5.5), and asthma attacks in the past 6 months (0 vs. ≥1 attacks). As only a parent or child reported health services utilization for each case, parent and adolescent reports were not compared to each other for the same participant.

3. Results

3.1. Sample characteristics

The 545 respondents who consented to participate had higher average incomes (CAD$62,866 vs. CAD$54,120, P <0.0001) and were more educated (received degree/diploma, 62.5% vs. 47.2%, P <0.0001) compared to non-consenters. Demographic data for the 545 children were collected from 459 parent proxies (parent group) and 86 adolescents (adolescent group). The majority (91%) of parent respondents were mothers. Also, most (59%) of the 545 children were male, and less than 10 years of age (73%), with an average age of 7. Seventy-eight percent of the participants had a household income of more than CAD$45,000 and 76% of the children’s parents had some college/university education or received a university degree/college diploma (Table 1).

Table 1.

Sample demographics

Characteristics		n	%
Age	1 to <4 yr	168	31
	4 to <7 yr	136	25
	7 to <10 yr	92	17
	10 to <14 yr	98	18
	14 yr or more	51	9
	Mean, S.D.	7.00, 4.33
Sex	Male	323	59
	Female	222	41
Respondent	Child	86	16
	Parent	459	84
Household incomea	Less than $25,000	37	6
	$25,000 to <$45,000	60	11
	$45,000 to <$60,000	80	15
	$60,000 to <$80,000	120	22
	$80,000 or more	221	41
	Unknown	27	5
Parent respondent	Mother	416	91
	Father	41	9
	Other	2	<1
Parent’s education	No schooling or elementary	4	1
	Some secondary/high school	27	5
	Completed secondary/high school	95	17
	Some college/university	78	14
	Received university or college degree/diploma	340	62
	Unknown	1	<1

^aCanadian dollars.

As seen in Table 2, the median scores for PAQLQ activity limitations, emotions, and symptoms in the sample were similar: 6.00, 6.00, and 5.90, respectively. The majority of children (94%) were reported to have at least one outpatient physician visit in the prior 6 months, with a median of three per child. Over half (51%) the respondents reported at least one ED visit in the past year. Only one-quarter of respondents reported at least one inpatient admission in the past year. The median number of asthma attacks in the past 6 months for the participants was two.

Table 2.

Quality of life and reported health service utilization

Measure	Median	Minimum	Maximum
PAQLQ symptom score	5.90	1.10	7.00
PAQLQ activity limitations score	6.00	1.80	7.00
PAQLQ emotions score	6.00	1.38	7.00
Asthma attacks in past 6 mo	2	0	50
Asthma outpatient physician visits in past 6 mo	3	0	32
Asthma ED visits in past year	1	0	14
Asthma admissions in past year	0	0	12

3.2. Analysis of agreement

The overall agreement results are presented in Table 3. Agreement between self/proxy report and OHIP data was substantial for inpatient admissions (93%, κ =0.80), moderate for ED visits (80%, κ =0.60), and slight for outpatient visits (79%, κ =0.13).

Table 3.

Overall agreement between respondent reports and claims data

	++	+−	−+	−−	Obs (%)	κ	95% CI
Outpatient visits	411	22	93	16	79	0.13	0.00, 0.27
ED visits	189	24	85	244	80	0.60	0.53, 0.67
Admissions	114	17	23	388	93	0.80	0.74, 0.86

Notes: ++, visit present in claims data and self-reported; +−, visit present in claims data but not self-reported; 3−, visit not present in claims data but self-reported; −−, no visit present in claims data and no self-reported visit.

Recall period was 6 months for outpatient visits and was the past year for ED visits and admissions.

Abbreviations: Obs, observed agreement; CI, confidence interval.

Agreement between self/proxy report and OHIP data was evaluated for the effects of key demographic characteristics (Table 4). Observed agreement for the adolescent self-report group was higher than that for the parent proxy report group for outpatient physician visits (88% vs. 77%), ED visits (88% vs. 78%), and inpatient admissions (96% vs. 92%). However, chance-corrected agreement (κ) was only higher for the adolescent self-report group for outpatient physician visits (0.34, fair vs. 0.10, slight). There was a tendency for the adolescent group to be better reporters of health care utilization than the parent proxy group; however, this was not statistically significant. High-income respondents were better reporters than low-income respondents as observed agreement was higher for outpatient physician visits (80% vs. 78%), ED visits (81% vs. 76%), and inpatient admissions (93% vs. 92%). Although slight, chance-corrected kappa values were also better for high-income respondents compared to low-income respondents for outpatient physician visits (0.16 vs. 0.07). Kappa was substantial for ED visits in high-income respondents (0.62) and moderate in low-income respondents (0.52). There was no statistical difference in the kappa statistic between high- and low-income groups. Observed agreement was greater for respondents with high education compared to respondents with lower education for outpatient physician (80% vs. 76%) and ED visits (81% vs. 75%). The kappa statistic was also greater for respondents with high education for these two types of health services. Although respondents with higher education had a tendency to be better reporters of health care service utilization, there was no statistically significant effect of education on the kappa statistic.

Table 4.

Agreement by demographic characteristic

	++	+−	−+	−−	Obs (%)	κ	CI
Parent (n =457)
Outpatient visits	339	22	83	13	77	0.10	(0.00, 0.25)
ED visits	180	24	75	178	78	0.57	(0.50, 0.65)
Admissions	107	17	20	313	92	0.80	(0.73, 0.86)
Adolescent (n =85)
Outpatient visits	72	0	10	3	88	0.34	(0.00, 0.72)
ED visits	9	0	10	66	88	0.58	(0.34, 0.83)
Admissions	7	0	3	75	96	0.80	(0.59, 1.00)
High income (n =420)
Outpatient visits	322	17	67	14	80	0.16	(0.16, 0.32)
ED visits	139	18	61	202	81	0.62	(0.54, 0.69)
Admissions	82	13	18	307	93	0.79	(0.72, 0.86)
Low income (n =95)
Outpatient visits	72	4	17	2	78	0.07	(0.00, 0.42)
ED visits	37	6	17	35	76	0.52	(0.35, 0.69)
Admissions	23	4	4	64	92	0.79	(0.66, 0.93)
High education (n =415)
Outpatient visits	318	19	65	13	80	0.14	(0.00, 0.31)
ED visits	148	14	63	190	81	0.63	(0.56, 0.70)
Admissions	86	13	20	296	92	0.79	(0.72, 0.86)
Low education (n =127)
Outpatient visits	93	3	28	3	76	0.09	(0.00, 0.37)
ED visits	41	10	22	54	75	0.50	(0.34, 0.65)
Admissions	28	4	3	92	94	0.85	(0.75, 0.96)

Notes: ++, visit present in claims data and self-reported; +−, visit present in claims data but not self-reported; 3−, visit not present in claims data but self-reported; −−, no visit present in claims data and no self-reported visit.

Recall period was 6 months for outpatient visits and was the past year for ED visits and admissions.

High income, ≥CAD$45,000/yr; low income, <CAD$45,000/yr.

High education, some college/received degree or diploma; low education, secondary school education or less.

Abbreviations: Obs, observed agreement; CI, confidence interval.

Table 5 presents the effects of asthma morbidity on agreement. Reports for children with better PAQLQ symptom control scores had greater observed agreement than reports for children with lower PAQLQ symptom control scores for ED visits (81% vs. 77%) and inpatient admissions (95% vs. 91%). The κ statistic was greater for children with better PAQLQ symptom control scores for all three types of health care services utilization, but the differences in kappa values between high- and low–symptom control groups were not statistically significant. Reports for children who had at least one asthma attack in the past 6 months had greater observed agreement with administrative data than reports for children who did not have an asthma attack in the past 6 months for outpatient physician visits (79% vs. 76%) and ED visits (80% vs. 78%). The κ statistic was greater only for ED visits (0.61, substantial vs. 0.25, fair) for patients who had at least one asthma attack in the past 6 months, and the difference in agreement between groups was statistically significant (z =2.07, P <0.05).

Table 5.

Agreement by asthma morbidity characteristic

	++	+−	−+	−−	Obs (%)	κ	CI
High symptom control (n=285)
Outpatient visits	77	3	61	144	78	0.54	(0.45, 0.64)
ED visits	99	14	39	133	81	0.63	(0.53, 0.72)
Admissions	66	9	6	204	95	0.86	(0.79, 0.93)
Low symptom control (n=193)
Outpatient visits	51	5	54	83	69	0.41	(0.29, 0.54)
ED visits	70	9	35	79	77	0.55	(0.43, 0.67)
Admissions	40	5	13	135	91	0.75	(0.65, 0.86)
Any attacks (n =477)
Outpatient visits	363	21	79	14	79	0.13	(0.00, 0.28)
ED visits	183	18	76	200	80	0.61a	(0.54, 0.68)
Admissions	109	15	23	330	92	0.80	(0.73, 0.86)
No attacks (n =63)
Outpatient visits	46	1	14	2	76	0.14	(0.00, 0.52)
ED visits	5	6	9	43	76	0.25	(0.00, 0.59)
Admissions	5	2	0	56	97	0.82	(0.57, 1.00)

Notes: ++, visit present in claims data and self-reported; +−, visit present in claims data but not self-reported; 3−, visit not present in claims data but self-reported; −−, no visit present in claims data and no self-reported visit.

Recall period was 6 months for outpatient visits and was the past year for ED visits and admissions.

High symptom control, PAQLQ symptom score ≥5.5; low symptom control, PAQLQ symptom score <5.5.

Any attacks, one or more asthma attacks in the past 6 months; no attacks, no asthma attacks in the past 6 months.

Abbreviations: Obs, observed agreement; CI, confidence interval.

^aSignificant difference between the “any attacks” and “no attacks” groups (P <0.05).

4. Discussion

Agreement between administrative data and respondent reports was substantial for inpatient admissions (κ =0.80), moderate for ED visits (κ =0.60), and slight for outpatient physician visits (κ =0.13) according to the Landis and Koch [17] criteria. Although there was a trend toward higher agreement between reports and administrative data for groups with higher income, higher education, and better PAQLQ symptom control scores, the differences in kappa values were not statistically significantly different for any of the three types of health services examined. However, agreement for ED visits was significantly higher for children who had an asthma attack in the past 6 months compared to children who did not.

Our study results indicate that agreement was highest for health services that were less frequent and associated with serious adverse health events. A study conducted by Ungar and Coyte [5] with an adult asthma population comparing patient report to OHIP data found similar results. Agreement was perfect (κ =1.00) for respiratory admissions and substantial (κ =0.74) for specialist visits in the past year, but only fair (κ =0.34) for general practitioner visits in the past 6 months when OHIP data and patient reports were compared. Similarly, agreement between patient report and administrative data was almost perfect (κ =0.88) for hospitalizations in the past 4 weeks in an adult cystic fibrosis population [8]. Conversely, in that study, agreement was substantial (κ =0.69) for general practitioner visits in the past 4 weeks [8]. This is likely due to the shorter recall period of that particular study and the nature and frequency of outpatient visits for cystic fibrosis.

Agreement between OHIP data and respondent report of outpatient physician visits in this study was slight (κ =0.13). This may be due to several factors. First, although respondents remembered and reported visits to family/general practitioners and pediatricians as asthma related, the health care practitioners may have coded the visit as a regular checkup or immunization visit, resulting in misclassification. This misclassification bias is suggested by the finding that 17% of respondents overreported an outpatient visit as asthma related and only 4% of respondents did not report an asthma-related visit that was recorded as such in the OHIP database. In using administrative claims to identify a cohort of children with persistent asthma, Kozyrskyj et al. found that reliance on a single visit with an asthma diagnosis was insufficient and that case definition was improved when health care visits that occurred in nonwinter months were combined with claims for asthma medications [18]. Similarly, Blais et al. found that the positive predictive value of the Quebec provincial health services administrative claims database for predicting that an asthma diagnosis will be found in the patient’s chart improved when two or more visits in the claims data with a diagnosis of asthma were used [19]. Another reason for the low agreement may be that the kappa statistic is known to be influenced by prevalence [20]. The kappa statistic either increases or decreases as each of the 2 row or column totals differs from half of the total sample size, depending on whether the imbalances between column and row totals are asymmetric or symmetric [20]. Therefore, the high imbalance of yes (OHIP)/yes (respondent report) answers for outpatient physician visits is likely biasing the kappa value downward. The low kappa statistic is in contrast with the high level of observed agreement (79%) between respondent report and OHIP data for outpatient physician visits. This observed agreement is similar to the values that Ungar and Coyte [5] found for general practitioner and specialist visits (67% and 88%, respectively). Additionally, the results are consistent with other studies which show that physician outpatient/ambulatory visits are reported with substantially less accuracy than ED visits and inpatient hospitalizations [6,21].

Not only was agreement high for ED visits and admissions that represent serious adverse health events, but agreement for ED visits was significantly higher for children who had an asthma attack in the past 6 months compared to children who did not. More accurate reporting may thus be a reflection of improved recall as a function of past health events that are associated with potentially serious health consequences.

Our results suggest that although parents are accurate reporters of ED visits and inpatient hospitalizations for their children, they are less so for outpatient visits. Other studies had similar results. D’Souza-Vazirani et al. [9] found that the agreement between parent report and medical record for hospitalizations and ED visits since their child’s birth was substantial when the child was 2–4 months old and moderate for visits in the past 12 months when the child was 30–33 months old. Pless and Pless [10] found that 84.6% of parents had good recall of the number of hospitalizations their child ever had in comparison to physicians’ records. Only 51.3% of parents had good recall of the number of physician visits their child had that year [10]. Similar to the present study, Canino et al. [11] found higher agreement between parent report and medical records for hospitalizations (κ =0.60) for children compared to outpatient services (κ =0.55) in the past year. Likewise, low agreement (intra-class correlation coefficient [ICC] =0.37) was found between parent reports and medical records for attention deficit hyperactivity disorder (ADHD) primary care services in the past year in a study by Bussing et al. [12]. There was a trend for parents with higher family income, more education, and children with better PAQLQ symptom control to demonstrate higher agreement between reports and administrative data for all types of health care services; however, the differences were not statistically significant. Some studies have found that family annual income and education did not affect recall of health care utilization when parent reports were compared to a second data source [9,10]. Conversely, parents with high socioeconomic status were significantly better reporters of their children’s ADHD outpatient services compared to parents with low socioeconomic status when parent reports were compared to medical records [12]. However, other studies have found that the agreement between self-reported and computer database data for health care utilization is affected little by education and social class [22–24]. The lack of a statistically significant difference in agreement between high and low values for demographic characteristics in the present study may be related to the cut-point selected for comparing agreement. The study sample had a higher average education and income than those who did not consent to participate. Choosing CAD$25,000 rather than CAD$45,000 as a low income cutoff may have demonstrated significance, but the small sample size in the lowest-income category precluded this analysis. Similarly, defining lower education as not having completed high school may have rendered a significant result but the effect could not be assessed due to inadequate sample size in the lowest-education category.

Although other health services utilization validation studies have been completed, this is the first study to compare parent proxy reports of health care utilization to administrative data for an asthmatic pediatric population. Asthma is the most common chronic disease impacting children in developed countries [25,26]. In Canada, the prevalence of asthma in children aged 5–19 years is between 10% (Sherbrooke, Quebec) and 18% (Prince Edward Island) [27]. Given the high prevalence of this condition and the importance of epidemiologic and health economic evaluations of the burden of disease, it is vital to assess the accuracy of parent proxy reports. Additionally, this study captures reports of health service utilization along a wide age range, from the proxy reports for early stages of childhood development (1 year of age) through to the self-reports for adolescents. Other studies investigating parent reports of child health services were limited to children 13 years of age or younger [9,10]. There remains a paucity of data regarding pediatric health services utilization reporting. Most published agreement studies focused on health services utilization by older populations [6,7,28].

A limitation of our study was that the sample was not randomly selected; however, a sampling strategy that enabled inclusion of asthma patients recruited from primary and specialist clinics and residing in urban and suburban areas representing the full spectrum of asthma severity and management was used. Inadequate variation in demographic variables such as income and education in the study sample precluded an analysis of the effects of very low income or education on agreement, and this requires further study. Another limitation of this study is that in the absence of a true gold standard, we considered the OHIP and Canadian Institute for Health Information administrative databases to be the gold standards for comparison. Although the Canadian Institute for Health Information data are considered highly valid and are subjected to quality control audits [29], the accuracy of OHIP data remains unknown [30,31]. Several other studies have used OHIP data as the gold standard for comparison to patient reports [5,7,8] as it is a more efficient and cost-effective method than using patient charts or a combination of data sources. The advent of electronic patient charts may facilitate the use of this data source for future assessments of the accuracy of reporting.

Although the present study was not designed to compare an adolescent’s self-report of health services use to his or her own parent’s proxy report, adolescent reports were compared to parent proxy reports. It would be of value in future to assess the effects of demographic characteristics such as family income and education on the agreement between adolescent self-reports and parent reports of the same cases. In future, it would also be of value to investigate agreement between parent report and administrative data for health care services utilization in other childhood diseases and to compare multiple sources of health services information (self-report, medical chart, administrative) to determine which sources or combination of sources provides the most accurate result. This information could be used to make recommendations regarding the optimal source(s) of health service data for epidemiologic and health economic research.

In conclusion, the results of this study indicate that agreement was substantial and moderate between respondent reports and administrative data for ED visits and inpatient admissions in the past year, respectively, suggesting that parents are accurate reporters for their child’s urgent respiratory-related health care utilization. Also, parents whose child had one or more asthma attacks in the past 6 months were better reporters. This study is the first to focus on an asthmatic pediatric population and parental report of health services utilization. The results support the use of parental proxy reports of urgent health services use (ED visits and inpatient admissions) in epidemiologic studies and economic evaluations for children with asthma.

What is new?

• Parents of asthmatic children were found to be reliable reporters of their child’s emergency visits and hospital admissions.

• Agreement between parent reports of emergency visits and administrative data was significantly higher for children with prior emergency visits.

• Although parents are routinely relied upon as proxy reporters of their children’s health service use, there has been little prior investigation of the validity of parent reports for children with a chronic childhood disease.

• This is the first study to compare parent proxy reports of health care utilization to administrative data for an asthmatic pediatric population.

• The use of parent proxies for reports of urgent health services use in children with asthma should be considered for epidemiologic and economic evaluations that entail primary data collection.