Measuring Resource Utilization in Patient-Oriented Comparative Effectiveness Research: A Psychometric Study of the Resource Utilization Questionnaire

Abstract

In the United States, rising healthcare costs have led to discussion about bending the cost curve. To understand the true burden of disease and its treatment, costs of care including those incurred by patients and their families must be comprehensively assessed using psychometrically sound instruments. The Resource Utilization Questionnaire (RUQ) is a 21 item self-report questionnaire first developed to measure the costs incurred by families of infants who had required intensive care during the newborn period. The purpose of this article is to describe the conceptualization of resource utilization and costs and other methodological issues in conducting economic analyses, the process of adapting the RUQ for use in children and families with Type 1 diabetes mellitus (T1DM), and the psychometric evaluation to establish content and criterion validity of the instrument. The finalized modified RUQ for T1DM (mRUQ-T1DM) contained 25 items reflecting direct (5 items) and non-direct (3 items) healthcare, patient/family time (8 items), and patient/family productivity (9 items) costs using a 3 month recall. The mRUQ-T1DM validly measures cost incurred by children and families with T1DM and is easily completed by parents. Furthermore, the mRUQ-T1DM may be adapted for use in other populations using a similar process.

In the United States, rapidly rising healthcare costs have led to much discussion about bending the cost curve by improving the quality of services and incorporating the findings of patient centered comparative effectiveness research (CER) into decision-making. An important consideration for CER is treatment of patients with chronic conditions such as type 1 diabetes mellitus (T1DM).

T1DM is the most frequent metabolic disease in childhood and adolescence and affects 186,300 children in the United States (Matthews, Berendt, & Lipsky, 2007). The incidence of T1DM has increased globally in children in all age groups (DIAMOND Project Group, 2006) with the highest age specific increase in young children under the age of five years (Dabelea et al., 2007). Diabetes is costly to individuals and society. In 2007, healthcare costs for the one million U.S. children and adults living with T1DM were estimated at $14.9 billion (Dall et al., 2009). Individuals with T1DM, on average, have medical expenditures of $14,856 per year (Dall et al., 2009), approximately 2.3 times higher than health expenditures would be in the absence of diabetes. Understanding how to provide effective, high quality care efficiently for these patients is of utmost importance.

Findings of the Diabetes Control and Complications Trial (DCCT) clearly demonstrated the importance of blood glucose control in preventing or forestalling long term complications of diabetes (Diabetes Control and Complications Trial Research Group, 1994). Furthermore, the DCCT research group modeled the life-time costs and benefits of intensive versus conventional treatment and found that implementing intensive treatment would cost approximately $29,000 per life year gained (“Lifetime benefits and costs of intensive therapy as practiced in the diabetes control and complications trial. The Diabetes Control and Complications Trial Research Group,” 1996). Researchers from the Epidemiology of Diabetes Interventions and Complications (EDIC) Study have prospectively monitored DCCT subjects for more than 15 years and provide ongoing evidence that the improved metabolic control achieved for those in the intensively treated group remains protective against the long term complications of diabetes (Albers et al., 2010; White et al., 2008). To help achieve this tight glucose control the use of insulin pumps and other technologies has dramatically increased in youth over the past decade. However, rigorous economic evaluations of these expensive new technologies and the burden they may place on youth and their families have not been conducted.

The purpose of this article is to review the conceptualization of resource utilization, methods of valuation of costs and other methodological issues important in the conduct of economic analyses as well as report on the modification and validation of a self-report resource utilization questionnaire (RUQ). We provide a general overview and present a critical review of instruments that have been used by researchers to collect cost data in T1DM. We then describe a psychometric evaluation of a RUQ for use with youth and families with T1DM.

Conceptualization of Resource Utilization, Costs and Methodological Issues

Perspectives and Resource Utilization Components

“Perspective” is a term used in health economics to denote the viewpoint from which an economic evaluation is conducted and drives the set of resources considered as costs in the evaluation. Before commencing any type of economic evaluation, the perspective of the study should be determined and explicitly stated (Gold, Siegel, Russell, & Weinstein, 1996). The most comprehensive perspective is a “societal” approach where all costs are considered regardless of who incurs them (Gold et al., 1996). A societal perspective is most often recommended since it is the broadest approach and concerned with society’s welfare; thus, arguably, the impact of an intervention on the welfare of the whole society, not just on the individual or organization directly involved, should be evaluated (Byford & Raftery, 1998).

In practice, issues related to feasibility, funding, time as well as the primary question at hand have often led to adopting a narrower perspective such as a hospital or insurer (Byford & Raftery, 1998). For example, a hospital decision maker may be interested in assessing the costs (and savings) related to a specific educational program. While patient length of stay and labor are appropriate resources to include in an economic evaluation conducted from the hospital perspective, resources related to outpatient and/or home health services may not be appropriate. A broader perspective, where these resources may apply is that of the insurer. The broadest perspective is societal. In all cases, researchers should be explicit about the adopted perspective since a non-societal perspective may result in suboptimal resource allocation decisions and a corresponding loss in the total welfare of society (Byford & Raftery, 1998). Table 1 lists examples of resource utilization components congruent with various perspectives. Each component is further described below.

Table 1.

Examples of conceptually congruent resource utilization components by various perspectives

Types of Resource Utilization Components	Perspective
	Societal	Insurer	Hospital
Direct healthcare
Intervention	X	X
Hospitalization	X	X	X
Outpatient visits	X	X
Long-term care	X	X
Other health care	X	X
Non-direct and non-healthcare
Transportation	X
Social services	X
Value of time and lost productivity
Patient time receiving care	X
Family/informal caregiver time	X

Direct healthcare resources encompass those used in the provision of an intervention or in dealing with the side effects or other current and future consequences linked to the intervention or illness (Luce, Manning, Siegel, & Lipscomb, 1996). An intervention or illness can affect both current and future resource use (Luce, et al., 1996). Luce and colleagues (1996) consider attributable resources as a stream of resource use that can span time from a year or less for a simple procedure to a lifetime for a preventive intervention or chronic disease treatment regimen. Resources related to the direct provision of healthcare may include the intervention (e.g. the introduction of a new insulin pump or a behavioral management program) or hospitalizations, outpatient visits (e.g. urgent care or emergency room visits), long-term care admissions (e.g. rehabilitation and nursing home care), and other aspects of healthcare (e.g. laboratory or devices not specific to the intervention or the hospitalization) (Stone, Chapman, Sandberg, Liljas, & Neumann, 2000). Overhead items such as rent (sometimes known as fixed costs) are sometimes included in the various categories listed above or calculated separately (Stone et al., 2000).

Other important direct healthcare resources that are often excluded from study are those incurred by the patient and family such as health services not covered by insurance, health education programs (e.g. for the management of diabetes), prescription medications, home supplies (e.g. glucose tablets, medic-alert bracelet), and equipment (e.g. device for inserting insulin pump infusion set) (del Rocio, Vintimilla, Castro, & Grossi, 2008; Elrayah et al., 2005; Songer, LaPorte, Lave, Dorman, & Becker, 1997).

Non-direct and non-healthcare resources are those consumed due to demands of the illness. Non-direct healthcare resource utilization may include travel to and from the healthcare delivery site (e.g. mileage, parking), food while attending a hospital appointment, child care for a parent attending a diabetes education session and/or health related social services (Luce et al., 1996; Stone et al., 2000). Examples of non-healthcare resource use include utilization of special education services and home tutoring (Stone, et al., 2000; Tolbert, 2009).

Patient and family time are important resources to include when the economic evaluation is being conducted from the societal perspective (Hodgson & Meiners, 1982; Luce et al., 1996). This time includes that spent by the patient, family member (e.g. a parent) and/or another caregiver while attending to the patient’s healthcare needs (Stone et al., 2000). Time consumed waiting for and receiving treatment, monitoring blood glucose levels, and providing insulin injections may also be important (Stone et al., 2000; Ziaian et al., 2006).

Decreased productivity or increased morbidity associated with the lost or impaired ability to work, to attend school, or to engage in leisure activities due to disease is also an important resource. For example, when a child has a chronic illness such as diabetes both child and parent productivity may be affected; a child misses educational instruction when he/she requires monitoring in the school nurse’s office because of hypoglycemia. In the case of a parent, productivity at work may suffer when communication with either the school or after school caregivers for diabetes management is needed. While there are guidelines recommending the inclusion of time and productivity (Canadian Agency for Drugs and Technologies in Health, 2006; Drummond, Sculpher, Torrance, O’Brien, & Stoddart, 2005; Gold et al., 1996) along with a general acceptance that these are important resources to be valued, the method of collecting data, valuing the resource monetarily and incorporating these costs into an economic evaluation has been controversial and inconsistently used (Jacobs & Fassbender, 1998; Sculpher, 2001; Tranmer, Guerriere, Ungar, & Coyte, 2005).

In a systematic review of 228 cost-utility studies published from 1975 to 1997, researchers (Stone, Chapman, Sandberg, Liljas, & Neumann, 2000) found that resources related to time and productivity were included in less than 10% of studies reviewed. Nonetheless, if an analysis is conducted from the societal perspective, inclusion of time costs is recommended (Gold, et al., 1996). In addition, because healthcare is becoming more community-based, nursing interventions may directly influence these resources. For example, a home visit by a nurse case manager may not only increase the ability to provide holistic patient-centered care, but also may save resources related to patient time, transportation, family care-giving, and improve the patient’s ability to engage in activities in a meaningful way (i.e. productivity).

Monetary Valuation or Assigning Costs to Resource Utilization

Economists and analysts often use a “two-step” approach to determine the costs attributable to an intervention. The first step in the estimation is determining the amount of resources attributable or consumed. Once the attributable resources have been determined, the “money” valuation or costs of the resources may be estimated. Using a two-step approach increases the clarity and transparency of the analysis and allows readers of the analysis to understand how the costs of attributable resources may be similar or different in their own setting.

Once the consumption of resources has been estimated, the resource must be assigned a monetary value. In healthcare, due to cost-shifting and negations between providers and insurers, it is clear that charges do not equate to cost. A common source of valuation for hospital costs is the hospital’s own cost-accounting system. For researchers internal to the institution, these will often be easy to access. These cost-accounting systems are developed by finance departments to help administrative decision making and are based on past accounting studies and algorithms. Another alternative is to use hospital cost-to-charge ratios (CCRs), which are calculated by dividing the total costs in a cost center by the total charges for the same resource. Published sources and governmental fee schedules may also be used to value a particular resource.

Collecting Resource Utilization Data

Researchers have used insurer or system level administrative data that includes both inpatient and outpatient services to collect data on both resource utilization and the valuation of those resources. But, these data sources often do not capture pharmaceutical and or physician/provider services that may be important. Additionally, the generalizability of the valuation (cost) data may not be accurate due to cost-shifting. The best cross-setting population-based insurer data sets in the United States come from Medicare, Veterans Affairs or large health maintenance organizations such as Kaiser Permanente. However, all of these have limited generalizability based on the populations served. Furthermore, non-direct healthcare resource utilization and costs (such as patient time) are not included. Recently, researchers conducted a systematic review of websites and literature to develop an inventory of data sources to estimate healthcare costs in the United States and identified 88 data sources (Lund et al., 2009).

When researchers wish to use a societal approach self-report methods of resource utilization (sometimes with cost valuations) are generally used due to the limited accessibility and comprehensiveness of other data sources. Self-report methods include surveys, diaries, and face-to-face or telephone interviews (Evans & Crawford, 1999). These techniques are often used in several combinations, such as a patient diary supplemented with a telephone contact, with data collection occurring at baseline, shortly after an intervention, at hospital discharge, and/or at regular intervals (Evans & Crawford, 1999). Researchers have determined that patient self-reports lead to reliable estimates of time, resource utilization and out-of-pocket costs; however, these estimates will vary by recall time, patient condition (e.g. cognitive deficits, and risk status), data collection method (e.g. diary versus survey), and resource use (e.g. medication use tends to be recalled less accurately compared to hospitalizations) (Drew et al., 2004; Evans & Crawford, 1999; Merkesdal et al., 2005). The guiding principle is that validity of estimates decline over time when relying on patient recall; yet, varied recall periods have produced acceptable estimate levels (Evans & Crawford, 1999). Efforts to incorporate patient self-report in future economic evaluations are strongly encouraged (Merkesdal, et al., 2005). The RUQ is an example of a self-report tool that can be used in economic analyses. Literature Review of Economic Evaluations in Diabetes

A recent systematic review (Icks, Holl, & Giani, 2007) revealed a paucity of research examining the attributable cost of diabetes (n = 4 studies) (Icks, Rosenbauer, Haastert, Rathmann, Gandjour et al., 2004; Icks, Rosenbauer, Haastert, Rathmann, Grabert et al., 2004; Nordfeldt & Jonsson, 2001; Stern, Benbassat, Nahshoni, & Blum, 2001) and economic consequences of interventions (n = 6 studies) (Allen, Yarnie, Murray, & Reiter, 2002; Beck et al., 2004; Chase et al., 2003; Dougherty, Schiffrin, White, Soderstrom, & Sufrategui, 1999; Eastman, Leptien, & Chase, 2003; Meinhardt, Ammann, Fluck, Diem, & Mullis, 2003) in pediatric T1DM. In the majority of the reviewed studies, researchers employed an insurer perspective to assess resource utilization and costs; and, in only 3 studies (Chase, et al., 2003; Dougherty, et al., 1999; Nordfeldt & Jonsson, 2001) was a societal perspective taken. Due to the methodological inconsistencies such as the use of varying perspectives, designs and prices, Icks and colleagues (2007) found limited opportunity to compare costs across studies and synthesize findings. In further review, we found that several important resource utilization components, including direct healthcare non-direct healthcare and time were excluded. (Allen et al., 2002; Beck, et al., 2004; Eastman et al., 2003; Icks, Rosenbauer, Haastert, Rathmann, Gandjour et al., 2004; Icks, Rosenbauer, Haastert, Rathmann, Grabert et al., 2004; Meinhardt et al., 2003; Stern et al., 2001). This may be due to either use of a non-societal perspective and/or an inconsistent adherence to recommended guidelines for economic evaluation. These exclusions suggest that the magnitude of the economic consequences of the interventions studied were most likely underestimated..

The researchers that adopted a societal perspective gathered data using self-report methods (Chase, et al., 2003; Dougherty, et al., 1999; Nordfeldt & Jonsson, 2001). It was assumed that the instruments used to collect costs were researcher-developed and there was no discussion of the psychometric properties of any of the instruments. Additionally, not all relevant items were identified, measured, or valued and there was an inconsistent reporting of the important resource utilization components across studies and therefore, uncertainty of the total cost estimates. The lack of consistency has been previously reported in economic evaluations pertinent to care of families of children with pediatric T1DM (del Rocio Vintimilla Castro & Grossi, 2008; Elrayah, et al., 2005; Songer, et al., 1997); and similar inconsistencies have been found in other patient populations (Neumann, Stone, Chapman, Sandberg, & Bell, 2000; Stone, Braccia, & Larson, 2005).

Understanding the implications of diabetes management interventions on the utilization of direct healthcare, non direct healthcare, and non-healthcare resources is increasingly important to better appreciate how different treatment strategies may or may not help to reduce costs from a societal perspective. Pivotal to the assessment is the utilization of a validated costing instrument.

Study Methods

In this study we modified an existing instrument, the RUQ (Cimiotti, Stone, & Larson, 2004; Melnyk et al., 2006; Stone, Curran, & Bakken, 2002), for use in children/families with T1DM using insulin pump technology. Content and criterion validity were assessed using an iterative process and conducted in four phases: (I) review by research experts, (II) convening an expert panel of pediatric diabetes health professionals and parents of insulin pump users, (III) conducting a focus group of parents of insulin pump users, and (IV) conducting in-person interviews of parents who had piloted mRUQ-TDM.. The instrument employs a societal perspective.

Instrument

The RUQ was initially developed for a previous National Institute of Health (NIH) funded study (R01NR005077) for use in a population of families of infants who had required intensive care during the newborn period (Melnyk, et al., 2006). The instrument was further used in another NIH funded study of hospitalized elders (R01NR008455). The original instrument was divided into three sections: direct healthcare and non-direct healthcare resource utilization of the (1) child (7 items), (2) mother (7 items) and (3) father (7 items). For each item, the respondent was asked to recall resources that had been utilized during the previous 6 months.

In the child section, items related to the direct healthcare resource components included number of emergency room visits, hospitalizations (including number of nights and reason for hospitalization), number of well and sick child visits stratified by provider type (i.e., nurse practitioner, pediatrician, family practice physician and physician assistant); home healthcare visits (visiting nurse or other home care); special support services stratified by type (e.g., special education received at a center or at home) and medical equipment. The last items related to non-healthcare resources and included place of child care during the day or working hours, as well as paid and unpaid caregivers in the home. Both the maternal and paternal sections included direct healthcare components such as preventive care, hospitalizations, and phone consultations with providers. Additionally, non-healthcare resource components included questions about productivity (days of work missed due to caring for child and due to own health).

The research team reviewed the RUQ (Phase I) to determine its applicability and modified existing categories to be more consistent with the needs of children and families with T1DM in preparation for its validation in a new population. For the category direct healthcare resources, outpatient visits were tailored to the types of visits incurred by children with diabetes, e.g. pediatrician, pediatric endocrinologist, dietician, and ophthalmologist and stratified by well or sick child visit. Parent and child time for caring for diabetes were also addressed by inclusion of items relating to parent telephone contact with health care team to review blood glucose records, seek guidance for insulin adjustment, and time lost to school for diabetes related illness. In addition, items that were specific to infants such as early intervention services were removed. Following this step, the RUQ was reviewed by a health economist external to the research team.

Sample

Using purposive sampling methods we recruited three independent samples from two academic medical centers located in Manhattan and in Suffolk County, New York. At each site, subjects were recruited by a member of their diabetes team.

The first sample included diabetes health professionals and parents from site 1who participated in an expert panel (Phase II) to review each item of the RUQ for its applicability from varying perspectives. Diabetes health professionals could provide expertise to RUQ modification regarding inclusion of resources needed to meet the standard of care for management of children with type 1 diabetes whereas parents could provide their real world viewpoint of the resources required to live and care for a child with diabetes.

The second sample was parents from site 2. This sample participated in a focus group (Phase III) to further review and refine the instrument following its modification by the expert panel to establish content validity.

We then recruited a third sample of parents from site 1 to pilot the modified RUQ for T1DM (mRUQ-T1DM). Following mRUQ-T1DM completion each parent participated in an individual interview (Phase IV) to substantiate each item response with another form of documentation to establish criterion validity.

All participants were English speaking. The parents all had a child ≤18 years of age with T1DM who currently used an insulin pump for at least one year, and received diabetes treatment at one of the two participating sites. Prior to initiation, the study received approval from Columbia University Medical Center Institutional Review Board and the Committees on Research Involving Human Subjects at the State University of New York at Stony Brook. All participants were monetarily compensated for their time.

Data Collection

We developed semi-structured interview guides and probes for conduct of the expert panel and focus group to lead the participants through each mRUQ-TIDM item, determine its appropriateness for children and adolescents with diabetes, and make recommendation for retention, deletion or modification. Both the expert panel and the focus group were facilitated by an expert in costs incurred by families of children with diabetes.

The expert panel discussion, focus group and individual interviews were audiotaped and field notes taken. The purpose of the in-person interviews was to review mRUQ responses and validate responses using copies of receipts of medical bills, cancelled checks, or other type of documentation. A variety of sources for resource validation was used because one unified source was lacking. For example, although documentation of a diabetes visit could be verified in the outpatient diabetes medical record, a visit to an ophthalmologist, unless a university based provider was seen, pediatrician, or purchase of pharmaceutical items could not. A short individual interview format (10–15 minutes) was used to allow discussion and validation of potentially sensitive matters (Rice & Ezzy, 1999) such as healthcare costs and missed work days.

Data Analysis

Expert panel and focus group audiotapes were analyzed for content by a researcher ( ) experienced in qualitative analysis to identify additional resource utilization categories specific to the pediatric diabetes population and existing items that could be deleted or collapsed into single categories (Waltz, Strickland, & Lenz, 1991). Following phases I to III, the mRUQ-T1DM was revised in accord with recommendations/findings and an audit trail of instrument modifications maintained. During phase IV criterion validity was established by verification of questionnaire responses with actual healthcare bills and/or medical records for hospitalization days, number and type of outpatient visits, and prescribed pharmaceutical items (Waltz, et al., 1991). Descriptive statistics were used to summarize response frequency and validation by resource type.

Results

Two health professionals (pediatric endocrinologist and pediatric nurse practitioner specializing in diabetes management; age 45.8±9.9 years) and 16 parents (age 46.9±5.8 years, 88% female, 94% non Hispanic white, 63% bachelor’s degree or higher) of 14 children with T1DM who used insulin pumps (age 13.6±4.0 years, diabetes duration 7.7±4.1 years, insulin pump use 4.6±3.1 years) participated in either an expert panel (5 parents, 2 health professionals), a focus group (6 parents), or an individual interview (5 parents). The majority of participants were mothers (81%) with health insurance for their children obtained through a parent’s employment (93%); one child was insured through the State Children’s Health Insurance Program. Of the 14 mothers who participated in the study, 10 (71%) were full time homemakers.

Content validity of the mRUQ-T1DM was established through an iterative process. Table 2 summarizes changes made to the RUQ at each phase of its development organized by resource type. As part of phase I modifications, on advice from the health economist, original caregiver productivity questions were replaced with a validated instrument, the Work Productivity and Activity Impairment (WPAI): General Health questionnaire (Reilly, Zbrozek, & Dukes, 1993). The WPAI is a 6 item scale that measures the impact on employment productivity due to absence from work or decreased productivity during the work day. The WPAI has been validated in adults with a variety of health problems (Chen et al., 2008; Reilly, Bracco, Ricci, Santoro, & Stevens, 2004). Recently the WPAI was slightly modified to reflect caregiving-related productivity in a sample of adults/spouses caring for disabled elderly adults (Giovannetti, Wolff, Frick, & Boult, 2009) and found to have good construct validity.

Table 2.

Establishing content validity for the modified Resource Utilization Questionnaire for Type 1 Diabetes Mellitus (mRUQ-T1DM)

Resource type	Phase I Research Experts (n=6)		Phase II Expert Panel (n=7)		Phase III Focus group (n=6)		Final mRUQ-T1DM
	Items (Sub-items)	Recall timeframe (months)	Items (Sub-items)	Recall timeframe (months)	Items (Sub-items)	Recall timeframe (months)	Items (Sub-items)	Recall timeframe (months)
Direct healthcare
Emergency room visits	1	6	1	3			1	3
Hospitalizations	1	6	1	3			1	3
Outpatient medical visits*	1 (8)	6	1 (14)	3	1 (20)	3	1 (20)	3
Equipment/medications	1	3			1 (23)	3	1 (23)	3
Health insurance premium/deductible					1	12	1	12
Non-direct healthcare
Home insurance rider to cover loss of insulin pump			1	12			1	12
Mental health counseling†					1	3	1	3
Child care to attend medical appointments					1	3	1	3
Time
Attending medical appointments*	1 (8)	6	14	3	1 (20)	3	1 (20)	3
Communicating diabetes information	1	1			1 (4)	1	1(4)	1
Receiving diabetes/insulin pump education	1	6	1	3			1	3
Receiving home tutoring					1	3	1	3
Receiving mental health counseling†			1	3			1	3
Time lost to diabetes self-management at home					1	3	1	3
Attending to insulin pump technical issues					1	3	1	3
Productivity
Diabetes related school absence	1	3					1	3
Diabetes self-management at school			1	3			1	3
Impact on mother	6	7 days	6‡	7 days	6‡	1	6	1
Impact on father	6	7 days

^*One questionnaire item includes two resource utilization components: direct healthcare and time;

^†One questionnaire item includes two resource utilization components: non-direct healthcare and time;

^‡Parent completing RUQ would complete “impact on parent” questions for self only

During phase II, based on the expert panel recommendations, the direct healthcare sub-items for outpatient medical appointments were expanded from 8 to 14 categories to include providers such as dentists and podiatrists as well as reason for the visit. In addition, new items to reflect non-direct healthcare (purchase of a rider to the home insurance policy to cover insulin pump loss), time (home tutoring) and productivity (diabetes self-management at school and home tutoring) were added to the instrument. Recall time for most items was decreased from 6 to 3 months.

In phase III, the focus group reviewed the modified instrument and made further suggestions for new items, and tailored other questions. For example, the item for direct healthcare for equipment and medications was changed from free text entry to an alphabetized list of frequently used diabetes care items to make the instrument easier to complete. The item to capture time spent communicating by telephone with the diabetes team or school was also expanded to include additional types of communication with the insulin pump company or the pump trainer and alternate means of communication such as email or fax. The group was unanimous in its suggestion that direct healthcare items be reordered by frequency of use: outpatient visits, emergency room, and hospitalization. Categories of outpatient providers were increased from 14 to 20 (e.g. mental health professional and alternative medicine sub-items were included). A recommendation was made to restrict data collection regarding impact on work to the parent completing the instrument for improved accuracy and to increase the recall time from 7 days to 1 month using the rationale that impact of diabetes on a parent varies from week to week and a 1 month recall would provide a more accurate perspective. As part of the focus group, each parent completed the instrument (mean completion time approximately 10 minutes). Based on expert panel and focus group suggestions, the measure was refined at each phase of review. A copy of the final mRUQ-T1DM is found in the Appendix.

Criterion validity was established by parent completion of the mRUQ-T1DM (3 mothers and 1 couple) followed by participation in an individual interview during which the completed form was reviewed. All parents reported that the form was comprehensive, easy to complete, and included all categories needed for a child with T1DM. Table 3 provides a summary of mRUQ-T1DM responses for each resource category by method of validation. The majority of direct healthcare items including visits with the diabetes team (100%), pediatrician (75%) and purchase of diabetes supplies (67%) were verifiable whereas other health visits such as dental appointments were not possible to validate (0%). Verification of time and productivity items varied. One mother obtained school attendance records to support her son’s absence from school when ill. However, methods to verify decreased productivity, such as time spent in caring for diabetes during the school day or at home, were not available. None of the mothers who participated in phase IV were employed in the labor market. Therefore, of the 6 mRUQ-T1DM items to measure productivity, only one item (During the past month, how much did your child’s health problems affect your ability to do your regular daily activities, other than work at a job?) applied. Responses to this item varied ranging from 1 to 6. One father completed the mRUQ-T1DM assigning a score of 2 out of a possible 10 to represent the impact of his child’s diabetes on both his work and regular daily activity productivity.

Table 3.

Criterion validity (Phase IV) for modified Resource Utilization Questionnaire for Type 1 Diabetes Mellitus (mRUQ-T1DM)

Resource	Response Frequency	Validation method				Validated
		Bill/cancelled check	Health Insurance/distributor statement	School attendance record	Medical record	N	%
Direct healthcare
Outpatient visits diabetes	4		1		3	4	100
Outpatient visit Pediatrician	4	2	1			3	75
Outpatient visit Ophthalmologist	1		1			1	100
Other health visits	3					0	0
Equipment/medications	6		4			4	67
Health insurance premium/deductible	2		1			1	50
Time [indirect]
Communicating diabetes information	5					0	0
Time lost to diabetes self-management at home	4					0	0
Productivity
Diabetes related school absence	1			1		1	100
Diabetes self-management at school	4					0	0

Discussion

In this study, we described our process and findings in modifying and validating the RUQ for use in families of children with T1DM. In this study the RUQ was evaluated with parents of children with T1DM who used insulin pump technology to intensively manage their blood glucose levels. All received diabetes care in tertiary care settings. The families commute from various geographic settings and had varying health insurance plans. To our knowledge this is the first published report of validation of a self-report questionnaire to measure resource utilization using a societal perspective. The RUQ has been used previously to measure costs for families whose infant had received neonatal intensive care and in family caregivers of hospitalized elders (Li et al., 2003; Melnyk, et al., 2006). We found the content of the mRUQ-T1DM to be comprehensive.

The mRUQ-T1DM validly measured resources utilized by children and families and was easily completed by parents. Additionally, the mRUQ-T1DM can be tailored for use in families of children with other complex health problems and we encourage such use. The mRUQ-T1DM was validated in a sample that included both mothers and one couple. When using a societal instrument such as this, an important aspect to consider would be any potential inequities related to household size. Indeed, a two-parent family may incur more costs (e.g., both parents have decreased productivity and time away from work) or less costs (e.g., a stay-at-home parent may be better able to provide the needed caregiving). However, it is important to note, all time (not just time away from work) is included in the instrument.

Managing the day-to-day care of a child with T1DM requires constant vigilance (Sullivan-Bolyai, Deatrick, Gruppuso, Tamborlane, & Grey, 2003). In the present study, the day-to-day management of the child’s T1DM was noted by the parental inclusion of mRUQ-T1DM items related to time and productivity. The addition of these items suggests that parental care-giving is multi-dimensional, requires a competent skill-set, and consists of varying responsibilities to ensure appropriate disease management. Parents devoted their time attending medical appointments, communicating diabetes information, attending educational sessions, and troubleshooting with industry regarding insulin pump malfunctions. Moreover, the parental request to include their child’s school absenteeism and their child’s time allocated for T1DM self-management provides preliminary evidence that children with T1DM may incur costs as well. The advantage of including children’s resource utilization in future studies will ensure a more accurate assessment of costs (Tsimicalis, McKeever, Kavanagh, & Stevens, 2006).

The WPAI was imbedded in the mRUQ to measure loss of productivity at work and during non work usual activities due to caregiver burden. The majority of women in our sample were currently not employed in the workforce. Engagement in the workforce of mothers caring for children with diabetes is frequently not reported; however, the employment characteristics of our sample was consistent with that reported in two other studies (Sullivan-Bolyai et al., 2003; Smaldone & Ritholz, in press) where lack of full time employment outside the home was reported as greater than 50% and 42% respectively. Employment of mothers caring for children with other chronic diseases has been found to be variable (Miedema, Easley, Fortin, Hamilton, & Mathews, 2008; Powers, 2001; Rosenzweig, Brennan, Huffstutter, & Bradley, 2008). Barriers to maternal employment in the TIDM population may include limited access to child care, or respite services (Sorenson, 2010). A substantial opportunity cost may be associated with caring for a child with TIDM. More research is needed to examine the preferences, facilitators and barriers to employment of mothers of children with complex healthcare needs.

Failure to ignore the opportunity costs associated with TIDM in future research will grossly underestimate the potential burden faced by families and fail to recognize the importance of providing them with support. Currently, there is limited agreement regarding appropriate methods to identify, measure, and value time and there are several distinct and competing theories (Gold et al., 1996; Sculpher, 2001).. These theories include the following the human capital approach, the friction cost method, the willingness-to-pay (or accept) approach, and the US Panel approach (Jacobs & Fassbender, 1998; Sculpher, 2001). These theories are not mutually exclusive and each has its own set of limitations.

In the present study, we included the WPAI in the mRUQ-T1DM. The WPAI includes only one item to measure the productivity loss of individuals in the non work setting. Although the sample participating in Phase IV of our study was small, we noted variability of responses to this item with parents reporting a range of 10 to 60% decrease in productivity related to usual activities. In one study of informal caregivers of older, medically complex adults (Giovannetti et al., 2009), loss of productivity in the non-work setting was similar for both employed and non employed caregivers with both groups reporting, on average, a 27% decline in usual activities due to caregiver burden.

In our sample, concordance between the mRUQ-T1DM and healthcare bills was highest for direct healthcare expenditures particularly diabetes team outpatient visits and supplies. Based on prior research (Guerriere et al., 2006; Roberts, Bergstralh, Schmidt, & Jacobsen, 1996) and past experience with the instrument, we expected high concordance between questionnaire responses and healthcare bills in this population. Time and productivity costs were more difficult to validate; however the addition of the WPAI as a measure of work productivity is a strength as it has already undergone other validation processes (Chen, et al., 2008; Reilly, et al., 2004).

As part of the American Recovery and Reinvestment Act of 2009, the Department of Health and Human Services was awarded $1.1 billion for comparative effectiveness research (CER) (Clancy & Collins, 2010). Furthermore, the Patient Protection and Affordable Care Act established a new, nonprofit body, the Patient-Centered Outcomes Research Institute (PCORI). The PCORI will spearhead and fund CER in the United States and set research priorities (Clancy & Collins, 2010). Over the past 2 to 3 decades, other countries have developed similar institutes (e.g., England’s National Institute of Health and Clinical Excellence and Germany’s Institute for Quality and Efficiency in Health Care). These countries use CER results to systematically determine the relative value of a new intervention to inform policy reimbursement decisions (Sorenson, 2010). In the United States, healthcare spending is currently estimated at $2.4 trillion dollars or 17 percent of the gross domestic product (GDP) and at the current rate of growth is estimated to reach $4.4 trillion or 20.3 percent of the GDP by 2018 (Sisko et al., 2009). Therefore, “bending the cost curve” by improving the quality of services based on CER evidence has received increased attention (Ghaferi, 2010). This is especially true for chronic diseases such as T1DM. However, it must be noted that including costs in CER is not without debate. Those opposed to including cost into CER are concerned that it could lead to limiting access and benefits. Others feel strongly that cost is an integral component of informed decision-making and that a wide range of stakeholders, including employers, policy makers and public health departments need cost information to make informed allocation decisions. Nevertheless, the recently appointed Administrator of the Centers of Medicare and Medicaid (CMS), Donald Berwick has identified reducing per capita costs by providing patient-centered care that avoids waste (Alonso-Zaldivar, 2010).

A strength of this instrument is that builds upon previously developed questionnaires (the RUQ and the WPAI) and applies them to new populations. Additionally, the validation was done in samples from two different settings, which may increase the generalizability. However, both samples come from the Northeast, and care and resulting resource utilization may be different in other regions. Also, the sample size is small. Further validation in larger samples and in other regions is encouraged.

The mRUQ-T1DM is a valid instrument to measures resource utilization using a broad base societal perspective. Assigning a monetary value to these resources allows the researcher to incorporate the costs of the patient and family. This is consistent with patient centered care and the goals of CMS and PCORI. We encourage others to use the instrument in research on T1DM and to modify the instrument for use in other populations.