Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Aug 17.
Published in final edited form as: J Prim Prev. 2012 Aug;33(4):209–222. doi: 10.1007/s10935-012-0276-x

Design of a Randomized Controlled Trial of a Web-Based Intervention to Reduce Cardiovascular Disease Risk Factors Among Remote Reservation–Dwelling American Indian Adults with Type 2 Diabetes

Jeffrey A Henderson 1,, Jessica Chubak 2, Joan O’Connell 3, Maria C Ramos 4, Julie Jensen 5, Jared B Jobe, On Behalf of the LOWPK Project Team6
PMCID: PMC4539001  NIHMSID: NIHMS414690  PMID: 23001642

Abstract

We describe a randomized controlled trial, the Lakota Oyate Wicozani Pi Kte (LOWPK) trial, which was designed to determine whether a Web-based diabetes and nutritional intervention can improve risk factors related to cardiovascular disease (CVD) among a group of remote reservation–dwelling adult American Indian men and women with type 2 diabetes who are at high risk for CVD. Enrollment on a rolling basis of 180 planned participants began during 2009; an average 18-month follow-up was completed by June 2011. The primary outcome variable is change in glycosylated hemoglobin level after an average 18-month follow-up period. Secondary outcome variables include changes in low-density lipoprotein cholesterol, systolic blood pressure, body mass index, and smoking status, as well as an evaluation of intervention cost-effectiveness. If effective, the LOWPK trial may serve as a guide for future chronic disease intervention trials in remote, technologically challenged settings.

Keywords: American Indians, Cardiovascular disease, Risk reduction, Intervention, Web-based

Introduction

Background and Context of the Trial

Cardiovascular disease (CVD) is a leading cause of mortality and morbidity in the American Indian (AI) population (Indian Health Service [IHS], 2000). Approximately 30 % of AI deaths at all ages are associated with CVD, and the number of deaths associated with CVD among AI adults aged 45 years and older exceeds the next three leading causes of death (cancer, diabetes, unintentional injuries) combined (Centers for Disease Control and Prevention [CDC], 2004). Furthermore, the decline in age-adjusted CVD death rates experienced by the general population in recent decades has not been observed in the AI population (Howard et al., 1999). Indeed, among most tribes, CVD morbidity and mortality rates are increasing (Galloway, 2005). The increasing prevalence of type 2 diabetes, obesity, and smoking account for a considerable portion of the CVD risk among AI populations (Lee et al., 1995, 2002;CDC, 1998, 2001; Welty et al., 2002;Nez Henderson, Jacobsen, Beals, & AI-SUPERPFP Team, 2005). Although each of these is clearly linked with behaviors that increase the risk for CVD, only a few behavior-based randomized controlled trials (RCTs) have been conducted to reduce CVD risk among AI populations (Davis et al., 1999). Interventions targeting undesirable CVD risk profiles, though theoretically appealing, have historically been difficult to implement in AI communities because of their remote location, expense and intensity, lack of local infrastructure, and dearth of health care providers and resources (Teufel-Shone, Fitzgerald, Teufel-Shone, & Gamber, 2009). However, several rigorous behavioral RCTs have recently been implemented in these communities, which are described in other articles in this issue of The Journal of Primary Prevention (see Adams et al., Lee et al., Karanja et al., and Walters et al.).

Innovations in technology and home-based care have led to new paradigms for chronic disease management. Notably, the Institute of Medicine’s report, Crossing the Quality Chasm: A New Health System for the Twenty First Century, has recommended a shift from provider-and clinic-centered care based primarily on sporadic office visits toward “care based on continuous healing relationships…not just face-to-face visits” (Committee on Quality of Health Care in America, 2001).

In response to the documented need to test whether behavioral interventions can mitigate the excess CVD risk that AI populations experience, the National Heart, Lung, and Blood Institute (NHLBI) initiated a Request for Applications titled “Community-Responsive Interventions to Reduce Cardiovascular Risk in American Indians and Alaska Natives” (RFA-HL-06-002). The primary aim of this initiative is to conduct 5-year studies in AI and Alaska Native (AN) communities to test the effectiveness of behavioral interventions designed to promote the adoption of healthy lifestyles and/or to improve behaviors related to CVD risk, such as weight reduction, regular physical activity, and smoking cessation.

In this article, we provide and explain the rationale for the Lakota Oyate Wicozani Pi Kte (LOWPK) trial design, and describe the analysis plan.

Objectives

The primary objective of the LOWPK trial is to test whether a remote, primarily Web-based behavioral intervention can reduce CVD risk factors in remote reservation-dwelling AI adults with type 2 diabetes at high risk for CVD, but free of CVD at baseline (i.e., no history of angina, coronary artery disease, or cerebrovascular disease). Specifically, our primary outcome variable is change in glycosylated hemoglobin (HbA1c) level at an average follow-up of 18 months. Our secondary outcome variables are changes in systolic blood pressure (SBP), low-density lipoprotein cholesterol (LDL-C), body mass index (BMI), and smoking status, as well as intervention cost-effectiveness.

Theory Underlying the Intervention

Co-management programs based on a social cognitive view of self-regulated learning are feasible (Goldberg, Ralston, Hirsch, Hoath, & Ahmed, 2003). Such programs should enhance disease knowledge, and facilitate the processes of self-observation, self-judgment, and self-reaction. The process of self-regulation must be incorporated into patients’ daily lives. Perceptions of self-efficacy—the extent to which a person believes he or she can successfully complete specific tasks in a given situation—serve as a “thermostat” that regulates the self-regulation process (Clark & Zimmerman, 1990). Patients with diabetes, for example, need to be able to observe and judge the influence of diet, exercise, and medication on blood sugar control in order to self-manage their illness by adjusting food intake, level of physical activity, and/or medications. Provider feedback and coaching that are relatively immediate and ongoing have the greatest likelihood of enhancing patients’ ability to self-regulate.

Trial Design

Overview

To test the effectiveness of a Web-based patient-centered approach to CVD risk-factor reduction among AI adults with type 2 diabetes, we designed a two-arm RCT (Fig. 1). The intervention group receives a home-based collaborative or “co-management” program that is conducted via the Internet and telephone with a certified physician’s assistant (PA-C) and a registered dietitian. The control group receives usual care, along with the same computing system and Internet access provided to the intervention group. All eligibility checks, basic computer training, and baseline data collection were completed prior to randomization. The study and its procedures were formally approved by the participant Tribe; the Aberdeen Area IHS, University of Washington and University of Colorado Denver institutional review boards (IRBs); and an external independent protocol review committee formed in concert by the grantee and NHLBI. In addition, informed consent was obtained from all individuals who were interested in participating in the study, not only those who were ultimately randomized; this was done so that study staff could access protected health information to fully ascertain study eligibility. Field clinic staff were responsible for recruitment, consent obtainment, and data collection; they were not directly involved in the randomization procedure, and they were purposefully blinded to group assignment.

Fig. 1.

Fig. 1

Overview of the LOWPK study design

Changes to Protocol

Unanticipated circumstances encountered after the study began necessitated several changes to our original study protocol. In this section, we describe two protocol-related changes to our inclusion criteria and one related to the study outcomes. Minor adjustments to the intervention delivery are described in context.

We originally planned to include only those adults who had previously participated in the Education and Research Towards Health (EARTH) Study, conducted on the Cheyenne River from 2003 to 2006 (Slattery et al., 2007). The reason for this was the substantial amount of baseline data available for these individuals, which would allow for both efficient characterization of baseline CVD risk and highly targeted recruitment efforts. However, within 6 months of starting recruitment, it became clear that there were too few potentially eligible participants in the EARTH Study sample to satisfy our targeted recruitment number. After receiving IRB approval, we opened the trial to adults who had not participated in the EARTH Study but who in every other way met our original inclusion criteria.

After 12 months of recruitment, we also relaxed the requirements necessary to meet the prerequisite conditions of hypertension and/or hyperlipidemia. Initially, we had required formal diagnoses of either of these conditions, reflected on the potential participant’s IHS medical record. However, prior research has shown a tendency for a significant delay in the diagnosis of either condition among AI populations (Welty et al., 1995; Lee et al., 2000). In addition, former IHS providers on the project team noted the historically poor charting of IHS providers to add and/or update diagnoses to the antiquated Resource Patient Management System used by the IHS. Therefore, we changed the inclusion criteria to define pre-existing (1) hypertension as any blood pressure recording found on the individual’s IHS medical record on at least one occasion in the past 12 months to be greater than 130/80 mmHG plus an average blood pressure greater than 130/80 mmHG via direct automated testing (HEM-907XL, Omron Healthcare, Kyoto, Japan) at the time of trial screening at our field clinic site and (2) hyperlipidemia as any prior documentation of a fasting LDL level greater than 100 mg/dl on the individual’s IHS medical record, or their EARTH Study clinical record, which included a fasting tabletop determination of lipids and blood glucose.

Finally, the extended period of rolling recruitment, coupled with the challenging and slowed pace of recruitment itself, necessitated a change from a planned average 36-month follow-up period to an average 18-month follow-up period.

Study Population

Eligibility Criteria

To be eligible for the study, participants must:

  1. Be ≥20 years of age

  2. Have completed a baseline EARTH Study examination (criterion later dropped)

  3. Be eligible for and primarily receive health care from the Eagle Butte IHS

  4. Not be pregnant, if female

  5. Have type 2 diabetes mellitus and either hypertension and/or hyperlipidemia (criterion involving co-morbid conditions later clarified)

  6. Be CVD-free at baseline (i.e., have no history of angina, coronary artery disease, or cerebrovascular disease)

  7. Be able to read and understand English

  8. Be able to walk

  9. Not be currently receiving active treatment for any non–skin cell cancer

  10. Not have any condition that would likely significantly interfere with trial participation (e.g., cognitive impairment, chronic renal failure requiring dialysis, multiple sclerosis)

  11. Not be presently indebted to the Cheyenne River Sioux Tribe (CRST) Telephone Authority, or be living in the same household with someone who is, and

  12. Be willing to provide written informed consent for all study procedures

Setting

The LOWPK study is set on the Cheyenne River Sioux Indian Reservation in north-central South Dakota, in partnership with the CRST, the EARTH Study (a National Cancer Institute–funded cohort study conducted on the Cheyenne River from 2003 to 2006), the Eagle Butte Public Health Service Indian Hospital, the CRST Telephone Authority, the Universities of Colorado and Washington, the Seattle-based Group Health Research Institute, and the project office at the NHLBI. Participants for this trial include AI adults, primarily Lakota Sioux, who reside on or near the Cheyenne River Sioux Reservation, and who are eligible for and principally receive health care from the IHS hospital and clinic located on that reservation.

The Cheyenne River Sioux Reservation was established by the Indian Appropriations Act of March 2, 1889 (1889), which divided the Great Sioux Reservation into six smaller reservations. The reservation encompasses 2.8 million square acres, roughly equal to the size of the state of Connecticut. It consists of rolling prairie land, timbered river bottoms, lake edges, rivers and creeks, rough river breaks, river valleys, buttes, and some slight badlands. Eagle Butte is the center of the Tribal government, and although the Tribe provides government services throughout the reservation, many federal and tribal services are available only in Eagle Butte. The reservation is made up of six representative districts, each with its own community building. District councils have an advisory role in the Tribal government. Residents from the outlying 17 communities often travel to Eagle Butte for federal and tribal services.

Recruitment and Enrollment

During the initial phase of recruitment, participants from the EARTH Study who were age-eligible for the LOWPK study were contacted by a research study assistant by phone and by mail with information about this study. Newspaper and television advertisements were also used to reach out to EARTH Study participants. However, due to low recruitment, after 6 months we opened the study to persons who did not participate in the EARTH Study, but who were otherwise eligible. Recruitment approaches included continued print, radio, and television advertising; discussions with health care providers on the reservation, including the CRST diabetes program staff (who ultimately shared their tribal diabetes registry with us); and word-of-mouth.

Figure 2 outlines the process from the initial clinic visit to randomization. The eligibility screening described above takes place during two screening visits at the study field clinic in Eagle Butte. During the first screening visit, the study nurse and research assistant explain the study procedures to the potential participant and obtain signed informed consent. Self-report and verified medical record data from IHS and/or the EARTH Study are used to preliminarily assess inclusion criteria 1, 2, 3, 5, and 6 above. For criterion 4, we used EARTH Study and/or IHS data that were available on menopause status for potential female participants, as well as a urinary beta-hCG test if needed (through the Eagle Butte IHS lab). A brief medical record review combined with interview is undertaken to document that individuals have remained CVD-free either since their initial EARTH Study examination or ever in the case of non–EARTH Study participants (criterion 6); that they continue to have hypertension and/or hyperlipidemia (criterion 5); and that they are not receiving active treatment for a non–skin cell cancer (criterion 9). We verify the status of criterion 6 through repeat questioning, electronic medical record review and, if needed, by electrocardiogram (ECG) confirmation. Criteria 7, 8, and 9 are verified by repeat questioning and observation using a standard checklist. Criterion 10 is assessed through a screening questionnaire and medical record review. Criterion 11 is assessed by the CRST Telephone Authority, a major study collaborator, who receives the names of prospective participants from the study field clinic.

Fig. 2.

Fig. 2

Flowchart of the LOWPK trial enrollment procedure

Physical examinations, including blood pressure tests, anthropometric measures, and ECGs, are also performed during the first screening visit. A clinical referral algorithm was developed to be used in concert with the physical examination. Briefly, the LOWPK nursing staff determines the acuteness of the findings, as well as whether or not the condition is being monitored by a physician. When the potential participant is aware of and being followed medically for a condition, discretion is used to determine if a medical referral is needed. The standard IHS referral form or other written summary is used to provide appropriate clinical information to the health care professional who will evaluate the patient. A copy of this referral is retained with the research forms to document the referral that was made. Specific examples of the referral algorithm include an “emergent” referral (immediate transport to nearest medical facility) for SBP greater than 200 mmHG, or an “immediate” referral (a same-day appointment/referral) for SBP 180–199 mmHG. Other referral criteria involve predefined criteria for diastolic blood pressure and blood glucose, as well as acute cardiovascular and neurologic symptoms (e.g., angina, heart failure, claudication, transient ischemic attack).

Requisite Computer Training

During the first screening visit, potential participants also complete a brief computer experience screening questionnaire developed by the project’s computer trainer in order to schedule them to an appropriate-level computer training session, or to give them the opportunity to test out of the basic training by completing instructions on a sheet provided by the computer trainer. Potential participants are then scheduled to participate in at least 4 h of basic computer instruction in order to learn how to minimally operate and understand the computer systems that are delivered and installed in their homes for their use. In practice, as many hours of computer instruction as necessary are provided to potential participants to help them attain sufficient proficiency (as assessed by the trainer) in using their personal computers.

A multi-stage test-out procedure was developed for potential participants who believe they have the requisite computer experience to not need the basic training. To test out, the participant must (1) type a URL address from an instruction sheet into their Internet browser, (2) navigate to that Web page, (3) copy text from that Web page and insert it into a new Microsoft Word document, and (4) e-mail the document as an attachment to the computer trainer. Few potential participants have been either willing to attempt or have successfully tested out of the basic computer training. All participants are also trained to use provided personal computers to access health-related Internet knowledge resources. Materials and links to knowledge resources regarding diabetes and CVD risk factors were collected and preloaded on participants’ computers. Unlike information generally available on the Web, however, all content is recommended to participants as having been sanctioned by the Medical Director of the University of Washington’s Diabetes Care Center. (Additional computer training provided to the intervention group is described in the “Intervention” section).

At the end of each basic computer training session, or upon completion of the test-out procedure, the participant is then scheduled for an appointment for the second clinic visit. Before the second clinic visit, the participant picks up, or the LOWPK study staff deliver, a 24-h urine collection container and the paper-based instruments (i.e., baseline survey, slightly modified [addition of several tribal foods] Block Food Frequency Questionnaire [FFQ], and present medications form) to take home and return at the next clinical visit.

The second clinic visit, which takes place on average 14 days after the first clinic visit, includes fasting blood draws, return of the 24-h urine sample and subsequent processing of the urine specimen, and return of the paper-based instruments. On average, this clinical visit takes about 40 min to complete. Participants who do not complete their baseline surveys in advance have the opportunity to do so during the clinic visit.

The second clinic visit also affords the LOWPK study staff the opportunity to examine and test each prospective participant’s blood glucometer device. The devices are checked for proper settings and calibration. If any potential participant either does not have a glucometer or if the one that they have is inoperative, they are referred to the Tribal Diabetes Nurse who is able to help them obtain one, along with the requisite test strips.

The final steps in the recruitment and eligibility process are completion of the CRST Telephone Authority application and signing of media release documentation. The Telephone Authority application is essentially a fiscal review performed to ensure that the prospective participant does not have an unpaid past telephone debt, or that they sign an agreement to pay such a debt on an agreed-upon schedule. Once the Telephone Authority application has been approved, participants are notified of eligibility and randomized.

Computer Delivery and Installation

Computer delivery and installation occur after a participant has been randomized to either intervention or usual care. Computers for intervention and control participants are identical (Dell Optiplex 740 w/AMD 64 Processor with Windows Vista OS, productivity, and anti-virus software), with the exception of certain software packages and programs described in the “Intervention” section. The CRST Telephone Authority delivers and installs the computer to a participant’s home, verifies that the correct software is installed, confirms satisfactory Internet access, and completes tracking paperwork. The installation technician also records geographic coordinates for the participant’s residence. The vast majority of our trial participants access the Internet through copper-wire dialup Internet access. Few participants are paying, or can afford to pay, for broadband access, whether by DSL in and immediately around the central town of Eagle Butte or via Wild Blue satellite broadband outside of Eagle Butte. Participants are required to maintain and pay for basic telephone service, and the study fully covers the cost of their subsidized (by the CRST Telephone Authority) Internet access.

Biological Sample Handling

At each time point, all blood and urine samples are labeled with a unique number that is unrelated to a participant’s study identification number. The LOW-PK study staff who draw the blood, receive the urine specimens, and label these samples are blinded to the participants’ assigned group. We will conduct all assays at the end of the study so that all of an individual’s samples can be assayed in the same batch. This reduces the likelihood that observed changes in biomarker levels in an individual over the course of the study are due to inter-batch assay variation (Tworoger et al., 2004). In addition, we plan to batch samples such that, within each batch, subject randomization dates are similar, and the number of intervention and control participants is balanced. Laboratory staff will be blinded with respect to the identity of all samples.

Intervention

Intervention Group Software

Computers delivered to participants in the intervention group include the Internet co-management module Diabetes Partner™ software (NuMedics, Inc., Portland, OR, USA), and an additional program to enable the computer to recognize the USB cable for their glucometers, thereby enabling the Web-based upload of glucometer results to a participant’s own Diabetes Partner™ profile. Diabetes Partner™ is the corresponding “patient” Web site to the provider-based CliniPro® software package used by our interventionists.

Participants in the control group are not given access to either the Diabetes Partner™ Web site or the related case-management services being evaluated.

Additional Intervention Group Training

Intervention participants have an initial visit with the study’s PA-C and registered dietitian case-managers at the beginning of the intervention period. This visit takes place in a CRST Telephone Authority training center in Eagle Butte with broadband Internet access. Here, intervention participants are trained in the use of the Internet diabetes co-management module (Diabetes Partner™) during this ~2-h session, and each receives a detailed user-documentation manual. Originally developed for the type 2 diabetes pilot test at the University of Washington, the manual has been updated and modified for use in the LOWPK study. This initial visit also affords intervention participants the opportunity to upload their first glucometer readings to their Diabetes Partner™ profile, which then links to the clinicians’ corresponding CliniPro® clinical interface, and to learn how to use a Web-based version of CalorieKing™ (Borushek, 2005), a diet diary software program. The PA-C, registered dietitian, and intervention participant then collaboratively generate a first short-term, achievable action plan meant to enhance self-efficacy by teaching problem-solving skills. The inclusion of such plans in patient education efforts appears to be a marker for success in improving clinical outcomes, especially among less-motivated individuals distracted by competing life problems not directly related to their chronic illnesses (Goldberg, Lessler, Mertens, Eytan, & Cheadle, 2004). Intervention participants are welcomed to voluntarily schedule a return visit for “refresher” computer trainings; roughly half of our intervention participants chose to do so.

Intervention Delivery

Location, Training, and Supervision of the PA-C and Registered Dietitian

The LOWPK trial involves two skilled clinicians to staff the Web-based diabetes co-management intervention, a PA-C and a registered dietitian. Each is directly supervised by the principal investigator (PI). During the planning year, the PA-C and the PI developed protocols for clinical management; for example, standing orders, referral patterns, medication dosage changes, and interactions with clinical providers at the Eagle Butte IHS service unit. Treatment goals are consistent with those employed for persons with type 2 diabetes in the Third National Health and Nutrition Examination Survey (NHANES III): an HbA1c level <7 %, a mean blood pressure <130/80 mmHg, an LDL-C level <100 mg/dl, and a total cholesterol level <200 mg/dl (National Center for Health Statistics, 1994). The PA-C provides clinical guidance from Rapid City, SD, about 170 miles from the trial reservation. The PA-C was trained throughout the course of the year-long planning period by NuMedics personnel in the use of the diabetes co-management program/software, the provider-oriented CliniPro®, and the patient-oriented, interfaced Diabetes Partner™ companion Web site.

The registered dietitian and the PI also worked together during the planning year to develop protocols for nutritional management, including referral patterns, interactions with the IHS dietitian, and food prescriptions. The registered dietitian is also located and works from Rapid City, SD. The registered dietitian was also trained throughout the course of the year-long planning period by NuMedics personnel in use of CliniPro® and Diabetes Partner™ simultaneously with the PA-C, with particular attention to the program’s nutrition-oriented aspects.

Case Management by Study Interventionists

Initially, intervention participants are asked to upload their glucometer readings from home at least once weekly via serial-port connections to their computers. The PA-C reviews the IHS medical record activity of participants monthly to check for the appearance of new data and then emails feedback to the participants after the review. During periods of active insulin adjustment, for example, the PA-C may encourage the participant to upload glucose readings more frequently and thus increase the frequency of feedback accordingly. The PA-C assists participants with interpreting values and in jointly agreeing on the next steps needed to reach treatment goals including adjusting medication dosages or modifying action plans. The PA-C, along with the registered dietitian, if needed, reviews weekly nutrition logs and emails feedback to participants. When appropriate, both clinicians counsel participants in smoking cessation using motivational interviewing. During the year-long planning period, both clinicians received training in the use of motivational smoking cessation techniques from a Black Hills Center for American Indian Health (BHCAIH) project coordinator with expertise in tobacco cessation methods among AI populations (Nez Henderson et al., 2005).

Intervention participants who asynchronously exchange data with trial interventionists over the Internet between scheduled IHS visits have the opportunity for greatly increased (i.e., more frequent and timely) interactions with health care providers. As is standard practice for any therapy change initiated following an intercurrent visit or telephone exchange with clinical staff other than the primary provider, any change in therapy resulting from trial procedures is communicated to the primary health care provider of record as indicated.

The PA-C and registered dietitian manage care of intervention participants remotely, primarily over the telephone. In practice, this has been the preferred method of communication for many intervention participants versus e-mail. In addition, the registered dietitian sends a quarterly newsletter to only the intervention participants with seasonally relevant information about diet and health.

Because potential contact between study field clinic visits is a specific component of the intervention, no enhanced communication with control participants between visits is planned (for example, via phone, mail, or e-mail), other than those contacts that might ordinarily occur as part of the usual care that control participants continue to receive. The sole exception to this is monthly study newsletters highlighting general individual and family health messages (e.g., holiday eating, food handling, winter preparedness, etc.) sent from the BHCAIH offices to all study participants.

Outcomes

The LOWPK trial’s primary outcome variable is change in serum HBA1c level over an average follow-up of 18 months. Change in HbA1c level was chosen as the primary outcome variable for several reasons: (1) A recent meta-analysis showed that there is a relationship between HbA1c level and CVD in persons with diabetes (Selvin et al., 2004),which is stronger in persons with types 2 diabetes than in those with type 1 diabetes; (2) HbA1c level has been and is still being routinely measured every 3 months during the usual clinical care of participants, so they either are or should be most familiar with this measure among various clinical measures; and (3) case management and team approaches have been shown to significantly impact HbA1c levels in other case management trials for CVD risk reduction (Gaede et al., 2003; Rothman et al., 2005). Our secondary outcome variables are changes in SBP, LDL-C, BMI, and smoking status, as well as intervention cost-effectiveness. Though this trial is at its core a behavioral trial, smoking status per se is our only discrete behavioral outcome variable. This is both because of smoking’s direct relationship to CVD and because of the lack of a general consensus supporting clinical counseling approaches to other behavioral changes that might influence cardiovascular risk and CVD.

Summary of Study Measures

The LOWPK trial collects data on risk factors, confounding and mediating factors, and cardiovascular function, including standard self-report measures of risk factors for CVD as well as results from clinical and laboratory tests. Table 1 lists key clinical and sociodemographic variables, their method of assessment, and their role in the analysis. All questionnaire data are collected via self-administered survey instruments using an easy-to-read, standardized format suitable for later scanning. Persons requiring assistance can elect to have a question(s) read and/or explained to them.

Table 1.

Summary of study measures and evaluation time points in the LOWPK trial

Data element Instrument/evaluation protocol Length in
minutes
Purpose Collection
time points
Cardiovascular disease risk factors
Blood pressure Clinical measurement 5 2° outcome Baseline
6 months
18 months
Anthropometric measures (e.g., body mass index) Clinical measurement 5 2° outcome Baseline
6 months
18 months
Fingerstick blood glucose Participant measurement 1 2° outcome Continuous self-monitoring
Blood lipids Laboratory measurement Blood draw for all labs 2° outcome Baseline
6 months
18 months
Hemoglobin A1c Laboratory measurement 1° outcome Baseline
6 months
18 months
Sociodemographic information
Age
Education
Marital status
Social support
Self-administered questionnaire 2 Potential confounders and/or effect modifiers Baseline
6 months
18 months
Functional/mood constructs
Short Form-12 (Ware et al., 1996)
Depressive scale (Radloff, 1977)
Self-efficacy
(Clark & Dodge, 1999)
Self-administered questionnaire 15 Potential confounders and/or effect modifiers Baseline
6 months
18 months
History of diabetes, hypertension, dyslipidemia Self-administered questionnaire 2 Potential confounders and/or effect modifiers Baseline
6 months
18 months
Behavioral risk factors
Smoking
Physical activity
Alcohol consumption
Modified block Food frequency Questionnaire
Behaviors
Self-administered questionnaire 15 Potential confounders and/or effect modifiers
2° outcomes
Baseline
6 months
18 months
Cultural factors
Acculturation
Ceremonial tobacco use
Traditional health practices
Self-administered questionnaire 2 Potential confounders and/or effect modifiers Baseline
6 months
18 months
Current medication use Interview 2 Potential confounders and/or effect modifiers Baseline
Cost-related measures
Intervention costs
Medical service use
Self- and interviewer-administered; electronic medical record query 10 2° outcomes Baseline
6 months
12 months
18 months
Geocoding Staff measurement <1 Potential confounder and/or effect modifier in cost analyses Baseline

Minimal Detectable Differences and Sample Size

We estimated the minimal detectable mean difference for the primary and each of the secondary outcome variables based on the following assumptions: (1) a two-tailed alpha of 0.05; (2) 80 % statistical power; (3) a total sample size of 180, equally divided between the intervention and control groups; (4) a 10 % attrition rate; and (5) a two-group intention-to-treat analysis of mean differences. The sample size of 180 was selected based on an expected 60 % participation rate among those with type 2 diabetes who are from the Cheyenne River EARTH Study site. In calculating the detectable differences, we used estimated means and variances for the primary and secondary outcomes from both our own EARTH Study data and published literature. We will have 80 % power to detect a mean 1 unit difference in HbA1c level, which is our primary outcome variable (Table 2). All other mean differences are consistent with potential treatment effects observed in previous intervention studies, except for BMI: A 9 % difference in BMI is ambitious and may not be achievable.

Table 2.

Minimal detectable mean differences with 80 % statistical power and a total sample of 180 participants with type 2 diabetes

Measure Estimated mean
in the control
group
Minimally
detectable
mean difference
Hemoglobin A1c 8.4 mmol/l   1
Systolic blood pressure 138 mmHG   8
Diastolic blood pressure 85 mmHG   5
Low-density lipoprotein cholesterol 104 mg/dl 14
Body mass index 33 kg/m2   3

Randomization

Randomization occurs after potential participants have met all eligibility criteria, passed basic computer training, and completed baseline data collection. In order to ensure similar distributions of age and sex across the intervention and control groups, we implemented a blocked and stratified assignment procedure. Participants were stratified by sex and age (20–39 years; 40–49 years; 50–59 years; 60–69 years; 70 or more years of age). For each of the ten age–sex strata, we generated a list of group assignments, randomizing participants in blocks of four, to be consulted as participants enrolled sequentially in the trial. This list is concealed from the project manager, PI, and the field staff. Each week, the project manager in South Dakota calls the biostatistician in Seattle to obtain the random group assignment for eligible participants based on the previously generated list. The statistician records the assignments in ink in a bound notebook as the official record. The project manager then contacts the participant by phone and letter to inform him or her of his or her group assignment, and informs the intervention staff of the identity of participants assigned to their group. Field clinic personnel responsible for screening and examinations (outcome assessment) are not informed of the participants’ group assignments.

Blinding

Members of the field clinic staff, namely those responsible for outcome assessment, are blinded to group assignment of the participants. Additionally, the trial protocol calls for central laboratory personnel, analytic staff, and data management staff at the University of Washington to also be blinded to participants’ group assignments.

Data Management

Data management is coordinated and performed at the University of Washington (in Seattle) for two primary reasons: (1) because it houses the data management and analysis cores for both a National Institute on Aging-and an Agency for Healthcare Research and Quality-funded center on AI/AN health disparities and (2) because of their familiarity and experience with AI/AN health disparities research.

Clinical data are recorded on standardized forms in the field, which are then copied for the participant file before the originals are sent to the University of Washington for manual data entry directly into a customized Microsoft Access database. For quality control purposes, a 10 % double entry standard is used for all trial-related data entry, in addition to standard range-and error-checking functions available within the Microsoft Access program. Subsequently, a SAS master file will be generated with the new data to be provided to the biostatistician for analysis.

Statistical Methods

Clinical Outcomes

All analyses will be performed according to intention-to-treat principles (i.e., all participants will be analyzed in the treatment arm to which they are randomized). Our primary outcome variable is change in HbA1c level over an average follow-up of 18 months, and our secondary outcome variables are changes in SBP, LDL-C, BMI, smoking status, and intervention cost-effectiveness.

First, we will use the t test to compare the mean values of each continuous outcome variable and the Chi-square test to compare the categorical outcome variable (e.g., smoking status) between the two study groups at each time point. These analyses will provide an initial indication of whether the intervention was associated with a change in our primary outcome variable or secondary outcome variables at any point during the study, even if the change was not necessarily maintained through the end of the intervention period. For our main analysis, we will use Generalized Estimating Equations (GEE) models with the identity link for continuous outcome variables and the logit link for categorical outcome variables to evaluate the intervention’s overall effect on the primary outcome variable and secondary outcome variables. Outcome variables will be the values measured at different time points. For blood pressure measurements, however, the average of three values measured at the same time point will be used in the model (SBP and diastolic blood pressure). Each outcome model will include an indicator variable for intervention group assignment, an indicator variable for each follow-up time point, and interaction terms between intervention and time point variables. The GEE models will assume an unstructured correlation matrix with robust standard error estimates. Models will be adjusted for potential confounders that were not equally distributed by the randomization process. Planned analyses will use STATA10 (StataCorp, College Station, TX, USA).

Cost Analyses

For this trial, we will use program cost, program savings, and net program cost estimates to construct cost indicators such as the cost/net cost per patient served. Cost-effectiveness analysis is used to compare costs of alternative interventions for a similar health outcome (Gold, Siegel, Russell, & Weinstein, 1996; Haddix, Teutsch, & Corso, 2002). We will calculate a cost-effectiveness ratio (CER), the net cost per health outcome, for both the intervention and control groups. We will calculate the net cost per unit decrease of HbA1c level, an intermediate CER, and net cost per unit increase in quality-adjusted life year, a final CER. The quality-adjusted life years will be estimated from SF-12 data, using published algorithms (Brazier & Roberts, 2004). We will also calculate incremental CERs, which measure the differences in net costs and effectiveness between the intervention and control groups. Decision-analysis software and second-order Monte Carlo probabilistic sensitivity analyses will be employed to examine the influence of variability in all cost parameters on the estimated cost outcomes and to provide a 95 % credible range for each estimated cost outcome (Sullivan, Buist, & Weiss, 2003; O’Connell, Brunson, Anselmo, & Sullivan, 2005).

It is important to note that we will estimate only short-term net program costs, not long-term costs that may result with improved participant self-management. To address this limitation, we will situate our findings to those in the published literature on the relationship between changes in clinical status (e.g., reductions in HbA1c level) and long-term outcomes (e.g., lower prevalence of CVD) in diabetes care to discuss the implications of our findings (Wagner et al., 2001; CDC Diabetes Cost-Effectiveness Group, 2002; Palmer et al., 2004; Zhang, Engelgau, Norris, Gregg, & Narayan, 2004).

Discussion

The LOWPK trial is an RCT designed to determine whether a Web-based diabetes and nutritional intervention, which has previously been found to improve diabetes control in disadvantaged inner-city populations (Goldberg et al., 2004), can significantly reduce CVD risk factors among a group of adult AI men and women with type 2 diabetes who are at high risk for CVD. The primary endpoint is change in HbA1c level after an average 18-month follow-up period. Secondary endpoints include changes in LDL-C, SBP, BMI, and smoking status; these outcome variables are included because they are well-known CVD risk factors.

This trial is unique in that it is the one of the first RCTs to be implemented in a remote reservation setting and the only one to employ a Web-based intervention. Its numerous strengths include the highly experienced field staff team; the familiarity of this AI community to health research (prior EARTH Study and ongoing Strong Heart Study community [Lee et al., 1990]); the refined, detailed trial protocol; and the collaboration and support of the IHS provider staff. If effective, this intervention may significantly reduce cardiovascular morbidity, reduce health care costs, and improve the quality of life for remote reservation– dwelling AI populations. It may also point toward a potentially highly significant evolution in the health care delivery system for reservation-based AI, as well as remote-dwelling AN, communities with chronic disease. The CliniPro® platform, being used by the University of Washington’s Roosevelt Outpatient Clinics, estimates that a well-trained mid-level provider can capably and effectively care for a panel of 2,000 persons with type 2 diabetes.

The uniqueness of the study population, which includes adult AI men and women residing on a very remote, socioeconomically disadvantaged AI reservation, is important. AI populations have been greatly underrepresented in RCTs; therefore, a significant aspect of this trial is whether these interventions are effective in or even feasible with this population. Our first 2 years in the field have already revealed significant challenges with feasibility, including (1) lack of fiber cable to support broadband Internet access, (2) very uneven cell phone coverage across the reservation, and (3) significant credit-worthiness issues with our participants, impacting the continuity of basic telephone service. These factors and others that are unmeasured have also resulted in a much slower pace of recruitment than originally anticipated. Nonetheless, our participants, IHS provider staff, and tribal leadership are all very pleased with the conduct of this trial in their community.

This study has several additional limitations. First, our study is both limited and complicated by the differing intensity and duration of the intervention. The first 2 years have revealed that there are some participants who quickly engage with our provider staff whereas others seem very reluctant or otherwise unable to effectively engage with them, despite concerted efforts to promote such engagement. Second, our study population is relatively unique for its frontier remoteness and profound socioeconomic disadvantage. One of the two counties that make up our participant tribal reservation, Ziebach County, has for years been widely acknowledged as one of the United States’ poorest counties on the basis of US Census Bureau data. Our results may not be applicable to other AI groups, or to urban or less remotely living AI/AN populations.

The lessons learned through the conduct of LOWPK are likely to have important implications for the design and conduct of future chronic disease intervention projects in AI/AN communities, particularly those employing Web-based strategies in remote frontier locations. Only through cumulative research efforts, which build sequentially upon each other, can we begin to reverse the rising tide of CVD among the AI population (Howard et al., 1999) and, in so doing, return the members of this special population to a path of wellness.

Acknowledgments

The Lakota Oyate Wicozani Pi Kte study was funded by the National Heart, Lung, and Blood Institute (UO1 HL087422). The contents of this manuscript are solely the responsibility of the authors and do not necessarily represent the official view of the National Heart, Lung, and Blood Institute. We acknowledge the contributions and support of the Indian Health Service, the Cheyenne River Sioux Tribe; staff of the University of Washington, including Corinne Hunt, Phu T. Van, Andrew Bogart, Carolyn Noonan, and Odile Lallemand; Dorene Levie and Khazi Ahmed of NuMedics, Inc.; staff associated with the Black Hills Center for American Indian Health, including Monique Giago, Stephanie Big Crow and Crissy Whitewolf; and staff of Missouri Breaks Industries Research, Inc., including Marcia O’Leary, Kendra Enright, Anne Chasing Hawk, Marie Gross, Jay Kunf, and Lillian Brown. We also acknowledge Lifescan, Inc., who provided replacement One Touch Ultra glucometer devices and USB cables (to connect the glucometers to participant computers) to the study at no cost.

Footnotes

Disclaimer: The opinions expressed in this paper are those of the author(s) and do not necessarily reflect the views of the Indian Health Service or National Heart, Lung and Blood Institute.

Contributor Information

Jeffrey A. Henderson, Email: jhenderson@bhcaih.org, Black Hills Center for American Indian Health, 701 St. Joseph St., Suite 204, Rapid City, SD 57701, USA.

Jessica Chubak, Group Health Research Institute, 1730 Minor Avenue, Suite 1600, Seattle, WA, USA.

Joan O’Connell, Department of Community and Behavioral Health, Colorado School of Public Health, University of Colorado Denver, Aurora, CO, USA.

Maria C. Ramos, Black Hills Center for American Indian Health, 701 St. Joseph St., Suite 204, Rapid City, SD 57701, USA

Julie Jensen, Black Hills Center for American Indian Health, 701 St. Joseph St., Suite 204, Rapid City, SD 57701, USA.

Jared B. Jobe, Division of Cardiovascular Sciences, National Heart, Lung, and Blood Institute, Bethesda, MD, USA jobej@mail.nih.gov; jljobe@hotmail.com.

References

  1. Borushek A. The Calorie King® calorie, fat & carbohydrate counter. Costa Mesa, CA: Family Health Publications; 2005. [Google Scholar]
  2. Brazier JE, Roberts J. The estimation of a preference-based measure of health from the SF-12. Medical Care. 2004;42:851–859. doi: 10.1097/01.mlr.0000135827.18610.0d. [DOI] [PubMed] [Google Scholar]
  3. CDC Diabetes Cost-Effectiveness Group. Cost-effectiveness of intensive glycemic control, intensified hypertension control, and serum cholesterol level reduction for type 2 diabetes. Journal of the American Medical Association. 2002;287:2542–2551. doi: 10.1001/jama.287.19.2542. [DOI] [PubMed] [Google Scholar]
  4. Centers for Disease Control and Prevention. Tobacco use among U.S. racial/ethnic minority groups, African Americans, American Indians and Alaska Natives, Asian Americans and Pacific Islanders, Hispanics: A report of the Surgeon General (Executive summary) Morbidity and Mortality Weekly Report. 1998;47(No. RR-18):v–xv. 1–16. [PubMed] [Google Scholar]
  5. Centers for Disease Control and Prevention. Cigarette smoking among adults—United States, 1999. Morbidity and Mortality Weekly Report. 2001;50:869–873. [PubMed] [Google Scholar]
  6. Centers for Disease Control and Prevention. Disparities in premature deaths from heart disease—50 States and the District of Columbia, 2001. Morbidity and Mortality Weekly Report. 2004;53:121–125. [PubMed] [Google Scholar]
  7. Clark NM, Dodge JA. Exploring self-efficacy as a predictor of disease management. Health Education & Behavior. 1999;26:72–89. doi: 10.1177/109019819902600107. [DOI] [PubMed] [Google Scholar]
  8. Clark NM, Zimmerman BJ. A social cognitive view of self-regulated learning about health. Health Education Research. 1990;5:371–379. doi: 10.1177/1090198114547512. [DOI] [PubMed] [Google Scholar]
  9. Committee on Quality of Health Care in America, Institute of Medicine. Crossing the quality chasm: A new health system for the 21 st century. Washington, DC: The National Academies Press; 2001. [Google Scholar]
  10. Davis CE, Hunsberger S, Murray DM, Fabsitz RR, Himes JH, Stephenson LK, et al. Design and statistical analysis for the Pathways Study. American Journal of Clinical Nutrition. 1999;69(4 Suppl):760S–763S. doi: 10.1093/ajcn/69.4.760S. [DOI] [PubMed] [Google Scholar]
  11. Gaede P, Vedel P, Larsen N, Jensen GV, Parving HH, Pedersen O. Multifactorial intervention and cardiovascular disease in patients with type 2 diabetes. New England Journal of Medicine. 2003;348:383–393. doi: 10.1056/NEJMoa021778. [DOI] [PubMed] [Google Scholar]
  12. Galloway JM. Cardiovascular health among American Indians and Alaska Natives: Successes, challenges, and potentials. American Journal of Preventive Medicine. 2005;29(5) Suppl 1:11–17. doi: 10.1016/j.amepre.2005.07.023. [DOI] [PubMed] [Google Scholar]
  13. Gold MR, Siegel JE, Russell LB, Weinstein MC, editors. Cost-effectiveness in health and medicine. New York, NY: Oxford University Press; 1996. [Google Scholar]
  14. Goldberg HI, Lessler DS, Mertens K, Eytan TA, Cheadle AD. Self-management support in a web-based medical record: A pilot randomized controlled trial. Joint Commission Journal on Quality and Patient Safety. 2004;30:629–635. 589. doi: 10.1016/s1549-3741(04)30074-2. [DOI] [PubMed] [Google Scholar]
  15. Goldberg HI, Ralston JD, Hirsch IB, Hoath JI, Ahmed KI. Using an internet comanagement module to improve the quality of chronic disease care. Joint Commission Journal on Quality and Patient Safety. 2003;29:443–451. doi: 10.1016/s1549-3741(03)29053-5. [DOI] [PubMed] [Google Scholar]
  16. Haddix AC, Teutsch SM, Corso PS, editors. Prevention effectiveness: A guide to decision analysis and economic evaluation. New York, NY: Oxford University Press; 2002. [Google Scholar]
  17. Howard BV, Lee ET, Cowan LD, Devereux DB, Galloway JM, Go OT, et al. Rising tide of cardiovascular disease in American Indians: The Strong Heart Study. Circulation. 1999;99:2389–2395. doi: 10.1161/01.cir.99.18.2389. [DOI] [PubMed] [Google Scholar]
  18. Indian Appropriations Act of March 2, 1889, 25 U. S. C. § 888. 1889 [Google Scholar]
  19. Indian Health Service. Trends in Indian health, 1998–1999. Rockville, MD: Office of Public Health, U.S. Department of Human Health and Services; 2000. [Google Scholar]
  20. Lee ET, Howard BV, Go O, Savage PJ, Fabsitz RR, Robbins DC, et al. Prevalence of undiagnosed diabetes in three American Indian populations. A comparison of the 1997 American Diabetes Association diagnostic criteria and the 1985 World Health Organization diagnostic criteria: The Strong Heart Study. Diabetes Care. 2000;23:181–186. doi: 10.2337/diacare.23.2.181. [DOI] [PubMed] [Google Scholar]
  21. Lee ET, Howard BV, Savage PJ, Cowan LD, Fabsitz RR, Oopik AJ, et al. Diabetes and impaired glucose tolerance in three American Indian populations aged 45–74 years: The Strong Heart Study. Diabetes Care. 1995;18:599–610. doi: 10.2337/diacare.18.5.599. [DOI] [PubMed] [Google Scholar]
  22. Lee ET, Welty TK, Cowan LD, Wang W, Rhoades DA, Devereux R, et al. Incidence of diabetes in American Indians of three geographic areas: The Strong Heart Study. Diabetes Care. 2002;25:49–54. doi: 10.2337/diacare.25.1.49. [DOI] [PubMed] [Google Scholar]
  23. Lee ET, Welty TK, Fabsitz R, Cowan LD, Le NA, Oopik AJ, et al. The Strong Heart Study: A study of cardiovascular disease in American Indians: Design and methods. American Journal of Epidemiology. 1990;132:1141–1155. doi: 10.1093/oxfordjournals.aje.a115757. [DOI] [PubMed] [Google Scholar]
  24. National Center for Health Statistics. Plan and operation of the third national health and nutrition examination survey, 1988–1994. Hyattsville, MD: US Department of Health and Human Services; 1994. DHHS Publication No. 94-1308. [Google Scholar]
  25. Nez Henderson P, Jacobsen C, Beals J AI-SUPERPFP Team. Correlates of cigarette smoking among selected Southwest and Northern plains tribal groups: The AI-SUPERPFP Study. American Journal of Public Health. 2005;95:867–872. doi: 10.2105/AJPH.2004.050096. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. O’Connell JM, Brunson D, Anselmo T, Sullivan PW. Costs and savings associated with community water fluoridation programs in Colorado. Preventing Chronic Disease. 2005;2:A06. (Special Issue) [PMC free article] [PubMed] [Google Scholar]
  27. Palmer AJ, Roze S, Valentine WJ, Spinas GA, Shaw JE, Zimmet PZ. Intensive lifestyle changes or metformin in patients with impaired glucose tolerance: Modeling the long-term health economic implications of the Diabetes Prevention Program in Australia, France, Germany, Switzerland, and the United Kingdom. Clinical Therapeutics. 2004;26:304–321. doi: 10.1016/s0149-2918(04)90029-x. [DOI] [PubMed] [Google Scholar]
  28. Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1:385–401. [Google Scholar]
  29. Rothman RL, Malone R, Bryant B, Shintani AK, Crigler B, Dewalt DA, et al. A randomized trial of a primary care-based disease management program to improve cardiovascular risk factors and glycated hemoglobin levels in patients with diabetes. American Journal of Medicine. 2005;118:276–284. doi: 10.1016/j.amjmed.2004.09.017. [DOI] [PubMed] [Google Scholar]
  30. Selvin E, Marinopoulos S, Berkenblit G, Rami T, Brancati FL, Powe NR, et al. Meta-analysis: Glycosylated hemoglobin and cardiovascular disease in diabetes mellitus. Annals of Internal Medicine. 2004;141:421–431. doi: 10.7326/0003-4819-141-6-200409210-00007. [DOI] [PubMed] [Google Scholar]
  31. Slattery ML, Schumacher MC, Lanier AP, Edwards S, Edwards R, Murtaugh MA, et al. A prospective cohort of American Indian and Alaska Native people: Study design, methods, and implementation. American Journal of Epidemiology. 2007;166:606–615. doi: 10.1093/aje/kwm109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Sullivan SD, Buist AS, Weiss K. Health outcomes assessment and economic evaluation in COPD: Challenges and opportunities. The European Respiratory Journal Supplement. 2003;41:1s–3s. doi: 10.1183/09031936.03.00077603. [DOI] [PubMed] [Google Scholar]
  33. Teufel-Shone NI, Fitzgerald C, Teufel-Shone L, Gamber M. Systematic review of physical activity interventions implemented with American Indian and Alaska Native populations in the United States and Canada. American Journal of Health Promotion. 2009;23:S8–S32. doi: 10.4278/ajhp.07053151. [DOI] [PubMed] [Google Scholar]
  34. Tworoger SS, Yasui Y, Chang L, Stanczyk FZ, McTiernan A. Specimen allocation in longitudinal biomarker studies: Controlling subject-specific effects by design. Cancer Epidemiology, Biomarkers and Prevention. 2004;13:1257–1260. [PubMed] [Google Scholar]
  35. Wagner EH, Sandhu N, Newton KM, McCulloch DK, Ramsey SD, Grothaus LC. Effect of improved glycemic control on health care costs and utilization. Journal of the American Medical Association. 2001;285:182–189. doi: 10.1001/jama.285.2.182. [DOI] [PubMed] [Google Scholar]
  36. Ware J, Jr, Kosinski M, Keller SD. A 12-item short-form health survey: Construction of scales and preliminary tests of reliability and validity. Medical Care. 1996;34:220–233. doi: 10.1097/00005650-199603000-00003. [DOI] [PubMed] [Google Scholar]
  37. Welty TK, Lee ET, Yeh J, Cowan LD, Go O, Fabsitz RR, et al. Cardiovascular disease risk factors among American Indians: The Strong Heart Study. American Journal of Epidemiology. 1995;142:269–287. doi: 10.1093/oxfordjournals.aje.a117633. [DOI] [PubMed] [Google Scholar]
  38. Welty TK, Rhoades DA, Yeh F, Lee ET, Cowan LD, Fabsitz RR, et al. Changes in cardiovascular disease risk factors among American Indians: The Strong Heart Study. Annals of Epidemiology. 2002;12:97–106. doi: 10.1016/s1047-2797(01)00270-8. [DOI] [PubMed] [Google Scholar]
  39. Zhang P, Engelgau MM, Norris SL, Gregg EW, Narayan KM. Application of economic analysis to diabetes and diabetes care. Annals of Internal Medicine. 2004;140:972–977. doi: 10.7326/0003-4819-140-11-200406010-00039. [DOI] [PubMed] [Google Scholar]

RESOURCES