Implementing a Palm Pilot Intervention for Primary Care Providers: Lessons Learned

INTRODUCTION

Many investigators have identified "gaps" in the quality of care delivered in the outpatient setting in the U.S[1]. High blood cholesterol was identified by the Department of Health and Human Services (DHHS) as one of 15 high priority conditions for initial focus. One technique for improving the quality of care and reducing variation is through the use of clinical practice guidelines [2]. Studies have shown that non-adherence to the cholesterol guideline Third Report of the Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (ATP III) is as high as 64% in some populations indicating a tremendous potential for improvement [3].

The Guideline Adherence for Heart Health (GLAD Heart) study was designed to test a strategy to improve quality of care through increased adherence to ATP III cholesterol guidelines. Using Cabana et al.’s framework for improving physician adherence to clinical practice guidelines, we developed a multi-modal intervention and tested it through a randomized, controlled, practice-based study design [4]. The GLAD Heart study was designed to answer three questions. First, can physician adherence to complex clinical practice guidelines be promoted by use of a hand-held computerized decision support tool providing patient-specific recommendations, documentation, and drug dosing assistance? Second, will increased adherence to Clinical Practice Guidelines (CPG), as measured by chart review, reduce variation in management by age, gender and race/ethnicity such that disparities in healthcare are reduced or eliminated? Finally, what are the cost implications of using Personal Digital Assistant (PDA)-based technology to promote CPG adherence? This paper describes the overall study design including the multi-faceted intervention and outcome measures.

METHODS

GLAD Heart is a primary care practice-based, randomized controlled trial of a multi-faceted intervention. The unit of randomization was the practice while the primary target of the intervention was the healthcare provider. Outcomes were measured at the patient level. 68 practices were recruited, and 61 primary care practices located in the central and western portions of North Carolina participated. Our inclusion criteria were self-described primary care practices, staffed by either internal medicine or family medicine providers, who were willing to be randomized, and to have chart abstraction performed to obtain information about lipid management. Furthermore, at least half of the providers (physicians, physician assistants, nurse practitioners) in each practice agreed to participate, and the practice was within a 3-hour driving radius of Winston-Salem, NC. We excluded practices with a direct affiliation to a medical school or residency program, practices which had not been open at least one year after the publication of ATPIII, practices providing exclusively sub-specialty care (e.g. OBGYN, cardiology), and sites outside of NC. The recruitment effort for this study is reported elsewhere [5]. Practices were stratified for size and practice type (family medicine, internal medicine or mixed) and then randomized using permuted blocks of size 8. The randomization was conducted by an investigator not involved in either the recruitment or the intervention. To assess the impact of this multi-faceted intervention on adherence to ATPIII we used an attention-control condition with a similar intervention. Control practices received an intervention similar in intensity and frequency of contact but focused on the Joint National Committee on Prevention, Detection, Evaluation and Treatment of High Blood Pressure (JNC7) CPG [6]. The study protocol was approved by the Wake Forest University Institutional Review Board.

Intervention

The intervention and control conditions were designed simultaneously and to mirror one another, since our aim was to change physician behavior; a “usual care” control approach might lead to improved ATPIII adherence simply due to increased contact and raised expectations in the intervention arm. The components common to both arms were provider education regarding both guidelines prior to randomization, continuing medical education (CME) credits, baseline feedback regarding cholesterol and hypertension management,, lunch and learn sessions in a case-based format surrounding the randomized guideline, and patient education materials based on the respective guideline. The arms differed by the guideline assigned and the technology that was distributed to enhance adherence to each guideline.

The intervention condition was designed to improve compliance to ATPIII [7]. The control condition was designed to improve compliance to (JNC7) Clinical Practice Guideline. The providers in practices randomized to the intervention condition were given a software tool and practices randomized to the control condition an automated blood pressure device to assist with accurate blood pressure measurement.

Shared components between study arms

An initial two-hour medical education session was provided to assure that all healthcare providers had similar knowledge about both guidelines. All health care providers received a CD of the presentation with voice-over and copies of the executive summaries of ATPIII and JNC7. Additional CME was offered throughout the study as an incentive in the form of a voice-over CD of PowerPoint slides with a short multiple-choice test at the end. Two were prepared (“Lifestyle Interventions for Hyperlipidemia and Hypertension” and “Drug Therapy to Reduce Cardiovascular Disease Risk”) and distributed to all participating providers by mail.

Four times during the intervention period, investigators and staff visited each practice enrolled in the study to present educational material to providers to enhance adherence to the respective guidelines and engage providers in the study. Education was delivered by physician investigators. Instructional topics centered on the specific guideline to which each practice was randomized. Each visit lasted one hour, lunch was provided and an attempt was made to include all providers. The educational presentation was case-based when possible and educational materials for staff and patients were left for each practice to utilize as they saw fit. To ensure consistency, group training prior to each round of educational outreach was conducted with written instructions. Checklists were developed to assure completion of all tasks and the visits were pilot-tested to determine length and effectiveness. Please see Table 1 for a list of sessions and information included.

Table 1.

Patient Education Materials Distributed at Each Academic Detailing Session.

Round 1	JNC – 7	ATP III
	Patient Blood Pressure Pads
	Laminated JNC-7 Cards for exam rooms	Laminated APTIII Cards for exam rooms
	High Blood Pressure Handouts	High Blood Cholesterol Handouts
Round 2	Dash Diet Handout	Food Label Guide
	High Blood Pressure Handout	High Blood Cholesterol Handouts
	Exercise Prescription Pads	Cholesterol Poster
	Patient Blood Pressure Pads	Exercise Prescription Pads
	TLC Posters
	Food Label Flip Charts
Round 3	Threefold BP Handouts	Threefold Cholesterol Handouts
	Patient Blood Pressure Pads	Exercise Prescription Pads
	Exercise Prescription Pads	Pillboxes
	Pillboxes	Exam Room Posters
	Exam Room Posters	NC Prescription Drug Plan
	NC Prescription Drug Plan
Round 4	Patient Blood Pressure Pads	Threefold Cholesterol Handouts
	Exercise Prescription Pads	AHA Diet Handout
	Dash Diet Handouts	Exercise Prescription Pads
	Pillboxes (if requested)	Pillboxes (if requested)
	Posters (if requested)	Posters (if requested)

At the first lunch and learn session, participating clinics were provided with their baseline rates of compliance, based on medical record abstraction, with both the ATPIII and the JNC7 guidelines. The report described the methodology used in the chart review and determination of guideline compliance and bar charts describing their overall screening and treatment compliance for cholesterol indicators (including the proportion of patients insufficiently treated, appropriately treated, over treated, and at goal LDL level). Hypertension outcomes included the proportion of patients with hypertension who had a documented hypertension diagnosis, percent on therapy and percent at goal blood pressure. For comparison, benchmarks of 20%, 50% and 80% determined from the GLAD Heart study population as a whole were also provided.

During lunch and learn sessions, the study team also provided each practice with patient education materials to support their efforts to implement study guidelines. The goal of the patient education materials was to give participating physicians and practices additional resources to promote adherence to therapeutic lifestyle change and medications that may be added as providers began to follow the CPGs more closely. Approximately half of the materials were developed by the study team and the other half were purchased from state and national organizations that provide patient education materials related to ATP III, JNC7 and cholesterol and blood pressure management.

Differences between study arms

The case-based education sessions were based on the implementation of JNC7 in the attention control practices and ATPIII in the intervention condition. The patient education materials which were distributed focused on cholesterol management and ATPIII in the intervention condition practices and JNC7 and blood pressure management in the attention control condition. Each practice randomized to the attention-control condition was given 1–4 automated blood pressure devices (based on the size of the practice) to use in the day to day operation of their practice. This technology was chosen to reduce the error in blood pressure measurement and these practices were educated regarding proper blood pressure measurement. Each participating provider in the intervention arm was given a Palm® PDA with a novel cholesterol management tool designed to improve adherence to ATPIII.

The Glad Heart software was initially developed by the National Institutes of Health (NIH) to identify specific risk factors to reduce the overall risk for cardiovascular disease and death. The software, named ATPIII, was developed for Palm PDAs. The tool allowed providers to enter a patient’s total cholesterol, LDL, HDL, age, gender and other risk factors such as: established CHD, presence of other clinical atherosclerotic diseases, diabetes, cigarette use, hypertension (HTN) (or Rx for HTN) and family history of early CHD. From this information, the program calculated a patient’s Framingham risk score and provided suggestions to reduce that risk.

We chose a PDA platform because we wanted the tool to be accessible at the point of care. This would allow providers to enter patient information and provide instant feedback. We were also aware that, although integrating a decision support tool in an electronic medical record (EMR) could also be effective, EMR use was not widespread at the inception of the trial. Alternatively, a web-based application was also considered, however was felt to be technically and logistically challenging with additional potential privacy implications. GLAD Heart added new features to the existing program to enhance its function. The GLAD tool provides a list of cholesterol lowering drugs and doses that can be prescribed to aid the patient in achieving their LDL goal to the programmed treatment recommendations. Study programmers created both a print function and a record management program which allows for batch printing so patient information can be included in the patient’s chart. Printing of the patient information proved to be a challenge as the program required infrared technology. In many cases, practice printers could be made infrared capable by installing an adapter. For those practices that did not have a printer or the adapter could not be added, an infrared ready printer was provided. The upgraded study program, named GLAD ATPIII, enabled providers to enter all patient information from the original program along with a patient identifier. In addition to the GLAD ATPIII software, two additional medical software programs were provided on the palm to further encourage use: ePocrates and 5-Minute Clinical consult.

Mid-study, the GLAD Heart team added two additional features to the GLAD ATPIII program. First, a dose escalation program was added. This program enables providers to enter a patient’s current LDL, goal LDL, current therapy, and drugdose. The program calculates the patient’s risk and displays recommendations to achieve the LDL goal. Secondly, a data entry option for the triglyceride level was added. The software algorithm was revised to first check the triglyceride value and determine if action should be taken based on high triglycerides[8]. If the patient’s triglyceride count is below goal, the Framingham risk category for the patient is displayed. If the triglycerides are elevated, a screen providing treatment recommendations is instead displayed.

Study Measures

Intervention Development

Providers in the participating practices were surveyed prior to the intervention to elicit any potential barriers to the intervention. Items included prior use of hand held devices, use of the JNC7 and ATPIII guidelines, and use of other CPGs pertaining to blood pressure and cholesterol management. Study staff also convened a focus group of providers to elicit feedback on the electronic tool development and ideas on how best to eliminate disruptions in practice flow.

Resource Utilization Measures

Time spent installing and maintaining the palm pilots and associated software and hardware at each practice visit was recorded. Also, the absolute number of help requests study staff received was recorded and each request was categorized with regard to the resources required to resolve it.

Intervention Fidelity Measures

Several indicators were used to assess participating providers’ use of the multiple intervention components. The number of academic detailing visits and the number of providers which attended each were maintained. A checklist of educational message to be delivered at each on-site educational session was used to ensure consistent intervention delivery. A software program, AppTracker, was installed on the PDAs that were distributed to providers. This program allowed the GLAD Heart team to monitor use of the PDA, our GLAD ATPIII program, and other programs. This data was extracted at AD visits to calculate usage reports. Version 2 of the GLAD ATPIII program had this technology included.

Technical support was provided for the life of the study. This proved to be particularly challenging due to differing hardware and operating systems within the different practices. Additionally, many providers already had PDAs, so installation had to be customized to the individual provider.

Study Outcome

The trial’s primary study outcome was the difference in the mean change in compliance to ATPIII guidelines from baseline to follow-up between practices randomized to intervention compared to control. Adherence to the guideline was defined as the proportion of patients treated appropriately with respect to lipid-lowering drug therapy within four months after testing. Several secondary outcomes were also included in the study design. These included the proportion of patients appropriately screened for lipids, the proportion of patients appropriately treated with therapeutic lifestyle change, and the cost effectiveness of the GLAD Heart intervention.

The primary and secondary outcomes were determined through a chart review of a random sample of the medical records at each practice. To assure appropriate abstraction of the medical records and to reduce privacy concerns, medical record review was contracted to the Medicare Quality Improvement Organization (QIO) in North Carolina, Carolina Centers for Medical Excellence.

At baseline, charts were randomly selected for abstraction. Information on patient demographics and medical conditions was abstracted from all charts. Medical records with indication of a recent lipid screening qualified for a more complete abstraction. Additional data elements included those necessary for determination of compliance to guidelines including actual lipid levels, lipid lowering medications started after screening, therapeutic lifestyle counseling provided, and follow-up lipid labs, as well as information on diagnosis, treatment and control of hypertension.

The follow-up chart review follows similar procedures with respect to selection of charts for initial and complete review. Additional data elements were added to facilitate the cost-effectiveness study including counts of patient visits and follow-up laboratory measures. To augment the cross-sectional chart review, a sample of patients from the baseline chart review were resampled at follow-up abstraction and reviewed at a later time point. This longitudinal cohort will provide additional estimates of lipid control for comparison.

The cost-effectiveness model will use accepted decision tree modeling techniques with probabilities and cost based on actual estimates obtained from the GLAD Heart study. The outcome will be the cost per screened patient compliant to the guideline using a one year time outlook and a payer perspective.

Statistical Analysis Plan

The primary outcome for this trial was the proportion of patients treated appropriately with respect to lipid-lowering therapy within 4 months after lipid testing. The analyses for this study took into account the nested design where randomization and intervention were performed at the level of the practices, but the actual outcomes data were collected on individual patients within practices.

The primary and secondary outcomes assessed represent dichotomous variables that are usually reported either in terms of the proportion or the odds of success or failure. These include appropriate medication treatment decisions (primary), and the components of appropriate decision making: appropriate prescribing, over-prescribing, and appropriate screening. We will analyze the data using a generalized estimating equations (GEE) approach with a logit link.[9] utilizing robust variance estimates.

To determine sample size we had estimated that appropriate treatment at baseline would be 70% in both arms. Recruiting 32 practices per group (64 total) yielded 80% power to detect a minimum difference of 9% in treatment rates between control and intervention practices at follow-up with a 5% two-sided significance level, assuming 30 patients assessed per practice, a between practice variance of 0.21, and an intraclass correlation of 0.006.

RESULTS

GLAD Heart Software tool

The refinement and development of the GLAD Heart software took one programmer working one FTE approximately 14 weeks to complete. Four physician investigators also spent time testing the tool in their own clinic assignments, approximately 10 hours. Installation of the palm pilot software at each practice was challenging due to the variability in computer systems and internet access. An average of two hours per practice was needed to complete the installation with a range from 1 to 3 hours. Installation was performed by experienced software technicians. Over the course of the study, there were 74 technical issues requiring assistance for the Palm Pilot from 23 participating practices in the intervention arm. Most of these requests for help were the result of dead batteries and printing issues. Thirty of the issues were resolved during a regularly scheduled visit to the practice. An additional 31 were resolved on the phone and 13 required a non-scheduled visit to the practice. As described above, two updates were made to the tool (dose escalation and triglyceride management). Updates were performed in conjunction with scheduled educational outreach visits.

Almost half (49.1%) of providers responded by survey at the beginning of the intervention that they did not currently use a PDA or handheld device to assist in providing patient care, so the use of this technology was fairly novel. Monitoring indicated mean total PDA use was 1,814 (SD = 3061, n=56 providers) minutes during the first six months. Over the course of the study, due to Palm failures, provider turnover, and attendance differences at lunch and learn sessions, data collected at subsequent sessions could not be compared to the initial data.

Initial Continuing Medical Education Session and Educational Outreach Visits

A total of eight sessions were scheduled. Of the 219 providers practicing at the 61 practices, 75 attended the CME session. 100% accepted the CD Rom. A total of 244 educational outreach visits were conducted. The sessions lasted approximately one hour. A training session for investigators and staff was held prior to each round of educational outreach with average attendance by investigators of 80% and 100% by staff. Number of providers at each practice in the JNC arm ranged from 1 to 14 with mean of 4.0 (3.3) and in the ATP arm ranged from 1 to 11 with mean of 3.0 (2.3). Attendance by providers at the educational outreach sessions in the JNC group averaged 80.3% while the ATP group averaged 86.1% with p=0.15 for intervention difference.

Baseline feedback

Sixty practices received feedback from the baseline chart review (one practice was enrolled but had been open for less than a year and thus a chart review was not conducted). An average of 84 charts per practice was reviewed with a range of 37 to 436 charts.

DISCUSSION

The GLAD Heart project was completed successfully with some impact on cholesterol management. [10;11] However, the implementation of the intervention proved to be a challenge for several reasons. Provider turnover, technical issues with Palms requiring staff assistance, technology differences among practices, provider and practice schedules, and differences in provider familiarity with technology all proved to be challenges for which we had not originally planned. Providers reported, anecdotally, becoming more familiar with the tool and the Framingham risk calculations and felt less reliant on the tool for risk determination and medication evaluation as the intervention progressed. In addition, as time since study enrollment increased, the novelty of the handheld decision support tool decreased. Consequently, providers seemed to use the tool less as the study progressed. It is possible that this familiarity was achieved through increased awareness of the CPG and not necessarily through the technology. Therefore, it may be possible to obtain similar impact on patient management through other methods of provider education. We were interested in the impact of the technology as part of a multi-faceted intervention and therefore did not test other methods of education.

We attempted to track usage of our tool through the software program, however, when providers did not bring the handheld device to intervention sessions, we were unable to capture this data, as it was stored on the Palm. Palm Pilot battery failures also posed a challenge due to the lack of an internal memory on the Palm model used. Usage data were erased when batteries were allowed to lose their charge, thus usage data collection restarted for some users and had to be thrown out. Therefore, usage data was inaccurate.

When considering a similar study that involves the planning and execution of a technology intervention, study teams should bear the following in mind: First, staff and investigators must be familiar with the technology to be used and anticipate the degree of technology support that will be needed by participating practices. A survey of the technology support conducted prior to starting the intervention may be necessary. Second, future study teams should anticipate turnover among the providers and change in the computer systems among practices. Reinstallation of software may be necessary throughout the duration of the intervention and the appropriate staff should be retained to provide this service if practices do not have technology support. Third, any methods used for monitoring the adoption of the technology must be robust enough to account for user lapses such as failure to charge the batteries. Finally, if the intervention is to include some form of educational session at the medical practice, the study team should plan for attendance problems by the healthcare providers. Patient schedules get behind and providers become unable to attend lunch and learn sessions.

Technology has the potential to improve the quality of care provided in the healthcare setting. However, potentially expensive interventions such as that conducted in GLAD Heart should undergo rigorous testing to assure their efficacy before widespread adoption. The lessons learned by the GLAD Heart study team can assist future groups in the planning and execution of these studies.