Patient reminder and recall systems to improve immunization rates

Julie C Jacobson Vann; Peter Szilagyi

doi:10.1002/14651858.CD003941.pub2

. 2005 Jul 20;2005(3):CD003941. doi: 10.1002/14651858.CD003941.pub2

Patient reminder and recall systems to improve immunization rates

Julie C Jacobson Vann ^1,^✉, Peter Szilagyi ²

PMCID: PMC6485483 PMID: 16034918

Abstract

Background

Immunization rates for children and adults are rising, but coverage levels have not reached optimal goals. As a result of low immunization rates, vaccine‐preventable diseases still occur. In an era of increasing complexity of immunization schedules, rising expectations about the performance of primary care, and large demands on primary care physicians, it is important to understand and promote interventions that work in primary care settings to increase immunization coverage. A common theme across immunization programs in all nations involves the challenge of determining the denominator of eligible recipients (e.g. all children who should receive the measles vaccine), and identifying the best strategy to ensure high vaccination rates. Strategies have focused on patient‐oriented interventions (e.g. patient reminders), provider interventions, and system interventions. One intervention strategy involves patient reminder and recall systems.

Objectives

To assess the effectiveness of patient reminder and recall systems in improving immunization rates, and compare the effects of various types of reminders in different settings or patient populations.

Search methods

A systematic search was performed for the initial review using MEDLINE (1966‐1998) and 4 other bibliographic databases: EMBASE, PsychINFO, Sociological Abstracts, and CAB Abstracts. Authors also performed a search of The Effective Practice and Organisation of Care (EPOC) register in April 2001 to update the review. Two authors reviewed the lists of titles and abstracts, and used the inclusion criteria to mark potentially relevant articles for full review. The reference lists of all relevant articles and reviews were back searched for additional studies. Publications of abstracts, proceedings from scientific meetings, and files of study collaborators were also searched for references. In May 2007 the EPOC register was searched to identify relevant articles to update the review. In May 2007, the EPOC register search was supplemented by searches of CINAHL and PubMed. One study was identified through prior knowledge of this study.

Selection criteria

Study Design Randomized controlled trials (RCT), controlled before and after studies (CBA), and interrupted time series (ITS) studies written in English.

Types of participants Health care personnel who deliver immunizations and children (birth to 18 years) or adults (18 years and up) who receive immunizations in any setting.

Types of interventions Any intervention that falls within the EPOC scope and that includes patient reminder or recall systems, or both, in at least one arm of the study.

Types of outcome measures Immunization rates, or the proportion of the target population up‐to‐date on recommended immunizations. Outcomes were acceptable for either individual vaccinations (e.g. influenza vaccination) or standard combinations of recommended vaccinations (e.g. all recommended vaccinations by a specific date or age).

Data collection and analysis

Data Collection Each study was read independently by two reviewers. Disagreements between reviewers were resolved by a formal reconciliation process to achieve consensus.

Analysis Results are presented for individual studies as relative rates for randomized controlled trials, and as absolute changes in percentage points for controlled before and after studies. Pooled results for RCTs only were for presented using the random effects model.

Main results

Four new studies were added for the 2007 update for a total of 47 studies. Increases in immunization rates due to reminders were in the range of 1 to 20 percentage points. Reminders were effective for childhood vaccinations (OR = 1.47, 95% CI =1.28, 1.68), childhood influenza vaccinations (OR = 2.18, 95% CI = 1.29, 3.70), adult pneumococcus, tetanus, and Hepatitis B (OR = 2.19, 95%CI = 1.21, 3.99), and adult influenza vaccinations (OR = 1.66, 95% CI = 1.31, 2.09). The effectiveness of patient reminders for childhood influenza vaccinations declined overall from an odds ratio of 2.87 in the previous review. However, reminders were not effective in the one new study of adolescent immunizations in an urban setting (OR = 1.14, 95% CI = 0.98, 1.31). All types of reminders were effective (postcards, letters, telephone or autodialer calls), with telephone being the most effective but also the most costly.

Authors' conclusions

Patient reminder and recall systems in primary care settings are effective in improving immunization rates in developed countries.

Keywords: Adult, Child, Humans, Reminder Systems, Immunization, Immunization/utilization, Immunization Programs, Immunization Programs/organization & administration, Randomized Controlled Trials as Topic

Does reminding people to have vaccinations increase the number of people who receive vaccinations?

Increasing the number of people who are vaccinated could lower death and disease rates throughout the world. Many strategies to increase vaccinations have been used. One way is to remind people to receive their vaccinations. This review of 47 studies evaluated whether reminding people worked.

This review found that reminding people to have vaccinations increased the number of people vaccinated, whether the people were due or overdue for vaccinations. The increases were observed in both children and adults for all types of vaccines, but not among urban adolescents in one study. Reminding people over the telephone, sending a letter or postcard, or speaking to them in person increased vaccinations. Providing numerous reminders was more effective than single reminders. Reminding people over the telephone was more effective than postcard or letter reminders, but reminders over the telephone may be expensive compared with alternative approaches. Reminders also worked whether it was from a private doctor's office, a medical center, or a public health department clinic. The studies in this review were from developed countries; and, it is therefore not clear whether reminders to patients would work in low‐ and middle‐income countries.

Background

In 1974 immunization rates in children worldwide were reported to be as low as 5% (GlobalAlliance 2001). A number of programs, such as the Expanded Programme on Immunization (EPI), launched by the World Health Assembly in 1974, helped to increase immunization rates throughout many parts of the world. By 1990 the vaccination coverage rate of children under five was 80%; this fell to 74% in 1999 (GlobalAlliance 2001). Global coverage of infants with DTP3 (3 doses of DTP) was estimated to be 79% in 2006, an increase of 59 percentage points since the 1980 level of 20% (WHO 2007). Yet, an estimated 26.3 million children did not receive DTP3 in 2006 (WHO 2007).

Access to immunizations, prevalence of vaccine‐preventable diseases, and vaccination rates varies by geographic area or country. High levels of immunization coverage (86% to 92%) were achieved in the Americas, Europe, the Western Pacific, and the Eastern Mediterranean regions in 2006 (WHO 2007). In contrast, African countries suffer from a disproportionately high level of vaccine‐preventable diseases (Arevshatian2007). The African Regional Strategic Plan of the Expanded Programme on Immunization established a goal for 80% of countries in Africa to reach 80% immunization coverage by 2005 (Arevshatian2007). This goal was not achieved; more than one‐third of African Region districts did not reach 50% DTP3 coverage by the end of 2004 (Arevshatian2007). However, DTP3 coverage in Africa did increase from 54% in 2000 to 69% in 2004 (Arevshatian2007). The lowest vaccination coverage is found in sub‐Saharan Africa (GlobalAlliance 2001).

Throughout the United States immunization rates for children and adults are rising, (CDC1997657; CDC1998547) but coverage levels have not reached national goals (CDC1999243). In 1998, coverage levels for children 19 to 35 months of age were at 79% for the 4:3:1:3:3 combined series of 4 DTP, 3 polio, 1 measles‐containing vaccine, and 3 Hib; 87% for 3 hepatitis B, and 43% for varicella (CDCwebpage). In this same age group, coverage for the 4:3:1:3:3:1 series (which adds varicella vaccine to the 4:3:1:3:3 combined series) increased slightly from 76.1% in 2005 to 77.0% in 2006 (MMWR 2007). Coverage levels for adults are lower; in 1997 only 65% percent of adults over the age of 65 received the influenza vaccine, and only 45% had ever received pneumococcal vaccine (CDC1998797). During the 2005 to 2006 season, influenza vaccination rates were slightly higher (69.3%) (MMWR 2007a). Furthermore, immunization coverage levels are not evenly distributed, with low rates among impoverished populations, (CDC1997956) and some primary care practices (Massoudi 1999). Influenza immunization rates also vary by race or ethnicity, with reports of 71.9% among non‐Hispanic whites compared with 58.3% among other racial or ethnic groups during the 2005‐06 season (MMWR 2007a). As a result of low immunization rates, vaccine‐preventable diseases still occur as evidenced by the measles epidemic during 1989‐1991, (NVAC 1991) the large number of annual cases of varicella, pertussis, and hepatitis B, (CDC19981) and the more than 50,000 annual deaths in adults from influenza or pneumococcal infections (Fedsom 1994). There is also concern that incorporation of new vaccinations will be slow, as evidenced by the slow uptake of varicella vaccine (CDC1999829). Recent concerns about vaccine safety, (CDC199910; CDC1999996) may cause health care providers, patients, or parents to become more cautious about vaccinations.

In an era of increasing complexity of immunization schedules, rising expectations about the performance of primary care, and large demands on primary care physicians, it is important to understand and promote interventions that work in primary care settings. Published reviews have identified several promising strategies to improve immunization rates (Gyorkos 1994; Shea 1996; Udovic 1998; Task Force 1999; Shefer 1999). One strategy involves patient reminder or recall systems, which was recommended by the Task Force on Community Preventive Services (Task Force 1999) and the Standards for Immunization Practices (AdHocWorkingGroup).

Few primary care providers actually use reminder or recall systems, or both (Szilagyi 1994; Schaffer 1998). Because many patients cannot remember the recommended immunization schedule (Santoli 1998), the burden falls on primary care providers to ensure that their patients receive immunizations on a timely basis. Recently, in the U.S., the burden on the private sector has increased as more patients receive immunizations from their comprehensive primary care provider rather than at health department immunization clinics (Szilagyi 2000; Rodewald). While in some nations (such as the United Kingdom) (UK 2002) private providers administer the majority of vaccinations. In many other nations a variety of publicly funded clinics provide vaccinations. Nevertheless, the issue of determining who is eligible (i.e., the denominator), and how to vaccinate the entire population using either patient‐oriented or provider‐oriented strategies is a common one irrespective of type of setting or funding mechanism.

With experts recommending reminder or recall systems, or both, and individual studies demonstrating their effectiveness, why are these systems not utilized more frequently in primary care settings? Several factors may impede their incorporation. First, providers may not perceive that individual studies apply to their own practices. Pediatricians may not focus on studies involving elderly adults, and internists may not be aware of studies involving children. Some studies were performed in public health department clinics or academic teaching hospital clinics, and private providers may not believe those findings could be applied to their settings. Furthermore, some vaccinations are given only once, while others require multiple booster doses, making it more difficult to extrapolate findings from individual regimens to all vaccinations.

A second barrier is that recommendations about reminder or recall systems have not been very specific (Udovic 1998; Task Force 1999). Patient reminders can be delivered by a variety of methods (e.g. telephone, mail), and levels of intensity (e.g. single, or multiple reminders). The most useful recommendations are ones that apply to large numbers of providers but are specific enough to be applicable in real‐world settings. A third barrier is that many primary care practices have lacked the computerized technology to track their patients' immunization status. However, recent advances in billing systems and computerized immunization registries (Linkins 1998) are making such technology attainable for a growing number of primary care practitioners.

Some countries, including a number in Europe, have made substantial progress toward developing computerized immunization registries (Paunio1991T89). Immunization registries offer the potential for becoming the backbone of patient reminder or recall systems by: (a) providing a denominator of potential vaccine recipients, (b) providing algorithms to determine who is eligible based on vaccination recommendations, and (c) providing means to send postcards, letters, or even telephone reminders.

Several common themes apply to both childhood and adult immunizations in all countries, including developing countries. First, immunizations are a major public health intervention by protecting not only those vaccinated but also the general population through prevention of spread of disease. Vaccine‐preventable diseases are a major cause of mortality and morbidity throughout the world, due in large part to insufficiently high vaccination rates (WHO 2001). Second, while recommendations for specific vaccinations vary across nations, there are international efforts to improve immunization rates and reduce vaccine‐preventable diseases (WHO 2002; Global02). Third, similar strategies are being considered across nations, including strategies to increase demand by focusing on patients (such as patient reminder and recall systems), strategies to focus on providers (such as reducing missed opportunities, or provider prompts), and system‐wide strategies (such as financial interventions). Thus, the current review is widely applicable.

This is an update of an earlier Cochrane, last updated in 2005 (Jacobson Vann 2005).

Objectives

The study objectives were to:

assess the overall effectiveness of patient reminder or recall systems, or both, in improving immunization rates;
compare the effectiveness of different types of reminder or recall interventions (e.g. postcard, letter, telephone), or a combination of both reminder and recall.

Methods

Criteria for considering studies for this review

Types of studies

Randomized controlled trials (RCT), controlled before‐and‐after studies (CBA), and interrupted time series studies (ITS) were included. Non‐English language publications were excluded.

Types of participants

The study selection criteria for types of participants included health care personnel who deliver immunizations and children (birth to 18 years) or adults who receive immunizations in any setting (academic or non‐academic, developed or developing countries).

Types of interventions

The study selection criteria for intervention type included patient reminder or recall interventions, or both, that either reminded patients of upcoming immunizations or immunization visits that were due (reminders) or were overdue (recall). Reminder and recall systems could be delivered by letter, postcard, telephone, autodialer (a computerized telephone dialer programmed to generate multiple telephone calls during a short time period), or in person (e.g. provider gives face‐to‐face reminder). Reminder and recall cues could also vary in their specificity (generic reminders or personal reminders that address issues specific to the patient), in their number (one‐time or multiple reminders) and whether or not combined with other interventions such as provider reminders or outreach.

Types of outcome measures

Key outcome measures were immunization rates, or the proportion of the target population up‐to‐date on recommended immunizations. We accepted outcomes for either individual vaccinations or standard combinations of recommended vaccinations (e.g. all recommended vaccinations by a specific date or age).

Search methods for identification of studies

For the initial review, a search strategy was performed using the following bibliographic databases: MEDLINE (1966‐1998), EMBASE, PsychINFO, Sociological Abstracts, and CAB Abstracts. All databases were searched from their date of inception. The search terms included the following, including text words (tw) or MeSH headings (mh): remind$ (tw), track$ (tw), autodial$ (tw), postcard$ (tw), mail$ (tw), recall$ (tw), telephone$ (tw), registry (tw), registries (mh and tw), reminder systems (mh), appointments & schedules (expanded mh), appointment$ (tw), information systems (expanded mh), computers (mh), immunization (expanded mh), immuniz$ (tw), immunization programs (mh), vaccination (mh), vaccin$ (tw), innoculat$ (tw), prevention health services (mh), diphtheria (mh), tetanus (mh), whooping cough (mh), poliomyelitis (mh), polioviruses (mh), haemophilus (mh), influenza (mh), measles (mh), mumps (mh), rubella (mh), hepatitis b (mh), pneumococcal infections (mh), vaccines (expanded mh), tetanus toxoid (expanded mh), and diphtheria toxoid (expanded mh). The reference lists of all relevant articles and reviews were back searched for additional studies. Publications of abstracts, proceedings from scientific meetings, and files of study collaborators were also searched for references. In April 2001 the EPOC register was searched to update the review. For the 2005 review update, a search of the EPOC Register was performed on December 14, 2004 using the search terms: immuniz$, vaccin$, and innoculat$. The 2007 update involved a search of the EPOC register during May 2007 using the search terms: immunis*, immuniz*, vaccin*, and inoculat*. The EPOC search with supplemented on July 17, 2007 with a search of CINAHL using the terms immunization reminder, patient reminder, immunization recall, vaccine recall, vaccine reminder, and patient recall, and a search of PubMed using the term vaccine reminder. One study was identified through prior knowledge of this study. The searches were restricted to articles written or translated into English. Studies written in languages other than English may be reviewed in an update.

Data collection and analysis

For the initial review, two authors reviewed the lists of titles and abstracts, and used the inclusion criteria to mark potentially relevant articles for full review. For the updated reviews, one author reviewed the titles and abstracts to select relevant articles for full review. Each study that was selected as potentially relevant in the search process was read and abstracted independently by two reviewers (P.G.S. and J.C.J.V). Reviewers were not blinded to authors. Disagreements between reviewers on abstraction results were resolved by a formal reconciliation process to achieve concensus. Data abstraction utilized a checklist developed by the Cochrane Collaboration Effective Practice and Organization of Care Group (EPOC) (EPOC 2007). For each included study, information was collected on the method of randomization or assembly of control groups, blinding, characteristics of subjects, setting and nature of the interventions, and results. Numerous quality criteria were assessed for each study design (EPOC 2007).

For randomized controlled trials, which were the vast majority of included studies, assessment criteria included: concealment of allocation, proportion of participants followed up, blinded assessment of primary outcomes, documentation of baseline data, reliability of outcome measures, and protection of contamination between study groups. These are reported in the "Characteristics of Included Studies" table. As part of the 2005 update all new and previously included studies were reviewed for cluster allocation and unit of analysis errors. The review criteria in 2007 were unchanged from the 2005 update.

Quality Criteria ‐‐ Concealment of Allocation Concealment of allocation, which refers to "how well the allocation to treatment group was concealed," was scored in the "Characteristics of Included Studies" table using the Review Manager scores of "A" through "D" (Higgins 2006). These scores are defined as: A: "indicates adequate concealment of the allocation (for example, by telephone randomization, or use of consecutively numbered, sealed, opaque envelopes)." B: "indicates uncertainty about whether the allocation was adequately concealed (for example, where the method of concealment is not known)." C: "indicates that the allocation was definitely not adequately concealed (open random number lists, for example, or quasi‐randomization such as alternate days, odd or even date of birth, or hospital number)." D: "indicates the score was not assigned."

Quality Criteria ‐‐ Follow‐up of allocation unit (professionals or patients) Follow‐up of professionals or patients or episodes of care was scored using the "A" through "D" scoring system as was used for "allocation concealment." A score of: A: indicates that outcome measures were obtained from 80‐100% of subjects randomized. Follow‐up must be explicitly documented. B: not clear; not specified in the paper. C: not done, if outcomes were obtained for less than 80% of subjects randomized. D: score was not assigned.

Quality Criteria ‐‐ Blinded Assessment of Primary Outcomes Blinded assessment of primary outcomes was scored in the "Characteristics of Included Studies" table using the "A" through "D" scoring system. A score of: A: indicates that the authors specifically documented that the primary outcome variables were assessed blindly or the outcome variables are objective measures. B: not clear; not specified in the paper. C: not done; if the outcomes were not assessed blindly. D: score was not assigned.

Quality Criteria ‐‐ Reliable Primary Outcome Measure The reliability of the primary outcome measure was scored in the "Characteristics of Included Studies" table using the "A" through "D" scoring system as listed above. A score of: A: indicates two or more raters with at least 90% agreement or kappa greater than or equal to 0.8, or the outcomes is obtained from some automated system. B: not clear; if reliability is not reported for outcome measures that are obtained by chart extraction or collected by an individual. C: not done; if agreement is less than 90% or kappa is less than 0.8. D: score was not assigned.

Quality Criteria ‐‐ Protection against contamination Protection against contamination was scored in the "Characteristics of Included Studies" table using the "A" through "D" scoring system as listed above. A score of: A: indicates the allocation was by community, institution or practice and it is unlikely that the control group received the intervention. B: not clear; if professional were allocated within a clinic or practice and it is possible that communication between experimental and group professionals could have occurred. C: not done; if it is likely that the control group received the intervention (e.g. cross‐over trials or if patients rather than professionals were randomized). D: score was not assigned.

Analysis Procedures For the initial review EXCEL software (Excel 2005) was used to track the review process, manage the study‐level and comparison‐level data, compute odds ratios and 95% confidence levels for each study arm as a reliability check of results computed in Meta View and RevMan, sort studies by selected characteristics, record absolute changes in immunization rates (expressed as percentage point changes), and prepare funnel distribution displays of study‐level data to assess for potential publication bias. Study‐level data were analyzed using Meta View (RevMan 3.1), EXCEL software, and then RevMan. RevMan was used to compute odds ratios and 95% confidence intervals for individual RCT studies or study arms, assess heterogeneity of the study data, and compute summary odds ratios for RCT studies using the random effects models.

The Table of Comparisons was structured to examine study results (odds ratios) by type of patient reminder: (a) postcard; (b) letter; (c) person‐to‐person telephone call; (d) autodialer (computerized phone message system); (e) postcard and telephone call (combination); (f) tracking (patient reminder or recall) and outreach; (g) summary of all patient reminders; and (h) combinations of patient and provider reminders. Subcategories were set up within each intervention type listed above to perform subgroup analyses by major immunization category: (1) childhood influenza immunizations only; (2) routine immunizations delivered to preschool children; (3) adult influenza vaccinations only; and (4) other routine adult immunizations. In 2007, a new subcategory, adolescent immunizations, was added to two comparison categories, autodialer and patient reminder summary. This subcategory includes studies that focus exclusively on immunizations in adolescents age 11 to 19 years. Dichotomous data from included RCT studies only, and those not identified with potential cluster randomization limitations, were entered into Meta View and RevMan Analyses data tables for appropriate study arms, reminder type, and immunization category. In addition, data for these included RCT patient reminder or recall studies were re‐entered into the patient reminder summary category to provide an overall measure of results. Data entered for each selected study arm includes: number of persons immunized (or up‐to‐date) in the intervention group; total number of persons in the intervention group; number of persons immunized (or up‐to‐date) in the control group; and total number of persons in the control group.

For studies with more than one patient reminder or recall intervention group with similar intervention types (e.g. two different types of postcards), intervention group data were combined. For RCT studies with outcomes reported in more than one immunization outcome category (based on the table of comparisons), multiple outcomes were reported at the individual reminder‐type level, and data were combined for the summary comparison. If an RCT described a series of interventions (e.g. postcards sent at 1 month, 3 months and 5 months) with immunization outcomes measured after each intervention, the sample sizes for each cell (e.g. number of persons immunized in the intervention group) were averaged before being entered into RevMan for analysis.

In the initial review, the Peto odds ratios (fixed effects model) were computed in MetaView (RevMan 3.1) for the reminder types and immunization categories as listed above. The results were then tested for heterogeneity using the chi square distribution, with a 0.1 level of significance because of the low sensitivity of this test. The chi square tests were examined at the subgroup level as well as for the patient reminder groups to assess study heterogeneity. After testing for heterogeneity, the pooled measures were ultimately computed using the random effects model, including for the review update.

To assess whether possible publication bias exists, funnel distribution displays were created in EXCEL for the initial review. Sample size was plotted against both absolute measures of effect (percentage point changes in immunization rates) and relative measures (odds ratios).

Results

Description of studies

Of the 398 studies that were reviewed for potential inclusion in this review, 97 were identified by literature searches, 17 by back‐tracing, and 283 from the EPOC register (193 of these in 2007) and 1 from prior knowledge. Of the 122 studies that were fully reviewed, 47 met eligibility criteria and were included in the final review, including 5 new studies that were added in the 2005 update (Daley2004T513; Daley2004T515; Hull2002T511; LeBaron2004T512; Sansom2003T514) and 4 in the 2007 update (Irigoyen2006T702; Kempe2001T706; Kempe2005T707; Szilagyi2006T718) (see "References to Studies, Included Studies" and the table of "Characteristics of Included Studies"). Approximately two‐thirds of excluded studies did not meet study design inclusion criteria. I ntervention Type Five types of patient reminder or recall interventions and combinations of some intervention types were reviewed. Each intervention type was a mechanism to inform patients or families of the need for a vaccination that is due or overdue. The methods include: letters to patients (22 studies) (Baker 1998T96; Brimberry1988T33; Campbell1994T87; Carter1986T104; Daley2004T513; Hogg1998T101; Kempe2005T707; Kemper1993T11; Lieu1997T69; Lieu1998T82; McDowell1986T46; Moran1992T16; Mullooly1987T67; Nexoe1997T92; Oeffinger1992T27; Ornstein1991T30; Rosser1991T61; Rosser1992T47; Satterthwaite1997T93; Siebers1985T36; Szilagyi1992T15; Young1980T63), postcards (9 studies) (Baker 1998T96; Buchner1987T34; Buffington1991T29; Campbell1994T87; Irigoyen2006T702; Larson1982T39; Puech1998T99; Spaulding1991T28; Tollestrup1991T18), person‐to‐person telephone calls (8 studies) (Brimberry1988T33; Ferson1995T57; Hull2002T511; Lukasik1987T85; McDowell1986T46; Rosser1991T61; Rosser1992T47; Sansom2003T514), autodialer (computerized phone messages) (5 studies) (LeBaron2004T512; Lieu1998T82; Linkins1994T49; Stehr‐Green1993T10; Szilagyi2006T718), postcard and phone combination (4 studies) (Alto1994T54; Daley2004T515; Kempe2001T706; LeBaron1998T78), and tracking and outreach (2 studies) (Rodewald1999T95; Wood1998T105). Provider reminders, in combination with patient reminders, were also reviewed (5 RCT & 1 CBA) (Becker1989T23; Frame1994T52; Ornstein1991T30; Rodewald1999T95; Soljak1987T35). The total number of studies sorted by intervention type exceed the 47 included studies because many studies had more than one intervention arm.

Countries Approximately three‐fourths (36) of the included patient reminder or recall studies were performed in the United States (USA). The remaining 26% of studies were performed in Australia (2) (Ferson1995T57; Puech1998T99), Canada (5) (Hogg1998T101; Lukasik1987T85; McDowell1986T46; Rosser1991T61; Rosser1992T47), Denmark (1) (Nexoe1997T92), New Zealand (2) (Satterthwaite1997T93; Soljak1987T35), and United Kingdom (1) (Hull2002T511).

Participants More than one‐third of included studies (16) examined routine vaccinations of infants and children (Alto1994T54; Campbell1994T87; Daley2004T515; Ferson1995T57; Irigoyen2006T702; Kempe2001T706; LeBaron2004T512; Lieu1997T69; Lieu1998T82; Linkins1994T49; Oeffinger1992T27; Rodewald1999T95; Stehr‐Green1993T10; Tollestrup1991T18; Wood1998T105; Young1980T63), and four studied influenza vaccinations in high‐risk children and infants (Daley2004T513; Kempe2005T707; Kemper1993T11; Szilagyi1992T15). Less than half (20) (Baker 1998T96; Brimberry1988T33; Buchner1987T34; Buffington1991T29; Carter1986T104; Hogg1998T101; Hull2002T511; Larson1982T39; Lukasik1987T85; McDowell1986T46; Moran1992T16; Mullooly1987T67; Nexoe1997T92; Puech1998T99; Rosser1991T61; Satterthwaite1997T93; Siebers1985T36; Spaulding1991T28) of the studies assessed the effectiveness of patient reminder or recall interventions on adult influenza immunization rates for patients 65 years of age or older, those with chronic illnesses, or both. Six (Hogg1998T101; Ornstein1991T30; Rosser1991T61; Rosser1992T47; Sansom2003T514; Siebers1985T36) included studies assessed the effectiveness of patient reminder recall on any or all of adult tetanus, pneumococcal, or hepatitis B vaccine. 0ne new study examined the effect of patient reminders on adolescent immunizations (Szilagyi2006T718). The total number of studies described exceeds the 47 included studies as some studies examined more than one type of vaccine.

Settings The patient reminder or recall studies were performed in diverse settings, ranging from urban to rural, and public to private to university‐based. Examples of study settings are state health departments, health maintenance organizations (HMO), public health departments, urban teaching facilities, private practices, senior centers, rural practices, and schools.

Description of Excluded Studies The "Characteristics of Excluded Studies" table briefly indicates the reason for exclusion from this review. Of the 75 studies in this table, 65% (49) of excluded articles did not meet the study design definitions for RCT, CBA or ITS. These studies either lacked a true control group, were descriptive or ecological studies, had insufficient data points to be considered ITS, were cost studies, or were reviews or editorials instead of studies. Three studies compared two interventions without a true control group. Thirteen studies examined the effectiveness of interventions that did not meet the review definition of the evaluated interventions, such as immunization stickers and patient‐carried immunization cards, or individualized blended interventions to preclude evaluation of specific intervention types. Six studies used an outcome other than immunizations (e.g. preventive visits or services, cases of measles). One article reported a study of mammography with discussion of immunizations. The remaining three references were abstract only (Szilagyi2002T717; Wojciechowski1993T88) and an article that was later retracted (Abramson1995T66).

Risk of bias in included studies

Three (LeBaron1998T78; Margolis1992T17, Tollestrup1991T18) of the included studies used a CBA design; all of the remaining studies used a RCT design. One of the CBA studies (Tollestrup1991T18) had equivalent baseline immunization rates in the control and intervention study groups. Concealment of allocation, as defined in the EPOC Data Collection Checklist and as outlined above in Methods of the Review, was explicitly described in 36.4 percent (16 of 44) (Brimberry1988T33; Daley2004T513; Daley2004T515; Hogg1998T101; Hull2002T511; Irigoyen2006T702; Kemper1993T11; LeBaron1998T78; Lieu1997T69; Linkins1994T49; McDowell1986T46; Puech1998T99; Rodewald1999T95; Rosser1991T61; Rosser1992T47; Szilagyi2006T718) of the included RCT studies. Allocation concealment was not clearly documented in 52.3 percent (23 of 44) (Alto1994T54; Baker 1998T96; Becker1989T23; Buchner1987T34; Buffington1991T29; Campbell1994T87; Carter1986T104; Ferson1995T57; Frame1994T52; Kempe2001T706; Kempe2005T707; Larson1982T39; Lieu1998T82; Moran1992T16; Mullooly1987T67; Ornstein1991T30; Satterthwaite1997T93; Siebers1985T36; Spaulding1991T28; Stehr‐Green1993T10; Szilagyi1992T15; Wood1998T105; Young1980T63) of the included RCT studies.

Follow‐up of the allocation unit (patient or provider), as specified by The Data Collection Checklist, was clearly documented in 54.3 percent (26 of 47) (Brimberry1988T33; Campbell1994T87, Carter1986T104; Hogg1998T101; Hull2002T511; Irigoyen2006T702; Kemper1993T11; Lieu1997T69; Lieu1998T82; Linkins1994T49; Lukasik1987T85; Margolis1992T17; McDowell1986T46; Moran1992T16; Nexoe1997T92; Ornstein1991T30; Puech1998T99; Rodewald1999T95; Rosser1992T47; Sansom2003T514; Siebers1985T36; Soljak1987T35; Spaulding1991T28; Szilagyi2006T718; Tollestrup1991T18; Wood1998T105) of the included studies, "not clear" in 31.9 percent (15 of 47, (Alto1994T54; Baker 1998T96; Becker1989T23; Buffington1991T29; Daley2004T513; Daley2004T515; Frame1994T52; LeBaron1998T78; LeBaron2004T512; Mullooly1987T67; Oeffinger1992T27; Satterthwaite1997T93; Szilagyi1992T15) of the studies, and not done in 12.8 percent (6 of 47) (Buchner1987T34; Ferson1995T57; Larson1982T39; Rosser1991T61; Stehr‐Green1993T10; Young1980T63).

Almost half (22 of 47; 46.8%) of the included studies (Becker1989T23; Brimberry1988T33; Buchner1987T34; Carter1986T104; Hogg1998T101; Hull2002T511; Kempe2005T707; Kemper1993T11; LeBaron1998T78; Lieu1997T69; Lieu1998T82; Linkins1994T49; McDowell1986T46; Ornstein1991T30; Puech1998T99; Rodewald1999T95; Rosser1991T61; Rosser1992T47; Siebers1985T36; Soljak1987T35; Szilagyi1992T15; Szilagyi2006T718) clearly documented blinded assessment of the primary outcome(s). This was not clearly documented in 46.8 percent (22 of 47) of included studies (Alto1994T54; Baker 1998T96; Campbell1994T87; Daley2004T513; Daley2004T515; Ferson1995T57; Frame1994T52; Irigoyen2006T702; Kempe2001T706; Larson1982T39; Lukasik1987T85; Moran1992T16; Mullooly1987T67; Nexoe1997T92; Oeffinger1992T27; Sansom2003T514; Satterthwaite1997T93; Spaulding1991T28; Stehr‐Green1993T10; Tollestrup1991T18; Wood1998T105; Young1980T63).

Fourteen (29.8%) included studies (Baker 1998T96; Carter1986T104, Hogg1998T101; Kempe2001T706; Lieu1997T69; Lieu1998T82; Linkins1994T49; McDowell1986T46; Moran1992T16; Puech1998T99; Rosser1991T61; Soljak1987T35; Spaulding1991T28; Szilagyi2006T718) clearly documented the reliability of the primary outcome measure. Protection against contamination (of the study groups) was present in only six studies (Buffington1991T29; LeBaron1998T78; Linkins1994T49; Margolis1992T17; Rosser1992T47; Tollestrup1991T18) or 12.8% of the included studies using the criteria in The Data Collection Checklist because most studies of patient reminder or recall systems allocated patients to study groups rather than allocating clinical settings or practices to study groups.

Effects of interventions

Overall Patient Reminder Results Patients receiving the patient reminder or recall interventions were more likely to have been immunized or up‐to‐date with immunizations compared with control subjects, with an odds ratio (OR) of 1.57, using the pooled random effects model (95% CI: 1.41, 1.75). There were ten studies (Buffington1991T29; Frame1994T52; Lukasik1987T85; McDowell1986T46; Ornstein1991T30; Puech1998T99; Rodewald1999T95; Rosser1991T61; Rosser1992T47; Spaulding1991T28) not included in the meta‐analysis because of potential unit of analysis errors but were included in the review. Together, these ten studies have a median odds ratio of 3.37. An additional three studies (LeBaron1998T78; Margolis1992T17; Tollestrup1991T18) are also analyzed separately because they are CBA studies; together, the three CBAs have a median odds ratio of 1.57.

Routine Childhood Immunizations In patients receiving routine childhood vaccinations, reminder or recall interventions increased the likelihood of being vaccinated or up‐to‐date with immunizations with a pooled random effects OR of 1.47 (95% CI: 1.28, 1.68). Two studies (Daley2004T515; LeBaron2004T512) and two comparisons were added to this set of analyses for the 2005 updated review and two (Irigoyen2006T702; Kempe2001T706) to the updated review in 2007. Both new studies in 2007 demonstrated higher rates of age‐appropriate immunizations for patients in the patient intervention arms, compared with controls, with improvements not being statistically significant. In one new study both the intervention and comparison groups received provider prompts (Kempe2001T706). One study reviewed in 2005 (Daley2004T515) examined age‐appropriate immunization rates after a very brief two‐month study period, which may account for the post‐intervention one percentage point difference between study groups. The magnitude of the effects in the included comparisons are in the range of 2 percentage point decrease to 34 percentage point increase in immunization rates. Only one eligible RCT study was excluded for a potential unit of analysis error (Rodewald1999T95); the odds ratio for this study was 6.79 (95% CI: 4.56, 10.11). One CBA study, which demonstrated a positive effect in this population (Tollestrup1991T18), had an Odds Ratio of 4.11 (95% CI: 2.18, 7.76). The other CBA study (LeBaron1998T78) demonstrated a non‐significant effect, but the baseline rates between the study arms were substantially different.

Childhood Influenza Immunizations The three originally included studies of childhood influenza immunizations demonstrated 17 to 26 percentage point improvements in influenza immunization rates for patient reminder or recall intervention groups over very low baseline rates of controls, with a pooled random effects odds ratio of 2.87 (95% CI: 1.65, 4.98). The third study (Daley2004T513), added in the 2005 updated review, showed the least effect and decreased the pooled random effects odds ratio in that update. In 2007 a fourth study of childhood influenza immunizations was added. This study (Kempe2005T707) examined the effect of patient reminder and recall letters on the receipt of influenza vaccination for healthy 6‐ to 23‐ month old children, in contrast to the previous studies which targeted children with high risk conditions. The Odds Ratio of 1.20 (1.07, 1.34) in the Kempe study (Kempe2005T707) reduced the overall effect of patient reminders and recall on childhood influenza immunizations to an Odds Ratio of 2.18 (95% CI: 1.29, 3.70). This new study (Kempe2005T707) reported several limitations, including a vaccine shortage, a pandemic with extensive media coverage, and the use of a telephone survey prior to the intervention to assess attitudes and intentions regarding influenza vaccination.

Adult Pneumococcal, Tetanus, Hepatitis B Immunizations ("other adult") All six included studies and 12 comparisons of adult pneumococcal, tetanus or Hepatitis B immunizations demonstrated higher immunization rates in the patient reminder or recall intervention groups compared with controls. Vaccination increases ranged from 1.8 to 27.4 percentage points, with five comparisons showing at least 20 percentage point increases. In all but one study (Hogg1998T101) (and eight of 12 comparisons) the improvements were statistically significant. The pooled random effects OR for the three studies of adult pneumococcal, tetanus, or Hepatitis B vaccinations without unit of analysis errors (Hogg1998T101; Sansom2003T514; Siebers1985T36) is 2.19 (95% CI: 1.21, 3.99). The above results include one study (with one comparison) which was added to this subgroup for the 2005 updated review. This study (Sansom2003T514) demonstrated a statistically significant increase in Hepatitis B vaccinations for the intervention group, compared with controls. No new studies were added to this subanalysis in the 2007 update. There were three eligible studies (Ornstein1991T30; Rosser1991T61; Rosser1992T47) with potential unit of analysis errors which were not included in the analysis; the median odds ratio for these three studies was 13.32.

Adult Influenza Immunizations Among the 20 included studies of adult influenza immunizations, two studied patient reminders in combination with provider reminders (Becker1989T23; Margolis1992T17), and one study (Carter1986T104) used a less‐intense patient reminder (standard letter) as the comparison. Patient and provider reminder combinations are discussed under "Effectiveness of Different Types of Reminder or Recall Systems." The changes in vaccination rates in the patient reminder or recall groups ranged from 8.5 percentage point decrease to 47 percentage point increase compared with the controls, with half of the comparisons exceeding a 15 percentage point increase. Of the 18 studies of patient reminders only, data from six were not entered in RevMan analysis because of unit of analysis errors. The pooled random effects summary OR for the remaining group of 12 RCT studies (Baker 1998T96; Brimberry1988T33; Buchner1987T34; Carter1986T104; Hogg1998T101; Hull2002T511; Larson1982T39; Moran1992T16; Mullooly1987T67; Nexoe1997T92; Satterthwaite1997T93; Siebers1985T36) without unit of analysis errors was 1.66 (95% CI: 1.31, 2.09). One study (Hull2002T511) of telephone reminders with modest relative increases in influenza vaccinations (OR = 1.3) was added as part of the 2005 review update. This additional study, and the omission of data from studies with unit of allocation errors, resulted in a decrease in the pooled random effects odds ratio for adult influenza vaccinations from the initial review. No additional studies were added to these subanalyses in the 2007 update. There were six eligible studies with potential unit of analysis errors. The median odds rations for these six studies (Buffington1991T29; Lukasik1987T85; McDowell1986T46; Puech1998T99; Rosser1991T61; Spaulding1991T28) was 3.08. One study (Hull2002T511) had a potential unit of analysis error, but the authors reported an adjusted odds ration as well. The adjusted OR is very close to unadjusted, and the impact using a generic inverse variance method of analysis showed minimal effect on the overall odds ratio.

Adolescent Immunizations One study of urban adolescent immunizations (Hepatitis B and Td), added in the 2007 update, was not successful with significantly increasing immunizations using autodialer (OR = 1.14, 95% CI = 0.98, 1.31) (Szilagyi2006T718). This study faced challenges of unstable telephone numbers.

Effectiveness of Different Types of Reminder or Recall Systems We examined the pooled results for randomized controlled trials to compare the effectiveness of different types of reminder and recall systems for routine vaccination of preschool children, child influenza vaccination, adult influenza vaccination, other adult vaccines, and adolescent vaccines. The Table of Comparisons was set up to facilitate these sub‐analyses. All six types of reminder or recall systems appeared to improve immunization rates compared to controls, with five of six types demonstrating statistically signficant improvements. Person‐to‐person telephone reminders were the most effective single approach (OR = 1.92, 95% CI: 1.20, 3.07). Letter reminders were represented by 20 comparisons in the analysis, and were close to the level of effectiveness of phone reminders, with a pooled random effects odds ratio of 1.79 (95% CI: 1.50, 2.15). The effectiveness of postcard, autodialer, and postcard combined with telephone reminders were equivalent in the 2007 update, when excluding adolescent vaccines. The effectiveness of autodialer decreased with the addition of one urban adolescent study (Szilagyi2006T718). Six comparisons from RCT studies were included in the analysis of postcard reminders, with an odds ratio of 1.44 (95% CI: 1.09, 1.89). One CBA study (Tollestrup1991T18) of postcards also demonstrated significant effects with an odds ratio of 4.11. The pooled random effects odds ratios for other statistically significant reminder and recall systems follow: autodialer (OR = 1.29; 95% CI: 1.09, 1.53), and postcard and telephone call combined (OR = 1.45, 95% CI: 1.11, 1.89). Patient reminder and recall in combination with outreach is the least effective approach in this analysis based on the non‐significant odds ratio of 1.37 (95% CI: 0.98, 1.98); however, this subanalysis included only two comparisons. This result for patient reminder and recall with outcomes in the updated review (in 2005) is in stark contrast to the initial review where this intervention type was second in effectiveness to phone reminders. When results of four comparisons of patient reminder recall interventions combined with provider reminder were pooled, the effectiveness exceeded those of patient reminder or recall systems alone (OR = 3.65; 95% CI: 1.54, 8.67). This result did not change from 2005 to 2007. There was also one CBA study (Margolis1992T17) which showed a positive effect of a combined patient and provider intervention with an odds ratio of 1.32.

Heterogeneity In the initial review, the pooled odds ratios were first computed for each of the comparisons within Metaview using the fixed effects model. Within Metaview, a chi square test was available to test for statistical diversity or heterogeneity of the treatment effects that exists among the different sets of data. This chi square test was employed using a 10 percent one‐tailed level of significance because of the low sensitivity of this test. The chi square tests showed heterogeneity in the studies. This finding was anticipated because of the wide range of study methods and procedures employed, intervention types, settings, immunization types, and other study characteristics. It is also possible that some of the heterogeneity can be attributed to the variable baselines levels in immunization rates. After performing tests for heterogeneity, the results for the initial review were re‐calculated using the random effects model. All results in the updated reviews in 2005 and 2007 were performed using the random effect model.

Costs of Patient Reminder or Recall Systems Sixteen studies reported basic cost data, including eight pediatric studies (Stehr‐Green1993T10;Linkins1994T49; Young1980T63; Lieu1997T69; Lieu1998T82; Campbell1994T87; Rodewald1999T95; Wood1998T105) and six studies of adults (Buchner1987T34; McDowell1986T46; Rosser1992T47; Nexoe1997T92; Baker 1998T96; Hull2002T511). Eight studies estimated cost‐effectiveness of reminder and recall systems (McDowell1986T46; Rosser1992T47; Lieu1998T82; Lieu1997T69; Young1980T63; Rodewald1999T95; Young1980T63; LeBaron2004T512). Costs varied widely across studies, due to variability in methods of calculating costs and items included in analyses (such as existing staff or computer programming); different types of reminders used, with telephone reminders being more costly than letter or postcard reminders; different levels of intensity of interventions, from single postcard reminders to repeated reminders plus home visits; and, different study time periods. As a result of the limited cost data reported and variations in the methods the cost information is of limited use.

Discussion

The findings from this systematic review of the literature support the general recommendation (Udovic 1998; Task Force 1999; Shefer 1999; AdHocWorkingGroup) that primary care providers should consider implementing patient reminder and recall systems to improve immunization coverage levels of their practices. However, all included studies were performed in developed country health care systems; therefore, these findings may not be generalizable to low and middle income countries, given the variations in primary care organization and fewer resources in primary care. We found that reminder and recall systems were effective for both children and adults, in all types of medical settings including private practices, academic medical centers, and public health department clinics, and for universally recommended vaccinations such as routine childhood vaccinations as well as targeted vaccinations such as influenza vaccine. Patient reminders were not, however, effective in one new study of adolescent immunizations in an urban setting. In addition, all types of patient reminder and recall were found to be effective, with increases in immunization rates tending to be on the order of 5‐20 percentage points. Telephone reminders were most effective, while letter reminders were somewhat more effective than postcard reminders among mailed reminders. More intensive reminder and recall, such as those using multiple reminders, appeared to be more effective than single reminders.

This study has several potential limitations. First, the scope of the review was limited to studies published in English. At least one study has found that randomized controlled trials published in English were more likely to have positive findings than studies published in German journals (Egger 1997). However, such language bias was not noted in another study (Heidenreich 1999).

A second potential limitation involves publication bias, because the majority of studies were located from EPOC, MEDLINE or references from other studies. Because publication bias typically results in failure to publish studies with negative or null findings, (Chalmers 1990; Easterbrook 1991; Dickerson 1992) it is possible that our findings of positive outcomes in the majority of reviewed studies is partly affected by publication bias and that the impact of reminder and recall is lower than noted in this review. We attempted to minimize publication bias in the initial review by searching the files of the investigators and immunization experts, searching references of published reviews for abstracts, and reviewing abstracts or proceedings of major scientific meetings. In addition, a funnel plot analysis in the initial review examining the relationship between effect size and sample size did not find more precise studies clustered around null results, thereby increasing the plausibility of the positive findings.

A third potential limitation involves aspects of the systematic review process. We grouped studies according to key characteristics of either the patient population or the intervention. We defined these groups a priori, and they represent standard groupings used in other studies. However, it is possible that where differences were noted by group, factors other than the intervention might have accounted for these differences. Limitations of the standard Cochrane review criteria are published elsewhere (Jadad 1998).

A fourth limitation resulted from omitting studies with potential unit of analysis errors from from meta‐analysis. Twelve generally well‐designed studies included in the initial review, were omitted from the analyses in the 2005 and 2007 updates because of potential unit of analysis errors. This loss of information tended to reduce the strength of the measures of effect.

Our method of pooling data has limitations, particularly in light of heterogeneity of some of the data which is often present in meta‐analyses (Gottlieb 1982; Thompson 1991). Because these reminder or recall studies were performed for a variety of populations, using different interventions, in multiple settings, and across 3 decades (in the initial review), it is not surprising that there is between‐study heterogeneity in the results. Because of this heterogeneity, we performed a qualitative analysis of study characteristics that might explain differences in findings among the eight studies that had negative findings; and although it was easy to find explanations for the negative findings in each study, we did not note consistent trends. We used random effects analyses which had consistently more "conservative" (wider confidence intervals) results than fixed effects models. In one subgroup, a single study by Baker (Baker 1998T96) had more than 24,000 subjects and small but significant effects of reminder or recall, while most of the other studies in that group had clinically larger positive effects of reminder or recall. The large sample size and small effect of that one study resulted in heterogeneity within this subgroup, but also resulted in a conservative effect on the pooled results by reducing the overall impact of reminder or recall.

One final limitation pertains to the inevitable lack of perfection in any study selected for inclusion in this review. Individual study results may have been biased as a result of: vaccination status is collected from only one care delivery location (Baker 1998T96; Moran1992T16); the control group includes a less intense intervention, thereby potentially diminishing the observed effect (Carter1986T104; Ferson1995T57; LeBaron2004T512); subjects are lost to follow‐up (Ferson1995T57; Larson1982T39; Stehr‐Green1993T10); extensive media campaigns may contaminate the study groups (Puech1998T99; Kempe2005T707), and vaccine shortages (Kempe2005T707). Some flaws in study design are not very amenable to change, such as external media campaigns. However, practices can work to optimize the effects of patient reminder and recall systems by improving internal operational systems and learning from those programs with the greatest successes.

Authors' conclusions

In all settings that were evaluated, patient reminder and recall systems appear to be effective for improving immunization rates. As such, methods to incorporate reminder and recall systems into practices should have a positive impact on vaccine‐preventable diseases. Different types of reminder and recall systems can be tailored to suit specific provider or practice needs. While person‐to‐person telephone reminders are most effective, they are also generally more costly than other methods, and have not been studied extensively in children except for the use of autodialers, which were found to have smaller but positive effects. Practical issues relevant to choices of reminder and recall systems include: characteristics of current computer systems, staffing, perceived accuracy of patient telephone numbers or addresses, availability of computer programmers, and estimated patient responsiveness to different types of reminders. Because these factors vary widely across nations or geographic regions, immunization leaders will want to interpret the findings in this review with respect to their own setting. For example, settings with widely used computerized immunization registries could adopt postcard reminders sent by the registries. Practitioners today can tailor their own billing systems to function as reminder and recall systems for simple procedures, such as selecting all patients over 65 years of age for reminders about influenza or pneumococcal vaccination. Many billing systems have recently incorporated separate modules that can track immunization status.

A critical issue involves the complexity of "rules" required for a reminder or recall system. The simplest scenario involves elderly adults, because no special immunization algorithm is needed and eligible patients can be selected by birth dates. A slightly more complex scenario involves "flagging" patients with chronic problems, such as asthma or heart disease, that would require influenza or pneumococcal (for adults) vaccination. More sophisticated algorithms are required to track prior immunization status, particularly for the complicated pediatric immunization schedule. A very promising route involves practitioners linking with computerized immunization registries that are being developed throughout the U.S. (CDCP1998; NVAC 1999; USDHHS2000), already available in several nations (Canadian 1998), and being discussed in various forms in developing nations. These registries already contain the necessary algorithms to assess up‐to‐date status of children, and could be modified to deliver patient reminders. Finally, databases of managed care organizations can be modified to become reminder and recall systems. For practitioners, the usefulness of such databases depends on the proportion of a practice's patients covered by the managed care plan and the accuracy of the database information.

Overall, the technology exists, in the developed world, to incorporate patient reminder and recall into routine primary care practice. There are additional benefits to the patient and practice, beyond improving immunization rates. Studies have shown that patients behind with immunizations are also behind in other measures of preventive care, (Fairbrother 1996; Rodewald 1995) and that reminder or recall systems targeting immunizations can also have "spillover effects" to improve other aspects of preventive care, (Rodewald 1999) if they are used within primary care practices. Second, in fee‐for‐service settings, patient reminder and recall systems can increase revenues by increasing visits.

This review focused on patient reminder and recall systems in settings in which the potential recipients are followed by health care providers over time. This is the case throughout the developed countries, in which many or most children and adults have primary health care providers whom they see on a regular or as‐needed basis. The providers could be public or private, physicians or other health care experts, generalists or more specialized providers (such as pediatricians in the US). The cornerstone is that there is a population of potential recipients who would need annual influenza vaccinations (in the case of adults or children with chronic respiratory diseases), or periodic vaccinations on some schedule (in the case of children). In many developing countries or regions, such a situation does not exist; and although health care providers do serve patients, there is little ability to determine the population of eligible vaccine recipients. In these countries, the concept of "vaccination days" or vaccination programs have been employed (WHO 2001; WHO 2002; Global02; UNICEF1996). These strategies involve massive vaccination programs during single days or weeks, in which entire populations are vaccinated. Much of the world success in eradicating polio is due to such programs (WHO 2003), as well as to more "traditional" methods of vaccinating eligible populations within public or private health care systems. However, this review has not evaluated the use of "vaccination day" strategies. Obviously, patient reminder and recall programs are not likely to be applicable in those settings that rely on vaccination days or single, short‐term vaccination programs. However, in virtually all settings in which patient reminder and recall interventions were rigorously evaluated (and all these settings were ones in which patients were connected with a health care system), the reminder and recall systems were found to be effective in improving immunization rates.

The use of patient reminder and recall systems provides the primary care practitioner with real‐life experience at practicing population‐based care, by improving the care for the entire population served by the practice (Halpern 2000). Although medicine is traditionally taught and practiced one patient at a time, and preventive services such as immunizations are delivered to individual patients, the measures of success (such as immunization rates) are population‐based. Such population‐based primary care, while not easy to do in a busy practice, has the potential to improve the quality of care and performance of primary care providers (Halpern 2000; OConnor 1998; Rivo1998).

This study also has implications for research. With the plethora of studies showing that patient reminder and recall systems improve immunization rates in all types of settings, future researchers should consider not simply repeating prior studies but rather building on them. For example, more research is needed on defining the most cost‐effective types of reminder and recall interventions. There was only one study reviewed on adolescent immunization delivery in an urban setting; this study did not demonstrate significant improvement with use of autodialer reminders. With the rising importance of adolescent immunizations and multiple settings that adolescents receive care, additional studies of adolescents would be useful. Further, the rapid implementation of computerized immunization registries presents opportunities for research in implementing, on a community‐wide basis, reminder and recall interventions that appear to be effective in single practice settings. In addition, managed care plans have databases that could be used as the backbone of reminder and recall interventions; studies incorporating such linkages would be helpful. Studies about "fine‐tuning" patient reminder and recall interventions would be helpful, such as investigations of the degree to which different combinations improve outcomes, or the degree to which combinations of patient reminder and recall and other types of interventions improve outcomes. Finally, because the majority of reviewed studies of patient reminder and recall interventions found positive effects, any studies that do not find improved immunizations should carefully investigate the reasons for lack of improvement. Such detailed investigations may uncover important barriers to care delivery that are likely to be useful in better understanding how to improve services for patients.

Feedback

study arms

Summary

1. There are a couple of labelling problems with the graphs as they appear in MetaView. At the moment the intervention groups retain their default labels: "Treatment" and "Control" and it might be better to change the former to "Reminders". More importantly, though, the labels at the bottom of the graphs have not been changed from the defaults, so they are still "Favours treatment" on the left and "Favours Control" on the right. This is wrong, because the outcome is a good one (immunisation) and it may be confusing to some users to find so many black squares and diamonds sitting above a label that says "Favours Control", when the results and conclusions are that reminders are beneficial in relation to increasing immunisation.

2. The large 4‐arm Baker study is commented on a few times in the review and the reviewers do sensitivity analyses that exclude it. However, the whole trial does not seem to be included in the analyses and it probably should be. Specifically, analysis 01.01 (postcards) has a single intervention group versus the single control group (but there seem to have been two postcard groups in the trial) and analysis 08.01 also has a single intervention group when all three reminder groups should probably be included. Although this is unlikely to have any major impact on the general findings, it is likely to affect the values for the Odds Ratios in the text and abstract.

Reply

From the EPOC editorial base and Julie Jacobson Vann

We have made the editorial changes to address the first point.

Concerning the second point. We will work on the best way to include the other study arms of the Baker study, and addition study arms for other multi‐arm studies. We anticipate being able to include this in the next version of the library (Issue 2, 2003)

Contributors

Mike Clarke, UK Cochrane Centre

Acknowledgements

We wish to acknowledge the CDC for providing funding for the research (grant # 5‐29029) related to the initial review, Department of Health (England) Cochrane Review Update Incentive Scheme for their funding for the 2005 update, Sharon Humiston for her help and wisdom, and Clay Bordley, Peter Margolis, Ron Kraus, Page Abrahamson, Ann Chelminski and Lance Rodewald for their contributions made during the initial review. We also wish to acknowledge Alain Mayhew and Jessie McGowan for their administrative assistance and guidance provided during the 2005 updated review, Doug Salzwedel for assistance in 2007, and Lisa Bero and Jeremy Grimshaw for editorial assistance and guidance with the review process.

Data and analyses

Comparison 1.

postcard reminder vs. control

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Immunized	6	20749	Odds Ratio (M‐H, Random, 95% CI)	1.44 [1.09, 1.89]
1.1 Influenza‐child	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Preschool‐child	3	1484	Odds Ratio (M‐H, Random, 95% CI)	1.70 [0.85, 3.42]
1.3 Influenza‐adult	3	19265	Odds Ratio (M‐H, Random, 95% CI)	1.33 [0.91, 1.93]
1.4 Other‐adult	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Immunized	18	28329	Odds Ratio (M‐H, Random, 95% CI)	1.79 [1.50, 2.15]
1.1 Influenza‐child	4	7264	Odds Ratio (M‐H, Random, 95% CI)	2.18 [1.29, 3.70]
1.2 Preschool‐child	5	1311	Odds Ratio (M‐H, Random, 95% CI)	1.58 [1.26, 1.99]
1.3 Influenza‐adult	9	18319	Odds Ratio (M‐H, Random, 95% CI)	1.78 [1.32, 2.40]
1.4 Other‐adult	2	1435	Odds Ratio (M‐H, Random, 95% CI)	3.45 [1.51, 7.88]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Immunized	4	2465	Odds Ratio (M‐H, Random, 95% CI)	1.92 [1.20, 3.07]
1.1 Influenza‐child	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Preschool‐child	1	103	Odds Ratio (M‐H, Random, 95% CI)	4.25 [1.85, 9.75]
1.3 Influenza‐adult	2	1838	Odds Ratio (M‐H, Random, 95% CI)	1.64 [0.84, 3.22]
1.4 Other‐adult	1	524	Odds Ratio (M‐H, Random, 95% CI)	1.59 [1.00, 2.55]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Immunized	4	11589	Odds Ratio (M‐H, Random, 95% CI)	1.29 [1.09, 1.53]
1.1 Influenza‐child	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Preschool‐child	3	8583	Odds Ratio (M‐H, Random, 95% CI)	1.43 [1.30, 1.57]
1.3 Influenza‐adult	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.4 Other‐adult	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.5 Adolescent	1	3006	Odds Ratio (M‐H, Random, 95% CI)	1.14 [0.98, 1.31]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Immunized	5	3535	Odds Ratio (M‐H, Random, 95% CI)	1.45 [1.11, 1.89]
1.1 Influenza‐child	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Preschool‐child	5	3535	Odds Ratio (M‐H, Random, 95% CI)	1.45 [1.11, 1.89]
1.3 Influenza‐adult	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.4 Other‐adult	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Immunized	3	3057	Odds Ratio (M‐H, Random, 95% CI)	3.65 [1.54, 8.67]
1.1 Influenza‐child	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Preschool‐child	2	2689	Odds Ratio (M‐H, Random, 95% CI)	3.57 [1.03, 12.41]
1.3 Influenza‐adult	1	104	Odds Ratio (M‐H, Random, 95% CI)	3.4 [1.10, 10.50]
1.4 Other‐adult	1	264	Odds Ratio (M‐H, Random, 95% CI)	4.34 [1.15, 16.42]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Immunized	33	60922	Odds Ratio (M‐H, Random, 95% CI)	1.57 [1.41, 1.75]
1.1 Influenza‐child	4	7264	Odds Ratio (M‐H, Random, 95% CI)	2.18 [1.29, 3.70]
1.2 Preschool‐child	15	15704	Odds Ratio (M‐H, Random, 95% CI)	1.47 [1.28, 1.68]
1.3 Influenza‐adult	12	32989	Odds Ratio (M‐H, Random, 95% CI)	1.66 [1.31, 2.09]
1.4 Other‐adult	3	1959	Odds Ratio (M‐H, Random, 95% CI)	2.19 [1.21, 3.99]
1.5 Adolescent	1	3006	Odds Ratio (M‐H, Random, 95% CI)	1.14 [0.98, 1.31]

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1 Immunized	2	1894	Odds Ratio (M‐H, Random, 95% CI)	1.37 [0.98, 1.92]
1.1 Influenza‐child	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.2 Preschool‐child	2	1894	Odds Ratio (M‐H, Random, 95% CI)	1.37 [0.98, 1.92]
1.3 Influenza‐adult	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]
1.4 Other‐adult	0	0	Odds Ratio (M‐H, Random, 95% CI)	0.0 [0.0, 0.0]

Date	Event	Description
14 August 2008	New citation required but conclusions have not changed	New search July 2007, 4 new studies
12 June 2008	Amended	Converted to new review format.
15 February 2008	New search has been performed	New searches, no changes to findings
25 May 2005	New citation required and conclusions have changed	Substantive amendment

Methods	Study Design: RCT Study Duration: 6 months (1/1/91‐6/30/91) Follow‐up: B Outcome Assessment Blinding: B Reliable Outcome Measure: B Contamination: C (patients randomized)
Participants	Inclusion: actively enrolled in practice; not up to date with immunizations Age: 2 months to 7 years Setting: family practice residency clinic (USA) n=464 randomized, 446 analyzed
Interventions	Intervention: Postcard reminder to parent & phone contact 6 weeks after postcard Control: no intervention
Outcomes	#/% of children immunized: 8.8% point increase; #/% children brought up to date: 8.7% point increase
Notes	Did not describe allocation procedure; Abstracted records, but did not report reliability
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: perhaps 1 flu season Follow‐up: B Outcome Assessment Blinding: B Reliable Outcome Measure: A Contamination: C (patients allocated)
Participants	Inclusion: high‐risk adult patients aligned with primary care physician Age: adults; mean age=67.2 years Setting: multispecialty group practice, southeastern Michigan (USA) n=24,743 randomized
Interventions	Intervention: (1) generic postcard to patient; (2) personalized postcard from physician; (3) personalized letter from physician, specific to health risk; Control: no reminder, but comprehensive immunization program for all 4 groups
Outcomes	#/% receiving influenza vaccination (group 1) 2.9% point increase (group 2) 4.1% point increase (group 3) 4.6% point increase
Notes	Patient reminders were 1 component of a comprehensive influenza immunization program. Used billing data for outcomes. Did not capture vaccinations given at other locations (having free vaccine). possible threshold
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: 8 months Follow‐up: B Outcome Assessment Blinding: A Reliable Outcome Measure: B Contamination: C (patients allocated)
Participants	Inclusion: has recorded phone #, at least 1 clinic visit within 18 months, residence outside Long Term Care facility (nursing home or psychiatric) Age: 40‐60 years; Setting: U of Virginia internal medicine clinic (USA) n= 360 patients (immunization study arms)
Interventions	Intervention (1) mailed memo to patient & physician reminder clipped to chart; (2) physician reminder clipped to chart; Control: no reminder; no intervention
Outcomes	Immunization rates: pneumococcal (0.8% point increase), tetanus (8.2% point increase), influenza (16% point increase)
Notes	Multiple interventions (patient & provider reminders). Variable follow‐up for each outcome measure
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: 3 months Follow‐up: A Outcome Assessment Blinding: A Reliable Outcome Measure: B Contamination: C (patients randomized)
Participants	Inclusion: in active patient computer files; high risk for influenza & complications Age: not clear Setting: Family Medical Center, University of Arkansas (USA) n=787 patients
Interventions	Intervention: (1) mailed form letter; (2) telephone reminder ; Control: No intervention
Outcomes	#/% receiving influenza vaccination (1) 5.9% point increase (2) 5.5% point increase
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

Methods	Study Design: RCT Study Duration: possibly 1 influenza season (1984) Follow‐up: C Outcome Assessment Blinding: A Reliable Outcome Measure: B Contamination: C (patients randomized)
Participants	Inclusion: active patients, not nursing home resident, no flu shot or egg allergy Age: >=65 years Setting: private practices of 3 internists near Seattle, Washington (USA) n=655 patients randomized; 540 analyzed
Interventions	Intervention: Postcard reminder in business envelope Control: no intervention
Outcomes	% receiving influenza vaccination; 1.0 percentage point increase
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: 7‐13 months Follow‐up: A Outcome Assessment Blinding: B Reliable Outcome Measure: B Contamination: B
Participants	Inclusion: infants enrolled at clinic, but not those receiving well care from first author Age: infants birth ‐ 7 months Setting: pediatric continuity clinic in teaching hospital, Rochester, New York (USA) n=288 patients enrolled & analyzed
Interventions	Intervention: (1) letter one week before appointment; (2) postcard reminder one week before appointment Control: no reminder letter or postcard
Outcomes	#/% receiving 3 DTP by 7 months of age (group 1) 5.9% point increase (group 2) 2.5% point increase
Notes	Chart audits performed to determine date DTP received; Letters reminded patients of appointments & discussed several topics; postcards reminded patients of appointments only
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT, stratified by age & diagnosis Study Duration: 2 week flu shot clinic in October Follow‐up: A Outcome Assessment Blinding: A (done for survey) Reliable Outcome Measure: A Contamination: C (patients allocated)
Participants	Inclusion: patients at high risk for influenza complications who had not received flu shot in previous year Age: adults Setting: Veterans Administration Medical Center, general medical clinic, Seattle, Washington (USA) n=284 patients
Interventions	Intervention: (1) standard letter & informational brochure; (2) augmented letter; (3) augmented letter & informational brochure; Control: standard letter used as comparison
Outcomes	#/% receiving influenza immunization; (group 1) 13% point increase (group 2) 7% point increase (group 3) 23% point increase
Notes	Control group includes a patient reminder (standard letter), so no true control group
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: 7/2002 through 5/2003 (11 months) Follow‐up: B Outcome Assessment Blinding: B Reliable Outcome Measure: C Contamination: C (patients allocated)
Participants	Inclusion: pediatric patients with high‐risk conditions, record in registry & billing database, & clinic visit within 18 months Providers: pediatricians & mid‐level provders Age: 6 to 72 months Setting: 4 private pediatric practices, Denver, Colorado (USA) n = 1851
Interventions	Intervention: staged reminder letter & postcard recall Control: standard practice (may have included some personal reminders)
Outcomes	#/% receiving influenza vaccination; 17 percentage point increase
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

Methods	Study Design: RCT Study Duration: 2 months (June ‐ July 2000) Follow‐up: B Outcome Assessment Blinding: B Reliable Outcome Measure: B Contamination: C (patients allocated)
Participants	Inclusion: children not up to date with immunizations Age: 5 to 17 months Setting: pediatric primary care clinic of an inner‐city teaching hospital, Denver, Colorado (USA) n = 420
Interventions	Intervention: postcard reminder to parents; phone recall if not seen or scheduled Control: standard practice (includes quality improvement initiative, chart prompts, provider reminders)
Outcomes	#/% up to date with immunizations (point estimates); 1 percentage point increase
Notes	Very brief study period (2 months) to get children up‐to‐date (may not be sufficient); QI did not improve accuracy of parent contact information; other SES barriers may have contributed to results
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

Methods	Study Design: RCT Study Duration: approximately 3‐5 months Follow‐up: C Outcome Assessment Blinding: B Reliable Outcome Measure: C Contamination: C (patients randomized)
Participants	Inclusion: school children located where child health screening occurred in 1991 Age: 5‐6 years, in kindergarten Setting: 28 primary schools in Eastern Sydney (Australia) n=239 children
Interventions	Intervention: Telephone call, letter & brochure to parents Control: letter & brochure to parents
Outcomes	#/% immunized for measles, mumps & DTP; 34% point increase
Notes	25.8% of intervention & 34.4% of control subjects lost to follow‐up. Control group also included patient reminder; Outcomes obtained verbally from parents
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT (stratified using 4 criteria) Study Duration: 2 year study; 1 year followup per intervention Follow‐up: B Outcome Assessment Blinding: B Reliable Outcome Measure: B Contamination: C (randomized families)
Participants	Inclusion: families active in the practice Age: 21 years of age or older Setting: rural, multiple office, nonprofit, fee‐for‐service, family practice; Dansville, New York (USA) n=1008 families; 1665 adult family members
Interventions	Intervention: telephone reminders to patients, computer‐generated health maintenance status report on chart & 2 hour provider instruction session; Control: Manual flowchart based health maintenance tracking system
Outcomes	Provider compliance with health maintenance protocol (% immunized for Tetanus‐Diphtheria); 20% point increase
Notes	Randomized families; data not entered in RevMan
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: 1990‐91 (1 year) Follow‐up: A Outcome Assessment Blinding: A Reliable Outcome Measure: A Contamination: C (patients allocated)
Participants	Inclusion: registered patients who made >= 1 visit in previous 2 years Age: mean = 37.1‐41.6 years Setting: community‐based care; rural family medicine center (Canada) n=1998 patients; 719 families
Interventions	Intervention: (1) computer‐generated customized letters; (2) form letter to patients; Control: no letters, but physician reminder system existed for all patients
Outcomes	#/% overdue who received adult tetanus, flu > 65 years, flu (chronic disease), MMR, HIB, DPT, & TOPV immunizations; Outcome range: 5.9% point decrease to 2.6% point increase
Notes	Computerized allocation of families; Baselines differed significantly
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

PERMALINK

Patient reminder and recall systems to improve immunization rates

Julie C Jacobson Vann

Peter Szilagyi

Abstract

Background

Objectives

Search methods

Selection criteria

Data collection and analysis

Main results

Authors' conclusions

Does reminding people to have vaccinations increase the number of people who receive vaccinations?

Background

Objectives

Methods

Criteria for considering studies for this review

Types of studies

Types of participants

Types of interventions

Types of outcome measures

Search methods for identification of studies

Data collection and analysis

Results

Description of studies

Risk of bias in included studies

Effects of interventions

Discussion

Authors' conclusions

Feedback

study arms

Summary

Reply

Contributors

Acknowledgements

Data and analyses

Comparison 1.

Analysis 1.1.

Comparison 2.

Analysis 2.1.

Comparison 3.

Analysis 3.1.

Comparison 4.

Analysis 4.1.

Comparison 5.

Analysis 5.1.

Comparison 6.

Analysis 6.1.

Comparison 7.

Analysis 7.1.

Comparison 8.

Analysis 8.1.

What's new

History

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Characteristics of excluded studies [ordered by study ID]

Contributions of authors

Sources of support

Internal sources

External sources

Declarations of interest

Notes

References

References to studies included in this review

References to studies excluded from this review

Additional references

References to other published versions of this review

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases

Methods	Study Design: RCT Study Duration: 9/00 ‐ 10/00 (2 months) Follow‐up: A Outcome Assessment Blinding: A Reliable Outcome Measure: B Contamination: C (patients allocated)
Participants	Inclusion: registered patients w/o chronic disease; Age: 65 ‐ 74 years Setting: 3 general practices (UK) n = 1261 patients
Interventions	Intervention: phone call to patient Control: untargeted activity (city sent letter & brochure)
Outcomes	Influenza immunization rates; 5.9 percentage point increase
Notes	Reported differences as percent rather than percentage point changes (probable error). allocated households resulting in unit of analysis error; adjusted OR in paper (minimal effect). Included data
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

Methods	Study Design: RCT Study Duration: 9/11/01 ‐ 3/31/02 (6.5 months) Follow‐up: A Blinding: B Reliable Outcome Measure: C Contamination: C (patients allocated)
Participants	Inclusion: visit to network & due for DTaP Age: 6 weeks to 15 months Setting: 5 community‐ based peds practice, New York city (USA) n = 1662
Interventions	Intervention: (1) continuous reminders (weekly postcards) (2) limited reminders (up to 3 postcards) Control: no intervention
Outcomes	Up to date with DTaP; 4.3 percentage point increase
Notes	25.6% misclassification of DTaP; postcards returned for 13.6% children; vaccine shortage
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

Methods	Study Design: RCT Study Duration: 9/1/03 ‐ 2/29/04 Follow‐up:B Blinding: B Reliable Outcome Measure: A Contamination: C (patients randomized)
Participants	Inclusion: children visiting practices in previous 18 months & in immunization registry Age: 6 to 21 months Setting: 5 peds practices in metropolitan Denver, Colorado (USA) n = 5193
Interventions	Intervention: up to 3 reminder/recall letters generated by registry Control: standard practice
Outcomes	Receipt of 1 or more influenza immunizations 2003 ‐ 2004 season; 4.4 percentage point increase
Notes	Possible contamination of both groups may have attenuated the observed effect: telephone survey before intervention & pandemic w extensive media coverage
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: January ‐ July 1999 (6‐7 months) Follow‐up: B Blinding: A Reliable Outcome Measure: B Contamination: C (patients randomized)
Participants	Inclusion: seen for well‐child care or acute illness in clinic Age: 5 to 17 months Setting: inner‐city hospital‐based teaching clinic, Denver, Colorado (USA) n = 603
Interventions	Intervention: Postcard & attempts to call; provider prompts; Control: provider prompts
Outcomes	Up to date with immunizations; 4 percentage point decrease to 12 percentage points increase
Notes	Provider prompts for both groups may have influenced results; data somewhat unclear; inadequate immunization records for ˜ 18%; unable to contact 28.1%
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: 1 flu season (fall 1991) Follow‐up: A Outcome Assessment Blinding: A Reliable Outcome Measure: B Contamination: C (patients randomized)
Participants	Inclusion: received primary care at 1 children's clinic; 2 or more emergency/clinic visits in past year for asthma Age: children at least 6 months old Setting: primary clinic serving poor, urban children in Seattle, Washington (USA) n=96 randomized
Interventions	Intervention: one computer‐generated letter to parent & standing order; Control: standard practice (memo to providers on recommendations)
Outcomes	#/% children immunized with influenza vaccine; 26 percentage point increase
Notes
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

Methods	Study Design: CBA Study Duration: 1 year; 9/1/92 ‐ 8/31/93 Follow‐up: B Outcome Assessment Blinding: A Reliable Outcome Measure: B Contamination: A
Participants	Inclusion: patients of 4 clinics or residents of 9 communities Age: 3 to 59 months Setting: community based organization; Fulton County, Georgia (USA) n= 4 public clinics; 9 inner city communities
Interventions	Intervention: (1) "clinic": phone, mail or home visit with family; (2) "community": door‐to‐door campaign; Control: no intervention
Outcomes	Age‐appropriate vaccination rates; series completion rates; Immunizations increased by 15 % points in intervention groups; no change in controls
Notes	Data not entered in RevMan; CBA, allocation by practice
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	D ‐ Not used

Methods	Study Design: RCT Study Duration: 9/1996 ‐ 8/1998 (2 years) Follow‐up: B Outcome Assessment Blinding: C Reliable Outcome Measure: B Contamination: C (patients allocated)
Participants	Inclusion: inner city birth cohort, patients of public facilities Providers: city‐wide hospital, clinic, health dept. Age: 1 to 14 months Setting: Atlanta, Georgia (USA) n = 3050
Interventions	Intervention: (1) autodialer; (2) outreach; (3) autodialer & outreach Control: standard practice (may include postcard reminders)
Outcomes	Age‐appropriate vaccination rates' (group 1) 6% point increase (group 2) 3% point increase (group 3) 4% point increase
Notes	Limitation: postcard recall systems in control group may have attenuated the results; lacked vaccination records from providers not in registry
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

Methods	Study Design: RCT Study Duration: 4 months per subject Follow‐up: A Outcome Assessment Blinding: A Reliable Outcome Measure: A Contamination: C (patients allocated)
Participants	Inclusion: enrolled children at 2 medical centers Age: 20 months between 1/94 & 11/94 Setting: Kaiser Permanente (group model Health Maintenance Organization), northern California (USA) n=321 patients randomized
Interventions	Intervention: personalized letter & brochure; Control: no letter
Outcomes	#/% MMR recorded on Kaiser immunization tracking system or parental report of MMR received outside the system (by 24 months): 19% point increase
Notes	Randomized patients using random number generator
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate

Methods	Study Design: RCT with nonrandomized controls Study Duration: 9/96 ‐ 1/97 Follow‐up: A Outcome Assessment Blinding: A Reliable Outcome Measure: A Contamination: C (patients randomized)
Participants	Inclusion & Age: underimmunized 20 month olds identified by HMO (Health maintenance organization) Setting: HMO, Northern California (USA) n: 752 randomized, 648 analyzed
Interventions	Intervention: (1) automated phone message followed by letter; (2) automated phone message; (3) letter; (4) letter followed by automated phone message Control: no systematic intervention
Outcomes	#/% needed immunizations received by 24 months; Odds Ratios for combined interventions = 2.1 & 2.5; (group 1) 17.7% point increase (group 2) 8.2% point increase (group 3) 8.6% point increase (group 4) 22.2% point increase
Notes	Computerized immunization tracking system may not have complete vaccine information for children enrolled after 42 days of age; randomized to 4 intervention groups; control groups were nonrandomized
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Unclear risk	B ‐ Unclear

Methods	Study Design: RCT Study Duration: 1 month Follow‐up: A Outcome Assessment Blinding: A Reliable Outcome Measure: A Contamination: A
Participants	Inclusion: if computerized immunization record had phone # Age: < 2 years Setting: 14 counties in urban & rural Georgia (USA) n=8,002 patients
Interventions	Intervention: computer‐generated phone reminders (general vs. specific reminders) Control: no intervention
Outcomes	rates of immunization visits (childhood vaccines); 7.9% point increase
Notes	contacted 70.3% of households; allocated after determined immunization status; randomized patients in 14 counties
*Risk of bias*
Bias	Authors' judgement	Support for judgement
Allocation concealment?	Low risk	A ‐ Adequate