A meta-analysis of influenza vaccination following correspondence: Considerations for COVID-19

Abstract

Background

High vaccination rates are needed to protect against influenza and to end the COVID-19 pandemic. Health authorities need to know if supplementing mass communications with direct correspondence to the community would increase uptake.

Objectives

The primary objective is to determine if sending a single written message directly to individuals increases influenza vaccine uptake, and a secondary objective is to identify any identified content shown to increase influenza vaccine uptake.

Methods

MEDLINE, Embase, Cochrane CENTRAL, PsycINFO, and PubMed were searched for RCTs testing a single correspondence for members of the community in OECD countries to obtain influenza vaccination. A meta-analysis with inverse-variance, random-effects modelling was used to estimate a mean, weighted risk ratio effect size measure of vaccine uptake. Studies were quality assessed and analysis was undertaken to account for potential publication bias.

Results

Twenty-eight randomized controlled trials were included, covering 45 interventions. Of the 45 interventions, 37 (82.2%) report an increase in influenza vaccination rates. A formal meta-analysis shows that sending a single written message increased influenza vaccine uptake by 16%, relative to the no contact comparator group (RR = 1.16, 95% CI [1.13–1.20], Z = 9.25, p < .001). Analysis shows that the intervention is effective across correspondence type, age group, time, and location, and after allowing for risk of publication bias.

Limitations

The generalizability of results across the OECD may be questioned.

Conclusions and implications

The implication for public health authorities organizing vaccination programs for influenza, and arguably also for COVID-19, is that sending written vaccination correspondence to members of the community is likely to increase uptake.

This study is pre-registered on osf.io; details can be found at https://osf.io/98mr7.

1. Introduction

Mass vaccination has a vital role to play in ending the COVID-19 pandemic. Given higher transmissibility of new variants, and an optimistic estimate of efficacy across the available vaccines of 0.80, achieving herd immunity requires a high rate of uptake of available vaccines.[1], [2] When examining global trends in vaccination from 2015 to 2019, confidence in vaccination was identified as a key driver of improved vaccine uptake.[3] As such, across many countries, public health authorities are using mass communications to address public confidence in the perceived safety, effectiveness, and importance of vaccination programs.[4], [5], [6], [7] Mass communications include public service announcements, media campaigns, notices to healthcare providers, and news coverage. A practical question for public health authorities, especially in OECD countries as they have well developed immunization programs and community wide access to social media, is whether supplementing mass communications with direct correspondence would increase vaccine uptake.

Previous experience in promoting adult influenza vaccination programs is relevant to answering this important question for COVID-19 vaccination programs. Crucially, both programs target the decision of adults on whether or not to vaccinate themselves. The transferability of learning about decisions from children’s vaccination programs to the uptake of COVID-19 vaccines by adults is limited, however, because concerns for dependent children can be very different to the concerns that parents and guardians have for themselves as vaccine recipients.[8] Furthermore, it is plausible that perceived personal threat and risk with regard to age and health vulnerability to the adult influenza is patterned in the same way as COVID-19, thus highlighting the relevance of research examining influenza vaccination to the present pandemic.[9] Examining the experience of influenza vaccination programs also has the advantage of providing results over four decades and covering a virus that was the cause of the most recent global pandemic prior to COVID-19: the 2009 pandemic hemagglutinin type 1 and neuraminidase type 1 (H1N1) influenza which had marked similarities to COVID-19 in terms of rapid spread.[10]

Several previous systematic reviews examine methods for increasing influenza vaccine uptake[11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23] Previous qualitative reviews typically conclude that direct communications increase[12], [18], [20] or show potential to increase[14], [16], [17] vaccine uptake, which has been further reinforced through meta-analyses that indicate a positive pooled effect size of direct communications, contributing to increased vaccine uptake, relative to control[11], [13], [15], [19], [21], [22], [23].However, previous qualitative reviews have tended to be broad-ranging, covering multiple vaccine preventable diseases and not solely influenza[12], [14], [17], [20], but also somewhat limited in the evidence-based recommendations that can be offered as only a subset of communication types were considered (e.g., solely email[12], personal electronic health records[20], or new media[17]), focused solely on people living with a chronic illness[18] or in a single country[14]. Additionally, most reviews included non-randomized as well as randomized studies[12], [14], [16], [17], [20] thus, it is possible that confounding factors to some degree, account for intervention effects, owing to the inclusion of non-randomized trial designs. Amongst the relevant meta-analyses, estimates have included a pooling across correspondence and non-correspondence interventions (e.g., the estimates also include face to face and poster messages[15] or telephone calls and voicemail messages[22]) or have exclusively considered a subgroup of the population (such as people aged 60 plus[13] or with chronic illness[21]). Whilst two systematic reviews with accompanying meta-analyses demonstrated that mailed print material[23], or exclusively letters or postcards[19], significantly increased vaccination rates in the community, relative to control, such have not focused on single correspondence and new primary studies have since been published. Single messaging is less time and resource intensive than sending multiple reminders, but previous reviews do not address the issue of whether a single direct correspondence alone increases flu vaccine uptake by people living in the community in OECD countries. For public health organizations with responsibility for communicating with the public and for clinicians with responsibility for patients, there is uncertainty as to whether sending a single written message directly to individuals is likely to increase influenza vaccine uptake. There is also uncertainty as to whether an effect on uptake due to correspondence differs by type of correspondence (e.g., letter, postcard, portal message, smartphone message), by continent, year of publication, by age group or by the institutional setting of the message sender.

The aim of this systematic review and meta-analysis is to compile evidence from randomized control trials (RCTs) on the effectiveness of sending a single written message to an individual to encourage influenza vaccination. Our primary question is: “does sending a single written message directly to individuals increase influenza vaccine uptake?” Since the specific content and design elements of correspondence may play an important role in vaccine uptake, secondary questions are: “what content is included in any correspondence that is shown to increase influenza vaccine uptake?” and “is there any evidence of the comparative effectiveness of different content or design elements?” Several previous systematic reviews, listed in the methods section, examine methods for increasing influenza vaccine uptake but none of the previous reviews address our specific research questions.

2. Methods

2.1. Inclusion criteria

This study is pre-registered on osf.io; details can be found at https://osf.io/98mr7. Studies were included if they met the following criteria: compared influenza vaccination rates where a single correspondence was sent versus no correspondence; was a randomized controlled trial with an appropriate control group; was not part of a reminder for a pre-existing appointment; was published in a peer-reviewed journal; was not exclusive to health care workers or children; and was conducted in an OECD country (given the particular relevance of such countries to the primary study question). The reference to “appropriate control group” refers to the fact that two studies were excluded because they were reported as randomized controlled trials with a control group but the control group was judged not to be appropriate: participants in the control group of one study comprised pregnant women who reported not participating in an ‘opt in’ SMS information service[24] and in another study those with a report of ‘pending’ message, ‘unknown number’, or without available mobile telephone number were categorized as the control group. Each record was screened by RM and the results independently reviewed by CT.

2.2. Moderators

Information was recorded for each study on (a) the country in which it was conducted, (b) year of publication, (c) age group, and (d) institutional setting of correspondence sender. We also classified interventions into (e) type of correspondence and (f) summary assessment of risk of bias. We attempted to identify if interventions were personalized or not, using the definition of a previous systematic review[16] of personalized communication as that “which aims to make a personally relevant appeal to individuals by, for example, using direct contact or individually addressed correspondence”, but insufficient detail was reported to classify all interventions (see Table A.1). Subgroup analysis was conducted based on the classification of (a) to (f).

2.3. Search strategies

A search was undertaken of MEDLINE, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), PsycINFO, and PubMed up to the 19th of August 2021 using the search string below. CT and RM also hand-searched the references of 13 systematic reviews found in the search[11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23].

The search string used was:

• ((vaccin* OR *immunis*) AND (flu OR influenza) AND (letter* OR email* OR SMS OR text OR postcard* OR brochure* OR reminder* OR invitation* OR “portal message”) AND (vaccinated OR vaccination rate* OR uptake OR take-up OR “take up” OR effect*) AND (RCT OR random* OR trial OR quantitative OR empirical OR experiment* OR test*))

2.4. Coding of outcomes and content

CT and RM collected data on the outcome of interest: the effect of interventions on vaccine uptake. This data was reviewed by EA, with odds/risk ratios calculated for the purposes of meta-analysis. Where data was missing, percentage vaccinations were collected. Study/sample characteristics were also collected for use in subgroup analyses: year of study; country; patient type (e.g., chronic illness, elderly, healthcare insured); high risk group (Y/N); patient age group.

Content of messaging interventions was coded in part by RM and completed by CT, with each reviewing the other’s coding. Sixteen studies showing an effect included information on the content of correspondence; this content was coded into 18 elements, grouped as follows: (a) recommendation to get the vaccine; advice to get the vaccine soon; advice to get the vaccine every year; (b) information about the clinical manifestations of influenza; statement on the seriousness of influenza / possible complications from influenza; statement that the vaccine helps avoid serious complications / is effective; (c) statement that the vaccine is safe / has minimal side effects; statement that the vaccine can cause minor side effects; addresses common concerns about the vaccine; (d) statement on the importance of the vaccine for high-risk people; statement of who is at high risk of complications from the flu; statement that the recipient is at high risk of complications / a serious case of the flu; (e) information on how and where to get the vaccine / scheduling information; access to online scheduling; clinic operating hours; clinic locations; information on the availability of the vaccine; statement that the vaccine is free. The template data collection forms and the data extracted from included studies is available upon request.

2.5. Assessment of risk of bias

The Cochrane Risk of Bias tool was administered to assess the risk of bias across the studies included in the systematic review and meta-analysis.[25] This tool consists of six bias domains assessed across seven items: selection bias (random sequence generation, allocation concealment), performance bias, detection bias, attrition bias, reporting bias, and other bias. A judgement of high, unclear, or low risk of bias was assigned based on the reported trial characteristics. Each study record was assessed by either EA or GMcM, with a sample (13/27; ∼50%) of records blindly and independently assessed by both EA and GMcM for reliability. Any conflicts between assessors were discussed in relation to the supporting information provided by the assessor for the bias judgement and a final consensus was agreed between EA and GMcM. In accordance with the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) guidelines 2015[26], [27], the quality of evidence on the effectiveness of direct written correspondence interventions on vaccine uptake was assessed as high, moderate, low, or very low.

For the studies included in the meta-analysis (k = 27; n = 41 subsamples) a summary assessment of risk of bias was computed using three of the domains from the risk assessment tool and studies were categorized into three groups based on the summary assessment using the framework as recommended by the Cochrane Risk of Bias tool: high, unclear, and low risk of bias. The three domains used were selection bias (concealment of allocation prior to randomization), performance bias (blinding of participants and study personnel), and detection bias (blinding of outcome assessors). This selection was informed by previous work that identified allocation concealment and blinding as the components of methodological quality most closely associated with the estimate of intervention effect.[28] For example, inadequate concealment of allocation can introduce a bias if the investigator (and/or healthcare professional) has strong beliefs about the potential benefits of the intervention, which (in-)directly may confound the intervention process.

2.6. Statistical methods for estimating effect size

The events of vaccination and total events (i.e., subsample size, inclusive of events and non-events) from the intervention and control groups were inputted into Review Manager v5.4 to generate risk ratio effect sizes. This was calculated as (SI / NI) / (SC / NC), where SI / NI = the number of ‘success’ events (vaccination) divided by the total events in the intervention group and SC / NC = the number of ‘success’ events (vaccination) divided by the total events in the control group. When only the percentage vaccination rate for both the intervention and control groups was reported, the absolute risk was derived from this percentage using the relevant denominator (i.e., subsample size of the intervention group or control group) reported in the respective study.[29]

Inverse-variance weighted, random-effects modelling was conducted to determine the mean risk ratio across the included studies. A random-effects model was selected to account for variability between studies which can likely be explained by factors other than sampling error[30], for example, variance in the sample characteristics and the intervention components between studies. The risk ratio effect size contributed by each study was weighted by its inverse variance so that studies with a larger sample size were given more weight in the analyses to ensure precision in the mean, weighted effect size estimate.[30] Each study contributed only one effect size to the meta-analysis per written correspondence intervention; this avoided weighting individual studies by the number of subsamples reported (e.g., if vaccination was reported by age group for the respective intervention) and also to ensure statistical independence of effect sizes.[31]

A mean, weighted effect size and 95% confidence intervals were generated for the meta-analysis and presented visually in a Forest plot along with the study-level effect sizes. The Z statistic was interpreted against a 0.05 alpha level to test the null hypothesis that the mean, weighted effect size was 0; a significant Z statistic indicated that the mean, weighted effect was significantly different from 0. Heterogeneity, resulting from differences between the study-level effect sizes that contributed to the mean, weighted estimate, was evaluated with the Q statistic Chi-square test. Due to low power in a meta-analysis with a small number of studies, the alpha level was set to 0.10, as recommended.[29] The I2 index was applied to quantify the amount of heterogeneity between studies that could be explained by true heterogeneity rather than chance. This was interpreted in accordance with the recommended criteria: 25–49% = small, 50–74% = moderate, and 75%+ = large heterogeneity.[30]

Categorical variables such as the characteristics of the sample (age group), intervention (type of written correspondence) and study (location (continent), year of publication (decades), risk of bias assessment) were considered for subgroup analyses. A minimum of two studies were required per category in the subgroup analyses to ensure sufficient power to determine whether the categorical variable was a significant moderator of effect size.[30]

2.7. Assessment of risk of publication bias

A funnel plot (log risk ratio by standard error) was generated and visually inspected for asymmetry to determine the presence of publication bias. This is typically observed by missing studies towards the bottom of the graph on one side of the weighted, mean effect size line, indicating an absence of non-significant or unfavorable outcome studies with small sample sizes (publication bias). Egger’s test was conducted to quantify the funnel plot asymmetry and statistically determine the presence of publication bias. In the detection of publication bias, Duval and Tweedie’s trim and fill analyses were conducted to trim or remove extreme, positive, small studies and then impute the mirror of these studies to produce a symmetric plot and an unbiased, mean estimate of the intervention effect.[30]

3. Results

3.1. Sample of studies

The full texts of 83 articles were screened for eligibility. A total of 28 randomized controlled trials were included in the review (see Fig. 1 ). A description of the 45 interventions used in the studies is provided in the Appendix. One of the studies[32] which accounted for 4 subsamples/intervention arms did not report the required statistics for inclusion in the meta-analysis so 27 studies (inclusive of 44 subsamples) were included in meta-analysis.

Fig. 1.

PRISMA Flow Diagram.

3.2. Study characteristics

The studies were conducted in the USA, Canada, Spain, Denmark, New Zealand, South Korea, Germany, and Australia. Study populations were specified as at-risk or medical condition groups (k = 11); older people (≥65 years) (k = 8); Medicare beneficiaries (k = 4); adults in the general population (k = 3); and adults and children (>6 months) (k = 2). Table A.1 provides a brief description of each intervention. A total of 45 types of broad intervention were used: letter (n = 26); postcard (n = 10); patient portal message (n = 3), educational brochure (n = 1), lottery (n = 1), brochure + lottery (n = 1), mobile app (n = 2), SMS (n = 1). 26 interventions were characterized as “personalized”, with the remainder considered to be generic letters, postcards, SMS, mobile app messages or portal messages.

3.3. Risk of bias

The quality assessment for 27 studies revealed the lowest risk of bias for the first four domains: selection (random sequence generation and allocation concealment), performance, and detection bias. For each of these four domains less than 15% of the studies were judged to be of high risk of bias (details are available in the online supplemental file). However, for the first two of these domains, approximately half of the studies were judged to be unclear due to the lack of transparency; selection domains of random sequence generation (13/27; 48%) and allocation concealment (16/27; 59%). The domain judged to have the lowest risk was detection bias, (18/27; 66%), reflecting blinding of participants and researchers; most studies measured outcomes with objective health records[33] or insurance claim records[34]. The risk was judged to be high in more than half of the studies for the domains of reporting bias (15/27; 55%) - often only the percentage vaccination rate was reported without the corresponding frequencies of events and non-events - and attrition bias (14/27; 52%), due to the lack of explanation for attrition within some studies. High risk of other biases was noted in 33% (9/27) of studies. This was most often related to the possibility of sampling/recruitment bias. For instance, non-random sampling methods were often reported (e.g., site selection[35]) or the criteria used for exclusions may have limited the generalizability of findings (e.g., participant exclusion if believed to object to vaccination[36]).

As shown in Fig. 2 , four studies were deemed low risk across all seven domains[33], [37], [38], [39] 2015, with a further eight studies judged as low risk in at least 4/7 domains[34], [36], [40], [41], [42], [43], [44], [45]. Although no study was deemed high risk across all domains, four studies were deemed high risk across at least 5/7 domains[6], [46], [47], [48]. Risk of bias assessment was conducted on 27 of 28 studies included in the meta-analysis; risk of bias was unable to be determined for one non-English study[49]. The quality of the available evidence, as per GRADE criteria, was deemed as moderate (see Table S.1 in the Supplementary file).

Fig. 2.

Traffic Light Plot: Risk of Bias Assessment.

3.4. Overall effect of correspondence reported in each study

Information on the reported effective size for each of the 28 studies is provided in Table 1 . Of the 45 interventions, 32 (71.1%) are reported to have significantly increased influenza vaccination rates, 7 (15.6%) are reported not to be effective (no effect or negative effect), and the effectiveness of 6 (13.3%) interventions is not explicitly reported in four studies[36], [41], [47], [48]. Sending a reminder letter did not significantly increase vaccination rates in one study[50], while in another study combining an educational brochure with a financial incentive (a lottery to receive a gift certificate) was not effective[51]. In a study where patients were sent customized or form letters reminding them of outstanding preventive care procedures, the effect of these interventions on flu vaccination did not achieve significance[53]. Sending a postcard to older people who had previously received a vaccine was not effective[52], and in another a personalized postcard from a physician to older patients also failed to increase vaccination rates[35]. In one study the intervention was reported to have a negative effect on vaccination rates when pharmacists sent a personalized letter to asthma and COPD patients[53].

Table 1.

Intervention Type, Target Group and Effectiveness.

Studies	Intervention Category	Target group	Vaccination Rate			RR/ OR	95% CI
			Control	Intervention	Abs. Difference
Klassing et al, 2017[53]	PL	Asthma and COPD patients	88.6%	83.7%	−4.9%	p = 0.02
McCaul et al, 2002^[41]	PL (Action)	≥65 years Medicare recipients	19.6%	28.2%	8.6%	z = 12.01, p = 0.01
	PL (PRO)			24.4%	4.8%	Not given. Significance not reported relative to the control.
	PL (Loss)			24.5%	4.9%	Not given. Significance not reported relative to the control.
	PL (Gain)			23.5%	3.9%	Not given. Significance not reported relative to the control.
McDowell et al, 1986[48]	PL	≥65 years	9.8%	35.1%	25.3%	Not given. Significance not reported relative to the control.
Moran et al, 1992[50]	PL	High risk patients	38.2	40%	1.8%	Reported that not significantly different (chi-square p > 0. 10)
Mullooly et al, 1987[42]	PL	≥65 years	30.1%	38.9%	8.8%	RR = 1.29	[1.15;1.45]
Nexøe et al, 1997[55]	PL	≥65 years high risk patients	25%	49%	24%	p < 0.01
Roca et al, 2012[39]	PL	≥60 years	39.5%	43.8%	4.3%	OR = 6.33	[1.15;1.45]
Satterthwaite et al, 1997[56]	PL	>65 years	17%	27%	10%	RR = 1.55	[1.28; 1.88]
Terrell-Perica et al, 2001[44]	PL	Medicare recipients	17.1%	19.8%	2.7%	p = 0.023 [2.70; 3.40]
CDC 1995a[32] (Wyoming)	GL + B	Medicare recipients	33.1%	40.4%	7.3%	OR = 1.91	[1.81; 2.02]
	PL + B			42.7%	9.6%	OR = 1.79	[1.69; 1.90]
CDC 1995b[32] (Montana)	GL + B	Medicare recipients	46.7%	52.5%	5.8%	OR = 1.51	[1.42; 1.61]
	PL + B			49.9%	3.2%	OR = 2.07	[1.45; 2.20]
Minor et al, 2010[57]	PL + B	Hypertension clinic	33%	46%	13%	OR = 1.8	[1.3; 2.5]
Yokum et al, 2018*[34]	PL (NVPR)	Medicare recipients	25.9%	26.6%	0.7%	p = 0.01 [1.01–1.07]
	PL (USSG)			26.8%	0.9%	p < 0.001 [1.02; 1.08]
	PL (Imp)			26.4%	0.5%	p < 0.001 [1.02; 1.07]
	PL (Active)			26.3%	0.4%	p < 0.001 [1.02; 1.07]
Brimberry et al, 1988[47]	GL	High risk patients	11.4%	10.6%	−0.8%	Not given. Significance not reported relative to the control.
Schulte et al, 2019	L (patient centred)	Patients with chronic renal failure	29.1%	37.4%	8.3%	Significant, 95% CI [-0.3; 16.8]
Buchner et al, 1987[35]	PP	≥65 years	54.1%	55.1%	1%	Not significant
Puech et al, 1998[36]	PP	≥65 years	45%	54.5%	11.5%	Not given. Significance not reported relative to the control.
Spaulding et al, 1991[43]	PP	Military family practice	9.1%	25.2%	16.1%	RR = 2.77	[2.05; 3.75]
Clayton et al, 1999[52]	P	≥65 Received vaccine previous year	77.2%	78.6%	1.4%	p = 0.222
Song et al, 2000[49]	P	≥65 years	46.7%	56.3%	9.6%	OR = 1.55	[1.18; 2.02]
Moran et al, 1996[51]	GEB	High risk patients	20%	36%	16%	OR = 2.29	[1.45; 3.61]
	Lottery			29%	9%	OR = 1.68	[1.05; 2.68]
	GEB + Lottery			26%	6%	OR = 1.41	[0.88; 2.27]
Baker et al, 1998[46]	GP	≥65 years <65 years	40.6%	43.5%	2.9%	----	[1.22; 4.79]
	PP			44.7%	4.1%	----	[2.43; 5.98]
	PL			45.2%	4.6%	----	[2.97; 6.53]
Larson et al, 1982[6]	GP	>65 years or various diagnoses	20.2%	25%	4.8%	p < 0.1
	PP			41%	20.8%	p < 0.025
	HBP			51.5	31.3%	p < 0.001
Cutrona et al, 2018[40]	PPM	≥18 years	11.6%	13.4%	1.8%	OR = 1.20	[1.06; 1.35]
Szilagyi et al, 2020[33]	PPM	Adults and children > 6 months	37.5%	38%	0.5%	p = 0.008
Wijesundara et al, 2020[38]	PPM	≥18 years	27.1%	28.4%	1.3%	OR 1.07	[1.02–1.12]
Regan et al, 2017[45]	SMS	High-risk patients	8.9%	12.4%	3.5%	RR = 1.39	[1.26; 1.54]
Lee at al, 2020[37]	MA (conspicuous points)	18–65 years	22%	23.4%	1.4%	RR = 1.06	[1.02; 1.11]
	MA (not conspicuous points)		22%	22.9%	0.9%	RR = 1.05	[1.0; 1.08]
Hogg et al, 1998[58]	PL	≥65 years	19.1%	20%	1%	Not reported as significant.
	GL	≥65 years	19.1%	16.7%	−2.4%	Not reported as significant.

PL = Personalised Letter; GL = Generic Letter; L = Letter; PL + B = Personalised Letter + Brochure; GL + B = Generic Letter + Brochure; PP = Personalised Postcard; P = Postcard; GEB = Generic Educational Brochure; GP = Generic Postcard; HBP = Health Belief Model Postcard; PPM = Patient Portal Message; SMS = Short Message Service; MA = Mobile App.

^ McCaul et al tested four letter types: Action = letter on when and where to get a flu shot; PRO = letter from state peer review organisation (PRO); Loss = PRO letter with loss frame; Gain = PRO letter with gain frame. * Yokum et al tested four letter types: NVPR = letter + picture of National Vaccine Program Officer; USSG = letter + picture of US Surgeon General; Imp = letter + picture of US Surgeon General + implementation intention prompt; Active = letter + picture of US Surgeon General + active choice implementation prompt. ^$ With a billed Medicare service in the previous year.

3.5. Overall estimates of effect size for correspondence

The main analysis included 41 subsamples (intervention arms) across 27 studies (see Fig. 3 ); one study did not report sufficient statistical information for inclusion in the meta-analysis[32]. Sending a single written message increased influenza vaccine uptake by 16%, relative to the no contact comparator group (RR = 1.16, 95% CI [1.13–1.20], Z = 9.25, p < .001).1 There was substantial heterogeneity among the included 41 samples (k = 27 studies), χ2 (40) = 428.71, p < .001, I² = 91%) which warranted further subgroup analyses to determine the influence of patient, intervention and/or institutional characteristics on the effect size measure.

Fig. 3.

Overall Effect Size Estimate.

3.6. Subgroup analyses

Type of correspondence. A single direct message is shown to be effective across all primary correspondence types (see Fig. 4 ). A postcard (35% increased uptake, RR = 1.35, 95% CI [1.15, 1.60]), direct letter (18% increased uptake, RR = 1.18, 95% CI [1.12, 1.24]), smartphone message (14% increased uptake, RR = 1.14, 95% CI [1.02, 1.28]), and portal message (4% increased uptake, RR = 1.04, 95% CI [1.00, 1.09]), format contributed to significant improvements in vaccination rates, relative to control. However, for the small number of studies (k = 2) where a letter/postcard was supplemented with brochure, no significant improvement in vaccination uptake was observed (RR = 1.17, 95% CI [0.87, 1.58], Z = 1.02, p = .31). Significant differences were observed in the effectiveness of messaging based on the type of correspondence (letter, postcard, letter/postcard + brochure, portal message, smartphone message), χ2 (4) = 19.52, p < .001. Specifically, a portal message was significantly less effective than a letter, χ2 (1) = 13.54, p < .001, or postcard, χ2 (1) = 8.81, p = .003, at increasing vaccination uptake relative to control. Of note, large heterogeneity was still observed within each subgroup (I² = 75% − 93%) so the estimate of subgroup treatment effects warrants caution as individual study results considerably varied.

Fig. 4.

Effect Size Estimates by Correspondence Type.

Continent. Increased vaccination rates were observed across all continents following a written direct message, relative to control (see Fig. 5 ). Significant differences were observed in the effectiveness of direct written correspondence based on study location (continent: North America, Europe, Australia), χ2 (2) = 13.98, p < .001, with vaccination uptake significantly better in Australia (40% increased uptake, RR = 1.40, 95% CI [1.25, 1.57]) than in North America (13% increased uptake, RR = 1.13, 95% CI [1.10, 1.17]), χ2 (1) = 12.75, p < .001). Heterogeneity within the subgroup of studies conducted in Australia was deemed to be small (I² = 33%), whilst large heterogeneity was observed for the studies conducted in Europe and North America (I² = 86% − 91%).

Fig. 5.

Effect Size Estimates by Location.

Year of publication. Subgroup analysis by year of publication was carried out in two-decade intervals, i.e., 1980–1999 and 2000–2020. The effect of sending a direct written correspondence held over both periods but was higher in the earlier period. Studies published in 1980–1999 saw a 32% increase on control (RR = 1.32, 95% CI [1.22, 1.43]) while the increase was 12% in those published from 2000 to 2020 (RR = 1.12, 95% CI [1.08, 1.16]), χ2 (1) = 14.57, p < .001 (see Fig. 6 ). Large heterogeneity was observed within the subgroups (I² = 88% − 92%), which warrants caution in the estimate of the subgroup treatment effects due to individual study differences.

Fig. 6.

Effect Size Estimates by Year of Publication.

Age group. No significant differences were observed in the effectiveness of direct written correspondence based on age group, χ2 (1) = 1.85, p = .17 (see Fig. 7 ). Following a message, the increase in vaccine uptake was comparable between young and middle-aged adults (typically 18–64 years; 9% increase in vaccine uptake, relative to control; RR = 1.09, 95% CI [1.01, 1.18]) and older adults (typically ≥ 65 years; 16% increase in vaccine uptake, relative to control; RR = 1.16, 95% CI [1.12, 1.20]). Such findings are to be considered considering the large heterogeneity observed within the age subgroups (I² = 81–91%), demonstrating variance in individual study estimates of effect.

Fig. 7.

Effect Size Estimates by Age Group.

Institutional setting of correspondence sender. Direct written correspondence increased vaccine uptake for all three of the institutional settings for the organization from which the correspondence was provided. A 24% increase in vaccination was observed for correspondence from primary care providers, relative to control (RR = 1.24, 95% CI [1.17, 1.32]), whilst correspondence from hospital care providers and health insurance (public/private) providers contributed to a 22% (RR = 1.22, 95% CI [1.09, 1.37]) and 12% (RR = 1.12, 95% CI [1.07, 1.16]) increase, respectively. The effectiveness of direct written correspondence on vaccine uptake significantly differed dependent on the institutional setting of the organization from which the correspondence was provided, χ2 (2) = 9.58, p = .008 (see Fig. 8 ). Specifically, correspondence from primary care providers was more effective at promoting vaccine uptake, relative to control, when compared to correspondence from health insurance providers, χ2 (1) = 8.84, p = .003. No heterogeneity was observed within the hospital provider subgroup (I² = 0%; k = 2), whilst large heterogeneity was evident in the subgroups of primary care providers and health insurance providers (I² = 89% − 94%). Thus, tests of subgroup effects should be interpreted with caution due to varying individual study effects.

Fig. 8.

Effect Size Estimates by Institutional Setting of Correspondence Sender.

Risk of bias assessment. Significant differences were observed in the overall effectiveness of direct written correspondence on vaccine uptake when risk of bias assessment was considered, χ2 (2) = 16.37, p < .001 (see Fig. 9 ). In particular, the effectiveness of messaging differed significantly between low and unclear risk studies, χ2 (1) = 4.06, p = .04, as well as between low and high-risk studies, χ2 (1) = 15.19, p < .001; the effectiveness of messaging also significantly differed between unclear risk studies relative to high-risk studies, χ2 (1) = 8.47, p = .004. On average, messaging contributed to a 9% increase in vaccination (RR = 1.09, 95% CI [1.04, 1.15]) for low-risk studies, a 17% increase in vaccination for unclear risk studies (RR = 1.17, 95% CI [1.12, 1.22]) and a 42% increase in vaccination (RR = 1.42, 95% CI [1.26, 1.61]) across high-risk studies. Large heterogeneity was observed within each subgroup (I² = 86% − 92%), precluding the possibility of drawing valid conclusions for tests of subgroup differences.

Fig. 9.

Effect Size Estimates by Summary Assessment of Risk of Bias.

3.7. Publication bias

A visual representation of the publication bias via funnel plot (log risk ratio by standard error) was produced (available in the online supplemental file). Egger’s test was performed to quantity the funnel plot asymmetry and indicated that publication bias was present in the meta-analysis, with small sample studies with non-significant or smaller than average effect sizes likely to be missing (Egger’s intercept = 3.18, p < .001). Following trim and fill analyses to account for publication bias by imputing the effect sizes of 13, hypothetical, missing studies, the overall, mean, weighted effect size was adjusted to RR = 1.09 (95% CI [1.05, 1.12]). This corresponds to a 9% increase in vaccination following the messaging intervention, relative to control.

3.8. Content analysis of correspondence

Of the studies with intervention arms reporting an effect, 16 provided information on the content of correspondence: primarily a descriptive summary of the correspondence tested rather than the correspondence text in full. The most commonly reported content elements per study were: a recommendation to get the vaccine (k = 11); a statement of the seriousness of the influenza / possible complications from influenza (k = 7); a statement that the vaccine helps avoid serious complications / is effective (k = 6); information on how and where to get the vaccine / scheduling information (k = 6); a statement that the vaccine is free (k = 6); and advice to get the vaccine every year (k = 4).

Other reported content elements were: an address to common concerns about the vaccine (k = 4); a statement that the vaccine is safe / has minimal side effects (k = 3); clinic operating hours (k = 3); clinic locations (k = 3); a statement of who is at high risk of complications from the flu (k = 2); access to online scheduling (k = 2); information on the availability of the vaccine (k = 2); a statement that the recipient is at high risk of complications / a serious case of the flu (k = 2); statement on the importance of the vaccine for high-risk people (k = 2); statement that the vaccine can cause minor side effects (k = 1); advice to get the vaccine soon (k = 1); information about the clinical manifestations of influenza (k = 1).

3.9. Difference in effectiveness across intervention arms

Of the seven studies that report an intervention is effective and have multiple intervention arms, four found a difference in results between intervention arms. The most effective interventions in these studies highlight design elements that might influence vaccine uptake. In one study the effectiveness of the intervention increased with more personal modes of contact: ‘the reminder postcard from the patient’s primary care physician was more effective than the generic postcard and the personalized tailored letter was more effective than either postcard intervention’[46]. Another study tested three postcard types and found that all were more effective than no reminder,[6] a postcard designed according to the Health Belief Model was most effective (32.1% increase), followed by a personalized postcard (20.8% increase), while a ‘neutral’ reminder postcard showed a comparatively lower increase in vaccine uptake (4.8% increase). Another study found that in testing four different letter designs, it was found that only the action letter (giving the exact time and places of vaccination clinics) was markedly more effective than the others: ‘First, differential framing was no more effective than providing a simple reminder. Second, providing action instructions had a powerful incremental effect on vaccination rates.[41] An earlier study found that sending an educational brochure alone was more effective than either a financial incentive or sending both brochure and incentive: ‘the educational brochure more than doubled the likelihood of influenza immunization (odds ratio [OR] = 2.29, 95% confidence interval [CI] 1.45 to 3.61), whereas the incentive had less of an effect on immunization (OR = 1.68, 95% CI 1.05 to 2.68). Immunization for the group mailed both interventions was not significantly different from control.’[51]

Three of the seven studies did not find a difference between intervention arms. One found no difference in sending a personalized versus a generic letter[32]: ‘The likelihood of vaccination was similar for persons who received a personal letter and for those who received a form letter.’ A study testing four letter types ‘found that a single mailed letter significantly increased influenza vaccination rates compared with no letter. However, there was no difference in vaccination rates across the four different letters tailored with behavioural science techniques.’[34] Another study testing app messages – one offering reward points, the other not - found ‘there were no significant differences between conspicuous incentives and generic messaging[37].’

4. Discussion

This is the first study to provide a meta-analysis and detailed subgroup analysis of the effect of providing a single direct correspondence on the uptake of the influenza vaccine by adults living in the community. The current review offers evidence from previous influenza vaccination programmes to inform future programmes targeted towards influenza and also the uptake of COVID-19 vaccination. The seasonal and consistent burden of influenza meant that infrastructure, prevention, and treatment strategies could be more promptly implemented in response to the 2009 pandemic, and this has not been the case in response to SARS-CoV-2.[10] Nonetheless, we argue that there is important learning available.

Firstly, the certainty of evidence, as assessed in accordance with GRADE criteria, was moderate, indicating that a single written correspondence to individuals likely increases the uptake of the influenza vaccine. This review found direct written correspondence increased influenza vaccination uptake, direct written correspondence significantly improved vaccine uptake by approximately 16%, relative to control. This conclusion is consistent with the assessment of two previous qualitive systematic reviews that such interventions have been implemented successfully[16], [18] and it is consistent with the two previous systematic reviews that provided a pooled estimate of the effect size of mailed print material on the uptake of the influenza vaccine[19], [23]. Our study’s estimate of the pooled effect of a direct letter and estimate of the pooled effect of a postcard are consistent with, but differ in magnitude, to the previously available estimate as our review includes more studies[19]. Our study estimated a 18% increase in uptake due to a direct letter (RR = 1.18, 95% CI [1.12, 1.24], k = 14) compared to a previous estimate of 35% (RR = 1.35, 95% CI [1.19, 1.52], k = 11)[19], and a 35% increase in uptake due to a postcard (RR = 1.35, 95% CI [1.15, 1.60], k = 6) compared to a previous estimate of 15% (RR = 1.15, 95% CI [0. 95, 1.39], k = 3)[19].

This review also confirms that an increase in uptake can be achieved with just one correspondence, and this applies not just to letters or postcards but also with other types of correspondence such as smartphone messages or portal messages. This review also found that the effect of direct correspondence on uptake of the influenza vaccine holds across different age groups (18–64 and 65 + years), across continents (North America, Europe, or Australia), across time periods (1980–1999 and 2000–2020), and across institutional setting. While the small number of available studies and heterogeneity within subgroups precluded the possibility of drawing meaningful conclusions on moderator effects, the treatment estimates nonetheless suggest better vaccination rates following a direct, single correspondence intervention. The positive effect also holds after considering possible risk of study bias and potential publication bias.

Strengths of this review are that it exclusively included randomized trials with a control group, followed the PRISMA statement, undertook quality assessment, and it accounted for the possibility of study and publication bias when estimating intervention effects. There are several limitations to the evidence included within this review which have implications for the certainty with which certain conclusions can be made. The number of studies within some categories of the subgroup analysis is small, which limits the certainty of evidence in relation to differences in effect size - in particular, for correspondence type, continent, and age group. It is also the case that the effect size estimate is lower for studies with a low risk of bias than for studies with an unclear or high risk of bias. Regarding our secondary questions not all studies published content, preventing full analysis, and there are too few studies to produce an estimate of pooled effect. This review only included studies undertaken in OECD countries as these were felt to be most pertinent to the primary review question of whether supplementing mass communications with direct correspondence increases influenza vaccine uptake, as in these countries public health systems are well developed and most members of the community have access to mass media. The generalizability of results across the population of OECD countries may be questioned, as the studies were undertaken in eight countries and 18 of the 27 studies in the meta-analysis related to older adults (65+ years) or groups with specific medical conditions that might be considered at high risk from influenza.

These limitations and caveats aside, there is a second important implication for public health authorities organizing vaccination programs for influenza, and arguably also for COVID-19. Sending written vaccination correspondence directly to members of the community is likely to increase vaccine uptake more than using mass communications alone. When designing correspondence to support the uptake of the influenza vaccine, public health authorities should consider including the most reported content used in correspondence shown to increase influenza uptake. In particular, it is important to give a clear and strong recommendation to be vaccinated; provide information on vaccine effectiveness, the seriousness of influenza and how vaccination can avoid complications; state that the vaccine is safe; as well as providing information on cost and instructions on how and where to get vaccinated. These factors are also likely to be relevant for inclusion in correspondence to support the uptake of COVID-19 vaccines as they address many of the most frequently cited reasons by citizens in OECD countries for willingness and unwillingness to obtain COVID-19 vaccines, as identified in a recent review of peer-reviewed papers.[54] Based on the findings in the same review it would also be advisable for correspondence supporting the uptake of COVID-19 vaccines to briefly explain the speed at which COVID-19 vaccines were developed and tested, and for mass communications to support trust in health professionals, government agencies and in science.

Further research is needed on designing direct written communications to maximize vaccine uptake, whether in paper format or electronic media. In publishing results it is advised to quote the full text of tested correspondence to allow comparative analysis of effective design elements. To conclude, this meta-analysis provides evidence for single, direct messaging in increasing vaccination uptake for the influenza vaccine and can provide important insights for the rollout of vaccination programs for COVID-19.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A.

See Table A1 .

Table A1.

Summary of Interventions.

Studies	Interventions
Baker et al, 1998 USA n = 24,743	Control: No reminder Intervention: (1) A generic postcard that included a standard message, (2) a personalized postcard from the primary care physician, addressed to the patient at risk and containing the standard message, (3) a personalized letter from the primary care physician, addressed to the patient at risk, and contained a message tailored to the patient’s risk factors for influenza. The standard message of these materials included a description of who is at risk of contracting influenza, a statement that influenza can be serious, and assurance that the vaccine is safe and effective. The printed materials also advised individuals to get the influenza vaccine and listed the influenza clinic locations and operating hours. Category and basis: (1) GP = Generic Postcard, (2) PP = Personalised Postcard, (3) PL = Personalized Letter. Personalised letter described as such by authors, also personalised correspondence signed by physician and/ or tailored to the patient’s risk factors.
Brimberry et al, 1988 USA n = 787	Control: No reminder Intervention: (1) Patients received a reminder letter or, (2) a personal telephone reminder. The second intervention is not discussed further in this review. The letter emphasized that, because of “certain medical problems (for example, diabetes or heart disease),” influenza can be a serious threat to health, and that the patient’s physician had recommended that the patient be vaccinated. As a form letter was used, each patient’s personal diagnosis could not be mentioned, and the signature of a designated “influenza vaccination director” was used because of the difficulty of obtaining the signature of each patient’s personal physician. To make the vaccination convenient for the patient, no appointment was necessary, and the patient was informed of the cost. Category and basis: GL = Generic Letter.
Buchner et al, 1987 USA n = 655	Control: No reminder Intervention: Postcard reminder; short message on 3-inch by 5-inch card, mailed in business envelope with physician’s return address; message indicated flu season was coming, some people are at greater risk for influenza and complications, flu shots can decrease risks with minimal side effects, and it is needed each year; also provided instructions for where to obtain flu shots. By having the physician sign the cue and by using the physician’s business envelope, the cue emphasised that the physician recommended the flu shot. Category and basis: PP = Personalised Postcard (signed by physician)
CDC 1995 USA n = 190,000 Cluster-randomized trial	Control: No letter. Measures to increase influenza vaccination coverage including public service announcements and notices to health-care providers Intervention: (1) A personalized letter and informational brochure from the Montana-Wyoming Foundation for Medical Care (MWFMC) medical director encouraging vaccination, or (2) a form letter and informational brochure from the MWFMC encouraging vaccination. Category and basis: (1) PL + B, (2) GL + B = Personalised Letter and Brochure, and Generic Letter and Brochure (described as such by authors, also from named medical director)
Clayton et al, 1999 USA n = 5278	Control: Standard member educational materials sent by mail. Intervention: Postcard reminder mailed in addition to standard materials. Category and basis: P = Postcard (unclear if generic or personalised as no information given in paper)
Cutrona et al, 2018 USA n = 20,000	Control: Usual care Intervention: (1) A portal message promoting influenza vaccines, or (2) an Interactive Voice Reminder (IVR) call. The second intervention is not discussed further in this review. Portal messages appeared in letter format; the signature line contained the name of the patient’s PCP. Messages were delivered through standard channels. A generic message (without personal health information or reference to vaccines) was delivered to the patient’s email account, prompting login to the secure portal via a hyperlink. Once logged in, patients clicked on a message labelled “Brief Flu Questionnaire” to view. Message included access to direct online scheduling of influenza vaccination appointments. Information about CDC vaccine website(s) appeared within the body of the message as a hyperlink. Message also included opportunities to report community-administered influenza vaccinations, barrier questions, and targeted information dispelling misconceptions. Category and basis: PPM = Patient Portal Message
Hogg et al. 1998 Canada n = 1971 patients in 719 families	Customized preventive care reminder letters vs form letters vs usual care Control: Usual care Intervention: (1) Computer-generated individualized letters were sent to patients reminding them of outstanding preventive care procedures. The tone of each letter was positive and nonthreatening. Letters included dates on which participants had last received the recommended procedures,and were generated from the computerized medical record. (2) Patients received a form letter outlining all the recommended preventive procedures for all ages and sexes. The letter was identical to the individualized letter received by patients in Group 1, except the date of previous procedures was not provided. Category and basis: (1) is PL = Personalised Letter; (2) is GL = Generic Letter; see above for basis.
Klassing et al, 2017 USA n = 311	Control: No contact Intervention: (1) standardized letter, or (2) phone call. A phone call script was utilized for the phone call intervention; patient specific questions were fielded on an individual basis. This second intervention is not discussed further in this review. The letter intervention group received a standardized letter addressed to each specific patient. Both the phone call script and letter referenced the 2014 CDC immunization schedule and guidelines Category and basis: PL = Personalized Letter (addressed to each patient)
Larson et al, 1982 USA n = 395	Control: No reminder Intervention: (1) Patients sent a neutral postcard mentioned influenza vaccine now available; listed telephone number for nurse appointments; addressed to “Dear Patient”; or (2) health belief model postcard, emphasizing severity of influenza, susceptibility of at-risk persons to influenza, and benefits of vaccination; addressed to “Dear Patient”, or (3) personal postcard; addressed to patient’s name and signed by clinician; postcard mentioned that influenza season is approaching and recommended the patient come in for flu shot; it listed telephone number to call and make appointment with nurse (more details in Chapter 4 and Appendix). Category and basis: (1) GP = Generic Postcard (2) HBP = Health Belief Model Postcard (3) PP = Personalised Postcard
Lee et al, 2020 USA n = 50,286	Control: No reminder Intervention: (1) App message with reward: “Flu Shot = 200 Points. Get a flu shot at a pharmacy next time you refill a Rx. Tap Activities to submit proof” or (2) app message with no reward: “Flu Shot. Get a flu shot at a pharmacy next time you refill a Rx” Category and basis: MA = Mobile App, see above for basis.
McCaul et al, 2002 USA n = 23,733 Cluster-randomized trial	Control: No reminder Intervention: (1) Reminder letter from state peer review organisation (PRO), or (2) reminder letter with loss or gain frame from PRO, or (3) action letter from county public health office with date, time and place of vaccination clinics. The reminder letter highlighted four main points: (a) “You should have a flu shot every year,” (b) “Medicare will pay for your flu shot this fall,” (c) “The flu shot is safe,” and (d) “You should have your shot soon.” In addition, the framing letter stated, “As a person 65 or older, you are at risk for getting a serious case of flu.” The framing letter was accompanied by one of two inserts. The gain insert featured the picture and testimonial of a North Dakota woman who had received a flu shot the previous year and had not gotten the flu; the loss insert featured the picture and testimonial of another North Dakota woman who had not received a flu shot last year and had spent several days in bed, sick with the flu. More detail on the arms is provided in Chapter 4. Category and basis: PL = Personalised letter (reference to age; addressed to individual; signature of doctor)
McDowell et al, 1986 Canada n = 939	Control: No reminder Intervention: (1) A personal reminder by the physician, or (2) a telephone reminder by the nurse, or (3) a letter reminder. Only the latter intervention is discussed here. The letter was signed by the patient's physician and the practice nurse. The letter read: “As you know, each fall we recommend immunization against influenza for our patients who are 65 years of age or older. The vaccine is now available and if you would like to be immunized, please call to schedule an appointment.” Category and basis: PL = Personalised Letter (addressee selected by age, signed by physician and practice nurse)
Minor et al, 2010 USA n = 1371	Control: Standard clinical practice Intervention: (1) A letter addressed from the clinic and signed by the clinic pharmacist and physician medical director and a copy of the Centers for Disease Control and Prevention (CDC) Influenza Vaccine Information Statement, or (2) a telephone reminder. The latter is not discussed further in this review. Category and basis: PL + B = Personalised Letter and Brochure (letter signed by the clinic pharmacist and physician medical director)
Moran et al, 1992 USA n = 409	Control: Usual Care Intervention: (1) Reminder letter offering free vaccination with an appointment, or (2) two sequential reminder letters, offering the same. The sequential reminder intervention is not discussed further in this review. The reminder letters were written at fifth-grade reading level and emphasized that: 1) immunization was medically indicated, 2) immunization did not cause influenza, 3) immunization could result in minor side effects, and 4) immunization was free and available without an appointment. Category and basis: PL = Personalised Letter (advising high risk patient that immunisation is medically indicated)
Moran et al, 1996 USA n = 797	Control: No intervention Intervention: (1) A large print, illustrated educational brochure emphasizing factors important to patients in making a decision about influenza immunization, or (2) a lottery-type incentive announcing that all patients receiving influenza immunization would be eligible for grocery gift certificates, or (3) both educational brochure and incentive. Category and basis: (1) GEB; (2) Lottery; (3) GEB + Lottery. Generic Educational Brochure and financial incentive (gift certificate lottery)
Mullooly et al, 1987 USA n = 2217	Control: Did not receive the mailed cue. Intervention: Personalized letter stressing the importance of influenza vaccination for high-risk elderly individuals who had been hospitalized during the past year. It was explained that immunization could help to avoid serious complications from the bout of flu and that the CDC and their personal Kaiser Permanente doctors recommend that they get a flu shot each year. Information about how and where to obtain a vaccination was also provided. Category and basis: PL = Personalised Letter (described as personalised by author, letter also makes reference to people discharged from hospital in last year)
Nexøe et al, 1997 Denmark n = 585	Control: No letter Intervention: (1) Patients were invited for vaccination and had to pay the GP’s usual fee, or (2) patients were invited for free vaccination. The second intervention is not discussed further in this review. Category and basis: PL = Personalised Letter. Letter included patient’s name and GP’s signature in print.
Puech et al, 1998 Australia n = 325	Control: Usual care, considered to be an ad hoc approach, influenced by news coverage of potential epidemics, media campaigns by vaccine manufacturers, opportunistic reminders and other secular events. Intervention: A postcard encouraging patients to attend the practice for an influenza vaccination before the end of the month. The postcard stressed the seriousness of influenza as opposed to the effectiveness and safety of influenza vaccine; it also gave availability and cost information. For ease of reading, the postcard was large (A5 format) and had clear, black-on-white large print. The postcard had a Flesch readability score of 68,14 requiring a minimum IQ of 90 to understand it (75% of the general population would understand it). Postcards had the practice logo and were mailed in a handwritten, personally addressed envelope also printed with the practice logo. Category and basis: PP = Personalised Postcard (personally addressed envelope)
Regan et al, 2017 Australia n = 12,354	Control: No reminder Intervention: SMS for adults or guardians of children: addressed patient by name, stated patient may be eligible for government-funded vaccine and gave a number to call to schedule an appointment. Category and basis: SMS = Short Message Service, see above for basis.
Roca et al, 2012 Spain n = 2402	Control: No intervention Intervention: A personalized letter including basic information about the clinical manifestations and possible complications of influenza, and about the efficacy of the vaccine to prevent the disease, according to recommendations of the Centers for Disease Control and Prevention and the local authorities of the Comunidad Valenciana. The letter addressed common concerns about the flu shot and was written in easy-to-understand language. Category and basis: PL = Personalized Letter (described as personalized by author, paper also makes reference to where patients’ postal addresses were obtained from)
Satterthwaite et al, 1997 New Zealand n = 2791	Control: No reminder Intervention: (1) Personalised letter recommending vaccination, or (2) personalised letter recommending visit to receive vaccine at no charge. Both letters were signed by principal. The second intervention is not discussed further in this review. Category and basis: PL = Personalised Letter.
Schulte et al, 2019 Germany n = 460	Control: No reminder Intervention: (1) Patients received a letter from a physician, or (2) nephrologists were called upon by the Association of Statutory Health Insurance Physicians (ASHIP) to recommend vaccination, or (3) patients received a letter from their health insurance fund. Only (1) meets this reviews inclusion criteria. Category and basis: L = Letter (no further information given)
Spaulding et al, 1991 USA n = 1068	Control: No postcard and received routine care. Intervention: A reminder postcard advising patients that their physician had determined that they were at high risk of complications should they catch the “flu,” and strongly urging them to come to the Family Practice Clinic for immunization. Category and basis: PP = Personalised Postcard (the letter stated that their physician had determined that they were at high risk of complications should they catch the flu)
Szilagyi et al, 2020 USA n = 164,205	Control: No reminder Intervention: One, two or three patient portal reminder letters on the importance and safety of influenza vaccination. Only the result of one reminder letter is analysed here. The letter included (a) information that influenza season was coming, the disease can cause substantial morbidity and the vaccine is the best way to protect against influenza, (b) recommendation to receive an influenza vaccine by calling for an office appointment or going to a pharmacy or other setting, (c) a website link to input influenza vaccinations received elsewhere into the UCLA Health System record, and (d) another website link to a UCLA webpage containing information about influenza vaccine and video testimonials about influenza vaccination. Letters were in English, included the name of the patient’s primary care physician, and had a below seventh grade reading level per Flesch-Kincaid analysis. Category and basis: PPM = Patient Portal Message
Terrell-Perica et al, 2001 USA n = 6528	Control: No letter. During the study period, the State of Hawaii Department of Health conducted routine promotional activities for influenza immunization, including press releases, immunization clinics held at pharmacies and retail stores, and health education at a large annual senior fair. In addition, pneumococcal education kits produced by the National Institute on Aging were mailed to physicians. Intervention: (1) A letter encouraging recipients to take advantage of their new Medicare benefits to receive influenza immunization, or (2) a letter encouraging them to take advantage of their new Medicare benefits to receive influenza and pneumococcal immunizations – this intervention is not discussed further in this review. The one-page influenza immunization reminder letter was formatted in an easy-to-read, 14-point font with two prominent bullets: “Have you had your FLU shot this year?” and “Medicare covers FLU shots!” Category and basis: PL = Personalized Letter (did not apply to all households, new Medicare members)
Wijesundara et al, 2020 USA n = 97,607	Control: No reminder Intervention: Patient portal message that listed upcoming clinics and provided an option for online scheduling. The message was sent in a letter format and signed by named physician. Category and basis: PPM = Patient Portal Message, see above for basis.
Yokum et al, 2018 USA n = 228,000	Control: No letter Intervention: (1) A letter with vaccination information + picture of National Vaccine Program Officer, or (2) a letter with vaccination information + picture of Acting US Surgeon General, or (3) a letter with implementation intention prompt + picture of Acting US Surgeon General, or (4) a letter with enhanced active choice implementation prompt + picture of Acting US Surgeon General (more details in Chapter 4). Category and basis: PL = Personalised Letter (addressed to recipient’s first name)
Song et al, 2000 South Korea n = 2017	Control: No reminder Intervention: (1) A telephone reminder, or (2) a postcard reminder. The first intervention is not discussed further in this review. Category and basis: P = Postcard (no further information as paper was not translated from Korean)

Appendix B. Supplementary material

The following are the Supplementary data to this article: