Abstract
Objective
To determine effectiveness of text/telephone outreach messages, with and without coronavirus disease 2019 (COVID-19) vaccine information.
Study design
We conducted an intent-to-treat, multiarm, randomized clinical trial with adolescents aged 12-17 years. Eligible patients did not have an adolescent well-care visit in the past year or scheduled in the next 45 days or an active electronic health record portal account. We randomized participants to the standard message, COVID-19 vaccine message, or no message (control) group and delivered 2 text messages or telephone calls (per family preference) to the message groups. The primary outcome was adolescent well-care visit completion within 8 weeks, and secondary outcomes were adolescent well-care visit scheduled within 2 weeks and receiving COVID-19 vaccine within 8 weeks.
Results
We randomized 1235 adolescents (mean age, 14 ± 1.5 years; 51.6% male; 76.7% Black; 4.1% Hispanic/Latinx; 88.3% publicly insured). The standard message group had higher odds of scheduling an adolescent well-care visit compared with the control group (OR, 2.07; 95% CI, 1.21-3.52) and COVID-19 vaccine message group (OR, 1.66; 95% CI, 1.00-2.74). The odds of completing an adolescent well-care visit did not differ significantly (standard message group vs control group; OR, 1.35; 95% CI, 0.88-2.06; COVID-19 vaccine message group vs control group, OR, 1.33; 95% CI, 0.87-2.03). In per-protocol analyses, adolescents in the standard message group were twice as likely as the control group to receive the COVID-19 vaccine (OR, 2.48; 95% CI, 1.05-5.86).
Conclusions
Outreach messages were minimally effective. Efforts are needed to address widening disparities.
Clinical Trial Registration
Keywords: preventative services, pediatrics, disparities
Numerous preventative services were delayed during the coronavirus disease 2019 (COVID-19) pandemic, including adolescent well-care visits.1, 2, 3 Adolescent well-care visit adherence was challenging before the pandemic, with only 30%-50% of adolescents completing the recommended yearly evaluation.4, 5, 6 After the pandemic shutdown, thousands of adolescent well-care appointments were cancelled or delayed and vaccination administration rates decreased.7, 8, 9 Black and Latinx vaccination rates were found to be disproportionately worsening, creating an imperative to address structural racism with intentional and deliberate intervention.9, 10, 11, 12 Additionally, increasing rates of obesity and depression have heightened the importance of adolescent well-care preventative services.13 , 14
On May 10, 2021, the Pfizer-BioNTech COVID-19 vaccine age eligibility expanded to adolescents 12-15 years of age, amending the emergency use authorization originally issued on December 11, 2020, for those 16 years or older.15 With COVID-19 vaccine hesitancy more common in Black and Latinx and uninsured populations, possibly linked to pervasive mistreatment, medical mistrust, and poor access, systematic outreach approaches hold promise to encourage adolescents to return for care and complete vaccinations, including text and email messages, phone calls, mailings, school campaigns, community navigators, and home visitors.11 , 16, 17, 18, 19, 20 Outreach has proven benefit for completion of adolescent well-care visits and vaccinations, but it was uncertain whether this would be effective amid the pandemic and among Black and Latinx communities who experienced the collective suffering of the pandemic with additional layers of trauma.12 , 21, 22, 23, 24, 25, 26, 27
We sought to assess the effect of automated text and telephone outreach messages to parents, with and without information about COVID-19 vaccine availability, on the scheduling and completion of adolescent well-care visits among adolescents due for preventative services. We hypothesized that outreach messages would be superior to no message, and messages with mention of COVID-19 vaccine availability would outperform messages without it, given that a subset of our population might be early adopters and open to COVID-19 vaccination.16
Methods
We conducted a multiarm, randomized clinical trial (NCT04904744) from May 28 to August 5, 2021, to remind patients due for an adolescent well-care visit. Study personnel generated a random allocation sequence stratified by clinic location using block randomization (with a block size of 3) and randomly assigned eligible patients (1:1:1) to 1 of 3 arms: standard message, standard plus COVID-19 vaccine message (hereafter referred to as COVID-19 vaccine message), or control group (no message). Our institutional review board approved the study, granting a waiver of informed consent.
Setting and Population
The study took place at 3 academic pediatric primary care practices. These practices serve a predominantly non-Hispanic Black, low-income population. Annually, these practices provide more than 54 000 visits to 36 000 patients. Parents or legal guardians (hereafter referred to as parents) schedule appointments using a central scheduling center that is available Monday to Friday, 8:30 am to 5:00 pm.
We included patients aged 12-17 years old who met the following eligibility criteria: (1) seen at 1 of our 3 primary care practices within 2 years, (2) no adolescent well-care visit in the last 365 days; (3) no adolescent well-care visit scheduled; (4) preferred language of English and/or Spanish; and (5) parent telephone number recorded in the hospital registration system (Figure ). We also excluded patients with an active electronic health record (EHR) portal account (n = 271 [17.8%] of 1522 potentially eligible patients) to conduct a separate trial focused on delivering portal messages to that population. At the time of this study, 32% of the entire patient population served by our practices had active portal accounts. We collected the following variables from our EHR to characterize our sample and for consideration as covariates in our analyses: parent communication preference; absence of past patient receipt of measles, mumps, rubella vaccine and diphtheria, tetanus and acellular pertussis vaccine as a proxy for declining childhood immunizations; patient lifetime historical institutional no-show rate; and practice appointment availability. We also tracked the community incidence of COVID-19.28
Figure.
CONSORT 2010 flow diagram.
Interventions
We sent 2 automated outreach messages 24 hours apart (ie, Thursday at 12:00 pm and then Friday at 12:00 pm) because it was feasible to implement, and a recent adult intervention was successful with a short interval between 2 text messages.29 Messages were sent either by text or phone calls to parents using Televox, a Health Insurance Portability and Accountability Act-compliant platform, based on the parent's preferred mode of reminder and preferred language. Parent preferences are elicited at first-time registration and recorded in the EHR. Interpreter services at Cincinnati Children's Hospital Medical Center translated automated messages into Spanish.
We crafted the messages delivered based on the extant literature and feedback from parents, as well as outreach medical assistants who call and text with our families daily. Please see the Appendix (available at www.jpeds.com) for examples of automated messages. The standard messages referenced the patient's first name, reminding parents that their child was due for an adolescent well-care visit, and asked them to call to schedule using the number provided. The COVID-19 vaccine messages included the information in the standard message plus mention that the COVID-19 vaccine was available for the child and parent during their future visit. Study personnel downloaded a report from the third-party vendor to ascertain receipt of text and phone call messages by parents.
Outcomes
Our primary outcome was adolescent well-care visit completion within 8 weeks of the date we sent the first message or the date of randomization for the control group. All practice locations maintained ample adolescent well-care appointment availability within 2 weeks throughout the study. Secondary outcomes included appointment scheduled within 2 weeks, and receipt of first COVID-19 vaccination within 8 weeks. Post hoc secondary outcomes included eligible patient's receipt of tetanus-diphtheria-acellular pertussis (Tdap) vaccine, human papillomavirus (HPV) vaccine, and meningococcal conjugate vaccine (MCV4). We assessed all outcomes using data from our EHR. Adolescent well-care completion was determined by a well-visit billing code. Vaccinations administered at any Cincinnati Children's facility were recorded in our EHR. In addition, our team members can document vaccines received elsewhere manually. Our EHR imports vaccine administration data from external sources, including the Ohio immunization registry and Care Everywhere (Epic's information exchange application). We were unable to accurately determine if a vaccine was offered to but declined by the family.
Sample Size and Blinding
We calculated the sample size for the study based on the hypothesis that 20% of patients in the standard message group, 30% of patients in the COVID-19 vaccine message group, and 2% of those in the control group would complete an adolescent well-care visit within 8 weeks. We based our hypotheses on effect sizes documented for automated text reminders on immunization rates.18 To detect these differences in adolescent well-care visit completion with 80% power at 5% significance level, we required a sample size of 293 per group (879 subjects overall).
We blinded patients and their parents by concealing the clinical trial. We blinded clinical practice teams and our outcome assessor by concealing group allocation.
Statistical Analyses
We conducted descriptive analyses to characterize the participants in terms of demographics and other potential covariates. We used χ2 tests or AMOVA, as appropriate, to assess for differences across the 3 groups. We conducted an intention-to-treat analysis to compare outcomes among the 3 randomized groups. We used logistic regression models to examine each of the following binary outcomes: scheduled appointment within 2 weeks, completed appointment within 8 weeks, and receipt of vaccine within 8 weeks. We did not adjust for any covariates in these logistic regression models because the 3 arms were balanced on patient characteristics and potential covariates. Because our study design did not account for siblings or those living in the same household, we conducted a sensitivity analysis excluding patients who shared the same phone number. Because some patients did not receive messages as intended, we also conducted a per-protocol analysis excluding these patients.
Results
Of the 1522 adolescents assessed for eligibility, 1235 were eligible and randomized into either the standard message group (n = 412), COVID-19 vaccine message group (n = 411), or the control group (n = 412) (Figure). The 3 groups were very similar, with no statistically significant difference on any characteristics (Table I ). Most adolescents were Black (76.7%), non-Hispanic (95.7%), and had public insurance (88.3%). Overall, 71.7% preferred to receive a text message and 28.3% preferred to receive a phone call. Few patients (0.73%) seemed to have declined childhood vaccine based on our proxy measure of not having received at least 1 past dose of measles-mumps-rubella and diphtheria-tetanus-acellular pertussis. Community incidence rates of COVID-19 based on 7-day moving average remained low during the study; increases in our local area owing to the Delta variant of COVID-19 occurred in August 2021.28
Table I.
Baseline characteristics
| Characteristics | Standard message | COVID-19 vaccine message | Control group | Total |
|---|---|---|---|---|
| Unique patients, No. | 412 | 411 | 412 | 1235 |
| Age, years | ||||
| 12 | 74 (18.0) | 72 (17.5) | 78 (18.9) | 224 (18.1) |
| 13 | 103 (25.0) | 110 (26.8) | 99 (24.0) | 312 (25.3) |
| 14 | 105 (25.5) | 106 (25.8) | 98 (23.8) | 309 (25.0) |
| 15 | 57 (13.8) | 54 (13.1) | 61 (14.8) | 172 (13.9) |
| 16 | 39 (9.5) | 29 (7.1) | 33 (8.0) | 101 (8.2) |
| 17 | 34 (8.3) | 40 (9.7) | 43 (10.4) | 117 (9.5) |
| Sex | ||||
| Female | 192 (46.6) | 210 (51.1) | 196 (47.6) | 598 (48.4) |
| Male | 220 (53.4) | 201 (48.9) | 216 (52.4) | 637 (51.6) |
| Race | ||||
| Black or African American | 313 (76.0) | 319 (77.6) | 315 (76.5) | 947 (76.7) |
| White | 69 (16.8) | 60 (14.6) | 65 (15.8) | 194 (15.7) |
| Multiracial | 10 (2.4) | 17 (4.1) | 13 (3.2) | 40 (3.2) |
| Asian | 3 (0.7) | 2 (0.5) | 6 (1.5) | 11 (0.9) |
| American Indian and Alaska Native | 1 (0.2) | 0 (0) | 0 (0) | 1 (0.1) |
| Missing data | 16 (3.9) | 13 (3.2) | 13 (3.2) | 42 (3.4) |
| Ethnicity | ||||
| Non-Hispanic/Latinx | 390 (94.7) | 396 (96.4) | 396 (96.1) | 1182 (95.7) |
| Hispanic/Latinx | 21 (5.1) | 15 (3.6) | 15 (3.6) | 51 (4.1) |
| Not reported | 1 (0.2) | 0 (0) | 1 (0.2) | 2 (0.2) |
| Insurance | ||||
| Public (ie, Medicaid) | 375 (91.0) | 357 (86.9) | 358 (86.9) | 1090 (88.3) |
| Private | 30 (7.3) | 47 (11.4) | 44 (10.7) | 121 (9.8) |
| Self-pay | 7 (1.7) | 7 (1.7) | 10 (2.4) | 24 (1.9) |
| Communication preference | ||||
| Text message | 279 (67.7) | 306 (74.5) | 301 (73.1) | 886 (71.7) |
| Phone call | 133 (32.3) | 105 (25.5) | 111 (26.9) | 349 (28.3) |
| Decline childhood vaccine | ||||
| No past MMR and DTaP | 3 (0.7) | 4 (1.0) | 2 (0.5) | 9 (0.7) |
| Patient lifetime historical no-show rate | 0.15 ± 0.3 | 0.19 ± 0.4 | 0.15 ± 0.3 | 0.16 ± 0.3 |
DTaP, diphtheria-tetanus-acellular pertussis; MMR, measles-mumps-rubella.
Values are number (%) or mean ± SD.
Intention-to-treat Analyses
There were 43 (10.4%), 27 (6.6%), and 22 (5.3%) patients who scheduled an adolescent well-care visit within 2 weeks in the standard message, COVID-19 vaccine message, and control groups, respectively. The standard message group had significantly higher odds of scheduling an adolescent well-care visit compared with both the COVID-19 vaccine message group (OR, 1.66; 95% CI, 1.00-2.74) and the control group (OR, 2.07; 95% CI, 1.21-3.52) (Table II; available at www.jpeds.com). Across the standard message group, COVID-19 vaccine message group, and control group, there were relative differences in the rates of completed adolescent well-care visits within 8 weeks (13.6% vs 13.4% vs 10.4%), and adolescent receipt of COVID-19 vaccination within 8 weeks (3.9% vs 2.4% vs 1.9%), but there were no statistically significant differences in the odds of these outcomes. Among the 34 adolescents who received a COVID-19 vaccine within 8 weeks, 30 (88.2%) received the vaccine at our institution and 4 (11.8%) received the vaccine elsewhere in the community. Among those who were eligible to receive the Tdap, HPV, and/or MCV4 vaccine during the study period, the odds of receiving Tdap and MCV4 within 8 weeks differed significantly between groups, with the message groups outperforming the control group (Table II). Rates of eligible patients receiving these vaccines within 8 weeks across the standard message group, COVID-19 vaccine message group, and control group were as follows: Tdap 13% vs 17% vs 2%; HPV 12% vs 10% vs 7%; and MCV4 12% vs 10% vs 2%. By the end of the trial, the study population had high rates of vaccination with Tdap (89.5%) and first doses of MCV4 (89.5%) and HPV (82.8%), but a lower percentage of eligible participants had completed the HPV series (54.6%) or received the MCV4 booster (35.3%).
We identified 237 patients (19% of 1235 sample; 77 in the standard message, 83 in the COVID-19 message, and 77 in control groups) who shared the same parent phone number with 1 or more other patients included in the study. Excluding these patients did not change the overall pattern of results, but differences between the standard message group and the control group on adolescent well-care visit completion approached significance (P = .06). The proportion of those who received the Tdap reached statistical significance between the standard message group, COVID-19 vaccine message group, and the control group (13% vs 15% vs 0%; Fisher exact test, P = .02).
Process Measures
Delivery of text messages had similar success rates across the standard message (84%) and COVID-19 vaccine message (83%) groups (Figure). Common reasons for text messages failing to be delivered included 12% did not have SMS capabilities, 6% failed at the carrier, and 1% previously blocked the number from which the messages were generated. The delivery of messages by phone was impacted by vendor technical difficulties on 1 day when no calls were sent, leading to the COVID-19 vaccine message group receiving only 1 rather than 2 phone calls. On the days phone calls were delivered, the success rates were similar to texting. Common reasons for phone call failures included the following: 1% had a busy tone or did not answer the phone and 0.4% did not have a working phone number. Overall, 342 (83%) of the standard message group and 242 (59%) of the COVID-19 vaccine message group received the intended intervention on both days. Therefore, we conducted a per-protocol analysis including only patients who received the intended intervention (a total of 996 patients), with the standard message group including recipients of text (n = 230) and phone (n = 112) messages, the COVID-19 vaccine message group only including recipients of text messages (n = 242), and the control group including those who received no messages (n = 412).
Per-protocol Analyses
Per-protocol outcome analyses are presented in Table III (available at www.jpeds.com). Overall, the pattern of results was similar to the intention-to-treat analyses. Across the standard message group, COVID-19 vaccine message group, and control group, there were relative differences in the rate of scheduling an appointment within 2 weeks (11.4% vs 6.6% vs 5.3%). The standard message group had a higher odds of scheduling an adolescent well-care visit within 2 weeks of receiving the first intervention message compared with the control group (OR, 2.28; 95% CI, 1.32-3.93), but differences compared with the COVID-19 vaccine message group were no longer significant (OR, 1.82; 95% CI, 0.99-3.34). Although there were relative differences in adolescent well-care completion within 8 weeks across the standard message, COVID-19 vaccine message, and control groups (14.0% vs 12.8% vs 10.4%), there were no significant differences between the groups on odds of completing the adolescent well-care visit within 8 weeks. Completion of COVID-19 vaccine within 8 weeks differed across the standard message (4.7%), COVID-19 vaccine message (3.3%), and control (1.9%) groups, with significant differences in the odds between the standard message group and the control group (OR, 2.48; 95% CI, 1.05-5.86). The standard message group still had higher odds of receiving an MCV4 compared with the control group, but differences between the COVID-19 vaccine message group and control group were no longer significant.
Discussion
In this clinical trial of automated text and telephone outreach messages among adolescents due for preventative services, the standard message positively impacted adolescent well-care visit scheduling but not completion. Both message groups outperformed the control group on MCV4 vaccine receipt within 8 weeks among those eligible, suggesting that messaging may ultimately impact routine adolescent vaccination. Per-protocol analyses suggest the standard message positively impacted adolescent receipt of the COVID-19 vaccine compared with the control group.
There was a significant increase in the scheduling of appointments within 2 weeks of receiving the standard message compared with the COVID-19 vaccine message and no message, indicating that gentle outreach nudges may remind families of delayed care and prompt some action. The COVID-19 vaccine message group had rates of visit scheduling similar to the control group. Despite known vaccine hesitancy in many populations similar to ours, we anticipated that notification of the COVID-19 vaccine availability directly in their primary care medical home, which is familiar and often trusted, would provide an incentive to families seeking access to the vaccine.30, 31, 32, 33 It is possible that messaging about the COVID-19 vaccine deterred some vaccine-hesitant families concerned they would be pressured to accept the vaccine if they scheduled and completed an appointment. It is also possible that this difference was due to lower intervention fidelity in the COVID message group, because this finding was no longer significant in per-protocol analyses.
The rate of adolescent well-care visit completion in our study was lower than previous studies, with estimates ranging from 14% to 72%.21, 22, 23 , 25 , 34 Our rate of 10.4% in the standard message group is similar to that achieved by O’Leary et al's bidirectional text messaging intervention.25 Interventions by Szilagyi et al and Suh et al yielded much higher rates. There are plausible explanations for this discrepancy.21, 22, 23 , 34 First, the duration of follow-up in these other studies was markedly longer, ranging from 6 to 18 months. We chose 8 weeks as a more proximal outcome measure given easy appointment access in our system and wanting to yoke the appointment to our specific intervention. Second, more than one-half of the sample in previous studies had an adolescent well-care visit in the 12 months before intervention vs none in our study.21 , 23 , 34 Our families may have been underusers of healthcare and less likely to return for adolescent well-care. Third, prior studies deployed higher intensity interventions. In contrast with other studies that provided many more messages over a longer time period, we provided 2 messages 1 day apart to examine the use of a simple, easy-to-implement strategy. Comparative studies included their whole patient population, whereas we excluded patients who had an active EHR portal account. This difference may have impacted results, because the included population who lacked portal access might be even harder to reach.35 Simple outreach messages may not be sufficient to re-engage patients who have fallen away from the medical system, especially during the pandemic. Vaccine hesitancy, which seems to be increasing nationally, may also have deterred some families.36 It is unclear if the challenge to get families to return to well-care relates to worsening trust or skepticism of the medical system, or the profound impact of the pandemic on Black families, including higher rates of unemployment and death, or other factors.12 , 37 , 38 Regardless, more is needed to help adolescents catch-up, specifically those most at risk and traditionally under-represented and not engaged by the medical community. Some have advocated for converting sick visits to adolescent well-care visits, when possible.10 Tiered approaches to outreach, including home visitation, may also be warranted.21
The 2-fold increase in odds of COVID-19 vaccine receipt observed in per-protocol analyses among adolescents in the standard message group compared with the control group was encouraging and consistent with strong evidence of effect for a variety of reminder systems increasing pediatric vaccine initiation.18 That said, the overall vaccination rates were extremely low: 3.9% vs 1.8%. More than 80% of our sample had become eligible for COVID-19 vaccine when the emergency use authorization was expanded 3 weeks before the receipt of intervention messages in this study. It is possible that many were not ready to accept the COVID-19 vaccine and that multiple messages and conversations will be needed to answer questions and address concerns.
Additional qualitative research is needed to develop effective strategies to promote the uptake of the COVID-19 vaccine in culturally relevant ways.11 The framing of future outreach messages may be informed by recent intervention studies designed to increase influenza vaccination. Reminders were not effective at increasing the initiation in a large pediatric study, regardless of whether the message was framed as gains or losses, but reminders did promote completion among those who needed a second shot.39 In contrast, reminder messages to adults who already had a visit scheduled were effective at increasing flu vaccine rates, with the most effective messages framed as “a vaccine has been reserved for you.”29 In addition to outreach, effective communication approaches are needed to engage vaccine-hesitant adolescents and their parents during visits.32 It is unclear whether past approaches to address vaccine hesitancy when discussing routine childhood vaccines will remain effective in the context of COVID-19 vaccine.40
Study strengths include the use of a clinically relevant intervention, a large, marginalized population, ascertainment of important outcomes (visit completion and vaccination rates), and an intention-to-treat design. Our study has several limitations. It was conducted in academic primary care practices serving low income, predominantly Black patients, which limits the generalizability of these results. Owing to challenges related to attribution, it is likely we included some adolescents who were not current patients of the practices. Our groups were contaminated with adolescents in the same household (ie, the same parental phone number) being allocated to different groups. This factor biased our results toward the null hypothesis, although sensitivity analyses suggest the impact was minimal. There are currently barriers to identifying household members within EHRs, providing researchers and EHR vendors an opportunity to create innovative population management tools. Like other studies, the delivery of automated messages was limited by incorrect phone numbers, lack of SMS capabilities, and so on.41 In addition, we experienced technical difficulties for phone call delivery, which further decreased the fidelity of our automated message intervention. Finally, the racial composition of our study team does not reflect that of our patient population. We engaged parents and practice outreach medical assistants to craft the messages, but would have benefited from collaboration with family members throughout the research process.
These data reinforce the challenges of recovering from the COVID-19 pandemic; delays in adolescent well-care threaten future health outcomes. However, we found messages had some beneficial spillover effects for vaccine rates. Additional study is needed to understand whether the timing of follow-up messages could realize additional successes in the completion of adolescent preventative services and address widening disparities.
Footnotes
Funding support from the Center for Clinical & Translational Science & Training (CCTST) at the University of Cincinnati which is funded by the National Institutes of Health (NIH) Clinical and Translational Science Award (CTSA) program, grant 2UL1TR001425. W.B. has common stock holdings in the following publicly traded companies: Pfizer, Merck, Abbott Laboratories, Viatris, and Johnson & Johnson. The other authors declare no conflicts of interest.
Portions of this study were presented as an abstract during the Pediatric Academic Society meeting, April 2022, Denver, Colorado.
Appendix
Table II.
Intention-to-treat analysis
| Outcomes | Comparison | OR (95% CI) | P value |
|---|---|---|---|
| Adolescent well-care visit scheduled within 2 weeks | Standard message vs control group | 2.07 (1.21-3.52) | .008 |
| COVID-19 vaccine message vs control group | 1.25 (0.70-2.23) | .457 | |
| Standard message vs COVID-19 vaccine message | 1.66 (1.00-2.74) | .049 | |
| Adolescent well-care visit completed within 8 weeks | Standard message vs control group | 1.35 (0.88-2.06) | .165 |
| COVID-19 vaccine message vs control group | 1.33 (0.87-2.03) | .193 | |
| Standard message vs COVID-19 vaccine message | 1.02 (0.68-1.52) | .930 | |
| Receipt of COVID vaccination within 8 weeks | Standard message vs control group | 2.04 (0.86-4.82) | .104 |
| COVID-19 vaccine message vs control group | 1.26 (0.49-3.22) | .631 | |
| Standard message vs COVID-19 vaccine message | 1.62 (0.73-3.61) | .238 | |
| Receipt of Tdap within 8 weeks∗ | Standard message vs control group | 6.50 (0.74-311.21) | .117 |
| COVID-19 vaccine message vs control group | 8.66 (1.03-408.04) | .045 | |
| Standard message vs COVID-19 vaccine message | 0.75 (0.19-2.89) | .854 | |
| Receipt of HPV within 8 weeks† | Standard message vs control group | 1.86 (0.94-3.70) | .075 |
| COVID-19 vaccine message vs control group | 1.47 (0.72-3.03) | .292 | |
| Standard message vs COVID-19 vaccine message | 1.27 (0.67-2.38) | .465 | |
| Receipt of MCV4 within 8 weeks‡ | Standard message vs control group | 5.44 (1.52-19.48) | .009 |
| COVID-19 vaccine message vs control group | 4.59 (1.25-16.93) | .022 | |
| Standard message vs COVID-19 vaccine message | 1.19 (0.51-2.74) | .691 |
Analysis includes all patients eligible to receive the Tdap during the 8-week study period (total, n = 130; standard message group, n = 45; COVID-19 vaccine message group, n = 41; control group, n = 44). ORs estimated from exact logistic regression analysis, and wide 95% CI owing to small cell counts.
Analysis includes all patients eligible to receive the HPV series during the 8-week study period (total, n = 608; standard message group, n = 208; COVID-19 vaccine message group, n = 195; control group, n = 205).
Analysis includes all patients eligible to receive the MCV4 vaccine or booster during the 8-week study period (total, n = 344; standard message group, n = 116; COVID-19 vaccine message group, n = 106; control group, n = 122).
Table III.
Per-protocol analysis
| Outcomes | Comparison | OR (95% CI) | P value |
|---|---|---|---|
| Adolescent well-care visit scheduled within 2 weeks | Standard message vs control group | 2.28 (1.32-3.93) | .003 |
| COVID-19 vaccine message vs control group | 1.26 (0.65-2.44) | .503 | |
| Standard message vs COVID-19 vaccine message | 1.82 (0.99-3.34) | .054 | |
| Adolescent well-care visit completed within 8 weeks | Standard message vs control group | 1.4 (0.9-2.17) | .132 |
| COVID-19 vaccine message vs control group | 1.26 (0.77-2.06) | .356 | |
| Standard message vs COVID-19 vaccine message | 1.11 (0.68-1.8) | .670 | |
| Receipt of COVID vaccination within 8 weeks | Standard message vs control group | 2.48 (1.05-5.86) | .039 |
| COVID-19 vaccine message vs control group | 1.73 (0.64-4.66) | .281 | |
| Standard message vs COVID-19 vaccine message | 1.44 (0.6-3.41) | .413 | |
| Receipt of Tdap within 8 weeks∗ | Standard message vs control group | 7.76 (0.81-385.15) | .089 |
| COVID-19 vaccine message vs control group | 6.58 (0.49-364.47) | .208 | |
| Standard message vs COVID-19 vaccine message | 1.17 (0.20-8.45) | .999 | |
| Receipt of HPV within 8 weeks† | Standard message vs control group | 1.81 (0.88-3.70) | .105 |
| COVID-19 vaccine message vs control group | 1.56 (0.68-3.57) | .290 | |
| Standard message vs COVID-19 vaccine message | 1.16 (0.53-2.52) | .715 | |
| Receipt of MCV4 within 8 weeks‡ | Standard message vs control group | 5.39 (1.46-19.91) | .012 |
| COVID-19 vaccine message vs control group | 3.31 (0.76-14.30) | .110 | |
| Standard message vs COVID-19 vaccine message | 1.63 (0.54-4.94) | .388 |
Analysis includes all patients eligible to receive the Tdap during the 8-week study period (total, n = 98; standard message group, n = 32; COVID-19 vaccine message group, n = 22; control group, n = 44). ORs estimated from exact logistic regression analysis, and wide 95% CI owing to data with small cell counts.
Analysis includes all patients eligible to receive the HPV series during the 8-week study period (total, n = 483; standard message group, n = 171; COVID-19 vaccine message group, n = 107; control group, n = 205).
Analysis includes all patients eligible to receive the MCV4 vaccine or booster during the 8-week study period (total, n = 279; standard message group, n = 92; COVID-19 vaccine message group, n = 65; control group, n = 122).
References
- 1.Bode S.M., Gowda C., Mangini M., Kemper A.R. COVID-19 and primary measles vaccination rates in a large primary care network. Pediatrics. 2021;147 doi: 10.1542/peds.2020-035576. e2020035576. [DOI] [PubMed] [Google Scholar]
- 2.Dinleyici E.C., Borrow R., Safadi M.A.P., van Damme P., Munoz F.M. Vaccines and routine immunization strategies during the COVID-19 pandemic. Hum Vaccin Immunother. 2021;17:400–407. doi: 10.1080/21645515.2020.1804776. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Patel Murthy B., Zell E., Kirtland K., Jones-Jack N., Harris L., Sprague C., et al. Impact of the COVID-19 pandemic on administration of selected routine childhood and adolescent vaccinations - 10 U.S. jurisdictions, March-September 2020. MMWR Morb Mortal Wkly Rep. 2021;70:840–845. doi: 10.15585/mmwr.mm7023a2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Irwin C.E., Adams S.H., Park M.J., Newacheck P.W. Preventive care for adolescents: few get visits and fewer get services. Pediatrics. 2009;123:e565–e572. doi: 10.1542/peds.2008-2601. [DOI] [PubMed] [Google Scholar]
- 5.McInerny T.K., Cull W.L., Yudkowsky B.K. Physician reimbursement levels and adherence to American Academy of Pediatrics well-visit and immunization recommendations. Pediatrics. 2005;115:833–838. doi: 10.1542/peds.2004-1510. [DOI] [PubMed] [Google Scholar]
- 6.Child and Adolescent Well-Care Visits (W15, W34, AWC). National Committe for Quality Assurance. www.ncqa.org/hedis/measures/child-and-adolescent-well-care-visits/
- 7.Santoli J.M., Lindley M.C., DeSilva M.B., Kharbanda E.O., Daley M.F., Galloway L., et al. Effects of the COVID-19 Pandemic on routine pediatric vaccine ordering and administration - United States, 2020. MMWR Morb Mortal Wkly Rep. 2020;69:591–593. doi: 10.15585/mmwr.mm6919e2. [DOI] [PubMed] [Google Scholar]
- 8.DeSilva M.B., Haapala J., Vazquez-Benitez G., Daley M.F., Nordin J.D., Klein N.P., et al. Association of the COVID-19 pandemic with routine childhood vaccination rates and proportion up to date with vaccinations across 8 US health systems in the Vaccine Safety Datalink. JAMA Pediatr. 2022;176:68–77. doi: 10.1001/jamapediatrics.2021.4251. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Ackerson B.K., Sy L.S., Glenn S.C., Qian L., Park C.H., Riewerts R.J., et al. Pediatric vaccination during the COVID-19 pandemic. Pediatrics. 2021;148 doi: 10.1542/peds.2020-047092. e2020047092. [DOI] [PubMed] [Google Scholar]
- 10.Jenssen B.P., Fiks A.G. COVID-19 and routine childhood vaccinations-identifying gaps and informing solutions. JAMA Pediatr. 2021;176:21–23. doi: 10.1001/jamapediatrics.2021.4248. [DOI] [PubMed] [Google Scholar]
- 11.Balasuriya L., Santilli A., Morone J., Ainooson J., Roy B., Njoku A., et al. COVID-19 vaccine acceptance and access among Black and latinx Communities. JAMA Network Open. 2021;4:e2128575. doi: 10.1001/jamanetworkopen.2021.28575. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Egede L.E., Walker R.J. Structural racism, social risk factors, and Covid-19 — a dangerous convergence for Black Americans. N Engl J Med. 2020;383:e77. doi: 10.1056/NEJMp2023616. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Mayne S.L., Hannan C., Davis M., Young J.F., Kelly M.K., Powell M., et al. COVID-19 and adolescent depression and suicide risk screening outcomes. Pediatrics. 2021;148 doi: 10.1542/peds.2021-051507. e2021051507. [DOI] [PubMed] [Google Scholar]
- 14.Jenssen B.P., Kelly M.K., Powell M., Bouchelle Z., Mayne S.L., Fiks A.G. COVID-19 and changes in child obesity. Pediatrics. 2021;147 doi: 10.1542/peds.2021-050123. e2021050123. [DOI] [PubMed] [Google Scholar]
- 15.Coronavirus (COVID-19) update: FDA authorizes Pfizer-BioNTech COVID-19 vaccine for emergency use in adolescents in another important action in fight against pandemic [press release] U.S. Food & Drug Administration; 2021. [Google Scholar]
- 16.Kelly B.J., Southwell B.G., McCormack L.A., Bann C.M., MacDonald P.D.M., Frasier A.M., et al. Predictors of willingness to get a COVID-19 vaccine in the U.S. BMC Infect Dis. 2021;21:338. doi: 10.1186/s12879-021-06023-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Nguyen K.H., Srivastav A., Razzaghi H., Lindley M.C., Jorgensen C., Abad N., et al. COVID-19 vaccination intent, perceptions, and reasons for not vaccinating among groups prioritized for early vaccination - United States, September and December 2020. Am J Transplant. 2021;21:1650–1656. doi: 10.1111/ajt.16560. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Jacobson Vann J.C., Jacobson R.M., Coyne-Beasley T., Asafu-Adjei J.K., Szilagyi P.G. Patient reminder and recall interventions to improve immunization rates. Cochrane Database Syst Rev. 2018;1:CD003941. doi: 10.1002/14651858.CD003941.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Bar-Shain D.S., Stager M.M., Runkle A.P., Leon J.B., Kaelber D.C. Direct messaging to parents/guardians to improve adolescent immunizations. J Adolesc Health. 2015;56(5 Suppl):S21–S26. doi: 10.1016/j.jadohealth.2014.11.023. [DOI] [PubMed] [Google Scholar]
- 20.Morris J., Wang W., Wang L., Peddecord K.M., Sawyer M.H. Comparison of reminder methods in selected adolescents with records in an immunization registry. J Adolesc Health. 2015;56(5 Suppl):S27–S32. doi: 10.1016/j.jadohealth.2015.01.010. [DOI] [PubMed] [Google Scholar]
- 21.Szilagyi P.G., Humiston S.G., Gallivan S., Albertin C., Sandler M., Blumkin A. Effectiveness of a citywide patient immunization navigator program on improving adolescent immunizations and preventive care visit rates. Arch Pediatr Adolesc Med. 2011;165:547–553. doi: 10.1001/archpediatrics.2011.73. [DOI] [PubMed] [Google Scholar]
- 22.Suh C.A., Saville A., Daley M.F., Glazner J.E., Barrow J., Stokley S., et al. Effectiveness and net cost of reminder/recall for adolescent immunizations. Pediatrics. 2012;129:e1437–e1445. doi: 10.1542/peds.2011-1714. [DOI] [PubMed] [Google Scholar]
- 23.Szilagyi P.G., Albertin C., Humiston S.G., et al. A randomized trial of the effect of centralized reminder/recall on immunizations and preventive care visits for adolescents. Acad Pediatr. 2013;13:204–213. doi: 10.1016/j.acap.2013.01.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Marron R.L., Lanphear B.P., Kouides R., Dudman L., Manchester R.A., Christy C. Efficacy of informational letters on hepatitis B immunization rates in university students. J Am Coll Health. 1998;47:123–127. doi: 10.1080/07448489809595632. [DOI] [PubMed] [Google Scholar]
- 25.O'Leary S.T., Lee M., Lockhart S., Eisert S., Furniss A., Barnard J., et al. Effectiveness and cost of bidirectional text messaging for adolescent vaccines and well care. Pediatrics. 2015;136:e1220–e1227. doi: 10.1542/peds.2015-1089. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Staras S.A., Vadaparampil S.T., Livingston M.D., Thompson L.A., Sanders A.H., Shenkman E.A. Increasing human papillomavirus vaccine initiation among publicly insured Florida adolescents. J Adolesc Health. 2015;56(5 Suppl):S40–S46. doi: 10.1016/j.jadohealth.2014.11.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Stockwell M.S., Kharbanda E.O., Martinez R.A., Lara M., Vawdrey D., Natarajan K., et al. Text4Health: impact of text message reminder-recalls for pediatric and adolescent immunizations. Am J Public Health. 2012;102:e15–e21. doi: 10.2105/AJPH.2011.300331. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Center for Clinical & Translational Science & Training . 2021. COVID-19 data and research projects.www.cctst.org/covid19 [Google Scholar]
- 29.Milkman K.L., Patel M.S., Gandhi L., Graci H.N., Gromet D.M., Ho H., et al. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor's appointment. Proc Natl Acad Sci U S A. 2021;118 doi: 10.1073/pnas.2101165118. e2101165118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Szilagyi P.G., Thomas K., Shah M.D., Vizueta N., Cui Y., Vangala S., et al. The role of trust in the likelihood of receiving a COVID-19 vaccine: results from a national survey. Prev Med. 2021;153:106727. doi: 10.1016/j.ypmed.2021.106727. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Okoro O., Kennedy J., Simmons G., Jr., Vosen E.C., Allen K., Singer D., et al. Exploring the scope and dimensions of vaccine hesitancy and resistance to enhance COVID-19 vaccination in Black communities. J Racial Ethn Health Disparities. 2021;22:1–14. doi: 10.1007/s40615-021-01150-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Hofstetter A.M., Robinson J.D., Lepere K., Cunningham M., Etsekson N., Opel D.J. Clinician-parent discussions about influenza vaccination of children and their association with vaccine acceptance. Vaccine. 2017;35:2709–2715. doi: 10.1016/j.vaccine.2017.03.077. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Rane M.S., Robertson M.M., Westmoreland D.A., Teasdale C.A., Grov C., Nash D. Intention to vaccinate children against COVID-19 among vaccinated and unvaccinated US parents. JAMA Pediatr. 2021;176:201–203. doi: 10.1001/jamapediatrics.2021.5153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Szilagyi P.G., Schaffer S., Barth R., Shone L.P., Humiston S.G., Ambrose S., et al. Effect of telephone reminder/recall on adolescent immunization and preventive visits: results from a randomized clinical trial. Arch Pediatr Adolesc Med. 2006;160:157–163. doi: 10.1001/archpedi.160.2.157. [DOI] [PubMed] [Google Scholar]
- 35.Who isn’t using patient portals and why? Evidence and implications from a national sample of US adults. Health Affairs. 2018;37:1948–1954. doi: 10.1377/hlthaff.2018.05117. [DOI] [PubMed] [Google Scholar]
- 36.He K., Mack W.J., Neely M., Lewis L., Anand V. Parental perspectives on immunizations: impact of the COVID-19 pandemic on childhood vaccine hesitancy. J Community Health. 2022;47:39–52. doi: 10.1007/s10900-021-01017-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Gould E., Wilson V. Black workers face two of the most lethal preexisting conditions for coronavirus—racism and economic inequality. Economic Policy Institute. 2020. www.epi.org/publication/black-workers-covid/
- 38.Price-Haywood E.G., Burton J., Fort D., Seoane L. Hospitalization and mortality among Black patients and White patients with Covid-19. N Engl J Med. 2020;382:2534–2543. doi: 10.1056/NEJMsa2011686. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Lerner C., Albertin C., Casillas A., Duru O.K., Ong M.K., Vangala S., et al. Patient portal reminders for pediatric influenza vaccinations: a randomized clinical trial. Pediatrics. 2021;148 doi: 10.1542/peds.2020-048413. e2020048413. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Real F.J., DeBlasio D., Beck A.F., Davis D., Cruse B., Samaan Z., et al. A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal. Acad Pediatr. 2017;17:431–435. doi: 10.1016/j.acap.2017.01.010. [DOI] [PubMed] [Google Scholar]
- 41.Rand C.M., Vincelli P., Goldstein N.P.N., Blumkin A., Szilagyi P.G. Effects of phone and text message reminders on completion of the human papillomavirus vaccine series. J Adolesc Health. 2017;60:113–119. doi: 10.1016/j.jadohealth.2016.09.011. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.