ABSTRACT
Next-generation vaccine delivery technologies may provide significant gains from both a technical and behavioral standpoint, but no scale has yet been developed to assess public attitudes to novel vaccine delivery technologies. We therefore performed a cross-sectional validation study that included 1,001 demographically representative participants from the UK and US to develop and validate a novel scale, the Oxford Benchmark Scale for Rating Vaccine Technologies (OBSRVT). A sample of 500 UK participants was used to perform exploratory factor analysis with categorical variables (using a polychoric correlation matrix) followed by promax oblique factor rotation to develop the initial model. This yielded a 15-item 4-domain scale with domains including acceptance (6 items), effectiveness (4 items), comfort (3 items), and convenience (2 items). This model was tested for robustness on a 501-participant demographically representative sample from the US. A confirmatory factor analysis with a Satorra-Bentler scaled test statistic was performed, which demonstrated adequate goodness of fit statistics including the root mean squared error of approximation (0.057), standardized root mean squared residual (0.053), and comparative fit index (0.938). Reliability as internal consistency was excellent (alpha = 0.92). Convergent validity with the Oxford Needle Experience Scale was supported by an adequate correlation (r = 0.31, p < .0001), while discriminant validity was supported by a lack of correlation with an unrelated question (r = -0.03, p < .0001). These findings suggest that the OBSRVT scale represents a feasible, valid, and reliable scale that could be used to gauge the acceptability of existing and future vaccine delivery technologies, and further investigation and testing should be considered.
KEYWORDS: Vaccine delivery technology, vaccines, de-risking, public health, validation study, behavior rating scale, pain
Introduction
Developing novel vaccine delivery technologies is an international public health priority, driven by both the ongoing challenge of vaccine hesitancy and the need for more efficient, deployable, and innovative approaches to vaccine delivery.1,2 While traditional needle and syringe-based vaccination has represented the standard of care for decades, next-generation vaccine delivery technologies such as nasal sprays, microneedle patches, oral vaccines, and ultrasound are increasingly being investigated as alternative, and potentially superior, approaches to vaccine delivery.3–7
Understanding public willingness to consider the uptake of next-generation vaccine delivery technologies is important for two central reasons. First, alternative vaccine delivery technologies may help overcome some of the technical and logistical barriers to vaccination – cold storage, enhancing immune response, risk of sharps injuries, and cost – that could lead to more seamless deployment.6,7 Second, these approaches may help mitigate some of the hesitancy surrounding vaccination, a phenomenon that may present an even greater, or at least more intransigent, barrier to effective vaccine delivery than deployment and efficacy challenges.8
Yet despite decades of work in developing alternatives to needle and syringe-based vaccination, public attitudes to these individual approaches remains unknown, and little has been done to develop a validated approach to assessing patient and public preferences for novel vaccine delivery technologies.9 This is a critical lacuna, as our previous work has suggested that a fear of needles may be the only modifiable risk factor for vaccine hesitancy.10 That said, novel technologies deployed in a medical setting are also sometimes themselves associated with increased mistrust.11–13 Therefore, while developing needle-free approaches to vaccine delivery may represent a powerful modulatory tool in the armamentarium of public health researchers to mitigate vaccine hesitancy, it remains difficult to determine the net value of novel delivery technologies from the public perspective.
As new vaccine delivery approaches are considered, developed, and trialed, it will be crucial to provide researchers with validated instruments so that the preferences of the public and trial participants can be studied, considered, and addressed. On a practical level, while funding agencies and pharmaceutical companies are highly invested in developing new delivery approaches, a validated scale to assess acceptance and attitudes to specific approaches of interest could help with de-risking these technologies at an early stage, helping to target investment and research across the timeline of technological development.14
We therefore sought to develop and validate a feasible, psychometrically sound, and multidimensional benchmark scale – the Oxford Benchmark Scale for Rating Vaccine Technologies (OBSRVT) – that could be used at all phases of alternative vaccine delivery approach development to gauge the relative preferences of the public for these technologies. Ultimately, such a scale may provide evidence of the putative benefits of novel, and sometimes costly, delivery technologies from a societal standpoint that could help tailor future development to better align with public preferences while also helping to justify the research and development of these technologies. This scale will hopefully lead to a better understanding of the barriers to vaccine program delivery in all its myriad forms, ultimately resulting in enhanced vaccine coverage and improved public health outcomes.
Materials and methods
Data collection, population, and overall approach
Our overall methodology was similar to our prior work in developing instruments to assess needle attitudes and vaccine hesitancy.15–18 Initially, we developed a series of questionnaire-based surveys in Qualtrics (Qualtrics International, Provo, Utah, USA) with responses solicited using a survey panel design through Prolific Academic (Oxford, UK) from two separate 50-member cohorts. The first cohort was a convenience sample and the second represented a vaccine hesitant group who self-identified as never having received a vaccination for COVID-19.
After this initial exploratory study, two separate 500-respondent demographically representative samples from the UK and USA (based on UK Office of National Statistics or 2015 US national census bureau data, respectively, and matched by stratification on age, sex, and ethnicity) were targeted to develop the OBSRVT scale. The UK data were used for the exploratory factor analysis (EFA), while US data were used for confirmatory factor analysis (CFA) and reliability and validity testing. The US sample served to assess the scale’s robustness across a distinct, but similar, population, consistent with prior methodological approaches in this area. Sample sizes of approximately 500 participants in each cohort were selected based on two considerations. First, best practices suggest including approximately 10 participants per item in EFA and CFA19; with a total of 40 candidate items for inclusion, we conservatively estimated 500 participants per cohort. Second, our aim was to use demographically representative samples to improve our generalizability, and including a 500-subject sample size in each population permitted representative demographic stratification.
To maximize the generalizability of our scale to all current and future vaccine delivery technologies, we chose to benchmark all responses against respondents’ feelings about vaccine delivery with a traditional needle and syringe. Responses to all items in the scale were therefore scored on a 5-point Likert scale ranging from ‘much less than with a needle and syringe’ to ‘much more than with a needle and syringe,’ with a neutral option ‘the same as with a needle and syringe’ included. Items were reverse coded, as needed, to yield a final scale in which higher scores represent a greater degree of support for the delivery technology under question than with a needle and syringe.
This research was approved by the University of Oxford Medical Sciences Interdivisional Research Ethics Committee (approval reference R81585/RE001). Participants were reimbursed less than £5 for their time. All participants provided informed consent for participation and were permitted to withdraw at any time. Demographic details were self-declared and linked to responses using a unique 25-character alphanumeric identifier. Demographic data included age, self-identified gender, self-identified race and ethnicity, employment status, educational background, household income, and COVID-19 vaccination status.
Qualitative analyses were performed using NVivo release 1.4 (QSR International, Burlington, Massachusetts, USA) and statistical analyses were performed using Stata for Mac version 16 (Stata Corporation, College Station, Texas, USA). A p-value of 0.05 was considered significant for all analyses.
Domain and item generation
To develop a set of domains for consideration, we relied on a 3-part approach. First, a literature review was performed to determine what domains should be considered based on historical precedent. Second, the two 50-member survey cohorts were used to expand the range of domains for inclusion by asking both open- and closed-answer questions regarding priorities for vaccine delivery technologies. Finally, this qualitative research was followed by iterative feedback from the Oxford Vaccine Group and the Engineering Science biology/in vivo cluster meetings.
Closed-ended responses were reported with a 5-point Likert scale while open-ended responses were recorded in free text format. Open-ended questions included those such as: ‘What are your priorities when deciding on how much you value a vaccine delivery technology?’ A draft survey was also provided, with respondents given the opportunity to answer whether any items should be added, modified, or deleted, and whether the answer choices and questions were clear and appropriate.15,16,19,20
Domain and item reduction and exploratory factor analysis
Item reduction was performed by reassessing domain relevance, deleting redundant items, examining polychoric (item-item) and polyserial (item-test) correlations, and deleting items with low communality or factor loadings.21 Both Bartlett’s test of sphericity and the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy were assessed to confirm that data were amenable to factor analysis.22 A combination of an iterative approach, parallel analysis, scree plot, and assessment of the minimum average partial correlation were used to assess the number of domains to retain for inclusion.20,23,24 EFA with categorical variables was performed using a polychoric correlation matrix with iterated principal factor analysis followed by promax oblique factor rotation.25,26
Confirmatory factor analysis
We performed confirmatory maximum likelihood factor analysis with structural equation modeling and a Satorra-Bentler scaled test statistic on the US sample using the model developed in the UK sample to confirm the robustness of our model. Goodness of fit indices were assessed, including the root mean squared error of approximation (RMSEA), the standardized root mean squared residual (SRMR), and the comparative fit index (CFI). Values of RMSEA ≤ 0.08, SRMR ≤ 0.08, and CFI ≥ 0.90 suggest acceptable fit, while values of RMSEA ≤ 0.05, SRMR ≤ 0.06, and CFI ≥ 0.95 suggest excellent fit.27,28
Reliability and validity
Reliability as internal consistency was assessed using Cronbach’s alpha for the overall OBSRVT scale and each subscale.29
We assessed convergent validity using the Oxford Needle Experience (ONE) scale, a validated scale that measures public attitudes to needles.15 Discriminant validity was assessed using a single question regarding political outlook (‘I consider myself to be a conservative person’). For both convergent and discriminant validity, the Pearson correlation coefficient was used.
Results
Characteristics of the representative samples
UK EFA sample
A total of 541 individuals initially accessed our survey; of these, 21 failed to complete the survey (yielding a completion rate of 96.1%) and 20 (3.8%) failed at least one attention check, yielding 500 unique participants for analysis. The median (interquartile range [IQR]) age was 48 (33, 59) and respondents ranged in age from 18 to 82. Overall, 52.3% of respondents identified as female and the median (IQR) household income was £45,000 (26,000, 65000). Additionally, 15.6% of the participants identified as nonwhite, 42.3% reported having no children, 47.3% reported working full-time, and 91.4% reported receiving at least one vaccination for COVID-19.
US CFA sample
A total of 527 individuals initially accessed our survey; of these, 24 failed to complete the survey (yielding a completion rate of 95.4%) and 22 (4.4%) failed at least one attention check, yielding 501 unique participants for analysis. The median (interquartile range [IQR]) age was 47 (32, 59) and respondents ranged in age from 18 to 95. Overall, 50% of respondents identified as female and the median (IQR) household income was $65,000 (40,000, 100,000). Additionally, 34.3% of the participants identified as nonwhite, 44.5% reported having no children, 56.7% reported working full-time, and 81.4% reported receiving at least one vaccination for COVID-19.
Domain and item reduction and exploratory factor analysis
Our iterative approach yielded 10 potential domains including effectiveness, safety (including risks and side effects), ease of use (convenience), acceptance, comfort, cultural acceptability, likelihood of recommendation to others, perceived advantages, perceived disadvantages, and concerns (including anxiety or fear) associated with the delivery approach. We explored a total of 40 items for inclusion in the scale; 25 items were ultimately culled due to redundancy, low item correlations, and low factor loadings, yielding a final 15-item scale.
Bartlett’s test of sphericity (p < .0001) suggested the data were appropriate for EFA, which was confirmed by the KMO measure of sampling adequacy, which was excellent at 0.95. A scree plot, minimum average partial correlation analysis, and parallel analysis suggested a 2 to 4 factor solution.23,30,31
Domains included in the final scale
The four domains ultimately included in the scale were acceptance (6 items), effectiveness (4 items), confidence (3 items), and concerns (2 items). The acceptance domain reflects a generally positive response to the vaccine delivery technology in question, including likelihood of vaccination, willingness, preference, and comfort. Effectiveness, in contrast, reflects the degree to which the respondent believes that the delivery technology will be effective in vaccine delivery, providing immunity, and protection. Confidence highlights the perceived ease of use, ability of health care professionals to deliver the vaccine, and perception of vaccine safety. Finally, the concerns domain addresses fear or worry regarding side effects or safety associated with vaccine delivery using the technology under study.
Factor loadings are included in Table 1 and the OBSRVT scale is presented in Table 2.
Table 1.
Rotated factor loadings for the OBSRVT. Factor loadings >0.40 (our a priori minimum for consideration) are bolded.
| Item | Acceptance | Effectiveness | Confidence | Concerns |
|---|---|---|---|---|
| I would be likely to get vaccinated if this vaccine delivery method were available. | 0.9377 | −0.0221 | −0.0817 | −0.0374 |
| I am willing to try this vaccine delivery method in the future. | 0.8894 | −0.0169 | 0.0593 | −0.0194 |
| I would prefer this vaccine delivery method. | 0.8703 | −0.0124 | 0.0634 | 0.0175 |
| I believe this vaccine delivery method should be widely adopted. | 0.7570 | 0.2215 | −0.0222 | −0.0429 |
| I would recommend this vaccine delivery method to friends and family. | 0.7560 | 0.1965 | 0.0220 | −0.0286 |
| This vaccine delivery method is comfortable. | 0.6434 | −0.2154 | 0.0945 | 0.0015 |
| I trust that this delivery method will provide immunity. | 0.0166 | 0.9844 | −0.0676 | −0.0112 |
| I am confident in the long-term effectiveness of this vaccine delivery method. | 0.0428 | 0.9013 | −0.0279 | 0.0213 |
| I believe this vaccine delivery method would be effective in preventing disease. | −0.0230 | 0.8884 | 0.0284 | −0.0126 |
| I believe this method would be effective in delivering the vaccine as needed. | 0.0532 | 0.7948 | 0.1292 | 0.0155 |
| This vaccine delivery method can be easily used for the general population. | 0.1562 | −0.0909 | 0.7464 | 0.0712 |
| I am confident that health professionals can safely deliver vaccines using this method. | 0.0431 | 0.1287 | 0.7301 | 0.0389 |
| I feel this vaccine delivery method is safe. | 0.0623 | 0.0129 | 0.4538 | −0.2355 |
| I am fearful of the potential side effects of this vaccine delivery method. | −0.0516 | 0.0352 | 0.0740 | 0.8937 |
| I worry about the safety of this vaccine delivery method. | −0.0497 | −0.0307 | −0.0155 | 0.8479 |
Table 2.
The OBSRVT scale. All items are scored using a 5-point Likert scale ranging from much less than with a needle and syringe to much more than with a needle and syringe. Higher scores thus reflect a greater degree of desirability relative to traditional needle and syringe-based vaccination, with a score of 45 suggesting equivalence to needle and syringe-based vaccination. Items 14 and 15 are reverse scored. The total score can range from 15 to 75.
| # | Item | Domain |
|---|---|---|
| 1 | I would be likely to get vaccinated if this vaccine delivery method were available. | Acceptance |
| 2 | I am willing to try this vaccine delivery method in the future. | Acceptance |
| 3 | I would prefer this vaccine delivery method. | Acceptance |
| 4 | I believe this vaccine delivery method should be widely adopted. | Acceptance |
| 5 | I would recommend this vaccine delivery method to friends and family. | Acceptance |
| 6 | This vaccine delivery method is comfortable. | Acceptance |
| 7 | I trust that this delivery method will provide immunity. | Effectiveness |
| 8 | I am confident in the long-term effectiveness of this vaccine delivery method. | Effectiveness |
| 9 | I believe this vaccine delivery method would be effective in preventing disease. | Effectiveness |
| 10 | I believe this method would be effective in delivering the vaccine as needed. | Effectiveness |
| 11 | This vaccine delivery method can be easily used for the general population. | Confidence |
| 12 | I am confident that health professionals can safely deliver vaccines using this method. | Confidence |
| 13 | I feel this vaccine delivery method is safe. | Confidence |
| 14 | I am fearful of the potential side effects of this vaccine delivery method. | Concerns |
| 15 | I worry about the safety of this vaccine delivery method. | Concerns |
Confirmatory factor analysis
Using the final 4-factor 15-item model and structural equation modeling with a Satorra-Bentler scaled test statistic on the US sample, we demonstrated very good goodness of fit characteristics, with RMSEA = 0.057, SRMR = 0.053, and CFI = 0.958.
Scale reliability and validity
Reliability as internal consistency was measured using Cronbach’s alpha; values for the overall scale were very good at 0.92. Values for each of the four subscales are included in Table 3.
Table 3.
Subscale and overall reliability (assessed using Cronbach’s alpha).
| UK Sample | US Sample | |
|---|---|---|
| Acceptance | 0.91 | 0.92 |
| Effectiveness | 0.92 | 0.93 |
| Confidence | 0.70 | 0.71 |
| Concerns | 0.84 | 0.81 |
| Overall OBSRVT | 0.91 | 0.92 |
Convergent validity between the OBSRVT score and the Oxford Needle Experience (ONE) scale score was 0.31 (p < .0001), and discriminant validity between the OBSRVT scale and a single unrelated question (‘I consider myself to be a conservative person’) was −0.03 (p < .0001), supporting our validity argument.
Discussion
This manuscript outlines the initial development and validation of the OBSRVT, the first scale to assess the public’s attitude to novel vaccine delivery technologies. We developed a concise, feasible 15-item scale over four domains: acceptance, effectiveness, confidence, and concerns. By benchmarking responses relative to needle and syringe-based vaccination, we aimed to maximize the utility of the scale in assessing both existing and future vaccine delivery technologies. This may represent an important next step in better understanding public priorities regarding novel vaccine delivery technologies.
Indeed, prior studies evaluating a range of delivery technologies from microneedle array patch approaches to nasal sprays routinely report outcomes such as efficacy, safety, and stability,32–35 but acceptance is generally reported without a validated or standardized instrument.36,37 Given the importance of public buy-in, both to decrease hesitancy and justify the significant public and private cost of this research, this gap is striking and highlights the potential value of the present work.
We adopted a benchmarking approach to rate novel vaccine delivery technologies relative to the ubiquitous needle and syringe-based vaccination approach. Benchmarking not only allows us to assess preferences relative to needle and syringe vaccines, but also to compare different alternative vaccine delivery technologies to each other, since each is rated relative to a stable, familiar, and immutable benchmark.
Each of the four domains in our scale – acceptance, effectiveness, confidence, and concerns – includes a different number of items. Since the scale is scored on a Likert scale ranging from 1 (‘much less than with a needle and syringe’) to 5 (‘much more than with a needle and syringe’), a score of 3 on each item suggests equivalence to needle and syringe (‘the same as with a needle and syringe’). Thus, the neutral cutoffs for each subscale, as outlined in Table 2, are 18, 12, 9, and 6 (yielding an overall scale cutoff of 45), which allows for easy benchmarking of the technology under investigation; values higher than the cutoffs suggest that the technology under investigation is preferred to a needle and syringe. This approach is particularly powerful in the dynamic environment of vaccine delivery technology development, as the OBSRVT is flexible and adaptable enough to be used to evaluate future technologies that have not yet been considered.
The benchmarking approach is also powerful on a practical level; sample data (Table 4) for an assessment of a theoretical nasal spray vaccine, for example, suggest that the overall feelings regarding this approach in the UK and US are similar, with the same median score in each population, bolstering our validity argument. Even these sample data could suggest to researchers, funders, and governments that the public largely sees nasal spray as superior to needle and syringe-based vaccination, but that concerns regarding effectiveness (with scores slightly below the threshold, suggesting concern relative to needle and syringe-based vaccination) would need to be addressed carefully to encourage broad adoption. These findings also further support the face validity of the OBSRVT scale, since (as expected) participants were enthusiastic about the potential for nasal vaccine delivery based on acceptance scores while – not surprisingly – they had some residual concern regarding effectiveness, presumably given the relative novelty of the technology.
Table 4.
Representative comparison of scores for the OBSRVT scale when assessing a nasal spray vaccine between UK and US samples. The table presents the median and interquartile range (IQR) values for responses in the domains of acceptance, effectiveness, confidence, and concerns, as well as the overall OBSRVT score. P-values are derived from the Mann-Whitney U test, assessing the statistical significance of differences in the distributions between the two independent samples. Cutoffs for needle and syringe based equivalence are based on a score of 3 for each Likert scale item; this is computed easily based on the number of items in each domain. A p-value less than 0.05 suggests a statistically significant difference.
| UK Sample Median (IQR) | US Sample Median (IQR) | Cutoffs for needle and syringe equivalence | p-value | |
|---|---|---|---|---|
| Acceptance | 21 (18–25) | 21 (17–24) | 18 | 0.02 |
| Effectiveness | 10 (8–12) | 11 (8–12) | 12 | 0.07 |
| Confidence | 10 (9–12) | 10 (9–11) | 9 | 0.02 |
| Concerns | 6 (6–7) | 6 (4–7) | 6 | 0.27 |
| OBSRVT Overall | 48 (43–54) | 48 (41–54) | 45 | 0.17 |
Our work has several other strengths. First, we used a rigorous iterative process for domain and item development, including qualitative data from open-ended questions, to ensure that we could capture public priorities adequately. Second, we specifically included a vaccine hesitant cohort when initially soliciting feedback to increase the chances that we would capture priorities of a traditionally marginalized – yet critical for increasing novel vaccine adoption – group. Third, we used large demographically representative samples for our exploratory and confirmatory factor analyses to improve the generalizability of our findings. Fourth, to assess the robustness of our model, we used a UK sample for exploratory factor analysis and a US sample for confirmatory factor analysis. Fifth, we performed our EFA using polychoric correlation matrices, an approach designed specifically for categorical data, to ensure that the factor analysis was performed appropriately given the Likert-scale responses. Finally, as alluded to above, the face validity of the scale was supported by the similarity in the baseline scores between the UK and US populations as well as the lower scores seen for the effectiveness subscale.
By design, there is some overlap between the items included within each of the scale’s domains. For example, in the Confidence domain, item 12 (“I am confident that health professionals can safely deliver vaccines using this method”) addresses confidence in the perceived ability of health professionals to practically use the delivery technology in question, while item 13 (“I feel this vaccine delivery method is safe”) addresses a more general sense of confidence in the overall perceived safety of the technology itself. The distinction between these items may be particularly notable when exploring novel vaccine delivery technologies that may be perceived as technically difficult to deliver in a reproducible and consistent manner.
Limitations of our study include our use of a survey panel for subject recruitment, so that participants may not truly reflect the population of interest, though this was mitigated by relying on large demographically representative samples. Indeed, the essentially identical scores seen in the UK and US populations (Table 4) suggests that our findings are indeed robust and thus likely representative. Second, our domains include an uneven number of items, and only two items were included in the concerns domain; still, since those items address the two key area of potential concern for participants – side effects and safety – and since their loading and structure is statistically valid, the practical utility of the factorial structure of our scale is supported. Third, while we included representative samples from both the UK and US in this study, future studies exploring whether our scale is valid across disparate cultural contexts, as well as invariance testing and test-retest reliability assessments, may be warranted. Fourth, most of the items in our scale were framed to elicit acceptance; while this is in part a function of our domain choices, and we included two items with reverse scoring in the Concerns domain, it is possible that this approach could introduce bias. Finally, given the potential for baseline attitudes to needle and syringe-based vaccination to change over time, individual ratings may not remain stable. Thus, future studies assessing the responsiveness of the OBSRVT and its subscales – that is, how individual scores change over time as a result of the evolution of preferences, changes in circumstance, or a history of exposure – may be considered.
This manuscript presents data on the development and validation of the OBSRVT, the first scale designed specifically to assess the public’s attitudes to next-generation vaccine delivery technologies. Given the public and private outlay of research funding in these areas,38,39 and the potential for such technologies to fundamentally shift the ways in which vaccine delivery is effectuated, a formal validated scale is necessary to better assess public buy-in, better justify research funding, better aid in de-risking technologies, and better understand potential acceptance bottlenecks prior to full vaccine technology deployment. By developing a short yet comprehensive 15-item, 4-domain scale that captures key considerations for vaccine delivery technologies – acceptance, effectiveness, confidence, and concerns – we hope that our work may serve as a tool for investigators as vaccine delivery technologies continue to evolve.
Biographies
Jonathan Kantor is Associate Professor of Medicine; at Oxford, he is part of the Vaccines and Society Unit of the Oxford Vaccine Group and the Institute for Biomedical Engineering.
Robert C. Carlisle is Professor of Biomedical Engineering, Director of the MSc for Nanotechnology in Medicine and Healthcare program, and the Oxford Director of the EPSRC Centre for Doctoral Training in Engineering Biology.
Samantha Vanderslott is Associate Professor and the head of the Vaccines and Society Unit of the Oxford Vaccine Group.
Andrew J. Pollard is Director of the Oxford Vaccine Group and the Ashall Professor of Infection and Immunity.
Michael Morrison is Senior Researcher in Social Science at HeLEX, the Centre for Health, Law and Emerging Technologies in the Faculty of Law at Oxford.
Funding Statement
SV received funding from AHRC-IRC [HNR02360] and NIHR [HNR02910]. RCC is grateful for the generous benefaction of Mr Donald Porteus which supported his contribution. The funding agencies and parties were not involved in study design, data collection, analysis, or presentation.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Author contributions
Study conception: JK, RCC
Study design: JK, SV, MM, AJP, RCC
Statistical analyses: JK, AJP, RCC
Writing: JK, SV, MM, AJP, RCC
Oversight: SV, MM, AJP, RCC
Data statement
Data are available from the corresponding author.
References
- 1.Kanekiyo M, Ellis D, King NP.. New vaccine design and delivery technologies. J Infect Dis. 2019;219(Supplement_1):S88–7. doi: 10.1093/infdis/jiy745. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Wallis J, Shenton DP, Carlisle RC. Novel approaches for the design, delivery and administration of vaccine technologies. Clin Exp Immunol. 2019;196(2):189–204. doi: 10.1111/cei.13287. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kumar M, Dogra R, Mandal UK. Nanomaterial-based delivery of vaccine through nasal route: opportunities, challenges, advantages, and limitations. J Drug Delivery Sci Technol. 2022;74:103533. doi: 10.1016/j.jddst.2022.103533. [DOI] [Google Scholar]
- 4.Tang J, Cai L, Xu C, Sun S, Liu Y, Rosenecker J, Guan S. Nanotechnologies in delivery of DNA and mRNA vaccines to the nasal and pulmonary mucosa. Nanomater (Basel, Switz). 2022;12(2):226. doi: 10.3390/nano12020226. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rodgers AM, Cordeiro AS, Donnelly RF. Technology update: dissolvable microneedle patches for vaccine delivery. Med Dev Evidence Res. 2019;12:379–398. doi: 10.2147/MDER.S198220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Hettinga J, Carlisle R. Vaccination into the dermal compartment: techniques, challenges, and prospects. Vaccines (Basel). 2020;8(3):534. doi: 10.3390/vaccines8030534. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hettinga JK, Lyons B, Balkaran J, Rijal P, Dunn‐Lawless D, Caproni L, Gray M, Suslick KS, Coussios CC, Carlisle RC. Cavitation‐mediated transcutaneous delivery of protein and nucleotide‐based antigen for rapid high‐level immune responses. Adv Ther. 2023;6(12):n/a. doi: 10.1002/adtp.202300102. [DOI] [Google Scholar]
- 8.Freeman D, Loe BS, Chadwick A, Vaccari C, Waite F, Rosebrock L, Jenner L, Petit A, Lewandowsky S, Vanderslott S, et al. COVID-19 vaccine hesitancy in the UK: the oxford coronavirus explanations, attitudes, and narratives survey (oceans) II. Psychol Med. 2022;52(14):3127–3141. doi: 10.1017/S0033291720005188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kaaijk P, Kleijne DE, Knol MJ, Harmsen IA, Ophorst OJAE, Rots NY. Parents’ attitude toward multiple vaccinations at a single visit with alternative delivery methods. Hum Vaccines & Immunotherapeutics. 2014;10(8):2483–2489. doi: 10.4161/hv.29361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Freeman D, Lambe S, Yu L-M, Freeman J, Chadwick A, Vaccari C, Waite F, Rosebrock L, Petit A, Vanderslott S, et al. Injection fears and COVID-19 vaccine hesitancy. Psychol Med. 2023;53(4):1185–1195. doi: 10.1017/S0033291721002609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Caplan AL, Ferguson K, Williamson A. Ethical challenges of advances in vaccine delivery technologies. Hastings Cent Report. 2024;54(1):13–15. doi: 10.1002/hast.1563. [DOI] [PubMed] [Google Scholar]
- 12.Jaiswal J, Halkitis PN. Towards a more inclusive and dynamic understanding of medical mistrust informed by science. Behavioral Med (Wash, DC). 2019;45(2):79–85. doi: 10.1080/08964289.2019.1619511. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Nguyen BT, Brown AL, Jones F, Jones L, Withers M, Ciesielski KM, Franks JM, Wang C. “I’m not going to be a guinea pig:” medical mistrust as a barrier to male contraception for black American men in Los Angeles, CA. Contraception (Stoneham). 2021;104(4):361–366. doi: 10.1016/j.contraception.2021.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Abou-El-Enein M, Duda GN, Gruskin EA, Grainger DW. Strategies for derisking translational processes for biomedical technologies. Trends In Biotechnol (Regular Ed). 2017;35(2):100–108. doi: 10.1016/j.tibtech.2016.07.007. [DOI] [PubMed] [Google Scholar]
- 15.Kantor J, Vanderslott S, Morrison M, Pollard AJ, Carlisle RC. The oxford needle experience (ONE) scale: a UK-based and us-based online mixed-methods psychometric development and validation study of an instrument to assess needle fear, attitudes and expectations in the general public. BMJ Open. 2023;13(12):e074466. doi: 10.1136/bmjopen-2023-074466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kantor BN, Kantor J. Development and validation of the oxford pandemic attitude scale-COVID-19 (OPAS-C): an internet-based cross-sectional study in the UK and USA. BMJ Open. 2021;11(7):e043758–e043758. doi: 10.1136/bmjopen-2020-043758. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kantor J, Carlisle RC, Morrison M, Pollard AJ, Vanderslott S. The oxford vaccine hesitancy scale (OVHS): a UK-based and us-based online mixed-methods psychometric development and validation study of an instrument to assess vaccine hesitancy. BMJ Open. 2024;14(10):e084669. doi: 10.1136/bmjopen-2024-084669. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Kantor J, Aasi SZ, Alam M, Paoli J, Ratner D. Development and validation of the oxford skin cancer treatment scale, a patient-reported outcome measure for health-related quality of life and treatment satisfaction after skin cancer treatment. Dermatol Surg. 2024. Nov 1;50(11):991–996. doi: 10.1097/DSS.0000000000004305. Epub 2024 Jul 12. PMID: 38996368. [DOI] [PubMed] [Google Scholar]
- 19.Boateng GO, Neilands TB, Frongillo EA, Melgar-Quinonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health. 2018;6:149. doi: 10.3389/fpubh.2018.00149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Watkins MW. A step-by-step guide to exploratory factor analysis with stata. New York (NY): Routledge; 2022. [Google Scholar]
- 21.Cappelleri JC, Lundy JJ, Hays RD. Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures comment. Clin Ther. 2014;36(5):648–662. doi: 10.1016/j.clinthera.2014.04.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Lorenzo-Seva U, Ferrando PJ. MSA: the forgotten index for identifying inappropriate items before computing exploratory item factor analysis. Methodol. 2021;17(4):296–306. doi: 10.5964/meth.7185. [DOI] [Google Scholar]
- 23.Dinno A. Implementing horn’s parallel analysis for principal component analysis and factor analysis. The Stata J: Promoting Commun On Stat Stata. 2009;9(2):291–298. doi: 10.1177/1536867X0900900207. [DOI] [Google Scholar]
- 24.Horn JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30(2):179–185. doi: 10.1007/BF02289447. [DOI] [PubMed] [Google Scholar]
- 25.Holgado–Tello FP, Chacón–Moscoso S, Barbero–García I, Vila–Abad E. Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables. Qual Quant. 2010;44(1):153–166. doi: 10.1007/s11135-008-9190-y. [DOI] [Google Scholar]
- 26.Flora DB, Curran PJ. An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods. 2004;9(4):466–491. doi: 10.1037/1082-989X.9.4.466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Browne MW, Cudeck R. Alternative ways of assessing model fit. Sociolog Methods & Res. 1992;21(2):230–258. doi: 10.1177/0049124192021002005. [DOI] [Google Scholar]
- 28.Browne MW, Cudeck R. The use of causal indicators in covariance structure models: some practical issues. Psycholog Bull. 1993;114(3):533–541. doi: 10.1037/0033-2909.114.3.533. [DOI] [PubMed] [Google Scholar]
- 29.Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Personality Assess. 2003;80(1):99–103. doi: 10.1207/S15327752JPA8001_18. [DOI] [PubMed] [Google Scholar]
- 30.Garrido LE, Abad FJ, Ponsoda V. Performance of velicer’s minimum average partial factor retention method with categorical variables. Educ Psychological Meas. 2011;71(3):551–570. doi: 10.1177/0013164410389489. [DOI] [Google Scholar]
- 31.Ledesma RD, Valero-Mora P, Macbeth G. The scree test and the number of factors: a dynamic graphics approach. Span J Psychol. 2015;18:18. doi: 10.1017/sjp.2015.13. [DOI] [PubMed] [Google Scholar]
- 32.Nguyen TT, Oh Y, Kim Y, Shin Y, Baek S-K, Park J-H. Progress in microneedle array patch (MAP) for vaccine delivery. Hum Vaccines & Immunotherapeutics. 2021;17(1):316–327. doi: 10.1080/21645515.2020.1767997. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Forster A, Junger M. Opportunities and challenges for commercializing microarray patches for vaccination from a MAP developer’s perspective. Hum Vaccines & Immunotherapeutics. 2022;18(4):2050123. doi: 10.1080/21645515.2022.2050123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Baker B, Hacker E, Siller G, Lee M, Mursaliyev N, Forster A. Evaluation of the self-administration potential of high-density microarray patches to human skin: a preliminary study. Hum Vaccines & Immunotherapeutics. 2023;19(1):2189409. doi: 10.1080/21645515.2023.2189409. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Kumar P, Bird C, Holland D, Joshi SB, Volkin DB. Current and next-generation formulation strategies for inactivated polio vaccines to lower costs, increase coverage, and facilitate polio eradication. Hum Vaccines & Immunotherapeutics. 2022;18(7):2154100. doi: 10.1080/21645515.2022.2154100. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Ma Y, Li J, Cao Y, Li W, Shi R, Jia B, Wang H, Yan L, Suo L, Yang W, et al. Acceptability for the influenza virus vector COVID-19 vaccine for intranasal spray: a cross-sectional survey in Beijing, China. Hum Vaccines & Immunotherapeutics. 2023;19(2):2235963. doi: 10.1080/21645515.2023.2235963. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Qu S, Zhou M, Campy KS, He W. Predictors of parental acceptance to live attenuated influenza vaccine for children. Hum Vaccines & Immunotherapeutics. 2024;20(1):2356343. doi: 10.1080/21645515.2024.2356343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Gouglas D, Thanh Le T, Henderson K, Kaloudis A, Danielsen T, Hammersland NC, Robinson JM, Heaton PM, Røttingen J-A. Estimating the cost of vaccine development against epidemic infectious diseases: a cost minimisation study. The Lancet Global Health. 2018;6(12):e1386–e1396. doi: 10.1016/S2214-109X(18)30346-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Lalani HS, Nagar S, Sarpatwari A, Barenie RE, Avorn J, Rome BN, Kesselheim AS. US public investment in development of mRNA COVID-19 vaccines: retrospective cohort study. Bmj. 2023;380:e073747. doi: 10.1136/bmj-2022-073747. [DOI] [PMC free article] [PubMed] [Google Scholar]