Plain Language vs Standard Format for Youth Understanding of COVID-19 Recommendations

This randomized clinical trial evaluates a plain language recommendation format vs a standard language version for understanding COVID-19 health recommendations among youths.

Key Points

Question

Is COVID-19 health information easier to understand for youths when presented in a plain language format compared with a standard language version of a health recommendation?

Findings

In this randomized clinical trial of 268 youths, no significant difference in understanding was found between participants who received plain language COVID-19 health recommendations and those who received a standard language version. Participants ranked the plain language format higher in terms of accessibility, usability, and satisfaction, suggesting that it may be preferred for communicating health recommendations to youths.

Meaning

The findings suggest that improvements for better understanding of PLRs among youths are warranted.

Abstract

Importance

To ensure that youths can make informed decisions about their health, it is important that health recommendations be presented for understanding by youths.

Objective

To compare understanding, accessibility, usability, satisfaction, intention to implement, and preference of youths provided with a digital plain language recommendation (PLR) format vs the original standard language version (SLV) of a health recommendation.

Design, Setting, and Participants

This pragmatic, allocation-concealed, blinded, superiority randomized clinical trial included individuals from any country who were 15 to 24 years of age, had internet access, and could read and understand English. The trial was conducted from May 27 to July 6, 2022, and included a qualitative component.

Interventions

An online platform was used to randomize youths in a 1:1 ratio to an optimized digital PLR or SLV format of 1 of 2 health recommendations related to the COVID-19 vaccine; youth-friendly PLRs were developed in collaboration with youth partners and advisors.

Main Outcomes and Measures

The primary outcome was understanding, measured as the proportion of correct responses to 7 comprehension questions. Secondary outcomes were accessibility, usability, satisfaction, preference, and intended behavior. After completion of the survey, participants indicated their interest in completing a 1-on-1 semistructured interview to reflect on their preferred digital format (PLR or SLV) and their outcome assessment survey response.

Results

Of the 268 participants included in the final analysis, 137 were in the PLR group (48.4% female) and 131 were in the SLV group (53.4% female). Most participants (233 [86.9%]) were from North and South America. No significant difference was found in understanding scores between the PLR and SLV groups (mean difference, 5.2%; 95% CI, −1.2% to 11.6%; P = .11). Participants found the PLR to be more accessible and usable (mean difference, 0.34; 95% CI, 0.05-0.63) and satisfying (mean difference, 0.39; 95% CI, 0.06-0.73) and had a stronger preference toward the PLR (mean difference, 4.8; 95% CI, 4.5-5.1 [4.0 indicated a neutral response]) compared with the SLV. No significant difference was found in intended behavior (mean difference, 0.22 (95% CI, −0.20 to 0.74). Interviewees (n = 14) agreed that the PLR was easier to understand and generated constructive feedback to further improve the digital PLR.

Conclusions and Relevance

In this randomized clinical trial, compared with the SLV, the PLR did not produce statistically significant findings in terms of understanding scores. Youths ranked it higher in terms of accessibility, usability, and satisfaction, suggesting that the PLR may be preferred for communicating health recommendations to youths. The interviews provided suggestions for further improving PLR formats.

Trial Registration

ClinicalTrials.gov Identifier: NCT05358990

Introduction

Since the beginning of the COVID-19 pandemic, the public has been overwhelmed with COVID-19 information. Clinical practice guidelines have been a primary source of guidance. However, as most guidelines are developed for health care professionals,¹ they may not be understandable or accessible to everyone. To encourage uptake of evidence-informed health behavior by the public, the availability of clear, accessible, and understandable information is critical.

In response to the surge of COVID-19 guidelines, we created a global catalog, the COVID-19 map of recommendations and gateway to contextualization (RecMap),² which aims to identify credible COVID-19 guidelines, extract relevant information, and provide high-quality guidance in plain language for the public.¹ To promote public autonomy in COVID-19 decision-making, we developed a process to draft COVID-19 plain language versions of recommendations (PLR) and publish them on the RecMap website.^2,3,4

Research shows that PLRs with behaviorally specified information, compared with original text, are associated with increased likelihood of implementation.⁵ Plain language recommendations are most effective when they include information on the relevance of the a recommendation to the user,^3,6 provide clarity on how to apply the recommendation to make decisions,^3,6 advise on the quality of evidence underpinning the recommendation,^3,7 and detail the expected benefits and harms.³ These items have been adopted by guideline organizations to help users making health decisions.^8,9,10

Yet, despite an increased awareness of the importance to produce accessible and understandable versions of health care guidelines for the public,⁶ PLRs have been poorly explored as a tool to support mobilization of knowledge to the public.² Given the scarcity of literature and guidelines on how to develop and present PLRs, we designed 3 randomized clinical trials (RCTs) to test adults’, parents’, and youths’ interpretation and understanding of COVID-19 guidelines when presented in a PLR format tailored to the target population vs the original guideline text format, herein referred to as the standard language version (SLV). The trials aimed to investigate the understanding, accessibility, usability, satisfaction, and intention to implement the recommendation and to elicit a preference for health recommendations when presented as either PLRs or SLVs.

Methods

Trial Design

The protocol for the trials in the 3 populations (adults, parents, and youths) is included in Supplement 1.¹¹ In brief, each was a pragmatic, 2-arm, allocation-concealed, blinded, superiority RCT conducted online using a survey for measuring primary and secondary outcomes combined with qualitative interviews among a subset of self-selected trial participants. Researchers from The Hospital for Sick Children in Toronto, Ontario, Canada, led the design and conduct of the youth trial and recruited youths aged 15 to 24 years worldwide to investigate superiority of either PLRs or SLVs from May 27 to July 6, 2022. We hypothesized that health information would be easier to understand by youths when presented in a PLR format. We reported this trial in accordance with the Consolidated Standards of Reporting Trials (CONSORT), CONSORT-Outcomes 2022, and Consolidated Criteria for Reporting Qualitative Research (COREQ) reporting guidelines.^12,13,14 Research ethics approval was obtained through Clinical Trials Ontario from the Hamilton Integrated Research Ethics Board, and participants provided electronic informed consent. Youth participants who completed the survey had the opportunity to enter a drawing to win 1 of 10 gift cards valued at 25 CAD; all interview participants received a gift card valued at 25 CAD. This was a token of appreciation for their contribution. All youth advisors who participated in the pretrial workshops were compensated for their time and contribution.

Participants

We used the World Health Organization’s definition of youths as individuals aged 15 to 24 years.¹⁵ Youths from any country who self-reported being aged 15 to 24 years, had internet access, and could read and understand English were eligible.

We identified organizations with a youth network from around the world through web searches and a social media platform and emailed the youth trial’s recruitment flyer to those organizations, asking them to share it with their network through their social media platforms or email or in upcoming newsletters. We also asked the Cochrane Consumer Network to share our flyer with their youth networks and shared the flyer on our page of a social media platform, tagging the identified global youth organizations and asking them to share our post with their followers.

Participant Inclusion

After youths agreed to participate using our study recruitment flyer (eFigure 1 in Supplement 2), they accessed an online trial link via a survey platform and first answered a set of eligibility questions. Eligible youths then reviewed general study information and the consent form. After providing electronic consent to participate in this anonymous trial and selecting their geographic region, participants were randomized to 1 of the 2 recommendation topics and then, in a 1:1 ratio, to the intervention (PLR) or control (SLV) format of the recommendation (eAppendices 1 and 2 in Supplement 2). We developed quality-control measures to exclude poor-quality responses. Based on our piloting work, we determined a minimal time needed to complete all tasks (6 minutes); as such, the online survey platform response quality tool flagged participants who completed the survey in an inappropriately short time (ie, <6 minutes [speeding]) or who were labeled by the platform as straight-lining (eg, respondents selected a single option for all questions). Additionally, we identified and excluded spam responses to ensure exclusion of suspicious or robotic responses; spam was defined as responses generated by an identical internet Protocol address and with identical start and stop times.

Interventions

We used the following 2 COVID-19 recommendations: (1) “the World Health Organization suggests that people with comorbidities that increase their risk for severe illness when infected with COVID-19 should take the Pfizer-BioNTech vaccine to protect against COVID-19”¹⁶ and (2) “the Centers for Disease Control and Prevention and the Advisory Committee on Immunization Practices recommended that people aged 16 years and older should get 2 doses of the Pfizer-BioNTech COVID-19 vaccine.”¹⁷ We refined previously developed PLR templates^3,4 through pretrial engagement work with Canadian youth advisors (eAppendix 3 in Supplement 2). The PLR information was extracted from the original SLVs using a process that we developed as part of the RecMap work; the draft PLR was edited by a clinical expert (K.P.) and reviewed by other authors.⁴

Randomization, Masking, and Procedures

Randomization was stratified by region (Africa, North and South America, Eastern Mediterranean, Europe, Southeast Asia, and Western Pacific) to ensure an equal chance for participants from each region to be randomized to the intervention or control arm. The allocation sequence was concealed using SurveyMonkey software (Momentive), which uses a proprietary algorithm without a preidentified sequence. Participants first reviewed the recommendation to which they were randomized and then answered outcome questions. While participants were informed that the trial was testing different formats to present COVID-19 guidelines, they were blinded to their allocation to the intervention or control until after completion of all tasks at the end of the survey. After submitting all responses, participants received the alternative format of the recommendation and were asked to compare both formats before indicating which format they preferred. Lastly, participants completed a set of demographic questions and self-reported their gender, region, ethnicity (including Black, East Asian, Indigenous, Latino, Middle Eastern or North African, South Asian, Southeast Asian, White, other, or prefer not to respond), educational level, and level of health literacy. Demographic information was ascertained to help us characterize the diversity of the study population; for ethnicity, participants were given multiple-choice options defined by the research team or the option of “other” with a free-text box for reporting a specific category. Team members were blinded while interpreting the data and drafting the manuscript as we used general group labels for the intervention (format 1) and control (format 2) before revealing the group allocation.^18,19

Outcomes

The trial primary outcome was understanding, defined as the correct interpretation of COVID-19 recommendation content, including its target audience, the benefits and adverse effects of the specified recommendation, and its year of publication. We chose this outcome based on prior work testing plain language versions of scientific information and because measuring actual behavior or health outcome was not feasible as part of this trial.²⁰ Furthermore, this outcome was deemed to be the most relevant to youths’ ability to consume and apply health information. Understanding was measured using 7 multiple-choice questions that tested key content to be found in both recommendation formats (PLR and SLV). Participants were given a score ranging from 0 to 7, based on the number of correct responses; the score was converted into a percentage of correct responses (Supplement 1). Based on previous literature that referenced a baseline understanding score of 60% correct answers when reading health information, we hypothesized that an increase of 10% or more in understanding between the PLR and SLV groups would be an important difference.^{20,21,22,23,24}

Secondary trial outcomes were based on prior trials^{20,24,25,26,27} and included (1) accessibility and usability (the ability to find and use information in the recommendation), (2) user satisfaction (participants’ impression of the recommendation’s presentation [eg, length of the document]), (3) intended behavior (participants’ intention in adopting and following the recommendation they read), and (4) preference (liking 1 format more than the other [PLR vs SLV]). Definitions and measurement details are available in Supplement 1.

The outcome assessment survey contained Likert scale questions, multiple-choice questions, and open-ended questions (eAppendix 4 in Supplement 2). No trial outcomes were changed after commencement.

Qualitative Study

Upon successful completion of the survey, participants indicated their interest in completing a 1-on-1 semistructured interview to reflect on their preferred digital format (PLR or SLV) and their outcome assessment survey responses. The Morville honeycomb model,²⁸ which distinguishes among 7 dimensions of user experience (eFigure 2 in Supplement 2), was used to develop the interview guide (eAppendix 5 in Supplement 2). The guide was pilot tested by members of the study team. Prospective participants were selected using convenience sampling and contacted via email. Informed consent was obtained verbally from participants by the researcher prior to starting the interview. Interviews were conducted in English by a female research coordinator (L.S.) with a master’s degree and previous experience conducting qualitative interviews over a video conferencing platform and took between 15 and 30 minutes. Consenting participants were informed at the beginning of the interview of the study objectives and had no prior relationship with L.S. Demographic data, including country of origin, ethnicity, gender, educational level, and comfort in reading health information, were collected. Interviews were recorded with participants’ permission, transcribed verbatim, cleaned, and analyzed using a thematic analysis.²⁹ Interviews were conducted until data saturation was reached.

Statistical Analysis

For a 2-sided (α = .05), modified, intention-to-treat superiority analysis, we used a relative risk of 1.25 (ie, a 75% correct response rate in the PLR group) for the understanding scores to calculate the sample size^21,24 with a t test of the null hypothesis of no difference between groups. With β = .8 and an allocation ratio of 1:1, we aimed to recruit 240 participants (120 in each arm) using Stata/SE, version 16.0 (StataCorp LLC).³⁰ Using 2 COVID-19 recommendation topics, we conducted a preplanned interim analysis at half of our intended sample size to test for interaction between the recommendation topic and our primary outcome—understanding. All analyses were conducted using SPSS, version 23 (IBM Corp).

Descriptive and Inferential Analyses

Participant baseline characteristics and outcomes were summarized using means (SDs) for continuous variables and proportions for categorical variables. A modified intention-to-treat analysis was conducted on the data from all participants who completed the entire survey. For the primary outcome, we used 2 tests and reported the risk difference with 95% CIs. For overall understanding, we calculated the proportion of correct responses, calculated a mean of this proportion for all understanding questions, and then compared the mean proportion of correct responses between groups using a t test. For the outcome preference, we presented preference as mean (SD) overall and for both trial arms. Skewness, Shapiro-Wilk tests, and histograms were used to evaluate whether the distribution was shifted toward the same preference in both groups. Analysis techniques for other outcomes are described in Supplement 1. All t tests were conducted after verifying for homogeneity of variances using a Levene test. Two-sided P < .05 was considered significant.

Interview Analysis

Interview transcripts, coded by A.S. using a thematic analysis approach,²⁹ were compared, verified, and merged and categorized into 3 to 5 themes by L.S. Emerging themes were identified. NVivo, version 12 (Lumivero) was used for data management.³¹ Member checking was not conducted. Describing the sample of 14 participants, we defined “no participants” as 0 (0%), “few participants” as 1 to 3 (1%-25%), “several participants” as 4-6 (26%-49%), “half of the participants” as 7 (50%), “majority of participants” as 8 to 10 (51%-75%), “almost all participants” as 11 to 13 (75%-99%), and “all participants” as 14 (100%).

Results

Participants

Through our team’s networks and the RecMap collaborators, we reached out purposively to 272 organizations, including 209 in North and South America, 6 in Africa, 9 in Southeast Asia, 34 in Europe, 2 in the Eastern Mediterranean, and 12 in the Western Pacific. Of the 918 youths who accessed the trial link, 495 were randomized to either the PLR (n = 254) or the SLV (n = 241) group. Of these youths, 227 (PLR: n = 117; SLV: n = 110) were excluded from the final analysis due to speeding, straight-lining, or having a duplicate internet Protocol address. In total, we included 268 youths in the final analysis, of whom 137 were in the PLR group (48.4% female; 49.6% male) and 131 were in the SLV group (53.4% female; 47.4% male) (Figure 1); their characteristics are detailed in Table 1. A total of 22.8% were Black; 5.6%, East Asian; 3.0%, Indigenous; 10.4%, Latino; 1.0%, Middle Eastern or North African; 7.8%, South Asian; 2.2%, Southeast Asian; 45.5%, White; and 1.1%, other ethnicity; 1.0% preferred not to respond. Of those who reported other ethnicity, 1 was Asian American, 1 was mixed Asian British, and 1 was White and South Asian. Characteristics of randomized participants who were included and excluded from the analysis are presented in the eTable in Supplement 2.

Figure 1. CONSORT Diagram.

PLR indicates plain language recommendation; SLV, standard language version.

Table 1. Baseline Characteristics of Participants per Group.

Characteristic	Participants, No. (%) (N = 268)
	PLR (n = 137)	SLV (n = 131)
Demographics
Gendera
Female	65 (47.4)	70 (53.4)
Male	68 (49.6)	63 (48.4)
Transgender girl or woman	2 (1.5)	1 (0.8)
Transgender boy or man	1 (0.7)	0
Gender queer	1 (0.7)	0
Gender fluid	1 (0.7)	0
Nonbinary	4 (2.9)	2 (1.5)
Prefer not to respond	1 (0.7)	1 (0.8)
Region
Africa	7 (5.1)	6 (4.6)
North and South America	119 (86.9)	114 (87.0)
Eastern Mediterranean	1 (0.7)	0
Europe	7 (5.1)	3 (2.3)
Southeast Asia	2 (1.5)	3 (2.3)
Western Pacific	1 (0.7)	5 (3.8)
Ethnicitya
Black	29 (21.2)	32 (24.4)
East Asian	7 (5.1)	8 (6.1)
Indigenous	7 (5.1)	1 (0.8)
Latino	17 (12.4)	11 (8.4)
Middle Eastern or North African	0	2 (1.5)
South Asian	7 (5.1)	14 (10.7)
Southeast Asian	3 (2.2)	3 (2.3)
White	63 (46.0)	59 (45.0)
Otherb	2 (1.5)	1 (0.8)
Prefer not to respond	2 (1.5)	0
Educational level
Never enrolled	1 (0.7)	2 (1.5)
Grades 1-6	5 (3.6)	5 (3.8)
Grades 7-9	11 (8.0)	6 (4.6)
Grades 10-12	41 (29.9)	29 (22.1)
College diploma or equivalent	22 (16.1)	22 (16.8)
Bachelor’s degree	38 (27.7)	51 (38.9)
Master’s degree	8 (5.8)	9 (6.9)
Doctorate	4 (2.9)	5 (3.8)
Other	4 (2.9)	2 (1.5)
Prefer not to respond	3 (2.2)	0
Comfort in reading health information
Very uncomfortable	6 (4.4)	10 (7.6)
Uncomfortable	5 (3.6)	11 (8.4)
Somewhat uncomfortable	12 (8.8)	5 (3.8)
Neutral	15 (10.9)	8 (6.1)
Somewhat comfortable	19 (13.9)	24 (18.3)
Comfortable	41 (29.9)	45 (34.4)
Very comfortable	39 (28.5)	28 (21.4)
Previously looked for COVID-19 guidelines
Yes	121 (88.3)	119 (90.8)
Internet search	86 (62.8)	74 (56.5)
Organization websites	78 (56.9)	73 (55.7)
Scientific journals	31 (22.6)	23 (17.6)
Social media	60 (43.8)	53 (40.5)
Prefer not to respond	5 (3.6)	3 (2.3)
Not applicable	0	0
Other	1 (0.7)	1 (0.8)
No	15 (10.9)	11 (8.4)
Unsure	1 (0.7)	1 (0.8)
Plan to look for COVID-19 guidelines
Yes	108 (78.8)	113 (86.3)
Internet search	87 (63.5)	88 (67.2)
Organization websites	95 (69.3)	86 (65.6)
Scientific journals	35 (25.5)	28 (21.4)
Social media	63 (46.0)	59 (45.0)
Don’t know where to look	5 (3.6)	1 (0.8)
Prefer not to respond	2 (1.5)	0
Other	2 (1.5)	1 (0.8)
No	19 (13.9)	5 (3.8)
Unsure	10 (7.3)	13 (9.9)

Abbreviations: PLR, plain language recommendation; SLV, standard language version.

^a Percentages do not add to 100% because of a multiple-select question.

^b Other includes Asian American (n = 1; SLV group), mixed Asian British (n = 1; PLR group), and White and South Asian (n = 1; PLR group).

Randomization to recommendation topic was skewed by many spam responses, resulting in participants not being allocated in an equal manner to 1 of the 2 COVID-19 recommendation topics, as predicted by the survey platform algorithm. Across participants, the mean time in which the survey was open varied from 1 minute to 143 hours (mode, 9.6 minutes). Moreover, 166 youths were flagged for speeding. After spam responses were excluded, 106 and 162 youths had been presented either topic. The interim analysis showed no significant interaction between the primary outcome (understanding) and the recommendation topics; no changes to the original sample size estimation and target inclusion numbers were necessary.

Quantitative Findings

No significant difference was found in understanding scores between the PLR and SLV groups; the mean (SD) proportion of correct answers was 50.9% (28.5%) in the PLR group compared with 45.7% (24.1%) in the SLV group (mean difference, 5.2%; 95% CI, −1.2% to 11.6%; P = .11). Analysis of subquestions showed a mean difference in the overall mean of 5.2% (95% CI, 0.5%-9.9%; P < .001) (Figure 2). Most participants (233 [86.9%]) were from North and South America, and region was not associated with understanding; regardless, we conducted a preplanned analysis stratified by region as a sensitivity analysis (mean difference, 5.3; 95% CI, −1.11 to 11.65; P = .10).

Figure 2. Risk Differences and Total Mean Difference for Correct Responses to 7 Questions Measuring Understanding.

Squares represent risk differences, and horizonal lines represent 95% CIs. PLR indicates plain language recommendation.

Participants in the PLR group found that the PLR was more accessible and usable (mean score difference, 0.34; 95% CI, 0.05-0.63; P = .02) (Table 2). Similarly, these participants were more satisfied with the presentation of the recommendation (mean score difference, 0.39; 95% CI, 0.06-0.73; P = .02). No significant difference was found between groups for the outcome of intended behavior (mean difference, 0.22; 95% CI, −0.20 to 0.74). Lastly, after reviewing both formats, participants had a preference toward the PLR (mean difference, 4.8; 95% CI, 4.5-5.1 [4.0 indicates a neutral response]).

Table 2. Results of Secondary Outcomes Comparing Mean Score Difference of PLR With SLV.

Secondary outcomes	Group score, mean (SD)a		Mean difference (95% CI)
	PLR (n = 137)	SLV (n = 131)
Accessibility and usability	5.12 (1.1)	4.78 (1.3)	0.34 (0.05 to 0.63)
Satisfaction	5.24 (1.3)	4.85 (1.5)	0.39 (0.06 to 0.73)
Intended behaviorb	5.24 (1.6)	5.02 (1.7)	0.22 (−0.20 to 0.74)

Abbreviations: PLR, plain language recommendation; SLV, standard language version.

^a Scores were 1 to 7 based on the number of correct responses.

^b Denominator for the SLV arm was 92 and for the PLR arm was 90.

Qualitative Findings

All 14 participants interviewed were from North and South America and had diverse educational backgrounds (high school students and postgraduate students); 11 (78.6%) were female. Overall, they reported a high degree of self-rated health literacy, ranging from 4 to 7 (neutral to very comfortable) on a 7-point scale of comfort in reading health information, and interviewees revealed unanimous agreement that the PLR was easier to understand. The thematic analysis yielded 4 themes: presentation, content, usability, and credibility. Table 3 summarizes the participants’ perceptions for each theme and suggestions for key improvements to make the PLR a more valuable tool for youths to consume and share reliable health information.

Table 3. Themes and Concepts Emerging From 14 Participants From the Qualitative Study.

Theme, concepta	Participant perceptionsb	Representative quotation
Presentation
Aesthetics	Almost all felt that the PLR was more aesthetically pleasing and friendly for the lay public relative to the SLV, commenting on the visual appeal of the PLR format	“[The PLR] is easier to read and it’s more appealing to the eyes. I like the color, and the font size is bigger.”
Formatting	All stated that the PLR format was easier to understand and navigate when reading health information and appreciated the PLR’s formatting and layout	“The colors [of the PLR] are beautiful, but the thing I like the most is that it is like in bullet form like I know what I’m going to read next and what information I’m going to get if I click there, so that’s the thing I liked the most.”
Length	The majority of participants commented on the length of the SLV format, which was overwhelming for many	“[The SLV] is just words and words and words, and I feel like I’m in like my research course and I’m reading a lot, but in the [PLR] there’s a lot of…white space which I like and…it’s just more clear and neat.”
Content
Understandability	All participants felt that the PLR format gave them a better understanding of the health recommendation	“For a lay person to understand [the PLR] is a lot easier, like there are simpler words; it doesn’t seem as complicated as if you go to the long document.”
Comprehensiveness	Almost all youths felt that the PLR provided a comprehensive overview of the topic	“I thought [the PLR] was much more concise and clear what they wanted to convey, and I think it was more tailored to the reader in terms of what’s important and meaningful to them.”
Complexity	When reading the SLV, the majority of participants had issues with the complexity of the language and information	“The sentences are hard to understand with a lot of scientific jargon, even for someone like myself, like I’m going into my third year of medical school.”
Usability
Navigation	When using the formats to locate key information, almost all participants noted that the PLR was easier to navigate; yet, many participants cited having issues with finding information in the SLV format, including a few who decided to use the “find” command to answer questions in the survey	“I’m looking for information that’s relevant to me and things I have questions about, but the [PLR] just makes it a lot easier.” “I’d have to read really into [the SLV], in order to find the recommendations, I just CTRL+F, and I mean I find recommendations, but I’d still have to read a lot, because the recommendation word comes up on a lot of pages.”
Efficiency	For the majority of participants, the efficiency of the PLR format made it more appealing to use	“I think [quickness] definitely is important because people don’t want to spend a lot of time necessarily searching for information when there is another option available where it’s presented really quick.”
Formatting	Several participants also noted various formatting and technological features of the PLR	“On the side [of the PLR], there’s the legend and then, when you press on it, then like you’ll automatically scroll down to the category that you want, and then the title of each category is big compared to the rest of the font underneath so it stands out more, and I can find it quickly.”
Credibility
Trustworthiness	Despite the strong preference of participants toward the PLR for consuming information, the majority of youths mentioned that the PLR lacked credibility, indicating that having the SLV would improve the overall credibility	“I just feel like the [PLR] just doesn’t look as credible, you know,…because it, it looks like anybody wrote it or like I wrote it.” “I would say [I would use the PLR], because I mean it seems like the [PLR], it’s very organized and it’s easier and faster to find the answers that you’re looking for, and if it is coupled with like references and like that source (SLV) I think I would probably use the [PLR].”
Sharing	Almost all participants suggested that the PLR format would be more appropriate to share with their friends, family, and the general youth audience but only if the information is trustworthy	“[I would share the PLR] just because it’s quicker and not everyone’s interested in knowing, like all the details, but again, like I would only feel comfortable sharing with my family and friends if I knew it was like legit, like if it was real.”
Decision-making	Half of the participants stated that if they had to choose one to make decisions about their health, they would rather use the SLV as it seemed more credible and detailed	“In that case, if I want to have like as much information as I can possibly have, I would use the [SLV], because I think it’s more reliable and it has like kind of more information, even though it’s like the same thing, but yes in that case I would use the [SLV], even though it’s kind of boring.”
Suggested improvements
Aesthetics	Several participants suggested aesthetic improvements for the format	“I think it’s good, I think the font size is a little bit tiny for me. Maybe not necessarily tiny, but it seems to be out of proportion because the subtitles are so large and then the actual blurbs are tinier. Maybe it’s also because the colors are different, like the headings’ colors, the background, that is a little bit different, so there is a little bit of contrast there, but it’s not a big deal.”
Accessibility	A few participants suggested to ensure that the PLR has accessibility features consistent with any relevant regional or national accessibility guidelines	“The only other thought that I would have is just from like an accessibility standpoint for both documents, just making sure that if they’re like online that they’re accessible for people with like screen readers and stuff like that so having those open boxes—like I’m not a professional in that like I don’t know how it works—but I just know some people who have used it, and so I think whenever you’re presenting information, it’s important to go through those accessibility features; make sure that they can be used online.”
Supplemental information needs	For the majority of participants, having the SLV as supplemental information would improve their overall experience of using the PLR	“For the [PLR], I think it would be helpful to include a link to the actual article (SLV), just in case people like say they look at the [PLR] format first, but then they think like, ‘Oh, I want to go to the original source to read about it myself even more, maybe there are extra details, I just, I’m curious about.’ Or if I’m doing a project or writing a research paper or something like that, um, maybe they would want sources and that they have to reference articles, so it would be helpful to put a source or link to the [SLV] that they can click on.”

Abbreviations: PLR, plain language recommendation; SLV, standard language version.

^a Presentation relates to how the information was presented, including aesthetics, formatting, and length. Content relates to the type of information presented, including understanding, comprehensiveness, and complexity of the information. Usability relates to the user experience of youths with the formats, including navigation, efficiency, and formatting. Credibility relates to the trustworthiness of the formats when making decisions or sharing health information with others.

^b Participant descriptions: a few, 1 to 3 (1%-25%); several, 4 to 6 (26%-49%); half, 7 (50%); the majority, 8 to 10 (51%-75%); almost all, 11 to 13 (76%-99%); and all, 14 (100%).

Discussion

In this RCT of youths aged 15 to 24 years worldwide, no significant difference was found in understanding scores between the PLR and SLV groups, but after reviewing both formats, participants had a preference toward the PLR. Combining the randomized design with qualitative interviews provided insight into the complexity of youth uptake of health recommendations. Their direct feedback on understanding, preference, usability, and impact highlighted the importance of credibility of information when making decisions about their health. Ideally, youths would prefer to have references and additional resources to enhance their experience with the PLR.

In the SLV group at baseline, the mean proportion of correct answers related to understanding was 45.7%, considerably lower than anticipated. This somewhat compromised our ability to observe the expected difference in understanding between groups but also suggests that there is room for improvement and that we made some progress in communication of recommendations. While the 5.2% improvement in understanding did not meet our a priori minimum important difference, we gathered important information from the qualitative interviews to further improve the PLR.

Although participants rated the PLR as more accessible and usable and satisfying, our results suggest that while knowledge is important, it may not be sufficient to influence behavior. This is consistent with research in advice response theory³²; further research is needed to explore how to increase implementation of recommendations.

Youth Partnership and Engagement

We used previously validated outcomes from similar trials^{20,21,22,24,25} to strengthen the credibility of our results and engaged youths in developing all study materials. Our study showed that successful collaboration and partnership with youths are possible and made vital contributions to the trial’s core elements. We enhanced interview reliability by conducting pilot test interviews and restating what participants shared to ensure that we comprehended their statements correctly. We concluded that partnering and engaging youths in the design of this study was key, especially since available evidence regarding guideline presentation to the general public was based on adult studies³³ except for 1 identified study in a review⁶ that could not be generalized in a meaningful way to adolescents without their input.

Limitations

This study has limitations. As our study was conducted in English and being able to comprehend English was part of the eligibility criteria, the generalizability of our study is limited. We are in the process of translating the PLRs into other languages; future research around knowledge and uptake of PLRs in other languages is needed. Additionally, youths around the world completed our survey in an uncontrolled environment, and we could not control how they used the intervention (eg, control the duration of how long participants read the recommendation or whether they read it while completing the survey).³⁴ Across participants, the mean time in which the survey was open varied from 1 minute to 143 hours (mode, 9.6 minutes). Moreover, 166 youths were flagged for speeding, which could indicate that the participants were not reading the recommendation in its entirety or were not answering the survey questions thoroughly. We determined a priori to remove these results from the final analysis. Another limitation of an online trial includes the inability to verify demographic data or the extent to which participants provide true responses, as described by Murray et al.³⁴ Although our intention was to target the general public, many participants were individuals with health and digital literacy with access to the internet and a device. As suggested by Murray et al,³⁴ we applied strategies to reduce the chances of these limitations (eg, user feedback on recruitment materials and data collection instruments, consent forms, and study process)³⁵; these strategies did not eliminate all challenges of this online RCT. Finally, as in previous trials,^20,24,25 3 of the secondary outcomes were assessed using more than 1 question. We did not evaluate the reliability of these questions among youths.

Conclusions

In this RCT comparing 2 formats of digital COVID-19 health recommendations among youths, no significant difference was found in understanding scores between the PLR and SLV groups. Participants ranked the PLR higher in terms of accessibility, usability, and satisfaction. Youths found the PLR format easier and faster to navigate, with content that was more accessible to understand, yet at the cost of some loss in credibility. Valuable information on how to further improve PLR formats for youths was garnered from the qualitative interviews. Further work could focus on how youths would increase uptake of recommendations and make decisions about their health using PLRs and on the uptake of PLRs for other health care topics among youths with a large burden of disease. Our findings may raise awareness among guideline organizations and national health policy makers and encourage them to develop PLRs alongside any new, emerging health guidelines for youths.

Supplement 1.

Trial Protocol

Supplement 2.

eTable. Characteristics of Randomized Participants Who Were Included and Excluded From the Analysis

eAppendix 1. Plain Language Recommendation/Standard Language Version for Recommendation 1

eAppendix 2. Plain Language Recommendation/Standard Language Version for Recommendation 2

eAppendix 3. Pretrial Youth Engagement Activities Related to Trial Design, Conduct, and Analysis

eAppendix 4. Youth Survey

eAppendix 5. Interview Guide

eFigure 1. Recruitment Flyer

eFigure 2. Morville’s “Honeycomb” Model

Supplement 3.

RecMap Members

Supplement 4.

Data Sharing Statement