. 2022 Oct 30;8:33. doi: 10.21037/mhealth-22-11

Table 2. HLO strategies and corresponding author recommendations.

HLO strategy	Author/year	Recommendation
Section 1 [2015]: What we know about users with limited literacy skills
1.1 Reading & cognitive processing challenges	Ben-Zeev [2013]	Apps and technological systems must be usable by people with low literacy levels and cognitive impairments
	Chaudry [2013]	Recommend using literacy tests other than REALM to reduce discomfort when speaking aloud
	Connelly [2016]	Differences in health literacy better identified with Newest Vital Sign than Short Assessment of Health Literacy with usability best tested in situ
	Siedner [2015]	Coded messages, which obviate the need for literacy, were as effective as direct messages and might augment privacy
1.2 Understanding navigation	Coughlin [2017]	Varying levels of eHealth literacy will be addressed by using simple navigation features and providing straightforward instructions about how to use the app and connect it to commercially available products
1.2 Understanding navigation	Wildenbos [2019]	Using feedback messages in interfaces should not only inform users on (the result of) their actions but should also offer the user options to recover from wrong actions and return to previously retrieved information or actions
1.3 Using search	N/A	NA
1.4 Mobile considerations	Ben-Zeev [2013]	Deploying existing mHealth resources intended for the general population may prove problematic
	Miller [2017]	Design apps for those with low health literacy and low computer literacy: use a simple interface displaying only one question per screen with large response buttons, similar to what would be found at an automated teller machine or self-checkout kiosk
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
Section 1 [2010]: Learn about your users & their goals
1.1 ID your users. Who are they?	Mueller [2020]	Co-design of images with intended population
	Siedner [2015]	In person confirmation of mobile phone competency was highly predictive and should be considered for future similar interventions
	Siedner [2015]	End-user characteristics, particularly literacy and technology experience are important predictors of an mHealth intervention for PLWH in rural Uganda.
1.2 Understanding their motivations. Why are they here?	Bahadori [2020]	Consider specific needs of target population
1.2 Understanding their motivations. Why are they here?	Wildenbos [2019]	Advise to put more emphasis on addressing motivational barriers of older adults within user interface design and guidelines
1.3 Understanding their goals. What are they trying to do?	N/A	NA
Section 2: Write actionable content
2.1 Identify user motivations & goals. Why are they here?	Ceasar [2019]	Use focus groups as a collaborative tool to inform app development
2.1 Identify user motivations & goals. Why are they here?	Wildenbos [2019]	Advise to put more emphasis on addressing motivational barriers of older adults within user interface design and guidelines
2.2 Put the most important information first	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
2.2 Put the most important information first	Schnall [2015]	Present information with simplicity
2.3 Describe the health behavior – just the basics	Fontil [2016]	In addition to simplifying overall language, we simplified explanations of scientific concepts, preserving core concepts while improving understandability
2.3 Describe the health behavior – just the basics	Schnall [2015]	Present information with simplicity
2.4 Stay positive. Include the benefits of taking action	Povey [2020]	Engagement via humor, music, vibrant colors, relatable images, and stories about positive change
	Povey [2016]	Graphics and animation perceived as supporting motivation
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
2.5 Provide specific action steps	Dunn-Lopez [2020]	Essential elements in providing health literate content at a 6th grade reading level include plain language, short sentences, brief paragraphs, bulleted or numbered lists, and actionable content
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
	Coughlin [2017]	Varying levels of eHealth literacy will be addressed by using simple navigation features and providing straightforward instructions about how to use the app and connect it to commercially available products
2.6 Write in plain language	Abujarad [2018]	Desired reading level was at 8th grade level
	Bahadori [2020]	Target a Gunning Fog Index and Flesch Kincaid Grade Leve of 6 and Flesch Reading Ease Score of 70
	Boyd [2015]	Write text at a sixth-grade reading level and provide narration as an additional method for individuals with low literacy to understand text-based information
	Dunn-Lopez [2020]	Essential elements in providing health literate content at a 6th grade reading level include plain language, short sentences, brief paragraphs, bulleted or numbered lists, and actionable content
	Fontil [2016]	To address concerns about the complexity of the curriculum, we adapted the readability level of each lesson (originally 9th grade or higher) to mostly a 5th-grade level or below
	Miller [2017]	Use simple language and include audio narration to assist those with literacy barriers
	Poduval [2018]	Consideration of literacy levels and audio/visual media for usability
	Poduval [2018]	Text written for people with a reading age of 12, all essential information was provided in video as well as text format
	Schnall [2015]	Present information with simplicity
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
2.7 Check content for accuracy	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
Section 3: Display content clearly on the page
3.1 Limit paragraph size. Use bullets & short lists	Bahadori [2020]	Decrease number/length of sentences
	Dunn-Lopez [2020]	Essential elements in providing health literate content at a 6th grade reading level include plain language, short sentences, brief paragraphs, bulleted or numbered lists, and actionable content
	Srinivas [2019]	Specific to reducing burden while capturing a user response, we suggest designing a system that uses simple-worded, direct questions with fewer words that are easier to read and quicker for the participant to understand and has simple response options that are easier to read, quicker for the participant to understand and select from
3.2 Use meaningful headings	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
3.3 Use a readable font that’s at least 16 pixels	Wildenbos [2019]	User-interface design elements such as font size and buttons should be adjusted to the older adult user population
3.3 Use a readable font that’s at least 16 pixels	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
3.4 Use white space & avoid clutter	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
3.5 Keep the most important information above the fold – even on mobile	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
3.6 Use links effectively	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
3.7 Use color or underline to ID links	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
3.8 Use images that help people learn	Bender [2016]	Use visuals with simple text and culturally tailored themes and imaging
	Boyd [2015]	Include audio and images as supplemental information for people below a sixth-grade reading level
	Connelly [2016]	Interface has larger pictures with short labels and could be read aloud
	Gibbons [2014]	Use symbols that have been found to be common across culture
	Giunti [2018]	Personas were created to represent persons with MS at different eHealth and health literacy levels
	Mueller [2020]	Pictogram sets can be switched out for newly designed pictograms that are contextualized and localized to other study areas, countries or topics
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
	Povey [2020]	Audio and intuitive visuals encouraged for lower literacy
	Povey [2016]	Culturally relevant graphics, voices, animation, and optional short video clips may assist in engagement with content and overcome literacy issues
3.9 Use appropriate contrast	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
3.10 Make web content printer friendly	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
3.11 Make your site accessible to people with disabilities	Abujarad [2018]	Text-to speech translation is a key feature of VIC and is achieved by online and automated text-to-speech translation
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
	Wildenbos [2018]	Encourage designers, programmers and developers should be to create mHealth interventions with inclusive design, or flexible flexible enough to be usable by people with no limitations as well as by people with functional limitations related to disabilities or old age
	Wildenbos [2018]	The MOLD-US framework can aid mHealth designers in inclusive design efforts. The visual overview of MOLD-US enables a quick assessment of aging barriers and medical conditions that involve deteriorating capacity
3.12 Make websites responsive	NA	NA
3.13 Design mobile content to meet mobile user’s needs	NA	NA
Section 4: Organize content & simplify navigation
4.1 Create simple & engaging homepage	Casey [2014]	Reduction, or simplifying a task to influence behavior, was evident by the reports that the app was easy to use, required basic numerical literacy, and was highly visible on the home screen
4.1 Create simple & engaging homepage	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
4.2 Label & organize content with your users in mind	Casey [2014]	Reduction, or simplifying a task to influence behavior, was evident by the reports that the app was easy to use, required basic numerical literacy, and was highly visible on the home screen
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
	Wildenbos [2019]	Cognitive load should be minimized, i.e., by a clear navigational structure and aligning an interface with expectations of older adults
4.3 Create linear information paths	Casey [2014]	Reduction, or simplifying a task to influence behavior, was evident by the reports that the app was easy to use, required basic numerical literacy, and was highly visible on the home screen
4.3 Create linear information paths	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
4.4 Give buttons meaningful labels	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
4.5 Make clickable elements recognizable	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
4.6 make sure the browser “Back” button works.	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
4.7 Provide easy access to home & menu pages	Casey [2014]	Reduction, or simplifying a task to influence behavior, was evident by the reports that the app was easy to use, required basic numerical literacy, and was highly visible on the home screen
4.7 Provide easy access to home & menu pages	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
4.8 Give users options to browse	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
4.9 Include a simple search	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
4.10 Display search results clearly	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
Section 5: Engage Users
5.1 Share information through multimedia	Abujarad [2018]	Text-to-speech interfaces addresses literacy issues and makes the IC process an option for inexperienced computer users
	Dev [2019]	Increasing graphics, audio, and video were recommended to overcome literacy barriers
	Huang [2015]	The voice-to-text bilingual function will be used to assist the patients with low health literacy
	Lord [2016]	Use speech-to-text functionality to help individuals with low literacy
	Mackert [2017]	Need app design to be engaging and interactive, from adding videos and games inside the application, to personalizing the experience to changing font size and color
	Miller [2017]	Use simple language and include audio narration to assist those with literacy barriers
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
	Muscat [2020]	Supplement written content with audiovisual formats
	Poduval [2018]	Consideration of literacy levels and audio/visual media for usability
	Poduval [2018]	Text written for people with a reading age of 12, all essential information was provided in video as well as text format
	Povey [2020]	Engagement via humor, music, vibrant colors, relatable images, and stories about positive change
	Povey [2020]	Audio and intuitive visuals encouraged for lower literacy
	Sox [2010]	Alternative text explanations and audio files to support lower health literacy
5.2 Design intuitive interactive graphics & tools	Dev [2019]	Increasing graphics, audio, and video were recommended to overcome literacy barriers
	Mackert [2017]	Need app design to be engaging and interactive, from adding videos and games inside the application, to personalizing the experience to changing font size and color
	Ownby [2012]	Present numeric dosing data in a graphic calendar format
	Srinivas [2019]	Specific to reducing burden while capturing a user response, we suggest designing a system that uses simple-worded, direct questions with fewer words that are easier to read and quicker for the participant to understand and has simple response options that are easier to read, quicker for the participant to understand and select from
5.3 Provide tailored information	Ceasar [2019]	Increase relatability with local information
	Mackert [2017]	Encouraged dynamic personalization allowing users to input personal data
	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
	Poduval [2018]	Personal stories included
	Povey [2020]	Sustain use with customization, interactive activities, and challenges/records of progress over time and options for sharing.
5.4 Create user-friendly forms & quizzes	Miller [2017]	Design apps for those with low health literacy and low computer literacy: use a simple interface displaying only one question per screen with large response buttons, similar to what would be found at an automated teller machine or self-checkout kiosk
	Muscat [2020]	Calculate readability statistics
	Muscat [2020]	Incorporate micro-learning and interactive quizzes
	Srinivas [2019]	Specific to reducing burden while capturing a user response, we suggest designing a system that uses simple-worded, direct questions with fewer words that are easier to read and quicker for the participant to understand and has simple response options that are easier to read, quicker for the participant to understand and select from
5.5 Consider social media sharing options	Monkman [2013]	Although the majority of the recommendations from the HLO guide for Web sites were applicable for assessing mobile usability, the heuristics generated in this study may benefit from being complemented with other evidence-based heuristics specific to mobile devices
Section 6: Test your site with users with limited literacy skills
6.1 Recruit users with limited literacy skills – and limited health literacy skills	Bahadori [2020]	Involve patients in app development and user acceptance testing
	Ben-Zeev [2013]	Deploying existing mHealth resources intended for the general population may prove problematic
	Ceasar [2019]	Use focus groups as a collaborative tool to inform app development
	Connelly [2016]	Mobile app focus groups to explore app design
	Gibbons [2014]	Include a target population with low health literacy during usability evaluation
	Siedner [2015]	Thorough assessments of end-user written literacy and technology experience should be made before and during implementation design
	Wildenbos [2019]	Advise to involve older populations as co-creators in the requirements analysis and design phases when developing mHealth
6.2 ID & eliminate logistical barriers to participation	Chaudry [2013]	Recommend using literacy tests other than REALM to reduce discomfort when speaking aloud
6.2 ID & eliminate logistical barriers to participation	Wildenbos [2019]	Usability evaluation approaches may need adjustments to prevent reporter bias and become better suited for testing mHealth services with the older adult and chronically ill patient populations
6.3 Create plain language testing materials	Connelly [2016]	Differences in health literacy better identified with Newest Vital Sign than Short Assessment of Health Literacy with usability best tested in situ
6.4 Test whether your content is understandable and actionable	Bahadori [2020]	Monitor patient experience to see if readability needs to be improved
	Ben-Zeev [2013]	Deploying existing mHealth resources intended for the general population may prove problematic
	Ceasar [2019]	Use focus groups as a collaborative tool to inform app development
	Ceasar [2019]	Check-ins or IT support to address technical difficulties
	Connelly [2016]	Iterative, user-centered design process with focus groups was essential for designing the app rather than merely replacing words with icons and/or audio
	Connelly [2016]	Provide a case study of design of an ecological momentary assessment mobile app for a low-literacy population
	Muscat [2020]	Calculate readability statistics
	Muscat [2020]	Apply the Patient Education Materials Assessment Tool
	Siedner [2015]	Coded messages can have similar efficacy as text messages, while maintaining confidentiality
6.5 Use moderators who have experience with users with limited literacy skills	Fontil [2016]	Creating technical assistance tools for various stages of the program to address lower technology literacy
6.6 Pretest your moderator’s guide	NA	NA
6.7 Use multiple strategies to make sure participants understand what you want them to do	NA	NA
6.8 Test on mobile	Ceasar [2019]	Check-ins or IT support to address technical difficulties

HLO, Health Literacy Online; REALM, Rapid Estimate of Adult Literacy in Medicine; NA, not applicable; PLWH, persons living with HIV; IC, informed consent; IT, information technology.