Smartphones-Based Assistive Technology: Accessibility Features and Apps for People with Visual Impairment, and its Usage, Challenges, and Usability Testing

Abstract

Smartphones are less likely to be considered as assistive technology for visual impairment among a large majority of health care providers, excluding vision rehabilitation professionals, and the general public who are not familiar with accessible features and apps. The present review aims to highlight accessible features and apps along with usages, including educational, and access to smartphones as assistive technology for visual impairment and blindness. It also includes advantages and challenges faced by users, and usability testing by app developers. There have been significant recent developments in mobile technology that incorporate computer technology relating to electronic information, communication, and touch-screen accessibility. Such advances in technology are transforming the use of smartphones from a traditional visual interface to a truly visual free interaction using alternative body senses, such as haptic, gesture, and so on. There are many built-in accessible features and third-party accessible applications that enable people with visual impairment to perform daily activities, independent functioning, movement, social inclusion and participation, education, etc. They are universally designed, so they are unlikely to induce social stigma or negative reactions from peers or public. Healthcare practitioners, not limiting to eye care, and caregivers, family members, teachers, or special educators should be informed about the potential uses and benefits of smartphones for visually impaired in developing nations. Evidence shows that most of the users train by themselves. Enhancing the awareness along with training for teachers and caregivers would be helpful to improve access and skills among users with visual disabilities. Developers are continuously producing more innovative applications for visual impairment, which indicates the need for having a training guideline on the use of smartphones.

Introduction

Globally, there are 43 million blind and 295 million people with moderate-to-severe visual impairment.¹ The great majority of them live in low- and middle-income countries (LMICs).¹ Visual impairment has an impact on the quality of life and wellbeing of individuals. Limitations in daily functioning, independence, indoor and outdoor movement, social inclusion, communication and employment extend beyond the affected individual, and can have repercussions for the family and the community.² In addition, the current COVID-19 pandemic and its preventive measures pose a new set of challenges for people with visual impairment in performing daily living tasks by restricting their usual routine support and supplies.³

People with visual impairment need to live independently and cope with daily challenges faced at home, workplace, school, and market. Assistive technology enables people with visual impairment to achieve their optimal potential while executing a wide range of daily living activities, thereby helping to improve healthy and productive living in the society.⁴ The Article 25 of the UN Convention on the Rights of Person with Disabilities reiterates that persons with visual disabilities should not be ignored or left behind in achieving the highest possible healthcare and quality of life.⁵ Similarly, at the heart of Sustainable Development Goal, the World Health Organization recognizes the inclusion of visual disabilities under its pledge “leave no one behind”.⁶ Research, including literature reviews accompanied by dissemination using various platforms on assistive technology would be helpful to not only healthcare professionals or practitioners but also to rehabilitation professionals, general public, special educators, and end users.

In the recent past, smartphones have become one of the most advanced assistive technologies of the early twenty-first century. The continued advance in digital assistive technologies, containing electronic information and communication technology provides new opportunities to overcome many of the challenges encountered by people with visual disabilities.^7–9 Smartphones are one of them which is continuously evolving over time with many integrated computer technologies.¹⁰ They provide many accessible built-in features and operate accessible applications for persons with different types of disabilities.

In LMICs like India, the usefulness of various benefits of smartphones needs to be informed to the entire gamut of healthcare practitioners, caregivers, family members, teachers and special educators, not confined to eye care providers, thereby promoting smartphones use among the population with low vision and blindness. Therefore, the aim of this narrative review is to provide a comprehensive overview of smartphone features specifically designed for people with visual impairment, such as accessible software technologies (built-in and third-party apps), their usage, and advantages, along with potential challenges among users and practitioners. Finally, the review also highlights usability testing and evaluation of accessible applications that need to be emphasised while designing special apps by the developers.

Search Methodology

A comprehensive electronic search of all relevant technical and non-technical materials was performed without limiting any timeframes, such as published scientific articles both biomedical and computer interface science or smartphones technology related, reports on digital assistive technology for visual impairment, practice reports, editorials, commentaries, smartphone ergonomics, was carried out using various digital libraries, namely, PubMed (US National Library of Medicine), Web of Science Core Collection, Google Scholars, ResearchGate, website of various organizations working in assistive technology for visual impairment, etc. We also included case studies, book chapters, observational studies, population-based survey in the online literature search and searched on Google Play Store for accessible applications on Android and iOS smartphones. The searches were limited to the English language only.

Medical Subject Heading terms, Boolean Operators, Truncation symbols, wildcard symbols were used during the search. The keywords for search were “smartphones”, “mobile”, “accessible apps*”, “accessible features”, “assistive devices”, “assistive technology”, “use”, “usage”, “usability”, “access”, “impaired vision”, “vision impairment”, “visual disabilities”, “visual loss”, “vision loss”, “blindness”, “low vision”, “functional low vision”, “legal blindness”, “vision rehabilitation”, “advantages”, “challenges”, “gaps”, “adoption”, “usability testing”, “usability evaluation”. To filter out the papers or literature, we screened title and abstract of articles or literature for inclusion in the narrative review. Features and apps that are not specifically designed for visual impairment were excluded, such as Skype, Facebook and WhatsApp. We also performed free text and phrase searching using smartphones accessible apps for “calls”, “messaging”, “shopping”, “financial management”, “reminder”, “calendar”, “reading”, “website browsing”, “voice recorders”, that can potentially be performed with applications.

Results

We retrieved 2053 articles. All duplicates were removed, reducing to 560 articles, including non-technical materials. Articles (453) that were out of scope or not related to special accessible features and apps for visual impairment of smartphones excluded after screening the title and abstracts. The full text of the articles was screened and excluded studies reported inconclusive results in terms of special apps uses (77). A total of 30 articles were deemed appropriate for further review and included in the study (Figure 1: PRISMA flow chart indicating the methodology of selecting articles).

Figure 1.

PRISMA flowchart.

Note: Adapted from: Liberati A, Altman D, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of Clinical Epidemiology. 2009;62(10)e1–e34. Creative Commons.⁵⁹

Discussion

Smartphone Technology

Advances in research on “human-computer-interaction” have driven the development of innovative assistive technologies with new interface designs that makes them more accessible and user-friendly for people with visual impairment^11,12 For instance, the technology of mainstream assistive devices, such as mobile phones and tablets, has evolved substantially over the past two decades from a simple basic phone, such as NOKIA 8110, to the high-end touch-screen smartphones or tablets using iOS and Android operating systems. The use of touch screens shifts the need for visual function to alternative body senses, and through gestures, haptic or audio, they facilitate a truly eyes-free human-smartphone interaction.^13–15 These changes substantially improve the accessibility of smartphones to individuals with visual impairment. Smartphones have large touchscreens and can access mobile internet data, Wi-Fi and Bluetooth, enabling access to digital platforms. The touch-screen accessibility with audio or tactile feedback is a benefit to visually impaired individuals and helps them overcome many daily living challenges.

Smartphone technologies can help a person with visual impairment understand the immediate surroundings and access a large amount of information for independent functioning, movement, social inclusion, participation, educational activities and news.¹⁶ As smartphones have now become part of our everyday lives, they are gradually replacing traditional assistive devices such as Braille, optical and electronic magnifiers for the performance of a significant number of routine daily living activities.¹⁷ Today, the market is dominated by Android (85%) and Apple iOS (15%), both of which have accessible built-in screen reading technology for visual impairment and host many third party accessible applications.^18,19

Since operating smartphones more or less relies on reasonably good visual function, many assume that such a technology is not particularly accessible to individuals with visual impairment.^20–22 While a few visual rehabilitation centres already consider smartphones as assistive technologies, there is a sense of limitation among the public, including target users, general practitioners, caregivers and the mainstream eye care professionals on the use of smartphones by visually impaired people, particularly in LMICs.^6–9,23 There are limited studies available on the integration of smartphones and tablets into low vision and rehabilitation services in LMICs.

Accessible Built-in Features for Visual Impairment

There are a number of built-in accessibility features in smartphones that enable a person with vision impairment to interact with the contents in the smartphone.²⁴ Table 1 represents some of the most used built-in accessible features.^13–17,25 They not only provide support to visually impaired users to use smartphone devices but also operate other applications.

Table 1.

Commonly Used Accessible Built-in Features of Smartphone

Features	Operating System	Operating Mode	Descriptions
Voice assistant (Google assistant, Siri)	iOS/Android	Audio based	Ask a question. Tell it to do things
TalkBack	Android	Audio based	Screen reader
VoiceOver	iOS	Audio based	Screen reader
Text to speech/Voice recognition	iOS/Android	Audio based	Read aloud
Select to speak	iOS/Android	Audio based	Speak selection
Zoom magnification/Font size	iOS/Android	Visual based	Zoom in the display
Contrast	iOS/Android	Visual based	Differences between an object and its background
Invert colours	iOS	Visual based	White becomes black, black becomes white, orange becomes blue
Voice Inputs Keyboards	iOS/Android	Audio based	Typing through voice command

The built-in accessibility features most widely used by people with visual impairment are screen readers: TalkBack for Android and Voice Over for Apple iOS (Figure 2).²³ The Talkback feature allows the user to easily identify the content or application icons on the smartphone’s screen. By simply placing a finger on the icons, the smartphone will read aloud the icon or application name. Similarly, Voice Over provides voice feedback on the contents appearing on the screen of an iOS smartphone, so it can be used without the need for visual function.²⁷ An individual with vision impairment can slide the finger on the screen until the desired icon is located.

Figure 2.

Accessible built-in Features for visual impairment (a-Android; b-iOS mobile).

A person with low vision can use the zoom magnification feature to magnify the entire phone screen to the required level. “Invert colours” allows to invert the display colours for users who cannot read the normal interface. A study on the use of iPhone accessibility features showed that the Zoom Magnification and Large Text were the most commonly used features among people with visual impairment (low vision).²⁴ The study also reported that many participants were interested in using other accessibility features if they were trained to do so.

Third-Party Accessible Applications for Visual Impairment

New advances in mobile technology allow people to operate smartphones using “eyes-free” third-party accessible applications. Table 2 presents some of the accessible apps for people with visual disabilities.^13–17,25 The majority of these accessible apps are freely available online, and visually impaired persons can install them to customise the smartphone functions (Figure 3). In addition to built-in features, these applications enable the smartphones to serve as assistive technology for visually impaired people and facilitate daily independent living activities, engagement in educational and societal activities.

Table 2.

Commonly Used Accessible Apps for Visual Impairment

Apps	Operating System	Operating Mode	Descriptions
Kibo	Android	Audio based	Image reading, (pdf, e-book, doc, reader Hindi and English), voice recorder
Be My Eyes	Android, iOS	Audio based	Help by sighted person (volunteer) through video calling
Supersense	Android, iOS	Audio based	Information about surrounding, Artificial intelligence (AI)
Visor	Android, iOS	Video based	Magnifier (near objects)
Binoculars	Android, iOS	Video based	Distance viewing (super zoom camera)
Mani	Android, iOS	Audio based	Mobile aided note identifier
BlindSquare	iOS	Audio based	Navigation, search places, etc.
Khabri	Android	Audio based	Audio news, current affairs, jobs, horoscope, stories, and promotional podcasts (Hindi)
Phonepe or Google Pay	Android, iOS	Audio based	Easy and reliable way to make payments online, E-Transaction
AccessNote	iOS	Audio based	Note-taking
KNFB Reader	Android, iOS	Audio based	Text-to-speech, text-to-Braille, and text highlighting tools
Seeing Eye GPS	iOS	Audio based	Fully accessible turn-by-turn GPS iPhone app with all navigation features
Audible	Android, iOS	Audio based	Provides audio books
TapTapSee	Android, iOS	Audio based	Photograph any two or three-dimensional object at any angle and speak the identification back to the user (Note: Spoken identification requires VoiceOver/talkback to be turned on).
Optelec	iOS	Video based	Reading text and viewing photographs using the simple magnification and high contrast functions.
Color picker	Android, iOS	Audio based	App to identify color from camera or image with many extra features.
Color Grab	Android, iOS	Audio based	Point your phone at anything and get its exact color, including the hexadecimal color value. Contains a huge database of color names!
Dolphin EasyReader	Android, iOS	Audio based/video based	Browse & download from the World’s largest collection of talking book and newspaper libraries.

Figure 3.

Third party accessible apps for visual impairment (left to right: Kibo, Mani, Supersense, Be My Eyes, Visor).

Usages Among People with Visual Impairment

The accessible features and applications of smartphones are being used for a wide range of daily living tasks that were previously executed using traditional assistive devices, such as magnifiers or Braille. These smartphone-based digital accessible technologies can be used for object and obstacle identification, for sighted help, GPS navigation, currency identification and social networking and entertainment.²⁸ They also play a role in communication, emailing, reading e-Books and news or listening, writing and typing, calendar functions, recording memos, colour identification and talking calculator.

There is a paucity of studies on the use of smartphones by individuals with visual impairment in LMICs. A recent study in Ghana showed that the majority of respondents with visual impairment and blindness either had no phone or used a basic “not-smart” phone. It was also reported that 90% of those who had smartphones were not aware of the various functionalities incorporated in a smartphone.²⁹ A study from Brazil including 28 people with low vision reported the use of 50 apps and 9 accessibility resources in smartphones and tablets. The apps’ functionalities included access to textual content and everyday tasks related to work, aesthetics and navigation.³⁰

In an online exploratory study conducted in high-income countries on smartphone usage among 466 people with visual impairment, over 90% of respondents reported that they used their mobile for calls, sending and receiving messages, browsing the web and reading emails; approximately 80% of them used them for calendar functions, listening to music and social media and networking; some 70% for reminders and to take photos for reading with optical character recognition.¹⁷ Eighty per cent of participants used a smartphone for outdoor GPS navigation. The study also highlighted that smartphones are replacing many traditional assistive devices in performing multiple daily living tasks, though the latter are still in use for specific purposes.¹⁷

A similar online survey reported that 81% of visually impaired respondents from the United Kingdom, the United States of America and Portugal used smartphone apps for various routine activities including calls, messages, internet, use of the camera to see things better, audiobooks and e-books.²⁰

Further, a global online survey of 259 participants with visual impairment in the United States found that more than 95% of participants used the apps to accomplish their daily activities.³¹ The most frequently used apps were BeMyEyes, ColorID, CamFind, followed by screen readers and writing apps such as TalkBack, KNFB Reader and BrailleTouch. In many cases, the smartphone apps could function for multipurpose tasks that facilitated independent living.

Several studies report that participating in the digital arena and accessing digital technology, including smartphones, reduces the feeling of isolation, improves social contact, information sharing and promotes better interaction with friends and family.^32–34 A study on tele-rehabilitation for visual impairment reported that smartphones offer a safe and efficient way of providing reliable information about the COVID-19 pandemic, including the various preventive strategies in place during the emergency lockdown. Smartphones also helped in psychological counselling for fear, facilitating and addressing many unseen challenges faced by visually disabled people during the lockdown period.³⁵

Smartphones Use for Academic Activities

Generally, students with visual impairment are more likely to be excluded and ignored from mainstream schooling, especially in LMICs. The literacy rate among children with disabilities is very poor in developing nations, as compared to their able-bodied peers. The World Bank estimates that children with any disability are five times more likely to be ignored from education than children without disability.³⁶ For example, in India nearly 80% of children with visual disabilities are illiterate, which is highest among all the disabilities.³⁶ Studies and reports are available that smartphones are used successfully for educational purposes by students with visual disabilities from LMICs, such as Nepal, Pakistan, and India, where screen reader eg TalkBack is being most frequently used.^37–39 The study in Nepal also reported that most students learned to use screen readers by themselves or from peers. Access to smartphones among students is not a major issue, but lack of training on the use of accessible features and apps is a challenge for them as reported by Prakash et al.³⁷ To the best of our knowledge, there has been no prevalence study available to date on the usage of smartphones for educational purposes among visually challenged students in developing nations. Smartphones would be a great help to students with visual impairment for inclusion in mainstream schools. Therefore, it is necessary to assess the current usage of smartphones and digital technology.

Advantages and Access to Assistive Technology for Visual Impairment

Conventionally, patients with low vision and blindness receive optical and non-optical vision rehabilitation services using various traditional assistive devices, such as magnifiers, Braille, audiobooks, video magnifiers, and closed-circuit television (Figure 4). Although the benefits of traditional devices are widely documented in the literature, their adoption and consistent use amongst visually impaired individuals is impeded by various factors such as discomfort or difficulty to use and carry, cost, social stigma, lack of training support and skills in caregivers, and finally lack of availability or supplies.^{10,37,40–42}

Figure 4.

Assistive technology for visual impairment (First row: traditional products: (A) Braille slate and stylus, (B) optical magnifiers, (C) video magnifiers; second row: smartphones accessible apps: (D) Kibo, (E) Visor, (F) operating TalkBack.

The World Health Organization reiterates lack of affordability as one of the key reasons for not acquiring assistive products in LMIC.⁴³ The Closed-Circuit Tele Vision or electronic video magnifiers are not affordable, especially due to associated high costs, whereas the cost of a smartphone is relatively lower in comparison.⁴⁴ Therefore, smartphone technology provides a cost-effective assistive technology platform to people with visual impairment due to its low cost. For instance, a study reported that the cost of smartphones has significantly reduced over the last few years.⁴⁵ Indeed, around 44% (760 million people) are estimated to be using a smartphone in India in 2021 which is expected to increase up to 51% by 2025.⁴⁶ However, there is a paucity of population-based data on access to smartphones among people with visual impairments, particularly in LMIC. Nevertheless, the WHO estimates that one in ten people who need assistive technology, irrespective of types of disabilities, are able to access assistive devices. This indicates that access to smartphones among people with visual disabilities is very poor in developing nations.

In addition, the abandonment and disuse of traditional devices ranges from 30% to 75% among individuals who used it.⁴⁷ In contrast, the mainstream smartphones are widely adopted by the general population; therefore, they are less likely to draw public attention when used as an assistive device and carry less social stigma compared to traditional assistive devices.^17,48 Smartphones are available at any time and any place, are universally designed and are easy to carry.²⁵ Smartphones can be connected with internet WiFi data for various purposes, which helps in accessing online information, and accomplishing many activities from home, reducing needless travel.

Gaps and Challenges in Applications

Many eye care professionals may not view smartphones as assistive technology for visual impairment, due to lack of awareness and understanding of the beneficial aspects of smart technologies.²² To date, there have been no or limited studies formally exploring the awareness of accessible apps and features among eye care professionals, general practitioners and caregivers. Similarly, a lack of awareness on how smartphones can be used for daily independent living activities among people with visual impairment is a major barrier to access.

At the same time, the number of accessible apps and features for people with visual impairment continues to grow at an unprecedented rate.⁴⁹ People with visual impairment need to be kept abreast of advancing accessible applications to obtain benefits from them. Designers are developing more innovative and readily accessible user-friendly mobile technology along with accessible apps. These lead to increased complexity of choice since each app has a special function that may also overlap with other apps.⁴⁹ There is not a single app that fits all purposes. Therefore, professionals recommending smartphones as assistive technologies will need to have access to the requirements and functions of different apps in order to tailor the training program to meet the specific needs of the person with visual impairment.

Training on how to use smart devices is critically important for a productive and consistent use of smartphones. Every user needs to receive training on the use of screen readers first and further expanded training for other accessible apps. This is particularly important in students and teachers of schools for the blind as indicated by Ari and Inan.⁵⁰ A Texas study revealed that a large proportion of teachers who teach students with disabilities have poor skills and knowledge of assistive technologies.⁵¹ The emergence of smartphones in today’s world would help in developing the confidence and training motivation for both teachers and students as pointed out by Prakash et al.³⁷ Zhou et al illustrated that the extent of use of assistive technologies in schools is significantly determined by the level of training received by teachers and teachers’ familiarity with assistive devices.^51,52 Therefore, a formal training for special educator or teachers would be helpful in improving access and smartphones usage among students with visual impairment. The same view was indicated in other literature.⁵³ Prakash et al study also highlighted that organizing training program would help to uncloak the opportunities provided by current smartphone technology.³⁷

Training a person with visual impairment to use assistive technology usually requires didactic sessions and considerable time. Other factors that could influence training are age, educational level, type of disabilities (low vision and absolute blind) and degree of the problem. There is a need to develop clinical training protocols for caregivers, family members, and eye care professionals to guide them when recommending smartphones as accessible technologies and training people with visual impairment to use them. Another challenge faced by people with visual impairment is that these specially designed features and apps are not available in their local language.⁵⁴ The accent and pronunciation of audio output from screen reader, and apps is hard to understand by a disabled person.³⁷

Given the continuous growing of new mobile technology alongside the decreasing average lifetime of a particular smartphone, it is likely that people with visual impairment will have to update their smartphones like the general public do. This could cause a challenge to end users who are visually impaired. Further, a technical challenge for smartphones is that the battery consumption is more while using several apps at the same time. Good-quality batteries that can operate for extended periods of time, especially for smartphones used by people with visual impairment will be essential. Additional due attention should be given to data security and back-up solutions in the event of loss or theft of smartphones.

Usability Testing for Smartphones

Developers or evaluators might have difficulty identifying the usability problems that people with visual disabilities experience. People with visual impairment have unique requirements for interacting with and using smartphones comfortably and efficiently. There must be confluence between the ideas of developers and the needs of the target users. Apps and features should be tested by users with visual impairment and evaluated accordingly. Such usability testing will be essential for the product to be accepted and adopted by the end users. Developers need to take a user-centered approach to produce accessible apps for visual impairment that meet the users’ expectations and needs.⁵⁵

A common method of usability testing is the Think-Aloud Protocol. In a Concurrent Think Aloud (CTA), the experts request the participants to verbalise their thoughts as they perform the tasks with an assistive device. Experts also use Retrospective Think Aloud (RTA), where participants are asked to retrace their steps after they complete the tasks.^56,57 Identifying the causes of usability problems helps differentiate whether the problems are due to individual’s disability or an imperfect interface design for the products.⁵⁸ It is important that every mobile assistive product, both hardware and software, should undergo such evaluations to improve adoption and acceptance. There is a crucial need for collaboration between vision rehabilitation professionals, computer scientists and people with visual impairments in designing innovative, acceptable, and adaptable mobile assistive technology.

Conclusion

There is a misconception, especially among non-eye care professionals and general public that smartphones are less likely to be inaccessible to people with visual impairment as they largely rely on more or less good visual function. However, in recent times, mobile technology has incorporated advanced computer and information technology features that use sound, haptic and gestures to interact with smartphones, substituting visual interaction, whereby making accessible all digital contents of smartphones for people with visual disabilities. Such innovative technology has provided a new platform for people living with visual impairment to overcome challenges encountered in their everyday lives. There is a need to disseminate the potential applications and use of smartphones assistive technology among visually impaired community, at the same time, efforts to increase the awareness other than vision rehabilitation professionals, to general healthcare practitioners, physicians, special educators, family members and even general public. Due to the widespread availability of “eyes-free” smartphone features and applications, there is a corresponding need for developing training guidelines for professionals, even teachers and caregivers, which will enhance smartphones use among visually challenged individuals.

In addition, the present manuscript provides information on the gaps, challenges in application and accessing to smartphones assistive technology. Such information will help the operational researcher to design a study focusing on how to improve the access to smartphones in the community. Considering the paucity of data on the use of smartphones assistive technology, eye care professionals and public health professionals can plan a prevalence study on the use of smartphones apps among visually impaired population along with barriers in terms of their adaptation. The article shows the need for having evidence-based guidelines on the training on the use of smartphones accessible features and apps under the principle of “from the developers to the practitioners or special trainers”.

Funding Statement

There is no funding to report.

Disclosure

None of the authors has any conflicts of interest related to this work to disclose.