User-Centered Development of a Web-Based Preschool Vision Screening Tool

Abstract

Although amblyopia is most successfully treated when detected in early childhood, many preschool-aged children are not being screened. This project explored the delivery of Web-based vision screenings, integrated with patient education, to parents and children, aged 3 to 6 years. Through a user-centered design methodology involving requirements gathering, iterative prototype development, and usability testing, a highly usable screening Website was created. Interviewing and testing parents and children in the home were essential in gathering accurate data about environments where the tool would actually be used. Frequent iterations of designing, testing, and modifying the tool were useful in identifying and correcting usability problems. Usability goals were set early in the project, and in the final phase a satisfaction questionnaire was administered to participants. Twenty-one out of 22 final usability objectives were achieved and the feasibility of Web-based vision screening was demonstrated.

INTRODUCTION

Amblyopia is “a nonspecific loss of visual acuity of at least two lines of difference that is not caused by pathology nor correctable by ordinary refractive means.”¹ Approximately 2.5% of the world’s population is affected with amblyopia, which is more than 7 million people in the United States alone. ^1,2 Amblyopia has also been reported to be the leading cause of irreversible monocular vision loss in adults between the ages of 20 and 70 years, and before the age of 45 years, it causes more vision loss than all other ocular disease and trauma combined.^1,3

Chances for improvement or full recovery are increased when the condition is detected and treated before about 9 years of age when vision is mature. Vision screening of preschool children has been advocated as a means for early detection. However, in the United States, an effective mass screening program has not yet been achieved and it is estimated that fewer than 25% of preschool-aged children receive vision screenings either from a government or private program.⁴ In addition, up to 60% of primary care providers do not perform vision screening on preschool-aged children, and others perform screenings inconsistently.⁴

The purpose of this study was to create a highly usable Web-based vision screening tool for parents to screen their preschool-aged children (3 to 6 years) for amblyopia, to learn more about this condition, and to find ophthalmologists in their area. Providing screening in the home was the focus of this research but the tool could easily be adapted for preschools or other settings. This site (www.lazyeyetest.org) was envisioned as a free, public service, with no ties to a particular clinic or advertisements for any products or services.

METHODS

It was important to maintain a focus on the needs of the end users in building an easy to use screening tool. In traditional software design actual users are often involved infrequently or too late in product development to make a difference. Consequently, user interface issues and usability testing are often only superficially addressed. A contrasting approach, user-centered design, is guided by three distinguishing principles: (1) An early focus on users and tasks, (2) empirical measurement of product usage, and (3) iterative design whereby a product is designed, modified, and tested repeatedly.⁵ Direct contact between users and designers occurs throughout the development lifecycle and emphasis is placed on measuring ease of use and ease of learning through the iterative development and testing of prototypes.⁶

The main phases of the project were completed as follows:

1. User needs analysis/requirements specification

2. Low-fidelity prototype development/exploratory testing

3. Website development/assessment testing

4. Final usability verification testing

Institutional Review Board approval was obtained and in each phase approximately 6 parents and 6 children participated in interviews and usability testing. Most participants were newly recruited in each phase. Within each phase, 3 children were included with known amblyopia and 3 with normal vision or unknown vision status. This technique utilizing a small number of participants, sometimes called “discount” usability testing, has been shown to be effective in detecting 85% of usability problems in a design.⁷

The main goals of the user needs analysis/requirements specification phase were to determine the characteristics of parents and children who would be using the site and to create a list of technical specifications. Meetings were arranged with parents and children in the home and data were collected through a background questionnaire, semi-structured interviewing, and an analysis of the home-computing environment. Variables such as monitor size and resolution, Internet connection speed, room lighting conditions, and distance in front of computer screen were measured to determine the baseline technological requirements for the site. Measurement of visual acuity was selected as the screening method for this study and the Amblyopia Treatment Study visual acuity testing protocol was used as a model for adaptation.⁸ Usability goals were constructed to be measured in the final phase with a user satisfaction questionnaire. Examples include: (1) the home page is convincing that preschool vision screening is essential, (2) the instructions for setting up the test are easy, and (3) the test results are clear and helpful. During this phase the decision was also made to link to an existing physician locator site.⁹

In the second phase a low-fidelity prototype of the site was created using PowerPoint. Low-fidelity prototyping is a technique used early in development to produce simple, cheap, and quick mock-ups that help support the exploration of alternative ideas and designs.¹⁰ We met with parents once again in the home, walked through the prototype screen by screen, and recorded their feedback.

The low-fidelity prototype phase provided a strong foundation for the actual development of the site. Although the main portion was built in HTML, Macromedia Flash was used to implement the actual vision screening because of its excellent interactivity and ability to provide high-quality animation requiring minimal network bandwidth. ColdFusion was used for data transfer to an MS Access database. Usability testing was accomplished by observing parents and children while they used the site and parents were asked to “think aloud” to reveal their decision making process as they accomplished tasks. Data were collected on problems observed as participants used the site, their comments during testing, and their answers to debriefing questions.

The final phase of the project involved correcting usability issues from the previous phase, observing parents and children using the improved site, and then administering a Likert-scale satisfaction questionnaire with topics phrased in the style of the Questionnaire for User Interaction Satisfaction (QUIS).¹¹

RESULTS

User needs analysis/requirements specification

The parents who participated in the user needs analysis were well-educated, generally female and aged 30–35 years and spent time weekly using a computer and the Internet. Although parents reported that more than 70% of the children had attention spans longer than 11 min, most believed that if the test took more than 5 min children would not be able to maintain attention. A common theme was that parents would need to pick “a good time of day” to test younger children in order to ensure the test could be completed. Another concern was that some younger children might have trouble recognizing the 4 letters used in the test (H, O, T, V), and that it would be important to provide them with a matching guide. Parents strongly believed that a unifying theme, animation, and sound would be important to sustain a child’s attention. They also emphasized the need for streamlined instructions that would give a concise overview of the setup process and time required.

Based on data from analysis of the home computing environment, the site would target users with slow dialup Internet connections and the Microsoft Internet Explorer browser. An 800x600 screen resolution and a 15-inch screen size would need to be accommodated. The Macromedia Flash Player was a common plug-in and sound capability seemed to be a standard feature. In terms of the distance in front of computer screens, all participants were able to accommodate at least 5 ft, but not all had 10 ft, the ideal distance determined. Consistent and appropriate lighting was a potential problem and would need to be addressed in the setup instructions.

Low-fidelity prototype development/exploratory testing

The main finding from exploratory testing of the low-fidelity prototype was that too much information was presented on the home page and in the test setup, and that more pictures and diagrams were needed to illustrate concepts. Six main sections of the site were planned for implementation: (1) the home page (see Figure 1), (2) the vision test setup, (3) the vision screening in Flash, (4) a physician locator introductory page, (5) the amblyopia Q&A page, (6) the “about us” page, and (7) the privacy policy page.

Figure 1.

Lazyeyetest.org home page.

Website development/assessment testing

In the vision test setup section the parent would read an overview of the setup process and then a disclaimer. After entering brief background information about the child, the parent was instructed to obtain a tape measure and a dollar bill for the test image calibration step. The parent would then measure and enter the test distance, adjust the room lighting appropriately, calibrate the size of test images by holding up a dollar bill to a line on the screen and marking a point with the mouse, and then print out a matching card for the child if necessary. Using the logMAR scaling method¹² and a trigonometric formula, appropriately sized test images were dynamically created in the vision test based on the dollar bill calibration and test distance.

After completing the test setup the Flash portion of the site would open in a new window and contained (1) test instructions, (2) the actual test, and (3) the test results. After parents and children viewed the instructions the child would view single letters with both eyes open, with the left eye covered, and finally with the right eye covered (Figure 2). During the actual test, parents would click on a “yes” or “no” button depending on the child’s response to the letter shown (Figure 3). Short animations with encouraging audio messages were included to help motivate the child. After completing the test, a results page displayed the child’s score for each eye and a recommendation. If a difference of two scoring levels was detected between the eyes the recommendation would alert the parent to seek the help of an eye care professional.

Figure 2.

Instruction screen before right eye is tested.

Figure 3.

Instruction screen with test image example.

Usability testing of the first live version of the site revealed that the home page could be more effective in convincing parents that screening was essential. Contrary to our original assumption, setting up the test took less time (5 min) than administering it (generally 10 min). Parents needed additional help in understanding how precisely to measure the test distance and in entering fractions as decimals into the input box. In the actual vision screening, children had a tendency to cover the eye with the fingers of the hand rather than the palm, as was instructed. Some 3- and 4-year-olds had difficulty maintaining attention and they invariably removed the hand covering their eye when they became fatigued. Also, during testing parents sometimes ran back and forth from the child to the computer to help the child cover the eye or hold a letter matching card. Having a second person to assist would allow the test to run more smoothly. Despite these usability issues, the vision screening test correctly identified the children with amblyopia.

Final usability verification testing

After an attempt was made to correct uncovered usability issues the final “verification” usability testing was conducted. After using the site, 7 parents completed a satisfaction questionnaire covering 22 objectives based on the original usability goals and areas of the site where usability issues had been previously identified. On a scale of 1–5 (5 being best), a mean score of 4 was considered success. Twenty-one out of the 22 objectives were achieved. The measurement of satisfaction with the amount of time required to take the test was the only objective that did not meet expectations with a mean score of 2.9 (Table 1). In addition, not every child in the age range of 3 to 6 years was able to complete the test on the first try, as was hoped.

Table 1.

Final usability questionnaire results.

Topic	Mean score (out of 5)
1. Layout of information on the home page	4.4
2. Home page’s effort to convince me it is important to screen my child for amblyopia	4.0
3. Downloading the Flash 6 plug -in	4.0
4. Entering background information about my child	5.0
5. Instructions for setting test distance	4.7
6. Instructions for adjusting room lighting	4.6
7. Understanding how to calibrate screen using dollar bill	4.7
8. Instructions to print and use the matching card	4.0
9. Overall impression of vision test setup	4.9
10. Instructions at the beginning of the vision test (when the new window opens)	5.0
11. Child’s impression of cartoons during vision test	4.9
12. Instructions explaining how to cover child’s eye	4.9
13. Length of time requiring child’s attention for vision test	2.9
14. Information on test results page	4.1
15. Instructions (on the LazyeyeTest.org site) for using the physician finder site	4.6
16. “Amblyopia Q&A” questions and answers content	4.6
17. Language of “Amblyopia Q&A” questions and answers	5.0
18. Links to other amblyopia resources on Web	5.0
19. Navigation through pages of the site	4.9
20. Layout, pictures, and graphics on site	4.9
21. The relative worth of this site to you and your child	5.0
22. Overall experience using site	4.6

Among the remaining usability issues after the final round of testing, the screening was still long for some younger children (3–4 years) and the method of using the hand to occlude the eye still needed improvement. Preliminary results indicated however, that having the child use a dollar bill as an occlusion tool may be more reliable and this method is shown in Figure 2. More than one parent suggested that the test should use pictures rather than letters so that it would be less intimidating and younger children would be better able to recognize the test images. There may be benefits to a picture-based test, and this is a future direction to investigate.

DISCUSSION

The user needs analysis was successful in separating the critical from the less important issues early in the project. Actually visiting parents and children in the home was essential to the success of the project and various distractions in the home environment provided a realistic testing experience that would have been missed in a more sterile usability lab.

This project demonstrated how trimming away nonessential elements is a critical task in interface design but that it can be difficult to find the appropriate balance between too much and too little information. Also, it was surprising that in the last 3 rounds of testing, 6 sets of users seemed adequate, if not too many. Generally by the time the 5^th user was tested no major new usability issues were identified. In retrospect, it may have been more effective to use the same total number of participants but double the number of cycles of testing and redesign, using half as many participants in each cycle.

Another observation was that showing instructions with pictures or diagrams could be much more effective than with words. There is definitely an art to providing content on the Web that can be easily scanned and understood by a wide range of users.

Additionally, the test required more time than seemed reasonable for some younger children with shorter attention spans but perhaps it was unrealistic to expect all 3-year olds to be able to complete it on the first try with the test protocol implemented. Further refinement of the protocol will allow younger children to complete the test successfully. Also, finding an effective eye occlusion method for a consumer-oriented vision test was a continual challenge.

Despite the challenges encountered, this project successfully demonstrated that the World Wide Web can be used to provide computerized vision screenings to relatively primitive personal computers over slow Internet connections. Macromedia Flash was a good method for providing the actual vision screening test and for integrating multimedia content. A streamlined or discount approach to user-centered design and usability testing was effective, particularly in this project where resources and number of study participants were limited.

LIMITATIONS

A larger sample size would be needed to generalize the results to a broader population and establish external validity. However, for detecting major usability issues, the sample size was adequate and followed established techniques. Also, the reliability and validity of the user profile questionnaire, verbal questions, and final usability satisfaction questionnaire were not formally assessed. Lastly, bias may have been introduced by combining the roles of designer and usability test moderator.^6,13 Remaining neutral and avoiding leading questions while moderating usability testing was challenging.

FUTURE DIRECTIONS

Although the results of the last two rounds of usability testing indicate that the test was potentially effective in screening for amblyopia, it was not formally validated against a gold standard ophthalmologic exam. The vision screening should be validated to ensure its accuracy. Second, the screening algorithm should be further simplified to make the test faster but still effective. Third, it would be wise to integrate the test setup into the Flash presentation and make the entire presentation more game-like and interesting for both parents and children. Fourth, different options for picture-based test images should be explored to help improve the ability of younger children to complete the test. Lastly, various eye occlusion methods should be explored and tested to improve on the current method.

Another possibility may be to market the tool in the preschool or grade school classroom. This would help reach a greater number of children than home screenings alone. In addition, physicians and organizations that provide vision screenings may be interested in using this tool instead of a traditional paper-based vision chart.

CONCLUSION

This project was successful in adapting established user-centered design techniques to create a high-quality, free-of-charge, online vision screening test and amblyopia education resource. The needs of parents and children with respect to the tool were identified, effective prototypes were created and tested with actual users, and an easy to use Website that a wide range of parents and children could use was developed.

It was demonstrated that the World Wide Web could be successfully used to provide computerized vision screenings and a patient education resource, but the unpredictability of young children (3 years) provided unique challenges. This project also demonstrated that a streamlined or discount approach to user-centered design and usability testing could be an effective development approach when resources and study participants were limited. Interviewing and testing users in the actual environment where the tool would be used and small, frequent rounds of usability testing were important aspects of this approach. Through the utilization of Web-based technology a cost-effective solution was found that may provide easy to use vision screenings and patient education to a much larger population.