A Mobile Phone Wayfinding System for Visually Impaired Users

In past work [1] we described a novel system for aiding blind and visually impaired persons in wayfinding – the process of finding one’s way to a destination. This system consists of a mobile phone held by the user that detects and reads specially designed printed signs (Fig. 1) using its built-in camera. Experiments have demonstrated the ability of blind and visually impaired persons using the system to locate these signs (which are read aloud using synthetic speech) at distances of up to several meters [2].

Fig. 1.

System concept. Bottom: system detects wayfinding sign and provides appropriate directional guidance. Top: Building floor plan shows placement of multiple signs, each identifying a unique location.

We have augmented our system with new functionality that provides directional guidance to a desired destination, rather than simply reading aloud every sign that is visible. Each sign identifies a unique location in the building, and thus the system is able to use these signs as a form of “indoor GPS” to determine the user’s current location. Whenever a sign is detected the system refers to a building map (previously loaded into memory) and provides audio directions (e.g. “turn left”) to guide the user to any desired destination from his/her current location (see Fig. 1). The ID of a sign, which identifies its spatial location, is encoded by the colors it contains (the four colors in the sign can be combined to form 24 different spatial arrangements). The particular arrangement of colors in a sign is deciphered by the mobile phone during detection. Special provisions have been added with respect to the original algorithm of [1] to improve detection robustness via shape analysis. If a larger set of ID’s is needed, then a bar code can be added to the sign, as in [1].

Finally, we conducted experiments showing that visually impaired subjects are able to use the system to find locations on one floor of an office building.