Screen Magnification for Office Applications

Abstract

People with low vision use screen magnifiers to interact with computers. They usually need to zoom and pan with the screen magnifier using predefined keyboard and mouse actions. When using office productivity applications (e.g., word processors and spreadsheet applications), the spatially distributed arrangement of UI elements makes interaction a challenging proposition for low vision users, as they can only view a fragment of the screen at any moment. They expend significant chunks of time panning back-and-forth between application ribbons containing various commands (e.g., formatting, design, review, references, etc.) and the main edit area containing user content. In this demo, we will demonstrate MagPro, an interface augmentation to office productivity tools, that not only reduces the interaction effort of low-vision screen-magnifier users by bringing the application commands as close as possible to the users’ current focus in the edit area, but also lets them easily explore these commands using simple mouse actions. Moreover, MagPro automatically synchronizes the magnifier viewport with the keyboard cursor, so that users can always see what they are typing, without having to manually adjust the magnifier focus every time the keyboard cursor goes of screen during text entry.

1. INTRODUCTION

Office productivity tools, such as Microsoft Word, Excel, Power-Point, etc., are indispensable tools of modern society. To interact with these applications, people with low vision typically rely on screen-magnifier assistive technology [3, 6, 8] that enables them to enlarge original screen content, and also pan the magnified content (i.e., move the magnifier focus) using special keyboard shortcuts and mouse actions. However, given that only a portion of the screen is visible at any instant with a screen magnifier, interaction with productivity applications can be very challenging, since UI components in these applications are spread out over the entire screen; e.g., in Word, text area is in the middle, upper ribbon and lower ribbon feature menus are at the top, and so on. Therefore, users have to pan back-and-forth between these elements repeatedly to complete their tasks. Usability issues faced by low-vision screen-magnifier users with productivity tools remain an understudied research topic. Even the few works on this topic have focused on either understanding the general interaction experience of low-vision users [4, 9], or have mainly targeted web browsing [1, 2] and smartphone applications [5, 7].

In this demo, we will demonstrate MagPro that augments the interface of productivity application so as to facilitate almost instant access to the application features in the ribbons with little-to-no panning effort. The fundamental idea underlying MagPro is to push all the application commands within close proximity to the user’s present magnifier focus in the edit area, preferably within the user’s current viewport (see Figure 1). This eliminates the need to switch context and search for these features by panning and zooming. MagPro augments the application interface with an additional Instant Menu. The Instant Menu contains all the application features arranged in the form of a scrollable list. Furthermore, the Instant Menu is displayed within close proximity to the current user focus in the edit area, and is also compact, thereby it not only offers the user an instant visual feedback, but also lets them focus only on a small fragment of the screen to view all the commands, which in turn reduces their panning, zooming, and even their eye-movements. Furthermore, MagPro automatically adjusts the magnifier viewport (by moving the mouse cursor) in response to the movement of the keyboard cursor, so that the user can always see what they are typing, without having to readjust their magnifier focus constantly while typing.

Figure 1:

A low-vision user accessing the desired command with a screen magnifier and MagPro. The user focuses on just one fragment of the screen and scrolls over the ribbon commands in MagPro’s Instant Menu.

2. MAGPRO DEMO SCENARIO

Consider a scenario where a screen-magnifier user is editing a document in Microsoft Word, and wants to add a WordArt style to a certain text. In order to do this task with a screen magnifier, the user has to first select that text and then pan the magnifier lens from the current view showing the text to the top of the screen containing ribbons. Then in the Insert ribbon, the user pans to find the WordArt command and select one of the options in its drop-down grid menu. The user then pans back to the selected text in the main edit area to check the resulting effect. If the resulting effect is not satisfactory, the user repeats this process all over again, once for each option of the WordArt command. This repeated panning back-and-forth between the text content and the ribbons makes the interaction experience tedious and cumbersome. MagPro mitigates this excessive panning by presenting the ribbon commands close to the user’s current context (i.e., selected text), as explained next.

For doing the same task with MagPro, instead of panning to the ribbons, the user can simply execute a middle click on the mouse wheel to bring up an Instant Menu that is displayed right next to the selected text (see Figure 1). The Instant Menu houses all the ribbons, their commands, and command options as scrollable lists (see Figure 2). By default, the Home ribbon commands are shown in the Instant Menu when it is initially brought up with a middle click mouse action. As shown in Figure 2a, for each of the commands, both command label and its corresponding icon are provided side by side so that the user can easily identify them. To navigate this list of commands, the user simply needs to scroll with the mouse wheel, and the commands will be brought into the viewport without the user having to pan the magnifier lens. To select a command, the user simply needs to execute a left mouse click. If a command has options (e.g., options for the Line and Paragraph Spacing command shown in Figure 2b), the option list will be shown as an overlay on top of Instant Menu so as to avoid panning and refocusing. To access other ribbons (e.g., the Insert ribbon that contains the desired WordArt command), the user can right-click on the ribbon tab to see the list of ribbons (see Figure 2c) and then select the desired ribbon using a left click, which causes MagPro to refresh the contents of the Instant Menu with the commands of the selected ribbon. The user can then scroll over to the WordArt command and left click to select it, which will refresh the Instant Menu with WordArt options. As these options will be shown right next to the selected text, the user can instantly observe the effects of choosing any option, and then settle for the one that best matches the user’s needs.

Figure 2:

Instant Menu design; (a) list of commands for the default Home ribbon, (b) command options for Line and Paragraph Spacing command overlaying Instant Menu, and (c) list of ribbons overlaying Instant Menu.

In addition to Instant Menu, MagPro also keeps track of the user’s current location in the edit area, and continuously updates the location of the magnifier lens such that the users can always view what they are typing, without having to manually adjust the magnifier lens each time the keyboard cursor goes of screen while editing content. To achieve this, MagPro leverages the fact that the magnifier viewport is tied to the mouse pointer, and therefore makes the mouse pointer automatically follow the keyboard cursor during text entry.

3. CONCLUSION

Interaction with productivity tools using screen magnifiers is tedious and stressful for people with low vision. MagPro is an easy-to-use application-interface augmentation that can significantly reduce the panning and zooming effort while not only accessing commands, but also during text entry in the productivity applications. User studies validating the benefits of MagPro are the scope of future work.

CCS CONCEPTS.

• Human-centered computing → Human computer interaction (HCI); Accessibility technologies; User studies.