Table A2.
“Seha” app (versions 1.0.35 and 1.0.36): usability issues identified based on Nielsen’s heuristics.
| # | Heuristic Item | Evaluation Period | Usability Issue Description | Location | Rating | Redesign Recommendation |
|---|---|---|---|---|---|---|
| 1 | Visibility of system status | First (2020) | No identified | |||
| Second (2021) | The timer in the consultation room: unclear what it reflects. Does it reflect the user consultation time limit or the waiting time to see the physician? | During consultation | 3 | Add description of timer, i.e., waiting time to see physician or consultation duration. | ||
| It took a while to load the consultation page to start consultation with physician. | Waiting for physician | 1 | Add a message indicating “loading”. | |||
| 2 | Match between system and the real world | First (2020) | User may not understand the meaning of “artificial intelligence” feature named “Smart Seha” | Log in | 1 | Add a definition of “Smart Seha” for the user in lay terms such as an “electronic tool that helps you understand your symptoms and recommends some actions”. |
| Second (2021) | None identified | |||||
| 3 | User control and freedom | First (2020) | On the “consult physician” screen- when the user enters information, chooses “live session”, then chooses to cancel after seeing the waiting time, the app doesn’t go back to the previous screen “consult physician”, the app takes the user to the home screen. | Consultation initiation | 3 | Allow the app to take the user to the previous screen and not the home screen. |
| † If the user screen goes static, the app does not give a notification to the user that a physician is present in the session and the app automatically ends the consultation without the option of going back to the session. | Waiting for physician | 3 | Allow the app to send a notification with sound to alert the user when a physician is present in the session and reply to the user. | |||
| Second (2021) | The “back” icon in the consultation room takes the user to the home screen and not to the previous page (page where the user entered the consultation details). This happens without giving a notification where the back icon will take the user. | Consultation initiation | 3 | Change the icon of the icon to show a “home” icon rather than an arrow indication “back”—or program the app to go to the previous page instead of the home. | ||
| The “back” icon and “end consultation” icon have the same functionality. | End of consultation | 3 | Differentiate between both icons by creating pages that reflect the functionality of the standard icon. | |||
| 4 | Consistency and adherence to standards | First (2020) | None identified | |||
| Second (2021) | Both “Smart Seha” and “health check” are artificial intelligence functionalities and it is unclear what the differences between these services are. | Log in | 2 | Add a description for each functionality in the home page. | ||
| The term “health check” may also reflect a tele-consultation with a physician. | Log in | 3 | Add a description for each functionality in the home page. | |||
| 5 | Error prevention | First (2020) | † No notification indicating that if the user screen is static, the consultation will end and will be counted towards the user’s monthly consultation limit. | Waiting for physician | 4 | Create a notification for the user upon entering the chat room, which indicates that the consultation session will end if no response comes from the user. |
| Second (2021) | When the user clicks on consultation by mistake, the app does not send a confirmation message to the user to start the consultation. This then counts as a consultation limit if the user decides to leave without seeing the physician. | Consultationinitiation | 3 | Allow the app to count active sessions (interaction between the physician and user)—as part of the monthly consultation limit and provide a follow-up on the experience of the consultation. | ||
| 6 | Recognition rather than recall | First (2020) | † Waiting time is only displayed before entering the consultation session room reflecting the time the user gets access to the room. When the user is in the room, waiting time for the physician to start the session is not displayed. | Waiting for physician | 2 | Provide a countdown timer within the consultation session screen showing the estimated waiting time for the physician to join. |
| After the user leaves the open consultation session, the icon for reentering the consultation is not clear for the user. | During consultation | 3 | Add “open consultation” icon with visible instructions in every page. | |||
| Second (2021) | When the user goes out of the consultation room by mistake, the app does not show a notification that “ you are in consultation”. | During consultation | 3 | Show a notification to user “you are in consultation”. | ||
| 7 | Flexibility and efficiency of use | No accelerators or ability to tailor frequent actions based on inexperienced and experienced users were found. | ||||
| 8 | Esthetic and minimalist design | First (2020) | Irrelative and unclear icons shown at the end after the “consultations page” to view the history of consultations indicating “closed”. This icon is “action required”. | End of consultation | 1 | Remove the “action required” icon from the closed consultations. |
| There are two icons that lead to the same function “starting the telemedicine consultation”. One accessed in the home screen “consultations” and the other in consultations “new”, which brings the user back to the home screen. | Consultation initiation | 1 | Remove the “new” tab from the consultations screen. Main dashboard might provide a summary of the features offered on a high-level. | |||
| Second (2021) | None identified | |||||
| 9 | Recognition diagnosis, and recovery from errors | First (2020) | None identified | |||
| Second (2021) | None identified | |||||
| 10 | Help and documentation | First (2020) | Quick start guide is only displayed to the users when the app is opened for the first time. | Log in | 3 | Provide users with ongoing access to help through an icon or tab placed in the chat room and/or in the home screen as user instructions. |
| Second (2021) | The app does not provide clear directions on how to use the app, and what each icon or label means. | Orientation | 4 | Provide any extra information that would be useful to users, along with the label. | ||
| Total Issues identified from the two evaluations: 18 | ||||||
† The issue was resolved—no longer identified in our second evaluation.