Table 2:
Usability Testing Results
| Feedback Area | Feedback Themes | Change Made to e-ASTER Application |
|---|---|---|
| Value & Use of e-ASTER application | Participants (both children and parents) believe the application is useful and can help them better manage asthma symptoms. | N/A |
| Parent participants indicated if they may stop using the tool consistently (a) when their child’s asthma symptoms were well-controlled for a sustained period of time or (b) when there was an increase in their child’s asthma symptoms that might reflect poorly on their job as parents | No change was made but we did add positive feedback message when patients are doing well to reinforce regular use of the tool. To address parents concern we added in the tutorial that low scores do not reflect parenting skills. | |
| Parent participants indicated they wanted some way to document reasons for changes in asthma symptoms (e.g., specific triggers) on the tool, particularly when the severity of symptoms increases | A comment box was added to the application. Comments recorded by patients can be printed with the graph. | |
| Instruction Materials (Tutorial) | All participants struggled with how to advance from the tutorial/orientation material after completing the first ACT questionnaire. They suggested having some clearly labeled button like “Start” or that the tool automatically advance to the start screen once the tutorial ends. | A button saying “Start using the e-Asthma Tracker” was added. After clicking the submit button, the application takes the user to the questionnaire |
| Requested that instructions about how to print the graph be added to the tutorial material | Was completed | |
| Requested adding an explanation regarding mousing over highlighted or underlined words for pop-up messages with clarifications. | Was completed | |
| Parents mistakenly believed the “Personal Information” section on the profile page was for their information rather than for their child’s information as it was intended. They suggested the title of that section be changed to “Patient Information” or something else that might more clearly define its intent. | Changed “Personal Information” to “Patient Information” | |
| Pre-populated fields appeared active and participants were confused whether to leave or change them | Fields were grayed out and changed to read only format (cannot be edited). | |
| When a required field is missed, red lettering reminders are too small to see | Increased font size. | |
| Wanted to add a possibility for multiple providers or clinics alerts such as PCP and Pulmonologist | Feature added | |
| Did not know what ACT stood for on the menu stating “complete ACT”. Did not know that ACT was an acronym | Spelled out Asthma Control Test | |
| Half of participants indicated they were unsure how to select and unselect the checkbox regarding reception of alert reminders via email or text messages. They suggested having a “yes” box and a “no” box, so they could click one or the other rather than only unclick a box if they didn’t want such notices or reminders sent | No change was made but we added an explanation in the instructions. | |
| ACT questionnaire | Half of the participants (both parents and children) believed the final question on the ACT about asthma control was asking them to rate their actions or efforts (i.e., were they doing everything they needed to do) rather than to rate the level of asthma symptoms they were experiencing (i.e., were the symptoms in check) | The word “symptoms” was highlighted and a definition now appears with a mouse over |
| Half of the participants (parents and children) had difficulty understanding what was meant by “controller medication” and how it differed from the medication referenced elsewhere in the questionnaire | Added a drop down box, which identifies available controllers and allows patients to choose; also added a mouse over definition. | |
| Enable users to see previous responses even if they cannot change them | Instruction was added to the tutorial that if you click on a dot, previous responses can be seen but cannot be changed | |
| Want font to be larger | Increased font | |
| Graph | On the first round of the test, most participants had difficulty finding the button to print the graph and to adjust the timeframe to be printed. They suggested the button be moved to the menu list at the top of the screen where other functions/features are listed. | We moved the button to top of the screen and added a drop down arrow to select the time frame. |
| Wanted to be able to see their comments about reasons for changes in asthma symptom scores on the graph itself | Not feasible since it will complicate the graph but added additional pages that will print all comments with the graph. | |
| Label the x and y axes | Add labels to both axes. X-axis=date ACT was completed and Y-axis=Asthma Control Test score | |
| When you mouse over a plotted data point on the graph, the pop-up message related to the score shows up but they did not know that by clicking the dot on the graph, it will take them to previous responses | Information was added to the training materials. | |
| What is the meaning of the picture that appears with the pop-up message including sun, cloud, and cloud with lightning | Explanation was added to the tutorial | |
| Other Suggestions | Consider having a “chat room” so parents or children with asthma can communicate with each other | This will be a future enhancement |
| Three participants suggested a smart phone version of the application. | This is part of our future plan |