The usability of two mobile health assistive technologies for wheelchair-related in-seat movement and pressure

Abstract

Introduction

This study aimed to understand the degree to which two different mobile health assistive technologies, AW-Shift© and Sensoria® Mat, addressed seven constructs for managing wheelchair-related in-seat movement and pressure.

Methods

After using each intervention system, participants answered questions regarding the general usability and usefulness of the systems.

Results

System Usability Survey scores ranged from 5 (Poor) to 97.5 (Excellent), with a median response of 60.0 (Okay) for AW-Shift© and 76.3 (Good) for Sensoria® Mat. Participants reported using AW-Shift© to check areas of high pressure on their cushion, the quality of their weight shifts, and their posture significantly more often than to check the condition of their cushion or to track their movement goals. Participants reported using Sensoria® Mat to check the quality and number of weight shifts, and their posture significantly more often than to check the condition of their cushion.

Conclusions

The findings of this study highlight that there is no one-size-fits-all solution and that different subpopulations of wheelchair users may have different needs and preferences. Optimizing the design for specific cohorts or constructs can result in an effective product that consistently provides meaningful and accurate information about behavior and performance.

Background

In order to ensure better acceptance and avoid abandonment of assistive technology, it is crucial to involve input from and evaluation by the intended users throughout the design and development process.^1,2 The technology alsso needs to be assessed for effectiveness in addressing the intended constructs. ³ Furthermore, if the intended purpose of the technology is to change users’ health-related behaviors, successful implementation and effectiveness depends on how well its use is grounded in behavior change theory. ⁴

Assistive technologies to help wheelchair users manage pressure on their sitting surface have emerged and evolved over time.^5,6 Existing mobile health (mHealth) self-management tools for wheelchair users typically focus on providing the user with pressure injury education and resources. VA PUR is a tool published by the VA that includes an extensive list of evidence based self-management strategies, educational content, links to resources (such as doctors and clinics) as well as the ability to journal and set reminders. ⁷ iMHere 2.0 is described as an “innovative self-care reminder and mobile health delivery system” that is in development at the University of Pittsburgh. It provides reminders, tracking and communication. ⁸ However, after reviewing these and other similar approaches, these approaches do not respond objectively to the individual’s behavior. ⁹

More recently, personal-use in-seat movement tracking and pressure management mHealth systems have emerged that include instrumentation added to the chair or cushion with communication to a smartphone for self-monitoring of seating behaviors.^10–14 Technologies often focus on users with a spinal cord injury (SCI) because the combination of mobility and sensory impairments puts them at especially high risk for pressure injuries.^15,16 However, all wheelchair users have some risk due to impaired mobility and periods of prolonged sitting. ¹⁷ The intent of in-seat tracking and pressure management systems has been to help reduce pressure injury incidence through features such as reminders to perform weight shifts or alerts to areas of high pressure. Some such systems, such as sensomative _wheelchair (sensomative GmbH, Rothenburg, Switzerland) and MisterGaspard (Captiv, Nantes, France), are systems currently commercially available outside the United States for use with the user’s existing wheelchair cushion. ¹⁸ It is well-established that pressure at the sitting interface contributes to skin and tissue damage and there are evidence-based strategies for mitigating prolonged pressure. ¹⁹

For prevention of pressure injuries related to wheelchair use, assistive technologies commonly address some or all of the following pressure management constructs: relieving pressure (weight shifts), setting and tracking goals for weight shift performance, monitoring posture, checking the condition of the seat cushion, and monitoring the seat interface for areas of high pressure. Weight shifts can be further described by their frequency, duration, and magnitude. ¹⁵ A number of studies have highlighted the difficulty in sustaining attention to pressure all day, every day, resulting in poor adherence to clinical guidelines.^20,21 More recently, guidelines have emphasized the need for individualizing pressure management strategies based on varying needs and personal risk factors. ²²

Wheelchair users across diagnoses (SCI, spina bifida, cerebral palsy, etc.) are at high risk of pressure injury development, ¹⁷ but have varied mobility and sensory impairments, which results in a wide range of seating and mobility equipment and pressure management needs. This diversity in needs requires careful attention to which features address the various pressure management constructs. In other words, do the features have the intended effect, do the users want to use them, and for which users are the different features most useful or usable? One challenge in assessing the usefulness and usability of innovative technologies is that evaluating them in the intended environment and contexts is costly and complex, particularly when they have not yet been fully developed or refined. This problem can result in technologies being commercialized before they have been fully evaluated, leading to poor adoption or abandonment. The Diffusion of Innovation theory (DOI) suggests that the following aspects of innovation be considered: pros and cons of its features, adopter characteristics, and the complexity of learning and using the system. ²³

Drawing from best practices in assistive technology evaluation, the objective of this study was to assess the usability of two distinct mobile health assistive technology systems designed to manage wheelchair-related in-seat movement and pressure. The study aimed to examine the extent to which these systems addressed seven specific constructs and to evaluate users’ perceived usability of each construct. Secondary objectives were to identify which subpopulations of wheelchair users identified most strongly with each construct and to assess subjective usability of the various features within subpopulations. The findings from this study can guide clinicians and wheelchair users in selecting the most appropriate assistive technologies for preventing pressure injuries, as well as help guide the design of other similar technologies.

Methods

Participant information

A convenience sample of full-time wheelchair users who were currently using or were determined to qualify for a skin protection (E2603/2604 and E2622/E2623 or a combination skin protection and positioning cushion (E2607/E2608 and E2624/E2625) ²⁴ were recruited. Participants were required to own and operate a smartphone with an Apple or Android operating system and to be able to perform weight shifts independently, either by leaning or using a power tilt.

In addition to basic demographic information, such as age, gender, race, and ethnicity, participants were asked about their diagnosis, level of injury (if SCI), level of sensation on their sitting surface, and whether they had volitional movement below their level of injury.

Mobile health assistive technologies

The study considered two systems: AW-Shift© and Sensoria® Mat (Sensoria Health Inc., Redmond, WA, USA) (Table 1). Participants were scheduled to use one or both systems for up to 4 weeks each. Participants were encouraged, but not required, to use systems daily. Participants were allowed to end or pause use of the system at any time, for any reason. All participants who successfully set up and reasonably had access to the system (i.e., equipment and software were functional for at least a day following setup) were included in this analysis. The first system used was assigned on an alternating basis to ensure as close to a 50/50 split. A 2-week washout period with no device use was included after the participants used their first system to mitigate any carry over effects.

Table 1.

Characteristics of the two systems used by participants on their wheelchairs in this study.

Hardware features	AW-Shift©	Sensoria® Mat
Primary feature	Stream live pressure map	“Wheelchair wellness Coach” a
Commercially available	No	Yes
Pressure sensing regions	256 sensors, 1” × 1”	4 sensors, 1” × 1”
Location of pressure sensors	In cover, top of the seat cushion	In fabric, under seat cushion
Interface hardware	Box with a USB charging port	“Core” removed to charge
Software features	AW-Shift©	Sensoria® Mat
Reminders to perform weight shifts	X	X
Customizable weight shift settings	X	X
Customizable weight shift goals	X b	X
Track and review weight shift performance	X	X
Automatically detect weight shifts	X b	X
Customizable weight shift detection thresholds	X b	X
Alerts about high pressure	X
Review pressure distribution during past events	X b
View pressure maps in real time on phone	X

Note:

^aAccording to https://store.sensoriafitness.com/sensoria-mat-wheelchair-wellness-coach-with-cushion/

^bAW-Shift© features under development at the time of study but available to the participants to use.

AW-Shift© (Assisted Weight-Shift) System

The AW-Shift© system consists of a 4-way stretch, modified BodiTrac ²⁵ pressure mapping mat (21.85” × 21.85”) with 256 sensing regions spaced one” apart, a custom 4-way-stretch and water-resistant cover, and an interface box that transmits pressure readings (5 Hz) using Bluetooth® to a mobile application (Figure 1). The pressure mapping mat is secured to the top of the user’s wheelchair cushion with a fitted-sheet-style custom cover. The thickness of the cover and mat material was 3 mm. A pocket on the cover holds the interface box and a USB cable extending from the mat. The interface box has a magnetic charging port, power switch, and indicator lights for power and battery charging. The interface box measures 8 × 12.5 × 2 cm and connects to the pressure mapping mat via a USB cable. The interface box must be removed for daily charging. The custom cover has Velcro® tabs on each corner to secure them during transfers. The free edges of the mat otherwise rest loosely on top of the cushion and can be gently draped over the edges of the cushion.

Figure 1.

AW-Shift© hardware and software. (a) AW-Shift© interface box. (b) AW-Shift© pressure mapping mat and cover with a pocket for the interface box. (c) AW-Shift© app displays the user’s interface pressure map and countdown timer to when their next weight shift is due. (d) The AW-Shift© cover wraps over the user’s cushion fitted-sheet style.

The AW-Shift© app displays the user’s real-time pressure map and up to five areas depicting the peak pressure index (PPI), as defined in the International Organization for Standardization (ISO) Technical Report related to pressure mapping. ²⁶ A banner on the home screen shows the user when their next weight shift is due. The banner changes when activity related to redistributing pressure has been detected and includes a countdown timer for their weight shift duration goal. The pressure map view maintains the location of the PPI immediately before starting the activity so the user can see they have removed pressure from the intended area. The default setting for this study was three shifts/hour for 15 s each; however, users could select different frequency, duration, and notification preferences for weight shift activity goals. During the setup, users indicated whether they used power tilt/recline or leaning to relieve pressure, and the thresholds for activity detection and alerts to high pressure were determined. After setup, users could modify the thresholds to increase or decrease the sensitivity. Pressure map data was streamed to a cloud-based server at 5 Hz the entire time the system was used, allowing the user to review past weight shift performance and progress toward their daily goals. They could also replay pressure maps recorded before, during, and after all weight shift and alert events. The app is compatible with both the iOS and Android operating systems.

Sensoria® Mat

The Sensoria® Mat is a thin 16” × 16” fabric mat that consists of four force sensors arranged approximately under the user’s ischial tuberosities and upper thighs (Figure 2). The mat is placed directly on the wheelchair seat underneath the user’s cushion. Velcro® is used on both sides of the mat to secure it to the seat and keep the cushion in place. Each mat has a pigtail with a connection port for the logger, referred to as the Core. The pigtail hangs off the front of the chair, behind the user’s thighs, so that the logger is accessible when sitting on the cushion. The Core is a 3 × 3 × 1 cm, Bluetooth®-enabled box with a single LED indicator light. The LED light indicates the power, charging, and Bluetooth® connection status of the Core. To charge, the Core is removed from the connection port on the mat and placed in a charging dock that connects to a standard microUSB charging cable. For this study, an adaptive handle (shown in Figure 2(b)) had to be designed to make the Sensoria® Mat Core accessible to participants with limited dexterity or hand strength. The 3D-printed handle fits over the Core and provides a larger surface area for pushing it into the ports and more leverage for removing it from the ports. Some versions of the adapted handle included a ring on the back, enabling the user to hook a finger through and pull the Core out of the port. Small rubber pucks were added to the top and bottom of all four sensors to increase sensor contact with the wheelchair seat and cushion.

Figure 2.

Sensoria® Mat hardware and app. (a) Sensoria® Mat with four pressure sensors. (b) Sensoria® Mat Core in its charging port with an adaptive handle designed to make getting the Core out of the charger easier. (c) The Sensoria® Mat placed properly on the wheelchair seat under the cushion so the Core is easily accessible. (d) The Sensoria® Mat app home screen displays the user’s weight shifts per hour and a countdown to when their next weight shift is due.

The Sensoria® Mat app opens to a home screen that displays the user’s number of hours in a chair and their pressure reliefs per hour. The app tracks the number of front leans, side-to-side leans, and push-ups completed by the user throughout the day compared to the number they were expected to complete each hour based on their goal settings. Another screen shows a live display of the user’s center of pressure as calculated from the sensors on the mat. The app sends notifications to the user whenever too much time has passed since their last pressure relief, when they have successfully completed a pressure relief, and when they start but fail to complete pressure relief (called an interrupted pressure relief). The user can define three types of settings: the time between pressure reliefs, the duration of pressure reliefs, and the cushion and lean sensitivities. The time between pressure reliefs defines the maximum time allowed between successful pressure reliefs. The duration of pressure reliefs defines how long the user must hold their lean to successfully complete a pressure relief. The cushion and lean sensitivities define how far the user must lean for the app to detect a pressure relief.

Surveys

After using each system, participants answered questions regarding the general usability and how they used the system. The System Usability Scale (SUS), a ten-item questionnaire that is commonly used to assess a product’s usability, ²⁷ was administered electronically through REDCap® after approximately 4 weeks of using the intervention system. Responses from the SUS were used to calculate a score between zero and 100, where scores between 100 and 80.3 are considered Excellent, scores between 80.3 and 68 are Good, between 68 and 51 are Okay, and scores below 51 are considered Poor. ²⁸

To investigate how participants used the systems, they were asked on a scale of 1 (Never) to 5 (Always) how often they used each system for different contexts of use. Seven different constructs related to managing pressure were defined:

• - Checking their posture

• - Checking for areas of high pressure on their cushion

• - Checking if they were moving far enough to relieve pressure (weight shift quality)

• - Checking how long they were holding their weight shifts (weight shift duration)

• - Checking how many weight shifts they performed (weight shift quantity/frequency)

• - Monitoring the condition of their cushion

• - Setting and tracking their movement goals

Participants were then asked to rate their likeliness to recommend each system between zero (Not at All Likely) and ten (Extremely Likely) for each construct. From these responses, a Net Promoter Score ²⁹ was calculated for each construct as another measure of perceived usefulness. Participants’ ratings of nine or 10 are considered Promoters, those rating seven or eight are Passives, and those ratings six and below are Detractors. The Net Promoter score was calculated as follows:

$$Net\hspace{0.17em}Promoter\hspace{0.17em}Score=\left(\frac{Promoters-Detractors}{Total}\right)*100$$

Usability interviews

Participants answered 13 open-ended questions at the end of each intervention phase (Table 2). Qualitative responses to the questions were summarized using a rapid assessment process ³⁰ to identify the usefulness of the feedback and features of the systems, usability challenges of the systems, and suggested improvements to the systems.

Table 2.

Open-ended usability questions were presented to participants at the end of each intervention phase.

Open-ended usability questions
1. On average, how often did you open or use the app? Did you open it less often or more often as the time went on?
2. When using the app, which screens did you look at and how did you use them?
3. Did you ever change your pressure relief settings or think about changing them?
4. How often did logger lose connection with your phone? Did you have issues with reconnecting?
5. How was your experience with charging the logger?
6. Did you ever need to interact with the sensor mat or hardware? If so, can you tell me more about that experience?
7. Were you surprised by the information reported by the app (e.g., hours in chair, pressure relief stats)? Did you feel the information was accurate?
8. Did you find notifications for pressure reliefs helpful? Did they come too often or not often enough? Did you ever ignore or silence the notifications?
9. What aspects or parts of the system did you find easy to use?
10. What aspects or parts of the system did you find complex or difficult to use?
11. What would you recommend to improve the system?
12. If you had the option, would you continue to use the system after the study ended? Would you recommend others to use the system?
13. Are there other things we need to understand about you (travel, hospitalizations, mechanical issues, physical function) in order to understand your experience?

Data analysis

Differences in system usability scores between participant populations were analyzed with ANOVAs and Tukey post-hoc tests. Differences in reported frequency of use were also analyzed with ANOVAs and Tukey post-hoc tests. Additionally, when appropriate, qualitative feedback was tallied to indicate how many participants commented on specific factors.

Results

A total of 50 participants were enrolled in this study. Seven participants withdrew from the study before completing any intervention phase. Forty-three total participants were included in the analysis; 39 participants used Sensoria® Mat and 37 used AW-Shift©, including 32 participants who used both systems. Participants were most commonly middle-aged (45 +/− 13 years) white men with SCI (Table 3) but included 37% women and 21% individuals with other diagnoses

Table 3.

Demographic characteristics of participants by device.

Device	AW-Shift© (n = 39)		Sensoria® Mat (n = 37)		Combined (n = 43)
	M	SD	M	SD	M	SD
Age (years)	45	13	45	12	45	13
Height (in.)	67	6	68	6	68	6
Weight (lb.)	169	38	175	37	172	37
	n	%	n	%	n	%
Sex
Male	24	61.5	23	62.3	27	62.8
Female	15	38.5	14	37.8	16	37.2
Race
Asian	2	5.1	1	2.7	2	4.7
Black	1	2.6	1	2.7	1	2.3
Native Hawaiian/Pacific islander	0	0	1	2.7	1	2.3
White	32	82.1	32	86.5	35	81.4
Hispanic	3	7.7	1	2.7	3	7.0
More than one/other	2	5.1	1	2.7	2	4.7
Chose not to disclose	2	5.1	1	2.7	2	4.7
Diagnosis
Spinal cord injury	30	92.3	30	81.1	34	79.1
Spina bifida	3	7.7	3	8.1	3	7.0
Cerebral palsy	1	2.6	1	2.7	1	2.3
More than one/other	5	12.8	3	8.1	5	11.6
	n	%	n	%	n	%
Level of injury
N/A	6	15.4	6	16.2	5	11.6
C5-C8	13	33.3	11	29.7	15	34.9
T1-T6	8	20.5	9	24.3	11	25.6
T7-L1	10	25.6	9	24.3	10	23.3
>L1	2	5.1	2	5.4	2	4.7
Level of sensation below injury
Full sensation	4	10.3	4	10.8	4	9.3
Some sensation and/or pain	18	46.2	17	45.9	22	51.2
No sensation	17	43.6	16	43.2	17	39.5
Movement below injury
Yes	16	41.0	14	37.8	17	39.5
No	23	59.0	23	62.3	26	60.5

System usability

Participants reported a wide variety of responses in usability for the two systems, varying from a total SUS score of 5 (Poor) to 97.5 (Excellent), with the median response for each system at 60.0 (Okay) for AW-Shift© and 76.3 (Good) for Sensoria® Mat (Figure 3).

Figure 3.

System Usability Scale (SUS) Scores had a median value of 60.0 for AW-Shift© and 76.3 for Sensoria® Mat, with most responses in the Okay-Good range.

AW-Shift©

On the SUS, 13 participants rated the AW-Shift© system as Poor, 10 as Okay, 7 as Good, and 8 as Excellent (Table 4). To understand the wide distribution of usability scores, SUS scores were compared across the levels of injury and sensation (Figure 4). ANOVA testing revealed that participants with a C5-C8 level of injury scored AW-Shift© significantly higher than participants with a lower T7-L1 level of injury (p = 0.005, Tukey post hoc). While not significant, the SUS scores also varied across the level of sensation, with increasing SUS scores corresponding to individuals with decreasing levels of sensation (p = 0.106).

Table 4.

AW-shift© system usability scale scores by score rankings.

	N	MEDIAN	MIN	MAX
Overall score
AW-Shift©	38	60.00	25.00	97.50
Score rankings
Poor (<51)	13	40.00	25.00	50.00
Okay (51–68)	10	58.75	52.50	65.00
Good (68–80.3)	7	70.00	70.00	80.00
Excellent (>80.3)	8	93.75	87.50	97.50

Figure 4.

AW-Shift© System Usability Scale (SUS) Scores by Participants’ Level of Injury (left) and Sensation (right). Participants with a C5–C8 level of injury scored AW-Shift© significantly higher than participants with a lower T7-L1 level of injury (p = 0.005, ANOVA & Tukey post hoc). There was no significant difference in the SUS scores by the participants’ level of sensation (p = 0.106).

Sensoria® Mat

Seven participants rated the system as Poor, 3 as Okay, 12 as Good, and 12 as Excellent (Table 5). As the distribution of usability scores is also wide, further comparison was made across the level of injury and sensation (Figure 5). There was no significant difference in SUS scores across different levels of injury (p < 0.05, ANOVA) or sensation (p <0.05, ANOVA) for Sensoria® Mat.

Table 5.

Sensoria® Mat system usability scale scores by score rankings.

	N	MEDIAN	MIN	MAX
Overall score
Sensoria® Mat	34	76.25	5.00	97.50
Score rankings
Poor (<51)	7	45.00	5.00	50.00
Okay (51–68)	3	57.50	52.50	60.00
Good (68–80.3)	12	75.00	70.00	80.00
Excellent (>80.3)	12	90.00	82.50	97.50

Figure 5.

Sensoria® Mat system usability scale scores by participants’ level of injury (left) and sensation (right). There was no significant difference in the SUS scores by participants’ level of injury (p = 0.957, ANOVA) or level of sensation (p = 0.379, ANOVA).

System promotion

Participants were the strongest promoters of AW-Shift© for checking areas of high pressure (31.6%) and the strongest detractors for monitoring their cushion condition (−23.7%) and tracking their movement goals (−21.1%) (Figure 6). Participants had more passive responses, neither willing to promote nor detract, related to the use of AW-Shift© for checking the quality (−2.63%) and the quantity (2.63%) of weight shifts. Similarly, participants were the strongest detractors of Sensoria® Mat for monitoring their cushion condition (−45.5%). Participants were the strongest promoters of Sensoria® Mat for checking the quality (18.2%) and quantity (30.3%) of their weight shifts.

Figure 6.

Net promoter scores for AW-Shift© and Sensoria® Mat for different contexts of use.

Reported uses

AW-Shift©

Participants reported using AW-Shift© to check areas of high pressure on their cushion, the quality of their weight shifts, and their posture significantly more often than to check the condition of their cushion or to track their movement goals (p < 0.05, ANOVA) (Figure 7). There were no significant differences in the reported use of the AW-Shift© system between participants with different levels of injury or sensation. However, participants without movement below their level of injury reported using AW-Shift© to track their movement goals significantly more often than those with movement below their level of injury (p < 0.05, ANOVA) (Table 6).

Figure 7.

Frequency with which participants reported using AW-Shift© to check different movement and pressure related concerns. Participants reported using AW-Shift© to check areas of high pressure on their cushion, the quality of their weight shifts, and their posture significantly more often than to check the condition of their cushion or to track their movement goals (p < 0.05, ANOVA).

Table 6.

Frequency with which participants reported using AW-Shift© to track their movement goals. Participants without movement below their level of injury reported using AW-Shift© to track their movement goals significantly more often than those with movement below their level of injury (p < 0.05, ANOVA).

	N	Never/Rarely (%)	Sometimes/Often/Always (%)
Track goals
Movement
Yes	15	66.7	33.3
No	23	47.8	52.2

Sensoria® Mat

Participants reported using Sensoria® Mat to check the quality of their weight shifts, their number of weight shifts, and their posture significantly more often than to check the condition of their cushion (p < 0.05, ANOVA) (Figure 8). Participants with a C5-C8 level of injury reported using the app less often to check areas of high pressure than those without SCI (p < 0.05, ANOVA). Participants with no sensation on their sitting surface reported using the system to check the quality of their weight shifts significantly more often than those with only some sensation and/or pain (p = 0.05) (Table 7). Additionally, participants without movement below their injury used Sensoria® Mat to check the quality and quantity of their weight shifts more often than those with movement (p < 0.05, ANOVA).

Figure 8.

Frequency with which participants reported using Sensoria® Mat to check different movement and cushion related concerns. Participants reported using Sensoria® Mat to check the quality of their weight shifts, their number of weight shifts, and their posture significantly more often than to check the condition of their cushion (p < 0.05, ANOVA).

Table 7.

Frequency with which participants reported using Sensoria® Mat to check their quantity and quality of weight shifts and areas of high pressure. Participants with no sensation on their sitting surface reported using the system to check the quality of their weight shifts significantly more often than those with only some sensation and/or pain (p = 0.05). Participants without movement reported using the app to check their number and quality of weight shifts more often than those with movement (p < 0.05, ANOVA). Participants with a C5–C8 level of injury reported using the app less often to check areas of high pressure than those without SCI (p < 0.05, ANOVA).

	N	Never/Rarely (%)	Sometimes/Often/Always (%)
Quality of WS
Movement
Yes	11	45.5	54.5
No	22	9.10	90.9
Sensation
Full	4	0.00	100.0
Some	14	50.0	50.0
None	15	0.00	100.0
Quantity of WS
Movement
Yes	11	27.3	72.7
No	22	13.6	86.4
High pressure
Level of injury
N/A	5	0.00	100.0
C5-C8	11	63.6	36.4
T1-T6	9	44.4	55.6
T7-L1	6	33.3	66.7
> L1	2	100.0	0.00

Qualitative responses

Usefulness of feedback and features of the systems

Overall, participants reported using the feedback of the systems in two ways: in real time or retrospectively to review past performance. Real-time feedback was most commonly used to view pressure distribution or the timer countdown as they performed weight shifts (n = 20). Feedback about past performance was most commonly used to check metrics related to weight shift performance, see total time spent in the wheelchair, compare actual performance to their goals, or observe the variety or type of weight shifts the system detected (n = 21). Participants also described how they used the feedback to guide posture and check the performance of the cushion.

Notifications received mixed responses from participants depending on the context and conditions. Notifications were perceived as helpful by many participants across both technologies, and most customized them in some way, either by changing the reminder frequency, adjusting detection thresholds, or changing the notification delivery method. Notifications and feedback from the systems were perceived as helpful (n = 26) for being aware of time passing, to validate their perceptions about pressure relief, to remind them to move, and to identify patterns in their weight shift activity. Conversely, notifications were also perceived as annoying or disruptive in some contexts, such as work or in social situations (n = 21), leading them to make changes in the settings. Many participants (n = 17) reported they did not believe they needed the feedback provided by the systems, their comments reflecting that they were not concerned and described themselves as very active. Over half of the participants admitted to turning notifications off completely.

Participants were divided about whether they found the feedback accurate or not. Users of both systems commented that feedback felt accurate when the systems worked as expected. Folds or wrinkles in the cover created artifacts that reduced the accuracy of the AW-Shift© feedback; however, users perceived AW-Shift©’s weight shift detection as accurate. Sensoria® Mat users commented that they found the system to have a lag or delay in detecting weight shifts and that shifts would be counted as incomplete even though the user held their shift for the required period of time. They also reported inconsistency in the identification of lean direction or type with Sensoria® Mat use.

As far as interpreting the feedback, a small number (n = 8) commented that either the graphs were confusing or that they did not understand some of the information, while nearly all (n = 39) indicated that they found the feedback from both systems easy to interpret and that the software was easy to use.

Usability challenges of the systems

User-perceived usability challenges centered around the following features: battery capacity, charging method, Bluetooth® connectivity, software issues, sensing mat, and hardware form factor (Table 8). Charging issues were attributed to the hassle of daily charging, difficulty getting Sensoria® Mat’s Core out of the charger (but not out of the mat’s pigtail) or removing the box from the pouch for charging AW-Shift©. Those who found the magnetic charging cable easy to use tended to have less hand function; others thought it disconnected too easily.

Table 8.

Mobile health assistive technologies for wheelchair-related in-seat movement and pressure usability challenges and suggested improvements – selected exemplar statements from participants.

Challenges - participant statements	Suggested improvements
Battery charging and battery capacity
“...Battery life frustrating.” “...Battery didn't last long enough.” “...Box didn't always make it a full day and the battery would die.”	Make getting the logger in and out easier. Longer battery life - charge weekly instead of daily. Make it easier to manipulate small parts. Improve access to parts for charging. Notify when the battery is low
Bluetooth® connectivity
“…Never one approach that seemed to work consistently.” “...Hard to know if it was actually connected ...eventually couldn't get it to reconnect at all.” “...If your phone went too far away it would disconnect.” “...Connectivity was the biggest problem.”	Improve the consistency of the Bluetooth® connection. Reduce disconnection when separated from the phone or make reconnection automatic Do not require a consistent Bluetooth® connection (e.g., on-board data processing with intermittent data uploads)
Sensing Mat
“...Bunches up during sliding transfers.” “...Getting mat straightened was cumbersome.” “...doesn't have a handle - hard to get cushion in and out of chair.” “...Mat was too big - difficult getting sides tucked in.” “...Made cushion heavier.” “...doesn't like on top of cushion “...Cover felt slippery.” “...Mat not washable or waterproof.” “...Not breathable, surface not ‘friendly’ to skin.”	Secure the mat so it does not move when transferring into the chair. Better fit to cushion, customize to cushion size. Reduce the chance of wrinkles. Lighter overall (weight) and increase stretch. Put the mat inside or underneath the cushion. Use a less bulky mat with no excess material. Air permeable surface, but also waterproof. More Velcro® for secure connection to seat pan. Avoid using seams in the cover material where it contacts the body
Software
“...Occasionally the app would glitch and shut down…frustrating.” “...Delay in recognizing pressure relief activity start.” “...didn't seem to actually register any weight shifts.” “...Map color would not show up even though box was connected to phone.” “...Seemed like it would take a long time for it to sense her movement.”	Option to turn off different types of notifications. Option for different sounds for different types of notifications. Improve the accuracy of weight shift detection. Provide low battery notifications and a battery level indicator. Provide an indicator to show when data is sent to the cloud. Include more feedback on error messages. Include a longer time range for between shift times. Detect pushups. Reduce or eliminate delay in detecting the start of a weight shift. Add rewards for performance or gamify the system. Detect shorter leans as valid measures of performance. Detect tilt activity on power chairs (Sensoria® Mat). Include reminders to check the skin. Provide an end-of-day summary about issues from day
Hardware
“...Distracting light on logger.” “...Mat made his cushion the heaviest part when breaking chair down, hardware bulky.” “After the first couple of years (in w/c) you start getting rid of things on your w/c, you go as narrow and light as possible to be as minimal in your chair as possible.” “...Location of pouch was a nuisance.” “...Battery pack hanging down in the way” “...Core fell out during a transfer - on sidewalk”	Eliminate indicator lights. Minimize the size of hardware. Conceal logger out of the way. Use shorter cables. Add a button to reset all default settings. Esthetics are important – make it look nice Consider durability – reinforced to protect from damage

Bluetooth® connectivity was a primary issue for many participants. If they were out of range of their phone, the system would disconnect. Comments reflected that they did not always know the process or sequence for reconnecting the system. Software glitches resulted in unexpected behavior of notifications (both systems), poor responsiveness of the app in real time (Sensoria® Mat), and trouble getting the pressure map to connect even when Bluetooth® was connected (AW-Shift©).

Mat issues were described for many AW-Shift© users (n = 28). The two primary issues were that the mat was too large, bulky, or folded/wrinkled during transfers. Sensoria® Mat users wanted more Velcro® to keep the mat in place under their cushion or while they prepared the chair for transport in their vehicle. Hardware issues revolved around esthetics or parts of the system getting in their way as they completed daily activities.

Suggested improvements to the systems

Suggestions for improvements to both systems are summarized in Table 8. The issues most likely to result in a user not wanting to continue using the systems were poor battery capacity, Bluetooth® connectivity, lag or accuracy in detecting weight shifts, and the size of the AW-Shift© pressure map and cover. Nearly all participants (n = 32) indicated they would recommend one or both systems to other wheelchair users in specific contexts: acute spinal cord injuries, new wheelchair users, those more prone to pressure injuries, those with poor sensation, and less active wheelchair users.

Discussion

The objective of this study was to understand the system usability of two mobile health assistive technology systems designed for managing in-seat movement and pressure related to wheelchair use in the community. Overall, the findings suggest that variability in usability scores as well as in how the systems were used may be attributed to key differences between the two systems’ hardware and software, combined with mobility and sensory function of the user. SUS scores and responses to usability questions suggest that there is no one-size-fits-all solution, and that subpopulations of wheelchair users based on motor and sensory function seem to have different needs and preferences, some of which can be grouped by a user’s functional abilities. The results of the study provide valuable design considerations for optimizing usability provided by personal-use mHealth assistive technologies that aim to improve in-seat activity to reduce pressure injury risk.

Overall usability

Key drivers of SUS scores varied between the systems in terms of subpopulation characteristics. For AW-Shift©, usability scores (SUS) were more favorable for users with poor sensation and users with more severe mobility impairments. These user characteristics tend to result in lower overall activity levels. ²² Users who were less active likely had fewer challenges related to the pressure-mapping cover moving or wrinkling during transfers or with the mat and cover interfering with operation of the wheelchair once they were set up each day. Users with more mobility often perform more transfers during the day and may need to dismantle or fold their chair for transport in a vehicle compared with more impaired users who may be in a power wheelchair, requiring less interaction with the sensing mat once set up. AW-Shift© was developed and refined through a user-experience design approach with both power and manual wheelchair users with SCI.^13,31 A specific challenge inherent in the AW-Shift© system is the placement of the pressure map on top of the seat cushion and the size of the mat not customized to the cushion size. The pressure map and cover, while modified over time based on user feedback to optimize them for research purposes, have not yet been the primary focus of development and there are known limitations with use of commercially available pressure sensing mats. ³² In prior studies using earlier versions of the AW-Shift© system, usability findings were consistent with the current study in that power wheelchair users, who typically have more impaired mobility, indicated a higher satisfaction with the design than manual wheelchair users. ¹³

Usability scores (SUS) after use of Sensoria® Mat were similar across mobility and sensory level subpopulations. This lack of significant variability across subgroups could be explained by users not needing to interact with the sensing mat once it is set up under the cushion. Set-up occurs during the initial visit, and then typically did not need to be interacted with daily. Critical feedback on usability when using Sensoria® Mat tended to center around the feedback’s accuracy and difficulty manipulating the charging component. Sensoria® Mat is not designed to detect tilts, which means that participants who relied on tilts as their only form of weight shift could not use the system at all so their potential experiences are not reflected in this data. Additionally, adaptations were implemented during Sensoria® Mat use for certain subpopulations of wheelchair users, which may have improved usability (SUS) scores by reducing the impact of limited mobility. For example, the Sensoria® Mat app only provides feedback after a successful initial calibration is completed. Some users with limited trunk mobility had difficulty moving far enough to successfully complete the calibration, and in those cases, it was necessary to place weights on the four pressure sensors of the mat to simulate the leans. Furthermore, a small tool to extract the charging Core from the mat and the charger base was needed for individuals with poor hand function due to the tight fit and small profile of the Core. Had participants had to use Sensoria® Mat without these study-provided modifications, the overall SUS score would likely have been much lower.

Weight shift metrics

Both systems provided real-time and retrospective performance-based feedback to the user about weight shift duration, frequency, and quality. While users of each system reported that they used the systems for monitoring weight shift metrics, the Net Promotor Score was higher for Sensoria® Mat than for AW-Shift© for these specific constructs. For each system, more than 70% of the participants reported using the feedback to monitor weight shift metrics “Sometimes”, “Often”, or “Always”. This result is not surprising given that the home screen for both apps includes countdown timers and indicators for when the next weight shift is due. Thus, when the app is opened, weight shift metric information is readily seen without the need to interact with the app. Wheelchair users have been exposed to use of timers and reminders as part of standard patient education where they are instructed to complete weight shifts using a time-based schedule, and one strategy often recommended is to use timers to facilitate that performance.^15,33

The Net Promotor Score differs between the two systems in terms of promoting the system to others as a tool to observe feedback about weight shift metrics, which may be attributed to the type of information presented to the user on the home screen. The primary information presented on the Sensoria® Mat app is weight shift performance metrics, while AW-Shift© provides a live pressure map view that covers more than 2/3 of the screen in addition to the weight shift timers. This difference in how feedback is presented on the main screen may explain why users seemed more likely to promote Sensoria® Mat for weight shift performance feedback as it was the primary feedback provided. The Net Promoter Score was close to 0% for using AW-Shift© for weight shift feedback, which reflects a more neutral perspective regarding the promotion of the system to others.

High pressure

More than 80% of AW-Shift© users reported using the app’s feedback to monitor high pressure areas “Sometimes”, “Often”, or “Always”, which aligns with the primary functional feature of the AW-Shift© system and reflects what the user sees on the main screen of the app. Conversely, more than 60% of Sensoria® Mat users reported that they “Never”, “Rarely”, or “Sometimes” used the app feedback to monitor high pressure areas. The Net Promoter Score differences between AW-Shift© (30%) and Sensoria® Mat (−20%) align with the users’ variations in using the system to view areas of high pressure. The Sensoria® Mat app did not show the users interface pressure values but did provide indirect information about pressure through feedback about which direction the user had moved and which direction they needed to move next based on their center of pressure deviation from the center. Users of AW-Shift© in earlier studies to explore feasibility of viewing pressure map feedback outside of the clinical setting described finding value in the detailed pressure feedback. ¹³

Goal tracking

The use of the technologies to monitor their performance toward goals was not promoted for either device. It is not known whether users simply do not find value in tracking their performance toward goals, whether the duration of use was not long enough for them to become familiar with the feedback to allow them to see patterns in their weight shift performance over time, or whether there were technological challenges limiting goal-tracking performance within the apps. A possible limitation related to goal tracking for Sensoria® Mat is that the app displays only the current day’s metrics, which does not show performance over time. AW-Shift© provides daily, weekly, and monthly metrics, however, this feature is still under development which may have impacted how valuable the information was to the participants. It is also possible that some people are more inherently drawn toward self-monitoring technologies and goal tracking than others, regardless of mobility, sensory impairment level, or type of wheelchair used, and that there are other contributing factors.

Checking posture or cushion condition

Both technologies received negative Net Promoter Scores for checking both cushion condition and posture; however, participants reported they did use the feedback to guide their posture and check their cushion condition. Nearly 80% of AW-Shift© users, for example, indicated use of the feedback to guide their posture “Sometimes”, “Often”, or “Always, and the Net Promoter Score was just slightly negative and close to zero. On AW-Shift©, users can see the impact of their position or posture on overall pressure distribution, which may lead them to adjust their posture and observe the immediate outcome. Sensoria® Mat received a more negative Net Promoter Score (−12%) and a mixed response in use of the feedback to guide posture, with about half reporting they did not use the feedback for that purpose.

Differences in use between subpopulations

There were several instances where different subpopulations of the participants reported using the systems in different ways, and users’ comments from the post-intervention interviews provided additional context around what they found useful versus not useful and which features enabled versus created barriers in performing daily activities. Since participants with limited sensation do not receive physical feedback to indicate they have successfully relieved enough pressure during their shift, many reported that having a visual or auditory indication that they had leaned far enough, along with the pressure map feedback, was very helpful. This is a promising benefit of technologies like AW-Shift© and Sensoria® Mat that provide weight shift cues and performance feedback, considering the high risk of developing pressure injuries among the subgroups with more limited function.

Suggested improvements

While there were some consistently valued features across the systems, such as the audio and visual feedback on weight shift quality and duration, a few key improvements were indicated repeatedly throughout the study. Bluetooth® connectivity was one of the biggest points of frustration for almost all users. Participants often faced challenges maintaining a consistent connection due to disruptions in their daily lives, resulting in data loss and inconvenience. Therefore, it is crucial to prioritize a solution that addresses these concerns. One approach is to develop a more robust Bluetooth® connection that automatically reconnects without user intervention. By eliminating the need for users to manually manage the connection, the system becomes more user-friendly and reliable. However, it is also essential to explore alternative approaches that do not rely solely on Bluetooth® due to its inherent limitations. Recognizing that disconnections are inevitable - whether due to users leaving their mobile devices behind or external interference - it becomes necessary to devise a data transfer process that remains unaffected by such disconnects. Implementing intermittent or on-demand data uploads may be a viable solution. This approach allows users to access complete data sets whenever they open the app, regardless of their previous connection status. Furthermore, evaluating other communication technologies, such as Wi-Fi or cellular connectivity, may provide more reliable options in situations where Bluetooth® may not suffice.

Another significant finding from the qualitative interviews was the importance of customization regarding notifications and feedback. While certain participants appreciated the persistent nature of the notifications, which ranged from infrequent to regular intervals of every 20–30 min, or even near-constant presence, a considerable number grew frustrated with them. As a result, they resorted to modifying their app settings to decrease the frequency of notifications or to completely disable them. Some expressed a need to temporarily silence what they found to be generally useful notifications, such as when in a meeting at work. Additionally, many differed in the content of the notifications they wished to receive. Some only wanted reminders to move as compared to others who only wanted system-related notifications (i.e., low battery or Bluetooth® disconnects). Allowing the user to control how, when, and why they get notified is extremely important to overall user satisfaction.

Study limitations

This usability study had some limitations related to the study duration, population, and status of the systems. Participants only used the systems for roughly 4 weeks each, capturing only a small portion of their life experiences. This may not have been enough time for participants to fully integrate the systems into their regular life and accurately study technology adoption or abandonment. Participants did not represent the full range of abilities of wheelchair users who might benefit from AW-Shift© or Sensoria® Mat as most wheelchair users in this study used manual wheelchairs. Additionally, neither system was off-the-shelf ready as tested. AW-Shift© development is still in the prototype phase with several additional features and changes to be implemented in the future. Sensoria® Mat required several modifications to be successfully setup and used by participants. Almost all participants needed continuous support and guidance to manage and troubleshoot both systems. Had participants simply been provided the equipment and left to setup everything on their own, it is expected that usability scores would be significantly affected. The goal of this study was not to directly compare the AW-Shift© and Sensoria® Mat systems, therefore, all attempts were made to minimize such comparisons through participant instruction and the 2-week washout period. Participants who used both systems were instructed to only provide feedback on the system they had just used, however, some still drew direct comparisons between the systems in their qualitative interview responses. It is important to acknowledge that participants’ familiarity with other devices and their general exposure to mHealth apps may have exerted an influence on the usability scores.

Conclusion

Overall, the study suggests that mHealth assistive technologies that provide feedback to users about in-seat movement and pressure need to consider both the usefulness of the information provided to the user and the usability of the system when integrated into daily life. Most importantly, developers must understand the diverse needs of wheelchair users. When introducing a technology, it is essential to prioritize the individual's comfort and function, without compromising the validity of the feedback. Furthermore, the varied needs indicate different design constraints to ensure that all intended users can operate the system. Specifically, improvements to the Bluetooth® connectivity and data transfer of both systems are needed to make them more user-friendly on a commercial scale. The findings from this study suggest that different cohorts of users may benefit from distinct feedback constructs. Optimizing the design for specific cohorts or constructs can result in an effective product that consistently provides meaningful and accurate information about behavior and performance, a critical aspect of design. Additionally, allowing customization of the app features, such as notification content and frequency, would better meet the varying preferences found within the wheelchair user community. Finally, evaluating technologies for wheelchair users in their daily routines throughout the development process is crucial for understanding the impact of the technologies on daily life and their potential for ongoing use.

Appendix.

Table 1.

Usefulness of system feedback and features per selected exemplar open-ended participant responses.

Monitor real-time performance

“...To see when next weight shift was coming up.”

“...Checked it (map feedback) every time he did a weight shift.”

“...See how far back to tilt to show difference on map.”

“...Liked visually being able to see what his weight shifts looked like – ‘it made a difference’”.

“...Opened it every time it gave her a reminder to do a weight shift to see which direction she had to go.”

Weight shift duration

“...Very helpful to see how long to hold leans.”

“...Pressure relief time is a lot longer than you think it is.”

“...It was telling her to do something longer and that was helpful.”

Weight shift quality

“...Looked at main map screen when doing weight shifts to see if the pressure was being relieved.”

“...Used different tilt features and look at map to see where pressure was.”

“...See how far back to tilt to show difference on map.”

“...When she moved and watched map, could tell where pressure was being relieved.”

“...Looked to see how much COP moves when shifting right or left.”

Review past performance

“...it's great to be able to see your accomplishments because it motivates you to keep trying and maintain that level.”

“...Trying to see what kind of shifts she was doing, length of shifts, and if she was getting close to goal.”

“...To see the different leans he used most”

“...Liked the graph and looked at that a lot to make sure she as doing her shifts and seeing how long the shifts were.”

Weight shift frequency

“...Notorious for not doing his reliefs as much as he should – helpful.”

“...To see when next weight shift was coming up.”

Time in wheelchair

“...Surprised - in chair too long and should be transferring out more.”

“...Concerned with time spent in and out of chair because her MD had told her to spend less time in her chair.”

“...didn't realize how much time spent in chair.”

Check posture

“...Surprised by amount of time posture bad - too much pressure on one side.”

“...After adjusting posture, alerts were less frequent.”

“… To understand if one hip has more pressure than the other.”

“...Wanted to make sure she was sitting the best way possible for pressures.”

“...Wanted to make sure she was sitting evenly.”

“...Reminded to sit up straight.”

Check pressure distribution

“...Seeing where pressure was - really cool.”

“...Open his eyes to potential problem areas.”

“...Surprised to see where map showed areas of pressure, didn't notice he had such pronounced pressure points.”

“...Gave a quick visual of where the problem spots were….’the visual was so powerful’“

Check cushion condition or performance

“...Using it once a week to see if cushion is inflated correctly.”

“...See if cushion is doing its job.”

“...Adjusted gel cushion and it would make a difference in how the map read.”

Note. The exemplars are transcribed comments the participants made during their open-ended interview questions at the end of each intervention period.

ORCID iDs

Kathleen Jordan https://orcid.org/0000-0002-5174-8532

Tamara Vos-Draper https://orcid.org/0000-0002-9038-2764

Melissa Morrow https://orcid.org/0000-0002-6500-6572