Usability Assessment of Technologies for Remote Monitoring of Knee Osteoarthritis

Abstract

Goal: To evaluate the usability of different technologies designed for a remote assessment of knee osteoarthritis. Methods: We recruited eleven patients affected by mild or moderate knee osteoarthritis, eleven caregivers, and eleven clinicians to assess the following technologies: a wristband for monitoring physical activity, an examination chair for measuring leg extension, a thermal camera for acquiring skin thermographic data, a force balance for measuring center of pressure, an ultrasound imaging system for remote echographic acquisition, a mobile app, and a clinical portal software. Specific questionnaires scoring usability were filled out by patients, caregivers and clinicians. Results: The questionnaires highlighted a good level of usability and user-friendliness for all the technologies, obtaining an average score of 8.7 provided by the patients, 8.8 by the caregivers, and 8.5 by the clinicians, on a scale ranging from 0 to 10. Such average scores were calculated by putting together the scores obtained for the single technologies under evaluation and averaging them. Conclusions: This study demonstrates a high level of acceptability for the tested portable technologies designed for a potentially remote and frequent assessment of knee osteoarthritis.

I. Introduction

Knee osteoarthritis (OA) is a degenerative and debilitating musculoskeletal condition that primarily affects the articular cartilage and the subchondral bone, also causing inflammation in the synovium of the knee joint [1]. It is a a vastly widespread pathology, which causes pain and disability, particularly among elderly people who suffer from joint pain, stiffness, decreased mobility, with a significant reduction of their life quality [2].

The management of knee OA typically relies on periodic clinical visits, which may not fully capture the dynamic nature of the disease or provide immediate insights to tailor treatment plans to individual needs.

In this context, remote (at-home) and more frequent monitoring of the health status of patients affected by knee OA would be much desirable. This could be facilitated by wearable devices, mobile applications, and Internet of Things (IoT) platforms. This approach would allow, indeed, continuous and objective data collection on pain levels, joint function, and physical activity, which are critical factors in understanding the progression and impact of knee OA [3], [4].

In recent years, wearable technologies have emerged as valuable tools for monitoring knee OA. Devices such as smart knee braces, smartwatches, and accelerometers enabled continuous tracking of joint movements and physical activity levels, providing healthcare professionals with dynamic data for more informed decision-making and tailoring subsequent rehabilitation processes [5], [6], [7], [8]. Furthermore, static and dynamic measurements of the center of pressure (CoP) through force balance platforms have shown a correlation with the knee status and the quality of life of OA patients [9].

Although these technologies offer interesting insights, they do not reveal the underlying physiological alterations of the knee. To this aim, other technologies may be used: infrared thermography can monitor skin temperature and indirectly assess synovial inflammation processes, which play a pivotal role in the pathophysiology of OA [10], [11], [12]. Interestingly, other non-invasive sensing methods, such as wearable knee bioimpedance measuring systems, have recently been used to quantify the electrical impedance of knee joint tissues, which can be correlated with knee pain during daily activities [13]. Additionally, while some efforts are still needed to mature portable imaging acquisition system technologies for common practice use, tele-ultrasonography may offer a remote, repeatable, cost-effective, and rapid assessment of knee joint internal structures in the future [14].

In this context, telehealth platforms are constantly improving in enabling direct communication between patients and physicians. They facilitate virtual consultations, allowing healthcare providers to assess a patient's condition remotely and make necessary adjustments to treatment plans. This approach ensures continuity of care and reduces the need for in-person visits, which is especially beneficial for patients with limited mobility [15].

Dedicated mobile applications designed for knee OA patients have been recently proposed [16], [17]. These apps enable patients to record and report their daily symptoms, including pain intensity and joint stiffness. However, these apps are not integrated with data derived from wearable devices. Indeed, filling this gap would create a comprehensive profile of patients' conditions, to be monitored remotely.

Despite recent advancements in knee OA remote monitoring technologies, several challenges still need to be addressed to make this approach a reality in everyday life. A fundamental aspect that is too often neglected is the acceptability of these technologies by end-users. Usability is actually a critical factor that developers of such technologies must take into consideration if they intend to make these systems available in the market and expand the user base among both patients and healthcare professionals.

In this study, we recruited 11 patients affected by mild or moderate knee impairments and 11 caregivers, and we asked them to test a combination of different technologies intended for continuous and remote monitoring of the pathology. These technologies included a mobile app to guide the users during the various activities, a wristband with a built-in accelerometer for monitoring physical activity, an examination chair integrating goniometers for leg extension measurements, a thermal camera for thermographic data acquisition, a force balance for center of pressure measurements, and an ultrasound (US) imaging system for remote imaging.

The usability of each device and the feasibility of the procedures were evaluated through dedicated questionnaires. Additionally, 11 clinicians working in their clinical practice in the field of knee osteoarthritis as orthopedic surgeon or radiologist, evaluated the usability of the clinical portal software, through which they could monitor patients, schedule activities, and communicate with them. This portal completed the IoT architecture of the system, connecting patients and clinicians.

II. Materials and Methods

Eleven volunteers affected by mild or moderate knee cartilage tissue alterations (named patients in the manuscript) were recruited to test the usability of several technologies developed for at-home diagnosis of knee OA (geographical location, age and gender distribution are shown in Fig. S1). For each volunteer, a caregiver (a family member or an assistant) was also involved in the test, to assist the patient throughout the procedure.

All the tests were conducted at the Biorobotics Institute of the Scuola Superiore Sant'Anna in Pontedera (Pisa, Italy). This usability test was approved from the joint ethics committee established by Scuola Superiore Sant'Anna, Scuola Normale Superiore, and IMT Lucca (Protocol no. 36/2022).

Before performing all the activities, the patients filled out a pain report, replying to 24 questions regarding pain, stiffness, and physical function during daily life activities (the score ranged from none to extreme). Based on the answers provided by the patients (Fig. S2), the standard WOMAC (Western Ontario and McMaster Universities Arthritis) index was calculated [18]. The WOMAC score of the recruited patients ranged from 4 to 44 (Table S1). Before each task, patients and caregivers were instructed through a specific video tutorial describing step-by-step the procedure to be performed. All the tutorials were previously uploaded directly into the mobile App, except for the one regarding US imaging. For US imaging, the procedure was managed through a user-friendly graphical user interface, installed on a PC, which guided the user.

A. Mobile App

An Android mobile application was developed by the company H&D Wireless (Kista, Sweden). The program communicated with the connected devices and uploaded the data to the server, allowing the clinician to monitor each patient remotely. The App served as a guide for various tasks, enabling easy and fast communication with the assigned clinician through a direct chat function. Additionally, it included a calendar through which the users could access the daily activities scheduled. All enrolled patients and caregivers were asked to use the App on a smartphone (Redmi 10C, Android 12). Both the calendar and chat functions were tested. Fig. 1(a)shows the home page of the App on which the different activities were listed and a representative image of a volunteer testing it. To assist users in correctly performing the procedures, a series of animations and tutorial pages were also prepared (see Supplementary Material).

Fig. 2.

Screenshot of the clinical portal with the summary of patients’ data (left) and picture of a clinician during the usability test (right).

Fig. 1.

Usability evaluation of the devices intended for at-home assessment of knee OA: (a) Home-page, calendar section of the mobile app (left) and representative picture of the usability test(right). (b) Representative screenshot of the tutorial (left) and photo of a volunteer wearing the wristband and connecting it to the smartphone (right). (c) Representative screenshot of the tutorial about leg extension measurements (left) and volunteer and caregiver during the procedure (right). (d) Screenshot of the tutorial (left), and volunteer and caregiver during the measurement of knee skin temperature (right). (e) Representative screenshot of the tutorial (left), and volunteer and caregiver during the balance monitoring (standing position on both legs) procedure (right). (f) Anterior and posterior wearable brace (left), and volunteer and caregiver during the US imaging acquisition procedure (right).

B. Activity Monitoring

Physical activity levels correlate well with pain levels and are a reliable indicator of the patient's health status, as recently demonstrated in a longitudinal analysis carried out on 17454 patients with knee or hip osteoarthritis [19]. Consequently, although they have not yet been entirely validated as clinical predictors, wearable activity trackers are being proposed to monitor physical activity in OA patients [20] and constitute exciting tools to gather information on their health condition in real-time. For this reason, a silicone Box818 wristband with a built-in accelerometer and Bluetooth connection was used to measure the physical activity of the patients. The collected data were transferred to the App through the NFC technology. At the beginning of the test day, the patients wore the wristband and connected it to the App with the caregiver's help (Fig. 1(b)), following the instruction reported in the tutorial (Suppl. Video 1). At the end of the test, the data were uploaded in the App as minutes of activities, and the volunteer got the wristband off.

After the test day, the patients were also asked to wear the wristband passively (without recording data) for an entire week to evaluate any discomfort and interference with daily activities.

C. Range of Motion Measurement

The analysis of knee flexion and extension degrees can serve as a straightforward measure of a patient's recovery and an indicator of joint function improvements, being knee flexion range of motion reduced with the onset of knee OA [21], [22]. An examination chair with integrated goniometers capable of measuring leg extension was developed by Image Guided Therapy (Pessac, France) and depicted in Fig. 1(c). To assess leg extension capabilities, each volunteer, supported by the caregiver, was asked to perform two exercises, namely a maximum extension and a maximum flexion of the knee, repeated twice for each limb (Suppl. Video 2). For each exercise and limb, the range of motion was calculated as the difference between the maximum knee extension angle and the maximum knee flexion angle reached.

D. Temperature Acquisition

Thermography is an effective tool for assessing the presence of inflammatory processes. Previous evidence showed a correlation between a decrease in the differential skin temperature (i.e., temperature difference between the pathological and healthy knee) and patients' recovery [23]. An infrared thermal camera (FLIR One Pro) was connected by the caregiver to the smartphone, for thermographic image acquisition, as shown in Fig. 1(d) and Suppl. Video 3.

Two thermal images, one for each knee, were acquired to enable differential measurement (Fig. S3). Automatically, the App recorded and saved the images and recorded the higher temperature in the selected region of interest (i.e., the knee). This information was then uploaded to the server.

E. Balance Monitoring

Some evidence in the current state of the art suggests a correlation between the severity of OA (and thus the pain levels) and the body sway, namely the ability to maintain a specific posture [24]. For example, patients affected by hip OA show significantly lower proprioceptive accuracy and poorer functional balance than healthy subjects [25]. Indeed, recently, neuromuscular exercises and neuromuscular electrical stimulation have been proposed as methods to improve balance and consequently reduce fall risks, in OA patients [26]. Similar evaluations are the object of ongoing clinical trials [27], thus evidencing the importance of balance as a possible predictor of the OA pathology progression. The body sway can be quantified through different techniques; one of the most effective ones consists of monitoring the center of pressure (CoP) over time, through a force balance [28], [29]. The device selected to collect data on the CoP, in particular to compute the CoP length [30] was a commercial Kistler platform (model 9286B equipped with BioWare Software). To evaluate the body sway, the patients performed three exercises: (1) maintaining a standing position on both legs (Fig. 1(e)) for 20 seconds (repeated twice), (2) maintaining a standing position only on one leg (the right one) for 10 seconds (repeated twice), and (3) maintaining a standing position only on one leg (the left one) for 10 seconds (repeated twice). An example of data acquired is reported in Fig. S4. The caregivers played a supportive role, instructing the patients during the procedure and managing the app to start/stop the recordings (Suppl. Video 4).

F. US Imaging

US imaging is particularly valuable for assessing knee OA, even in the early stage, wherein cartilage damage may be minimal but soft tissue changes are evident [31], [32]. US images of the cartilage, knee joint and surrounding tissues can provide information about alterations in articular cartilage, synovial tissue, bony cortex, and joint effusion, thereby establishing the inflammatory status of the knee, the patient's healing process and the effectiveness of treatments [33]. In a previous study, eight pre-defined positions for the US probe were identified, to provide a comprehensive evaluation of the knee cartilage [14]. Based on these specifications, two wearable braces, one for imaging the anterior knee cartilage and joint knee effusion/inflammation, the other for imaging the posterior knee cartilage, were designed and developed. The braces included predefined openings, in which the US probe could be easily and reliably placed (Fig. 1(f)). Unlike all the other previous tasks managed directly by the App on the phone, US imaging was handled through a software installed on a PC. A user-friendly dedicated graphical user interface was developed to provide a step-by-step guide for operators during US acquisitions and data collection (see Suppl. Video 5) US acquisitions were performed using the ARTUS EXT-1H ultrasonography system from Telemed (Vilnius, Lituania) and a linear array probe with an upper frequency of 15 MHz.

The patients, with the assistance of the caregivers, wore the wearable braces and positioned themselves on the same examination chair (equipped with goniometers) used for Range of Motion measurements. At each step, the software guided patients and caregivers to acquire clinically relevant images, indicating which region of the knee should be imaged and in which position the knee should be placed. For each position, the user interface displayed the reference image (acquired by a professional radiologist, specialized in ultrasonic imaging of the knee cartilage in a previous test, carried out on a different day) and the real-time US transducer video stream. This helped the caregiver replicating as close as possible the image acquired by the professional radiologist. The caregiver acquired three videos (10 seconds each) for each brace opening. An image recognition algorithm based on correlation [34], included in the software, automatically selected the most similar frame from the video with respect to the reference image acquired by the clinician in the initial phase. The selected image is, finally, shown to the user that can either accept it or decide to perform the measurement again. An example of ultrasound images acquired by a caregiver in comparison with the clinicians ones are reported in Fig. S5.

G. Clinician Portal

All the data collected by the App on the phone and the Software on the PC, were sent to an IoT server (GRIFFIN^TM server), and the results uploaded in real-time to the clinician's portal. The clinician web page enabled clinicians to access information about the patients under evaluation. All data acquired by the patient's app were directly uploaded to the clinician's portal. On this platform, clinicians could download data, schedule activities, and communicate directly with patients.

Upon logging in, the portal displayed the list of assigned patients. By clicking on any patient in the list view, the clinician could access their healing status and some diagnostic and treatment data (see Suppl. Video 6).

Eleven clinicians (orthopedic surgeons and radiologists with expertise in knee OA) were recruited. After a 20-minute training session, clinicians were asked to log in into the portal, explore the various sections, download data, interpret the collected information, schedule a task on the calendar, and send a message to the patient.

H. Questionnaires for Usability Assessment

At the end of each activity described above, patients, caregivers and clinicians filled dedicated questionnaires about the usability of devices and procedures, providing a score from 0 (low usability) to 10 (high usability) in correspondence to each question. All the questionnaires were filled out by patients and caregivers on the same day of the visit, except from the one concerning the wristband use in passive mode. For this device, the questionnaire was filled out and sent back by e-mail after a week of continuous use at home.

III. Results and Discussion

A. Results of Usability Tests: Patients and Caregivers

The results of the usability questionnaires filled by the eleven patients and caregivers are reported in Fig. 3.

Fig. 3.

Results of the questionnaires used to assess the usability of the different technologies tested by patients and caregivers: (a) Mobile application; (b) physical activity monitoring ;(c) leg extension measurements; (d) temperature measurements; (e) force balance test; (f) US image acquisitions.

Regarding the evaluation of the phone app (Fig. 3(a)) the mean score was higher than 9 (9.4 for the patients and 9.2 for the caregivers), indicating a high level of user- friendliness.

The results of the physical activity data acquired by the wristband are reported in Fig. 3(b). Overall, the mean score was 9.2 for the active mode and 6.8 for the passive mode. The lower score associated with the use of the wristband for an entire week (in passive mode) was mainly due to a slight discomfort caused by its prolonged use and possible interferences during daily life activities. From the feedbacks received, the discomfort was mainly due to: i) the silicon material, which was not very comfortable, especially in warm weather, ii) the color, which was not appreciated from an aesthetic point of view, and iii) the poor adaptability of the strap to the wrist. These aspects played a relevant role only during prolonged use of the device. Surely, they can be easily improved in future versions of the device, by: i) using a more flexible and durable elastomeric material or elastic nylon, ii) using less vivid colors, iii) substituting the Velcro closure with a strap provided with holes to improve versatility and comfort.

Overall, the usability of the leg extension measurement using the examination chair has been positively evaluated (Fig. 3(c)) The mean score was 8.7 both for the patients and the caregivers. Further improvements, might be addressed to involve a larger and more comfortable seating in alignment with the received feedback.

Regarding temperature measurements, the mean score was 10 for the patients and 8.5 for the caregivers (Fig. 3(d)), demonstrating high usability of the thermal camera and of the whole procedure. Only a few subjects encountered slight difficulties in centering the knee with the thermal camera, placed at the bottom of the phone.

The mean scores about the usability of the force balance device were 8.2 and 9.5 for the patients and caregivers, respectively (Fig. 3(e)). It is worth remarking that the three patients with the highest WOMAC index (i.e., 44, 30, 29) gave the lowest average score in this exercise (i.e., 6, 6.6, 6.6) since they experienced some pain and fatigue during the test (especially the one in which they should keep their balance on a single leg).

About usability results of the US imaging procedure, the mean score was 8.6 for the patients and 8.2 for the caregivers (Fig. 3(f)). Only two caregivers rated the feasibility of the overall procedure as less than 5 out of 10. Such a low rating is likely due to the multi-step complexity of this procedure, which involves the simultaneous use of different technologies (examination chair with the goniometer, wearable brace, software on the PC, and use of a US probe). Future improvements will focus on more comfortable, flexible and easy-to-position braces, as well as a more intuitive and faster procedure.

In Fig. 4, the average scores for each technology are reported for patients (Fig. 4(a)) and caregivers (Fig. 4(b)). Each dot in the boxplot represents the average score of each participant, for the different questions posed. As shown, the lowest average score was equal to 6.8 and associated with the evaluation of the wristband in the passive mode. For all other technologies tested by patients and caregivers, an excellent average usability score was reported, exceeding 8. The overall mean score resulted 8.7 for the patients and 8.8 for the caregivers.

Fig. 4.

Overall mean scores given by patients and caregivers for the usability of the tested technologies: (a) patients evaluation and (b) caregivers evaluation.

B. Results of Usability Tests: Clinicians

The results of the questionnaires filled out by the clinicians are shown in Fig. 5. The mean score was 8.5, showing an overall high usability of the clinician portal and a high ease of use of the interface. The lowest score recorded was equal to 6 for three clinicians. Such scores were associated with the intuitiveness of the home page and the ease of scheduling a task. For these functions, clinicians gave helpful suggestions and feedbacks to improve these aspects in future versions of the software.

Fig. 5.

Results of the usability assessment regarding the clinician portal.

Generally, the mean usability score are high for all categories (patient, caregiver and clinicians) showing that the proposed monitoring solution had a very high acceptance among all the involved subjects. The feedback received from questionnaires will be precious to implement minor but tailored adjustments to the devices and procedures to further enhance their usability. For certain tasks, such as range of motion measurement and balance monitoring, the lowest scores derived from patients showing poorer mobility and higher WOMAC indexes. In a future clinical translation, based on the patient's conditions, clinicians could be able to better personalize, by selecting and scheduling case by case the most appropriate activities to be performed, thus to not cause pain and avoiding risks for the patients.

The integration of remote monitoring for knee OA patients offers the promise of a brighter future, where individuals can actively participate in managing their condition and healthcare providers can make data-driven decisions to optimize treatment strategies [35], [36].

Indeed, the introduction of IoT paradigms in the healthcare sector will likely facilitate the use of emerging medical devices and technologies, improving their connectivity and their potential use also in a domestic setting.

Portable devices could potentially enable OA patients to become more active in the management of their condition and fulfil their interest in personalized health information. At the same time, clinicians would have access to precious frequent and up-to-date data that they would not be able to collect by their own allowing a better follow-up, streamline also their workload [37].

In addition to several technologies that are already validated and available in the state of the art (physical activity, postural equilibrium, skin temperature, leg extension analyses), in this work, a new strategy has also been introduced for a remote quantitative imaging based on ultrasound technologies. This new approach could represent a significant added value for a more reliable and precise diagnosis, providing objective information about cartilage thickness, tissue echogenicity, and the presence of synovial fluids as well as other US signs associated with the inflammatory process.

This study focused on assessing the usability and feasibility of different technological tools, to be used in a remote monitoring setting, as well as collecting feedback from the end-users. Indeed, understanding the user experience and acceptability of technologies is always crucial for a successful future implementation in remote patient care. The actual clinical relevance of these technologies or their combination in detecting changes in the OA healing process will require a dedicated clinical trial. In such a trial, the outcomes derived from the proposed portable diagnostic tools should be compared with the ones obtained through gold-standard methods (MRI or CT) at specific time points (e.g., 3, 6, 9, and 12 months). The collected data should be processed offline, individually or collectively, to extract informative metrics describing the healing progression, and validating them against reference results obtained from the gold-standard diagnostic tools.

IV. Conclusion

In this work, we evaluated the usability of different technologies whose combination may enable a remote, reliable and frequent assessment of knee osteoarthritis condition. These technologies were a mobile app, a wristband, an examination chair, a thermocamera, a force platform for balance recordings, a US imaging system, and a portal software dedicated to clinicians.

The average usability scores arising from questionnaires resulted very good both from patients’/caregivers’ and clinicians’ perspectives. The collected feedback will be used to further improve the hardware and software components, to make them suitable to a clinical setting.

Open Access funding provided by ‘Scuola Superiore “S.Anna” di Studi Universitari e di Perfezionamento’ within the CRUI CARE Agreement

Funding Statement

This work was supported by the European Union's Horizon 2020 Research and Innovation Programme under Grant 814413 through Project ADMAIORA (Advanced Nanocomposite Materials for in Situ Treatment and Ultrasound-Mediated Management of Osteoarthritis).