Patient engagement in digital health: a preliminary observation on user-centred and stakeholder feedback

Abstract

Design

Prospective cohort study

Setting

In this pilot study, we recruited patients from a secondary pain clinic in Boston, Massachusetts.

Participants

In this pilot study, we recruited patients from a secondary pain clinic within the Spaulding Rehabilitation network in Boston, Massachusetts, USA. We enrolled 37 patients who initially came in for a clinical visit with the principal investigator of the study. Of the 37 patients, 14 patients who continued to enrol/join after December 2024 received the ‘DigitalPulse’ to drive engagement.

Objectives

To present a roadmap for our efforts to contextualise engagement in our digital health technology study and showcase our attempts to incorporate an engagement approach inspired by the Method for Program Adaptation through Community Engagement. Building on this, we further incorporated continued feedback and revision beyond the prototype of the user-centred feedback form (‘DigitalPulse’) to include expanded stakeholders such as clinicians and research assistants.

Results

From these patients, we observed that our approach produced highly variable changes in engagement with slight increases at the group level.

Conclusion

From our observations, we have found that it is important to incorporate iterative refinements and expanded stakeholder involvement in designing patient-centred digital health tools to improve engagement. Overall, we report a process to address engagement and emphasise the need for continuous personalisation in digital health interventions.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• Integration of Method for Program Adaptation through Community Engagement (M-PACE) principles in digital health research.

• Expansion of the original M-PACE model to capture evolving patient, clinician and researcher preferences.

• Addresses the lack of literature involving patient preferences in digital health technology research.

• Limitations from pilot work: small sample size that limits statistical power and generalisability.

Introduction

Background/rationale

Digital health technologies (DHTs) have become a mainstay in healthcare delivery across the USA.¹ In the years following the COVID-19 pandemic, surges in telemedicine visits² and Food and Drug Administration authorisations for health monitoring in consumer-marketed wearable devices³ have reshaped communication between clinicians and patients. Recent interests in artificial intelligence for drug development, mHealth apps for mental health support and wearables for athletic performance have demonstrated the multifaceted implementation of DHTs across different fields of medicine.⁴ The movement towards DHTs has allowed for the acceleration of incorporating big data in improving patient outcomes.⁵ DHTs offer clinicians and researchers access to new, previously unattainable data streams such as gait and fall quantification or predicting depressive symptoms through smartphone app data.^{6 7} By gathering a more continuous set of information from patients, DHTs have the potential to allow clinicians to create more informed, real-time decisions when treating their patients.⁸ Coupled with the ability to monitor clinically relevant digital biomarkers in a cost-effective manner,^{6 8} DHT application in research and clinical flow has shown to be a promising direction for the future of healthcare.

Despite the potential of DHTs in medicine, a persistent issue in DHT implementation is the loss of user engagement. In a clinical context, engagement can be conceptualised as a fit-for-purpose data sufficiency—that is, the minimal frequency and regularity of inputs needed to support timely, clinically meaningful clinical inferences—rather than an open-ended goal of ‘more data’. In a study or programme, engagement is defined as the amount of interaction required by a participant to achieve the overall goal of a study or programme. While the goal of DHTs varies, studies have shown remarkably high attrition rates ranging from 26% to 86%.⁹,¹² Our previous work in older patients with chronic musculoskeletal pain showed that the clinical relevance of inferences from DHTs was limited by data missingness, which was related to user engagement.¹³ Our earlier work showed that out of the 77 recruited patients, only 39, or 49%, of patients completed the study.¹³ Among the most salient issues with high attrition is that poor engagement results in data missingness, which impacts clinical relevance.¹⁴ While collected data do not need to be 100% complete, they must be frequent enough to reliably detect changes in function over time and any systemic missingness must be appropriately addressed via statistical methodology. When disengagement does occur, it remains unclear whether it reflects functional decline or other outcomes of interest.¹⁴ As such, promoting user engagement is crucial despite the consistent struggle in DHT studies to maintain retention among its study population.^{12 15 16}

We therefore sought to explore ways to address the problem of low engagement in our population. As an outpatient chronic pain programme, we routinely evaluate and follow patients for ∼6 months. Therefore, we chose 6 months as our study’s clinical decision window. Given our past experience with poor engagement, we designed a pilot study based on the Method for Program Adaptation through Community Engagement (M-PACE) model.¹⁷ The M-PACE model was first developed and applied to adapt an evidence-based arthritis self-management programme for older adults in New York City. In the M-PACE model, a group of participants underwent a ‘prototype’ of the arthritis self-management programme and, afterwards, provided structured feedback on how various components (whether content or presentation of those components) of the programme might be improved. This feedback was incorporated into the prototype and the modified programme was then tested in pilot with both the original group and a new, previously unexposed, group of participants. Effectively, the M-PACE model provides a way to elicit and incorporate detailed feedback from programme participants that can guide changes.¹⁷ Other DHT studies have already shown that user engagement improves when the study allows for users to ‘co-design’ the study or when there is an end-user approach.¹⁸,²⁰ Specifically, DHT studies where participants have had the opportunity to influence study design changes have demonstrated high levels of engagement.^{19 20} To observe the difference in participant engagement after the implementation of patient feedback in our study design, we created a data feedback form (called the ‘DigitalPulse’) based on user suggestions.

In this report, we outline our efforts to clarify the problem of engagement within our specific population. We present the M-PACE inspired development of a digital tool aimed at improving engagement and share our preliminary observations from data collected from 37 patients in our outpatient chronic musculoskeletal pain programme. We then share our expanded involvement across multiple stakeholders, including clinicians (physicians and registered nurses), patients and research staff (research assistants (RAs)). Figure 1 presents the schema of ‘DigitalPulse’ development for workflow visualisation. Finally, we discuss further iterations of the digital tool and planned future work, designed to enhance the effectiveness and usability of the form for all involved parties. We emphasise the importance of continued, personalised revision within the M-PACE model and the value of multiple stakeholders in expanded involvement.

Figure 1. Schema for feedback form continued development. The schema illustrates the different sources of feedback we incorporated to develop and revise the feedback form. The prototype drew from initial patient and laboratory member feedback, and the continued revision drew from a wider scope of physicians, RAs, nurses and patients. RA, research assistant; RN, registered nurse.

Developing and piloting ‘digital pulse’

Efforts to understand engagement in existing study

Existing study design and enrolment

This current report has been conducted as part of a larger, National Institutes of Health-funded study seeking to define quantitative measures of functional status in patients with chronic pain conditions.¹³ We have previously reported our overall study design,¹³ but provide a brief overview here for the benefit of the reader.

First, a patient’s eligibility is assessed during their initial clinical visit with a doctor participating in the Pain Intervention and Digital Research Program (Pain-IDR). Patients who meet inclusion and exclusion criteria meet with a study coordinator who presents the study and, if the patient consents, registers participants to instal the Beiwe application on their personal smartphone. Beiwe is a research platform for smartphone data that allows researchers to collect patient data (further details below).²¹ As will be relevant later in the manuscript, Beiwe is contracted by the research community through the Beiwe Service centre and the fee is set by Harvard University as a function of how many patients (n) and over how much time (t). The eligibility criteria for this study are as follows:

1. Pain-IDR programme patient.

2. Age 18–120 years old.

3. Owns a smartphone.

4. Fluent in English.

5. Access to Wi-Fi internet and a valid email address.

6. Diagnosis of a chronic musculoskeletal pain condition.

7. No active substance use disorder per patient report or patient’s medical record.

After patient consent, we collected two types of data from the Beiwe application: active and passive. Active data included daily ‘microsurveys’ from the 29-item self-reported health questionnaire from the Patient Reported Outcomes Measurement Information System (PROMIS-29), a daily pain score (0–10) and an audio-response questionnaire that asked users about their day. We standardised PROMIS-29 question responses by adjusting the Likert scale responses to range from ‘1’ to ‘5’ with ‘1’ indicating the response suggestive of greatest health and ‘5’ indicating a response suggestive of the worst health. Passive data included accelerometer and Global Positioning System (GPS) data. Patients were enrolled for a 6-month study period and then given an exit survey at completion expressing their thoughts about the study/app experience.

Patient and public involvement

In our study, patients were involved at every stage of development, from study design to continual feedback. This study’s objectives largely stem from patient guidance and we continue to enhance our design/approach through participant suggestions. Patients regularly access the results of their specific study tasks.

Context

Initially, our original institutional review board (IRB) protocol did not have a monthly minimum completion requirement.²² After observing that some participants no longer responded to the microsurveys a few weeks after being onboarded on the study, we realised we recognised the need for a structured, formal way to dismiss people from the study given the budget limitations of the principal investigator (grateful to have received a K-award, but limited by the K-award budget applied to the Beiwe Service Center). We therefore amended our IRB protocol such that study participants with 1 month of data with less than 20% survey completion would be notified of this and asked to troubleshoot; participants with more than two consecutive months of data with less than 20% survey completion would be dismissed from the study. It was at this time we began analysing the extent of missing data in our sample.

By examining the last day in which a user provided a pain score, we operationalised study participation by their active engagement with the pain score survey. The decision to choose this metric largely relied on the nature of this data that required users to open the research app to provide a single score. Unlike the other ‘microsurveys’ that hosted multiple questions, this allowed us to track engagement as a binary measure (answer or no answer) per day. Using the last day in which we have a record of a participant providing us with a pain score, we were able to see the duration of study participation for each user. From this data, we found the attrition rate at the end of our 6-month study to be 62.2%. To visualise the dropout rate across study duration of all participants recruited from January 2022 to September 2023, we created a survivability plot shown in figure 2. This figure accounts for 82 participants. As evident in the survival plot, study participation declines consistently throughout the study duration. The sharp drop-off of active participants in the last week, between weeks 25 and 26, is explained by participant offboarding for completing the 180-day study period. While engagement seems to increase when examining the density plot, this increase is likely explained by the dropout of users who were not engaging with the Beiwe app. As disengaged users were being offboarded by RA intervention throughout the 6 months, only users who were continuously active on Beiwe remained near the end of the study. As such, engagement becomes artificially inflated by RA intervention. Therefore, this figure illustrates the extent to which the lack of user participation posed a significant barrier in data interpretation for our repository. In line with the current literature on DHT studies,^{12 15 16} we struggled to promote participant engagement and retention for the initial participants before ‘DigitalPulse’ implementation.

Figure 2. Survivability and density plot across Phase 1. The graph is gathered from 88 participants enrolled in the study from January 2022 to September 2023. The survivability plot showcases the percentage of participants still active on the study (y-axis) and the number of weeks from study enrolment (x-axis). The density plot showcases the completion rate of all participants as a percentage in a given week, displayed by the shade of red on the graph.

Development of ‘DigitalPulse’

User and team inquiry

In efforts to boost user engagement across our 6-month observational study, we sought to convert an internal data tracking form to a user-centred data feedback form. The internal data tracking form was an automated form developed for laboratory members to keep track of each user’s involvement with the app every week. Figure 3 illustrates the evolution of our data form, based on two main sources of revisional comments: patient feedback and laboratory meeting feedback.

Figure 3. Change from internal use to patient feedback form (‘DigitalPulse’). The leftmost form displays the internal form that notified laboratory members of a user’s activity in a given week. The figure displays the transition of this form from internal use to patient-facing content. GPS, Global Positioning System; Pain-IDR, Pain Intervention and Digital Research Program; PROMIS, Patient Reported Outcomes Measurement Information System; VAS, Visual Analogue Scale.

Patient feedback started from a single patient exit survey, performed by the RA. During the exit survey, the patient expressed interest in the ability for patients to see a summary of their data on a regular basis. The patient expressed that this feedback summary would have boosted engagement and interest in completing daily surveys. In response to this, we drafted several data tracking options and presented them to all patients enrolled in the study at that time. Table 1 provides the options presented to patients including the type of data available, the frequency of data feedback and the level of agency each user desired in customising their feedback form. More details about this table can be found in the next subsection. User preferences are marked with an ‘X’ under each option.

Table 1. User engagement options preference.

	RA suggested options
	Data completion			Data monitoring			Records		Customisable option
Participant	GPS tracking	Survey	Steps	Accelerometer changes	Pain survey responses over time	Survey responses over time	Personal best tracker	Weekly/monthly record analysis	Pick 3, see those	Dynamic selection
P1		X			X	X			X
P2				X	X	X
P3				X	X	X
P4	X	X	X	X	X	X	X	X	X	X
P5	X	X	X	X	X	X	X	X	X	X

The table showcases user preferences to options provided by the research assistant. Options that were selected by each user are marked by a ‘X’.

GPS, Global Positioning System; RA, research assistant.

Laboratory meeting feedback was largely in response to improving the user interface (UI) of the data feedback form. As presented in figure 3, the transformation shows an intermediate iteration of the data form that resembled the internal study team form. This form took into account user responses and preferences. During laboratory meetings, the team would suggest revisions to the UI or order of data presentation to improve the readability of the form. Through these two sources of revision, the internal data tracking form transformed to the final version.

With the creation of the final version, we amended our existing protocol and began sending all enrolled patients with a weekly summary of their information through email.

Rationale

The options listed in table 1 were determined from the feedback provided during the exit survey and laboratory meeting discussions.

• The data completion section showcases the completion percentage of patients on given active and passive data.

• The data monitoring section looks at the patient responses and gives an overview of the changes in responses over time.

• The records section examined individual participation records.

• The personal best tracker allowed for patients to view their most engaged week with the app.

• The weekly/monthly record analysis allowed for patients to view their participation trends over time.

• The customisable option provided patients with the ability to pick any three records that they would want to see at the beginning of the study.

• The dynamic selection aimed at allowing patients to pick the data they want to see every week.

The aim of this table is to showcase the high level of agency we sought to provide our patients in customising their feedback form.

Data and presentation rationale

Our data presentation strategy was informed by patient feedback and refined through laboratory discussions. To follow the M-PACE emphasis on user feedback, we limited laboratory discussions to preferences chosen by users rather than introducing new data metrics. We structured the PDF feedback form to present high-interest, broader summary items first, followed by more detailed data. This approach aligns with cognitive load theory, as discussed by Sweller,²³ which emphasises the role of schemas in processing information. By first establishing a ‘study schema’ with general study progression, we provide context before delving into specific results and completion rates. This design aims to engage patients and encourage them to review the entire report.

The current version of the feedback form presents three key data sets: (1) days since enrolment, (2) pain score responses over time and (3) survey completion rates. Other data components from table 1 were omitted to keep the form concise and user-friendly. While accelerometer and survey response trends were of high interest, displaying these changes effectively proved challenging due to the nature of our microsurveys—rotating daily questions—and limitations in phone-based accelerometer data collection. Future iterations explored later in this paper will incorporate these data streams.

Although including all relevant data is possible, excessive information can overwhelm users, leading to disengagement. Business psychology research has shown that information overload induces stress and frustration,²⁴ and in healthcare, excessive clinical data in electronic medical records can increase cognitive burden on physicians.²⁵ To prevent similar disengagement among patients, we condensed the feedback into a single-page PDF.

Data presentation structure

Top third: study progression

Given that one of the most frequent patient inquiries concerns their study progress, we prioritised this information at the top. Participants, particularly those in longer studies, often lacked visibility into their progression due to limitations in the Beiwe platform. By providing a weekly update on study milestones, we offer patients a clear overview of their progress.

Middle third: pain score responses

Patients expressed strong interest in tracking their pain score progression, making it a key inclusion. Since our study focuses on individuals with chronic musculoskeletal pain, this measure is particularly relevant. If no data are available, an error message prompts users to engage with the app to begin tracking their pain scores.

Bottom third: data completion metrics

The final section provides a weekly summary of patients’ engagement with active (audio and PROMIS surveys) and passive (GPS tracking) data collection. It displays microsurvey completion rates and GPS data availability as percentages. This inclusion, based on patient preferences (table 1), also serves as a troubleshooting tool. If GPS data collection falls to 0%, an error message alerts patients to check location settings. This feature helps both participants and Pain-IDR RAs identify and resolve issues efficiently.

By structuring the feedback form in this way, we balance comprehensiveness with usability, ensuring patients receive meaningful insights without information overload.

Pilot testing of ‘DigitalPulse’

Study demographics

After implementing the user-centred form inspired by patient feedback and laboratory discussions, we sought to observe differences in user engagement. To measure this, we gathered engagement data from 37 users enrolled from February 2024 to February 2025. Of the 37, 23 provided data before the implementation of the user feedback form, while 14 provided data after. The data feedback form was implemented in December 2024. Of the 14 users, 2 provided data for 1 week after form implementation due to reaching the 6-month study completion period. Four users (excluding the two users who contributed for a week) provided data before and after form implementation. Specific study demographics for the 37 patients can be found summarised in online supplemental table 1.

Engagement observations

We sought to observe engagement differences from before and after the implementation of our ‘DigitalPulse’ through two measures: observing changes at the group level and individual level. Group-level changes were calculated as a summary statistic of all users who were involved at each stage of the study (before and after ‘DigitalPulse’ implementation). Individual-level changes were calculated by examining each user and their specific differences in engagement. These measures were taken from available data 3 months after implementing the feedback form to study design.

At the group level, figure 4 visualises the changes in engagement we observed across the two time periods. As noted above, there is an overlap of four users across the two time periods. As seen in the graph, there is a minimal, slight increase in engagement with the introduction of the feedback form. Notably, participant engagement in the audio question increased by 6 percentage points as opposed to the regular survey questionnaire that increased by approximately 1.5 percentage points.

Figure 4. Preliminary group-level change in engagement after feedback implementation. The bar graph showcases differences in user engagement from pre (before ‘DigitalPulse’) and post (after ‘DigitalPulse’) across the standardised PROMIS-29 questionnaire (survey) and the audio-response questionnaire (audio). Completion was quantified by examining the number of days each user provided responses for. The y-axis displays the completion as a percentage across all participants within the two stages of the study.

At the individual level, figure 5 visualises the changes in engagement we observed across the two periods, specifically for the four users who were on the study before and after the introduction of the feedback form. We can observe from this figure that the changes in engagement are highly variable. For instance, participants P3 and P4 saw an increase in engagement after the introduction of the ‘DigitalPulse’,” while the opposite was observed for P6 and P5.

Figure 5. Preliminary individual change in engagement after feedback implementation. The bar graph shows differences in user engagement using the same quantifications and denotations as figure 3. Specifically, this graph examines the four users who were active before and after the introduction of the ‘DigitalPulse’. For each user, the two bars on the left represent the pre (before ‘DigitalPulse’) stage of the study while the two bars on the right represent the post (after ‘DigitalPulse’) stage of the study. Participant numbers (P*) are continued from table 1.

Examining the changes in engagement across the two levels, we can observe that the introduction of the ‘DigitalPulse’ has a highly variable, slight positive impact on user engagement. Users who were involved in constant discussion throughout the form development, such as P3 and P4, showed a notable increase in engagement. Meanwhile, users such as P5, who responded once throughout development, and P6, who did not engage with development, did not show any improvements in engagement (P6 is not listed on table 1). While this difference might simply reflect a broader difference in user motivation (that is, users who are more engaged are more likely to respond to RA inquiry and complete tasks irrespective of ‘DigitalPulse’ implementation) or participant time availability, the observed increase across the group-level comparisons suggests there was an improvement across all users. Therefore, as supported in other M-PACE incorporated studies,²⁶,²⁸ we observed that users who were involved in the study design showed greater engagement with study tasks than users who were not responsive to the feedback form design.

Continued revision and expanded involvement

Continued patient involvement

In efforts to continue to drive engagement, we continuously sought to revise the ‘DigitalPulse’. Table 2 outlines the feedback we received after implementing the first iteration of the user-centred form. The feedback was framed in response to a brief survey asking three questions: (1) Do you like the PDF form? (2) Do you think the PDF form is useful? (3) Are you more likely to participate in our study because you can see your data? Participants had the option to expand on any responses (beyond the yes/no nature) if they wanted to share more information. The interview responses highlight participant favourability to the feedback form and its usefulness for participants to track their provided information. However, the interviews showed mixed responses when asking participants if they believe viewing data would drive engagement. Notably, patient number six on table 2 expressed that ‘the forms are definitely more useful than not…doesn’t change how I engage with the study’. This suggests that for some patients, the implementation of the feedback form does not necessarily change the way they interact with the study, but it was a useful feature for the repository.

Table 2. Continued patient feedback.

No.	Do you like the PDF form?	Do you think the PDF form is useful?	Are you more likely to participate because you see your data?	Other feedback?
1	Yes	Yes	Yes	‘No, not now. Maybe when I can watch my step count [on the form]; that will really tell me something!’
2	No	Yes	No	‘I just think the study is asking the same question about previous 7 days within 7 days. [On the form] instead of ‘past 7 days’ maybe make it 24/48 hours’.
3	Yes	Yes	No	‘Question [and the form] should ask for that day and not past 7 days if daily completion’.
4	N/A	Yes	N/A	N/A
5	Yes	Yes	Yes	N/A
6	Yes	Yes	No	‘This data is probably more useful for research, but it is good to know and helpful for patients. The forms are definitely more useful than not, but it doesn’t change how I engage with the study’.
7	Yes	Yes	N/A	N/A

The table showcases patient responses after a brief survey interview with active patients. Fields filled with ‘N/A’ represent fields that have no answers.

We have further provided the option for individual participants to select which data they would like to see on their PDF form (eg, would they prefer step count vs cadence; see online supplemental material 2). In a separate correspondence with all active participants, users were given the option to opt into receiving step count information. While we initially omitted this preference in the initial feedback form, users continued to express strong interest in this feature. Out of the 12 users who were active after ‘DigitalPulse’ feedback beyond the initial week, 7 users opted into receiving step count information. We have decided to incorporate this tracker with a warning message that the accelerometer readings can be incomplete. Online supplemental material 2 displays the changes in the revisional form. As evident in this refinement, it was clear that patient preferences were driving more personalised features in the ‘DigitalPulse’.

In response to the feedback we received in table 2 and the users who opted into receiving the expanded feedback form, it became increasingly evident that different patients preferred different information. While some patients preferred minor alterations (such as changes in the way the data is displayed or for the inclusion of step count changes), others did not feel these changes were necessary. As such, the continued revision process suggested the need for greater personalisation and customisation to meet each participant’s need.

Feedback scope extension: clinician and research staff involvement

Beyond participant feedback, we sought to expand our feedback base to involve other clinicians and research staff. To illustrate and simplify the iterative process to develop the ‘DigitalPulse’, we have included a development schema (figure 1) that showcases the steps we took for form development. Figure 1 illustrates the steps taken for initial form design and the expanded scope of stakeholders that we have included for continual revision. The schema illustrates the ongoing improvements that are made to the form in response to feedback from different sources including clinicians and RAs.

While the M-PACE model designed by Chen et al discussed the importance of participant-centred feedback rather than researcher and practitioner suggestions, we chose to involve other clinical and research staff (1) to accurately capture feedback from other parties that are involved in physician–patient interactions; (2) we sought to gain perspectives that can further improve our feedback form; and (3) because future expansion of our project will depend on the buy-in of the larger clinical and research staff.

Table 3 outlines the feedback we have received in response to a brief survey formatted similar to the patient-facing survey. The survey focused on the practicality and UI of the ‘DigitalPulse’ with the opportunity to expand on any specific data points. Interestingly, the expanded feedback we received from different sources closely aligned with the nature of each individual’s contribution to research. While the patient feedback we received largely focused on the user experience of the forms and the study design itself, RAs and clinicians largely focused on the effects of this form on research integrity and clinical utility.

Table 3. Clinical stakeholders feedback.

No.	What is your role?	Do you think data feedback is useful?	Are these types of information useful?	What might you add to this?	Is this visually appealing?	How would you improve this?
1	RA	Yes	Yes	‘Change the wording at the top; ‘You have made it to Week 24’ is kind of morbid.’	Yes	‘Visually, this is good.’
2	RA	Yes	Yes	‘I don’t think the GPS information or the data type is particularly useful. If they’re not doing it they won't care about it.’	Yes	‘Audio graph color should be changed since it matches the background but overall color is good.’
3	RA	Yes	No	‘I think the pain score could be confounding, I also think data here isn’t too useful if I were a patient as it can influence my interaction with the app. I would want this as a summary statistic at the end [of the study].’	No	‘This can definitely be improved. I think for many people who won’t scrutinize this, this is good. But for someone who really looks into the data, this can be improved.’
4	RA	Yes	Yes	‘Take out days on study graphic, its confusing. Move completion rate graph to the middle.’	Yes	‘Make the colors darker on the graph.’
5	RA	Yes	Yes	‘Make the datatype graph less scientifically worded, change the max limit of the y-axis on the datatype graph.’	Yes	‘Present grater granularity, time of the day can change survey responses.’
6	RA	Yes	No	‘Main concern about confounding by looking at past pain scores.’	No	‘The form looks good overall, I just think the design can be improved, rounder edges, better colors.’
7	RN	Yes	Yes	‘Make it more clear that this data is for each individual.’	Yes	‘Synopsis of research purpose at the top would be helpful.’
8	Physician	Yes	No	‘If [the data is] complete capture, then [it is] useful. If utility is there in reference, then it would be good. [The form] motivate patients to meet tangile metrics. (However,)the form can be disruptive, and can be cumbersome if they keep getting it. Pain score is very ‘in-the moment’ and is not as accurate in a complete capture. Fall prediction data would be useful. Life-changing for people who is about to fall.’	Yes	‘Flatten the peaks for the line graphs, its too rough and can show easeness throughout the day.’
9	Physician	Yes	Yes	‘Anything else encouraging, such as sharing the actual % increase in steps or % improvement in other areas.’	Yes	‘I’m not sure if looking at pain scores are beneficial or have a negative effect. I remember hearing that biopsychosocial pain programs (such as functional restoration programs) try to not emphasize pain scores. We’re still capturing that data, but wonder if it would be better to only share the progress in other areas.’
10	RN	Yes	Yes	‘(For the survey completion) patients often complain about uneeded questions. Patients don’t find the utility of the PROMs; questions can be limited and not so long.’	Yes	‘Form looks great! No complaints.’

The table showcases different stakeholder responses after a brief survey interview. Fields filled with ‘N/A’ represent fields that have no answers.

RA, research assistant; RN, registered nurse.

For instance, one RA expressed that the ‘pain score could be confounding…[if I were a patient,] it can influence my interaction with the app’. This concern was raised by multiple RAs, and it is an important point of discussion. Studies investigating pain responses have suggested that pain is largely subjective and is easily influenced by past experiences.^{29 30} Furthermore, there is evidence that viewing information on pain from other people substantially modifies subsequent perception of pain.³⁰ Because of this, it is not an unsubstantiated concern that releasing past pain scores can have an inadvertent effect on participant responses and subsequent study results for the pain scale. This variable must be accounted for in future analyses of research data.

Interestingly, physician feedback in table 3 expressed similar concerns for the pain scale, but concentrated on the effectiveness and utility of the pain scale. Specifically, one physician expressed that pain scores are very ‘in-the-moment’, while another was unsure if viewing this trend was beneficial or harmful. In either case, physician feedback largely focused on the utility of this information for patient wellness. In contrast to RA feedback that placed a microscope on research integrity, physician feedback was interested in the clinical application of the data and form. It is clear that through expanded involvement, we were able to gather varied insight from individuals that are not solely end-users.

Discussion

Engagement remains a persistent challenge in digital health research. The goal of engagement is not to allow ‘as much data as possible, forever’, but instead, to allow fit-for-purpose data towards a clinical or programme goal. In our case, patient engagement was designed to serve a 6-month clinical decision window during which most patients were seen at our outpatient chronic musculoskeletal pain clinic. Observing low engagement in our study, we adapted the M-PACE framework to actively involve participants, clinicians and RAs in iterative feedback loops. This led to the development and testing of ‘DigitalPulse’, a weekly report designed to enhance engagement by delivering personalised feedback on functional status. To preface, our small sample size of four patients severely limits our interpretations of the results of the study. Our desire here is to discuss the observations of these preliminary findings from our cohort. While overall group engagement remained largely unchanged, individual-level outcomes showed promise, reinforcing the need for tailored, dynamic engagement strategies.

Our limited findings highlight the interpersonal nature of engagement, as increased participation was observed with higher responsiveness to RA outreach (figure 4). Notably, the two engaged participants in figure 5 showed an increase in engagement, whereas the other two showed lower responsiveness. Additionally, preferences for the user feedback format varied: while some participants valued step count data, others did not express interest. In addition, patients varied in their opinions on whether they thought the form would promote engagement or not. In the context of emerging consumer engagement research with digital tools, it has become increasingly evident that greater personalisation drives user attention.³¹,³³ The produced user variabilities in our study underscore the need for customisable engagement strategies, where content and communication methods align with individual patient preferences.

A key contribution of our study is the expansion of the M-PACE model to incorporate a broader range of stakeholders, including patients, clinicians and RAs (table 3). Stakeholder feedback revealed contrasting priorities: RAs raised concerns about potential bias introduced by pain score tracking, clinicians emphasised the clinical utility of trend data and patients focused on the user experience. In a recent systematic review examining consumer behaviours through digital channels, the author highlighted a growing trend in using interdisciplinary perspectives and findings from diverse fields to inform marketing strategies.³⁴ In the context of this study, the diversity of perspectives from multiple stakeholders highlights the importance of iterative refinement in designing patient-centred digital health tools.

Strengths and limitations

Our approach has several strengths. Unlike previous M-PACE applications that focus on theoretical modelling before pilot testing,¹⁷²⁶,²⁸ our study integrates M-PACE principles with real-time feedback to iteratively refine engagement strategies. Furthermore, while modified M-PACE models have been deployed in medical education,²⁷ professional development²⁸ and sexual health education,²⁶ there is a lack of literature involving patient preferences specific to DHT research. By conducting stakeholder interviews post implementation, we captured evolving patient preferences, a critical aspect often overlooked in digital health research and one that would not have been possible in the original M-PACE model.¹⁷ The main limitations of our study are those common to all pilot work: our small budget limits our findings to a small sample size, thus limiting statistical power and generalisability. We note that our sample size, including clinician and RA feedback, is very much in line with past reports deploying the M-PACE model.^{26 27} However, due to our small sample size across the two measured time points, we cannot claim that there is statistically significant evidence that the ‘DigitalPulse’ increased engagement. Additionally, the potential for confounding effects—such as patients altering responses based on past pain scores—warrants careful consideration in future studies.

Looking ahead, our findings emphasise the need for continuous personalisation in digital health interventions. While engagement remains a challenge, the variability in patient preferences suggests that a one-size-fits-all approach is insufficient. Future research should explore scalable methods for dynamic user customisation and evaluate the impact of real-time modifications on long-term engagement. Larger, multisite studies that investigate the effects of personalisation can further illuminate the statistical significance of our approach in increasing user engagement as well. Importantly, further health disparity research into the practical implications of measuring user engagement (eg, structural limitations, such as time and accessibility, that prohibit user engagement) should be conducted. As DHTs continue to evolve, fostering patient participation through adaptive, user-driven feedback mechanisms will be essential for maximising their clinical utility.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.