Table 4.
Effectiveness of support measures, recruitment and compliance barriers, technical challenges, and lessons learned in the conduction of the BarKA-MS study and scalability potential.
| # | Key words | Description | Lessons learned | Concept reflected in following ISAT axes | Scalability potential Assessment (based on ISAT scale) 1) Not at all 2) To a small extent 3) Somewhat 4) To a large extent |
|---|---|---|---|---|---|
| Effectiveness of support measures | |||||
| 1 | Comprehensive onboarding | At the baseline meeting, the on-site study coordinator at the rehabilitation clinic equipped the study participants with a Fitbit, helped them download the Fitbit application onto their mobile phone, and helped with the first log in to their Fitbit study account. Next, the study coordinator created a new account for the study participant on the study platform (RMIS), checked that they could receive emails on their phone, and then completed the baseline questionnaire with them. This initial onboarding session lasted approximately one hour. | It was important to take time to discuss the study in detail with the participants and answer all their questions as the study involved quite a number of digital tools. | ISAT 1: “fidelity and adaptation” | Somewhat |
| 2 | Personal relationship | The on-site study coordinator had regular contact with the study participants and made sure that everything was going well during the rehabilitation phase. Also, the digital device expert team was always reachable for study participants by phone. | Regular contact with the study participants strengthens their commitment to the study. | ISAT 1: “fidelity and adaptation” | To a small extent |
| 3 | Research team support for on-site study coordinator | The on-site study coordinator performed the study participant onboarding but initially had limited experience in using digital tools. Despite this, everything worked well due to an initial knowledge transfer by the research team and a smooth hand-over to other experts in the research team for more complex technical questions. | Briefing the on-site study coordinator before the study start and being at their disposal (e.g., per phone) in case of a (technical) issue or questions was essential for the smooth conduction of the study. | ISAT 1: “fidelity and adaptation” and ISAT axis 4: “implementation infrastructure” |
To a large extent |
| 4 | Active monitoring | Research team closely monitored the participants. They controlled that the questionnaires were filled in every week and that the Fitbit were collecting data. If a questionnaire had not been completed or if Fitbit data were not being collected, the research team would pro-actively contact the study participants. | The collection of high-quality and complete data requires active monitoring and is, therefore, resource consuming. | ISAT axis 4: “implementation infrastructure” and ISAT axis 5: “sustainability” | Somewhat |
| 5 | Monitoring device satisfaction and adherence | Wearable devices, especially the Fitbit Inspire HR, was appreciated by the study participants, who also reported increased motivation to be physically active. | The convenience and discretion of the Fitbit Inspire HR and its immediate feedback were valued by the study participants and seemed to have led to very high wear adherence. | ISAT axis 2: “reach and acceptability” | To a large extent |
| 6 | Offering free Fitbit device as an incentive | Some people participated in the study mainly because the Fitbit was offered free-of-charge at the end of the study. | Offering a reward to the study participants can increase the recruitment fraction. | ISAT axis 4: “implementation infrastructure” | Somewhat |
| 7 | Involving significant others for peer support | The study participant's partner was sometimes of great support e.g., for study participants not having German mother tongue. | A study participant's partner can be an additional and valuable support for the participant in terms of study compliance. | Not applicable | To a small extent |
| Recruitment and compliance barriers | |||||
| 8 | Recruitment challenges | Conducting the study in a rehabilitation clinic led to a limited enrollment fraction (33%) due to the exclusion of persons with very advanced gait impairments. | The PwMS attending a rehabilitation stay tend to have a more advance disease stage. By choosing a different study implementation setting, a higher recruitment fraction could be reached. | ISAT axis 3: “delivery setting and workforce” | To a small extent (limited influence on source population) |
| 9 | COVID-19 | The study took place in 2021, during the COVID-19 pandemic, which led to delay in participant recruitment, as less PwMS attended the clinic for a rehabilitation stay. | The recruitment of PwMS was conditional on PwMS entering a rehabilitation stay. Again, by choosing a different study implementation setting, more PwMS could be reached and thus the recruitment speed could increase. | ISAT axis 3: “delivery setting and workforce” | Not applicable |
| 10 | Seasonality | During summer, less persons attended the rehabilitation clinic. It was thus more difficult to recruit participants during this period. | In planning the study, seasonality should be taken into consideration e.g., during the main holiday periods (in summer and at Christmas) people are less prone to attend a rehabilitation clinic. | ISAT axis 2: “reach and acceptability” | Somewhat (limited influence on clinical workflows and processes) |
| 11 | Digital literacy | Persons with less digital literacy required closer support. | When recruiting study participants, make sure that they understand well enough the technologies used and provide easy-to-understand instructions. This could be an inclusion criterion. | ISAT axis 2: “reach and acceptability” | Somewhat (exclusion could lead to limited generalizability) |
| 12 | Language barriers | Study participants with a different mother tongue than the one in which the study was conducted required more support. | If the person is not fluent in the languages in which the study is performed, this will pose a recruitment obstacle or may lead to a higher participation burden for study enrollees. Additional support (e.g., by relatives or study personnel) could be considered. | ISAT axis 2: “reach and acceptability” | Somewhat |
| 13 | Setting change | Survey completion fraction at home was lower than in inpatient settings. The participants’ return home was always a key moment because they were not so closely followed anymore, and the daily concerns were again present. | Sending a text message to participants a few days after their return home to remind them that they can reach us anytime if they need to could potentially increase their compliance to the study. | ISAT axis 3: “delivery setting and workforce” | To a large extent |
| 14 | Text messages vs. phone calls | Study participants would rarely pick up the phone, especially when it is an unknown number, but they would respond to text messages. | The participants in our study (solely PwMS) seemed to feel more at ease in responding to text messages or to calls that were scheduled in advance. | ISAT axis 2: “reach and acceptability” | Somewhat |
| 15 | Forgotten Passwords | Some study participants lost the password to their RMIS account or went on holiday without it. | The provision of a study “visiting card” with the login information could be provided to the study participants, so that they can easily keep it in their wallet. Participants should be reminded to also take this card on holiday. | ISAT 1: “fidelity and adaptation” and ISAT axis 5: “sustainability” |
To a large extent |
| 16 | New mobile phone | When a study participant changed their phone during the course of the study, they had to re-install the Fitbit application, log in again, and reconnect the application with their Fitbit tracker. Not every study participant would be able to do this on their own. | Manuals and instructions (e.g. short videos) could be provided to advise participants on how to set up a new phone. | ISAT 1: “fidelity and adaptation” and ISAT axis 5: “sustainability” |
Somewhat |
| Technical challenges | |||||
| 17 | Incompatible mobile phone operating systems | Certain mobile phones of the Huawei brand did not have an apps store, which hindered the download of the Fitbit application and thus participation in the study. | Be aware that not all mobile phones have an app store. | ISAT axis 4: “implementation infrastructure” | Not at all (unless phones are provided by study) |
| 18 | No email access on mobile phone | Some study participants had no access to their emails on their mobile phone. In such a case, the contact person in the rehabilitation clinic had to link the email account of the study participant to an email application on the participant's phone. This was not always successful because e.g., the participant did not have their email account credentials available. | Instruct participants to bring along their login credentials to enable on-site installation and set-up of necessary applications by study supporters. | ISAT 1: “fidelity and adaptation” and ISAT axis 4: “implementation infrastructure” |
Somewhat |
| 19 | Provision of a notebook during the rehabilitation stay | For some participants, it was difficult to complete the weekly questionnaires on their mobile phones, e.g. due to small fonts or hand control impairments. | The provision of a notebook, at least for the initial inpatient phase, should be considered. | ISAT axis 4: “implementation infrastructure” | To a large extent (for inpatient setting) |
| 20 | Login problems with survey platform | Sometimes during the rehabilitation stay, because of login problem to the RMIS survey platform or other difficulties to fill in the survey digitally, surveys were filled in on paper. However, afterwards the data had to be manually entered by the research team on the RMIS platform, which was time consuming with the added risk of errors in entries. | For different reasons, study participants were not always able to fill in the surveys digitally. A fallback option such as paper questionnaires could be considered, which may lead to a subsequent higher workload and potential data entry errors due to manual entry. | ISAT axis 4: “implementation infrastructure” | To a small extent |
| 21 | Email bounces | For unclear reasons, the RMIS platform could not send emails to certain email addresses provided by the study participants. | Asking the study participants for an alternate email address or other contact information at the beginning of the study could be useful in different situations. | ISAT axis 4: “implementation infrastructure” | To a large extent |
| 22 | Replies to automatically generated emails | Participants would sometimes reply to automatically generated emails from the RMIS platform to ask for help or mention a problem, which led to delays in their resolution. | When using automatically generated emails, a note should be added at the end to indicate not to reply to this email address. | ISAT axis 4: “implementation infrastructure” | To a large extent |
| 23 | Automatically timed invitation and reminder emails | Manually managing email invitations and reminders is labor intensive, with the burden and complexity growing exponentially with each additional participant. The setting transition also needed to be reflected in survey delivery: the first home setting survey differed from normal weekly surveys and had to be triggered manually. | The study platform ideally contains all necessary features for a proper and efficient study conduction, including automated timing of survey and invitation emails, as well as allowing for different study phases. | ISAT axis 4: “implementation infrastructure” | If manual: To a small extent |
| 24 | Extension of the rehabilitation stay in progress | The rehabilitation stay was sometimes prolonged during the stay by the health insurance. These changes were communicated to Kliniken Valens, who then had to inform the research team of the UZH. This made the weekly survey mailing dates planning challenging. | Rehabilitation stays can be extended and the study platform used should be flexible enough to account for such changes. | ISAT axis 4: “implementation infrastructure” | If manual: To a small extent |
| 25 | Synchronization issue with the Fitbit | The Fitbit did not always synchronize automatically with participants’ mobile phones. When this happened, the participants had to manually synchronize the Fitbit every few days for the entire study period. | Fitbit devices and synchronization status require regular monitoring (e.g., as available in the Fitabase data platform). If necessary, the study participants should be informed and instructed on how to manually synchronize the Fitbit with the Fitbit application. | ISAT axis 4: “implementation infrastructure” | To a small extent |
PwMS, people with MS; RMIS, Research Management Information System.