Optimizing an Outpatient mHealth Intervention for Children with Burns: A Convergent Mixed-Methods Study

Abstract

Burn injury is one of the most common traumatic injuries in childhood. Fortunately, 90% of pediatric burns may be treated in the outpatient setting after appropriate burn triage. Patients with burns face significant geographic disparities in accessing expert burn care due to regionalized care. To aid patients and their families during acute outpatient burn recovery, we developed a smartphone app, Telemedicine Optimized Burn Intervention (TOBI). With this app, we aimed to increase access to care by allowing secure, streamlined communication between patients and burn providers, including messaging and wound image transfer. The purpose of this study was to systematically evaluate user feedback to optimize the patient and provider experience. TOBI was evaluated using a convergent mixed-methods approach consisting of qualitative semi-structured interviews and quantitative measurements of app usability via the mHealth App Usability Questionnaire. Participants included 15 caregivers of pediatric patients with burns who used TOBI during treatment and ten burn providers. Users found TOBI to be a highly usable application in terms of usefulness, ease of use, satisfaction, and functionality. Qualitative data provided insight into user experience, satisfaction and preferences, difficulty navigating, usability and acceptability, and potential improvements. Although most users were highly satisfied, improvements were needed to optimize the burn app. We systematically made these improvements before we released TOBI for routine patient use. This study uncovered helpful recommendations for app improvements that can be generalized to other mobile health apps to increase their appeal and adoption.

Burn injury is a leading cause of mortality and morbidity in the United States. Each year, approximately 120,000 children present to emergency rooms with burn injuries, resulting in many referrals to dedicated burn centers.^{1, 2} Fortunately, up to 90% of pediatric burns can be managed effectively on an outpatient basis with at-home wound care or advanced wound dressings.^3–5 Over the past 20 years, advances in outpatient burn care have enabled providers to shift away from daily dressing changes to less frequent dressing changes using silver-impregnated dressings.⁶ This approach has improved pain associated with dressing changes and decreased the need for inpatient hospitalization. Wound care can be provided by the patient’s caregivers at home with close monitoring in the outpatient burn clinic.

Patient access to expert burn care is a critical issue. Telemedicine, or the remote delivery of healthcare using technology as a bridge, has the ability to improve patient access to care over geographic distances. This is particularly acute in burn care with the regionalization of burn centers that has occurred over the last 30 years.⁷ Less than half of the US population lives within a 2-h drive of a verified burn center,⁸ with even more geographic separation in other countries. Further, significant financial disparities impact rural and medically underserved areas, which are overrepresented in the population of patients with burns, particularly in children.⁹ Advances in telemedicine have provided ample opportunity to advance the delivery of burn care over the past 30 years. Starting with the burn triage system in the 1980s, burn centers could remotely evaluate patients with store-and-forward image evaluation^{7, 10–13} to help improve over- and under-triage in the burn transport system. Further evolution of burn telemedicine occurred over the past 20 years, with various iterations of a hub-and-spoke model of delivering burn care. In this model, burn professionals could treat patients remotely over synchronous video visits^14–18 or asynchronous store-and-forward image evaluations. These remote visits enabled patients to be treated in their own communities at authorized sites of origination, such as doctor’s offices or nursing homes.

The use of mHealth technology offers a promising approach to outpatient burn care,^{11, 19–21} particularly to support parent-administered dressing changes at home. Firstly, the assessment and treatment of burn injury and wounds are ideally suited for the use of telemedicine.^{7, 13, 22, 23} In fact, burn surgeons were early adopters of the hub-and-spoke telemedicine systems of the 1990s.²⁴ Further work in this area was codified by the American Telemedicine Association Guidelines for Teleburn in 2017.²⁵ Further, cell phones are utilized by ~95% of adults in the US, with widespread dissemination in all groups irrespective of race/ethnicity, socioeconomic status, or urban–rural classification. To date, mobile health applications for burn care have largely been limited to triage between referring hospitals and burn centers, with variable impact.¹³ A recent pilot study demonstrated mobile phone virtual reality to improve patient-reported pain scores during dressing changes for pediatric burn patients at home.²⁶ Mobile phones have surpassed the minimum requirements needed for burn assessment²⁷ but rigorous scientific validation aimed at addressing outcomes has not been achieved.²⁸ A systematic review of smartphone applications in burns found 31 apps in the Google Play Store and 29 in Apple’s App store. The apps fell into four groups: calculators, information apps, book/journal apps, and games. No mHealth apps have been designed to assist patients and families in trating serious burn injury in outpatient settings.

Due to these geographic and financial barriers to patient access, our pediatric burn center aimed to improve patient access to expert burn care and enhance communication between patients and their burn providers. To this end, we created a smartphone app called the Telemedicine Optimized Burn Intervention (TOBI). Initially created to improve burn care access to rural patients and their families, this app allows for the provision of expert clinical burn care in the patient’s home through text messaging, store-and-forward image transfer, frequently asked questions (FAQs), and instructional materials.

In this convergent mixed-methods study, we sought to systematically evaluate end-user feedback from both patients and clinicians who used a mobile health (mHealth) application during the acute phase of burn injury. Evaluation of app usability and quality can yield important information to optimize consumer-driven apps and promote user satisfaction and adherence. We then used stakeholder feedback to guide app refinements and optimize the patient and provider experience. Herein, we describe the results of qualitative semi-structured user interviews and quantitative usability and satisfaction data gathered from caregivers of pediatric burn patients who used TOBI during treatment of their child’s burn injury.

METHODS

Burn App

TOBI, initially called the “TeleBurn,” was developed in 2018 to provide expert clinical burn care in the home and enable burn physicians, advanced practice providers, and nurse specialists to monitor and intervene throughout treatment of a burn wound (Figure 1). The app allows the parent–child dyad to interface with the burn team through text messaging and image transfer in an asynchronous manner. Also, patients and caregivers can see instructional videos for dressing changes depending on the location of the burn and find answers to FAQs. Burn providers used a provider-facing portal to access patient communication and images. TOBI was developed by an interdisciplinary team including experts in pediatric burn care, health communication, nursing, public health, biostatistics, information technology, and clinical psychology. The TOBI app is HIPAA (Health Insurance Portability and Accountability Act) compliant, which has been a key feature from its inception. Because there is protected health information (patient photos) contained within the app, we have been required from inception to maintain HIPAA compliance in all apps developed by the in-house software development team. TOBI has security built into the mobile application (required secure user password, data encryption, etc.) and into the backend (firewalls, server security, SSL data transmission, database backups, role management security, etc.).

Figure 1.

Screenshot of the home screen of the original TeleBurn (A) and new, patient-optimized Telemedicine Optimized Burn Application (TOBI) (B, two right panels). Systematic stakeholder feedback from caregivers of pediatric patients with burns and burn providers was used to refine the TOBI app and improve usability and provider workflow.

Study Design

We conducted a convergent mixed-methods study at a single center to optimize an existing mHealth burn intervention. This study included burn stakeholders and consisted of semi-structured interviews and a validated questionnaire. All activities described in this study were approved by the Institutional Review Board. The study activities were conducted either on campus or remotely via secure videoconferencing software. In response to COVID-19, we pivoted from an in-person-only format to support both in-person and virtual appointments. Both the principal investigator and doctoral-level study coordinator conducted mock study interviews and assessments prior to the study to maintain consistency and reduce interviewer bias. All research data were stored in a password-protected REDCap® database.

After recruitment and screening for inclusion criteria, study staff obtained written or electronic informed consent from caregivers and provided an acknowledgment form outlining the study to burn providers. The study staff then allowed about 10 min for participants to re-explore TOBI and access each of the app/portal’s functions. This process was done because not all study participants had recently used TOBI. The interviewer then conducted and recorded a semi-structured interview and administered questionnaires. The questionnaires included a demographics form and measures about caregiver satisfaction with and usability of the app. Caregiver participants were seen individually in a private office or remotely via secure videoconferencing during one visit. Provider participants were interviewed in-person in groups of three to four people during one visit. Caregiver participants received a $50 gift card and burn providers received a $10 gift card for study participation.

Recruitment

Recruitment for this study occurred from both the patient–caregiver and provider populations. Caregiver participants were recruited through an institutional burn registry of pediatric patients with burns. Participants in the registry were cross-referenced with a password-protected database of patients who used TOBI during the treatment phase of their child’s partial thickness burn injury at our medical center over the past 12 months. Potential caregiver participants were recruited via telephone, informed about the study and a gift card for participation, invited to participate, and scheduled for an in-person or virtual consent and interview/assessment, if interested. Burn provider participants were recruited from the pediatric and adult burn teams at our academic medical center by email and scheduled for one of three small-group sessions. Pediatric burn care providers had prior experience using TOBI. Most adult care burn providers had trialed TOBI to improve access to outpatient care for their adult patients with burns. The inclusion of providers who trialed TOBI in their clinical practice with adult patients increased our provider sample and provided a greater diversity in assessing workflow and user-interface needs.

Inclusion Criteria

Study inclusion criteria required that caregiver participants: 1) were caregivers of children 0–18 years of age at the time of injury who used TOBI within the last 12 months, 2) were 18 years of age or older, 3) had at least one communication activity through TOBI, and 4) were English-speaking. English-speaking was an important screening tool because TOBI is currently only available in English and does not yet have translation capabilities. Providers were eligible for enrollment if they: 1) were burn surgeons, nurses, nurse practitioners, physician assistants, or therapists; 2) were 18 years of age or older; and 3) had at least one communication activity through TOBI in the past 12 months.

Semi-structured Interviews

We performed and recorded semi-structured interviews of stakeholders, including patients’ caregivers and burn providers, to assess app/portal usability, determine needed improvements in the app build, and evaluate the overall delivery of outpatient burn care using TOBI. The semi-structured interview lasted 30 to 45 min and consisted of open-ended questions based on a developed interview guide (Appendix 1 and 2). Interview questions were organized by app sections (eg, login, FAQs, and message board) and included open-ended questions about user experience, satisfaction and preferences, difficulty understanding and navigating, specific problems, app usability and acceptability, and potential improvements.

Validated Questionnaire Data

Usability data were collected using a psychometrically validated mHealth App Usability Questionnaire (MAUQ). The MAUQ consists of 21 questions with responses ranging from 1 = “Strongly Disagree” to 7 = “Strongly Agree.” The MAUQ yields an overall usability score and three subscale scores: 1) ease of use and satisfaction, 2) system information arrangement, and 3) usefulness. To determine the usability of an app, the total score was divided by the number of items to provide an average score, with higher average scores indicating greater acceptability. The MAUQ validation study provided evidence of strong construct and criterion validity, as well as high internal consistency of the entire questionnaire and three subscales.²⁹

Demographic Questionnaire Data

Participants were asked to provide standard demographic data, such as age, sex, educational achievement, marital status, racial and ethnic status, and household income.

Statistical Approach

TOBI was evaluated using a mixed-method approach consisting of qualitative semi-structured interviews and quantitative usability data. The qualitative interviews were transcribed by a transcription company that is HIPAA compliant and then coded by two trained independent coders. Data were inputted into an Excel spreadsheet for analyses. To analyze the interview transcriptions, we performed a manifest content analysis driven by our research questions specific to caregiver perceptions of smartphone-enhanced pediatric burn care and potential app improvements.³⁰ Since our transcribed text data reflected answers to specific questions as part of the semi-structured interviews, the content analysis approach was determined to be most suitable as the semi-structured questions naturally represented content categories. These questions focused on acceptability, usability, technical preferences, emotional perceptions, and suggestions for improvement. The questions were used to analyze the qualitative data; a priori categories derived from the semi-structured interviews were used to sort the data. Thus, manifest content analysis was the most appropriate method for this part of the data analysis.³⁰ Consensus was achieved through analysis by the first and senior authors, and an author who is a qualitative methodologist. Quantitative usability data were collected using the REDCap® survey and exported into IBM SPSS Statistics 27.0 software for descriptive analyses.

App Refinement Approach

Study findings pertaining to app refinement were discussed during weekly meetings between the principal investigator and study coordinator. At the meetings, potential refinements were consolidated and prioritized according to the defined needs of both caregivers and providers. Organized technology reports were given to the development team for optimization. We used an iterative process in which we provided a list of changes to the development team after retesting each new iteration of the app before the final launch.

RESULTS

Study Recruitment

Seventy-nine caregivers were identified for screening from the burn registry. Five caregivers were excluded for not meeting eligibility criteria: one due to a lack of communication activity, two who were Spanish-speaking only, one who was under 18 years of age, and one whose child was transferred to foster care. Of the remaining 74 participants, 20 were selected using stratified random sampling based on their children’s age groups that represent our patient population and align with the published literature (0–1 years old: 15%, 1–3 years old: 30%, 4–5 years old: 25%, 5–10 years old: 15%, and 10–18 years old: 15%).³¹ One major obstacle to participant recruitment and enrollment was the COVID-19 pandemic and an inability to contact some eligible participants. If a caregiver could not be reached by phone after three contact attempts, another potential participant was randomly selected from the respective child age group and contacted for recruitment. Study recruitment ended as determined by an ad hoc analysis conducted by the first and senior author after the 15th participant; data saturation indicated that no new data were emerging from participants. Data saturation determines sample size in qualitative research and is different for each study³²; additionally, since qualitative analysis begins with the first interview, it is common for data saturation to mark the end of recruitment. Twelve burn providers were contacted via email with a brief description of the study. Two providers were unavailable, and ten providers expressed interest in and consented to participate in the study.

Caregiver Study Sample

Caregiver study participants included 15 caregivers of pediatric patients with burns. Caregiver participants ranged from 22 and 52 years of age (M = 35.5, SD = 6.7), were predominantly female and mothers (14/15, 93%), White (9/15, 60%) or Black (5/15, 33%), married (11/15, 73%), sole caregivers (11/15, 73%), had a college degree (12/15, 80%), were employed (7/15, 47%) or unemployed (6/15, 40%), lived in an urban or suburban community (13/15, 87%) within 60 miles from the burn center (13/15, 87%), and had household incomes ranging from less than $10,000 to more than $200,000. The children’s ages ranged from 1 to 12 years (M = 3.4, SD = 3.1), and the average total body surface area of the burn was 3.3% (SD = 3.2; Table 1).

Table 1.

Demographic characteristics of participants

Characteristics	Caregivers (N = 15)	Providers (N = 10)
Caregiver age, y, mean (SD)	35.5 (6.7)
Caregiver sex, No. (%)
Female	14 (93.3)
Male	1 (6.7)
Caregiver relationship, No. (%)
Mother	14 (93.3)
Father	1 (6.7)
Caregiver race, No. (%)
Black or African American	5 (33.3)
White	9 (60.0)
Multiple or other	1 (6.7)
Caregiver marital status, No. (%)
Married	11 (73.3)
Single, never married	4 (26.7)
Caregiver education, No. (%)
High school or GED	3 (20.0)
Associate’s degree	3 (20.0)
Bachelor’s degree	6 (40.0)
Master’s degree	2 (13.3)
Doctoral degree	1 (6.7)
Caregiver employment, No. (%)
Full-time job	7 (46.7)
Part-time job	2 (13.3)
Unemployed	6 (40.0)
Caregiver income/year, No. (%)
Less than $10,000	1 (6.7)
$10,000–$50,000	5 (33.3)
$50,000–$100,000	3 (20.0)
More than $100,000	5 (33.3)
Sole caregiver, No. (%)
Yes	11 (73.3)
Caregiver community, No. (%)
City or urban	6 (40.0)
Suburban	7 (46.7)
Rural	2 (13.3)
Caregiver distance, No. (%)
Less than 60 miles	13 (86.7)
More than 60 miles	2 (13.3)
Child’s age, y, mean (SD)	3.4 (3.1)
TBSA, %	3.3 (3.2)
Provider age, y, mean (SD)		39.6 (5.7)
Provider sex, No. (%)
Female		5 (50.0)
Male		5 (50.0)
Provider race, No. (%)
Asian or Asian American		1 (10.0)
White		9 (90.0)
Provider role, No. (%)
Physician		8 (80.0)
Advanced practice practitioner		2 (20.0)
Nurse		1 (10.0)
Provider medical experience, y, mean (SD)		10 (5.9)
Provider burn experience, y, mean (SD)		7.5 (7.5)
Provider TOBI experience, months, mean (SD)		13 (15.5)

GED, general equivalency diploma; TBSA, total body surface area; TOBI, Telemedicine Optimized Burn Intervention.

Burn Provider Study Sample

Burn provider participants included ten providers from a southeastern academic medical center. Provider participants ranged in age from 30 to 49 years (M = 39.6, SD = 5.7), were equally represented by sex (female 5/10, 50%), were White (9/10, 90%), and included surgeons (7/10, 70%), advanced practice providers (2/10, 20%), and a burn registered nurse (1/10, 10%), each of whom had been practicing for 2–22 years (M = 10.0, SD = 5.9), had 1–22 years of experience treating burns (M = 7.5, SD = 7.5), and had used the TOBI app for 0–48 months with an average of 13 months (SD = 15.5; Table 1).

Patient Qualitative Data

Qualitative data revealed important consumer-driven feedback that was needed for app refinements (Table 2). Most (14/15, 93%) caregivers reported an overall positive experience and said the app was easy to use. One user stated, “I’m not very tech savvy but it was actually pretty easy to navigate, dive straight into point, the communication on the app was very easy.” All caregivers stated that the app was an acceptable way of monitoring child’s burns and that they would recommend the app to a friend in a similar situation: “It’s 100% convenient, I didn’t have to come down to the [doctor’s] office all the time,” “The answers that you can’t find anywhere else, you can find in the app.”

Table 2.

Qualitative data from semi-structured interviews of participants

	Patient Caregivers	Burn Providers
Study sample	15	10 (7 surgeons, 2 advanced practice providers, 1 burn registered nurse)
Most common suggestions	• Keep caregiver logged in • Time-stamp photos and messages • Consolidate text messages and pictures • Add push notifications and appointment reminders • Track pain level of child • Add stress-management techniques	• Modify user interface to change to smartphone app to streamline notifications • Link to EHR to streamline charting and billing
Exemplary quotes	• “They respond very quickly” • “It’s just—it’s just better [than] going to the hospital, more efficient, and safe at this point” • “It’s very easy, it means that you don’t have to get your child out of school. You don’t have to put everybody in the car. It’s very convenient”	• “It has the potential to keep patients closer to home to [sic] in the large catchment area, and in a competitive market, it’s one tool you can use to help keep patients close to home” • “I think also in the past, there was a lot of non-HIPAA compliant communication with patients, they should hopefully avoid that” • “If it decreases our clinic volume that could also be a bad thing. It can be, you know, there’s good and bad to everything, so…”

Regarding the specific app functions, many caregivers (11/15, 73%) reported that taking and sending photos of the burn was easy, and some (4/15, 27%) experienced problems. Most participants (14/15, 93%) said that the FAQs were easy to understand, and all users found the FAQs helpful. For instance, one participant said, “Very helpful because it saved me from having to Google questions that I know I would be dealing with anyway, it was all right there together.” Many participants (11/15, 73%) found the information they were looking for, and several suggested additional FAQs, such as “Can my child go to the beach or pool with a burn injury?” Only one caregiver experienced problems with messages, and several recommended revisions.

Most participants (7/9, 78%) were satisfied with the video consult experience. One participant described their experience: “I was very satisfied. The nurse answered questions very quickly, and actually helped me out in that particular moment because I was very lost.” All the caregivers found videos of dressing changes to be very helpful and easy to follow: “They are extremely helpful... because it walks you through step by step how to do the bandaging.” Five users requested adding specific videos (eg, how to improve range of motion, different size hand models).

Specific feedback to improve the app interface, features, and functions included: keeping caregivers logged in, time-stamping photos and messages, consolidating text messages and pictures, adding push notifications and appointment reminders, tracking the pain level of the child, adding stress-management techniques, adding rationales, and adjusting the font size. Some users commented on the challenges: “There wasn’t a way to notify me when I got a message from a nurse, so I just had to keep the app open and constantly refresh it” and “I’m all about choice because I can read it, but I know for example my mother would not be able to read it. So being able to have a choice as far as font size would be ideal for me.”

Overall, 47% (7/15) of participants preferred app-based burn care, 40% (6/15) preferred both face-to-face and app-based care, and 13% (2/15) preferred in-person-only care. Only one person preferred synchronous video-based care to asynchronous text messaging.

MAUQ Data

Overall MAUQ scores (M = 6.47; SD = 0.60) indicated high app usability. Ease of use and satisfaction (M = 6.66; SD = 0.40), system information arrangement (M = 5.93; SD = 1.05), and usefulness (M = 6.69; SD = 0.58) subscales indicated an average degree of agreement above “somewhat agree” with usability statements (>5). All subscales showed high internal consistency in our sample. For each subscale, Cronbach alphas were .79 (ease of use and satisfaction), .87 (system information arrangement), and .93 (usefulness).

Burn Provider Qualitative Data

Ten burn providers (seven surgeons, two advanced practice providers, one burn registered nurse) were surveyed in three small-group settings. Providers were given time to explore TOBI before answering open-ended questions, centering on the overall experience, logging in, interpreting patient images, messaging patients, scheduling, providing video consultation, interfacing with the portal, discussing barriers, and reporting. All 10 providers endorsed using the app for outpatient burn care, with comments such as: “very straightforward,” “messaging interface is very easy,” and “it has the potential to keep patients closer to home in the large catchment area, and in a competitive market, it’s one tool you can use to help keep patients close to home for their tool, for accuracy as a tool to keep patients close to home.” With respect to video conferencing through the app, all providers suggested discontinuing this feature and conducting video visits using institutionally licensed software with the workflow and billing occurring through the electronic health record (EHR).

All providers expanded on workflow barriers with this service, which was a major theme. Specific suggestions included: creating a provider app instead of a web-based portal; providing real-time notifications through the provider app; adding the ability to name a proxy provider during vacations, time-off, etc.; linking virtual encounters to the EHR for billing; enabling patient accounts to be deactivated; including instructions for appropriate photo-taking environments; and time-stamping all messages and pictures. Several concerns were voiced regarding the ability to bill for physician time using the app and the workflow of patients contacting burn surgeons during operating room time. Direct comments included: “The downside is it’s not billable stuff” and “For me, personally, because it’s not the way I do my practice, it’s a nurse-driven practice, nurse takes it, takes care of most of these calls, screens things out, and then … kicks up to their providers.” When asked whether there was a possibility of increased harm with app-based care, most providers felt that the risk was minimal compared to face-to-face care. However, one provider cited the potential for missing a dire complication that is not detected until the morning when the provider logs into email. Finally, several newer services were suggested to broaden the app’s use, including physical and occupational therapy exercises, survivor resources, and mental health resources.

Burn App Refinements

Based on stakeholder feedback, the following app changes were systematically implemented and tested to make TOBI more “user-friendly.” First, users now can remain logged into the app if they choose, which eliminated the need for password resets and burn center contact to retrieve this information. Second, the messages and images sections were integrated to support more streamlined communication. Both providers and patients can now see the time stamps on messages and whether the messages have been read. Also, users can now set up push notifications on their device and/or email notifications for new messages from providers. Third, we optimized rationales and explanations for various sections in the app to answer common user questions and clarify how various app components may be used.

Several new sections were added to improve app acceptability and usability, including interdisciplinary burn and mental health components (Figure 1). First, a section on pain tracking was added to allow caregivers and pediatric patients to rate pain level on a weekly basis. For this section, the Wong–Baker Faces Pain Rating Scale was used as a model.³³ We also included a visual graph in which providers and patients can track pain ratings over time. Second, we added a section containing Quick Tips, including Burn Tips, Wellbeing Tips for Adults, and Wellbeing Tips for Children/Adolescents. Third, to meet the caregiver demand for coping with stress during their child’s burn treatment, we incorporated a new section on stress management that applies to both adults and children. The subsections include relaxation techniques (body scan recordings for whole body or specific body parts), guided deep-breathing practice, guided mindfulness practice (mindful breathing, imagery, and walking), and distress tolerance skills (e.g., distracting and self-soothing techniques). Finally, self-help resources were added to the app to connect caregivers and patients to the appropriate burn- and mental health-specific agencies and resources, including the American Burn Association, Phoenix Society for Burn Survivors, Phoenix Survivors Offering Assistance in Recovery (SOAR), Model Systems Knowledge Translation Center, National Institutes of Health, National Institutes of Mental Health, Substance Abuse and Mental Health Services Administration, and the 988 Suicide and Crisis Lifeline.

On the provider side, we replaced the provider-facing web-based portal with a new provider app that communicates with the patient-facing app. Providers can now receive real-time notifications through the provider app, allowing for quicker response times. Because burn care is often provider-agnostic, we incorporated the ability to name a proxy provider when the designated provider is not on service. Also, to address a major workflow barrier, we linked virtual patient encounters to the EHR for billing. With respect to physician billing, a new feature was introduced to create a virtual visit encounter based on time spent. These encounters are billed as “Virtual Check-ins” and are reimbursable by both private and public third-party payers. Virtual Check-in is designed to be a brief technology-based communication between the healthcare provider and the patient using a phone, captured image, or integrated audio/video system.

DISCUSSION

After a pediatric burn injury, patients and caregivers are often discharged home with instructions to begin outpatient care for burn wounds. Unfortunately, most caregivers lack experience in outpatient care for a burn, leading to potential for delayed recovery, poor adherence to burn treatment, and significant anxiety during dressing changes.¹⁰ To address these deficiencies, we developed the TeleBurn app in 2018 to help parents with acute wound care at home. In a pilot study, 77 patients who used the TeleBurn for burn care had faster healing times, fewer in-person clinical encounters, and greater adherence to therapy than a similar cohort treated with traditional face-to-face care.¹¹

In the present study, we used a convergent mixed-methods approach to systematically evaluate user feedback from both caregivers of pediatric patients with burns and burn clinicians who used an mHealth application during the acute phase of burn injury. We then used stakeholder feedback to refine the app and optimize the patient and provider experience. Findings from qualitative semi-structured user interviews and quantitative usability measures indicated that the app was highly usable but required improvements. We systematically implemented these modifications before we released the app for routine patient use.

In this study, we sought to optimize the design and implementation of the TeleBurn (renamed as TOBI) using stakeholder feedback. Unfortunately, systematically derived stakeholder feedback is rarely used when creating and deploying new mHealth technologies.^{14, 19} Over 350,000 mHealth apps have been developed globally. However, despite many health benefits, about half of these apps stop being used due to factors such as high data entry burden and loss of interest.³⁴ With the results of our study, we better understand the patient–caregiver perspectives on usability/acceptability, satisfaction, and preferences. Overall, users reported TOBI was a highly usable app with high usefulness, ease of use, and satisfaction, as well as moderately high functionality. Specifically, on average, users rated TOBI’s ease of use and satisfaction statements between “agree” and “strongly agree”; system information arrangement statements between “somewhat agree” and “agree”; and usefulness-related statements between “agree” and “strongly agree.” Although most users were highly satisfied with the app and its health benefits, improvements were needed to create an optimal mHealth burn app. The identified improvements were systematically addressed and tested in collaboration with the digital development team before the release of TOBI for routine patient use. Qualitative data from this study provided helpful recommendations for app improvements that can be generalized to other mHealth apps to increase their appeal and adoption.

This study has limitations that may restrict its broad application in other mHealth applications. First, the sample size was relatively small given the diversity of ages and demographics of our patient population. For instance, a parent of a toddler may need different mHealth resources than a parent of a teenager. To address this difference, we sampled caregivers of patients in different age groups. After 15 patients, both interviewers coded and reviewed the data and felt that saturation was met. With respect to provider sample size, we were able to study all burn providers at our institution that had trialed TOBI in the clinical setting. TOBI had been trialed in a clinical setting by both pediatric and adult burn providers, giving a greater diversity of opinion about the utility, user interface, and workflow. Additionally, caregiver participants received a $50 gift card and provider participants received a $10 gift card. While these amounts were determined based on time and our institutional review board deemed these amounts as appropriate participant compensation, there may have been potential participation bias. Lastly, we have been able to incorporate mental health resources and tools into the app. Future app refinements may be needed to incorporate instructions from physical and occupational therapists to broaden the app’s use. Another limitation is that TOBI is designed for pediatric patients with burns, mHealth-based care can and should be extended to the adult burn population. We plan to implement TOBI in the adult burn unit in a future study. Finally, this service was only offered to English-speaking patients due to the communicative nature of text messaging used heavily in this service. Unfortunately, linguistic barriers between patients and providers remain a barrier to accessing health care. However, mHealth applications may provide a solution to reducing these inequalities through translation of apps and commercialization^{35, 36}. With novel interventions, such as TOBI, in which providers interact directly with patients, app translation must be paired with non-English-speaking providers to optimize health care delivery to these patient populations.

Unfortunately, in the United States, telemedicine systems have historically been encumbered by various technical and regulatory barriers. Federal and state statutes limited reimbursement based on patient and physician location, and technical connectivity issues plagued the system.^{19, 20} These limitations have been lifted since the onset of COVID-19 pandemic and significantly shifted the delivery of care as quarantine measures went into place. The Centers for Medicare and Medicaid suspended the sites of origination restriction and allowed the delivery of medical and surgical care in the patient’s home. These shifts challenged providers to explore new delivery platforms to provide direct-to-consumer health care, including burn care.³⁷ Importantly, some states have begun to adopt permanent telehealth benefits. While federal payers have not adopted all forms of telemedicine, Medicare has permanently agreed to coverage for Federally Qualified Health Centers and Rural Health Clinics. While the regulatory details of telehealth are beyond the scope of this paper, numerous new payment strategies have been employed by third-party payors, giving most patients and providers confidence that telehealth coverage will not collapse after the public health emergency ends.

As healthcare data generated by patients gain traction in medical decision-making, both patients and providers are major stakeholders in the process and its content. In previous mHealth development work, patients wanted reminder notifications, instructions, rationale explanations, incentives and rewards, and clear communication lines. On the other hand, providers valued data management, tracking, and summaries.^{38, 39} Increasingly, patients expect health care to be “on demand” and user-friendly, but healthcare systems have lagged behind these expectations. Understanding these stakeholder tensions is imperative to design effective mHealth systems of care.⁴⁰ The rapid evolution of telemedicine has followed the pace of technological advancement, enabling healthcare entities to provide healthcare in ever-changing settings, often blurring the lines of traditional care, and thus, the roles of the healthcare provider.⁴¹

Mobile health technology shows great promise in improving the delivery of expert burn care to patients who are outside of the burn center. TOBI is an innovative, scalable, personalized mHealth resource that has been refined to improve usability for both parental caregivers of pediatric patients with burns and the burn center team. This study shows the value of learning about caregiver experiences, which provided consumer-focused, systematically derived data to inform TOBI app modifications to improve its acceptability and usability. Overall, caregivers preferred asynchronous communication to synchronous video visits as an adjunct to in-person care. Now that the app has been improved with stakeholder feedback, we are conducting a randomized controlled trial to assess the feasibility and efficacy of the app to improve the quality of and access to expert burn care for patients and their families.

Funding: This work was supported by a generous grant from the Eunice Kennedy Shriver National Institute of Child Health and Human Development of the National Institutes of Health (K23 MH107641).

Conflict of interest statement: Dr. Lesher has submitted a record of invention for the TOBI app. The Medical University of South Carolina owns the intellectual property through the Foundation for Research Development. There are no patents issued. The Medical University of South Carolina has the right to commercially license TOBI. Should this happen, the principal investigator may be paid royalties from that licensure agreement.