Abstract
Background:
Children with medical complexity (CMC) account for high health care utilization. Telemedicine holds significant potential in CMC, as it allows a provider to engage with CMC in their home environment and can alleviate both financial and transportation burdens. Remote exam devices that enable the performance of a physical exam could expand the ability of providers to clinically assess CMC during a telemedicine visit. In this pilot study, our goal was to develop a process for integrating an in-home remote exam device into a complex care clinic during the COVID-19 pandemic and evaluate the usability and feasibility of this device.
Methods:
The remote exam device was distributed to caregivers of CMC cared for at a complex care outpatient center. Using deliberate practice framework, our onboarding process provided opportunities for caregivers and providers to learn how to utilize the device. Surveys examining usability and feasibility were administered to both caregivers and providers after each telemedicine encounter.
Results:
A total of 43 caregivers participated in the onboarding process, which included a total of 83 practice visits. The remote exam device was rated as having excellent usability by caregivers; however, providers rated the device as having lower usability. Feasibility was notable for technology issues contributing to 15% of encounters being cancelled or ending early.
Conclusions:
The remote exam device was successfully integrated into a complex care clinic. Data from this pilot study supports the usability and feasibility of deploying a remote exam device across a telemedicine platform in a busy and complex outpatient academic practice.
Keywords: telemedicine, pediatric complex care, children with medical complexity
Background
Children with medical complexity (CMC), defined as children with chronic, functional limitations, and technology-dependence, represent approximately 1% of pediatric patients, yet account for nearly half of all pediatric hospitalization days and cost.1,2 Care for these medically fragile patients is often fragmented and results in a variety of hardships for families. Specifically, up to 68% of families with CMC experience financial hardships related to caring for their child, and almost half of families have limited or no social support system.3 Over the past decade, research and clinical innovation has focused on identifying ways to provide comprehensive, timely, and accessible health care to improve patient outcomes and quality of life for this patient population and their caregivers.1,2,4
Telehealth, the use of asynchronous or synchronous electronic transmission of health care data, is a rapidly expanding field that affords providers increased opportunities to care for their patients remotely, especially in the setting of recent challenges during the COVID-19 pandemic.5 In particular, the use of telehealth for CMC holds significant promise for improving outcomes for this patient population and, ultimately, reducing hospitalization rates and healthcare costs.6 Since CMC are often dependent on highly specialized medical care and home medical technologies, establishing an opportunity for the primary care team to interact with CMC in the home environment can help providers identify and intervene in health deterioration events or troubleshoot malfunctions in medical technology in a timely manner. Ultimately, this more direct interaction could decrease acute care visits and hospital admission rates.6 Additionally, telehealth offers a viable mechanism to enhance care coordination in this vulnerable population, as well as potentially reduce the impact of physician shortages and transportation burdens on families.7,8
The use of telehealth medical devices presents additional opportunities to improve the quality of care that providers are able to offer in a patient’s home environment. The TytoCare TytoHome™ is a direct-to-consumer home-based device that enables the performance of a remote, high-fidelity physical exam. Consisting of an infrared thermometer, a digital stethoscope, a digital otoscope, a tongue depressor, and a built-in camera, this remote exam device expands the ability of providers to clinically assess patients virtually. The device is intended to remain with the consumer (patient) and the provider can provide instructions via video telehealth on a secondary tablet or computer to obtain the desired physical exam assessments.
In this preliminary study, our goal was to develop a process for integrating this in-home remote exam device into a busy pediatric complex care clinic during the COVID-19 pandemic in order to offer CMC a high-fidelity, socially distant option to access their primary care provider for routine care needs. Additionally, we aimed to evaluate the usability and feasibility of the remote exam device in delivering an ancillary telehealth physical exam among our CMC population.
Methods
TELEHEALTH DEVICE
The TytoCare Medical Exam Kit is a Food and Drug Administration cleared remote-exam device that allows providers to perform a virtual physical exam as part of a telemedicine encounter. It consists of a handheld, Wi-Fi-capable device that serves as the patient-facing interface, as well as three adapters: otoscope, stethoscope, and tongue depressor (Fig. 1). To use the device during a telemedicine encounter, the patient or caregiver must first connect the remote device to their home Wi-Fi. After the device is connected to the internet, the device should be linked to the patient’s telemedicine platform profile prior to an appointment with the provider.
Fig. 1.
Tyto™ device setup and caregiver view. Image provided by TytoCare, Inc.
During the telemedicine encounter, the caregiver can virtually interact with the provider through the institutional telemedicine platform using a secondary smartphone, tablet, or computer. When the provider is ready to use the remote device for their physical exam, the provider accesses the patient’s remote device using an integrated add-on within the telemedicine platform. The provider then selects the desired adapter needed to perform the intended component of the physical exam, which sends a signal to the patient-facing device interface instructing the caregiver how to attach the selected adapter. Once the adapter is attached, the provider can view a live-stream video from the adapter (or real-time auscultation) through the telemedicine platform. Importantly, the provider commonly needs to coach the caregiver how to adjust angles for improved visualization when using the otoscope or tongue depressor or provide instructions on how hard to press the stethoscope to the patient’s chest during auscultation.
INTEGRATING THE DEVICE WITH TELEMEDICINE APPOINTMENT
The Cincinnati Children’s Center for Telehealth developed an institution-specific application in concert with a commercial telemedicine platform where families could register for a telemedicine account and request appointments. This application was integrated with the remote exam device after our institution submitted a request for an integration build to the companies for both the telemedicine platform and the remote device. After this integrated telemedicine add-on was built, providers and families were educated on how to pair the device with the telemedicine platform during their first appointment. Documentation of telemedicine encounters occurred separately through our central electronic health record.
POPULATION AND ONBOARDING PROCESS
We developed an onboarding curriculum for a pilot group of caregivers and providers in the Cincinnati Children’s Hospital Medical Center’s (CCHMC) Complex Care Center, a patient-centered medical home for approximately 500 CMC with chronic diseases who are technology dependent and/or receive care from three or more subspecialties. Providers in the clinic include physicians and advanced nurse practitioners. Using deliberate practice as an educational framework, our onboarding process provided opportunities for caregivers and providers to engage with discrete functionalities of the device, receive targeted feedback, and re-engage with the task to build skills.9 The primary focus of this onboarding process was to establish a program that fostered provider skill acquisition in coaching a family to use of the device with specific attention to limiting disruptions to clinic flow.
In order to accelerate the remote device onboarding and integration processes, we provided CMC caregivers with educational content prior to their first telemedicine appointment with the provider. The caregiver educational content included: (1) a handout instructing caregivers how to connect the remote device to home Wi-Fi and the institution-specific telemedicine platform; (2) access to the training application developed by the remote device company; (3) a SuperUser directed 1-on-1 training session; (4) and practice visit(s) with a healthcare provider, which varied between 1–4 visits per family based on the phase of onboarding. Practice visits were used during the early onboarding period to rapidly increase provider experience and comfort with the device while also establishing a core of caregivers willing to offer formative feedback on the experience. Practice visits were scheduled as standard telemedicine appointments, but no charges were submitted to billing unless an acute medical concern was expressed by the caregiver and the provider addressed the issue. Routine practice visits consisted of the caregiver connecting the remote device to the telemedicine platform, the provider connecting to the remote device through the telemedicine platform, and the provider coaching the caregiver on using each physical exam adapter on the patient. As providers became more proficient with using the device and expanded their troubleshooting capabilities, practice visits were phased out. This strategy allowed us to minimize the number of nonbillable scheduled telemedicine visits to the smallest number needed for an individual provider to gain the requisite skills in using the remote device.
Additional provider educational content included: (1) a 1-on-1 training session with a SuperUser to onboard to the device and demonstrate how to integrate the device with the telemedicine platform, and (2) in-person SuperUser assistance during the first telemedicine encounter with each family until the provider demonstrated comfort using the device. SuperUsers were either clinical fellows with protected time for research (M.P., J.O.) or a member of the Cincinnati Children’s Center for Telehealth where the role of SuperUser was a part of their position (MR). The SuperUsers had experience in using the remote exam device and telemedicine platform, and they provided in-person support to troubleshoot connection issues and provide real-time feedback to providers on improving their caregiver coaching skills.
DATA COLLECTION & STATISTICS
We administered postencounter surveys examining usability and feasibility of the device to both caregivers and providers via REDCap after each appointment.10 Postencounter survey questions were developed using an iterative consensus process after consulting providers and CMC caregivers who were willing to provide serial formative feedback. Usability was determined using a modified Systems-Usability Scale (SUS) with questions formatted specifically to the device.11 Survey responses were aggregated by group (provider or caregiver) to develop a provider usability score and a caregiver usability score. Survey questions also targeted any caregiver perceived issues with patient discomfort or technical challenges in using each individual adapter.
Feasibility was determined by collecting provider reported data after each appointment regarding whether there were any issues with (1) ability to have the caregiver and provider connect on the telemedicine platform, (2) integrating the device with the telemedicine platform, and/or (3) connecting to any of the device adapters during the appointment.
Additionally, to obtain a better understanding of continued utilization, we collected data on how many devices had been distributed and how many telehealth appointments utilized Tytocare for the 6 months following this preliminary study.
This study was reviewed and approved by our institutional IRB as an exempted study under the umbrella of quality improvement. As such, we did not acquire any demographic data on patients or their caregivers for this study.
Results
Forty-three caregivers participated in the onboarding process, which included a total of 83 practice visits. Twenty caregivers (47%) completed one practice visit, 11 (26%) completed two visits, 5 (12%) completed three visits, and 2 (5%) completed four visits. Six Complex Care Center providers participated. We received 29 postvisit caregiver surveys (45% response rate) and 55 provider surveys (85% response rate). SuperUsers were present at 53 (63.8%) practice visits. This onboarding process was completed in 4 months, after which all providers reported adequate comfort in independently operating the remote device without help from a SuperUser.
Based on the modified SUS the caregiver usability score was 86.7, which categorized the device as “excellent usability.” The provider usability score was 58.3, categorized as “marginal usability” (Table 1). Caregivers were able to provide free text comments regarding their experience with the remote exam device (Table 2). Among the 15 caregivers (39.5%) that responded to survey questions regarding patient discomfort with individual adapters, 9 caregivers (60%) reported patient discomfort when using the tongue depressor, 3 (20%) with the otoscope, and 3 (20%) with the stethoscope.
Table 1.
The System Usability Scale for Remote Teledevice
| THE SYSTEM USABILITY SCALE (1–5 LIKERT SCALE, 1- STRONGLY DISAGREE, 5-STRONGLY AGREE) |
|---|
| Each individual survey response was calculated and yielded a score on a scale from 0–100. |
| 1. I think that I would like to use this system frequently |
| 2. I find the TytoCare device unnecessarily complex |
| 3. I think the TytoCare device was easy to use |
| 4. I need the support of a technical person to be able to use this system |
| 5. I find the various functions of the TytoCare device well integrated |
| 6. I thought there was too much inconsistency in this system |
| 7. I would imagine that most people would learn to use this system very quickly |
| 8. I found the system very cumbersome to use |
| 9. I felt very confident using the system |
| 10. I needed to learn a lot of things before I could get going with this system |
| Caregiver |
| Usability scores for surveys range: 75–100 |
| Average usability score: 86.7 |
| Provider |
| Usability scores for surveys range: 42.5–72.5 |
| Average usability score: 58.3 |
Table 2.
Caregiver Feedback
| “It was great! My family and I are excited to start using it to check in with the doctor!” |
| “Excited to see how this device prevents the need to go to the [doctor’s] office” |
| “My family and I are impressed with this device and the opportunity to use this in the future in the case of illness or semiemergency” |
| “I feel like if you can work a smart phone, then you can work this device” |
In assessing the feasibility of the device in this cohort, approximately 6% (5/83) of visits had difficulties connecting the caregiver and provider to the telemedicine platform, resulting in two cancelled visits (Fig. 2). Out of the 81 encounters that successfully connected to the telemedicine platform, 25% (20/81) of these encounters had difficulties integrating the remote exam device and platform, resulting in an additional nine visits being cancelled. Of the remaining 72 visits, 7% (5/72) had difficulty connecting to desired device adapters, leading to two encounters ending early. Overall, 84.3% (70/83) encounters were successfully completed.
Fig. 2.
Feasibility for Remote Teledevice.
In the 6 months following this study, 330 patients had been given a Tytocare device, and 102 telehealth visits at the complex care clinic had utilized the Tytocare device.
Discussion
Through the development of onboarding education for a novel remote exam device, we were able to successfully train complex care providers to independently coach caregivers how to integrate and use a remote exam device during a virtual patient encounter. The initial onboarding curriculum incorporated deliberate practice with multiple practice visits scheduled to allow for feedback and refinement of skills. However, given logistical challenges related to caregiver and provider availability the process underwent iterative changes to reduce the number of practice visits and modify the educational content based on stakeholder feedback and survey data. Ultimately, we determined that combining a SuperUser training session for caregivers with strong provider coaching led to successful remote exam device visits. Data collected after this study indicates continued distribution and utilization of the telehealth device.
Providers learned how to be successful coaches by participating in a 1-on-1 education session and through feedback from SuperUsers during their first several clinical encounters. SuperUsers have been used by other nascent telemedicine programs to augment efficiency in patient-to-provider telemedicine interactions, addressing issues from connectivity to interacting with novel telemedicine technology.12 In our program, SuperUsers reinforced the providers’ coaching skills and taught providers how to troubleshoot basic connectivity issues. Once a provider could perform these skills independently, the SuperUser no longer attended visits.
The remote exam device was rated as having excellent usability by caregivers; however, providers rated the device as having lower usability. While the SUS is well-validated tool, it is commonly deployed in situations evaluating a new technology exposure to a fairly homogenous population, such as end-users with a similar common goal.13 In our study, we used the SUS to evaluate the remote exam device from two distinctly-separate perspectives—the provider and the caregiver. This discrepancy in usability rating could be related to the differences in experiences during the onboarding process or different priorities for the telemedicine encounter. This pilot study relied heavily on providers becoming expert coaches and leading troubleshooting efforts when needed; thus, providers were first in experiencing and managing difficulties with the device. There is also the potential of added stress on providers when troubleshooting issues with device integration due to the pressures imposed by a busy clinic schedule.14 Future efforts need to focus on improving provider usability of the remote device; it will be particularly important to elicit direct provider input. Caregiver survey data also noted that certain adapters were more difficult to use, such as the tongue depressor. Additional educational efforts may focus on more direct education and/or best practices for particular adapters.
Feasibility was notably more problematic with technology issues contributing to 16% of encounters being cancelled or ending early. This is a higher rate of connectivity issues than experienced by Notario et al. in their study looking at telemedicine appointments for CMC, but is not necessarily a unique experience among telemedicine programs that rapidly expanded in response to the COVID-19 pandemic.15,16 Integration of the device to the telemedicine platform contributed to the largest number of cancelled visits. Future efforts need to focus on streamlining this integration, possibly with the creation of “pairing visits” with a telehealth team member prior to clinical appointment. Strong information technology support and availability for both providers and caregivers is paramount moving forward.17
This preliminary study has several limitations. This project was conducted at a single, large tertiary center with well-established institutional telemedicine infrastructure, which may potentially limit the generalizability of our findings. Additionally, our study was conducted during the COVID-19 pandemic, where caregivers of CMC may have been more eager to avoid in-person clinic visits, thus our experience may reflect a convenience sample of CMC caregivers most interested in increasing in-home telemedicine services. Lastly, our results are limited by the survey response rate among both caregivers and providers.
Next Steps
Data from this pilot study supports the usability and feasibility of deploying a remote physical exam device across a telehealth platform in a busy and complex outpatient academic practice. Future interventions will focus on improving connectivity rates and accessibility to non-English speaking caregivers. Currently, we are translating educational materials into other languages and incorporating interpreters into remote exam device visits to further increase accessibility. Additionally, continued qualitative caregiver and provider feedback about utilization and experience with the device will be important to better direct changes and growth. Finally, as more remote devices are deployed to CMC patients, we are excited to investigate this remote exam device’s impact on rates of acute care visits and hospitalizations for CMC.
Conflicts of Interest
The authors and CCHMC do not have any financial interest in the TytoCare TytoHomeTM device.
Funding Information
Funding for purchase of the devices was made available through the Federal Communications Commission via the CARES act. Funding for REDCap is provided through the CCTST at the University of Cincinnati is funded by the National Institutes of Health (NIH) Clinical and Translational Science Award (CTSA) program, grant UL1TR001425. The CTSA program is led by the NIH’s National Center for Advancing Translational Sciences (NCATS).
REFERENCES
- 1. Cohen E, Kuo DZ, Agrawal R, et al. Children with medical complexity: An emerging population for clinical and research initiatives. Pediatrics 2011;127(3):529–538; doi: 10.1542/peds.2010-0910 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Cohen E, Berry JG, Camacho X, et al. Patterns and costs of health care use of children with medical complexity. Pediatrics 2012;130(6):e1463–e1470; doi: 10.1542/peds.2012-0175 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Thomson J, Shah SS, Simmons JM, et al. Financial and social hardships in families of children with medical complexity. J Pediatr 2016;172:187–193.e1; doi: 10.1016/j.jpeds.2016.01.049 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Cohen E, Berry JG, Sanders L, et al. Status complexicus? The emergence of pediatric complex care. Pediatrics 2018;141(Suppl 3):S202–S211; doi: 10.1542/peds.2017-1284e [DOI] [PubMed] [Google Scholar]
- 5. Curfman A, McSwain SD, Chuo J, et al. Pediatric telehealth in the COVID-19 pandemic era and beyond. Pediatrics 2021;148(3):e2020047795; doi: 10.1542/peds.2020-047795 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Mosquera RA, Avritscher EBC, Pedroza C, et al. Telemedicine for children with medical complexity: A Randomized Clinical Trial. Pediatrics 2021;148(3):e2021050400; doi: 10.1542/peds.2021-050400 [DOI] [PubMed] [Google Scholar]
- 7. McConnochie KM, Ronis SD, Wood NE, et al. Effectiveness and safety of acute care telemedicine for children with regular and special health care needs. Telemed J E Health 2015;21(8):611–621; doi: 10.1089/tmj.2014.0175 [DOI] [PubMed] [Google Scholar]
- 8. McConnochie KM, Wood NE, Herendeen NE, et al. Telemedicine in Urban and Suburban childcare and elementary schools lightens family burdens. Telemed J E Health 2010;16(5):533–542; doi: 10.1089/tmj.2009.0138 [DOI] [PubMed] [Google Scholar]
- 9. Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79(Suppl 10):S70–S81; doi: 10.1097/00001888-200410001-00022 [DOI] [PubMed] [Google Scholar]
- 10. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42(2):377–381; doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Measuring and Interpreting System Usability Scale (SUS). UIUX Trend; 2017. Available from: https://uiuxtrend.com/measuring-system-usability-scale-sus/ [Last accessed: June 21, 2021]. [Google Scholar]
- 12. Leeman S, Wang L, Johnson BA, et al. Criteria-based assessment of a teleophthalmology diabetic retinopathy evaluation program in a primary care setting. Telemed J E Health 2022;28(6):865–872; doi: 10.1089/tmj.2021.0064 [DOI] [PubMed] [Google Scholar]
- 13. Kamal AH, Wolf S, Nicolla JM, et al. Usability of PCforMe in patients with advanced cancer referred to outpatient palliative care: Results of a randomized, active-controlled pilot trial. J Pain Symptom Manage 2019;58(3):382–389; doi: 10.1016/j.jpainsymman.2019.05.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Patel M, Berlin H, Rajkumar A, et al. Barriers to telemedicine use: Qualitative analysis of provider perspectives during the COVID-19 pandemic. JMIR Hum Factors 2023;10:e39249; doi: 10.2196/39249 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Notario PM, Gentile E, Amidon M, et al. Home-based telemedicine for children with medical complexity. Telemed J E Health 2019;25(11):1123–1132; doi: 10.1089/tmj.2018.0186 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. DeMauro SB, Duncan AF, Hurt H. Telemedicine use in neonatal follow-up programs—What can we do and what we can’t—Lessons learned from COVID-19. Semin Perinatol 2021;45(5):151430–151430; doi: 10.1016/j.semperi.2021.151430 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Laub N, Agarwal AK, Shi C, et al. Delivering urgent care using telemedicine: Insights from experienced clinicians at Academic Medical Centers. J Gen Intern Med 2022;37(4):707–713; doi: 10.1007/s11606-020-06395-9 [DOI] [PMC free article] [PubMed] [Google Scholar]