SMS-text messaging for collecting outcome measures after acute stroke

Abstract

Introduction

Traditional methods for obtaining outcomes for patients after acute stroke are resource-intensive. This study aimed to examine the feasibility, reliability, cost, and acceptability of collecting outcomes after acute stroke with a short message service (SMS)-text messaging program.

Methods

Patients were enrolled in an SMS-text messaging program at acute stroke hospitalization discharge. Participants were prompted to complete assessments including the modified Rankin scale (mRS) and Patient-Reported Outcomes Measurement (PROM) Information System Global-10 at 30, 60, and 90 days postdischarge via SMS-text. Agreement and cost of SMS-text data collection were compared to those obtained from traditional follow-up methods (via phone or in the clinic). Participant satisfaction was surveyed upon program conclusion.

Results

Of the 350 patients who agreed to receive SMS texts, 40.5% responded to one or more assessments. Assessment responders were more likely to have English listed as their preferred language (p = 0.009), have a shorter length of hospital stay (p = 0.01), lower NIH stroke scale upon admission (p < 0.001), and be discharged home (p < 0.001) as compared to nonresponders. Weighted Cohen’s kappa revealed that the agreement between SMS texting and traditional methods was almost perfect for dichotomized (good vs. poor) (κ = 0.8) and ordinal levels of the mRS score (κ = 0.8). Polychoric correlations revealed a significant association for PROM scores ($\rho$ = 0.4, p < 0.01 and $\rho$ = 0.4, p < 0.01). A cost equation showed that gathering outcomes via SMS texting would be less costly than phone follow-up for cohorts with more than 181 patients. Nearly all participants (91%) found the program acceptable and not burdensome (94%), and most (53%) felt it was helpful. Poststroke outcome data collection via SMS texting is feasible, reliable, low-cost, and acceptable. Reliability was higher for functional outcomes as compared to PROMs.

Conclusions

While further validation is required, our findings suggest that SMS texting is a feasible method for gathering outcomes after stroke at scale to evaluate the efficacy of acute stroke treatments.

Introduction

Stroke is a leading cause of acquired adult disability worldwide (1). Recent substantial advances in acute stroke treatments (2, 3) and novel approaches to stroke rehabilitation (4, 5) have resulted in significant improvements in poststroke outcomes. To systematically evaluate the real-world benefit of such interventions, it is essential to reliably collect outcomes for patients after acute stroke discharge.

Current approaches to outcomes data collection face many logistical barriers. Follow-up care, during which outcomes are traditionally collected, requires patients to return to specialized stroke centers and can be time-intensive, cost-prohibitive, and burdensome, relying on interaction with trained healthcare providers (6, 7). Phone calls to stroke patients to assess outcomes are also time-consuming and require dedicated and trained staff.

In recent years, there has been rapid adoption of digital and telehealth approaches in clinical care (8–10). Since mobile phones are one of the most popular forms of digital interaction (11) and are ubiquitous even among diverse demographic groups (11, 12), there is the potential to utilize short message service (SMS) texting for gathering assessments after stroke. SMS-texting programs have been used in a range of health conditions (13–15) for varying utilities, including intervention (16, 17), adherence (13), and data collection (18). Although app-based collection of outcomes after stroke has been explored (19), the feasibility of using SMS texting has not yet been examined in stroke. This study aimed to examine the feasibility, reliability, cost, and acceptability of an SMS-texting approach to gather health outcomes in the first 90 days after acute stroke.

Materials and methods

Our health system has articulated a goal of collecting functional outcomes after acute stroke discharge on all patients but has lacked the resources to accomplish this. As part of a clinical quality improvement initiative to assess barriers to success, we sought to increase the likelihood of data collection by leveraging an SMS-text messaging-based program on all discharged acute stroke patients for a several-month period. Using the services offered by a digital health technology company [Philips Patient Navigation Manager (formerly Medumo), Boston, MA, United States], we developed and launched an SMS-text follow-up program for patients discharged from the Massachusetts General Hospital (MGH) with a stroke ICD-10 code (I63, I60, I61, and G45) between June 8, 2020, and February 1, 2021. Patients were eligible to participate in the program if they had a valid mobile phone contact number in their medical chart. Patients who had not previously consented to receive SMS texts from their clinical care team at MGH received one consent SMS-text message at the time of acute hospital discharge, which remained active (i.e., giving the option to consent) for the duration of the program. If they did not consent, they did not receive any further messages. Patients had the option to decline participation in the program by responding “STOP” or simply not responding to the consent message.

Patients who consented to receiving SMS texts were enrolled in the program at the time of discharge regardless of their discharge destination (home or facility) and were provided instructions for unsubscribing (Supplementary Table S1). To familiarize patients with the SMS-texting method of communication and optimize patient engagement, patients also received weekly brain health educational tips developed by a multidisciplinary panel of clinicians, including neurologists, dietitians, and therapists (Supplementary Table S1).

Enrolled patients received an SMS text at 30, 60, and 90 days after hospital discharge, prompting them to complete the simplified modified Rankin scale (mRS) (20), a single-item, seven-level, ordinal measure of global disability, and then the Patient-Reported Outcomes Measurement Information System (PROMIS) Global-10, a 10-item measure of physical health, mental health, social health, pain, fatigue, and overall perceived quality of life. Individual items from each assessment were sent via SMS-text messages one at a time (i.e., each question of each assessment was one text message). Participants responded by directly texting the number corresponding to the answer of their selection. Participants had 1 week from receiving the prompt to complete the assessment at each time point. Responses were automatically saved in a secure database. Participants who completed all questions associated with the mRS at any given time point were considered responders, while those that did not were considered nonresponders. Participants who completed the mRS but did not complete all questions associated with the PROMIS Global-10 were still considered responders but were not scored on this assessment. The mRS can be dichotomized into good (score 0–2) and poor (score 3–6) outcomes (21). Global Physical Health (GPH) and Global Mental Health (GMH) z-scores (mean: 50; standard deviation (SD): 10) were derived from two 4-item summary scores extracted from the PROMIS Global-10 questions (22). At the conclusion of the program, enrolled patients received a satisfaction survey via an SMS text with six questions that had multiple-choice response options. Participants were counted as responders to the satisfaction survey if they responded to at least one question.

To evaluate the reliability of outcome measure scores obtained via SMS texting, mRS and PROMIS Global-10 scores closest in time to SMS-text responses were extracted from documented traditional follow-up encounters (clinic visit or follow-up phone call), when available. The clinically documented score was compared to the score from the closest SMS-text response in time.

To compare the yield and cost of the SMS texting approach to gather outcomes after acute stroke discharge with traditional methods of ascertaining outcomes, we added clinical staff and utilized a trained coordinator to call all consecutive patients discharged with stroke during a 3-month period to obtain their outcomes approximately 90 days after hospitalization discharge. The mRS and PROMIS Global-10 scores were also assessed via phone calls in the same order as SMS texts. Three attempts were made to reach each patient. Call attempts and time lengths were documented. The results of this intervention were then used to compare the cost between the two strategies (SMS texts vs. phone calls) for gathering poststroke outcomes.

Statistical analysis

Participant characteristics were examined with mean and SD, median and interquartile range (IQR), or n (%). Independent sample t-tests and chi-squared tests of independence were performed to compare clinical and demographic characteristics between those enrolled in the program and those not enrolled and between those who responded to the assessment prompts (responders) and those who did not (nonresponders).

Weighted Cohen’s kappa (23) was calculated to assess the agreement between SMS texts and clinician- or coordinator-gathered responses for the mRS, and polychoric correlations were calculated to assess the agreement between GPH and GMH scores (subscales of PROMIS Global-10). For the mRS, we examined agreement using the ordinal level (with quadratic weights, Supplementary Table S3) and the dichotomized level (good vs. poor) outcomes.

Descriptive statistics were used to characterize the time, yield, and cost comparison of SMS texting vs. phone call-based methods of gathering outcomes and to examine the results of the satisfaction survey.

The Massachusetts General Brigham Institutional Review Board (#2021P001342) approved this study, which was exempt from written informed consent as the data extracted for this study were gathered under the standard of care through a quality improvement initiative. Data will be made available from the corresponding author upon reasonable request.

Results

Of the 530 patients discharged from MGH with a stroke ICD-10 code between June 8, 2020, and February 1, 2021, 350 patients (66.04%) were enrolled in the program. Patients enrolled on average 6.8 ± 8.2 (mean ± SD) days after acute stroke hospital admission. Patients were not enrolled if they did not have valid contact information in the medical chart (n = 151) or declined to receive messages (n = 29) (Figure 1). Enrolled patients were more likely to have English listed as their preferred language (χ² = 5.44, p = 0.02), have a lower NIH stroke scale upon admission (t = 5.0, p ≤ 0.001), and have been discharged directly home from the hospital (χ² = 4.20, p = 0.04) compared to those patients who did not enroll (Table 1).

Figure 1.

Study flow diagram. Cohorts at each assessment time point are unique.

Table 1.

Cohort demographic and clinical characteristics.

	Enrolled (350)	Not enrolled (180)	p	Responders (142)	Nonresponders (208)	p
Age	66.3 ± 16.8	69.4 ± 16.7	0.5	65.2 ± 15.1	67.0 ± 17.7	0.06
Sex (male)	196 (56.0%)	92 (51.1%)	0.3	78 (54.9%)	118 (56.7%)	0.6
Hospital LOS	6.8 ± 8.2	8.1 ± 7.6	0.3	5.5 ± 6.4	7.6 ± 9.1	0.01*
NIH stroke scalea	3 [1–6]	5 [2–11]	<0.001*	3 [1–6]	8 [5–14]	<0.001*
Preferred language
English	312 (89.1%)	154 (85.6%)	0.02*	133 (93.7%)	179 (86.1%)	0.009*
Other	38 (10.9%)	26 (14.4%)		9 (6.3%)	29 (13.9%)
Discharge destination
Home	200 (57%)	86 (48%)	0.04*	100 (70.4%)	100 (48.1%)	<0.001*
Facility	150 (43%)	94 (52%)		42 (29.6%)	108 (51.9%)
Principal problem
Ischemic	237 (67.7%)	124 (68.5%)	0.2	94 (66.2%)	143 (68.8%)	0.5
Hemorrhagic	90 (25.7%)	51 (28.3%)		37 (26.1%)	53 (25.5)
TIA	23 (6.6%)	5 (2.8%)		11 (7.8%)	12 (5.8)

LOS, length of stay; TIA, transient ischemic attack.

Scores reported as mean ± SD, median [interquartile range], or n (%).

*denotes statistical significance.

^a NIH stroke scale scores at acute stroke hospital admission were available for 241/350 (68.9%) enrolled, 127/180 (70.6%) not enrolled, 100/142 (70.4%) responders, and 141/208 (67.8%) nonresponders.

Of those who enrolled, 40.5% (n = 142) responded to at least one SMS text to complete an assessment. The response rate at 30-day postdischarge was 28.6%, and the response rates at the 60- and 90-day time points were 24.3%. Of the responders, 30% (n = 42) responded at any two time points and 30% (n = 43) of responders responded at all three time points (Figure 2A). SMS-text response compliance is presented in Figure 2B. Responders to message prompts were more likely to have English as their preferred language (χ² = 9.33, p = 0.009), have shorter acute hospital length of stay (t = 2.5, p = 0.01), have a lower NIH stroke scale upon admission (t = 3.98, p < 0.001), and have been discharged directly home (χ² = 24.94, p < 0.001) (Table 1).

Figure 2.

(A) Number of SMS-text assessment responses at different time points depicted in a Venn diagram (N, N%). N indicates unique participant responses. For example, N = 13 participants responded at 60 days poststroke, N = 20 answered at both 30 and 60 days, and N = 43 responded at all three study time points. (B) SMS-text assessment compliance over time. Shades of gray (dark to light) correspond to those that responded at all time points, those that responded at two timepoints (T₁ and T₂, T₁ and T₃, or T₂ and T₃), and those that responded at a single time point (T₁, T₂, or T₃). SMS, short message service.

The median (IQR) modified Rankin scale score collected by SMS testing was 1 (0–3) at 30-, 60-, and 90-day postdischarge. The median (IQR) GPH scores were 44.9 (42.3–50.8), 47.7 (42.3–54.1), and 47.7 (41.1–54.1) and the median GMH scores were 45.8 (38.8–50.8), 43.5 (38.8–53.3), and 45.8 (36.9–50.8) at 30-, 60-, and 90-day postdischarge, respectively (Supplementary Table S2). The distributions of these outcomes gathered via SMS texting at 90 days are shown in Figure 3.

Figure 3.

Distributions of 90-day outcomes (mRS, PROMIS Physical, and PROMIS Mental) gathered via SMS texting. SMS, short message service; mRS, modified Rankin scale; PROMIS Physical, Global Physical Health Score; PROMIS Mental, Global Mental Health Score.

The mRS [median: 1 IQR: 1–3) from clinical follow-up encounters, within 13.0 ± 14.9 days of SMS-text responses across collection time points (Supplementary Figure S1), was available for 113 of the 142 patients who responded (Figure 4). Weighted Cohen’s kappa between mRS scores obtained from SMS texting compared to follow-up encounters revealed almost perfect agreement ($\kappa$ = 0.8, p < 0.001) for dichotomized (good vs. poor) and ordinal (with quadratic weights, Supplementary Table S3) levels ($\kappa$ = 0.8, p < 0.001) of the mRS. The PROMIS Global-10 score was not routinely collected and so was only available for 19 patients from clinical encounters. There were significant associations between traditional methods and SMS texting of ascertaining PROMIS subscores ($\rho$ = 0.4, p < 0.01, GPH, and $\rho$ = 0.4, p < 0.01, GMH) (Figure 4).

Figure 4.

(A) mRS compared by modality (traditional encounter vs. SMS texting) across all collection time points (30, 60, and 90 days) for N = 113. Bubble plot to assess agreement between outcomes collected by traditional programs vs. SMS texting. Sizes of bubbles are directly proportional to the number of participants who provided answers via traditional methods and SMS texting. Subjects with data at multiple time points were compared at the latest available time point. The stacked bar near axes shows mRS distributions as collected by each modality with dark to light gradient representing scores 0–5, respectively. (B) PROMIS Physical and PROMIS Mental compared by modality for N = 19. Bubble plot to assess agreement between outcomes collected by traditional programs vs. SMS texting. The size of bubbles is directly proportional to the number of participants who provided answers via traditional methods and SMS texting. SMS, short message service; mRS, modified Rankin scale; PROMIS Physical, Global Physical Health Score; PROMIS Mental, Global Mental Health Score.

To compare the yield of SMS texts with that of phone calls, we attempted to complete a 90-day phone call for all patients discharged within a 3-month time period. Of the 169 stroke patients discharged, we reached 104 (61.4%) by phone. Of those reached by phone, 59 (56.7%) were contacted on the first call attempt, 28 (26.9%) were contacted on the second attempt, and 17 (10.1%) were contacted on the third attempt. For every successful phone call, there were 2.5 unanswered calls. As compared to those who did not answer, patients who answered the phone calls were more likely to have English listed as their preferred language (t = 3.9, p ≤ 0.001). Phone calls took 5.4 ± 2.9 minutes to complete. Unanswered calls took approximately 1.5 minutes.

We estimated the cost per successful assessment achieved via SMS texts vs. phone calls. The total annual cost of the SMS-text program was $12,500 to configure and run, with a cost per assessment defined as $\frac{\mathrm{\$}12,500}{\mathrm{a}}$ (where a = # of assessments). For phone calls, we estimated a fixed cost of 20% of a coordinator’s time (for training and documentation) and added the average cost per phone call (both successful and unsuccessful) to each patient (n). In Boston, Massachusetts, coordinators make an average (avg.) of $60,000 and thus $0.48 per minute. Answered phone calls cost $2.50 $(5.4\min \times \mathrm{\$}0.48/\min )$, while unanswered phone calls cost $\mathrm{\$}0.70(1.5min\times \mathrm{\$}0.48)$ on average. This analysis yielded the following:

$$\frac{\begin{array}{c}[\mathrm{fixed}\mathrm{coordinator}\mathrm{cost}+(\mathrm{avg}.\mathrm{cost}\mathrm{of}\mathrm{a}\mathrm{successful}\mathrm{call}\times \mathrm{n})\\ +(\mathrm{avg}.\mathrm{cost}\mathrm{of}\mathrm{an}\mathrm{unsuccessful}\mathrm{call}\\ \times \mathrm{avg}.\mathrm{number}\mathrm{of}\mathrm{unanswered}\mathrm{calls}\times \mathrm{n})]\end{array}}{n}$$

or

$$\frac{[\mathrm{\$}12,000+(2.5\times \mathrm{n}+\mathrm{\$}0.70\times 2.5\times \mathrm{n})]}{\mathrm{number}\mathrm{of}\mathrm{subjects}},$$

suggesting that SMS texts become less costly than traditional phone calls if used for outcomes assessment in more than 181 stroke patients per year.

The results of the satisfaction survey revealed that the majority of patients found the educational material received via SMS texts to be helpful (53%), with only a small portion of patients finding information burdensome (6%). Most participants felt there was just the right number of tips (79%) and that the messages were clear and easy to understand (97%). Most felt that they were able to easily pick between the choices (91%) in the SMS texts that best described their recovery. Most participants (91%) reported feeling as comfortable answering questions by SMS texts than by answering in person or via phone calls (Table 2).

Table 2.

Satisfaction survey results.

Question	n (%)
1: How helpful was it to receive a text with information, educational content, and tips about stroke?
I did not find it helpful at all	2 (5.6)
Neutral	15 (41.7)
I found it very helpful	19 (52.7)
2: Did you find receiving information about stroke via text burdensome?
I did not find the program burdensome	31 (93.9)
I found the program burdensome	2 (6.1)
3: What did you think of the number of tips and questions?
Too few reminders	1 (3.0)
Just the right number of reminders	26 (78.8)
Too many reminders	6 (18.2)
4: The text messages about stroke were clear and easy to understand.
Agree	32 (97.0)
Disagree	1 (3.0)
5: I was able to easily pick between the choices in the text messages that best described my degree of stroke recovery.
Agree	30 (90.9)
Disagree	3 (9.1)
6: I was as comfortable answering questions by text as if I were answering in person or on the phone.
Agree	30 (90.9)
Disagree	3 (9.1)

Satisfaction survey results are reported at n (%).

Discussion

This is the first study to systematically examine the feasibility, reliability, cost, and acceptability of gathering outcomes after acute stroke via SMS texts. This novel method was found to be highly reliable for collecting the mRS scores and moderately reliable for collecting PROM scores. Even without specific program marketing or patient engagement campaigns, the SMS-text program yielded a 40.5% response rate. Compared to direct patient phone calls, SMS texting yielded fewer responses but is cost-saving for centers with annual stroke discharges exceeding 181 patients based on our costing equation. The experience of receiving text messages with assessments and brain health tips was overall very well received by stroke patients.

We found almost perfect agreement between mRS scores at both the dichotomized (good vs. poor outcome) and ordinal levels obtained via SMS texting compared to traditional methods. This high reliability provides the foundation for systematic evaluation of stroke survivors’ outcomes at a large scale, which could help evaluate the efficacy of stroke treatments. We found moderate agreement with traditional methods for patient-reported measures. This suggests that certain stroke outcomes (i.e., ordinal ratings of global disability) may be more suitable for text-message programs. Prior studies for smoking cessation (24) and depression (25) have also found mixed reliability (fair to substantial) for self-report assessments. Digital means for collecting stroke outcomes may be limited by stroke-related impairments (i.e., language or cognitive deficits). Further exploration is required to examine the reliability of collecting different types of outcomes via SMS texting, particularly in stroke. Furthermore, modality-specific outcome measures, such as motor or language assessments after stroke, may require other types of digital or sensor-based approaches (26–28).

SMS texting had an overall lower yield (40.5%) in this study compared to targeted phone call follow-up method (61%, phone calls). The first attempt at reaching participants via SMS texts also yielded lower responses (28.6%) than the first attempt at reaching participants via phone calls (56.7%). This differs from a prior study that received more SMS-text data than paper diary data after birth control insertion, although notably this patient population was substantially younger as compared to ours (29). Stroke survivors who tend to be older and often suffer stroke-related deficits may be limited in their ability to use cell phones or read and write SMS texts. Survivors might also have limited access to their mobile devices when discharged to a facility, as a majority (70.4%) of responders were discharged home. Engaging caregivers to help with outcomes data collection via digital technology may be helpful in these cases. Low SMS-text response rates could also be attributed to participants feeling more comfortable declining to participate via SMS texts rather than directly to a care team member via phone calls. Another reason could be that the number of required questions needed to complete the assessments by SMS texts was too burdensome, and future research should determine the optimal number of questions to response ratio. In addition, future programs with a dedicated patient and caregiver outreach and SMS-text reminders (30) will likely yield higher response rates.

While the majority of stroke patients consented to SMS-text communication, there were differences between those who consented and those who did not. Individuals who consented to receive SMS texts were more likely to have English listed as their preferred language than another language, have a lower NIH stroke scale at admission, and be discharged home rather than to a postacute care facility. Similar differences were found between those who responded to SMS-text assessments vs. those who did not. Furthermore, outcomes gathered via SMS texts revealed that responders had predominantly mild disability (median mRS of 1, with an interquartile range of 0-3). Outcomes gathered via SMS texts may not be representative of all stroke survivors. A nutrition education program for low-income parents that used SMS texts for program evaluation also found limitations in those who could be reached (31). In stroke, different approaches may be required to reach non-English speakers (32) and those with more severe disabilities (33). If the use of SMS-text programs can diminish the burden of manual collection, outcome collection systems can work in parallel to focus human resources on the more healthcare marginalized and disabled.

Due to the fixed cost of the SMS-text program for an unlimited number of participants, higher response rates would render a lower cost per participant. This contrasts with traditional methods such that outcomes collected during clinic visits or by phone calls require more resources for additional participants and thus cost increases per participant. Therefore, identifying the cost-to-response ratio that would favor SMS texting over traditional methods is essential for developing cost-saving programs. In our study, we show that gathering data via SMS texting would be cost-saving at scale for larger populations. Alternatively, it could also be cost-effective for smaller populations requiring outcome assessments at a greater frequency. For example, a previous study showed that SMS texting was more cost-effective for a weekly, two-question survey than the same paper-based survey (34) due to the frequency demand of the assessment. Future programs should consider the number of participants, data type, and sampling frequency and length in determining the cost-to-benefit ratio when using SMS texting.

Patients who participated in the program found it acceptable without additional burden. The majority found that the messages were clear, easy to understand, and easy to answer. The delivery of brain health tips was well received. These results are consistent with results from the acceptability of SMS texting in diverse clinical populations including individuals with depression (25), high blood pressure (35), and psychosis (36). Given the pervasive use of cell phones in modern society, cell phone-based outcome programs have significant promise for a wide range of patient populations including those with stroke. Although it is feasible to collect the mRS score via a mobile app (19), SMS texting leverages existing software without requiring a smartphone, additional download, or application knowledge.

Overall, our findings suggest that collecting functional outcomes via SMS texting during the first 3 months after stroke is feasible, acceptable, and reliable but that reach and cost-effectiveness should be further considered for broad clinical translation. Future programs should consider developing content in multiple languages and incorporating dedicated patient outreach materials. For example, educational material on how to view and respond to SMS-text communications delivered during the acute stroke inpatient stay could be helpful. Such content would help reach vulnerable populations. Increasing the yield of SMS-text outcome programs would decrease the cost per participant and make broad adoption across healthcare systems more feasible.

This study has several important limitations. The study was conducted at a single, urban, academic medical center in the northeastern United States with a predominantly White patient population. Findings may not be translatable to other hospitals in different locations with different patient populations. A limitation to communication via SMS texting is the chance that someone other than the intended recipient is receiving or interacting with the SMS texts. Moreover, the program could not gather information on the number of subjects who passed away during the study. In longitudinal data collection via SMS texting, subjects may see their responses from prior time points (in the SMS-text chain), which may lead to recall bias. Our sample of PROM data via both SMS texting and traditional methods was small (n = 19), and thus future studies with larger samples are needed to draw definitive conclusions. At last, the potential effects of our stroke and brain health educational program delivered via SMS texting were not systematically considered in our cost analysis.

Conclusion

Our study suggests that it is feasible, reliable, and acceptable to provide general stroke education and gather functional outcome measures via SMS-text messaging after acute stroke discharge. Replication of our results in an independent cohort and further validation of specific outcome types and assessment frequency and length are warranted. Our findings lay the foundation for using SMS texting to gather outcomes after stroke to better evaluate the real-world efficacy of stroke therapies.

Funding Statement

This research was funded in part by a gift from Marriott Foundation.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The Massachusetts General Brigham Institutional Review Board (#2021P001342) approved this study, which was exempt from written informed consent as the data extracted for this study was gathered under the standard of care through a quality improvement initiative.

Author contributions

All authors provided sufficient contributions to warrant authorship. JAD, KSE, MP, KE, PG, LHS, and DJL contributed to drafting and revising the manuscript for content. JAD, MP, KE, LHS, and DJL had a major role in acquiring data. JAD, LHS, and DJL contributed to conceptualizing and designing the study. JAD, KSE, PG, LHS, and DJL contributed to analyzing and interpreting the data. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2023.1043806/full#supplementary-material.