Electronic participation-focused care planning support for families: a pilot study

Abstract

AIM

To evaluate the feasibility, acceptability, and preliminary effects of Participation and Environment Measure-Plus (PEM+) 2.0, an optimized version of a web-based, participation-focused, care-planning tool.

METHOD

Twenty-two caregivers of children aged 0 to 5 years receiving rehabilitation services, who reported dissatisfaction with their child’s participation, had internet access, and could read and write English, were recruited for this 2-week, single-arm pilot trial. Feasibility was assessed through retention rates, completion time, percentage of care plans developed relative to caregiver reported need, and independent completion of PEM+ 2.0. Acceptability was assessed by the Usefulness, Satisfaction, and Ease of Use Questionnaire. Preliminary effects were assessed by two items on caregiver reported impact of PEM+ 2.0 on confidence for addressing their child’s participation.

RESULTS

Eighteen caregivers completed at least one iteration of PEM+ 2.0; of those, 17 were female and 15 were 30 to 39 years old. The median completion time was 12.99 minutes (quartile 1, 6.30; quartile 3, 17.33), mean care plan creation relative to need was 50% (standard deviation [SD] 31), and 17 completed PEM+ 2.0 independently. Mean acceptability scores were 3.80 to 4.97 (SD 1.25–1.97) and mean preliminary effect scores were 4.61 to 4.72 (SD 1.85–2.24), out of 7.0. There were strong and significant positive associations between two of the three estimates of PEM+ 2.0 acceptability and caregiver confidence (r=0.577–0.793, p<0.01).

INTERPRETATION

Electronic health tools have the potential for facilitating family-centered care in pediatric rehabilitation. PEM+ 2.0 is a feasible tool within pediatric rehabilitation and has potential to be an acceptable tool for improving caregiver confidence for promoting their child’s participation in valued activities.

Children’s participation in valued home, school, and community activities is a key patient-reported outcome and research priority in pediatric rehabilitation.¹ There has been a rapid proliferation of assessments to advance knowledge on children’s participation,² and a new conceptual framework to organize translational research on this important rehabilitation outcome.³ However, there are few effective interventions to promote participation, particularly among young children.

A growing number of studies have highlighted the importance of intervening on the child’s environment to improve their participation in activities.^4–7 Since caregivers are commonly tasked with addressing contextual factors (e.g. physical layout of space, sensory features) when caring for their child,^8,9 their expertise is critical for successfully designing and monitoring participation-focused rehabilitation.¹⁰ However, designing family-centered and participation-focused care takes considerable provider and patient resources, and often relies on a standard face-to-face approach.¹¹

Electronic assessment options may help organizations that have electronic data capture systems to reduce provider burden and expand provider reach for shared decision-making with caregivers when planning and monitoring participation-focused rehabilitation interventions.^12,13 The Participation and Environment Measure-Plus (PEM+) is a new electronic health application that facilitates family-centered and participation-focused intervention planning for young children receiving rehabilitation therapies. PEM+ helps caregivers to design plans of care that are directed toward changing their child’s attendance and/or involvement in meaningful activities and is congruent with the family of participation-related constructs framework.³ To use PEM+, caregivers first complete a baseline assessment of their child’s participation (i.e. the child’s frequency of attending activities and involvement while attending activities), using the Young Children’s Participation and Environment electronic patient-reported outcome (YC-PEM e-PRO).^14,15 Then they proceed to PEM+ where they get structured guidance on how to systematically use all of their YC-PEM e-PRO responses to create a care plan consisting of identified priorities for intervention, specific and meaningful goals for intervention, and intervention strategies for goal attainment. PEM+ was designed to mimic the clinical reasoning used to plan participation-focused care.

PEM+ design was informed by caregiver and provider input,^16,17 and an initial PEM+ prototype underwent usability testing with six caregivers of children 0 to 3 years old using the tool for only one iteration.¹⁸ Usability results informed significant optimizations to the PEM+ prototype, to include the following: (1) the addition of activity illustrations that selectively appeared at the start and end of each PEM+ step to visually prompt and focus the user about the activity for which they had chosen to create a plan; (2) a loopback function that enabled the user to seamlessly progress through multiple iterations of PEM+ without losing previous work; (3) new functionality to more directly route caregivers to the second strategy exchange function of PEM+; (4) simplified words and phrases to introduce each step; and (5) a storage function designed to give users direct access to each completed care plan. These optimizations resulted in PEM+ 2.0. Usability results also informed the decision to proceed with testing of PEM+ 2.0, a revised prototype, in one more round given current industry standards for the iterative development of health technologies.¹⁸ During subsequent testing of PEM+ 2.0, the need to expand study enrollment to families of children across the 0 to 5 year age range was based on the fact that PEM+ is designed to be paired with the YC-PEM, which is validated for use with caregivers of children 0 to 5 years old; therefore, PEM+ should be tested across the full 0 to 5 year age range as well.¹⁹

As a logical next step in non-pharmacological intervention development, piloting PEM+ 2.0 with children 0 to 5 years old is needed, to assess its feasibility (i.e. assessing whether this optimized tool can be used), acceptability (i.e. assessing whether the optimized tool is acceptable/satisfactory to its users), and the preliminary effects of using it on clinically relevant outcomes. This testing helps to determine whether the implementation of PEM+ 2.0 is practical by testing the process (e.g. recruitment and retention rates, user experience), resources (e.g. completion time, need for assistance), management (e.g. data management), and scientific basis (e.g. variance in clinical endpoints, identifying subgroup effects to test in future phases) for the intervention.^20–22 Feasibility and pilot studies have gained increased recognition for their importance in developing complex health interventions. They are critical at a pre-trial stage to assess the potential for successful implementation of an intervention and to reduce threats to study validity.^20,21 They assure individual site preparedness and identify prototype and protocol modifications before undertaking a definitive trial.^20,23

Given that PEM+ is a complex intervention in terms of its focus on participation and its electronic mode of administration to engage families, the primary aim of this pilot study was to examine its feasibility and acceptability as a technology-based option for family-centered and participation-focused intervention planning among caregivers of young children receiving rehabilitation services. The secondary study aim was to explore the preliminary effects of PEM+ 2.0 on caregiver confidence in promoting their child’s participation to identify appropriate outcome measures for future efficacy trials. Together, study results will inform PEM+ optimizations and protocol modifications before undertaking a definitive phase 2 clinical trial.

METHOD

Setting and participants

This was a single-arm pilot study which has been registered at www.clinicaltrials.gov under NCT03820011. Ethics approval was obtained from the Office for the Protection of Research Subjects at the University of Illinois at Chicago before recruitment and data collection (November 2017 to January 2018). Although formal sample size calculations are not appropriate for pilot studies, a target sample of 24 caregivers was planned on the basis of previous enrollment rates in our pilot phases of e-PRO implementation in early intervention.¹²

As shown in Figure S1 (online supporting information), a total of 27 eligible caregivers were recruited through convenience sampling from two community-based programs in the USA that provide rehabilitation services: an early intervention program (Denver, CO) (n=16) and an early childhood educational program (Chicago, IL) (n=11). These programs use participation-focused assessment within their processes of care and have the organizational capacity for research engagement as indicated by their previous engagement with research studies and quality improvement initiatives.²⁴

Caregivers were eligible for inclusion if they: (1) were 18 years or older; (2) read, wrote, and spoke English; (3) had a child aged 0 to 5 years who received rehabilitation services; (4) had broadband internet access; and (5) denoted at least one area of desired change in their child’s participation during using the YC-PEM e-PRO. All eligible caregivers confirmed their eligibility online. A total of 22 of the 27 participants recruited confirmed their eligibility, and 18 of the 22 participants who confirmed study eligibility continued with PEM+ 2.0 use.

Intervention

PEM+ 2.0 is a web-based care-planning application that is designed for use by caregivers of young children receiving rehabilitation therapies. Caregivers who complete the YC-PEM e-PRO to evaluate their child’s participation can click on a web link to begin the PEM+ application, whereby they build on their YC-PEM responses to create a participation-focused care plan to share with their child’s rehabilitation team. PEM+ is a five-step recursive process (i.e. the caregiver can repeat the five-step process to create as many care plans for their child as needed), and it offers tiered support to the user during completion (tier 1, frequently asked questions available in the application; tier 2, support provided by e-mail; tier 3, support provided by telephone). Details of the PEM+ process have been described elsewhere.¹⁸

For this study, PEM+ was optimized to include new functionality, resulting in PEM+ 2.0 that includes: (1) artist-rendered illustrations of each activity in the YC-PEM e-PRO, one of which is continuously displayed in the user interface to visually prompt and focus the user once they have selected a priority activity to address in the first step of the PEM+ process; (2) a loopback function that enables the PEM+ to be recursive, such that the user can complete multiple iterations of PEM+ without losing previous work; (3) new functionality to directly route caregivers to the second strategy exchange function of PEM+; (4) simplified words and phrasing to introduce each PEM+ step; and (5) a visible storage function in the user interface that is designed to give users direct access to each of their completed care plans. In contrast to previous phases of PEM+ testing, caregivers for this study were instructed to complete PEM+ 2.0 over a 2-week period. This time frame was selected by the research team to mimic what would be provided in the routine care-planning processes of early intervention and early childhood education at both study sites (Alexa Greif, personal communication, 13th July 2017).

Measures

We used measures that aligned with established guidelines for addressing feasibility (i.e. process, management, and resources), and the scientific basis (acceptability, preliminary effects) of the optimized PEM+ intervention (PEM+ 2.0).²⁰ To describe the study sample, select social and clinical characteristics of enrolled caregivers were drawn from caregiver completion of an online demographic questionnaire and service record abstraction. Both sample characteristics and results are reported for the total sample, and by age subgroups (0–3y, 4–5y).

Feasibility

The feasibility of completing PEM+ 2.0 was assessed quantitatively over a 2-week period. We used web analytics to obtain estimates of mean retention rates, median completion time for the first iteration, median completion time of subsequent iterations, percentage of care plans relative to the number of activities that caregivers indicated change desired, and the number of caregivers who completed a PEM+ iteration independently (i.e. no tiered support).

Acceptability

After completing their 2-week trial of PEM+ 2.0, caregivers reported on the acceptability of PEM+ 2.0 by completing the Usefulness, Satisfaction, and Ease of Use Questionnaire (USE).²⁵ The USE contains 22 items over four dimensions (usefulness, satisfaction, ease of use, and ease of learning) to evaluate the extent to which technologies are effective, efficient, and satisfying for individuals to use. The items are rated on a 7-point scale, from 1 (strongly disagree) to 7 (strongly agree). Examples include ‘I quickly became skillful with it’ and ‘It meets my needs’. A mean score for each of the four dimensions was calculated ranging from 1 to 7. In our previous phase of usability testing, participants were provided with a single opportunity to provide open-ended feedback at the end of the USE measure; however, this option was not used during the usability testing. As such, for this study, caregivers were provided with opportunities after each individual USE item to provide open-ended feedback.

Preliminary effects

The purpose of PEM+ is to empower caregivers to generate goals and initial intervention strategies so that they can collaborate effectively and efficiently with their child’s rehabilitation team in designing a participation-focused intervention. The primary aim of this study was to assess the feasibility and acceptability of PEM+ 2.0 before a definitive larger trial focused on intervention efficacy. To assure appropriate endpoints to evaluate during this subsequent trial,^20,21 we assessed preliminary effects of PEM+ 2.0 for the first time, by assessing for caregiver-reported initial impact of PEM+ 2.0 on caregiver confidence for addressing their child’s participation by two items developed for this project:²⁶ ‘This [PEM+] care plan material will influence what I do next’ and ‘This [PEM+] process has influenced what I think about my child’s participation and what can be done to improve it’. Each item was rated on a 7-point scale, from 1 (strongly disagree) to 7 (strongly agree).

Data collection

Eligible caregivers were directed online to a study website to confirm their eligibility and provide informed written consent. After consent, caregivers were asked to complete a demographic questionnaire and the YC-PEM e-PRO, as the PEM+ application pulls data from the completed e-PRO (e.g. activities with which caregivers identified being dissatisfied with their child’s participation) and displays it in an organized and tailored sequence so that caregivers can use the information from their completed assessment to create a care plan for their child.¹⁸ During YC-PEM e-PRO assessment, caregivers first evaluated their child’s current level of home and community participation, both in terms of attendance and involvement, and indicated whether they desired change in their child’s participation in specific activities (e.g. getting clean). If they desired change, they were then asked to list strategies they used to facilitate their child’s participation in that activity.¹⁴ Additionally, in the YC-PEM e-PRO, caregivers evaluated their child’s environment in terms of its impact on the child’s participation in a setting (e.g. home and community).

After YC-PEM e-PRO completion, caregivers were immediately directed to an automated YC-PEM report online and shown a web link to begin PEM+ 2.0 online with continued access for 2 weeks. In PEM+ 2.0, new functionality afforded a single automated e-mail notification to be sent to caregivers to log back in and continue using PEM+ when they were inactive for 48 hours (i.e. caregiver had not logged into their account for 48h). If caregivers had still not continued with PEM+ 48 hours after this e-mail notification, research staff contacted caregivers to encourage them to log back in and proceed in PEM+. New functionality also afforded caregivers to receive an automated ‘half-way through’ e-mail to notify them when seven out of their 14 days had passed. At 2 weeks, caregivers were directed by a web link to evaluate PEM+ 2.0 acceptability and report on the extent to which using PEM+ 2.0 changed their confidence for caring for their child, and they were then mailed a $40USD gift card payment.

Data analysis

All analyses were completed on the total sample and age subgroups (0–3y, 4–5y) using SPSS 24.0 (IBM SPSS Statistics, IBM Inc., NY, USA). The primary study aim was to assess the feasibility and acceptability of PEM+ 2.0. To assess PEM+ feasibility, we reported on mean retention rate, median completion time of the first iteration, median completion time of subsequent iterations, and the use of tiered support. To assess PEM+ 2.0 acceptability, total USE scores were reported as means with standard deviations, as well as minimum and maximum scores.

All open-ended responses were first independently analysed by four members of the research team (JMJ, VCK, DA, RT) using an inductive approach whereby categories were created to fit the content. Upon further discussion of coded content with a key informant (MAK), some nodes/categorizations were further collapsed or revised to yield preliminary results that were reviewed again by the remaining team members and the key informant to assure trustworthiness of results.²⁷

As a secondary aim, preliminary effects of mean caregiver confidence item scores were calculated, and correlation coefficients were calculated to ascertain associations between proportion of care plans created relative to areas of desired change and acceptability scores with caregiver confidence scores. Pearson product-moment correlation coefficients and Spearman’s ρ correlation coefficients were calculated depending on whether the data met relevant assumptions (e.g. normal distribution). Results were interpreted using the following criteria: r=0.10 to 0.29, weak; r=0.30 to 0.49, moderate; r≥0.50, strong.²⁸

RESULTS

Sample characteristics

As shown in Table 1, caregivers (n=18) were predominantly female (n=17), white (n=14), and had a family income above the US median household income (n=15).²⁹ On average, caregivers desired change in their child’s participation across 45% (SD 28) of home and community activities. The mean age of children was 3 years 5 months (SD 1y 6mo).

Table 1:

Sample characteristics

	Total sample (%)	0–3y (%)	4–5y (%)
	n=18	n=12	n=6
Mother or female guardian	94	92	100
Caregiver age (y)
21–29	6	8	0
30–39	83	83	83
40–49	11	8	17
Caregiver education
High school graduate or equivalent	17	8	33
Bachelor’s degree	61	67	50
Graduate degree	22	25	17
Caregiver ethnicity
Asian	12	8	20
Black	6	8	0
White	77	83	60
Other	6	0	20
Family incomea
≤$60 000	19	27	0
>$60 000	81	73	100
Percent desire change in participation	45 (28)	35 (24)	64 (29)
Child’s age (y:mo)	3:5 (1:6)	2:5 (0:10)	5:2 (0:6)
Child’s rehabilitation service use
Speech therapy	67	50	100
Occupational therapy	83	75	100
Physical therapy	56	42	83
Other	50	42	67

Continuous variables are presented as mean (SD) and categorical variables as percentages.

^aFamily income is reported relative to national estimates of the median household income in US dollars (USD).

PEM+ 2.0 feasibility

Eighteen of the 22 eligible participants who gave consent completed the 14-day PEM+ 2.0 trial. Seventeen caregivers completed the PEM+ 2.0 independently, one caregiver requested telephone support. There were no intervention-related adverse events reported.

For the total sample, the median completion time for the first iteration was 12.99 minutes (quartile 1 [Q1], 6.30; quartile 3 [Q3], 17.33) which decreased for subsequent iteration, ranging from 3.40 to 6.45 minutes (Table 2). The median number of care plans created by caregivers was 2.50 (Q1, 1.75; Q3, 4.00), and the mean proportion of care plans created, relative to the number of activities in which caregivers desired change, was 50% (Q1, 12%; Q3, 100%).

Table 2:

Participation and Environment Measure-Plus (PEM+) 2.0 feasibility, acceptability, and preliminary effects

	Total sample	0–3y	4–5y
	n=18	n=12	n=6
Number of care plans	2.50 (1.75, 4.00)	2.50 (1.25, 4.00)	3.00 (1.75, 8.00)
Proportion of care plans developed relative to need	50% (12–100)	55% (44)	41% (40)
Percentage of care plans for the home setting	83% (31)	88% (24)	73% (44)
Median time to complete care plan (min)
First care plan (n=18)	12.99 (6.30, 17.33)	11.75 (6.80, 16.80)	13.10 (2.50, 30.00)
Second care plan (n=14)	6.45 (3.95, 21.32)	6.40 (3.20, 24.30)	7.70 (4.57, 34.15)
Third care plan (n=9)	3.40 (2.10, 6.45)	3.55 (1.93, 9.18)	3.40 (2.10, 6.45)
Fourth care plan (n=7)	4.20 (1.30, 7.20)	2.75 (1.15, 7.13)	7.10 (2.40, 7.20)
Care plans downloaded by caregiver	2.56 (2.85)	2.25 (2.90)	3.17 (2.99)
USE
Usefulness	4.41 (1.54)	4.77 (1.31)	3.67 (1.81)
Satisfaction	3.80 (1.97)	4.11 (1.98)	3.19 (1.95)
Ease of use	4.64 (1.25)	4.93 (1.26)	4.06 (1.11)
Ease of learning	4.97 (1.57)	5.21 (1.73)	4.50 (1.83)
Caregiver confidence
‘PEM+ influenced what I will do’	4.61 (1.85)	4.83 (1.95)	4.17 (1.72)
‘PEM+ influenced what I think about my child’s participation and what can be done to improve it’	4.72 (2.24)	5.33 (1.88)	3.50 (2.59)

Continuous variables are presented as mean (SD) or median (quartile 1, quartile 3) and categorical variables as percentages. Usefulness, Satisfaction, and Ease of Use Questionnaire (USE) score: 1, strongly disagree; 4, neither agree nor disagree; 7, strongly agree.

PEM+ 2.0 acceptability

Mean PEM+ 2.0 acceptability scores were 4.41 (SD 1.54; Q1 3.38, Q3 5.53) for usefulness, 3.80 (SD 1.97; Q1 1.97, Q3 5.43) for satisfaction, 4.64 (SD 1.25; Q1 3.73, Q3 5.73) for ease of use, and 4.97 (SD 1.57; Q1 3.88, Q3 6.00) for ease of learning, all measured on a 7-point scale. Table S1 (online supporting information) summarizes primary areas of caregiver feedback from the open-ended comments within the USE organized according to the following categories: clarity of instructions and steps; perceived effort; perceived effects; and PEM+ 2.0 features. For each category, illustrative exemplars highlight features that were found to be helpful, areas of concern, and suggestions for improvement.

PEM+ 2.0 preliminary effects

On average, caregiver perceptions of PEM+ impacting their confidence, in terms of ‘what they will do next’ and ‘how they think about their child’s participation and how to support it’, were 4.61 (SD 1.85; Q1 3.75, Q3 6.00) and 4.72 (SD 2.24; Q1 2.75, Q3 7.00) out of 7.

There were strong and significant positive associations between two of the three estimates of PEM+ 2.0 acceptability and caregiver confidence. Caregiver report of how PEM+ 2.0 impacted ‘what they will do next’ had a strong and positive association with caregiver perceptions of PEM+ usefulness (r=0.785, p<0.001), satisfaction (r=0.793, p<0.001), and ease of use (r=0.577, p<0.01). Caregiver report of how PEM+ 2.0 impacted ‘how they think about their child’s participation’ had a strong and positive association with caregiver perceptions of PEM+ 2.0 usefulness (r=0.750, p<0.001), satisfaction (r=0.794, p<0.001), ease of use (r=0.729, p<0.001), and ease of learning (r=0.509, p<0.05). There was no significant association between the proportion of care plans created, relative to areas of desired change identified, and caregiver confidence (see Table 3).

Table 3:

Association between Participation and Environment Measure-Plus (PEM+) 2.0 acceptability and caregiver confidence

	‘PEM+ influenced what I will do’	p	‘PEM+ influenced how I think’	p
USE
Usefulness	0.785a	<0.001	0.750b	<0.001
Satisfaction	0.793a	<0.001	0.794b	<0.001
Ease of use	0.577a	<0.01	0.729b	<0.001
Ease of learning	0.329a	0.182	0.509b	<0.05
Proportion of care plans to desire change	0.123b	0.628	0.148b	0.558

^aPearson’s product-moment correlation coefficient.

^bSpearman’s ρ correlation coefficient. USE, Usefulness, Satisfaction, and Ease of Use Questionnaire.

DISCUSSION

PEM+ is an electronic rehabilitation intervention designed to increase organizational capacity to customize processes of care in ways that afford client–provider communication and shared decision-making when planning participation-focused care. Organizations that deliver rehabilitation services can opt to integrate e-PRO, such as the YC-PEM e-PRO, into their electronic data capture systems to improve client-centered clinical workflows. Organizations that have electronic data capture systems and use the YC-PEM e-PRO are poised to benefit from PEM+ to support family engagement during the design of rehabilitation care plans.¹⁸

Pilot studies of PEM+ are needed to prepare for a future clinical trial. Our usability testing of an initial PEM+ prototype focused on its technical features to drive optimizations to the intervention and study protocol before this feasibility study of PEM+ 2.0. Results of this study suggest that PEM+ 2.0 is feasible to deliver within an early childhood intervention context. Caregivers, on average, completed multiple care plans within a 2-week period. Mean completion time for the first iteration was similar to previous work.¹⁸ Additionally, there was a decrease in mean completion time when comparing the first and subsequent PEM+ 2.0 iteration(s), decreasing by nearly half between the first and second PEM+ 2.0 iteration. While most caregivers (n=13) created multiple care plans,³⁰ there was a wide range in the number of care plans created within the 2-week timeframe, which could reflect differences in caregiver need for professional support when planning care and/or their comfort with technology use. Since the number of care plans created does not provide evidence of care plan quality, a separate qualitative sub-study was undertaken to establish new evidence of PEM+ 2.0 care plan quality.³⁰

Mean acceptability scores indicated that caregivers’ responses to PEM+ 2.0 were neutral to somewhat favorable. Specifically, average scores for usefulness, ease of use, and ease of learning were slightly above neutral and approaching positive product ratings (i.e. scores above 5 on a 7-point scale).²⁵ Caregiver ratings on ease of use and ease of learning may be related to positive perceptions of the stepwise structure of PEM+ 2.0 and its flexible design (Table S1). However, neutral scores may indicate that PEM+ 2.0 needs further optimization or that it is an acceptable option for some, but not all, caregivers, highlighting the need for individualized approaches to engaging caregivers in the care-planning process. Flexibility in child care services has been identified as a priority concern of mothers whose child has long-term care needs.³¹

In contrast to the other three indicators of acceptability examined, caregiver satisfaction with the current PEM+ 2.0 prototype was lower than expected, as evidenced by mean USE ratings that were slightly below neutral and comparable amounts of positive and critical feedback obtained through open-ended responses. Caregivers voiced positive feedback on PEM+ 2.0 in terms of guiding them to think differently about their child’s goals, assist them to achieve more, and engage them more collaboratively in care planning. However, a recurrent point of critical feedback was caregiver concern that PEM+ 2.0 was burdensome. While PEM+ is designed for integration into a clinical workflow, PEM+ 2.0 was not tested in this manner and there was no structured opportunity for caregiver-provider discussion of the PEM+ report. As a result, caregiver effort for PEM+ 2.0 completion was not consistently reinforced in the clinical setting within the current study protocol, which may have contributed to lower satisfaction with PEM+ 2.0. Future trials will be designed to take place within the clinical workflow and will continue to assess PEM+ acceptability when used in this manner.

Previous studies have shown that caregivers can generate a wide range of participation-focused strategies,^32,33 and caregivers in this study commented positively about opportunities to seek and share expertise with other users by the PEM+ 2.0 strategy exchange feature. These results coincide with those from the qualitative sub-study that found most caregivers using PEM+ in this study (n=15) created complete and participation-focused care plans,³⁰ and that caregivers downloaded multiple care plans on average when prompted to do so in the optimized application. Therefore, caregivers can successfully use PEM+ 2.0 for its intended purpose of care planning and may respond positively to structured opportunity to share their PEM+ 2.0 report(s) with their child’s provider team, inclusive of their strategies for goal attainment. New hybrid designs in translational research that investigate both effectiveness and implementation to enable earlier implementation of interventions may be appropriate for subsequent phases of PEM+ testing.³⁴

PEM+ is a complex rehabilitation intervention. Therefore we chose to take on the task of preliminary exploration of intervention effects before designing an efficacy trial for the purpose of assuring an appropriate outcome to measure in the efficacy trial. This study was powered for feasibility and acceptability and not for efficacy testing, so the results should be interpreted within the context of informing a future trial. The preliminary effects of PEM+ 2.0 show that caregiver experience with PEM+ 2.0 is significantly associated with how they think about supporting their child’s participation and what they will do next. This is similar to findings in which young people’s confidence in planning and participating in social activities improved after completing a new electronic participation-focused intervention targeting transition-aged young people with acquired brain injuries.³⁵ Furthermore, the strong and positive associations between the acceptability estimates and caregiver confidence suggest that, as PEM+ 2.0 is optimized to increase its acceptability for caregivers, it has the potential to further impact their confidence in supporting their child’s participation in everyday routines. Future trials should therefore include measures related to caregiver confidence as primary clinical endpoints in future PEM+ testing, after further optimizations of the tool on the basis of caregiver feedback from this trial.²⁶

Limitations

Results should be interpreted in light of limitations pertaining to sampling and study measures, which may limit the generalizability of the results. With regard to sampling, as condition severity may impact PEM+ feasibility and the effect on caregiver confidence, subsequent phases of PEM+ testing will recruit larger samples at study sites with established electronic data capture systems, to afford the abstraction of demographic and clinical data for subgroup comparisons. Additionally, the study measures included two items that were developed to explore the potential effect of PEM+ 2.0 on caregiver confidence. There is a need to include a validated measure of caregiver confidence and/or self-efficacy in future PEM+ testing.

CONCLUSION

This was a single-arm pilot study to assess PEM+ 2.0, an electronic health application for organizations to engage caregivers in designing participation-focused intervention. Results support the feasibility of delivering the PEM+ 2.0 intervention in a clinical trial and highlight areas for optimization to improve acceptability and the likelihood of detecting significant effects of PEM+ 2.0 on caregiver confidence. Together, these results lay the foundation for further testing of PEM+, which is the first electronic health application that is designed to support family engagement with participation-focused care planning for young children receiving rehabilitation therapies. The next steps will involve further optimizations to PEM+ 2.0 on the basis of caregiver feedback, before future testing. Future testing will benefit from the engagement of organizations with electronic data capture and from using a hybrid effectiveness-implementation study design within clinical workflow.

Supplementary Material

supp figureS1

Figure S1: Enrollment and completion trends.

supp TableS1

Table S1: Caregiver feedback on Participation and Environment Measure-Plus 2.0 acceptability.

What this paper adds.

• Participation and Environment Measure-Plus (PEM+) 2.0 is feasible for collaboratively engaging caregivers in the care-planning process.

• Caregivers perceived PEM+ 2.0 as helpful for thinking about their child’s participation.

• Feasibility and acceptability results inform PEM+ 2.0 optimizations.

ABBREVIATIONS

PEM+

Participation and Environment Measure-Plus

USE

Usefulness, Satisfaction, and Ease of Use Questionnaire

YC-PEM e-PRO

Young Children’s Participation and Environment electronic patient-reported outcome