Improving Family Navigation for Children with Autism: A Comparison of Two Pilot Randomized Controlled Trials

Abstract

Objectives:

Family Navigation (FN), a care management strategy, helps families overcome systems and person-level barriers to care. We previously demonstrated FN’s feasibility, acceptability, and potential efficacy for increasing access and reducing time to autism-related diagnostic services among low-income, minority children. In this paper, we describe modifications to FN in response to concerns raised in our first pilot randomized controlled trial (RCT), and then assess these modifications in a second pilot RCT.

Methods:

An advisory group recommended modifications to recruitment procedures and study conditions. 40 parent-child dyad participants with autism-related concerns were randomized to receive modified usual care (UC) or modified FN. We compared whether the first and second pilot RCTs differed in: participant enrollment, satisfaction with clinical care, and timely completion of the diagnostic assessment.

Results:

Recruitment improved under the modified protocol with significantly fewer potentially eligible families refusing (19.5% vs. 4.8%, p < .05) or being excluded from study enrollment (43.6% vs. 0%, p < 0.01). Comparing the first and second pilot RCTs, regardless of study arm, families in the second pilot were more likely to complete diagnostic assessment (UC: HR 3.41, 95% CI [1.20, 9.68]; FN: HR 2.64, 95% CI [1.31, 5.30]) and report greater satisfaction with clinical care. In the second pilot, compared to UC, FN continued increase the likelihood of completing the diagnostic assessment (HR: 2.57; 95% CI [1.22, 5.40]).

Conclusions:

Easy-to-implement system-level enhancements improved study recruitment, satisfaction with care, and completion of a diagnostic assessment. With enhancement, FN continued to confer benefits to families.

Introduction

Autism spectrum disorder (ASD) can be diagnosed by age 2 years, and early intervention impacts core symptoms of the disorder.^1–6 To promote early detection, the American Academy of Pediatrics recommends universal autism screening at the 18 and 24-month well-child visits.⁷ However, despite recent advances,^8–10 minority and low-income children continue to be diagnosed later than white, more resource advantaged children,^10–14 receive fewer and lower quality services,^13,15,16 and wait longer for services.¹⁷ There is growing interest in strategies that explicitly aim to reduce racial and ethnic disparities in access to care by addressing these barriers in a culturally appropriate manner. Family Navigation (FN) is one such strategy.^18–21 FN is a lay-delivered care management intervention adapted from patient navigation, which has a strong evidence base in addressing disparities in cancer care.¹⁸ FN extends patient navigation to focus on the entire family rather than the individual patient.²²

Our team conducted a pilot randomized controlled trial (RCT) between 2012 and 2014 that demonstrated feasibility and acceptability of FN for families of children with ASD.²³ Forty parent-child dyads with children ages 15 months to 6 years who had a confirmed positive screen for ASD in primary care were recruited and randomized to either a FN or a usual care (UC) condition. Time to completion of diagnostic assessment was significantly shorter among families who received FN.²³ However, weaknesses were identified in this first pilot related to study recruitment and the FN intervention. Thus, after completing this first pilot study, and prior to initiating a large-scale effectiveness trial of FN,²⁴ we opted to conduct a second pilot parallel arm RCT in the same health system. In this second pilot, we redesigned recruitment procedures to align more closely with workflows in primary care. In addition, we refined aspects of the two study arms, with the goal of improving quality of care of UC and the efficacy of the FN intervention. This paper reports the results of the second pilot RCT, with specific comparisons between the first and second pilot RCTs. Our research questions were: 1) Did modifications to study recruitment procedures - embedded participant recruitment in primary care practices as opposed to a specialty care clinic - affect study enrollment? 2) Did modifications to UC affect satisfaction with care and time to completion of an ASD diagnostic assessment? 3) Did modification to FN affect satisfaction with care and time to completion of an ASD diagnostic assessment?

Methods

Modification Approach.

After completing the first pilot RCT and before initiating the second pilot RCT, we convened an advisory group consisting of the principal investigator, a research coordinator, a developmental and behavioral pediatrician, three bilingual, bicultural women with experience as family navigators, a biostatistician, and the parent of a child who was evaluated for ASD risk. The group reviewed process and outcome data from the first pilot study²³ and made recommendations for modifications in recruitment procedures and intervention conditions. A comparison of the original and modified recruitment, UC, and FN intervention approaches are presented in Table 1. Primary outcomes and measures of primary outcomes were identical across both pilot RCTs.

Table 1.

Modifications to original Usual Care and Family Navigation

Modification Area	First Pilot	Second Pilot
Recruitment Processes
Referral source	Developmental and Behavioral Pediatrics Specialty Care Clinic	Primary Care clinics
Eligibility	Age 15 months-6 years with concern for ASD based on positive M-CHAT-R/F or clinical judgment family must meet social risk criteria	Age 15–36 months with concern for ASD based on positive M-CHAT-R/F or clinical judgment; no social risk criteria
Usual Care
Diagnostic evaluation scheduling procedures	Three diagnostic evaluation visits scheduled separately	All three diagnostic assessment visits scheduled simultaneously. Added evening appointment confirmation calls to parents
Patient-centered coordination in medical home	No communication of activities or results after referral to study	Communicated level of risk from study M-CHAT-R/F screen and information regarding status of DBP referral back to primary care physician
FN Intervention
Dosage	Target number of navigator visits: 3	Target number of navigator visits: 4
Follow-Up Period	Final navigator meeting: at time of diagnosis	Final navigator meeting: 100 days after diagnosis
Navigator Training	Topics: navigation, ASD, and health care systems	Added motivational interviewing and collaborative decision-making training

Setting.

Participating sites were six pediatric primary care practices in an integrated health network in the Northeastern United States that served ethnically diverse, low-income families and shared a common electronic health record, and an academic medical center’s Developmental and Behavioral Pediatrics (DBP) specialty clinic. The DBP clinic, located at the safety-net hospital, was the major ASD referral center for children in this network.

Study Recruitment and Participants.

Following advisory group recommendations, study recruitment procedures were changed in three ways: 1) children were referred to the study by their primary care provider rather than from the DBP specialty clinic to decrease time from identification of ASD risk to study enrollment; 2) the upper age of eligibility criteria was decreased from 6 years to 36 months to align with clinical priorities to expedite diagnostic services for very young children demonstrated to gain the most from early intensive intervention;⁷ and 3) social risk criteria (i.e., racial/ethnic minority, teenage mothers, primary language other than English) were removed to increase the pragmatic character of the trial. Resulting inclusion criteria were child age between 15- and 36-months with confirmed risk for ASD based on results on the Modified Checklist for Autism for Toddler Revised with Follow-Up Interview (M-CHAT-R/F)²⁵ or parent/provider concern, and parent/caregiver spoke English, Spanish, or Haitian Creole. Children with a prior diagnosis of ASD were excluded. The study’s sample size was based on principles guiding sample size in pilot studies.²⁶ and the experience in the first pilot RCT, in which a sample size of 40 was found adequate to assess modifications of trial procedures. After completion of baseline measures, parent-child dyads were randomly allocated to one of two study conditions using a 1:1 computer-generated random number sequence created by the study statistician (HC). The allocation sequence was concealed from all study personnel in sequentially numbered opaque sealed envelopes. The study was approved by the Institutional Review Board at our institution’s medical campus.

Intervention Conditions

Usual Care.

Traditional UC for an ASD diagnostic assessment in the study’s health systems began with referral from the child’s primary care site to the DBP clinic for a three-session diagnostic assessment. The first intake appointment was scheduled and confirmed by a telephone call from the specialty clinic to the family. Children whose family could not be reached during working hours did not have an appointment scheduled; children who did not attend their initial appointment did not have subsequent appointments scheduled unless the family contacted the clinic. UC in this study was modified in 3 ways: 1) confirmatory M-CHAT-R/F screens were conducted by the study team per recommended algorithm²⁵ after the child’s DBP clinic referral was received; screening results were shared with the child’s primary care physician and the DBP clinic to better inform clinicians about the child’s ASD risk behaviors; 2) parents were given the dates for all three DBP clinic appointments upon referral to decrease delays in communication between the specialty clinic and family; and 3) confirmation calls for all DBP appointments were made in both daytime and evenings, and in the family’s preferred language. These changes were implemented by research staff and applied to all study participants.

Family Navigation.

Family navigators in the second pilot were trained and worked with families in a similar manner as navigators in the first pilot.²² Navigator activities with families included individual face-to-face, phone, and email encounters across home, clinic, and community settings. Navigators identified barriers to accessing services, provided emotional support, and coordinated care across the community, educational system, and healthcare system. Navigator workbooks included the action plans, a guide to navigation visits, documentation of services, and information about relevant community-based resources.²² The navigators were five women with no previous experience with ASD, two of whom were bilingual and bicultural. These characteristics were selected to increase capacity for cultural and linguistic brokering and to assess the capacity of diverse individuals to fill the navigator role regardless of prior clinical experience. Navigators were trained during a 2-day workshop that focused on four competency areas for navigation: fundamentals of community health work; linkage to services; cultural competency; and advocacy. In addition, navigators received ASD-specific training. They met weekly with a master’s level clinician for case supervision.

The advisory group determined that modifications to 1) intervention dosage, 2) follow-up timing, and 3) navigator training (Table 1) would likely improve parent engagement and satisfaction with navigation services, as well as intervention efficacy. The intervention dosage was increased from three to four in-person navigation visits. Follow-up timing was extended to 100-days after the completion of the diagnostic assessment to provide ongoing care coordination during the critical period after diagnosis when families often seek knowledge about their child’s new diagnosis and begin to access ASD-specific treatments and services. Lastly, motivational interviewing and collaborative decision-making training were added to the navigator training to improve the navigators’ abilities to engage and work collaboratively with families.

Measures

The measures and data collection procedures described below are the same as those used in the first pilot study.²³ All measures were administered by research staff blinded to intervention condition. Satisfaction with clinical care was assessed using the Information, Patient Autonomy, and Emotional Support subscales of the Hospital Care Questionnaire (HCQ).²⁷ The HCQ was previously shown to have high reliability and validity.²⁷ At face value, the items on these scales were relevant to our study population. In our sample, internal consistency on the HCQ showed high reliability, with a Cronbach’s alpha of 0.83.

Satisfaction with the navigator was assessed using the Patient Satisfaction with Interpersonal Relationship with Navigator (PSN-I), a 9-item parent questionnaire.²⁸ Although developed in the context of navigation for cancer care, the PSN-I assesses the navigator’s interpersonal skills such as listening ability, communication, and problem solving, which are relevant to the study population.²⁸ In the current sample, the PSN-I had very high reliability, with a Cronbach’s alpha value of 0.97. The HCQ and PSN-I were administered as in-person interviews in English, Spanish, or Haitian Creole two weeks after completion of the diagnostic assessment or, for families who did not complete the diagnostic process, 90 days after the child’s first scheduled DBP clinic visit.

Completion of the diagnostic assessment, the primary outcome, was defined as the number of days between confirmed risk for ASD and date of diagnostic ascertainment (e.g. when the diagnosis of ASD or other development disorder was conferred). We also measured completion of the diagnostic assessment within 200 days as a binary variable. These data were collected from medical records.

Study enrollment was measured through standard tracking of participants referred, eligible, and enrolled in the study. Prior to initiating the second pilot, the actual impacts of moving study recruitment from the specialty clinic to primary care were unknown to the investigators. Thus, the proportion of dyads referred to the study who were eligible for participation was the primary recruitment outcome of interest.

Statistical Analysis

To address our first research question, “Did modifications to recruitment procedures affect study enrollment?”, we assessed differences in 1) proportion of families eligible for the study who enrolled and 2) proportion of families who were excluded from the study because we were unable to complete baseline assessment prior to the first diagnostic assessment visit. We used chi-square tests to statistically compare the proportions between the first and second pilot studies. An alpha level of 0.05 was used to determine statistical significance in all analyses.

The second and third research questions examined the effects of changes to UC (Question 2) and FN (Question 3) from two perspectives. To assess satisfaction with care, we analyzed scores on the HCQ questionnaire among UC participants and FN participants in the two pilots. For families who received FN, we also analyzed results from the PSN-I using two sample t-tests. To examine how changes in the intervention conditions affected time to diagnosis, we used an intention-to-treat (ITT) analysis to assess time to diagnosis between the two trials. We examined three comparisons: UC in study 1 vs. study 2; FN in study 1 vs. study 2; and UC vs. FN in study 2. We created Kaplan-Meier plots and estimated the hazard ratio (HR), the relative likelihood of completing the diagnostic assessment, with 95% confidence intervals (CI) using a Cox proportional hazards model. We verified the proportional hazards assumption by including a group-by-time interaction term in the Cox regression models.

Results

We documented the proportion of patients who were referred to this study, not eligible, and declined. A total of 85 children were referred to the study: 38 did not meet inclusion criteria, 3 could not be contacted, and 4 declined participation. Forty parent-child dyads were recruited during the months of March through December 2014 (Figure 1). All dyads were randomized and received the allocated intervention. Participant follow-up was completed August 2015.

Figure 1:

Study CONSORT diagram

Average age was 24.5 months at time of enrollment; 32.5% were female. Families were largely minority and low-income, based on Medicaid participation. The only difference across study arms was gender. Within 200 days of follow-up, 72.5% of participants completed their diagnostic assessment; among those evaluated, 52% received an ASD diagnosis; 48% were diagnosed with another developmental or behavioral disorder. Sociodemographic factors of participants are listed in Table 2 (See Supplemental Table 1 for demographics of participants in the first pilot). Participants in this study were similar to those in the first pilot study with the exception of their age, due to a modification in age criteria for inclusion in the second pilot.

Table 2.

Sample characteristics in second pilot

	Study Condition
	FN (n = 20)	UC (n = 20)
Child Characteristics
Mean child age, months (SD)	23.2 (5.8)	25.8 (6.2)
Gender (%)
Female	50%	15%
Male	50%	85%
Mean M-CHAT-R/F score (SD)	7.2 (2.4)	7.9 (4.6)
Receiving EI services	85%	85%
Parent Characteristics
Race/Ethnicity (%)
Hispanic, Any Race	45%	60%
Black, Non-Hispanic	25%	25%
White, Non-Hispanic	5%	0%
Asian, Non-Hispanic	5%	0%
Other/Unknown	20%	15%
Language other than English spoken in the home (%)	30%	40%
Public Insurance (%)	80%	90%
High school graduate (%)	40%	55%
Married or living with partner (%)	60%	35%
Currently working outside of home (%)	30%	20%

Question 1: Did modifications of study recruitment pocedures – embedding recruitment in primary care practices as opposed to a specialty care clinic – affect study enrollment?

In the first pilot, 140 families were assessed for eligibility; 133 met eligibility criteria. However, 26 eligible families refused participation and another 61 were excluded because their child had a first DBP diagnostic assessment visit before enrolling in the study.²³ In this study, in which families referred to the study directly from primary care rather than the DBP clinic, only 2 of 42 eligible families refused to participate (19.5% vs. 4.8%, p < .05). Also in this study, no eligible families were excluded due to attending a DBP appointment prior to study enrollment (43.6% vs. 0%, p < 0.01).

Question 2: Did modifications to usual care affect satisfaction with care and time to completion of diagnostic assessment?

Comparing the UC conditions across the two pilot studies, parents assigned to the UC condition in this study reported greater satisfaction with clinical care (mean HCQ score, s.d. 86.3, 16.1 vs. 70.4, 22.3, p = 0.045); their children were significantly more likely to complete diagnostic assessment (60% vs. 26%; HR 3.41, 95% CI [1.20, 9.68]). The four curves in the Kaplan Meier plots in Figure 2 depict days-to-completion of the diagnostic assessment for the FN and UC groups in the first and second studies (days 0 to 200).

Figure 2.

Time to completion of ASD diagnostic assessment for children in usual care and FN conditions in the first pilot study (left) and second pilot study (right)

Question 3: Did the modifications to FN affect satisfaction with care and time to diagnosis?

Comparing FN conditions across the two pilot studies, parents in this study who received FN reported greater satisfaction with clinical care (HCQ mean score, s.d. 95.2, 5.5 versus 86.9, 12.4; p = 0.018). Satisfaction with the family navigator was not significantly different between the two studies (PSN-I mean score, s.d., 34.3, 3.8 (study 2); 34.5, 2.4 (study 1); p = 0.907). Children in this study who received the modified FN were significantly more likely to complete diagnostic assessment than children who received FN in the first study (95% vs 70%; HR 2.64, 95% CI [1.31, 5.30]). See Figure 2. We found children who received the modified form of FN continued to benefit even when improvements were made to UC (HR comparing FN to UC in Study 2 = 2.57; 95% CI = 1.22, 5.40; p=0.01, Figure 2).

Discussion

We described a set of changes to study recruitment procedures, enhancements to UC, and modifications to the FN intervention made to address weaknesses in our first pilot study. We then evaluated the impact of these changes on participant enrollment, satisfaction, and time of diagnosis. We found signs of improvements across all areas, while maintaining the significant effect of FN to increase the likelihood of diagnostic assessment completion. Overall, our findings show that the modified UC and FN intervention resulted in improved patient satisfaction and reduction in time to diagnostic ascertainment.

The first important finding was that the change in the recruitment site from the DBP specialty clinic to primary care facilitated more timely and efficient enrollment of study participants. The substantial difference in eligible families who enrolled in the study may reflect the importance of embedding trials in primary care settings, where families have trusted and ongoing relationships. This finding may be particularly important for trials that seek to enroll low-income and racial/ethnic minority families who have not been well represented in clinical trials. Furthermore, this finding has an important implication for clinical practice. FN may more effectively help patients access specialty care when the navigation process begins in primary care. A model of FN embedded in primary care likely results in increased continuity between primary care and specialty providers, thus strengthening the medical home for underserved children.

Study findings also suggest that easy-to-implement, low-cost changes to UC systems can lead to substantial improvements in timely completion of ASD diagnostic assessment among historically underserved families. The changes included scheduling dates for the three evaluation visits at the same point in time, thus reducing delays related to difficulty communicating with families, and outreach to families during the evening and in their preferred language. Such changes may be particularly salient for families for whom structural barriers related to accessing care exist and suggest that adaptations to UC alone may lead to improvements in clinical care. A third contribution of the study relates to the refinements made to FN. The additional training in motivational interviewing and collaborative problem solving were associated with the increased likelihood of children in this pilot study completing diagnostic assessment, compared to the first pilot study.

Consistent with results of the first pilot study, comparisons of FN and UC support the potential of FN to significantly affect satisfaction with clinical care and the likelihood of diagnostic assessment completion. We found that even with improvements among families who received UC, FN continued to confer additional benefit in this sample of predominantly low-income families from racial/ethnic minority backgrounds.

Additional, unexpected study findings highlighted the importance of pilot studies and iterative modifications of study conditions prior to conducting large scale trials. We found a diminution of hazard ratio between the first and second pilot studies, such that the first pilot showed larger effects on diagnostic assessment completion than the second pilot. Without data from both pilots, we would be unaware that this change was actually due to improvement in UC, not a decrease in the effectiveness (or “voltage drop”) of FN.²⁹ Moreover, had we powered our current large, multi-site trial on the first pilot’s study design, we may have overestimated effect sizes. This finding makes a compelling case for pilot studies of modified models of interventions, as well as the need to publish data from such studies. It also demonstrates that for this population, structural changes to UC improved care for all – but did not eliminate positive effect of having a navigator.

Limitations

This study has a number of limitations. First, it uses a design in which we compare results from a current trial to a historical trial. This method could be limited if additional changes in UC setting exist over time and should be considered when interpreting study results. Additionally, it is important to note differences in eligible children’s age range between the two pilot populations. It is possible that population differences could account for some of the differences we detected. Furthermore, the referral source change we implemented may have changed our population in other ways that was not captured in our data.

Conclusions

Despite the proliferation of FN as a strategy to reduce health care disparities,^18,19 it infrequently implemented and evaluated in a rigorous way. In this study, FN was a manualized intervention that was modified, implemented, and evaluated using a scientific approach. The study’s findings provide preliminary evidence that FN improved the time to ASD diagnostic assessment completion, even in the context of enhanced UC. It also suggests that easy-to-implement modifications to UC, as well as modifications to FN, can improve both participant satisfaction with clinical services and the likelihood of timely diagnostic assessment completion. For practitioners, findings can support development of patient navigation models in primary care pediatric care settings. For researchers, this work highlights the importance of carefully and iteratively refining interventions prior to implementing large-scale effectiveness trials.

FN was intended to be a brief intervention that addresses limitations of existing service systems; however, our pilot work has demonstrated some of FN’s broader implications. While FN interventions do not directly change the health care systems in which they are implemented, they provide a unique lens to understand the gaps that lead to system failures. For minority and low-income children who are disproportionately affected by these system failures, FN bridges the divide between evidence-based practice and service delivery and thus, has the potential to reduce health disparities. Family navigators are well poised to address cultural and stigma-related barriers that are likely to exist regardless of the quality of the health care system. Furthermore, the FN intervention in this study engaged families in collaborative decision-making, which is designed to help parents prepare to become better advocates for their children. In the large-scale effectiveness trial that is currently underway,²⁴ we are collecting data on service engagement over a one-year follow-up period. Given the need to demonstrate effects of FN with additional populations, this larger clinical trial is being conducted across large care networks in three cities. Future analyses will use these data to address the long-term impact of FN on child and family outcomes.

What’s New.

Targeted modifications to usual care and Family Navigation improved satisfaction with clinical services and likelihood of timely completion of autism diagnostic assessment. Even in the context of enhanced usual care, Family Navigation showed promise as a strategy to reduce disparities in timely autism diagnosis.