Development and psychometric testing of the AASPIRE Adult Autism Healthcare Provider Self-Efficacy Scale

Abstract

Our objective was to develop a measure of healthcare providers’ self-efficacy in providing healthcare to autistic adults and to better understand their training needs. We used a community-based participatory research approach with academic researchers, autistic adults, supporters, and healthcare providers. We developed a one-page questionnaire which included the new 6-item self-efficacy scale, two items on how challenging and rewarding it is to provide care to autistic adults, and seven items on provider characteristics. We surveyed 143 healthcare providers from eight primary care clinics in Oregon and California, United States. Preliminary psychometric testing found the Academic Autism Spectrum Partnership in Research and Education Adult Autism Healthcare Provider Self-Efficacy Scale to have good internal consistency reliability (Cronbach’s alpha 0.87) and consist of a single factor. A priori hypothesis testing found correlations in the expected directions. Only a minority of providers reported high confidence in communicating with patients (25%); performing physical exams or procedures (43%); accurately diagnosing and treating other medical issues (40%); helping patients stay calm and comfortable during visits (38%); identifying accommodation needs (14%); and making necessary accommodations (16%). While providers need training across all aspects of care related to autism in adulthood, interventions should pay particular attention to helping providers communicate with patients and identify and make accommodations.

Lay abstract

The adult healthcare system is ill-prepared to provide high-quality care to autistic adults. Lack of provider training may contribute to the problem, but there are few previously tested survey instruments to guide provider training efforts. Our objective was to develop and test a measure of healthcare providers’ confidence (or “self-efficacy”) in providing healthcare to autistic adults and to use it to better understand their training needs. We used a community-based participatory research (CBPR) approach, in partnership with academic researchers, autistic adults, supporters, and healthcare providers, throughout the project. We developed a one-page questionnaire and surveyed 143 primary care providers from eight primary care clinics in Oregon and California, United States. Preliminary testing of the AASPIRE Adult Autism Healthcare Provider Self-Efficacy Scale suggests that the measure is reliable and valid. Using this scale, we found only a minority of providers reported high confidence in communicating with patients (25%); performing physical exams or procedures (43%); accurately diagnosing and treating other medical issues (40%); helping patients stay calm and comfortable during visits (38%); identifying accommodation needs (14%); and making necessary accommodations (16%). While providers need training across all aspects of care related to autism in adulthood, interventions should pay particular attention to helping providers communicate with patients, and identify and make necessary accommodations. Future research is needed to further validate this scale and to understand how to meet providers’ training needs most effectively.

Autistic adults experience a greater number of co-occurring physical and mental health conditions than non-autistic adults (Croen et al., 2015; Fortuna et al., 2016), greater healthcare costs (Zerbo et al., 2019), higher rates of premature mortality (Hirvikoski et al., 2016), and higher inpatient mortality rates (Akobirshoev et al., 2020). Not surprisingly, studies have also found that autistic adults experience significant healthcare disparities, with higher reported rates of unmet healthcare needs, higher emergency department visits, lower use of preventive services, and lower ratings of patient–provider communication as compared to non-autistic adults (Nicolaidis et al., 2013). These findings, as well as advocacy from autistic adults themselves, have led to multiple calls for research to improve the health and healthcare of autistic adults (Interagency Autism Coordinating Committee, 2017).

Lack of provider training may significantly contribute to the healthcare disparities experienced by autistic adults. In a recent systematic review about barriers and facilitators to healthcare for autistic adults, challenges communicating with providers and providers’ lack of knowledge about autism were found to be common barriers across multiple studies (Mason et al., 2019). A small, but growing number of studies have documented that healthcare providers themselves feel ill-equipped to provide care to adult patients on the autism spectrum. For example, in a study of 922 adult healthcare providers in an integrated health system in the United States, 77% of providers rated their knowledge about autism in adulthood as “poor” or “fair” (Zerbo et al., 2015). Similarly, a study which used a convenience sample of 304 general practitioners in the United Kingdom found that providers had low to moderate confidence in their ability to provide care to patients on the autism spectrum (Unigwe et al., 2017). Moreover, in a study from the United States that included 139 physicians who care for adult patients on the autism spectrum, 69% stated they would like additional training on autism (Bruder et al., 2012). However, none of these studies use standardized instruments and many rely on very broad questions about self-efficacy or training needs.

Interventions to improve care must be informed by providers’ specific training needs. Such training needs may be assessed, at least in part, using measures of providers’ self-efficacy. Our objectives were to develop and psychometrically test the Academic Autism Spectrum Partnership in Research and Education (AASPIRE) Adult Autism Healthcare Provider Self-Efficacy Scale, a measure of providers’ self-efficacy in providing healthcare to autistic adults, and to use the scale to better understand the training needs of primary care providers (PCPs) who provide care to autistic adults.

Methods

Academic community partnership

The AASPIRE (www.aaspire.org) is a long-standing academic-community partnership that consists of academic researchers, autistic adults, family members, healthcare providers, and disability services professionals (with some members having dual roles). We used a community-based participatory research (CBPR) approach (Israel et al., 2003) throughout the project. Community and academic partners worked together as equal partners, learning from each other and making shared decisions throughout the project. Our participatory approach and collaboration processes are described in detail elsewhere (Nicolaidis et al., 2011, 2019, 2020; Nicolaidis & Raymaker, 2015; Raymaker & Nicolaidis, 2013).

Setting, participants, and recruitment

We conducted this study as part of a larger project to integrate the AASPIRE Healthcare Toolkit (www.autismandhealth.org; Nicolaidis et al., 2016) into three healthcare systems. The project was approved by the Institutional Review Boards of our university and of each health system. The study took part in (a) eight primary care clinics that are part of a large integrated health system in California; (b) two off-site primary care clinics affiliated with an academic medical center in Oregon; and (c) two primary care clinics affiliated with a private health system in Oregon. Two were family medicine clinics; four were internal medicine clinics; and six included a mix of both specialties. PCPs were medical doctors, nurse practitioners, or physician assistants. Five clinics included residents in training.

Data for this analysis come from baseline surveys conducted with PCPs in the participating clinics. Study or clinic personnel identified all PCPs who had at least one patient in their patient panels with an International Classification of Diseases, Ninth Revision (ICD-9) or International Classification of Diseases, Tenth Revision (ICD-10) code consistent with a diagnosis of autism (i.e. 299.0, 299.00, 299.8, 299.80, 299.9, 299.90, F84.0, F84.1, F84.5, and F84.9). PCPs were invited to participate in the survey regardless of whether or not their autistic patients participated in the intervention study. PCP participation in the survey was not tied to their participation in the intervention or overall study. As part of the intervention, PCPs could receive a personalized accommodations report about their autistic patient’s specific needs. The proposed intervention did not include any provider training activities besides receipt of the personalized accommodations report.

We used two recruitment methods. In California, study personnel from the partnering health system sent eligible PCPs an email invitation with a link to the survey. In Oregon, a staff person from each clinic distributed paper copies of the survey to PCPs and personally collected them. Surveys included a cover sheet with information about the study, including that participation was voluntary.

Instrument development and data collection

We used our prior qualitative research (Nicolaidis et al., 2015, 2016) and the lived experience of our CBPR team to develop a questionnaire for PCPs. Our prior studies included an informal survey of 129 PCPs and in-depth qualitative interviews with nine PCPs about their needs and preferences in caring for autistic adults. In addition to autistic adults and supporters, our CBPR team included three PCPs with different levels of experience in caring from autistic adults (ranging from one team member who practices almost exclusively with adults with developmental disabilities to another team member who has had extremely limited contact with autistic patients).

The questionnaire included the new AASPIRE Adult Autism Healthcare Provider Self-Efficacy Scale, which consists of six items on the provider’s confidence providing care to adult patients on the autism spectrum. Responses are on a 5-point Likert-type scale (1 = not at all confident to 5 = very confident). We also included two de novo items on how challenging and rewarding it is to provide care to adults on the autism spectrum on a 10-point scale (1 = not at all to 10 = extremely) and seven items on demographic characteristics, training, and experience with autism (See Supplemental Appendix). We purposefully limited the survey to one page to increase response rates.

Data analysis

We scored the self-efficacy scale by totaling the six items, with a possible range of 6 to 30. We based our psychometric testing on the COSMIN (COnsensus-based Standards for the selection of health Measurement Instruments) Initiative’s international consensus on the taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes (Mokkink et al., 2010). The creation of the tool by our team with lived experience as PCPs and autistic patients also helps support face validity (an aspect of content validity). In this preliminary study, we primarily focused on one type of reliability (internal consistency) and two aspects of construct validity (structural validity and hypothesis testing).

We assessed the self-efficacy scale’s internal consistency reliability using Cronbach’s alpha. In regards to structural validity, we conducted a principal axis factor (PAF) analysis to assess the unidimensionality of the scale (Velicer & Jackson, 1990). We tested a priori hypotheses using pair-wise correlations and t-tests. Specifically, we hypothesized that providers with higher self-efficacy scores would report higher levels of reward when caring for autistic adults and would find providing care for autistic adults less challenging than providers with lower self-efficacy scores.

For presentation purposes, we also dichotomized results of each item on the self-efficacy scale, with 4 or 5 coded as “high confidence,” as we felt that dichotomized results of individual items may be more meaningful than mean values on a Likert-type scale. Finally, we assessed the association of total self-efficacy score with provider demographic characteristics.

Results

Participants

A total of 143 PCPs took part in the survey, including 49 of 53 eligible PCPs in Oregon (92%) and 94 of 401 eligible PCPs in California (23%). Participants were evenly split between internal medicine and family medicine. Table 1 describes participant characteristics.

Table 1.

Participant demographic and training characteristics.

Characteristic	Mean (SD); range
Age	44.8 (9.6); 27–81
	N (%)
Gender
Male	63 (44%)
Female	80 (56%)
Degree
MD/DO	132 (92%)
PA/NP	11 (8%)
Specialty
Family medicine	71 (50%)
Internal medicine	70 (49%)
Med/Peds	2 (1%)
Resident
Yes	14 (10%)
No	127 (90%)
Any special training on autism or developmental disability	10 (7%)
Providers’ estimate of the number of adult patients on the autism spectrum in their panel
0	11 (8%)
1	7 (5%)
2	19 (14%)
3	24 (18%)
4	14 (10%)
5	25 (18%)
6–10	30 (22%)
>10	7 (5%)

MD: Medical Doctor; DO: Doctor of Osteopathic Medicine; PA: physician assistant; NP: nurse practitioner; Med/Peds: providers specializing in both medicine and pediatrics.

Psychometric testing

The AASPIRE Adult Autism Healthcare Provider Self-Efficacy Scale showed good internal consistency reliability with a Cronbach’s alpha of 0.87. The Kaiser–Meyer–Olkin measure verified the sampling adequacy for the factor analysis (KMO = 0.79). All KMO values for individual items in the anti-image correlation matrix were >0.70, which is well above the acceptable limit of 0.5 (Field, 2013; Velicer & Jackson, 1990). Bartlett’s test of sphericity indicated that correlations between items were sufficiently large for PAF (x² (15) = 477.20, p < 0.001), so we proceeded with the PAF. The eigenvalue and scree plot supported unidimensionality, with a single factor explaining 70.05% of the variance.

Our assessment of predicted associations confirmed our a priori hypotheses. Specifically, self-efficacy was positively correlated with feeling of reward in providing care to adults on the spectrum (r = 0.24, p < 0.01) and negatively correlated with reported feeling of challenge in providing care to adults on the spectrum (r = –0.5, p < 0.001). Participants who had provided care to six or more autistic adults had higher self-efficacy scores (mean 19.8; SD 3.6) than those with less experience (mean 17.6; SD 4.2 p < 0.01).

Provider self-efficacy

A minority of providers reported high confidence in communicating with autistic adult patients (25%), performing physical exams or procedures on these patients (43%), accurately diagnosing and treating these patients’ non-autism medical issues (40%), helping autistic adult patients stay calm and comfortable during visits (38%), identifying their accommodation needs (14%), and making necessary accommodations (16%). Table 2 shows more detailed results. There was no difference in self-efficacy score by provider gender, degree (MD/DO vs NP/PA), specialty (internal medicine vs family medicine, excluding the two providers who were double-boarded in medicine and pediatrics), training status (in-training yes/no), or state (California vs Oregon). There was a slight positive association between self-efficacy score and provider age (r = 0.17, p < 0.05).

Table 2.

Provider self-efficacy in caring for adult patients on the spectrum.

Self-efficacy scale items Please rate your confidence in doing the following for your Adult patients on the autism spectrum:	Mean (SD)	Providers who reported high confidence, N (%)
1. Successfully communicating with your patients	2.96 (0.87)	34 (25%)
2. Successfully performing physical examinations or procedures	3.34 (0.91)	61 (43%)
3. Accurately diagnosing and treating other medical issues in your autistic patients	3.32 (0.86)	58 (40%)
4. Helping your patients stay calm and comfortable during a clinic visit	3.21 (0.93)	54 (38%)
5. Identifying what accommodations will help facilitate your patients’ care	2.56 (0.91)	20 (14%)
6. Making the necessary accommodations to facilitate your patients’ care	2.62 (0.93)	23 (16%)
Total Self-Efficacy Score (alpha 0.87)	18.01 (4.25), range 6–29

Response options used a 5-point Likert-type scale, with anchors of 1 = Not at all confident to 5 = Very confident.

Discussion

Preliminary testing of the AASPIRE Adult Autism Healthcare Provider Self-Efficacy Scale demonstrated promising internal consistency reliability and construct validity. PCPs showed low confidence in all aspects of providing care to autistic adults, but training needs may be highest in the areas of communicating with patients, and identifying and making necessary accommodations.

There is a small, but growing literature on self-efficacy in providing services to autistic individuals. For example, Ruble et al. (2013) have developed a 30-item autism-focused self-efficacy scale for teachers, and Mazurek et al. (2017) describe a 57-item scale specifically focused on providing healthcare to autistic children. To our knowledge, there is only one other published self-efficacy scale that includes items related to providing healthcare to autistic adults (Unigwe et al., 2017). This 14-item self-report measure was tested in an online convenience sample of 304 general practitioners in the United Kingdom. While the scale had excellent reliability (alpha 0.95) and may be useful in some provider training projects, it may not be well-suited to projects that focus primarily on improving healthcare for autistic adults. For example, six items focus only on children or ask about self-efficacy related to both children and adults. Such items may be difficult to interpret when used with PCPs who only serve adults, such as those specializing in general internal medicine (as was the case in half of our sample and is common throughout the United States). Similarly, 6 of 14 items focus on self-efficacy related to the diagnosis of autism, allowing this aspect of self-efficacy to have a large influence on the total score. Self-efficacy related to autism recognition or diagnosis may be less relevant for providers who are taking care of patients who already have an autism diagnosis (as was the case in our intervention and others that focus on ongoing care for diagnosed adults). Most importantly, our prior qualitative data (Nicolaidis et al., 2015, 2016) and our lived experience as PCPs or autistic patients pointed to the importance of targeting specific aspects of care, such as providers’ self-efficacy in communicating with autistic patients, performing physical examinations, helping patients stay calm during visits, and identifying and making necessary accommodations. These types of issues have also surfaced in other qualitative studies of healthcare providers who work with autistic adults (Warfield et al., 2015). Such aspects of self-efficacy are not included in the self-efficacy scale by Unigwe et al. (2017) and may be particularly important targets of training interventions. We feel the two scales may have complimentary roles and could be used separately or in combination, depending on the goals of a specific project.

Our study benefited from the inclusion of PCPs from three different types of health systems across two states, but has several limitations. As has been the case with most other provider surveys about autism (Bruder et al., 2012; Zerbo et al., 2015), the response rate was low (23%) for providers from the integrated health system in California. However, responses from providers in California were consistent to those from providers in Oregon, where response rate was 92%, leading us to believe that response bias may have not contributed greatly to results. We found the clinic-based provider recruitment, with paper surveys distributed by clinic staff, to be more effective than recruitment of providers via email messages and would recommend that approach in the future in systems where that is possible. Knowing that providers face extreme time pressures, we prioritized reducing participant burden by limiting the survey to a single page. As such, we were not able to compare results of our new scale to other measures of provider self-efficacy. Furthermore, given no established reference standard, we were unable to assess criterion validity. Future research is needed to further validate this measure, including assessing criterion validity, test–retest reliability, and responsiveness to change. Finally, even though providers were selected because clinic records showed they had at least one autistic adult on their patient panels, 11 (8%) providers indicated they did not provide care to any autistic adults. It is unclear if the discrepancy was due to inaccuracies in medical records, if providers had not recently seen the autistic patients on their panels, or if, as has been seen in other studies (Zerbo et al., 2015), providers underestimated the number of autistic patients they cared for.

The one-page survey, including the 6-item provider self-efficacy scale, shows promise as an efficient, reliable, and valid way to assess healthcare providers’ confidence in caring for adult patients on the autism spectrum. Future research is needed to further validate this scale and to test its responsiveness to change. Given that a particularly low proportion of providers said they felt confident in identifying (14%) or making (16%) necessary accommodations, future research should pay attention to how providers interpret these items and what they understand about accommodations in healthcare. While providers need training across all aspects of care related to autism in adulthood, interventions should pay particular attention to helping providers communicate with patients, and identify and make accommodations. Tools such as the Autism Healthcare Accommodations Tool (www.autismandhealth.org/AHAT) are designed to help providers meet such goals (Nicolaidis et al., 2016), but additional research is needed to test their effectiveness and to understand how to best meet providers’ training needs.