Initial Concurrent and Convergent Validity of the Perceived Access Inventory (PAI) for Mental Health Services

Abstract

Access to high-quality healthcare, including mental healthcare, remains a high priority for the Department of Veterans Affairs (VA) and civilian healthcare systems. Increased access to mental healthcare is associated with improved outcomes, including decreased suicidal behavior. Multiple policy changes and interventions are being developed and implemented to improve access to mental healthcare. The Perceived Access Inventory (PAI) is a patient-centered questionnaire developed to understand the veteran perspective about access to mental health services. The PAI is a self-report measure that includes 43 items across five domains: Logistics (6 items), Culture (4 items), Digital (9 items), Systems of Care (13 items), and Experiences of Care (11 items). This paper is a preliminary examination of the concurrent and convergent validity of the PAI with respect to the Hoge Perceived Barriers to Seeking Mental Health Services scale (concurrent) and the Client Satisfaction Questionnaire (CSQ) (convergent). Telephone interviews were conducted with veterans from three geographic regions. Eligibility criteria included screening positive for post-traumatic stress disorder (PTSD), alcohol use disorder, or depression in the past 12-months. Data from 92 veterans were analyzed using correlation matrices. PAI scores were significantly correlated with the Hoge total score (concurrent validity) and CSQ scores (convergent validity). The PAI items with the strongest correlation with CSQ were in the Systems of Care domain and the weakest were in the Logistics domain. Future efforts will evaluate validity using larger data sets and will utilize the PAI to develop and test interventions to improve access to care.

Access to high-quality healthcare, including mental healthcare, remains a high priority for the Department of Veterans Affairs (VA). Unmet mental healthcare needs and barriers to mental healthcare are associated with more severe mental health symptoms, increased risk of suicidal ideation, and death by suicide (Becerra, Becerra, Hassija, & Safdar, 2016; Hoge & Castro, 2012; Pietrzak et al., 2010). Access to mental healthcare is reported as one of the protective factors associated with a lower likelihood of suicidal behaviors in the 2018–2028 National Strategy for Preventing Veteran Suicide (U.S. Department of Veterans Affairs [VA], Office of Mental Health and Suicide Prevention [OMHSP], 2018). Access to mental healthcare becomes critically important during a time when the age- and sex-adjusted suicide rate in the veteran population is higher than the same rate in the overall US population (VA, OMHSP, 2019).

Over the past several decades, the VA has invested heavily in strategies to improve access to mental healthcare including the introduction of community-based outpatient clinics (CBOCs), integration of mental health services into primary care settings, same day appointment scheduling, intensive case management, increased use of mobile clinics, widespread use of telemental health, and contracting with community providers (Bauer et al., 2016; Godleski, Darkins, & Peters, 2012; Kehle, Greer, Rutks, & Wilt, 2011; Kirchner et al., 2014; VA, Office of Public Affairs, 2014). Recent VA efforts to improve veteran access and outcomes include an Executive Order signed in 2018 which mandated creation of a Joint Action Plan by VA, Department of Defense, and Department of Homeland Security to provide seamless access to mental health services for veterans in the year following discharge, separation, or retirement from the military. VA continues to develop and test strategies to improve access to mental health treatment.

In 2012, the VA Office of Inspector General (OIG) recommended that VA “reevaluate alternative measures or combinations of measures that could effectively and accurately reflect the patient experience of access to mental health appointments” (VA, Office of Inspector General, Offices of Audits and Evaluations and Healthcare Inspections, 2012, p. 8). As reported previously, in response to the call for patient-centered measures of access, we used a multiphase, sequential mixed-methods approach to develop one such measure – the Perceived Access Inventory (PAI) for VA mental health services (Pyne et al., 2020). The PAI is a self-report measure that includes 43 items assessing perceived access to VA mental health services across five domains: Logistics (6 items), Culture (4 items), Digital (9 items), Systems of Care (13 items), and Experiences of Care (11 items) (see supplemental file). These domains reflect and extend the re-conceptualization of access to healthcare generated through the 2010 VA Health Services Research and Development Service (HSR&D) State-of-the-Art Conference (SOTA) on access (Fortney, Burgess, Bosworth, Booth, & Kaboli, 2011; VA, Health Services Research and Development Service, 2010).

This paper examines the concurrent validity of the PAI using the Perceived Barriers to Seeking Mental Health Services (Hoge et al., 2004) and convergent validity using the Client Satisfaction Questionnaire (CSQ) (Larsen, Attkisson, Hargreaves, & Nguyen, 1979). Concurrent validity is a form of criterion-related validity that focuses on the relationship of the measure in question to a separate gold standard or best available measure of the same concept. In this case we defined the Hoge measure as the best available perceived access measure because it is commonly used as a measure of mental healthcare access in veteran and active duty military samples. The Hoge measure was developed from the perspective of active duty personnel. In contrast, the PAI was developed from the perspective of veterans using VA services who identified a broader and more comprehensive set of mutable barriers than those included in the Hoge measure (Pyne et al., 2020). Convergent validity is a form of construct-related validity that focuses on the relationship of the measure in question to a measure of a separate, but theoretically related, concept. In this case, per the SOTA access model, access is theoretically related to satisfaction. Our selection of the CSQ as a convergent validation measure reflects the high priority VA gives to customer service in its policies and its nationwide assessment surveys (e.g., Survey of Healthcare Experiences of Patients (SHEP), Survey of Enrollees, and surveys from the Veteran Experience Office). In the remainder of this article, we report on the concurrent and convergent validity of the PAI across total and domain-level scores.

Method

Design.

This article reports on analyses of cross-sectional data collected via telephone interviews to evaluate the concurrent and convergent validity of the PAI.

Recruitment.

The procedures we used to identify the validation sample were similar to those used in identifying the PAI development sample (Pyne et al., 2020). Briefly, opt-out informational letters were sent to potentially eligible individuals (Miller et al., 2017), defined as United States military veterans aged 18 to 70 years who had at least one positive screen for post-traumatic stress disorder (PTSD), alcohol use disorder, or depression documented in their VA medical record in the previous 12 months, and resided in Arkansas, Maine or Northern California. PTSD, problematic alcohol use, and depression screening measures are routinely administered to veterans receiving care in the VA. At least one positive mental health screen in the past 12 months was used as the first step in establishing perceived need for mental health services. The second step occurred during the recruitment phone call (see below). Veterans with psychosis or dementia documented in the problem list of their medical record were excluded. Potential participants were categorized by geographic region, rural/urban residence, and gender. Selection of veterans to call was purposive, designed to ensure inclusion of specific groups of veterans needed to gain a comprehensive understanding of perceived access to VA mental healthcare (e.g., female veterans were over-sampled to ensure that the sample from each geographic region would include at least 20% women). Trained project personnel telephoned veterans who met purposive sampling criteria and did not opt-out within two weeks of the mailing date of the opt-out letter. During the recruitment phone call, potential participants were asked whether they had reliable access to a phone in case we needed to call them back on another line or at another time to conduct the interview. Potential participants were also asked if they had experienced any stress or emotional problems related to PTSD, alcohol use, or depression in the past year. We excluded veterans who reported no reliable phone access or no stress-related or emotional problems related to PTSD, alcohol use, or depression in the past year. Verbal informed consent was obtained prior to initiating the research interview. Participants were compensated $30 for completing the interview. This study was reviewed and approved by the VA Central Institutional Review Board.

Participants.

We mailed out 639 opt-out letters. A total of 272 veterans were contacted by phone. Ninety-nine (36.4%) of them completed the research interview, 33 from each of the following geographic regions: Northern California, Arkansas, and Maine. However, seven participants did not complete the full battery of measures, leaving a sample of 92 for the analyses reported here (see Table 1). Of the veterans we attempted to contact who did not participate (n=173), 54 were not interested, 44 were not eligible, 24 did not give a reason, 24 phone number did not work, 7 were too busy, 7 did not complete the interview and we were unable to recontact, and 13 other. Compared to the completed interview sample, the veterans who did not complete the interview were more likely to be male (141/173; 82%), urban (110/173; 64%), and Caucasian (136/173; 79%).

Table 1.

Demographics and Clinical Description of Veteran Participants

Variables	(N = 92)
Age, mean (SD)	50.8 (12.4)
Gender, male, N (%)	72 (78)
Race, Caucasian, N (%)	60 (65)
Education level, at least some technical school or college, N (%)	72 (78)
Rural residence, N (%)	46 (50)
PHQ-9, mean (SD)	14.2 (5.6)
PC-PTSD (4-item), mean (SD)	3.0 (1.3)
GAD-7, mean (SD)	11.7 (6.3)
AUDIT-C, mean (SD)	3.5 (3.5)
Hoge Barrier total score, mean (SD)	31.1 (9.5)
CSQ, mean (SD)	24.2 (5.4)
PAI Part 1 total score, mean (SD)	9.5 (6.2)
PAI Part 2 total score, mean (SD)	30.1 (26.8)

Note. PHQ-9, 9-item Patient Health Questionnaire depression module; PC-PTSD, 4-item Primary Care PTSD screen; GAD-7, Generalized Anxiety Disorder 7-item; AUDIT-C, 3-item Alcohol Use Disorders Identification Test-Concise; CSQ, 8-item Client Satisfaction Questionnaire; PAI, Perceived Access Inventory

Measures.

The PAI items were developed from qualitative interviews with rural and urban veterans across the three geographic regions listed above (Pyne et al., 2020). Each item has two parts. Part 1 is a yes=1/no=0 question which assesses the presence of a potential barrier, e.g., “did you have to travel a long distance to get VA mental healthcare?” Yes responses provide a prevalence measure for each item and contribute to total domain and overall Part 1 total scores. Part 2 questions are only administered to those who respond “Yes” to Part 1. Part 2 assesses the extent to which a barrier interfered with getting needed VA mental healthcare (not at all=1 to completely=5). Eight of the nine Digital domain items do not use the same response format as other PAI items and were not included in the total PAI Part 1 or Part 2 scores. The omitted items address use of specific digital resources in obtaining VA healthcare (i.e., My HealtheVet, secure messaging, smartphone apps, clinic-based video telehealth, home-based video telehealth, internet chat rooms, telephone, and internet for healthcare information) using a yes/no format only. Item #33, which is in the Experiences of Care domain and phrased as a facilitator (see supplemental file), was not included in this analysis because of the difficulty in interpreting the correlation coefficient for it. Validation analyses only used data from the 34 PAI items that had a common Part 1 and Part 2 format, i.e., all of the Logistics, Culture, and Systems of Care items; 10/11 Experiences of Care items; and 1/9 Digital items. Therefore, the possible range for the Part 1 total score was 0 to 34, with higher scores indicating more perceived problems with access. For Part 2, respondents who answered no to a specific item in Part 1 scored zero on that item for Part 2; those who answered yes to an item in Part 1 would have a Part 2 score from 1 to 5 on that item according to the level of interference with access they reported for it. The possible range for the Part 2 total score was 0 to 170, with higher scores indicating greater perceived interference in obtaining needed VA mental healthcare.

The comparison access measure was the 13-item Hoge Perceived Barriers to Seeking Mental Health Services (possible score range 13 to 65; higher scores indicate more severe barriers) (Hoge et al., 2004). The comparison measure for patient satisfaction with VA mental healthcare was the 8-item Client Satisfaction Questionnaire (possible score range 8 to 32; higher scores indicate greater satisfaction) (Larsen, Attkisson, Hargreaves, & Nguyen, 1979).

We also assessed demographic and clinical variables for descriptive analysis and possible casemix adjustment. These variables included symptom severity (higher scores indicate greater severity) using the 4-item Primary Care PTSD screen (possible range 0 to 4) (Prins et al., 2003), the 9-item Patient Health Questionnaire depression screen (PHQ-9, possible range 0 to 27) (Spitzer, Kroenke, & Williams, 1999), the 3-item Alcohol Use Disorders Identification Test-Concise (AUDIT-C, possible range 0 to 12) (Bradley et al., 2007), and the 7-item Generalized Anxiety Disorder screener (GAD-7, possible range 0 to 21) (Spitzer, Kroenke, Williams, & Lowe, 2006). Rural/urban status was determined using a ZIP code approximation of the census tract-based rural-urban commuting area (RUCA) codes (WWAMI Rural Health Research Center, n.d.). Demographic data were collected via self-report.

Analysis.

We used Pearson bivariate correlation analysis to examine the relationship of the PAI scores to the Hoge score, CSQ total score, demographics, and clinical variables. We examined PAI Part 1 and Part 2 total and domain scores in correlation analyses to explore whether the correlation with prevalence scores from Part 1 items differed from the correlation with interference scores from Part 2 items. Analyses were performed using PSPP statistical software (Free Software Foundation, Inc.) (GNU Project, 2015).

Results

A descriptive summary of sample characteristics appears in Table 1. Participants’ mean age was 50.8 years. The sample was 78% male, 65% Caucasian, and 50% rural residents. Eighty percent of participants screened positive for current depression (≥10 on PHQ-9), 71% screened positive for PTSD (≥3 on PC-PTSD), 62% screened positive for GAD (≥10 on GAD-7), and 46% screened positive for alcohol use disorder (AUD; 51% of males and 25% of females on AUDIT-C, using cutoff score of ≥5 for men and ≥4 for women) (Khadjesari et al., 2017). The mean PAI Part 1 total score was 9.5 (median = 9.0; range of 0–29). The mean PAI Part 2 total score was 30.1 (median = 21.0; range of 0–118).

Both PAI Part 1 and Part 2 total scores were correlated significantly with the Hoge total score (r=0.56 and r=0.57, respectively; both p<0.001) (Table 2). Each of the PAI Part 1 and Part 2 domain scores also significantly correlated with the Hoge total score, as did most of the individual PAI Part 1 and Part 2 item scores (Part 1: 23/34, 67.6% and Part 2: 27/34, 79.4%) (PAI item-level results are available from first author upon request).

Table 2.

PAI Correlations with Hoge Total Scale Score

	Hoge correlations with
	Part 1 score	Part 2 score
PAI Total	0.56***	0.57***
PAI Domain
Logistics	0.35**	0.44***
Systems of Care	0.53***	0.53***
Culture	0.47***	0.58***
Experiences of Care	0.39***	0.43***

Note.

^*p<0.05

^**p<0.01

^***p<0.001

PAI: Perceived Access Inventory

PAI total and domain scores and Hoge total scores were significantly and inversely correlated with the total CSQ score (Table 3). Among PAI domain scores, Part 1 and Part 2 Systems of Care (r=−0.69 and r=−0.73, respectively) and Experiences of Care (r=−0.68 and r=−0.68, respectively) domains had the strongest correlations with CSQ and the Logistics domain had the weakest correlations (r=−0.32 and r=−0.43, respectively). Among PAI item scores, 25/34 (73.5%) Part 1 and 30/34 (88.2%) Part 2 PAI item scores were significantly correlated with the CSQ total score. All the items that significantly correlated between Part 1 and CSQ also significantly correlated between Part 2 and CSQ. However, five items with Part 1 scores that were not significantly correlated with CSQ had Part 2 scores that did significantly correlate with CSQ. The number of PAI items per domain that were significantly correlated with CSQ for Part 1 and Part 2, respectively, were: Logistics 3/6 versus 5/6, Systems of Care 11/12 versus 11/12, Culture 3/4 versus 4/4, Experiences of Care 8/10 versus 9/10, and Digital 0/1 versus 1/1.

Table 3.

PAI and Hoge Correlations with CSQ

Variables	CSQ total score
PAI Part 1 total	−0.72***
Logistic	−0.32**
Systems of Care	−0.69***
Culture	−0.44***
Experiences of Care	−0.68***
PAI Part 2 total	−0.75***
Logistic	−0.43***
Systems of Care	−0.73***
Culture	−0.54***
Experiences of Care	−0.68***
Hoge total	−0.51***

Note.

^*p<0.05

^**p<0.01

^***p<0.001

PAI, Perceived Access Inventory, CSQ, 8-item Client Satisfaction Questionnaire

The specific Part 1 and Part 2 Systems of Care items with the strongest correlations with the CSQ were: lack of trust in any of your VA mental healthcare providers, lack of trust in the VA healthcare system, and VA mental healthcare providers were not available to you as soon as you needed them. The Experiences of Care items with the strongest correlations with the CSQ were: VA mental healthcare providers did not genuinely care about you, VA mental healthcare staff did not genuinely care about you, did you ever feel stuck in VA “red tape” or paperwork, and have you felt comfortable that you were aware of all the VA mental health services that were available to you. The PAI domain with the weakest correlation with CSQ was Logistics which was defined as geographical, temporal, and financial issues that are perceived by veterans to affect their use of VA mental health services.

The PAI and Hoge scores had similar patterns of correlation with demographic and clinical variables (Table 4). Younger age was associated with the reporting of more barriers to mental healthcare for both the Hoge and PAI measures. Participants with higher depression, PTSD, and anxiety scores also reported more barriers to mental healthcare on both the Hoge and PAI measures. Other variables shown in Table 4 (gender, education, rural residence, AUDIT-C) had non-significant correlations with both Hoge and PAI scores.

Table 4.

Demographic and Clinical Variables Correlation with Satisfaction and Barrier Scales

	Total Score
Variables	CSQ	Hoge	PAI Part 1	PAI Part 2
Age	0.22*	−0.34**	−0.29**	−0.29**
Gender, Male	−0.10	0.03	0.06	0.01
Race, Caucasian	0.12	−0.20*	−0.19	−0.17
Education level1	0.03	−0.09	−0.14	−0.17
Rural residence	0.04	−0.08	−0.08	−0.04
PHQ-9	−0.32**	0.42***	0.43***	0.43***
PC-PTSD	−0.32**	0.48***	0.43***	0.42***
GAD	−0.32**	0.40***	0.36***	0.38***
AUDIT-C	−0.18	0.19	0.17	0.14

Note.

¹Education level: less than some technical school or college vs. at least some technical school or college

^*p<0.05

^**p<0.01

^***p<0.001

PHQ-9, 9-item Patient Health Questionnaire depression module; PC-PTSD, 4-item Primary Care PTSD screen; GAD-7, Generalized Anxiety Disorder 7-item; AUDIT-C, 3-item Alcohol Use Disorders Identification Test-Concise; CSQ, 8-item Client Satisfaction Questionnaire; PAI, Perceived Access Inventory

Discussion

Our preliminary results support the concurrent validity of the PAI when compared with a measure of barriers to receiving mental healthcare frequently administered to military members and veterans (Hoge measure) as well as its convergent validity with scores on a measure of patient satisfaction (CSQ) commonly used in the VA. All of the PAI total and domain scores and most of the item scores were significantly correlated with the Hoge total score and with the CSQ score. Notably, as shown in Table 3, both the PAI Part 1 and PAI Part 2 total scores were more strongly correlated with satisfaction than was the Hoge score.

The PAI domains with the strongest bivariate correlation with CSQ were the Systems and Experiences of Care domains. These results suggest that gains in treatment satisfaction could be greatest by addressing Systems of Care and Experiences of Care barriers. The specific items in these domains with the strongest bivariate correlation included: trust, genuine caring, provider availability, red tape, and awareness of services. Each of these PAI items have varying degrees of trust embedded within them. Trust is learned and learning is dependent on repetition and interaction; both of these elements are threatened by current healthcare trends such as digitization (e.g., electronic health records, data driven policies, daily health monitoring, artificial intelligence) and the increasing number of clinicians caring for a given patient (Khullar, 2019). Trust is a critical metric for VA and non-VA hospitals (Khullar, 2019; VA, Office of Public and Intergovernmental Affairs, 2018). The VA’s FY18–24 Strategic Plan (VA, Office of Enterprise Integration, 2019) and the 2018 American Board of Internal Medicine Foundation Forum on [Re]Building Trust (Lee, McGlynn, & Safran, 2019) both reflect the importance of patient experience and building patient trust.

Demographic and clinical bivariate correlations are consistent with those in the literature. For example, younger veterans reported more barriers to getting needed mental health treatment across all PAI domains, as was the case in other veteran studies (Garcia et al., 2014; National Academies of Sciences, Engineering, and Medicine, 2018). In the Garcia, et al. study, for example, OEF/OIF (Operation Enduring Freedom/Operation Iraqi Freedom) veterans were more likely to agree (p<0.05) with half (8/16) of the barriers presented to them compared to veterans from prior service eras. Similarly, increased symptom severity (particularly PTSD) was associated with increased report of barriers in the current and other studies (Ouimette et al., 2011). Neither the PAI nor the Hoge measure was significantly correlated with gender, education or rural residence. Similar results using the Hoge measure were reported elsewhere for gender and education (Pietrzak, Johnson, Goldstein, Malley, & Southwick, 2009). The literature on rural residence is less straightforward because there is evidence that rural veterans received less mental health treatment than urban veterans but rural versus urban status did not predict perceived need for mental health services (Teich, Ali, Lynch, & Mutter, 2017). As suggested by the SOTA Access model, the stoicism and self-reliance associated with rurality may contribute to the lack of difference in perceived need and perceived access for rural versus urban veterans (Fischer et al., 2016).

Recent studies by RAND (Acosta et al., 2018) and the National Academies of Sciences, Engineering, and Medicine (National Academies of Sciences, Engineering, and Medicine, 2018) evaluated access to mental health services during a timeframe similar to that of the PAI study. The RAND study focused on current military service members while the National Academy study focused on recent veterans of the wars in Iraq and Afghanistan. The RAND study used qualitative and quantitative methods to create two item banks: one with 54 barriers and 6 facilitators and a short form with 15 barrier items. Many of the RAND long and short form items could be described as aspects of personal or public stigma. Similar to PAI item scores, the RAND items used a five-point Likert response set and the RAND barrier scores were strongly correlated with the Hoge scale total score (r=0.78) and mental health symptom severity. The RAND and PAI studies differed in that the PAI study focused on the perspective of veterans and on a broader range of items that were largely actionable by VA (e.g., the PAI includes personal stigma items but does not include items addressing social attitudes toward mental health treatment as the RAND items do).

The National Academy study also used mixed methods, focused on veteran access to mental health treatment, and concluded with eight broad recommendations and multiple specific actions for improving access to mental healthcare. The first specific action recommended was to solicit broad input from patients using VA mental health services about satisfaction and about barriers to accessing timely patient-centered mental healthcare; that was the focus of the current and previous PAI studies. The National Academy study and PAI studies differed in that the PAI enrolled veterans from all service eras and produced a list of perceived access items actionable by VA.

The PAI has several potential uses. PAI items or domains could be embedded in existing VA nationwide surveys (e.g., Survey of Healthcare Experiences of Patients (SHEP), Survey of Enrollees, and surveys from the Veteran Experience Office) as well as in more local surveys to understand national and local trends in perceived access over time. VA could administer these surveys prior to and following implementation of interventions to improve access so that the intervention’s impact on perceived access could be measured. The PAI could also be used as part of an intervention to improve access at the patient level by first identifying and then addressing actionable barriers for individual patients.

In future studies, we will evaluate the utility of the PAI as a predictor of subsequent service use. We will also assess the importance of collecting PAI Part 2 data. Similarities and differences in the results obtained using Part 1 versus Part 2 results need to be considered in this regard. For example, the correlations of PAI total, domain, and item scores with the Hoge scale and the CSQ were similar regardless of whether Part 1 or Part 2 data were used. More Part 2 items were significantly correlated with the CSQ than Part 1 items, but this needs to be weighed against the added respondent burden associated with obtaining Part 2 data. From an administrator’s perspective, the more nuanced Part 2 results on level of perceived interference might offer more realistic targets for improving satisfaction because they provide an opportunity to see gradual improvements in scores. In contrast, the yes/no response set for Part 1 results suggests that a barrier would need to be completely eliminated to improve satisfaction. Longitudinal data will also help clarify the relative value of Part 1 versus Part 2 results.

Limitations associated with the reported analyses include the relatively small sample size. This is also the first analysis of PAI concurrent and convergent validity; future studies will add to the validation evidence base. The version of the PAI administered in this study was specific to use of VA mental health services. It may not be appropriate in other clinical contexts (e.g., community care for veterans, use of physical healthcare services by veterans, civilian mental or physical health service use). A version of the PAI was developed to assess perceived access to community mental healthcare for veterans (Pyne et al., 2019). The community care version identified barriers that were unique to accessing community care, but it requires further validation within a larger sample of veterans utilizing community mental healthcare.

Conclusions

PAI concurrent and convergent validity are supported by strong correlations with the Hoge mental health barriers scale and the CSQ, respectively. This version of the PAI elicits users’ perspectives on access to VA mental health services. Those perspectives can and should inform development of innovative access interventions as well as evaluation of current and future interventions for veterans who use VA and community mental health services.

Impact Statements:

The Perceived Access Inventory was developed from the perspective of veteran patients who had experience accessing Veterans Affairs mental healthcare treatment. This study demonstrated strong correlations between the PAI and another access measure that was developed for use in active duty samples. The PAI is also strongly correlated with patient satisfaction suggesting that the PAI items may be used to identify ways to improve satisfaction with mental health services.