Abstract
The operational definition of first-contact accessibility is “the ease with which a person can obtain needed care (including advice and support) from the practitioner of choice within a time frame appropriate to the urgency of the problem”; accommodation is “the way healthcare resources are organized to accommodate a wide range of patients' abilities to contact healthcare providers and reach healthcare services, that is to say telephone services, flexible appointment systems, hours of operation, and walk-in periods.”
Objective:
To compare how well accessibility is measured in validated subscales that evaluate primary healthcare from the patient's perspective.
Method:
645 adults with at least one healthcare contact in the previous 12 months responded to six instruments that evaluate primary healthcare with four subscales that measure accessibility: the Primary Care Assessment Survey (PCAS), the Primary Care Assessment Tool – Short Form (PCAT-S, two subscales) and the first version of the EUROPEP (EUROPEP-I). Scores were normalized to a 0-to-10 scale for descriptive comparison. Exploratory and confirmatory (structural equation modelling) factor analysis examined fit to operational definition, and item response theory analysis examined item performance on common constructs.
Results:
The subscales demonstrate similar psychometric measures to those reported by developers. The PCAT-S First-Contact Utilization subscale does not fit the accessibility construct. The remaining three subscales load reasonably onto a single factor, presumed to be accessibility, but the best-fitting model has two factors: “timeliness of obtaining needed care” (PCAT-S First-Contact Access, some EUROPEP-I items) and “how resources are organized to accommodate clients” (PCAS Organizational Access and most of EUROPEP-I organization of care). Items in the PCAS and PCAT-S subscales have good discriminability.
Conclusion:
Only three of the four subscales measure accessibility; all are appropriate for use in Canada. The PCAT-S First-Contact Access subscale is the best measure for first-contact accessibility, and PCAS Organizational Accessibility has good metric properties and measures for accommodation.
Abstract
La définition opérationnelle de l'accessibilité de premier contact est « la facilité avec laquelle une personne peut obtenir les services requis (y compris des conseils et du soutien) du clinicien de son choix dans un délai approprié à l'urgence du problème »; l'accommodation est « la façon dont les ressources en santé sont organisées afin de permettre à une large gamme de patients d'entrer en contact avec les fournisseurs de soins avec les services de santé, c'est-à-dire accès téléphonique, flexibilité pour la prise de rendez-vous, heures d'ouverture et périodes allouées aux visites sans rendez-vous. »
Objectif :
Voir à quel point l'accessibilité est mesurée par les sous-échelles validées servant à évaluer les soins de santé primaires du point de vue du patient.
Méthode :
645 adultes ayant eu au moins un contact avec les services de santé au cours des 12 mois antérieurs ont répondu à six instruments servant à évaluer les soins de santé primaires, incluant quatre sous-échelles qui mesurent l'accessibilité : Primary Care Assessment Survey (PCAS), Primary Care Assessment Tool – version courte (PCAT-S, deux sous-echelles) et la premiere version de l'EUROPEP (EUROPEP-I). Les résultats ont été normalisés selon une échelle de 0 à 10 pour permettre des comparaisons descriptives. Les analyses factorielles exploratoires et confirmatoires (modélisation par équation structurelle) ont permis d'examiner l'adéquation à la définition opérationnelle, et l'analyse de réponse par item a permis d'en examiner la performance en fonction des construits communs.
Résultats :
Les mesures psychométriques des sous-échelles sont similaires à celles indiquées par les concepteurs. La sous-échelle « utilisation de premier contact » du PCAT-S ne concorde pas avec le construit de l'accessibilité. Les trois autres sous-échelles correspondent raisonnablement à un facteur unique, qui serait probablement l'accessibilité, mais le modèle qui s'ajuste le mieux comprend deux facteurs : « rapidité d'obtention des soins nécessaires » (« accessibilité de premier contact » du PCAT-S, certains items de l'EUROPEP-I) et « comment sont organisées les ressources pour accommoder les patients » (« accès organisationnel » du PCAS et la plupart des items « organisation de soins » de l'EUROPEP-I). Les items des sous-échelles du PCAS et du PCAT-S présentent une bonne discriminabilité.
Conclusion :
Seules trois des quatre sous-échelles mesurent l'accessibilité; toutes sont appropriées pour leur usage au Canada. La sous-échelle « accès de premier contact » du PCAT-S est la meilleure mesure pour l'accessibilité de premier contact et la sous-échelle « accessibilité organisationnelle » du PCAS possède de bonnes mesures et propriétés métriques pour l'accommodation.
Acessibility receives a lot of attention in research and policy debates. However, its definition and assessment pose significant challenges.
Background
Conceptualizing accessibility of healthcare services
Accessibility is a complex notion, as evidenced by the heterogeneity of definitions and conceptualizations in the literature and the almost interchangeable use of the terms “access,” “accessibility” and “utilization of healthcare services.” The Canadian Oxford Dictionary (1998) defines accessibility as the “condition of being readily approached.” In this sense, accessibility is a characteristic of something that can readily be reached, entered or used.
Donabedian (1973) describes accessibility as characteristic of health systems that impede or promote service utilization. Thus, health services are accessible if their specific characteristics – geographic availability, organization, price, acceptability and so on – allow a broad range of persons to reach, enter and use them (Bashshur et al. 1971; Donabedian 1973; Penchansky and Thomas 1981). From this perspective, evaluation of accessibility is amenable to both objective and subjective assessment of the geographic and temporal availability of services, their organizational availability, their costs and their social and cultural acceptability (Lévesque 2006).
Evaluating the accessibility of primary healthcare services
In a consensus consultation of primary healthcare (PHC) experts across Canada to formulate operational definitions of PHC attributes to be evaluated (Haggerty et al. 2007), two distinct definitions of accessibility emerged. The first, labelled first-contact accessibility, is “The ease with which a person can obtain needed care (including advice and support) from the practitioner of choice within a time frame appropriate to the urgency of the problem.” This is specific to PHC and is one of its essential functions. The second, accommodation, is applicable to all levels of healthcare: “The way healthcare resources are organized to accommodate a wide range of patients' abilities to contact healthcare providers and reach healthcare services (telephone services, flexible appointment systems, hours of operation and walk-in periods)” (Haggerty et al. 2007).
Various instruments have been developed to evaluate PHC accessibility from the user's perspective, but there is little comparative information about these to guide evaluators in their selection of tools. Our objective was to provide insight into how well validated subscales from different instruments measure accessibility. Specifically, we wanted to know how subscale scores from different instruments compare and whether the accessibility subscales measure a single construct, presumed to be accessibility. If analyses suggested more than one factor, we wanted to judge how those factors reflected our operational definitions of accessibility. Finally, we sought to examine how well individual items measured the dimensions that were common across instruments.
Method
The method of this series of studies has been described in detail elsewhere (Haggerty, Burge et al. 2011). Briefly, six validated instruments that evaluate PHC from the patient's perspective were administered to 645 healthcare users balanced approximately by English/French language, rural/urban location, high/low education, and poor, average or excellent overall PHC experience.
The analytic strategy, described in detail elsewhere (Santor et al. 2011) consisted of examining the distributional statistics and subscale correlations, followed by common factor analysis and confirmatory factor analysis (structural equation modelling) to identify dimensions common to the entire set of items. Finally, we examined the performance of individual items and response scales using parametric item response theory analysis against the original subscale and non-parametric analysis against the common factors that emerged across instruments.
Measure selection
Three of the instruments had accessibility subscales. The Primary Care Assessment Survey (PCAS) (Safran et al. 1998) six-item Organizational Accessibility subscale elicits ratings for obtaining services at the clinic using a six-point Likert response scale (1=very poor to 6=excellent). A parallel set of questions report average wait times, but these are not included in the validated subscale. The Primary Care Assessment Tool – Short Form (PCAT-S) (Shi et al. 2001) has two subscales: the three-item First-Contact Utilization (eliciting usual patterns of care-seeking), and the four-item First-Contact Access (eliciting the probability of being seen at the clinic under various scenarios with a four-point Likert scale from “1=definitely not” to “4=definitely”). However, during the attribute-mapping process, the PCAT-S First-Contact Utilization subscale was flagged as fitting poorly with the concept of accessibility, despite its label. The first version of the European general practice instrument, EUROPEP-I (Grol et al. 2000) has a seven-item Organization of Care subscale in which most items elicit ratings of accessibility using a five-point semantic differential response scale with “1=poor” and “5=excellent” as anchors of each extreme.
Results
Comparative descriptive statistics
The content of the four subscales and item distributors are summarized in Table 1; the detailed content and distributions are available online at http://www.longwoods.com/content/22635. Only two subscales have ≥5% true missing values (PCAS on being able to talk by phone to the doctor, and EUROPEP-I on the helpfulness of staff). However, both PCAT-S subscales and the EUROPEP-I offer respondents a “don't know/not sure” or “not applicable” option, which is used by a significant proportion of respondents in PCAT-S First-Contact Access items and four EUROPEP-I items. Approximately 17% were “not sure” about arrangements for getting care or advice outside office hours on the PCAT-S, or being able to contact the doctor by phone (13%) on the EUROPEP-I.
TABLE 1.
Summary of accessibility subscale content and distribution of item responses. (Detailed distribution available at http://www.longwoods.com/content/22635).
| Subscale and Item Description | Response Scale | Range Missing Values | Overall Modal Response | Range Item Discriminability | Comments on Distribution |
|---|---|---|---|---|---|
|
PCAS Organizational Access (6 items) Rate doctor's office for: Location, hours, usual wait for an appointment; usual wait at the clinic, ability to get through to the doctor's office or to speak to the doctor by phone |
Likert evaluative, 1=very poor to 6=excellent | 1%–6% | 4 (good) | 0.83 (office location) to 2.4 (speak to doctor by phone) | Approximately 60% of responses in two most positive categories; lowest rating for “speak to doctor by phone” |
|
PCAT–S First-Contact Utilization (3 items) Likelihood of seeking care for routine or a new problem and the need for referral to see a specialist |
Likert evaluative, 1=definitely not to 4=definitely | 1%–2% (true missing) 0%–4% not sure | 4 (definitely) | 0.87 (referral) to 4.7 (first place to seek routine care) | Over 65% of responses in the most positive category |
|
PCAT–S First-Contact Access (4 items) When clinic open: Likelihood of being seen same day; getting advice over the phone When clinic closed: Having a phone number to call; likelihood of being seen by doctor during the night |
Likert evaluative, 1=definitely not to 4=definitely | 2% (true missing) 6%–18% not sure | variable, according to the item | 1.06 (open) to 2.99 (phone number when closed) | Item score in decreasing order; 16%–18% “not sure” when clinic closed; 40% can definitely not be seen at night |
|
EUROPEP-I Organization of Care (7 items) Rate care by GP: Preparation for what to expect from specialty care; helpfulness of staff; getting a suitable appointment; getting through to the practice over the phone; speaking to GP on the phone; waiting time in waiting room; getting quick services for urgent problems |
Semantic differential rating, 1=poor, 5=excellent | 4%–5% (true missing) 1%–13% (n/a) | 5 (excellent) | 1.55 (preparation for other care) to 2.86 (quick services) | Lowest rating for “speak to GP on the phone.” Three items with “not applicable”: preparation for specialty care (12%), speak to GP on the phone (13%) and quick services (11%) |
TABLE 1.
Table 1. Distribution of responses to items in subscales measuring accessibility of primary healthcare services and discriminatory capacity of each item within its parent subscale. Modal response is shown in bold (n=645).
| Variable Name | Instrument: Subscale | Missing Values % (n) | Per Cent (Number) by Response Option | Item Discrimination | |||||
|---|---|---|---|---|---|---|---|---|---|
| Primary Care Assessment Survey (PCAS): Organizational Access | 1=Very poor | 2=Poor | 3=Fair | 4=Good | 5=Very good | 6=Excellent | |||
| PS_oa1 | How would you rate the convenience of your regular doctor's office location? | 1 (8) | 0 (1) | 1 (9) | 14 (93) | 27 (172) | 31 (202) | 25 (160) | 0.83 (0.17) |
| PS_oa2 | How would you rate the hours that your doctor's office is open for medical appointments? | 1 (5) | 1 (6) | 3 (17) | 17 (111) | 38 (245) | 28 (181) | 12 (80) | 1.84 (0.13) |
| PS_oa3 | How would you rate the usual wait for an appointment when you are sick and call the doctor's office asking to be seen? | 3 (22) | 5 (35) | 15 (95) | 21 (134) | 26 (168) | 19 (121) | 11 (70) | 2.51 (0.19) |
| PS_oa4 | How would you rate the amount of time you wait at your doctor's office for your appointment to start? | 2 (10) | 5 (34) | 12 (80) | 27 (177) | 29(190) | 16 (106) | 7 (48) | 1.81 (0.13) |
| PS_oa5 | Thinking about the times you have needed to see or talk to your doctor, how would you rate the following: ability to get through to the doctor's office by phone? | 1 (6) | 4 (27) | 5 (35) | 17 (110) | 31 (200) | 27 (172) | 15 (95) | 2.10 (0.15) |
| PS_oa6 | Thinking about the times you have needed to see or talk to your doctor, how would you rate the following: ability to speak to your doctor by phone when you have a question or need medical advice? | 6 (38) | 12 (79) | 16 (102) | 20 (132) | 25 (161) | 14 (91) | 7 (42) | 2.40 (0.16) |
| Primary Care Assessment Tool (PCAT-S): First-Contact Utilization | 1=Definitely not | 2=Probably not | 3= Probably | 4= Definitely | Not sure / Don't remember | ||||
| PT_fcu1 | When you need a regular general check-up, do you go to your Primary Care Provider before going somewhere else? | 1 (7) | 2 (12) | 1 (6) | 10 (64) | 86 (554) | 0 (2) | 4.70 (0.60) | |
| PT_fcu2 | When you have a new health problem, do you go to your Primary Care Provider before going somewhere else? | 1 (8) | 2 (13) | 2 (15) | 12 (79) | 82 (528) | 0 (2) | 4.59 (0.54) | |
| PT_fcu3 | When you have to see a specialist, does your Primary Care Provider have to approve or give you a referral? | 2 (10) | 2 (16) | 4 (27) | 23 (151) | 65 (418) | 4 (23) | 0.87 (0.13) | |
| Primary Care Assessment Tool (PCAT-S): First-Contact Access | 1=Definitely not | 2=Probably not | 3= Probably | 4= Definitely | Not sure / Don't remember | ||||
| PT_fca1 | When your Primary Care Provider is open and you get sick, would someone from there see you the same day? | 2 (11) | 6 (41) | 16 (103) | 43 (278) | 27 (175) | 6 (37) | 1.06 (0.12) | |
| PT_fca2 | When your Primary Care Provider is open, can you get advice quickly over the phone if you need it? | 2 (11) | 11 (72) | 17 (110) | 35 (225) | 25 (161) | 10 (66) | 1.06 (0.12) | |
| PT_fca3 | When your Primary Care Provider is closed, is there a phone number you can call when you get sick? | 2 (13) | 20 (129) | 10 (63) | 14 (90) | 39 (250) | 16 (100) | 2.99 (0.27) | |
| PT_fca4 | When your Primary Care Provider is closed and you get sick during the night, would someone from there see you that night? | 2 (12) | 40 (260) | 22 (140) | 9 (59) | 9 (58) | 18 (116) | 2.58 (0.22) | |
| EUROPEP-I: Organization of Care How would you rate the following care provided by your general practitioner in the last 12 months? | 1=Poor | 2 | 3 | 4 | 5=Excellent | NA (does not apply) | |||
| EU_oa1 | Preparing you for what to expect from specialist or hospital care | 4 (24) | 3 (20) | 7 (42) | 13 (87) | 29 (185) | 32 (209) | 12 (78) | 1.55 (0.14) |
| EU_oa2 | The helpfulness of staff (other than the doctor) | 5 (29) | 3 (18) | 6 (39) | 15 (98) | 30 (192) | 37 (236) | 5 (33) | 1.84 (0.15) |
| EU_oa3 | Getting an appointment to suit you | 4 (24) | 9 (57) | 10 (67) | 18 (113) | 25 (158) | 34 (220) | 1 (6) | 2.83 (0.18) |
| EU_oa4 | Getting through to the practice on the phone | 4 (23) | 5 (35) | 8 (53) | 19 (125) | 28 (180) | 34 (217) | 2 (12) | 2.16 (0.15) |
| EU_oa5 | Being able to speak to the general practitioner on the telephone | 4 (27) | 19(124) | 15 (99) | 17 (107) | 18(115) | 14 (89) | 13 (84) | 2.48 (0.19) |
| EU_oa6 | Waiting time in the waiting room | 4 (25) | 14 (91) | 12 (75) | 24 (156) | 30 (192) | 15 (99) | 1 (7) | 1.79 (0.13) |
| EU_oa7 | Providing quick services for urgent health problems | 4 (25) | 8 (49) | 8 (54) | 17 (108) | 25 (158) | 28(179) | 11 (72) | 2.86 (0.21) |
Only the PCAT-S First-Contact Utilization showed strong ceiling effects, with the majority endorsing the most positive response. The parametric estimate of discriminability within the original subscale generally showed good capacity to discriminate between different levels of the subscale score. Exceptions were the convenience of location on PCAS Organization Accessibility and need for approved referrals to specialists on PCAT-S First-Contact Utilization.
Table 2 presents the subscale scores as item means normalized to a common 0-to-10 metric to permit comparison. The normalized means and standard deviations differ substantially from one scale to another. All except the PCAT-S First-Contact Access subscale are quite skewed towards positive values, with the median higher than the mean. The PCAT-S subscales had relatively low reliability.
TABLE 2.
Mean and distributional scores for accessibility subscale scores normalized to 0 to 10 (n=645)*
| Developer's Scale Name (# of items in scale) | Cronbach's Alpha | Mean | SD | Minimum Observed | Quartiles | ||
|---|---|---|---|---|---|---|---|
| Q1 (25%) | Q2 (50%) | Q3 (75%) | |||||
| Normalized scores | |||||||
| PCAS Organizational Access (6) | .83 | 5.9 | 1.8 | 1.3 | 4.7 | 6.0 | 7.3 |
| PCAT-S First-Contact Utilization (3) | .59 | 8.5 | 1.5 | 0.0 | 7.5 | 9.2 | 9.2 |
| PCAT-S First-Contact Access (4) | .68 | 5.2 | 3.0 | 0.0 | 3.3 | 5.6 | 6.7 |
| EUROPEP-I Organization of Care (7) | .89 | 6.5 | 2.4 | 0.0 | 5.0 | 6.8 | 8.6 |
Subscale scores calculated as mean of item values and calculated only for observations where >50% of items were complete.
Table 3 presents the Pearson correlations between the accessibility subscales. With the exception of the PCAT-S First-Contact Utilization subscale, the accessibility subscales correlate strongly with one another (.45 to .68), suggesting a common construct. In contrast, the PCAT-S First-Contact Utilization subscale correlates only modestly (.24 to .29) and correlates most highly with relational continuity subscales (.28 to .37). The EUROPEP-I Organization of Care subscale correlates as strongly with subscales of other dimensions as it does with accessibility, suggesting that it is measuring an overall experience of care rather than accessibility specifically. In contrast, the PCAS Organizational Access subscale and the PCAT-S First-Contact Access subscale have much lower correlations with other attributes of care, suggesting they are more specific for accessibility.
TABLE 3.
Mean partial correlations between accessibility.* Only correlations significantly different from zero are provided.
| Questionnaire Subscale | Organizational Access PCAS | First-Contact Utilization PCAT-S | First-Contact Access PCAT-S | EUROPEP-I |
|---|---|---|---|---|
| PCAS: Organizational Access | 1.00 | 0.29 | 0.45 | 0.68 |
| PCAT-S: First-Contact Utilization | 0.29 | 1.00 | 0.24 | 0.29 |
| PCAT-S: First-Contact Access | 0.45 | 0.24 | 1.00 | 0.46 |
| EUROPEP-I: Organization of Care | 0.68 | 0.29 | 0.46 | 1.00 |
Controlling for language, education, achievement, geographic location.
Do all items measure a single attribute?
We excluded from factor analysis all subjects who had at least one missing value on any item (listwise missing). This dramatically reduced our effective sample size from 645 to 340. Most exclusions (267/340) were for selecting the “don't know/not sure” or “not applicable” response options. Those excluded from the factor analyses were more likely than those included to be English-speaking, to report better health status and to have lower usual wait times for an appointment; they did not differ by their overall experience of care and other individual characteristics. Because this conservative approach can introduce bias, we repeated all the confirmatory factor analyses using maximum likelihood imputation of missing values (Jöreskog and Sörbom 1996) to examine the robustness of our conclusions. Running the models on a sample of 559 improved the fit statistics of confirmatory factor analysis models but did not alter any of the general conclusions.
We had little expectation that all the items would load on a single construct because we had defined two dimensions of accessibility and the PCAT-S First Utilization subscales seem to fit poorly. Nonetheless, constraining common factor analysis to one factor found loadings >.30 for all but two items. The same model with structural equation modelling generated statistics suggesting only moderate fit, with a root mean squared error of approximation (RMSEA) of p=.11, considerably higher than the .05 standard for good fit. Fit statistics improved dramatically when we removed the PCAT-S First-Contact Utilization fit, with the RMSEA=.09, a normed fit index (NFI) of .98 (well above the .90 standard) and considerably lower chi-squared and model conditional Akaike information criteria (CAIC) values (better fit with lower values). Figure 1 presents a second-order model in which items are grouped within their original subscale as one construct (first-order latent variable), which in turn emerges from a single underlying construct, presumed to be accessibility (second-order latent variable). Compared to the one-factor model, grouping items by parent subscale improves fit and shows that PCAT-S First Utilization loads much less strongly on the construct presumed to be accessibility.
FIGURE 1.
Parameter estimations for a structural equation second-order model where a single underlying construct (second-order latent variable) leads to the four subscales (first-order variables) with loadings on their respective items
Based on this and our a priori concerns, we decided that the PCAT-S First-Contact Utilization subscale, despite its label, does not fit the construct of accessibility. Subsequent analyses indicated a best fit with comprehensiveness of care (Haggerty, Beaulieu et al. 2011). It was removed from further analyses on accessibility.
How do underlying factors fit with operational definitions?
Exploratory factor analysis suggested a two-factor model. Using our operational definitions as a guide, we judged that the first factor (eigenvalue=7.59) captured “obtaining needed care … in a time frame appropriate to the urgency of the problem,” or timeliness, and the second (eigenvalue=1.19) touched on “how resources are organized to accommodate clients,” or accommodation.
Using the operational definitions and the exploratory analysis as a guide, we grouped items on timeliness and accommodation for confirmatory factor analysis, shown in Figure 2. Goodness of fit improved slightly with respect to the unidimensional model (RMSEA=.084, NFI=.98). The correlation between the dimensions of timeliness and accommodation is .95. We tried other item groupings based on our judgment, but all other configurations resulted in poorer fit statistics. The model shows that the PCAS Organizational Access subscale relates to the sub-dimension of accommodation, whereas the PCAT-S First-Contact Access subscale relates to timeliness. The EUROPEP-I Organization of Care subscale mostly measures timeliness, but one item, wait time in the waiting room, loads highly on accommodation. Figure 2 shows that some items do not have high loadings and have a high proportion of residual error (shown to the right of each item). These items relate poorly to the construct, either because they are not discriminatory or because they relate better to another construct that is not part of the latent variable.
FIGURE 2.
Parameter estimations for a structural equation showing item loadings on items from different subscales on two sub-dimensions of accessibility (first-order variables), timeliness and accommodation
A number of items were identified a priori as potentially poor indicators of accessibility. For instance, the EUROPEP-I Organization of Care subscale item, “How would you rate your general practitioner's care in preparing you for what to expect from specialist or hospital care?” does not appear to measure accessibility. However, this item correlates well with other items in the subscale (item–total correlation=.55), and removing it does not improve the model's goodness of fit. Since we had decided to respect as much as possible the developers' factor solutions, we retained this item as part of the subscale.
Individual item performance
Non-parametric item response theory graphs were modelled on dimensions of timeliness and accommodation and provide further insight into item performance. For accommodation, we found that all but one item (PCAS convenient location) in the PCAS Organizational Access subscale demonstrate excellent performance. The probability of endorsing each response option is highest in a relatively narrow and unique zone of accommodation, and is clearly ordinal, reflecting the assigned value for each option. The item score varies linearly and strongly with accommodation, demonstrating excellent discriminability. The single EUROPEP-I item that measures accommodation (EU_oa6, wait in waiting room), shows good discriminability, but the extreme response options (“1=poor,” “5=excellent”) are overwhelmingly endorsed, suggesting that the assigned values for middle options may not be appropriate.
For timeliness, the first two items in the PCAT-S perform relatively well (probability of being seen same day when provider is open, and getting advice over the phone). The value assigned to each response option is mostly appropriate, except the option “probably,” which appears to be non-specific, as it is endorsed across the entire range of timeliness. The item score correlates highly with overall timeliness, indicating good discriminability. In contrast, the two items addressing more rare scenarios (when provider is closed) have modest discriminability, and the response options and assigned values perform less well. Only the options “1=definitely not” and “4=definitely” are endorsed with any frequency, raising the question of the appropriateness of a four-point response scale. The probability of choosing the “don't know” response option (over 15% of respondents) mimics the probability of endorsing options “2=probably not” or “3=probably,” depending on the item.
All but one EUROPEP-I item performed poorly on timeliness. Only the item “getting quick services for urgent problems” demonstrated good discriminability despite problems with middle response options. The remaining items demonstrate poor discriminability, and the behaviour of the middle response options indicate that differential weights may not be warranted. Based on the content of some items (preparation to see specialist, helpfulness of staff, phone contact with the clinic, and phone contact with the general practitioner), we tested them on accommodation rather than timeliness, but item performance did not improve.
Discussion
Capacity to measure accessibility
Of the four subscales used in the concurrent validation study, three seem to evaluate clearly the attribute of accessibility, whereas PCAT-S First-Contact Utilization appears to be measuring comprehensiveness of care or possibly relational continuity (Burge, Haggerty et al. 2011). This finding implies that evaluators interested in evaluating accessibility could use any one of these three subscales. However, the PCAT-S First-Contact Access subscale measures timeliness, and the PCAS Organizational Access subscale principally measures accommodation. The EUROPEP-I Organization of Care subscale appears to capture both dimensions, but despite high internal consistency, item performance on the constructs of timeliness or accommodation are problematic. The high correlation between the EUROPEP-I Accessibility subscale and all the other scales in the evaluation study further suggests it measures a generic experience of care rather than accessibility specifically. This is not surprising, because the scale developers did not specifically intend to evaluate accessibility, but rather organization of care based on patient priorities (Grol et al. 1999). It has adequate discriminability for its own construct. Recent revisions to the instrument are minor, but may improve metric properties (Wensing 2006).
The PCAT-S First-Contact Access is the best subscale for measuring first-contact accessibility as the dimension of accessibility that is specific to PHC. The item's content well reflects policy initiatives to enhance 24/7 access to PHC, and the items are ordered by difficulty, suggesting that a summary score is informative and meaningful. However, despite good discriminability, its performance could be improved by changing the response options. First, the option “probably” is non-specific, shown both on item response analysis and discussions with respondents (Haggerty, Lawson et al. 2011). Most importantly, the “don't know” option should not be offered as it creates missing values on analysis. The item response theory analysis suggests that the developers' recommendation of imputing “not sure” as “probably not” is not appropriate. The two items referring to more rare access experiences – when the office is closed or at night – would function better with a yes/no response format but can be used to identify those with excellent access.
The PCAS Organizational Accessibility subscale demonstrates excellent performance for measuring accommodation. The good discriminability and reliability might improve further if the item on convenience of location were removed. Since accommodation is not specific to PHC, it is likely equally relevant to general healthcare accessibility. Two questionnaires that were largely derived from the PCAS (General Practice Assessment Questionnaire – GPAQ and General Practice Assessment Survey – GPAS) include a question on timely care: “If you need to see a doctor urgently, can you normally get seen on the same day?” and may be a good option for including an indicator of first-contact accessibility as well as accommodation.
Aspects not covered in the studied instruments
The subscales we studied did not address the element of “obtaining care from the provider of choice” in our operational definition, though it could be argued this is captured by the PCAT-S First-Contact Utilization subscale that we excluded. Although studies show that patients prioritize timeliness over affiliation when faced with an acute illness (Love and Mainous 1999), on the whole, patients strongly prefer to consult their own physician for most care, as is shown on measures of relational continuity elsewhere in this special issue (Burge et al. 2011). The provider of choice – and therefore the PCAT-S First-Contact Utilization subscale – may be the link between accessibility and relational continuity, or may be particularly relevant to accessibility for non-urgent problems.
None of the identified questionnaires assess geographic accessibility, which is often the first aspect of accessibility considered (Frenk 1992). The PCAS item on “the convenience of location” is problematic owing to its skewed distribution and poor discriminability. Although long distances from care may affect a minority of Canadians, it would be important to develop a sensitive measure of geographic accessibility, given that Canadian geography makes it a major consideration in healthcare planning. We developed a measure of ease of reaching care and local availability of services that may meet this need (Haggerty, Lévesque et al. 2009).
We did not assess the subscales that evaluate economic accessibility, as these were not considered relevant for Canada because they refer to direct costs of services. However, indirect costs, such as transportation costs and pay lost when receiving medical care during working hours, can result in forgone care. We have developed a measure of economic accessibility that addresses these issues and performs very well in the Quebec context (Haggerty, Lévesque et al. 2009).
Study limits
Limiting the study to those having visited a regular provider in the previous 12 months may have selected subjects with good accessibility. However, including users and non-users in a study sample should increase the variance and improve, not compromise, measure performance. Eliminating subjects with missing values not only reduced statistical power, but may have biased the final sample, even though our sensitivity analysis using imputation of missing values did not alter our overall conclusions. The item response analysis shows that missing values tend to occur among respondents with more negative experiences of accessibility. This finding would underestimate the reported reliability and attenuate the factor analysis results, but is not expected to radically change overall differences among instruments. Finally, this study did not have the benefit of an objective assessment of aspects of accessibility at the usual source of care that could have enabled us to assess the correlation of different scales with actual measures of availability and barriers to care.
Conclusion
Despite the study's limitations, the results indicate that the PCAS Organizational Access subscale is an excellent choice for measuring accommodation. The PCAT-S First-Contact Access subscale is the best for timely first-contact care and is, therefore, highly relevant for accessibility reforms in Canada, but it could be psychometrically improved. The EUROPEP-I Accessibility subscale probably assesses a more general experience of care, including accessibility dimensions.
Acknowledgements
This research was funded by the Canadian Institutes for Health Research. During this study Jeannie L. Haggerty held a Canada Research Chair in Population Impacts of Healthcare at the Université de Sherbrooke. The authors wish to thank Beverley Lawson for conducting the survey in Nova Scotia and Christine Beaulieu in Quebec and Donna Riley for support in preparation and editing of the manuscript.
Contributor Information
Jeannie L. Haggerty, Department of Family Medicine, McGill University, Montreal, QC.
Jean-Frédéric Lévesque, Centre de recherche du Centre hospitalier de l'Université de Montréal, Montréal, QC.
Darcy A. Santor, School of Psychology, University of Ottawa, Ottawa, ON.
Frederick Burge, Department of Family Medicine, Dalhousie University, Halifax, NS.
Christine Beaulieu, St. Mary's Research Centre, St. Mary's Hospital Center, Montreal, QC.
Fatima Bouharaoui, St. Mary's Research Centre, St. Mary's Hospital Center, Montreal, QC.
Marie-Dominique Beaulieu, Chaire Dr Sadok Besrour en médecine familiale, Centre de recherche du Centre hospitalier de l'Université de Montréal, Montréal, QC.
Raynald Pineault, Département de Médecine sociale et préventive, Université de Montréal.
David Gass, Department of Family Medicine, Dalhousie University, Halifax, NS.
References
- Bashshur R.L., Shannon G.W., Metzner C.A. 1971. “Some Ecological Differentials in the Use of Medical Services.” Health Services Research 6: 61–75 [PMC free article] [PubMed] [Google Scholar]
- Burge F., Haggerty J.L., Pineault R., Beaulieu M.-D., Lévesque J.-F., Beaulieu C., Santor D.A. 2011. “Relational Continuity from the Patient Perspective: Comparison of Primary Healthcare Evaluation Instruments.” Healthcare Policy 7 (Special Issue): 124–38 [PMC free article] [PubMed] [Google Scholar]
- Canadian Oxford Dictionary. 1998. Toronto: Oxford University Press [Google Scholar]
- Donabedian A. 1973. “Capacity to Produce Services in Relation to Need and Demand.” In Aspects of Medical Care Administration: Specifying Requirements for Health Care. Cambridge, MA: Harvard University Press [Google Scholar]
- Frenk J. 1992. “The Concept and Measurement of Accessibility.” In Frenk J., Ordonez C., Paganini J.M., Starfield B., eds., Health Services Research: An Anthology. Washington, DC: Pan American Health Organization [Google Scholar]
- Grol R., Wensing M., Mainz J., Ferreira P., Hearnshaw H., Hjortdahl P., et al. 1999. “Patients' Priorities with Respect to General Practice Care: An International Comparison. European Task Force on Patient Evaluations of General Practice (EUROPEP-I).” Family Practice 16(1): 4–11 [DOI] [PubMed] [Google Scholar]
- Grol R., Wensing M.Task Force on Patient Evaluations of General Practice 2000. “Patients Evaluate General/Family Practice: The EUROPEP-I Instrument.” Nijmegen, Netherlands: Centre for Quality of Care Research, Raboud University; [PMC free article] [PubMed] [Google Scholar]
- Haggerty J.L. 2011. “Measurement of Primary Healthcare Attributes from the Patient Perspective.” Healthcare Policy 7 (Special Issue): 13–20 [PMC free article] [PubMed] [Google Scholar]
- Haggerty J.L., Beaulieu C., Lawson B., Santor D.A., Fournier M., Burge F. 2011. “What Patients Tell Us about Primary Healthcare Evaluation Instruments: Response Formats, Bad Questions and Missing Pieces.” Healthcare Policy 7 (Special Issue): 66–78 [PMC free article] [PubMed] [Google Scholar]
- Haggerty J.L., Beaulieu M.-D., Pineault R., Burge F., Lévesque J.-F., Santor D.A., Bouharoui F., Beaulieu C. 2011. “Comprehensiveness of Care from the Patient Perspective: Comparison of Primary Healthcare Evaluation Instruments.” Healthcare Policy 7 (Special Issue): 154–66 [PMC free article] [PubMed] [Google Scholar]
- Haggerty J., Burge F., Lévesque J.-F., Gass D., Pineault R., Beaulieu M.-D., et al. 2007. “Operational Definitions of Attributes of Primary Healthcare: Consensus Among Canadian Experts.” Annals of Family Medicine 5: 336–44 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haggerty J.L., Lévesque J.-F., Roberge D., Gauthier J. 2009. Assessing Access to Primary Care: Understanding the Context and Adapting Measurement Instruments to the Specificity of Rural, Remote or Isolated Areas. Unpublished report. Available from J. Haggerty, Department of Family Medicine, McGill University, Montreal, QC [Google Scholar]
- Jöreskog K.G., Sörbom D. 1996. LISREL 8: User's Reference Guide. Chicago: Scientific Software International, Inc [Google Scholar]
- Lévesque J.-F. 2006. Inequalities in Access to Health Care in Urban South India. Doctoral thesis dissertation. Montreal, QC: Université de Montréal [Google Scholar]
- Love M.M., Mainous A.G. 1999. “Commitment to a Regular Physician: How Long Will Patients Wait to See Their Own Physician for Acute Illness?” Journal of Family Practice 48: 202–07 [PubMed] [Google Scholar]
- Penchansky R., Thomas J.W. 1981. “The Concept of Access: Definition and Relationship to Consumer Satisfaction.” Medical Care 19: 127–40 [DOI] [PubMed] [Google Scholar]
- Safran D.G., Kosinski M., Tarlov A.R., Rogers W.H., Taira D.H., Lieberman N., et al. 1998. “The Primary Care Assessment Survey: Tests of Data Quality and Measurement Performance.” Medical Care 36: 728–39 [DOI] [PubMed] [Google Scholar]
- Santor D.A., Haggerty J., Lévesque J.-F., Burge F., Beaulieu M.-D., Gass D., Pineault R. 2011. “An Overview of Confirmatory Factor Analysis and Item Response Analysis Applied to Instruments to Evaluate Primary Healthcare.” Healthcare Policy 7 (Special Issue): 79–92 [PMC free article] [PubMed] [Google Scholar]
- Shi L., Starfield B., Xu J. 2001. “Validating the Adult Primary Care Assessment Tool.” Journal of Family Practice 50(2): 161–71 [Google Scholar]
- Wensing M. 2006. EUROPEP 2006: Revised Europep Instrument and User Manual. Nijmegen, Netherlands: Centre for Quality of Care Research, Raboud University; Retrieved July 23, 2011. <http://www.topaseurope.eu/?q=node/13>. [Google Scholar]