Setting standards based on patients' views on access and continuity: secondary analysis of data from the general practice assessment survey

Abstract

Objectives

To examine patients' views on access and continuity in general practice to derive quality standards.

Design

Secondary analysis of data from general practice research studies and routine quality assessment activities undertaken by practices and primary care trusts.

Setting

General practice.

Participants

General practice patients.

Results

Satisfactory standards of access were next day appointments with general practitioners and a 6-10 minute wait for consultations to begin. A satisfactory level of continuity was seeing the same general practitioner “a lot of the time.” Standards varied with the analytic method used and by sociodemographic group.

Conclusions

Standards expected by patients in primary care can be derived from linked report-assessment pairs. Patients may have expectations of access that are in excess of government targets. Patients also have high expectations of continuity of care. It is unclear the degree to which such standards are reliable or valid, how conflicts between access and continuity should be resolved, or how these standards relate to other priorities of patients such as high quality interpersonal care.

What is already known on this topic

Standards are increasingly being set for the provision of health services

Surveys and consultation exercises before the NHS plan helped set the standard for a maximum waiting time of 48 hours for appointments to see general practitioners

The optimal methods by which patients should be involved in setting standards and the utility of such standards are unclear

What this study adds

Satisfactory standards of access were next day appointments, a 6-10 minute wait for consultations to begin, and seeing the same general practitioner a lot of the time

Patients may have expectations for access to primary care in excess of current government targets

Introduction

The UK government has emphasised the measurement of performance in the NHS, highlighting the need to meet standards for appropriateness, effectiveness, and efficiency of care.¹ Standards are defined as “descriptions of specific aspects of healthcare practices to which are attached prescriptive values” and may legitimately include the views of patients.²

Patients have already influenced the setting of standards for access to primary care. The recent national survey of NHS patients (64.5% response rate from 100 000 patients, sampled from electoral registers) found that 34% of patients had got an appointment with a general practitioner of their choice on the same or next day, 29% had to wait two or three days, and 25% had to wait four or more days. Overall, 19% stated that the appointment should have been sooner.³ Waiting times were also an issue during consultation with patients about the NHS plan.⁴ This plan set a specific standard for waiting times by 2004 of seeing a primary care professional within 24 hours and a general practitioner within 48 hours.

This standard was not explicitly derived from the national survey, which only inquired about how much longer patients waited than they wanted to and whether they thought they should have been seen more quickly.³ An alternative approach to setting standards is provided by the general practice assessment survey, a self report questionnaire examining patients' views of several aspects of general practice.⁵ Some scales in the questionnaire use two types of items: report (the patient's experience of care) and assessment (the patient's evaluation of that experience). These report-assessment pairs relate to waiting times for appointments with a particular general practitioner, with any general practitioner, and for consultations to begin, and the proportion of consultations with the patient's usual general practitioner (continuity of care).

These questions provide a method for deriving standards for patients by examining the proportion of respondents who are satisfied or dissatisfied with particular levels of service. A second issue is whether such standards vary by sociodemographic group—for example, acceptability may relate to expectations or needs, which may vary with income, ethnicity, or health status.⁶

We aimed to examine relations between reports of access and continuity in general practice and assessments of acceptability to derive patient based standards and to examine differences in standards between patients from different sociodemographic groups.

Methods

The general practice assessment survey comprises multiple subscales, but we restricted our analysis to access and continuity items with report-assessment pairs.⁷ The relevant items are listed on bmj.com (a copy of the questionnaire is available at www.gpas.co.uk).

The data (21 905 patients) derive from a survey of quality of care, a questionnaire validation study, an evaluation of pilots for personal medical services, and data analysed for primary care groups and trusts by the National Primary Care Research and Development Centre.⁵,⁸,⁹ Some datasets used version 1 of the general practice assessment survey, whereas others used an updated version 2, but the relevant items are identical in both versions. Table 1 details the source of the data.

Table 1.

 Details of data in general practice assessment survey

Source	Sample size* (% response rate)	Type of administration	No of practices	Mean age (years) of patients†	% women	% white	% with chronic illness	% employed‡	% owner occupier
Research study5	7254 (66)	Consecutive attenders	55	39.8	63.6	64.5	47.8	No data	33.8
Research study8	4488 (38)	Postal survey	60	51.3	61.4	92.8	28.5	53.3	77.7
Research study9	2940 (65)	Postal	23	47.9	57.9	93.5	29.2	51.2	67.0
Routine assessment	2204 (54 in postal survey only)	Consecutive attenders and postal survey	13	No data	64.0	98.6	45.6	No data	74.9
Routine assessment	1672 (unknown)	Consecutive attenders	30	51.9	66.5	97.8	51.4	No data	81.4
Routine assessment	1492 (unknown)	Consecutive attenders	10	48.7	66.5	98.5	35.7	47.3	74.7
Routine assessment	1069 (unknown)	Consecutive attenders	13	46.5	64.6	97.5	34.2	52.1	74.5
Routine assessment	452 (unknown)	Postal survey	4	49.9	58.0	97.8	25.6	52.9	69.1
Routine assessment	176 (70)	Postal survey	1	53.8	60.2	100	66.3	50.3	86.5
Routine assessment	158 (79)	Consecutive attenders	1	60.6	56.9	98.7	34.4	34.9	94.1

^*Number of respondents returning questionnaire in each study (including cases with missing item data). 

^†Personal data (means and percentages) based on cases with complete data in each study. 

^‡Full or part time employment. 

The general practice assessment survey is completed by patients attending surgeries or sent by post to those on the practice list. However, items refer to care in general, not to specific consultations.

We excluded patients from the main analysis when data were missing on any of the eight key report or assessment items. We examined sociodemographic correlates of missing data on these variables. In addition, report items on the availability of a particular doctor and of any doctor each have a “not applicable” response option, so we excluded patients using this option. We also excluded patients from the sociodemographic analyses if the relevant sociodemographic variable was missing.

Statistical methods

We used cross tabulation to examine patterns of missing data and for the main analysis of relations between reports and assessments. For simplicity, we dichotomised assessments of satisfaction into dissatisfied (very poor, poor, or fair) and satisfied (good, very good, excellent).

To set standards, a minimum proportion of patients (for example, three quarters) might be prescribed who must be satisfied with a given aspect of the service. Such criteria are unambiguous but also arbitrary and are sensitive to relatively small differences in the proportion of satisfied patients (for example, between 74% and 76%). Alternatively, standards might be based on large discontinuities in the data. For example, if a large proportion of patients are satisfied at one level of service (with an appointment the next day, for example) and far fewer are satisfied with the next level (waiting two or three days), then this might suggest a degree of agreement among patients as to an acceptable level of service. Such an approach is less arbitrary and more sensitive to the actual distribution of data. However it is dependent on the existence of such discontinuities and on agreement as to what represents a large discontinuity.

We used both methods in our analyses. The first criterion was that three quarters of patients should report being satisfied, which we then relaxed to two thirds of patients. In addition, a large discontinuity was defined as an absolute percentage change of greater than 25% in the proportion of satisfied patients between different levels of service. Where there was more than one such discontinuity, we took the largest.

To examine effects of demographic characteristics, we stratified raw cross tabulations by age (16-30, 31-45, 46-59, ⩾60), sex, ethnicity (white or other), chronic illness (yes or no), employment (employed full or part time or other), and accommodation (owner occupied or other). Although regression analysis using interaction terms would be the usual method of examining the moderating effects of variables, this would consider differences in regression slopes, whereas we focused on identifying discontinuities in the data or specific criteria related to the proportion of satisfied patients.¹⁰ Our analysis also ignored the fact that scores are clustered at practice level.⁸

Results

Missing data in the four report-assessment pairs used in the main analysis were associated with all sociodemographic characteristics. The most consistent predictors of missing data were presence of a chronic illness, no full or part time work, and increasing age. The increase in missing data associated with these variables was in the order of 1-5%, with the largest difference being 8.9%.

Table 2 shows the raw cross tabulations of report and assessment items. We found at least one identifiable discontinuity in all analyses and more than one discontinuity in some. Table 3 shows the standards identified by the different methods.

Table 2.

 Proportion of patients satisfied with access and continuity in general practice (n=14 291)

	% (No) not satisfied	% (No) satisfied
Waiting time for appointment with particular doctor:
Same day	5.4 (123)	94.6 (2143)
Next day	19.4 (617)	80.6 (2570)
2or 3 days	62.8 (2608)	37.2 (1545)
4 or 5 days	85.5 (1891)	14.5 (320)
>5 days	95.3 (2358)	4.7 (116)
Waiting time for appointment with any doctor:
Same day	7.3 (368)	92.7 (4687)
Next day	29.6 (1370)	70.4 (3254)
2 or 3 days	71.6 (2335)	28.4 (925)
4 or 5 days	87.0 (758)	13.0 (113)
>5 days	92.7 (446)	7.3 (35)
Waiting time (min) for consultations to begin:
0	5.5 (33)	94.5 (563)
<5	6.9 (115)	93.1 (1552)
6-10	33.6 (1595)	66.4 (3154)
11-20	71.3 (3100)	28.7 (1249)
21-30	87.8 (1536)	12.2 (214)
31-45	93.7 (710)	6.3 (48)
>45	97.4 (411)	2.6 (11)
Continuity for seeing same doctor:
Always	2.4 (76)	97.6 (3098)
Almost always	8.1 (412)	91.9 (4651)
A lot of the time	27.1 (576)	72.9 (1552)
Some of the time	65.6 (1927)	34.4 (1010)
Almost never	80.2 (661)	19.8 (163)
Never	75.8 (125)	24.2 (40)

Table 3.

 Standards identified by different methods

Access issue	75% criterion	66% criterion	Discontinuity
Waiting time for appointment with particular doctor	Next day	Next day	Next day
Waiting time for appointment with any doctor	Same day	Next day	Next day
Waiting times for consultations to begin	<5 minutes	<5 minutes	6-10 minutes
Continuity for seeing same doctor	Almost always	A lot of the time	A lot of the time

Associations with sociodemographic variables produced relatively minor variations in the identified standards (the stratified cross tabulations are shown on bmj.com). Table 4 shows, for each method, which standards were moderated by sociodemographic variables and how the standard differed in each subgroup. Acceptable waiting time for consultations to begin was the issue most sensitive to sociodemographic factors. Age and ethnicity were the most consistent moderating factors, with patients from ethnic minorities generally having higher standards and patients in the two older age groups (46-59, ⩾ 60 years) having lower standards.

Table 4.

 Variations in standards associated with sociodemographic factors

Access issue	75% criterion	66% criterion	Discontinuity
Waiting time for appointment with particular doctor	Original standard: next day	Original standard: next day	Original standard: next day
	Moderating variables: member of ethnic minority, 16-30 years old, 31-45 years old
	Moderated standard: same day
Waiting time for appointment with any doctor	Original standard: same day	Original standard: next day	Original standard: next day
	Moderating variables: ⩾60 years old	Moderating variables: member of ethnic minority, employed, 31-45 years old
	Moderated standard: next day	Moderated standard: same day
Waiting times for consultations to begin	Original standard: <5 minutes	Original standard: <5 minutes	Original standard: 6-10 minutes
	Moderating variables: ⩾60 years old	Moderating variables: white, male, not employed, chronic illness, owner-occupier; 46-59 year, ⩾60 years old	Moderating variables: member of ethnic minority, 16-30 years old
	Moderated standard: 6-10 minutes	Moderated standard: 6-10 minutes	Moderated standard: <5 minutes
Continuity for seeing same doctor	Original standard: almost always	Original standard: a lot of the time	Original standard: a lot of the time
	Moderating variables: ⩾60 years old
	Moderated standard: a lot of the time

Discussion

Standards for primary care services can be derived from linked report-assessment pairs in the general practice assessment survey. Our methods suggest that patients may have expectations of access that are in excess of government targets and also have high expectations of continuity of care. Two key issues are raised. The first is the methodological adequacy of this approach to standard setting. The second, dependent on the first, concerns policy implications of the analysis.

Methodological issues

Our analysis was suggested by the nature of the items in the general practice assessment survey, but the questionnaire was not designed explicitly as an instrument for setting standards. The processes by which patients make judgments in situations designed to elicit standards may differ from those used in completion of routine questionnaires. It is not clear, however, whether direct methods are necessarily more valid than the indirect method used here.

The results depend on the validity and reliability of the questionnaire. The questionnaire is reliable and has an interpretable factor structure.⁵,¹¹ However, patient reports of waiting times have not been validated against objective measures, and validation of subjective assessments of acceptability is problematic. Concerns about validity might arise because a small proportion of patients remained dissatisfied with high standards of care (for example, some were dissatisfied with no waiting time for consultations to begin).

The standards identified obviously depend on the particular definition of satisfaction and the thresholds applied (66%, 75%, or discontinuities). The binary definition of satisfaction is similar to published recommendations, but ratings of “fair” might be considered indicative of some degree of satisfaction.¹²,¹³ If the data are reanalysed in such a way, different results occur (for example, both 66% and 75% of patients are satisfied with waiting two or three days for an appointment with a specific general practitioner, waiting 11-20 minutes for consultations to begin, and seeing the same general practitioner “some of the time”). In the identification of standards, the 66% and 75% criteria are obviously arbitrary. The presence of discontinuities in the data suggests that thresholds of acceptability do exist, but it should be noted that this method was suggested by preliminary analysis, and a discontinuity was not defined a priori.

However, the important issue may not be whether the methods are correct (as they will always be arbitrary to a degree) but whether they are intuitively reasonable and a useful reflection of patients' views.

The validity of the standards also depends on the representativeness of the sample. The data derive from several sources including practices using the general practice assessment survey for routine service evaluation, although 69% of the present sample derives from three research studies.⁵,⁸,⁹ One of these studies had a response rate of only 38%, whereas another achieved a rate of 66%, similar to the national survey (although additional cases were lost because of missing data).⁵,⁸ However, when our analysis was restricted to data from the study with the higher response rate, the results remained unchanged. The national survey was completed on a community sample, whereas the current data also include patients attending surgeries.

Interpretation of standards

Unfortunately the government's 48 hour standard can only be compared to the “two or three day” response option in the general practice assessment survey. It should, however, be noted that standards for access to any general practitioner varied between “next day” and “same day” (see table 3) and did not extend to “two or three days” in any analysis. The questionnaire does not include items relating to access to primary care professionals other than general practitioners.

Questions relating both to waiting times and to satisfaction are framed somewhat differently in the national survey and our data, complicating direct comparisons. If respondents to the national survey are considered dissatisfied when they responded that they should have been seen a bit sooner or a lot sooner, then 35% are dissatisfied with waiting one day longer than wanted, increasing to 57% dissatisfied with waiting two days longer than wanted, 66% with three days longer than wanted, 72% with 4-7 days longer than wanted, and 83% with eight or more days longer than wanted. If it is assumed that patients wanted an appointment the next day, then waiting one or two days longer would relate to the 63% dissatisfaction for two or three days in our study (see table 1). This compares with around 35% and 57% dissatisfaction for waiting one day or two days longer, respectively, among respondents in the national survey, which indicates even higher levels of dissatisfaction in our study.

The methods of standard setting used by us did not prove highly sensitive to sociodemographic factors (although the dichotomising of sociodemographic variables such as ethnicity may have concealed important variations). Also of interest are factors that were not associated with variation. For example, it might be expected that patients with chronic health problems would value higher standards of continuity of care, but this was not evident, although this may reflect the relatively high levels of satisfaction expressed with continuity. Patients may value continuity over convenience of access in relation to certain problems such as mental health, as opposed to more minor health problems or where immediacy is perceived to be paramount.¹³,¹⁴ However, we could not compare the relative importance of the different standards and it is unlikely that standards (or health systems) can ever be sensitive to the full range of patient characteristics and preferences.

Finally, the high standards relating to access and continuity derived from our analysis may reflect that they have not been explicitly compared with other aspects of primary care for their overall priority.¹⁵ Quality of care in primary care is a combination of access to care and effectiveness of the care provided, and surveys of primary care patients in Europe suggest that interpersonal aspects may be more important than access issues such as waiting times for consultations (ranked 34th of 38 aspects of primary care in the United Kingdom).¹⁶,¹⁷ It should, however, be noted that a quick service for emergencies was ranked first in the same survey (our study did not include a measure of perceived urgency), and rapid access to appointments has also been ranked highly in previous studies.¹⁷,¹⁸ In addition to the priorities of patients, there is also a possible tension between the priorities of different bodies such as patients, professionals, and government.¹⁸,¹⁹

Expectations potentially conflict, such that there is a tension between highly accessible care and continuity. Economists would generally focus on explicit choice based methods in determining patient priorities, and there is evidence from such methods that interpersonal issues are more highly valued than either waiting times for consultation or consultation length.^20–22 However, interpersonal attributes such as being able to talk to the doctor are far more complex concepts than waiting times. Although patient based assessments of interpersonal care have some validity, they also have conceptual and practical limitations and do not lend themselves to setting clear, demonstrable standards of care compared with more concrete issues such as waiting times.^23–25

Conclusion

Report-assessment pairs in the general practice assessment survey provide a method for examining patient views of general practice that may be of use in setting standards and monitoring performance. Patients have high expectations relating to access to care, which may support current government targets and the development of alternative models of service delivery such as walk-in centres.¹³ The interpretation of such standards must be sensitive to the methods used to derive them, however, and to issues of priority in other aspects of general practice, such as the effectiveness of interpersonal care. Although access to services is an important issue in itself, some definitions of access include notions of effectiveness, and thus evidence concerning the clinical and cost benefits of rapid access is also of importance in the wider debate.²⁶