Abstract
Background
The Patient Participation Program (PPP) was a patient satisfaction survey endorsed by the Royal Australian College of General Practitioners and designed to assist general practitioners in continuous quality improvement (CQI). The survey was been undertaken by 3500 practices and over a million patients between 1994 and 2003. This study aimed to use pooled patient questionnaire data to investigate changes in satisfaction with primary care over time.
Methods
The results of 10 years of the PPP surveys were analyzed with respect to 10 variables including the year of completion, patient age, gender, practice size, attendance at other doctors, and whether the practice had previously undertaken the survey. Comparisons were made using Logistic Generalized Estimating Equations (LGEE).
Results
There was a very high level of satisfaction with general practice in Australia (99% of respondents). An independent indicator of satisfaction was created by pooling the results of 12 questions. This new indicator had a greater variance than the single overall satisfaction question. Participants were shown to have higher levels of satisfaction if they were male, older, did not attend other practitioners or the practice was small in size. A minimal improvement in satisfaction was detected in this pooled indicator for the second or third survey undertaken by a practice. There was however no statistically significant change in pooled satisfaction with the year of survey.
Conclusion
The very high level of satisfaction made it difficult to demonstrate change. It is likely that this and the presentation of results made it difficult for GPs to use the survey to improve their practices. A more useful survey would be more sensitive to detect negative patient opinions and provide integrated feedback to GPs. At present, there are concerns about the usefulness of the PPP in continuous quality improvement in general practice.
Background
There is an extensive literature on patient satisfaction with health care but only a few that have been specifically designed and validated for their use in continuous quality improvement (CQI). CQI is a management concept that utilizes repeated cycles of data gathering, analysis, action and reappraisal. It seeks consumer feedback and uses this to generate change and improvement in a service. Examples of such surveys include the General Practice Assessment Questionnaire (GPAQ) used by the National Health Service [1-4], and one designed by the European taskforce (EUROPEP) for comparative evaluation of health quality between different countries in Europe. [5-10]
The Patient Participation Program (PPP) is an Australian survey designed by the Royal Australian College of General Practitioners (RACGP) in 1992 – 93 and which had been in use in general practice until 2003.[11,12] It resulted in over a million patients being surveyed from 3500 general practices over a 10 year period. GPs and practices chose to participate in order to provide points for practitioner's vocational registration and later for the practices accreditation.
There are two versions of the survey that we have named 45Q and 60Q according to the number of questions they contain. Each survey encompassed a range of topics including interaction with the doctor, accessibility of care and the range of services available within the practice. The survey was completed by the patient in the waiting room before and after a consultation. The initial 45Q survey was validated by factor analysis.[11,12] In 1999 the instrument was modified to include additional questions designed for practice accreditation.
In the literature the longest time period over which patient satisfaction has been analysed in general practice is only 15 months and the study involved showed that patient satisfaction improved over time. [13] No articles on the measurement of patient satisfaction over a ten year period were discovered in a thorough review of the literature. Unfortunately, the few longitudinal satisfaction studies that do exist, such as those originating from health funds in the United States, have had significant methodological limitations. [14]
The aim of this study was to investigate whether patient satisfaction varied with practice characteristics and time. It was postulated that changes in patient satisfaction might, in part, reflect consumer/patient acceptance of broader changes in general practice.
The secondary aim was to determine whether undertaking the PPP program would improve subsequent patient satisfaction results from participating practices. This would be a reasonable assumption if the practices were undertaking CQI processes effectively.
Methods
The survey results were stored by the RACGP in numerous ASCII databases. (ASCII is a standard 7-bit code for the transmission of data). The RACGP gave permission to undertake secondary data analyses, provided anonymity was maintained. The data were converted into two excel spreadsheets and analysed using Logistic Generalized Estimating Equations (LGEE).
Development of indicators of satisfaction
Each of the two surveys contained a single question that enquired about the respondent's overall level of satisfaction with the practice. (This variable we named "overall") The four point answer scale was dichotomized into satisfied ('very satisfied' and 'satisfied') and unsatisfied responses('dissatisfied' and 'very dissatisfied'). Despite excellent face validity, this question had problems as an indicator of satisfaction in that there was very poor response variability (figure 1). More than 99% of respondents were fully satisfied with their practice.
Figure 1.
note the categories on the x-axis are not of equal size.
A separate indicator or measure of satisfaction was derived, in the absence of any such indicator in the original survey. We chose 12 questions that represented a range of important determinants of satisfaction and had almost identical wording in the two versions of the questionnaire. (45Q and 60Q) Refer to table 1. We have named this indicator "multistat" (pooled results of multiple statistics). This indicator was dichotomized into a group who were satisfied with all 12 items and a group that were dissatisfied with one or more items. A small pilot of 28 patients completing both 45Q and 60Q surveys concurrently indicated 82% concordance of the "multistat" indicator derived from each version of the survey.
Table 1.
Comparison of wording for overall satisfaction question and the 12 selected questions from the 45-Q and 60-Q survey
| PPP 45 Question survey | PPP 60 Question survey | ||
| 16 | In general how satisfied are you with the medical care you receive at this practice? | 55 | Overall how satisfied were you with the consultation? |
| PPP 45 Question survey | PPP 60 Question survey | ||
| 20 | Are you satisfied with the ease of making an appointment to see the doctor | 35 | Were you able to obtain your appointment at a time that was convenient for you? |
| 23 | Are you satisfied with the ease of seeing the doctor out of normal working hours | 30 | How easy have you found it to see a doctor out of hours? |
| 24 | Are you satisfied with the ease of having the doctor see you at home | 29 | How easy have you found it to obtain a home visit during office hours? |
| 26 | Are you satisfied with the amount of time the doctor spends with you | 38 | Did the doctor spend enough time with you? |
| 28 | Are you satisfied with the handling of accounts by the doctors office | 21 | Are you satisfied with the handling of accounts by the doctors office? |
| 29 | Are you satisfied with the doctors ability to deal with children | 22 | Are you satisfied with the doctor's ability to deal with children? |
| 30 | Are you satisfied with the doctor's willingness to spend time with you | 23 | Are you satisfied with the doctor's willingness to spend time with you? |
| 31 | Are you satisfied with the doctor's willingness to answer your questions | 24 | Are you satisfied with the doctor's willingness to answer your questions |
| 32 | Are you satisfied with the respect shown to you by the doctor | 17 | Are you satisfied with the respect shown to you by your doctor? |
| 34 | Are you satisfied with the doctors ability to treat your problems | 25 | Are you satisfied with the doctors ability to treat your problems? |
| 35 | Are you satisfied with the doctors concern about you problems | 26 | Are you satisfied with the doctor's concern about your problems? |
| 37 | Are you satisfied with the ability to choose which doctor you see | 10 | Are you able to see the doctor of your choice at this practice? |
(Abreviations: sat. – satisfied with, WR – waiting room, Dr – doctor, SE – side effects, Rx – medication, avail. – availability)
Analysis
Multivariable analysis was undertaken comparing patient satisfaction, as measured by the "overall" and "multistat" indicators, to 10 independent variables. The independent variables were the patient's age, gender, years attending the practice and whether they saw a doctor from another practice, the practice size, the practice location, using the Accessibilty and Remoteness Index of Australia (ARIA code)[15], and socioeconomic status, using the Socio Economic Indexes For Areas (SEIFA code)[16], the year the survey was conducted, the number of times a practice had conducted the survey and the questionnaire that had been used. Logistic Generalized Estimating Equations (LGEE)[17,18] was chosen for the analysis. LGEE analyses data in discrete clusters, regarding all of the surveys from a single practice as being separate from surveys from other practices.
Results
The completed database included surveys collected from 1,119,688 patients. This represented 10,709 survey episodes undertaken by 3,554 distinct practices. We have no information on response rates. Also it was not possible to match 845 surveys (7.9%) to a known practice and these results were excluded from analysis. The earliest survey was scanned on the 12th January 1994 and the latest on the 8th December 2003. After a peak of 218,033 in 1996, the number of surveys per year has dropped to 28,448 in 2003. Figure 2 gives a breakdown of the number of patients surveyed each year.
Figure 2.
Patients surveyed by year.
Figure 1 illustrates the distribution of practices according to the proportion of dissatisfied responses for each of the two indicators (overall satisfaction and multistat) within discrete practices.
The median dissatisfaction rate for the "overall" indicator was only 0.5% with an intraquartile range from 0%–1.2%. The median dissatisfaction rate for the "multistat" indicator was more substantial at around 18% with a intraquartile range from 12%–26%.
It was noteworthy that 2 practices stood out with over 20% of patients dissatisfied (overall), and 2 practices had over 90% of patients dissatisfied with at least one of the 12 selected items (multistat).
Within both surveys, the questions with the most dissatisfaction included appointment availability, access to home visits, access to after hour care, waiting time, discussion of the costs of treatments and the cost of investigations.
Analysis
Figure 3 demonstrates the level of dissatisfaction in each of the two indicators for each year of the survey. The apparent change in the multistat indicator in 1999 is presumably due to the switch from the 45Q to the 60Q survey.
Figure 3.
note the 'overall' and 'multistat' have different y axes.
Multivariable analysis showed that satisfaction was related to all the variables examined except year of survey for the multistat (i.e. time). Although the overall indicator demonstrated significant change with time(p = 0.01) the size of this change was very small (Table 2) rendering this result unimportant or without meaning.
Table 2.
Multivariable analysis of 10 independent variables in relation to the two satisfaction variables.
| Independent Variable | Overall | Multistat |
| p | p | |
| Patient Age | <.01 | <.01 |
| Patient Gender | <.01 | <.01 |
| Yrs patient has attended Practice | 0.02 | <.01 |
| Pt attends different doctor | <.01 | <.01 |
| ARIA code of practice | <.01 | <.01 |
| SEIFA code of practice | <.01 | <.01 |
| Number of GPs in practice | <.01 | <.01 |
| Sequence (1st, 2nd survey etc) | 0.82 | <.01 |
| Year of survey | 0.01 | 0.27 |
| Questionnaire | <.01 | <.01 |
The odds ratios (OR) for the independant variables in the multivariable analysis are listed in tables 3 and 4. An OR greater than 1 indicates higher dissatisfaction. This was shown to diminish as measured by the "multistat" indicator with advancing patient age, male gender, smaller practice size, patients who do not visit other doctors, attendance at practices in highly accessible areas (ARIA) and high SE areas (SEIFA). Dissatisfaction slightly increased with respect to duration of attending a practice, particularly after the first year. All of these changes were statistically significant. The Overall indicator gave similar results with the exception of gender, survey instrument, and years attending the practice, where satisfaction was influenced in the opposite direction.
Table 3.
Multivariable analysis – Odds Ratios for Dissatisfaction "Overall"
| Variable | Category | Estimate | SE | Odds Ratio |
| Age | 14 yrs or less | 0.169 | (0.194) | 1.184 |
| 15–24 yrs | 0.265 | (0.093) | 1.303 | |
| 25–44 yrs | 0.172 | (0.081) | 1.188 | |
| 45–64 yrs | 0.053 | (0.081) | 1.054 | |
| 65–74 | -0.130 | (0.089) | 0.878 | |
| 75+ | - | - | - | |
| Gender | Female | -0.116 | (0.043) | 0.890 |
| Male | - | - | - | |
| Years Attending Practice | 0–1 yrs | 0.179 | (0.058) | 1.196 |
| 2–3 yrs | 0.108 | (0.055) | 1.114 | |
| 4–5 yrs | 0.096 | (0.051) | 1.101 | |
| 6+ years | - | - | - | |
| See Another Doctor | No | -0.922 | (0.045) | 0.398 |
| Yes | - | - | - | |
| ARIA | Highly Accessible | -1.268 | (0.356) | 0.281 |
| Accessible | -0.980 | (0.359) | 0.375 | |
| Moderately Accessible | -0.899 | (0.366) | 0.407 | |
| Remote | -1.123 | (0.394) | 0.325 | |
| Very Remote | - | - | - | |
| Sequence | 1 | -0.300 | (0.302) | 0.741 |
| 2 | -0.297 | (0.292) | 0.743 | |
| 3 | -0.290 | (0.293) | 0.748 | |
| 4 | -0.124 | (0.312) | 0.883 | |
| 5 | 0.019 | (0.467) | 1.019 | |
| 6 | - | - | - | |
| Year | 1994 | -0.403 | (0.413) | 0.668 |
| 1995 | -0.325 | (0.285) | 0.723 | |
| 1996 | -0.542 | (0.281) | 0.582 | |
| 1997 | -0.648 | (0.287) | 0.523 | |
| 1998 | -0.623 | (0.289) | 0.536 | |
| 1999 | -0.452 | (0.293) | 0.636 | |
| 2000 | -0.337 | (0.209) | 0.714 | |
| 2001 | -0.347 | (0.286) | 0.707 | |
| 2002 | -0.346 | (0.345) | 0.708 | |
| 2003 | - | - | - | |
| Survey Version | 45 | 0.397 | (0.153) | 1.487 |
| 60 | - | - | - | |
| Practice Size | 0.081 | (0.008) | 1.084 | |
| SEIFA | Low | 0.282 | (0.066) | 1.326 |
| Low/Medium | 0.285 | (0.109) | 1.330 | |
| Medium/High | 0.154 | (0.098) | 1.166 | |
| High | - | - | - | |
| Intercept | -2.538 | (0.494) |
Table 4.
Multivariable analysis – Odds Ratios for Dissatisfaction – Multistat
| Variable | Category | Estimate | SE | Odds Ratio |
| Age | 14 yrs or less | 0.638 | (0.052) | 1.893 |
| 15–24 yrs | 0.679 | (0.025) | 1.972 | |
| 25–44 yrs | 0.847 | (0.023) | 2.333 | |
| 45–64 yrs | 0.540 | (0.021) | 1.716 | |
| 65–74 | 0.120 | (0.021) | 1.127 | |
| 75+ | - | - | - | |
| Gender | Female | 0.118 | (0.010) | 1.125 |
| Male | - | - | - | |
| Years Attending Practice | 0–1 yrs | -0.494 | (0.018) | 0.610 |
| 2–3 yrs | -0.156 | (0.016) | 0.856 | |
| 4–5 yrs | -0.045 | (0.015) | 0.956 | |
| 6+ years | - | - | - | |
| See Another Doctor | No | -0.489 | (0.015) | 0.613 |
| Yes | - | - | - | |
| ARIA | Highly Accessible | -0.405 | (0.166) | 0.667 |
| Accessible | -0.271 | (0.168) | 0.763 | |
| Moderately Accessible | -0.270 | (0.175) | 0.763 | |
| Remote | -0.385 | (0.206) | 0.680 | |
| Very Remote | - | - | - | |
| Sequence | 1 | -0.190 | (0.157) | 0.827 |
| 2 | -0.257 | (0.154) | 0.773 | |
| 3 | -0.229 | (0.154) | 0.795 | |
| 4 | -0.203 | (0.156) | 0.816 | |
| 5 | -0.195 | (0.148) | 0.823 | |
| 6 | - | - | - | |
| Year | 1994 | -0.119 | (0.169) | 0.888 |
| 1995 | -0.152 | (0.115) | 0.859 | |
| 1996 | -0.095 | (0.110) | 0.909 | |
| 1997 | -0.069 | (0.106) | 0.933 | |
| 1998 | -0.056 | (0.105) | 0.946 | |
| 1999 | 0.006 | (0.096) | 1.006 | |
| 2000 | -0.011 | (0.095) | 0.989 | |
| 2001 | 0.068 | (0.098) | 1.070 | |
| 2002 | -0.035 | (0.108) | 0.966 | |
| 2003 | - | - | - | |
| Survey Version | 45 | -0.515 | (0.051) | 0.598 |
| 60 | - | - | - | |
| Practice Size | 0.061 | (0.005) | 1.063 | |
| SEIFA | Low | 0.220 | (0.032) | 1.246 |
| Low/Medium | 0.165 | (0.410) | 1.179 | |
| Medium/High | 0.094 | (0.046) | 1.099 | |
| High | - | - | - | |
| Intercept | -0.500 | (0.239) |
Figure 4 compares satisfaction according to survey sequence (the first, second, third or more survey conducted by a given practice). This should not be confused with the year of the survey. The change in relation to the Overall indicator did not reach significance. Although dissatisfaction with the multistat appeared to increase with each subsequent survey episode, the multivariable analysis indicates otherwise. The odds ratio (table 4) show that there was a small drop in dissatisfaction with the second and third surveys and then 4th and 5th surveys show the same level of dissatisfaction as the initial survey. It is noteworthy that the relative magnitude of this decrease in dissatisfaction (multistat) between first and second surveys is only 7%. (Odds ratio = 1.07) In other words dissatisfaction has only dropped from around 21% to 19.5%.
Figure 4.
Sequence indicates the order in which a survey was conducted, ie first survey, second, third etc. note the number of practices participating drops off as the sequence increases.
Discussion
The primary aim of this analysis was to look at change in patient satisfaction over time. On multivariable analysis we found that there was a significant change in the "overall" indicator but not in the more robust "multistat". The actual size of the change in "overall" satisfaction was less than 1 percent and must be regarded as inconsequential. Accordingly we conclude there was no meaningful change with time.
We had analyzed over a million surveys and it is unlikely that the survey lacks power.
It could be argued that the survey lacked sensitivity. The survey however performs as well as other surveys. Female patients, younger patients and those who regularly attend other doctors exhibited more dissatisfied responses. (multistat) Also, there was greater dissatisfaction with larger sized practices. These findings have been independently found in other surveys. [19-26] The replication of these finding offers a degree of criterion validity to the survey instruments.
It should be noted that the two indicators demonstrated opposite effects on some variables (gender, survey instrument, years attending the practice), suggesting that they have slightly different meanings. We have found the multistat to be more useful as it has greater response variability.
Alternatively the effect with time could have been confounded by the change in survey instrument midway through the 10 year period. We attempted to ameliorate this by including it as a variable in the multivariable analysis.
Despite the survey being a sincere attempt to provide practices with opportunities to practice CQI, there may have been limitations in the administration of the instrument. It is not possible to control the selection of patients, or the introduction of bias from reception staff or practitioners. Regardless, it represents a real life attempt at providing a survey for a large number of practices over a prolonged period of time.
The fact that patient satisfaction did not change in a decade that saw major changes to the structure of general practice in Australia, like accreditation, Divisions of General Practice, changes in the GP demographic, vocational registration and continuing medical education is itself surprising. It could be that patients remain satisfied despite these changes. It would seem more likely that "patient satisfaction" in the PPP survey did not measure satisfaction with the structure of general practice, either on the micro or macro scale. It implies that patient satisfaction may in fact be relatively stable over time.
The secondary aim was to compare satisfaction from practices undertaking the PPP for the first time, with second, third and subsequent surveys. Practices undertaking the program were required to review the results and identifying changes that could be made to their practice. It was hypothesized that subsequent surveys should show improvement in patient satisfaction. Multivariable analysis indicated that only the more robust "multistat" indicator showed significant change. This change however was rather meager. It should be noted that the power of the analysis drops off with the sequence, as fewer practices undertook the larger number of surveys.
This small improvement in patient's perceptions was noted only for practices undertaking the program at the 2nd and 3rd visit. The odds ratio 1.07 (between first and second surveys) represents only a small change from 21% to 19.5% dissatisfaction. The size of this change is so small in magnitude as to be rendered almost meaningless. If there is an improvement in patient satisfaction, it is eroded by the third cycle and is completely lost by the fourth or fifth cycle. In light of this result the effectiveness of the PPP survey as an instrument for CQI should be regarded as questionable.
The study uncovered several deficiencies in the survey design. These included the lack of an integrated index like the "multistat" in feedback to GPs, and the very high level of satisfaction, leaving no room to register improvement. Although many patient surveys report high satisfaction levels, they often fail to uncover the negative opinions of respondents. [27] In addition it has been noted that GPs are not disposed to respond to negative information.[13,28-30] There was evidence of this effect when we reviewed GP responses to their survey results.
Conclusion
In conclusion, the PPP has failed to identify changes in patient satisfaction with time, and has shown only small non sustained improvement with subsequent cycles of the program. Although minimal initial improvement in satisfaction was demonstrated, the small magnitude and transience brings it's usefulness in CQI open to question. It could be enhanced if future surveys address some of the major deficiencies of this survey, namely failing to elicit negative feedback from patients, lack of an integrated index and failing to address GP attitudes to negative feedback.
Abbreviations
45Q: 45 question survey; 60Q: 60 question survey; ARIA: Accessibilty and Remoteness Index of Australia; ASCII: American Standard Code for Information Interchange; CQI: continuous quality improvement; EUROPEP: General Practice Questionaire designed by European Taskforce; GP: General Practitoner; GPAQ: General Practice Assessment Questionnaire; LGEE: Logistic Generalized Estimating Equations; Multistat: Patient satisfaction measure derived from multiple questionnaire statistics; OR: Odds Ratio; p: Probability; PPP: Patient Participation Program; RACGP: Royal Australian College of General Practitioners; SEIFA: Socio Economic Indexes For Areas
Competing interests
The authors declare that they have no competing interests.
Authors' contributions
JA conceived and supervised the project. This was undertaken as a Masters thesis by distance education from Monash University, with support from the University of Adelaide Discipline of General Practice and funding supplied by the Primary Health Care Research Evaluation and Development (PHC RED) Program. He negotiated with the Royal Australian College of general Practitioners (RACGP) for access to the data, converted the data into an access database, proposed the research questions and drafted the manuscript. PS is Clinical associate Professor of the department of general practice, Monash University. He supervised the Masters thesis reviewing the proposal, analysis and manuscript. NS is Professor of the Discipline of General Practice at the University of Adelaide and Director of the Primary Health Care Research Evaluation and Development Program (PHC RED Program) at the University of Adelaide. He co-supervised the thesis. ER is a statistician within the discipline of General Practice, University of Adelaide. She undertook the statistical modeling and reviewed the manuscript/analysis. All authors read and approved the final manuscript.
Pre-publication history
The pre-publication history for this paper can be accessed here:
Acknowledgments
Acknowledgements
South Australian Faculty of the Royal Australian College of General Practitioners, Primary Health Care Research Evaluation and Development Program, University of Adelaide – discipline of General Practice, Department of General Practice Monash University, my wife and family.
Contributor Information
James Allan, Email: jandfallan@internode.on.net.
Peter Schattner, Email: Peter.Schattner@med.monash.edu.au.
Nigel Stocks, Email: nigel.stocks@adelaide.edu.au.
Emmae Ramsay, Email: emmae.ramsay@adelaide.edu.au.
References
- Ramsay J, Campbell JL, Schroter S, Green J, Roland M. The General Practice Assessment Survey (GPAS): tests of data quality and measurement properties. Fam Pract. 2000;17:372–9. doi: 10.1093/fampra/17.5.372. [DOI] [PubMed] [Google Scholar]
- Bower P, Roland MO. Bias in patient assessments of general practice: general practice assessment survey scores in surgery and postal responders. Br J Gen Pract. 2003;53:126–8. [PMC free article] [PubMed] [Google Scholar]
- Bower P, Mead N, Roland M. What dimensions underlie patient responses to the General Practice Assessment Survey? A factor analytic study. Fam Pract. 2002;19:489–95. doi: 10.1093/fampra/19.5.489. [DOI] [PubMed] [Google Scholar]
- Campbell JL, Ramsay J, Green J. Age, gender, socioeconomic, and ethnic differences in patients' assessments of primary health care. Qual Health Care. 2001;10:90–5. doi: 10.1136/qhc.10.2.90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grol R, Wensing M. Patients in Europe evaluate general practice care: an international comparison. Br J Gen Pract. 2000;50:882–7. [PMC free article] [PubMed] [Google Scholar]
- Klingenberg A, Bahrs O, Szecsenyi J. [How do patients evaluate general practice? German results from the European Project on Patient Evaluation of General Practice Care (EUROPEP)] Z Arztl Fortbild Qualitatssich. 1999;93:437–45. [PubMed] [Google Scholar]
- Kroneman MW, Maarse H, Zee J van der. Direct access in primary care and patient satisfaction: a European study. Health Policy. 2006;76:72–9. doi: 10.1016/j.healthpol.2005.05.003. [DOI] [PubMed] [Google Scholar]
- Vedsted P, Mainz J, Lauritzen T, Olesen F. Patient and GP agreement on aspects of general practice care. Fam Pract. 2002;19:339–43. doi: 10.1093/fampra/19.4.339. [DOI] [PubMed] [Google Scholar]
- Wensing M, Vedsted P, Kersnik J, Peersman W, Klingenberg A, Hearnshaw H, Hjortdahl P, Paulus D, Kunzi B, Mendive J, Grol Rl. Patient satisfaction with availability of general practice: an international comparison. Int J Qual Health Care. 2002;14:111–8. doi: 10.1093/oxfordjournals.intqhc.a002597. [DOI] [PubMed] [Google Scholar]
- Wensing M, Baker R, Szecsenyi J, Grol R. Impact of national health care systems on patient evaluations of general practice in Europe. Health Policy. 2004;68:353–7. doi: 10.1016/j.healthpol.2003.10.010. [DOI] [PubMed] [Google Scholar]
- Steven ID, Thomas SA, Eckerman E, Browning C, Dickens E. The provision of preventive care by general practitioners measured by patient completed questionnaires. J Qual Clin Pract. 1999;19:195–201. doi: 10.1046/j.1440-1762.1999.00332.x. [DOI] [PubMed] [Google Scholar]
- Steven ID, Thomas SA, Eckerman E, Browning C, Dickens E. A patient determined general practice satisfaction questionnaire. Aust Fam Physician. 1999;28:342–8. [PubMed] [Google Scholar]
- Vingerhoets E, Wensing M, Grol R. Feedback of patients' evaluations of general practice care: a randomised trial. Qual Health Care. 2001;10:224–8. doi: 10.1136/qhc.0100224... [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kahn KL, Liu H, Adams JL, Chen WP, Tisnado DM, Carlisle DM, Hays RD, Mangione CM, Damberg CL. Methodological challenges associated with patient responses to follow-up longitudinal surveys regarding quality of care. Health Serv Res. 2003;38:1579–98. doi: 10.1111/j.1475-6773.2003.00194.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Measuring Remoteness: Accessibility/Remoteness Index of Australia. Australian Government Department of Health and Ageing, Occasional Paper New Series: Number 14. 2001. http://www.health.gov.au/internet/main/publishing.nsf/Content/7B1A5FA525DD0D39CA25748200048131/$File/ocpanew14.pdf
- Socio-Economic Indexes for Areas (SEIFA index) Australian Bureau of Statistics, Catalogue No 20390, – Information Paper: An Introduction to Socio-Economic Indexes for Areas (SEIFA), 2006 – Latest ISSUE Released at 11:30 AM (CANBERRA TIME) 03/26/2008. http://www.abs.gov.au/AUSSTATS/ABS@.NSF/Latestproducts/2039.0Main%20Features32006?opendocument&tabname=Summary&prodno=2039.0&issue=2006&num=&view=
- Carriere KC, Roos LL, Dover DC. Across Time and Space: Variations in Hospital Use During Health Reform. Health Services Research. 2000;35:467–487. [PMC free article] [PubMed] [Google Scholar]
- Liang KY, Z S. Longitudinal data analysis using general linear models. Biometrika. 1986;73:13–22. doi: 10.1093/biomet/73.1.13. [DOI] [Google Scholar]
- Grol R, Wensing M. Patients Evaluate General/family Practice; The Europep Instrument. Mediagroep KUN/UMC. 2000.
- Greco M, Brownlea A, McGovern J. Impact of patient feedback on the interpersonal skills of general practice registrars: results of a longitudinal study. Med Educ. 2001;35:748–56. doi: 10.1046/j.1365-2923.2001.00976.x. [DOI] [PubMed] [Google Scholar]
- Kalda R, Polluste K, Lember M. Patient satisfaction with care is associated with personal choice of physician. Health Policy. 2003;64:55–62. doi: 10.1016/S0168-8510(02)00160-4. [DOI] [PubMed] [Google Scholar]
- Baker R. Characteristics of practices, general practitioners and patients related to levels of patients' satisfaction with consultations. Br J Gen Pract. 1996;46:601–5. [PMC free article] [PubMed] [Google Scholar]
- Wensing M, Vleuten C van de, Grol R, Felling A. The reliability of patients' judgements of care in general practice: how many questions and patients are needed? Qual Health Care. 1997;6:80–5. doi: 10.1136/qshc.6.2.80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McKinley RK, Manku-Scott T, Hastings AM, French DP, Baker R. Reliability and validity of a new measure of patient satisfaction with out of hours primary medical care in the United Kingdom: development of a patient questionnaire. Bmj. 1997;314:193–8. doi: 10.1136/bmj.314.7075.193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McKinley RK, Stevenson K, Adams S, Manku-Scott TK. Meeting patient expectations of care: the major determinant of satisfaction with out-of-hours primary medical care? Fam Pract. 2002;19:333–8. doi: 10.1093/fampra/19.4.333. [DOI] [PubMed] [Google Scholar]
- Campbell JL, Ramsay J, Green J. Practice size: impact on consultation length, workload, and patient assessment of care. Br J Gen Pract. 2001;51:644–50. [PMC free article] [PubMed] [Google Scholar]
- Williams B, Coyle J, Healy D. The meaning of patient satisfaction: an explanation of high reported levels. Soc Sci Med. 1998;47:1351–9. doi: 10.1016/S0277-9536(98)00213-5. [DOI] [PubMed] [Google Scholar]
- Rider EA, Perrin JM. Performance profiles: the influence of patient satisfaction data on physicians' practice. Pediatrics. 2002;109:752–7. doi: 10.1542/peds.109.5.752. [DOI] [PubMed] [Google Scholar]
- Kvamme OJ, Sandvik L, Hjortdahl P. [Quality of general practice as experienced by patients] Tidsskr Nor Laegeforen. 2000;120:2503–6. [PubMed] [Google Scholar]
- Wensing M, Vingerhoets E, Grol R. Feedback based on patient evaluations: a tool for quality improvement? Patient Educ Couns. 2003;51:149–53. doi: 10.1016/S0738-3991(02)00199-4. [DOI] [PubMed] [Google Scholar]