General Practitioners’ Concerns About Online Patient Feedback: Findings From a Descriptive Exploratory Qualitative Study in England

Salma Patel; Rebecca Cain; Kevin Neailey; Lucy Hooberman

doi:10.2196/jmir.4989

. 2015 Dec 8;17(12):e276. doi: 10.2196/jmir.4989

General Practitioners’ Concerns About Online Patient Feedback: Findings From a Descriptive Exploratory Qualitative Study in England

Salma Patel ^1,^✉, Rebecca Cain ¹, Kevin Neailey ¹, Lucy Hooberman ¹

Editor: Gunther Eysenbach

Reviewed by: Lisa Trigg, David Hanauer, Samantha Adams

PMCID: PMC4704896 PMID: 26681299

Abstract

Background

The growth in the volume of online patient feedback, including online patient ratings and comments, suggests that patients are embracing the opportunity to review online their experience of receiving health care. Very little is known about health care professionals’ attitudes toward online patient feedback and whether health care professionals are comfortable with the public nature of the feedback.

Objective

The aim of the overall study was to explore and describe general practitioners’ attitudes toward online patient feedback. This paper reports on the findings of one of the aims of the study, which was to explore and understand the concerns that general practitioners (GPs) in England have about online patient feedback. This could then be used to improve online patient feedback platforms and help to increase usage of online patient feedback by GPs and, by extension, their patients.

Methods

A descriptive qualitative approach using face-to-face semistructured interviews was used in this study. A topic guide was developed following a literature review and discussions with key stakeholders. GPs (N=20) were recruited from Cambridgeshire, London, and Northwest England through probability and snowball sampling. Interviews were transcribed verbatim and analyzed in NVivo using the framework method, a form of thematic analysis.

Results

Most participants in this study had concerns about online patient feedback. They questioned the validity of online patient feedback because of data and user biases and lack of representativeness, the usability of online patient feedback due to the feedback being anonymous, the transparency of online patient feedback because of the risk of false allegations and breaching confidentiality, and the resulting impact of all those factors on them, their professional practice, and their relationship with their patients.

Conclusions

The majority of GPs interviewed had reservations and concerns about online patient feedback and questioned its validity and usefulness among other things. Based on the findings from the study, recommendations for online patient feedback website providers in England are given. These include suggestions to make some specific changes to the platform and the need to promote online patient feedback more among both GPs and health care users, which may help to reduce some of the concerns raised by GPs about online patient feedback in this study.

Keywords: online reviews, physician quality, primary care, Internet, quality, patient empowerment, quality transparency, public reporting, attitude of health personnel, delivery of health care, feedback

Introduction

There has been growth in the use of online consumer feedback and review websites (eg, TripAdvisor), which some argue has allowed for transparent information and communication to influence change and has provided opportunities for consumers to read reviews and make more informed choices [1-3]. Similarly, there has been a growth in the volume of online patient ratings and comments, which suggests that patients in England (and other parts of the world) are embracing the opportunity to review their health care online [4-7]. There has also been a growth in the development of online patient feedback, with some patients now reviewing not just their experience of receiving health care, but also their medication and treatment plan [8].

Online patient feedback, in the context of this paper, can be defined as experiential feedback, ratings, reviews, and comments left by patients, carers, or service users on public Web-based platforms in England, such as NHS Choices, Patient Opinion, and iWantGreatCare, and on apps such as the iPhone-based Great Care app. Users can leave feedback and rate their experience with a general practitioner (GP) service, hospital, dentists, and other health care services, which is available in the public domain for other users to look at (see Figure 1 for an example of an online patient feedback website). The purpose of such websites is to give patients a voice by allowing them to leave feedback online, which some suggest will increase transparency, improve the quality of care, and could be used for service improvement [7,9]. Patients and carers could also then use these ratings and reviews to decide which health care provider to use; in England, this is part of the “patient choice” agenda [10-12].

Example of online patient feedback on NHS Choices website.

There are two major websites in the United Kingdom that (1) collect online reviews and ratings from patients about their experience of receiving care from their GP, and (2) allow the public to read patient feedback, which may be used by some patients or carers to choose a GP or a GP practice. The first is the NHS Choices website, which allows patients to leave comments under the GP practice’s name, but does not allow an individual GP to be named or identified in any feedback [13]. The second is the iWantGreatCare website, which allows patients to leave feedback under a GP’s name; therefore, GPs are named and identified in any review or feedback left on this website [14].

Research into online patient feedback or online physician-rating websites has been steadily increasing over the past few years with studies conducted in the United Kingdom [5-7,12,15-19], United States [20,21], Germany [22-26], and Australia [27] adding to the growing literature [28]. There appears to be some evidence to suggest that there is an association between online ratings and quality of care [4,5,21,29,30]. However, the results are not always consistent and, for some studies, the effect size is weak [28]. In particular, the extent to which the online ratings reflect the quality of primary care is less clear [6]. Furthermore, it is difficult to cross-apply the findings of different countries and different online patient feedback platforms because the characteristics of each online platform varies and the culture, context, and policies of each country and health care organization are different [9]. Despite this, some still argue that the “rich source of data” from online patient feedback has the potential to track quality of care [4,5,21,29,30].

Outside of the United Kingdom, a few studies have explored the type of patients who use online patient feedback platforms [24,31-34], whereas other studies have explored the attitudes of patients toward online patient feedback or doctor-rating websites [33,35,36]. A few physician representatives in the media have argued against the introduction of online feedback and rating websites by suggesting that they are dangerous and a waste of resources [37,38], and concerns about slander have also been raised by critics of such platforms [29,38-40]. However, there is very little research that explores health care professionals’ attitudes toward online patient feedback.

Because health care professionals are one of the primary recipients of online patient feedback, one of the aims of this study was to explore and describe GPs’ views about online patient feedback. This paper narrates the concerns raised by GPs in relation to online patient feedback only and the other findings, including the benefits of online patient feedback suggested by GPs, will be reported elsewhere. It is hoped that the findings from this study could be used to improve online patient feedback from the GPs’ perspective and this may help to increase usage of online patient feedback by GPs and, by extension, their patients too.

Methods

Data Collection

Because very little is known about GPs’ perceptions of online patient feedback, there was a need for in-depth exploration to capture attitudinal and experiential data; therefore, a qualitative approach using semistructured interviews was best suited. A topic guide was developed following the guidance suggested by Bryman [41], Matthews and Ross [42], and Tracy [43] (see Multimedia Appendix 1 for a copy of the topic guide). A literature review was used as a basis for the topic guide as well as information from discussions with key stakeholders, such as the main lead at one of the online patient feedback website providers in the United Kingdom and 4 GPs. Further issues raised by participants during the interviews were also pursued and participants were encouraged to draw on experiences to illuminate their responses. Three vignettes were also developed (see Multimedia Appendix 2) following the guidance of Barter and Renold [44] and were used as prompts if the participant had not seen an online feedback review website before.

Sampling and Recruitment

Participants were recruited from Cambridgeshire, London, and Northwest England. A probability sampling approach was employed initially to ensure a wide range of characteristics of participants as recommended by Bryman [41]. However, despite using various strategies (all described in Table 1), only 6 participants were recruited using probability sampling. Other researchers who attempted to recruit GPs for research in the United Kingdom also reported immense difficulties [45-51]. Therefore, because of the limited response rate and the difficulties with recruiting sufficient GPs for this study using probability sampling, we resorted to using snowball sampling and 14 further GPs were recruited this way using various approaches (detailed in Table 1). In total, 20 GPs were interviewed for this study because at that point thematic saturation [52] had been reached.

Table 1.

Recruitment strategies and number of general practitioners recruited for this study.

Recruitment strategy		Number of GPs recruited
Probability sampling
	Direct invitation to GP (postal invitations were sent to 25 practices in Cambridgeshire, which were then followed up by phone calls. A primary care research network [PCRN] also sent a letter on our behalf inviting and promoting the study to research active GPs and practices in Coventry)	1
	Invitation through practice managers (phone calls were made to 25 GP practice managers in Cambridge and a follow-up email was also sent; 13 further GP practices were then contacted through phone and then fax)	2
	Promoting study in email-based GP newsletters (the study was promoted in the following GP newsletters: Cambridgeshire NHS Newsletter, Lewisham Weekly Newsletter, Lambeth PCT newsletter, South NHS North West London Newsletter, Participate Autumn Magazine)	3
Snowball sampling
	Email to acquaintances with potential GP contacts	1
	Twitter call out to acquaintances with potential GP contacts	0
	GPs emailing their GP acquaintances	5
	Medical doctors phone calling their GP acquaintances	8
Total		20

Open in a new tab

Study Interviews and Participants

The interviews were conducted at the location that was most convenient to the participant, with a preference given to the GP practice where the GP worked. However, some GPs preferred to be interviewed at their home outside of working hours and one GP was interviewed at a private meeting room. GPs were paid £80 for their participation.

All participants were sent the participant information sheet beforehand either through email or in the post and this contained the aims of the study among other things. Informed consent was obtained from all participants before the start of the interview. Interviews were recorded using two digital voice recorders. The study had full ethical approval from the Biomedical and Scientific Research Ethics Committee at the University of Warwick.

The descriptive characteristics of the 20 participants interviewed are shown in Table 2. Although 60% (12/20) of the GPs interviewed were between the ages of 30 and 34 years, they represented a variation in relation to duration of experience as GP, type of GP, and gender.

Table 2.

Demographics and practice characteristics of participants (N=20).

Baseline characteristic		Frequency, n (%)
Age (years)
	25-29	1 (5)
	30-34	12 (60)
	35-39	3 (15)
	40-44	1 (5)
	45-49	1 (5)
	50-54	1 (5)
	55-59	1 (5)
Gender
	Male	12 (60)
	Female	8 (40)
Type of GP
	Salaried	6 (30)
	Partner	7 (35)
	Senior partner	2 (10)
	Lead	1 (5)
	Locum	4 (20)
Years practicing as GP
	1-5	14 (70)
	6-10	2 (10)
	11-15	1 (5)
	16-19	1 (5)
	≥20	2 (10)

Open in a new tab

Data Preparation and Analysis

Interviews were transcribed verbatim and each transcript was double-checked for inaccuracies. The transcripts were then exported to NVivo software and analyzed using the framework method. This is a form of thematic analysis developed in the 1980s by researchers at the NatCen Social Research and has been used widely since then, both in policy research and other areas [53-56]. A thematic framework was created which was refined and then applied to all the data. Categories and themes were refined and defined until explanations were formed and thematic maps were produced. The analysis was conducted by the first author (SP) and the thematic framework and thematic maps were checked by all authors.

Results

In this interview-based study, participants were asked about their experience, usage, and attitudes (both positive and negative) toward online patient feedback. Because of the richness, depth, and breadth of the interview data, it was only possible to report the concerns about online patient feedback raised by participants in this paper. Other findings, such as those related to GPs’ perceived benefits of online patient feedback or their attitudes toward social media, will be reported elsewhere. However, to place the GPs concerns into context, we believe it is important to understand participants’ usage of and experience with online patient feedback as well as their overall impression of patients leaving feedback online about them. This is narrated in the subsequent section.

Usage, Awareness, and Overall Impression of Online Patient Feedback

Three-quarters of GPs interviewed in this study were aware that patients can leave feedback for them or their practice on the NHS Choices website. Four GPs had direct experience with online patient feedback and their practice or GPs in their practice had received feedback online on the NHS Choices website. One of the GPs also admitted he had received negative personal feedback online on the iWantGreatCare website. The majority of GPs interviewed (n=17) did not currently consider online patient feedback as a way of collecting feedback from patients. However, 12 participants believed that patients do have a right to place feedback about their GP online as long as the feedback was factually correct and on an appropriate website. Five participants, however, disagreed suggesting that patients do not have the right to place feedback about their GPs online.

Concerns About Online Patient Feedback

In this study, 56 individual concerns were raised by GPs (31 of which were unique) when asked the open-ended question: “Do you have any concerns about online patient feedback?” Other concerns raised about online patient feedback during the interview were also included in the analysis. Figure 2 is a thematic map that shows a summary of the concerns raised by participants and the 7 main themes (highlighted and numbered in the diagram) are discussed in the subsequent sections.

A thematic map illustrating the downsides of online patient feedback (OPF) according to the GP participants in this study.

Theme 1: Online Patient Feedback is Biased

User Bias

Most participants were concerned that it is only young and middle-aged patients who are leaving feedback on online patient feedback websites. Some participants commented that the majority of their patients were elderly and were certainly not using this medium. This, according to them, indicates that the feedback and ratings that are online are not representative of the overall experience of their patients and, therefore, they questioned its validity and usefulness:

You are getting your opinions from again, one particular sector of the community...their perceptions, their understanding and attitudes are different to the rest of the population...You have to use it with a bit of scrutiny there, in terms of interpreting the data, how do you put that in practice? If you were to put in any changes?

P10

One participant felt that there was not enough publicity about the NHS feedback website and many patients do not even know about the website.

Online Patient Feedback is Mainly Negative Opinion

One of the most repeated concerns raised by a quarter of participants was that online patient feedback is mainly negative opinion and is and will become a channel for disgruntled patients:

There’s a bias towards putting negative feedback, if they’ve had a good experience, nonoutstanding one but an adequate one, they have no complaints but their needs are made [sic], they are unlikely to go on and post positive feedback.

P13

Other participants argued that it only takes one negative feedback to harm a GP’s reputation:

One unfortunate comment or bit of a negative feedback, which may be taken well out of context, can harm your reputation.

P16

Online Patient Feedback is Too Small in Number

A few participants (n=3) raised concerns that feedback left for a GP or a GP practice on online websites is too small in number and, therefore, it is not representative of their record of performance:

It’s a small number of comments, we’ve had 2 [online reviews] out of a population of 12,000 [on the NHS Choices website], I don’t think that would be representative enough for a potential patient to go on and go, “alright ok they’ve got 50% bad comments, right I’m not registering there” (laugh).

P2

GPs were concerned that patients could judge a practice or GP based on a very small number of reviews (and make an invalid “choice”) and this could also affect the doctor-patient relationship.

Reviews Could Be “Gamed”

Some participants (n=5) were concerned that reviews could be manipulated and that some GPs could encourage only satisfied patients to give feedback, which again would add bias to the data. Moreover, 4 participants from this study admitted they would only encourage those patients who they know will give positive feedback to leave feedback online:

No, obviously, if somebody has had a good experience, you might encourage it [leave feedback on NHS Choices]. But also, I think if somebody wants to make a complaint I would say you can write to the practice manager and there’s always, I may not actively promote it [giving feedback on NHS Choices].

P7

Theme 2: Risk of False Allegations and Absence of Regulation

Half of the participants felt that there was a very high risk of patients leaving false allegations about them or the practice on online patient feedback websites. Furthermore, a quarter of participants felt that the owners of such websites (eg, the NHS Choices feedback site) were not regulating feedback left on these websites and removing malicious or factually incorrect comments from patients. Participants were particularly concerned that their patients with psychiatric or personality disorders could leave factually incorrect or malicious comments about them and harm their reputation:

You will have everyone, including people with severe psychiatric illness [leaving feedback on online patient feedback websites]...so I think it’s [online patient feedback] potentially quite [a] dangerous tool.

P19

Also, a few participants (n=3) felt that even if GPs could respond online to such allegations, it would not be appropriate for them to respond online.

Theme 3: Transparency Versus Confidentiality

Eight participants agreed that patient reviews left online will seemingly help to increase transparency of care and improve the quality of care, and they were not concerned about the feedback being online and being so transparent as long as there was a “proper system” in place for online patient feedback:

It worries me if it’s [online patient feedback] not a proper system.

P18

By “proper system,” participants meant that the website was well regulated and validated. The website could verify, for example, that the patient leaving the feedback was an actual patient of that particular GP or that the patient did not have a malicious agenda. However, 8 participants were concerned about the platform being “too open” (P11) and in public due to the possibility of people making false allegations, and its damaging impact on the reputation and career of a GP and a GP’s own personal confidentiality:

If it’s [feedback] in public, particularly if I felt it was untrue...if you got y’know someone made an allegation...if that happened to a doctor it could destroy their career, and their self-esteem, and I just think that’s not fair on doctors.

P1

A few participants felt that these types of websites would fuel what they called the “me” culture and turn the NHS into a “customer service industry” and were concerned that it could lead to patients in the future thinking it was perfectly fine to leave feedback about physicians on social media, where according to them, it is impossible to validate or regulate the feedback. Another participant went on to explain that, particularly in health care, what a patient wants is not always what is best for them:

There can be a difference certainly between what people need and what people want, and if people don’t get what they want, often they can feedback negatively about that, even though actually the doctor or the medical provider or whatever who is looking after them, has done exactly the right thing.

P2

In addition to worrying about the confidentiality of GPs themselves, a few participants were also concerned that the transparent nature of the feedback meant that a patient’s own confidentiality may be at risk because they may feel the need to disclose personal health information about themselves on a public platform. Some participants (n=3) were also concerned that GPs are unable to respond to patient reviews online due to the possibility of violating doctor-patient confidentiality because they may need to disclose health information about the patient in their response.

Theme 4: Anonymity and its Impact on Validity and Usefulness of the Feedback

All participants (N=20) were aware that the feedback left on NHS Choices was left anonymously by patients. Some participants raised concerns that because the feedback was left anonymously, they would not know which consultation it referred to and, therefore, could not respond to the feedback nor make real use of it for improvement. Others (n=5) felt that the anonymous nature of the feedback meant they would not know if it was an actual patient from the practice that had left the feedback and questioned whether such feedback is even valid:

Again, if it is anonymous, then, with any feedback really, you really don’t know, is it somebody from this practice, or somebody, well it could be anybody really leaving a feedback there [on online patient feedback websites].

P10

Participants were then asked specifically if the feedback would be more useful to them if it was not left anonymously. Seven participants said that it would be more useful to them if feedback was left with the patient's real name so that they can then look up the consultation and see what went wrong:

If you had their name there, you could obviously understand where this is coming from, and then you can think about it or go back on it, and make ways of improving yourself during your consultation skills. But if it is very anonymous...out of seeing 40 patients in a day, 200 in a week, which one are we talking about, in terms of who?

P10

However, the remaining 13 participants disagreed, commenting that it would not be fair on patients to give their real name because, according to them, it will affect the doctor-patient relationship and patients will not leave feedback online if they cannot leave it anonymously. One participant appeared to suggest a solution that patients should leave their NHS number when they leave feedback to verify that they are a patient registered at that particular practice. Another participant raised the question that despite patients not naming themselves when leaving feedback online, would patients really remain anonymous because sometimes it was easy to identify a particular patient from an anonymous online comment.

Theme 5: Negative Impact on General Practitioners and Their Practice

In addition to the threat of defamation discussed previously and its impact on GP reputation and career, 2 participants were also concerned that negative feedback online could affect the self-confidence and self-esteem of GPs, which would in turn affect their practice, especially those GPs who are early in their career:

It [online patient feedback] will affect people in their early career a lot more, and could break their confidence and make them insular. Is that what you really want to be doing to your future doctors?

P11

Some participants also felt that people will start judging GPs based on online reviews instead of their own experience and this could also affect not only the doctor-patient relationship, but also their practice. Furthermore, participants raised concerns that due to the possibility of negative reviews going online (whether true or false), future practice could end up being defensive and it would be impossible to practice properly:

I don’t know how on earth we are going to have a decent relationship...doctors have become so defensive already...just to make sure they don’t get things online, or do you want them to actually do right for you...give you good care in the right manner in the right timeframe, in a manner which is satisfactory to, or do want them to just do things because they are so scared of litigation of online feedback.

P11

Other GPs raised concerns that it could become embarrassing for them if their practice became public and turned into a “competition” and this could impact patient care too. One participant was particularly concerned about the negative impact online patient feedback could have on her family:

I suppose it’s just the fact that something that’s online...you think about your family and other people, close to you nearest and dearest, sort of looking at things and getting upset on your behalf as well.

P12

Theme 6: Patients Cannot Judge the Professional Competence of a General Practitioner

Some of the participants who were not in favor of online patient feedback argued that the General Medical Council was already regulating them, so there was no need for patients attempting to “regulate” them online and, in fact, how can patients judge whether a GP was competent or not?

Can you really say a patient has that ability to say whether you are underperforming or not?...so the people that are doing appraisal and revalidation are also GPs, they know what you should be doing. I think they should police it, as opposed to patients.

P14

Theme 7: Little Evidence to Prove Usefulness of Online Patient Feedback to Patients or General Practitioners

Two participants argued that there was no evidence currently to prove the usefulness of online patient feedback to patients or GPs:

I think some things with Government policy or in the NHS policy are brought in without having any evidence of benefit, sometimes people jump at the chance “oh we will do this” and they don’t think why.

P1

Furthermore, a few participants argued that existing methods of collecting patient feedback, such as in-house questionnaires, were perfectly adequate and gave more useful data. However, when asked separately about offline feedback, more than half of the participants commented that they did not collect “useful” data.

Discussion

To our knowledge, this study is the first study that explores GPs’ concerns about online patient feedback. The study’s findings suggest that GPs have reservations about online patient feedback and question online patient feedback’s validity, value, usability, and its transparent nature, and are worried about how this will impact them, their practice, and their patients.

Validity and Value of Online Patient Feedback

In this study, GPs were concerned—among other things—about the bias of online patient feedback, both from the user and data perspective. The concern of potential bias due to the age of patients using online patient feedback (in favor of younger patients) has also been raised in literature by some [39,57-59] and some studies appear to support it [6,31,33,36]. However, it could be argued that even patient satisfaction results that are offline are influenced by age, education, and health status [60]. Furthermore, Greaves et al [5] argue that although there may be risks in using ratings from a small group of self-selecting patients, according to them it is outweighed by its positives, mainly that online patient feedback is low cost and has the ability to detect episodes of poor care that a traditional survey may miss. However, as some participants highlighted in this study, this does not address the question of whether a rating of a particular GP or a GP practice can truly be representative, valid, and fair if only the younger and middle-aged patients are leaving ratings or feedback. This is especially crucial for those practices and GPs that serve a largely elderly population.

Furthermore, other participants argued that online patient feedback is mainly negative opinion and will become a channel for disgruntled patients. This sentiment has been raised in opinion articles [37,39] and literature [29,61], but has been counteracted by the argument that many studies (including [6,19,20,22,29,34,62-66]) have found that the majority of feedback left on online physician review websites is actually positive [28]. However, Greaves et al [6] found that the recommendation level of GP and practices in England for the same period was 64% online and 82% in patient surveys. This, they suggest, does indicate that there may be a selection bias in online patient feedback toward less satisfied patients versus when patients are selected randomly and this appears to suggest that the concerns raised by participants in this study may be valid. Furthermore, Merrell et al [61] argue that the abundance of positive reviews cannot negate the impact of negative ones because negative ones, however few, can have long-lasting ramifications as a few GPs in this study highlighted. This is also supported by findings from a study by Hanauer et al [67] who found that parents who are exposed to a positive recommendation of a physician from a neighbor are less likely to choose that physician for their child if they were then exposed to negative reviews about that same physician online. However, Adams [68] found that patient reviews online are not always inherently positive or negative; rather, they contain a mixture of positive and negative comments as well as references to and comparisons with previous health care experiences.

Another argument put forward by GPs in this study was that patients cannot judge the professional competence of a GP; therefore, how can online patient feedback be a true representation of their practice? The concern of whether a patient can adequately judge quality of care received was also raised by Lagu and Lindenauer [58]. It would be useful to explore in future research whether patients are aware that patient-led ratings may be based primarily on the bedside manner of a GP, according to some GPs in this study, and not necessarily on the clinical competence of a GP.

The GPs interviewed were also concerned that feedback left online for them are too few in number and therefore not representative of their performance. This appears to be supported by a study on the NHS Choices feedback website, which found that only 61% of GP practices in England had been reviewed and the number of ratings left per practice was variable with an average of only 2 ratings per practice [6]. However, this study by Greaves et al [6] explored data from more than 5 years ago (between October 2009 and December 2010) and more up-to-date analysis of such websites is required to truly understand the current state because usage may have changed. Taking into account that reviews on the NHS website only correspond to 0.005% of all GP consultations [6] and that studies from the United States [29,66,69,70], Germany [34,71,72], and Australia [27] all indicate that less than 30% of physicians have been rated (and even those that have been rated have on average less than 4 ratings each), the assertions raised by the participants in this study may be true and valid, and need to be addressed by online patient feedback platform providers.

Strech [40] suggests that the solution to this may be that ratings should not be made available until they reach a certain baseline number (eg, 5-10). Although individual pieces of feedback could be displayed before a baseline number is reached, the overall star rating, for example, should not be shown until there are a reasonable number of ratings left for a practice or GP. If the NHS and other online patient feedback website providers want GPs to take these reviews seriously and for the ratings per practice to be “valid” and representative (so that patients can make an accurate “choice”), it needs to do more to get patients to leave reviews (see Textbox 1 for a list of recommendations for online patient feedback providers based on this study). Although the overall rating may not be representative of the quality of care provided by GPs at a GP practice, this does not mean that the individual patient feedback left, however few, may not be useful to GPs and practices to use to make changes and identify opportunities for improvement. This suggests that even if some feedback providers may choose not to publish reviews until a certain baseline has been reached, the unpublished reviews could be sent to GPs to review and use for improvement.

Recommendations for online patient feedback website providers based on findings from this study.

Based on the findings from this study with GPs, recommendations for online patient feedback website providers in England are as follows:

1. Promote online patient feedback more among GPs and patients

Promote online patient feedback among GPs and train GPs to use online patient feedback. This will help to reduce misunderstandings about online patient feedback among GPs, which may help to increase usage of online patient feedback by GPs and also by patients. If GPs believe online patient feedback is valid and useful, they are more likely to promote it to their patients and this may be one of the most effective ways to promote the platform with patients. This could be done through training or may even be as simple as creating a document entitled “A Guide to Online Patient Feedback [specific platform name] for GPs” and signpost it well, both online and offline.
Implement a campaign to promote online patient feedback to patients. This will help to increase the number of patients and type of patients leaving feedback and reviews and therefore the feedback left online is less likely to be biased and unrepresentative. This may mean GPs will take it more seriously and patients will be able to make a more valid “choice.” This could be done through traditional marketing routes through GP practices and digital methods, such as social media and TV ads.

2. Convince and reassure GPs about the value of online patient feedback

Outline precisely how feedback left on the website is moderated and regulated, especially in relation to malicious or personal comments about individual GPs.
Outline on the website and in any marketing leaflets what GPs can do with feedback that is left online for them, in particular, how to respond to it and use it for improvement.
Make patients aware that feedback and ratings left by other patients on these online patient feedback websites may be based primarily on bedside manner and that the majority of patients do not have the ability to judge the professional competence of a GP.

3. Consider some changes to the online patient feedback website

To eliminate concerns about patients judging a GP or a practice based on just a handful of reviews, have a larger number of reviews on the website per practice before the overall rating is calculated and shown.
Validate that the patient leaving feedback on the online patient feedback website is registered as a patient at the given practice through, for example, asking the patient for his/her NHS number. The NHS number could be concealed from the practice to protect the identity of the patient.
Allow patients to leave feedback both for individual GPs and for the practice.
Create an aggregated score of results of measures of competence of GP and patient feedback and reviews, left online and offline, instead of the rating being based on just a few reviews left online at the moment. This has been recommended by the Nuffield Trust [73].

The limited number of online reviews for GPs and practices may be partly explained by one participant’s comment that “patients do not even know about online patient feedback.” This appears to be supported by a study that found that only 15% of the 200 participants in one borough of London were aware of the existence of online feedback websites in health care [36]. However, this study was conducted almost 3 years ago and the awareness of online patient feedback among Londoners may have increased. Nevertheless, there is little evidence as to what extent the NHS Choices feedback website is known and used by patients in the United Kingdom. However, in the United States and Germany, recent studies found that approximately a quarter of respondents had used a physician-rating website [24,33]. This may be partly due to the higher usage and popularity of private health care in both the United States and Germany.

Even when reviews or feedback are left for GPs, some GPs in this study were concerned that the feedback or reviews left by patients could be manipulated without the GP or the practice doing anything “illegal” and this could add serious bias to the data and it would question the validity of the overall rating. This concern is similar to concerns raised in literature that ratings could be “gamed” by organizations or individuals and people could leave fake or multiple entries [6,58,74]. Lagu et al [66] analyzed feedback on review websites in the United States and found several reviews they felt had been written by the physician because they contained information only the physician would know. In another study, Kadry et al [63] found some reviews that they believed were acts of sabotage from competing providers.

A few other GPs in this study argued that there was not enough evidence to prove the usefulness of online patient feedback to patients or GPs. Although research into online physician-rating websites has been steadily increasing over the past few years and studies conducted in the United Kingdom, United States, Germany, the Netherlands, and Australia are all adding to the growing literature [4,5,21,29,30], there is currently a huge gap in the literature. For example, further research is needed to determine whether patients believe online patient feedback is “useful” to them to give feedback or to use to choose a health care provider.

Transparency of Online Patient Feedback

The transparent nature of online feedback websites is what has made it so attractive to patients and health policy makers because the understanding is that reviews left online will apparently increase transparency of care and improve the quality of care [63,68]. A few participants in this study appeared to support this view, whereas the majority had concerns about the platform being “too open” (P11) and in public due to the high risk of people making false allegations and its damaging impact on the reputation and career of a GP, a GP’s self-confidence, self-esteem, and personal confidentiality, all of which could affect their professional practice. Concerns about slander have also been raised by critics of online patient feedback platforms, mainly physician representatives such as the British Medical Association [29,38-40]. However, NHS Choices in England claim to have a strict set of regulations that they use to protect physicians and hospitals from content that may damage their reputation [13]. Despite the NHS Choices promising that all “inflammatory remarks” are removed, it is unknown how this is put into practice and to what extent, and also what constitutes “inflammatory.” Owners of such websites need to make this clear to their users [40]. Furthermore, a few participants in this study remarked that although NHS Choices may anonymize the doctor, it was easy for GPs and the public to work out which GP or staff member the comment was directed to; therefore, it does not really give them the anonymity and protection it claims to.

Usability of Online Patient Feedback

Others remarked that due to the GP being anonymous in comments, it was difficult to work out who the comment was for and, therefore, could not be used for improvement. This concern was also raised by McCartney [39] who as a practicing GP felt that it was difficult, if not impossible, for doctors to learn from anonymous comments. One participant explained that in his opinion the difference is related to the size of the practice; where there are fewer GPs in a practice, it is easy to work out who the feedback is for and their reputation could be harmed much more easily. Another participant felt that harming of reputation was happening offline too, so it made no difference whether it was online or offline. However, others remarked that being online was “too public” and hundreds and thousands of people could have access to it. Some physicians have gone as far as getting a court order to remove an online review according to Kadry et al [63], but they argue that it is very difficult to defend against online misinformation and defamation. Further research is required to determine how patients feel about remaining anonymous and naming their GP online when leaving feedback about their GP online and whether remaining anonymous and naming their GP are key criteria for them to leave feedback on online patient feedback websites.

Conclusions and Recommendations

The majority of GPs interviewed in this study had concerns and reservations about online patient feedback because they felt that online patient feedback was not an accurate representation of their performance due to user bias and data bias. They were also worried about the impact this could have on them, on their practice, and their patients, who may use these “questionable online ratings” to make an “invalid choice” of which health care provider to use. GPs in this study also felt that due to the transparent nature of the feedback online and what they perceive to be lack of regulation, there is a high risk of false allegations being left about them, which could have an impact on them personally, on their family, on their professional practice (more defensive medicine), and on their relationship with their patients. Other GPs questioned the usefulness of the online feedback if the feedback is left anonymously, but acknowledged the benefits to patients of leaving feedback anonymously. A few participants also argued that there was no current evidence to prove online patient feedback’s usefulness to GPs or patients.

Our findings suggest that most concerns raised by GPs may be valid and need to be addressed by online patient feedback providers and other online patient feedback stakeholders. If the NHS and other online patient feedback website providers want GPs to take these reviews seriously, for example, and for the ratings per practice to be valid and representative (so that patients can make an accurate “choice”), they need to do more to get patients to leave reviews. Promoting online patient feedback among GPs and reassuring them of the safety and usefulness of such platforms may also mean GPs are more likely to use online patient feedback for their own professional development and encourage their patients to leave feedback on online patient feedback websites. Other recommendations for online patient feedback website providers based on findings from this study can be found in Textbox 1.

Limitations of the Study

One of the aims of this descriptive study was to explore GPs’ concerns about online patient feedback and the qualitative findings from this study were not intended to be representative of all GPs in England. We do acknowledge that the sample size for this study was small (N=20) and because 60% of participants were between the ages of 30 and 34 years, there may have been a sample bias toward more technology-savvy GPs. However, we found little difference in Internet usage of all the different-aged participants in our sample. We did attempt to recruit participants randomly to get GPs from different age groups and backgrounds, and more than a quarter of our participants were recruited using probability sampling.

Despite our findings not being representative of all GPs in England and this paper being limited to narrating GPs’ negative attitudes to online patient feedback only, the findings highlight key concerns related to online patient feedback from GPs’ perspective and place them into the context of existing literature and viewpoints. This helped form recommendations for feedback providers and can help inform further research in this area.

Acknowledgments

This research was funded by the Engineering and Physical Sciences Research Council (EPSRC) under the Participation in Healthcare Environment Engineering Programme (#EP/H022031/1). Thank you to the 20 GPs who gave their time to contribute their experiences and viewpoints in interviews. A special note of thanks to those who assisted with recruiting GPs to the study, in particular, Dr Imran Satia, Dr Hamed Khan, and the South Midlands PCRN team.

Abbreviations

GP: general practitioner

Multimedia Appendix 1

A copy of the complete topic guide used for the interviews in this study.

jmir_v17i12e276_app1.pdf^{(24.3KB, pdf)}

Multimedia Appendix 2

A copy of the vignettes used during interviews with the GPs.

jmir_v17i12e276_app2.pdf^{(132.8KB, pdf)}

Footnotes

Conflicts of Interest: None declared.

References

1.Schuckert M, Liu X, Law R. Insights into suspicious online ratings: direct evidence from TripAdvisor. Asia Pacific Journal of Tourism Research. 2015 Apr 21;:1–14. doi: 10.1080/10941665.2015.1029954. http://www.tandfonline.com/doi/full/10.1080/10941665.2015.1029954 . [DOI] [Google Scholar]
2.Li H, Ye Q, Law R. Determinants of customer satisfaction in the hotel industry: an application of online review analysis. Asia Pacific Journal of Tourism Research. 2013 Oct 25;18(7):784–802. doi: 10.1080/10941665.2012.708351. [DOI] [Google Scholar]
3.Filieri R. What makes online reviews helpful? A diagnosticity-adoption framework to explain informational and normative influences in e-WOM. Journal of Business Research. 2015 Jun;68(6):1261–1270. doi: 10.1016/j.jbusres.2014.11.006. [DOI] [Google Scholar]
4.Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res. 2013;15(11):e239. doi: 10.2196/jmir.2721. http://www.jmir.org/2013/11/e239/ v15i11e239 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Greaves F, Pape U, King D, Darzi A, Majeed A, Wachter R, Millett C. Associations between Internet-based patient ratings and conventional surveys of patient experience in the English NHS: an observational study. BMJ Qual Saf. 2012 Jul;21(7):600–5. doi: 10.1136/bmjqs-2012-000906.bmjqs-2012-000906 [DOI] [PubMed] [Google Scholar]
6.Greaves F, Pape U, Lee H, Smith D, Darzi A, Majeed A, Millett C. Patients' ratings of family physician practices on the internet: usage and associations with conventional measures of quality in the English National Health Service. J Med Internet Res. 2012;14(5):e146. doi: 10.2196/jmir.2280. http://www.jmir.org/2012/5/e146/ v14i5e146 [DOI] [PMC free article] [PubMed] [Google Scholar]
7.Trigg L. Patients' opinions of health care providers for supporting choice and quality improvement. J Health Serv Res Policy. 2011 Apr;16(2):102–7. doi: 10.1258/jhsrp.2010.010010.jhsrp.2010.010010 [DOI] [PubMed] [Google Scholar]
8.Hughes S, Cohen D. Can online consumers contribute to drug knowledge? A mixed-methods comparison of consumer-generated and professionally controlled psychotropic medication information on the internet. J Med Internet Res. 2011;13(3):e53. doi: 10.2196/jmir.1716. http://www.jmir.org/2011/3/e53/ v13i3e53 [DOI] [PMC free article] [PubMed] [Google Scholar]
9.Greaves F, Millett C, Nuki P. England's Experience incorporating "anecdotal" reports from consumers into their national reporting system: lessons for the United States of what to do or not to do? Med Care Res Rev. 2014 Oct;71(5 Suppl):65S–80S. doi: 10.1177/1077558714535470.1077558714535470 [DOI] [PubMed] [Google Scholar]
10.Fotaki M. What Market-Based Patient Choice Can’t Do for the NHS: The Theory and Evidence of How Choice Works in Health Care. London: Centre for Health and the Public Interest; 2014. Mar, [2015-11-07]. http://chpi.org.uk/wp-content/uploads/2014/03/What-market-based-patient-choice-cant-do-for-the-NHS-CHPI.pdf . [Google Scholar]
11.House of Commons Library. 2015. Feb 03, [2015-11-30]. General practice in England today http://www.parliament.uk/mps-lords-and-offices/offices/commons/commonslibrary/
12.King D, Zaman S, Zaman S, Kahlon G, Naik A, Jessel A, Nanavati N, Shah A, Cox B, Darzi A. Identifying quality indicators used by patients to choose secondary health care providers: a mixed methods approach. JMIR Mhealth Uhealth. 2015;3(2):e65. doi: 10.2196/mhealth.3808. http://mhealth.jmir.org/2015/2/e65/ v3i2e65 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.NHS Choices. 2014. Sep 18, [2015-07-30]. Our terms, conditions and policies: comments policy http://www.nhs.uk/aboutNHSChoices/aboutnhschoices/termsandconditions/Pages/commentspolicy.aspx .
14.iWantGreatCare. [2015-10-16]. https://www.iwantgreatcare.org/
15.Greaves F, Millett C. Consistently increasing numbers of online ratings of healthcare in England. J Med Internet Res. 2012;14(3):e94. doi: 10.2196/jmir.2157. http://www.jmir.org/2012/3/e94/ v14i3e94 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Greaves F, Laverty A, Ramirez-Cano D, Pulman S, Moilanen K, Darzi A, Millett C. Analysis of patients' comments about hospitals in the English NHS via Twitter, and comparison with patient surveys. BMJ Quality & Safety. 2014 Mar 17;23(4):348–348. doi: 10.1136/bmjqs-2014-002893.5. http://qualitysafety.bmj.com/lookup/doi/10.1136/bmjqs-2014-002893.5 . [DOI] [Google Scholar]
17.Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res. 2013;15(11):e239. doi: 10.2196/jmir.2721. http://www.jmir.org/2013/11/e239/ v15i11e239 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Greaves F, Pape U, King D, Darzi A, Majeed A, Wachter RM, Millett C. Associations between Web-based patient ratings and objective measures of hospital quality. Arch Intern Med. 2012 Mar 12;172(5):435–6. doi: 10.1001/archinternmed.2011.1675.archinternmed.2011.1675 [DOI] [PubMed] [Google Scholar]
19.Lagu T, Goff SL, Hannon NS, Shatz A, Lindenauer PK. A mixed-methods analysis of patient reviews of hospital care in England: implications for public reporting of health care quality data in the United States. Jt Comm J Qual Patient Saf. 2013 Jan;39(1):7–15. doi: 10.1016/s1553-7250(13)39003-5. [DOI] [PubMed] [Google Scholar]
20.López A, Detz A, Ratanawongsa N, Sarkar U. What patients say about their doctors online: a qualitative content analysis. J Gen Intern Med. 2012 Jun;27(6):685–92. doi: 10.1007/s11606-011-1958-4. http://europepmc.org/abstract/MED/22215270 . [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Bardach N, Asteria-Peñaloza R, Boscardin W, Dudley R. The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Qual Saf. 2013 Mar;22(3):194–202. doi: 10.1136/bmjqs-2012-001360. http://europepmc.org/abstract/MED/23178860 .bmjqs-2012-001360 [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Emmert M, Meier F, Heider A, Dürr C, Sander U. What do patients say about their physicians? an analysis of 3000 narrative comments posted on a German physician rating website. Health Policy. 2014 Oct;118(1):66–73. doi: 10.1016/j.healthpol.2014.04.015.S0168-8510(14)00114-6 [DOI] [PubMed] [Google Scholar]
23.Emmert M, Sander U, Esslinger AS, Maryschok M, Schöffski O. Public reporting in Germany: the content of physician rating websites. Methods Inf Med. 2012;51(2):112–20. doi: 10.3414/ME11-01-0045.11-01-0045 [DOI] [PubMed] [Google Scholar]
24.Emmert M, Meier F, Pisch F, Sander U. Physician choice making and characteristics associated with using physician-rating websites: cross-sectional study. J Med Internet Res. 2013;15(8):e187. doi: 10.2196/jmir.2702. http://www.jmir.org/2013/8/e187/ v15i8e187 [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Emmert M, Halling F, Meier F. Evaluations of dentists on a German physician rating Website: an analysis of the ratings. J Med Internet Res. 2015;17(1):e15. doi: 10.2196/jmir.3830. http://www.jmir.org/2015/1/e15/ v17i1e15 [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Emmert M, Sander U, Pisch F. Eight questions about physician-rating websites: a systematic review. J Med Internet Res. 2013;15(2):e24. doi: 10.2196/jmir.2360. http://www.jmir.org/2013/2/e24/ v15i2e24 [DOI] [PMC free article] [PubMed] [Google Scholar]
27.Atkinson S. Current status of online rating of Australian doctors. Aust J Prim Health. 2014;20(3):222–3. doi: 10.1071/PY14056.PY14056 [DOI] [PubMed] [Google Scholar]
28.Verhoef L, Van de Belt TH, Engelen LJ, Schoonhoven L, Kool R. Social media and rating sites as tools to understanding quality of care: a scoping review. J Med Internet Res. 2014;16(2):e56. doi: 10.2196/jmir.3024. http://www.jmir.org/2014/2/e56/ v16i2e56 [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Gao G, McCullough J, Agarwal R, Jha A. A changing landscape of physician quality reporting: analysis of patients' online ratings of their physicians over a 5-year period. J Med Internet Res. 2012;14(1):e38. doi: 10.2196/jmir.2003. http://www.jmir.org/2012/1/e38/ v14i1e38 [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Segal J, Sacopulos M, Sheets V, Thurston I, Brooks K, Puccia R. Online doctor reviews: do they track surgeon volume, a proxy for quality of care? J Med Internet Res. 2012;14(2):e50. doi: 10.2196/jmir.2005. http://www.jmir.org/2012/2/e50/ v14i2e50 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Bidmon S, Terlutter R, Röttl J. What explains usage of mobile physician-rating apps? Results from a web-based questionnaire. J Med Internet Res. 2014;16(6):e148. doi: 10.2196/jmir.3122. http://www.jmir.org/2014/6/e148/ v16i6e148 [DOI] [PMC free article] [PubMed] [Google Scholar]
32.Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. JAMA. 2014 Feb 19;311(7):734–5. doi: 10.1001/jama.2013.283194.1829975 [DOI] [PubMed] [Google Scholar]
33.Terlutter R, Bidmon S, Röttl J. Who uses physician-rating websites? Differences in sociodemographic variables, psychographic variables, and health status of users and nonusers of physician-rating websites. J Med Internet Res. 2014;16(3):e97. doi: 10.2196/jmir.3145. http://www.jmir.org/2014/3/e97/ v16i3e97 [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Emmert M, Meier F. An analysis of online evaluations on a physician rating website: evidence from a German public reporting instrument. J Med Internet Res. 2013;15(8):e157. doi: 10.2196/jmir.2655. http://www.jmir.org/2013/8/e157/ v15i8e157 [DOI] [PMC free article] [PubMed] [Google Scholar]
35.The Tavistock Institute. [2012-01-23]. Potential benefits of web-based feedback for improving health service quality: Lessons emerging from the evaluation of Patient Opinion website Internet http://www.tavinstitute.org/lectures_and_presentations/powerpoint/potential-benefits-of-web-based-feedback-for-improving-health-service-quality-lessons-emerging-from-the-evaluation-of-patient-opinion-website/
36.Galizzi MM, Miraldo M, Stavropoulou C, Desai M, Jayatunga W, Joshi M, Parikh S. Who is more likely to use doctor-rating websites, and why? A cross-sectional study in London. BMJ Open. 2012;2(6):N/A–N/A. doi: 10.1136/bmjopen-2012-001493. http://bmjopen.bmj.com/cgi/pmidlookup?view=long&pmid=23148340 .bmjopen-2012-001493 [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Boffey D. The Guardian. 2011. Dec 03, [2015-11-07]. GPs’ fury over “poison pen” online tirades http://www.theguardian.com/society/2011/dec/03/gps-poison-pen-online .
38.Todd R. Digital Health. 2012. Mar 03, [2015-11-07]. GPs give low rating to Choices comments http://www.digitalhealth.net/news/27415/gps-give-low-rating-to-choices-comments .
39.McCartney M. Will doctor rating sites improve the quality of care? No. BMJ. 2009;338:b1033. doi: 10.1136/bmj.b1033. [DOI] [PubMed] [Google Scholar]
40.Strech D. Ethical principles for physician rating sites. J Med Internet Res. 2011;13(4):e113. doi: 10.2196/jmir.1899. http://www.jmir.org/2011/4/e113/ v13i4e113 [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Bryman A. Social Research Methods. Oxford: Oxford University Press; 2008. [Google Scholar]
42.Matthews B, Ross L. Research Methods: A Practical Guide for the Social Sciences. Harlow: Pearson Education Limited; 2010. [Google Scholar]
43.Tracy S. Qualitative Research Methods: Collecting Evidence, Crafting Analysis, Communicating Impact. Chichester: Wiley-Blackwell; 2013. [Google Scholar]
44.Barter C, Renold E. 'I wanna tell you a story': exploring the application of vignettes in qualitative research with children and young people. International Journal of Social Research Methodology. 2000 Jan;3(4):307–323. doi: 10.1080/13645570050178594. http://www.tandfonline.com/doi/abs/10.1080/13645570050178594 . [DOI] [Google Scholar]
45.Hummers-Pradier E, Scheidt-Nave C, Martin H, Heinemann S, Kochen MM, Himmel W. Simply no time? Barriers to GPs' participation in primary health care research. Fam Pract. 2008 Apr;25(2):105–12. doi: 10.1093/fampra/cmn015. http://fampra.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=18417465 .cmn015 [DOI] [PubMed] [Google Scholar]
46.Williamson M, Pirkis J, Pfaff J, Tyson O, Sim M, Kerse N, Lautenschlager NT, Stocks NP, Almeida OP. Recruiting and retaining GPs and patients in intervention studies: the DEPS-GP project as a case study. BMC Med Res Methodol. 2007;7:42. doi: 10.1186/1471-2288-7-42. http://www.biomedcentral.com/1471-2288/7/42 .1471-2288-7-42 [DOI] [PMC free article] [PubMed] [Google Scholar]
47.Shelton BJ, Wofford JL, Gosselink CA, McClatchey MW, Brekke K, Conry C, Wolfe P, Cohen SJ. Recruitment and retention of physicians for primary care research. J Community Health. 2002 Apr;27(2):79–89. doi: 10.1023/a:1014598332211. [DOI] [PubMed] [Google Scholar]
48.Wetzel D, Himmel W, Heidenreich R, Hummers-Pradier E, Kochen M, Rogausch A, Sigle J, Boeckmann H, Kuehnel S, Niebling W, Scheidt-Nave C. Participation in a quality of care study and consequences for generalizability of general practice research. Fam Pract. 2005 Aug;22(4):458–64. doi: 10.1093/fampra/cmi022. http://fampra.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=15814583 .cmi022 [DOI] [PubMed] [Google Scholar]
49.Templeton L, Deehan A, Taylor C, Drummond C, Strang J. Surveying general practitioners: does a low response rate matter? Br J Gen Pract. 1997 Feb;47(415):91–4. http://bjgp.org/cgi/pmidlookup?view=long&pmid=9101692 . [PMC free article] [PubMed] [Google Scholar]
50.Barclay S, Todd C, Finlay I, Grande G, Wyatt P. Not another questionnaire! Maximizing the response rate, predicting non-response and assessing non-response bias in postal questionnaire studies of GPs. Fam Pract. 2002 Feb;19(1):105–11. doi: 10.1093/fampra/19.1.105. http://fampra.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=11818359 . [DOI] [PubMed] [Google Scholar]
51.Asch S, Connor S, Hamilton E, Fox S. Problems in recruiting community-based physicians for health services research. J Gen Intern Med. 2000 Aug;15(8):591–9. doi: 10.1046/j.1525-1497.2000.02329.x. http://onlinelibrary.wiley.com/resolve/openurl?genre=article&sid=nlm:pubmed&issn=0884-8734&date=2000&volume=15&issue=8&spage=591 .jgi02329 [DOI] [PMC free article] [PubMed] [Google Scholar]
52.O'Reilly M, Parker N. 'Unsatisfactory Saturation': a critical exploration of the notion of saturated sample sizes in qualitative research. Qualitative Research. 2012 May 17;13(2):190–197. doi: 10.1177/1468794112446106. http://qrj.sagepub.com/cgi/doi/10.1177/1468794112446106 . [DOI] [Google Scholar]
53.Ritchie J, Lewis J, Nicholls CM, Ormston R. Qualitative Research Practice: A Guide for Social Science Students and Researchers. Thousand Oaks, CA: Sage Publications; 2013. [Google Scholar]
54.Dixon-Woods M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011;9:39. doi: 10.1186/1741-7015-9-39. http://www.biomedcentral.com/1741-7015/9/39 .1741-7015-9-39 [DOI] [PMC free article] [PubMed] [Google Scholar]
55.Smith J, Firth J. Qualitative data analysis: the framework approach. Nurse Res. 2011;18(2):52–62. doi: 10.7748/nr2011.01.18.2.52.c8284. [DOI] [PubMed] [Google Scholar]
56.Gale N, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117. doi: 10.1186/1471-2288-13-117. http://www.biomedcentral.com/1471-2288/13/117 .1471-2288-13-117 [DOI] [PMC free article] [PubMed] [Google Scholar]
57.Rozenblum R, Bates DW. Patient-centred healthcare, social media and the internet: the perfect storm? BMJ Qual Saf. 2013 Mar;22(3):183–6. doi: 10.1136/bmjqs-2012-001744.bmjqs-2012-001744 [DOI] [PubMed] [Google Scholar]
58.Lagu T, Lindenauer PK. Putting the public back in public reporting of health care quality. JAMA. 2010 Oct 20;304(15):1711–2. doi: 10.1001/jama.2010.1499.304/15/1711 [DOI] [PubMed] [Google Scholar]
59.Tritter JQ, McCallum A. The snakes and ladders of user involvement: moving beyond Arnstein. Health Policy. 2006 Apr;76(2):156–68. doi: 10.1016/j.healthpol.2005.05.008.S0168-8510(05)00131-4 [DOI] [PubMed] [Google Scholar]
60.Hekkert KD, Cihangir S, Kleefstra SM, van den Berg B, Kool RB. Patient satisfaction revisited: a multilevel approach. Soc Sci Med. 2009 Jul;69(1):68–75. doi: 10.1016/j.socscimed.2009.04.016.S0277-9536(09)00247-0 [DOI] [PubMed] [Google Scholar]
61.Merrell JG, Levy BH, Johnson DA. Patient assessments and online ratings of quality care: a "wake-up call" for providers. Am J Gastroenterol. 2013 Nov;108(11):1676–85. doi: 10.1038/ajg.2013.112.ajg2013112 [DOI] [PubMed] [Google Scholar]
62.Black E, Thompson L, Saliba H, Dawson K, Black NM. An analysis of healthcare providers' online ratings. Inform Prim Care. 2009;17(4):249–53. doi: 10.14236/jhi.v17i4.744. [DOI] [PubMed] [Google Scholar]
63.Kadry B, Chu L, Kadry B, Gammas D, Macario A. Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res. 2011;13(4):e95. doi: 10.2196/jmir.1960. http://www.jmir.org/2011/4/e95/ v13i4e95 [DOI] [PMC free article] [PubMed] [Google Scholar]
64.Detz A, López A, Sarkar U. Long-term doctor-patient relationships: patient perspective from online reviews. J Med Internet Res. 2013;15(7):e131. doi: 10.2196/jmir.2552. http://www.jmir.org/2013/7/e131/ v15i7e131 [DOI] [PMC free article] [PubMed] [Google Scholar]
65.Alemi F, Torii M, Clementz L, Aron DC. Feasibility of real-time satisfaction surveys through automated analysis of patients' unstructured comments and sentiments. Qual Manag Health Care. 2012;21(1):9–19. doi: 10.1097/QMH.0b013e3182417fc4.00019514-201201000-00004 [DOI] [PubMed] [Google Scholar]
66.Lagu T, Hannon N, Rothberg M, Lindenauer P. Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med. 2010 Sep;25(9):942–6. doi: 10.1007/s11606-010-1383-0. http://europepmc.org/abstract/MED/20464523 . [DOI] [PMC free article] [PubMed] [Google Scholar]
67.Hanauer DA, Zheng Kai, Singer Dianne C, Gebremariam Achamyeleh, Davis Matthew M. Parental awareness and use of online physician rating sites. Pediatrics. 2014 Oct;134(4):e966–75. doi: 10.1542/peds.2014-0681. http://pediatrics.aappublications.org/cgi/pmidlookup?view=long&pmid=25246629 .peds.2014-0681 [DOI] [PubMed] [Google Scholar]
68.Adams SA. Sourcing the crowd for health services improvement: rhe reflexive patient and "share-your-experience" websites. Soc Sci Med. 2011 Apr;72(7):1069–76. doi: 10.1016/j.socscimed.2011.02.001.S0277-9536(11)00075-X [DOI] [PubMed] [Google Scholar]
69.Mostaghimi A, Crotty B, Landon B. The availability and nature of physician information on the internet. J Gen Intern Med. 2010 Nov;25(11):1152–6. doi: 10.1007/s11606-010-1425-7. http://europepmc.org/abstract/MED/20544300 . [DOI] [PMC free article] [PubMed] [Google Scholar]
70.Ellimoottil C, Hart A, Greco K, Quek M, Farooq A. Online reviews of 500 urologists. J Urol. 2013 Jun;189(6):2269–73. doi: 10.1016/j.juro.2012.12.013.S0022-5347(12)05837-5 [DOI] [PubMed] [Google Scholar]
71.Emmert M, Maryschok M, Eisenreich S, Schöffski O. Arzt-Bewertungsportale im Internet—Geeignet zur Identifikation guter Arztpraxen? [Websites to assess quality of care—appropriate to identify good physicians?] Gesundheitswesen. 2009 Feb 16;71(04):e18–e27. doi: 10.1055/s-0028-1103288. [DOI] [PubMed] [Google Scholar]
72.Strech D, Reimann S. Deutschsprachige Arztbewertungsportale: Der Status quo ihrer Bewertungskriterien, Bewertungstendenzen und Nutzung [German language physician rating sites: the status quo of evaluation criteria, evaluation tendencies and utilization] Das Gesundheitswesen. 2012;74(8-9):502–503. doi: 10.1055/s-0031-1297254. [DOI] [PubMed] [Google Scholar]
73.The Nuffield Trust . Rating Providers for Quality: A Policy Worth Pursuing? London: The Nuffield Trust; 2013. Mar, [2014-09-01]. http://www.nuffieldtrust.org.uk/sites/files/nuffield/publication/130322_ratings_providers_for_quality_summary.pdf . [Google Scholar]
74.Segal J. The role of the Internet in doctor performance rating. Pain Physician. 2009;12(3):659–64. http://www.painphysicianjournal.com/linkout?issn=1533-3159&vol=12&page=659 . [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia Appendix 1

A copy of the complete topic guide used for the interviews in this study.

jmir_v17i12e276_app1.pdf^{(24.3KB, pdf)}

Multimedia Appendix 2

A copy of the vignettes used during interviews with the GPs.

jmir_v17i12e276_app2.pdf^{(132.8KB, pdf)}

[ref1] 1.Schuckert M, Liu X, Law R. Insights into suspicious online ratings: direct evidence from TripAdvisor. Asia Pacific Journal of Tourism Research. 2015 Apr 21;:1–14. doi: 10.1080/10941665.2015.1029954. http://www.tandfonline.com/doi/full/10.1080/10941665.2015.1029954 . [DOI] [Google Scholar]

[ref2] 2.Li H, Ye Q, Law R. Determinants of customer satisfaction in the hotel industry: an application of online review analysis. Asia Pacific Journal of Tourism Research. 2013 Oct 25;18(7):784–802. doi: 10.1080/10941665.2012.708351. [DOI] [Google Scholar]

[ref3] 3.Filieri R. What makes online reviews helpful? A diagnosticity-adoption framework to explain informational and normative influences in e-WOM. Journal of Business Research. 2015 Jun;68(6):1261–1270. doi: 10.1016/j.jbusres.2014.11.006. [DOI] [Google Scholar]

[ref4] 4.Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res. 2013;15(11):e239. doi: 10.2196/jmir.2721. http://www.jmir.org/2013/11/e239/ v15i11e239 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref5] 5.Greaves F, Pape U, King D, Darzi A, Majeed A, Wachter R, Millett C. Associations between Internet-based patient ratings and conventional surveys of patient experience in the English NHS: an observational study. BMJ Qual Saf. 2012 Jul;21(7):600–5. doi: 10.1136/bmjqs-2012-000906.bmjqs-2012-000906 [DOI] [PubMed] [Google Scholar]

[ref6] 6.Greaves F, Pape U, Lee H, Smith D, Darzi A, Majeed A, Millett C. Patients' ratings of family physician practices on the internet: usage and associations with conventional measures of quality in the English National Health Service. J Med Internet Res. 2012;14(5):e146. doi: 10.2196/jmir.2280. http://www.jmir.org/2012/5/e146/ v14i5e146 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref7] 7.Trigg L. Patients' opinions of health care providers for supporting choice and quality improvement. J Health Serv Res Policy. 2011 Apr;16(2):102–7. doi: 10.1258/jhsrp.2010.010010.jhsrp.2010.010010 [DOI] [PubMed] [Google Scholar]

[ref8] 8.Hughes S, Cohen D. Can online consumers contribute to drug knowledge? A mixed-methods comparison of consumer-generated and professionally controlled psychotropic medication information on the internet. J Med Internet Res. 2011;13(3):e53. doi: 10.2196/jmir.1716. http://www.jmir.org/2011/3/e53/ v13i3e53 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref9] 9.Greaves F, Millett C, Nuki P. England's Experience incorporating "anecdotal" reports from consumers into their national reporting system: lessons for the United States of what to do or not to do? Med Care Res Rev. 2014 Oct;71(5 Suppl):65S–80S. doi: 10.1177/1077558714535470.1077558714535470 [DOI] [PubMed] [Google Scholar]

[ref10] 10.Fotaki M. What Market-Based Patient Choice Can’t Do for the NHS: The Theory and Evidence of How Choice Works in Health Care. London: Centre for Health and the Public Interest; 2014. Mar, [2015-11-07]. http://chpi.org.uk/wp-content/uploads/2014/03/What-market-based-patient-choice-cant-do-for-the-NHS-CHPI.pdf . [Google Scholar]

[ref11] 11.House of Commons Library. 2015. Feb 03, [2015-11-30]. General practice in England today http://www.parliament.uk/mps-lords-and-offices/offices/commons/commonslibrary/

[ref12] 12.King D, Zaman S, Zaman S, Kahlon G, Naik A, Jessel A, Nanavati N, Shah A, Cox B, Darzi A. Identifying quality indicators used by patients to choose secondary health care providers: a mixed methods approach. JMIR Mhealth Uhealth. 2015;3(2):e65. doi: 10.2196/mhealth.3808. http://mhealth.jmir.org/2015/2/e65/ v3i2e65 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref13] 13.NHS Choices. 2014. Sep 18, [2015-07-30]. Our terms, conditions and policies: comments policy http://www.nhs.uk/aboutNHSChoices/aboutnhschoices/termsandconditions/Pages/commentspolicy.aspx .

[ref14] 14.iWantGreatCare. [2015-10-16]. https://www.iwantgreatcare.org/

[ref15] 15.Greaves F, Millett C. Consistently increasing numbers of online ratings of healthcare in England. J Med Internet Res. 2012;14(3):e94. doi: 10.2196/jmir.2157. http://www.jmir.org/2012/3/e94/ v14i3e94 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref16] 16.Greaves F, Laverty A, Ramirez-Cano D, Pulman S, Moilanen K, Darzi A, Millett C. Analysis of patients' comments about hospitals in the English NHS via Twitter, and comparison with patient surveys. BMJ Quality & Safety. 2014 Mar 17;23(4):348–348. doi: 10.1136/bmjqs-2014-002893.5. http://qualitysafety.bmj.com/lookup/doi/10.1136/bmjqs-2014-002893.5 . [DOI] [Google Scholar]

[ref17] 17.Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res. 2013;15(11):e239. doi: 10.2196/jmir.2721. http://www.jmir.org/2013/11/e239/ v15i11e239 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref18] 18.Greaves F, Pape U, King D, Darzi A, Majeed A, Wachter RM, Millett C. Associations between Web-based patient ratings and objective measures of hospital quality. Arch Intern Med. 2012 Mar 12;172(5):435–6. doi: 10.1001/archinternmed.2011.1675.archinternmed.2011.1675 [DOI] [PubMed] [Google Scholar]

[ref19] 19.Lagu T, Goff SL, Hannon NS, Shatz A, Lindenauer PK. A mixed-methods analysis of patient reviews of hospital care in England: implications for public reporting of health care quality data in the United States. Jt Comm J Qual Patient Saf. 2013 Jan;39(1):7–15. doi: 10.1016/s1553-7250(13)39003-5. [DOI] [PubMed] [Google Scholar]

[ref20] 20.López A, Detz A, Ratanawongsa N, Sarkar U. What patients say about their doctors online: a qualitative content analysis. J Gen Intern Med. 2012 Jun;27(6):685–92. doi: 10.1007/s11606-011-1958-4. http://europepmc.org/abstract/MED/22215270 . [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref21] 21.Bardach N, Asteria-Peñaloza R, Boscardin W, Dudley R. The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Qual Saf. 2013 Mar;22(3):194–202. doi: 10.1136/bmjqs-2012-001360. http://europepmc.org/abstract/MED/23178860 .bmjqs-2012-001360 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref22] 22.Emmert M, Meier F, Heider A, Dürr C, Sander U. What do patients say about their physicians? an analysis of 3000 narrative comments posted on a German physician rating website. Health Policy. 2014 Oct;118(1):66–73. doi: 10.1016/j.healthpol.2014.04.015.S0168-8510(14)00114-6 [DOI] [PubMed] [Google Scholar]

[ref23] 23.Emmert M, Sander U, Esslinger AS, Maryschok M, Schöffski O. Public reporting in Germany: the content of physician rating websites. Methods Inf Med. 2012;51(2):112–20. doi: 10.3414/ME11-01-0045.11-01-0045 [DOI] [PubMed] [Google Scholar]

[ref24] 24.Emmert M, Meier F, Pisch F, Sander U. Physician choice making and characteristics associated with using physician-rating websites: cross-sectional study. J Med Internet Res. 2013;15(8):e187. doi: 10.2196/jmir.2702. http://www.jmir.org/2013/8/e187/ v15i8e187 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref25] 25.Emmert M, Halling F, Meier F. Evaluations of dentists on a German physician rating Website: an analysis of the ratings. J Med Internet Res. 2015;17(1):e15. doi: 10.2196/jmir.3830. http://www.jmir.org/2015/1/e15/ v17i1e15 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref26] 26.Emmert M, Sander U, Pisch F. Eight questions about physician-rating websites: a systematic review. J Med Internet Res. 2013;15(2):e24. doi: 10.2196/jmir.2360. http://www.jmir.org/2013/2/e24/ v15i2e24 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref27] 27.Atkinson S. Current status of online rating of Australian doctors. Aust J Prim Health. 2014;20(3):222–3. doi: 10.1071/PY14056.PY14056 [DOI] [PubMed] [Google Scholar]

[ref28] 28.Verhoef L, Van de Belt TH, Engelen LJ, Schoonhoven L, Kool R. Social media and rating sites as tools to understanding quality of care: a scoping review. J Med Internet Res. 2014;16(2):e56. doi: 10.2196/jmir.3024. http://www.jmir.org/2014/2/e56/ v16i2e56 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref29] 29.Gao G, McCullough J, Agarwal R, Jha A. A changing landscape of physician quality reporting: analysis of patients' online ratings of their physicians over a 5-year period. J Med Internet Res. 2012;14(1):e38. doi: 10.2196/jmir.2003. http://www.jmir.org/2012/1/e38/ v14i1e38 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref30] 30.Segal J, Sacopulos M, Sheets V, Thurston I, Brooks K, Puccia R. Online doctor reviews: do they track surgeon volume, a proxy for quality of care? J Med Internet Res. 2012;14(2):e50. doi: 10.2196/jmir.2005. http://www.jmir.org/2012/2/e50/ v14i2e50 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref31] 31.Bidmon S, Terlutter R, Röttl J. What explains usage of mobile physician-rating apps? Results from a web-based questionnaire. J Med Internet Res. 2014;16(6):e148. doi: 10.2196/jmir.3122. http://www.jmir.org/2014/6/e148/ v16i6e148 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref32] 32.Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. JAMA. 2014 Feb 19;311(7):734–5. doi: 10.1001/jama.2013.283194.1829975 [DOI] [PubMed] [Google Scholar]

[ref33] 33.Terlutter R, Bidmon S, Röttl J. Who uses physician-rating websites? Differences in sociodemographic variables, psychographic variables, and health status of users and nonusers of physician-rating websites. J Med Internet Res. 2014;16(3):e97. doi: 10.2196/jmir.3145. http://www.jmir.org/2014/3/e97/ v16i3e97 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref34] 34.Emmert M, Meier F. An analysis of online evaluations on a physician rating website: evidence from a German public reporting instrument. J Med Internet Res. 2013;15(8):e157. doi: 10.2196/jmir.2655. http://www.jmir.org/2013/8/e157/ v15i8e157 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref35] 35.The Tavistock Institute. [2012-01-23]. Potential benefits of web-based feedback for improving health service quality: Lessons emerging from the evaluation of Patient Opinion website Internet http://www.tavinstitute.org/lectures_and_presentations/powerpoint/potential-benefits-of-web-based-feedback-for-improving-health-service-quality-lessons-emerging-from-the-evaluation-of-patient-opinion-website/

[ref36] 36.Galizzi MM, Miraldo M, Stavropoulou C, Desai M, Jayatunga W, Joshi M, Parikh S. Who is more likely to use doctor-rating websites, and why? A cross-sectional study in London. BMJ Open. 2012;2(6):N/A–N/A. doi: 10.1136/bmjopen-2012-001493. http://bmjopen.bmj.com/cgi/pmidlookup?view=long&pmid=23148340 .bmjopen-2012-001493 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref37] 37.Boffey D. The Guardian. 2011. Dec 03, [2015-11-07]. GPs’ fury over “poison pen” online tirades http://www.theguardian.com/society/2011/dec/03/gps-poison-pen-online .

[ref38] 38.Todd R. Digital Health. 2012. Mar 03, [2015-11-07]. GPs give low rating to Choices comments http://www.digitalhealth.net/news/27415/gps-give-low-rating-to-choices-comments .

[ref39] 39.McCartney M. Will doctor rating sites improve the quality of care? No. BMJ. 2009;338:b1033. doi: 10.1136/bmj.b1033. [DOI] [PubMed] [Google Scholar]

[ref40] 40.Strech D. Ethical principles for physician rating sites. J Med Internet Res. 2011;13(4):e113. doi: 10.2196/jmir.1899. http://www.jmir.org/2011/4/e113/ v13i4e113 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref41] 41.Bryman A. Social Research Methods. Oxford: Oxford University Press; 2008. [Google Scholar]

[ref42] 42.Matthews B, Ross L. Research Methods: A Practical Guide for the Social Sciences. Harlow: Pearson Education Limited; 2010. [Google Scholar]

[ref43] 43.Tracy S. Qualitative Research Methods: Collecting Evidence, Crafting Analysis, Communicating Impact. Chichester: Wiley-Blackwell; 2013. [Google Scholar]

[ref44] 44.Barter C, Renold E. 'I wanna tell you a story': exploring the application of vignettes in qualitative research with children and young people. International Journal of Social Research Methodology. 2000 Jan;3(4):307–323. doi: 10.1080/13645570050178594. http://www.tandfonline.com/doi/abs/10.1080/13645570050178594 . [DOI] [Google Scholar]

[ref45] 45.Hummers-Pradier E, Scheidt-Nave C, Martin H, Heinemann S, Kochen MM, Himmel W. Simply no time? Barriers to GPs' participation in primary health care research. Fam Pract. 2008 Apr;25(2):105–12. doi: 10.1093/fampra/cmn015. http://fampra.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=18417465 .cmn015 [DOI] [PubMed] [Google Scholar]

[ref46] 46.Williamson M, Pirkis J, Pfaff J, Tyson O, Sim M, Kerse N, Lautenschlager NT, Stocks NP, Almeida OP. Recruiting and retaining GPs and patients in intervention studies: the DEPS-GP project as a case study. BMC Med Res Methodol. 2007;7:42. doi: 10.1186/1471-2288-7-42. http://www.biomedcentral.com/1471-2288/7/42 .1471-2288-7-42 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref47] 47.Shelton BJ, Wofford JL, Gosselink CA, McClatchey MW, Brekke K, Conry C, Wolfe P, Cohen SJ. Recruitment and retention of physicians for primary care research. J Community Health. 2002 Apr;27(2):79–89. doi: 10.1023/a:1014598332211. [DOI] [PubMed] [Google Scholar]

[ref48] 48.Wetzel D, Himmel W, Heidenreich R, Hummers-Pradier E, Kochen M, Rogausch A, Sigle J, Boeckmann H, Kuehnel S, Niebling W, Scheidt-Nave C. Participation in a quality of care study and consequences for generalizability of general practice research. Fam Pract. 2005 Aug;22(4):458–64. doi: 10.1093/fampra/cmi022. http://fampra.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=15814583 .cmi022 [DOI] [PubMed] [Google Scholar]

[ref49] 49.Templeton L, Deehan A, Taylor C, Drummond C, Strang J. Surveying general practitioners: does a low response rate matter? Br J Gen Pract. 1997 Feb;47(415):91–4. http://bjgp.org/cgi/pmidlookup?view=long&pmid=9101692 . [PMC free article] [PubMed] [Google Scholar]

[ref50] 50.Barclay S, Todd C, Finlay I, Grande G, Wyatt P. Not another questionnaire! Maximizing the response rate, predicting non-response and assessing non-response bias in postal questionnaire studies of GPs. Fam Pract. 2002 Feb;19(1):105–11. doi: 10.1093/fampra/19.1.105. http://fampra.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=11818359 . [DOI] [PubMed] [Google Scholar]

[ref51] 51.Asch S, Connor S, Hamilton E, Fox S. Problems in recruiting community-based physicians for health services research. J Gen Intern Med. 2000 Aug;15(8):591–9. doi: 10.1046/j.1525-1497.2000.02329.x. http://onlinelibrary.wiley.com/resolve/openurl?genre=article&sid=nlm:pubmed&issn=0884-8734&date=2000&volume=15&issue=8&spage=591 .jgi02329 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref52] 52.O'Reilly M, Parker N. 'Unsatisfactory Saturation': a critical exploration of the notion of saturated sample sizes in qualitative research. Qualitative Research. 2012 May 17;13(2):190–197. doi: 10.1177/1468794112446106. http://qrj.sagepub.com/cgi/doi/10.1177/1468794112446106 . [DOI] [Google Scholar]

[ref53] 53.Ritchie J, Lewis J, Nicholls CM, Ormston R. Qualitative Research Practice: A Guide for Social Science Students and Researchers. Thousand Oaks, CA: Sage Publications; 2013. [Google Scholar]

[ref54] 54.Dixon-Woods M. Using framework-based synthesis for conducting reviews of qualitative studies. BMC Med. 2011;9:39. doi: 10.1186/1741-7015-9-39. http://www.biomedcentral.com/1741-7015/9/39 .1741-7015-9-39 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref55] 55.Smith J, Firth J. Qualitative data analysis: the framework approach. Nurse Res. 2011;18(2):52–62. doi: 10.7748/nr2011.01.18.2.52.c8284. [DOI] [PubMed] [Google Scholar]

[ref56] 56.Gale N, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117. doi: 10.1186/1471-2288-13-117. http://www.biomedcentral.com/1471-2288/13/117 .1471-2288-13-117 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref57] 57.Rozenblum R, Bates DW. Patient-centred healthcare, social media and the internet: the perfect storm? BMJ Qual Saf. 2013 Mar;22(3):183–6. doi: 10.1136/bmjqs-2012-001744.bmjqs-2012-001744 [DOI] [PubMed] [Google Scholar]

[ref58] 58.Lagu T, Lindenauer PK. Putting the public back in public reporting of health care quality. JAMA. 2010 Oct 20;304(15):1711–2. doi: 10.1001/jama.2010.1499.304/15/1711 [DOI] [PubMed] [Google Scholar]

[ref59] 59.Tritter JQ, McCallum A. The snakes and ladders of user involvement: moving beyond Arnstein. Health Policy. 2006 Apr;76(2):156–68. doi: 10.1016/j.healthpol.2005.05.008.S0168-8510(05)00131-4 [DOI] [PubMed] [Google Scholar]

[ref60] 60.Hekkert KD, Cihangir S, Kleefstra SM, van den Berg B, Kool RB. Patient satisfaction revisited: a multilevel approach. Soc Sci Med. 2009 Jul;69(1):68–75. doi: 10.1016/j.socscimed.2009.04.016.S0277-9536(09)00247-0 [DOI] [PubMed] [Google Scholar]

[ref61] 61.Merrell JG, Levy BH, Johnson DA. Patient assessments and online ratings of quality care: a "wake-up call" for providers. Am J Gastroenterol. 2013 Nov;108(11):1676–85. doi: 10.1038/ajg.2013.112.ajg2013112 [DOI] [PubMed] [Google Scholar]

[ref62] 62.Black E, Thompson L, Saliba H, Dawson K, Black NM. An analysis of healthcare providers' online ratings. Inform Prim Care. 2009;17(4):249–53. doi: 10.14236/jhi.v17i4.744. [DOI] [PubMed] [Google Scholar]

[ref63] 63.Kadry B, Chu L, Kadry B, Gammas D, Macario A. Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res. 2011;13(4):e95. doi: 10.2196/jmir.1960. http://www.jmir.org/2011/4/e95/ v13i4e95 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref64] 64.Detz A, López A, Sarkar U. Long-term doctor-patient relationships: patient perspective from online reviews. J Med Internet Res. 2013;15(7):e131. doi: 10.2196/jmir.2552. http://www.jmir.org/2013/7/e131/ v15i7e131 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref65] 65.Alemi F, Torii M, Clementz L, Aron DC. Feasibility of real-time satisfaction surveys through automated analysis of patients' unstructured comments and sentiments. Qual Manag Health Care. 2012;21(1):9–19. doi: 10.1097/QMH.0b013e3182417fc4.00019514-201201000-00004 [DOI] [PubMed] [Google Scholar]

[ref66] 66.Lagu T, Hannon N, Rothberg M, Lindenauer P. Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med. 2010 Sep;25(9):942–6. doi: 10.1007/s11606-010-1383-0. http://europepmc.org/abstract/MED/20464523 . [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref67] 67.Hanauer DA, Zheng Kai, Singer Dianne C, Gebremariam Achamyeleh, Davis Matthew M. Parental awareness and use of online physician rating sites. Pediatrics. 2014 Oct;134(4):e966–75. doi: 10.1542/peds.2014-0681. http://pediatrics.aappublications.org/cgi/pmidlookup?view=long&pmid=25246629 .peds.2014-0681 [DOI] [PubMed] [Google Scholar]

[ref68] 68.Adams SA. Sourcing the crowd for health services improvement: rhe reflexive patient and "share-your-experience" websites. Soc Sci Med. 2011 Apr;72(7):1069–76. doi: 10.1016/j.socscimed.2011.02.001.S0277-9536(11)00075-X [DOI] [PubMed] [Google Scholar]

[ref69] 69.Mostaghimi A, Crotty B, Landon B. The availability and nature of physician information on the internet. J Gen Intern Med. 2010 Nov;25(11):1152–6. doi: 10.1007/s11606-010-1425-7. http://europepmc.org/abstract/MED/20544300 . [DOI] [PMC free article] [PubMed] [Google Scholar]

[ref70] 70.Ellimoottil C, Hart A, Greco K, Quek M, Farooq A. Online reviews of 500 urologists. J Urol. 2013 Jun;189(6):2269–73. doi: 10.1016/j.juro.2012.12.013.S0022-5347(12)05837-5 [DOI] [PubMed] [Google Scholar]

[ref71] 71.Emmert M, Maryschok M, Eisenreich S, Schöffski O. Arzt-Bewertungsportale im Internet—Geeignet zur Identifikation guter Arztpraxen? [Websites to assess quality of care—appropriate to identify good physicians?] Gesundheitswesen. 2009 Feb 16;71(04):e18–e27. doi: 10.1055/s-0028-1103288. [DOI] [PubMed] [Google Scholar]

[ref72] 72.Strech D, Reimann S. Deutschsprachige Arztbewertungsportale: Der Status quo ihrer Bewertungskriterien, Bewertungstendenzen und Nutzung [German language physician rating sites: the status quo of evaluation criteria, evaluation tendencies and utilization] Das Gesundheitswesen. 2012;74(8-9):502–503. doi: 10.1055/s-0031-1297254. [DOI] [PubMed] [Google Scholar]

[ref73] 73.The Nuffield Trust . Rating Providers for Quality: A Policy Worth Pursuing? London: The Nuffield Trust; 2013. Mar, [2014-09-01]. http://www.nuffieldtrust.org.uk/sites/files/nuffield/publication/130322_ratings_providers_for_quality_summary.pdf . [Google Scholar]

[ref74] 74.Segal J. The role of the Internet in doctor performance rating. Pain Physician. 2009;12(3):659–64. http://www.painphysicianjournal.com/linkout?issn=1533-3159&vol=12&page=659 . [PubMed] [Google Scholar]

PERMALINK

General Practitioners’ Concerns About Online Patient Feedback: Findings From a Descriptive Exploratory Qualitative Study in England

Salma Patel, BSc (Hons)

Rebecca Cain, BA(Hons), PhD

Kevin Neailey, BSc (Hons), PhD

Lucy Hooberman, MA (Hons) Oxon, FRSA

Abstract

Background

Objective

Methods

Results

Conclusions

Introduction

Figure 1.

Methods

Data Collection

Sampling and Recruitment

Table 1.

Study Interviews and Participants

Table 2.

Data Preparation and Analysis

Results

Usage, Awareness, and Overall Impression of Online Patient Feedback

Concerns About Online Patient Feedback

Figure 2.

Theme 1: Online Patient Feedback is Biased

User Bias

Online Patient Feedback is Mainly Negative Opinion

Online Patient Feedback is Too Small in Number

Reviews Could Be “Gamed”

Theme 2: Risk of False Allegations and Absence of Regulation

Theme 3: Transparency Versus Confidentiality

Theme 4: Anonymity and its Impact on Validity and Usefulness of the Feedback

Theme 5: Negative Impact on General Practitioners and Their Practice

Theme 6: Patients Cannot Judge the Professional Competence of a General Practitioner

Theme 7: Little Evidence to Prove Usefulness of Online Patient Feedback to Patients or General Practitioners

Discussion

Validity and Value of Online Patient Feedback

Recommendations for online patient feedback website providers based on findings from this study.

Transparency of Online Patient Feedback

Usability of Online Patient Feedback

Conclusions and Recommendations

Limitations of the Study

Acknowledgments

Abbreviations

Footnotes

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases