Skip to main content
Journal of Medical Internet Research logoLink to Journal of Medical Internet Research
. 2011 Dec 6;13(4):e113. doi: 10.2196/jmir.1899

Ethical Principles for Physician Rating Sites

Daniel Strech 1,
Editor: Gunther Eysenbach
Reviewed by: Tara Lagu, Erik Black, Paul Wicks, Adrija Datta
PMCID: PMC3278099  PMID: 22146737

Abstract

During the last 5 years, an ethical debate has emerged, often in public media, about the potential positive and negative effects of physician rating sites and whether physician rating sites created by insurance companies or government agencies are ethical in their current states. Due to the lack of direct evidence of physician rating sites’ effects on physicians’ performance, patient outcomes, or the public’s trust in health care, most contributions refer to normative arguments, hypothetical effects, or indirect evidence. This paper aims, first, to structure the ethical debate about the basic concept of physician rating sites: allowing patients to rate, comment, and discuss physicians’ performance, online and visible to everyone. Thus, it provides a more thorough and transparent starting point for further discussion and decision making on physician rating sites: what should physicians and health policy decision makers take into account when discussing the basic concept of physician rating sites and its possible implications on the physician–patient relationship? Second, it discusses where and how the preexisting evidence from the partly related field of public reporting of physician performance can serve as an indicator for specific needs of evaluative research in the field of physician rating sites. This paper defines the ethical principles of patient welfare, patient autonomy, physician welfare, and social justice in the context of physician rating sites. It also outlines basic conditions for a fair decision-making process concerning the implementation and regulation of physician rating sites, namely, transparency, justification, participation, minimization of conflicts of interest, and openness for revision. Besides other issues described in this paper, one trade-off presents a special challenge and will play an important role when deciding about more- or less-restrictive physician rating sites regulations: the potential psychological and financial harms for physicians that can result from physician rating sites need to be contained without limiting the potential benefits for patients with respect to health, health literacy, and equity.

Keywords: Physician rating sites, ethics, patient empowerment, patient-physician relationship

Introduction

Physician rating sites allow patients to evaluate their experience and satisfaction with their health care providers, similar to other service-oriented businesses. The ratings are posted online and are intended as a source of information for people searching for a physician.

In addition to the more than 30 private physician rating sites [1,2], more and more publicly hosted physician rating sites have gone online in the last 5 years. In 2007, the National Health Service (NHS) in the United Kingdom launched the NHS Choices website, which allows patients to evaluate both physicians and hospitals. In 2010, the largest German health insurer (AOK) launched its own portal, Arzt-Navi, which initially went through a test phase in 3 out of 16 German states and has been open to all German residents since May 2011. In the United States, the Hospital Compare site, maintained by the Centers for Medicare & Medicaid Services (CMS) and other publicly funded sites, provides information on the quality of care, but it does not yet permit patients to rate physicians [3].

Increasingly, research results are being published on fundamental characteristics of physician rating sites in their present condition in regard to their frequency, content, and user assessment patterns [1,2,4-7]. However, direct evidence of potential benefits and harms of physician rating sites is still lacking.

Only a handful of discussion papers on this topic have been published in scientific journals [3,8-10]. However, the media are increasingly discussing opportunities and challenges of physician rating sites. On the one hand, government and insurance company representatives often express their support of such concepts [11]. Ben Bradshaw, the former British Minister for Health, for example, criticized the general lack of transparency as an argument supporting physician rating sites, stating that “I would never think of going on holiday without cross-referencing at least two guide books and using Trip Advisor. We need to do something similar for the modern generation in healthcare.” Other critics have referred to evidence related to questions similar to those of physician rating sites [8]. They highlight that key clinical measures and outcomes are closely linked to patient satisfaction [12,13] and that systematic feedback changes doctors’ clinical performance [14]. Nevertheless, physician representatives tend to argue against physician rating sites. Laurence Buckman, Chairman of the British Medical Association’s General Practitioners Committee, fears that physician rating sites could compromise physicians: “A website on which people can slander or praise irresponsibly is the wrong approach” [11]. Likewise, Frank Ulrich Montgomery, President of the German Medical Association, described these websites as “platforms for denunciation” [15].

Taking the current state of discussion and scientific analysis of physician rating sites into account, health policy decision making, with respect to the implementation and regulation of physician rating sites, is challenging for at least two major reasons: (1) the lack of outcomes research in the field of physician rating sites, and (2) the controversial but poorly structured (ethical) debate on the pros and cons of physician rating sites.

This paper has two aims. First, it aims to structure the ethical debate around the basic concept of physician rating sites—that is, allowing patients to rate, comment, and discuss physicians’ performance, online and visible to everyone. This provides a more thorough and transparent (and therefore more reasonable) starting point for further discussion and further decision making on physician rating sites: what should physicians and health policy decision makers take into account when discussing the basic idea of physician rating sites and its possible implications for the physician–patient relationship? Second, it discusses where and how the preexisting evidence from the partly related field of public reporting of physician performance can serve (at least) as an indicator for specific needs of evaluative research in the field of physician rating sites.

While this paper focuses on the preceding ethical discussion concerning the basic concept of physician rating sites, it does not analyze the wide range of issues surrounding the safety and validity of information provided by physician rating sites. It also does not analyze the various judicial aspects of physician rating sites such as ownership or liability, for example. Needless to say, controversy in eHealth ethics often relates to these implementation issues [16] and they are also relevant for physician rating sites. The issues related to the implementation and management of physician rating sites, however, become relevant and need to be discussed thoroughly after the basic concept of physician rating sites has been generally accepted by physicians and health policy decision makers (as is the case with the basic concept of online patient information about diseases and treatment options, for example).

The following analysis is based on three ethical principles that are presented in widely acknowledged ethical frameworks for health care and health policy (eg, [17,18]). The two major reasons for choosing these rather general ethical principles are the following: (1) the discussion on physician rating sites is still in its beginnings, and a too-specific framework (eg, specific eHealth ethics frameworks [19,20]) runs the risk of excluding relevant ethical issues and arguments, and (2) because the structure provided here aims to inform health care professionals and health policy decision makers that are not always trained in ethics, it is helpful to frame the analysis with well-known ethical principles.

The three substantial ethical principles are (1) patient (and physician) welfare, (2) patient autonomy, and (3) social justice. The analysis of these substantial ethical principles is complemented by a brief description of procedural criteria that aims to improve the fairness of the health policy decision making concerning physician rating sites.

Physician Rating Sites and Patient Welfare

Assuming that there are outcome-related quality differences between physicians and that physician rating sites can detect these differences to a certain degree, then physician rating sites could aim to improve health outcomes (patient welfare). One way to realize this goal is to make doctors aware of aspects of their work in need of improvement as identified by physician rating sites so that adjustments can be implemented in practice [14]. Second, patients who obtain information from physician rating sites are probably more likely to choose better-quality physicians and reap health benefits as a result. However, direct evidence that supports or negates these assumptions is lacking.

Can evidence from other fields be reasonably transferred to physician rating sites? The evidence related to the effects of public reporting of physician performance, based on either specific quality indicators or public report, is mixed [21]. Few researchers have examined the effects of public reporting on the actual delivery of health care [22,23] and even fewer have investigated how report cards might improve care [24,25]. Paradoxically, there is some evidence that public report cards can actually reduce quality by leading physicians to select patients based on risk profiles that best suit the specific quality indicator, which is critical for the interpretation of physician rating sites [26,27]. It is questionable, however, whether this scenario can be transferred to physician rating sites. Nevertheless, opponents of physician rating sites suggest that patients could be led to award more positive ratings based on nonmedical factors, such as pleasant waiting room music, or even persuasion by the physician.

Although physician rating sites cannot measure positive health outcomes directly, the presence of physician rating sites and the resulting transparency of medical performance could result in a (measurable) improvement in public trust in the system [21,28]. However, evidence either in support of or against this assumption is also lacking.

Physician Rating Sites and Patient Autonomy

Besides the direct improvement in health outcomes, another intervention goal of physician rating sites that is closely linked to the ethical principle respect of patient autonomy can be grouped under the heading improvement of health literacy [29,30]. According to the World Health Organization’s definition, health literacy is “the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand, and use information in ways which promote and maintain good health” [31]. Improving health literacy empowers and thereby respects patient autonomy in making various health-related choices. Physician rating sites could potentially influence health literacy on three different levels [29] (see Table 1 [29,32]).

Table 1.

The potential impact of physician rating sites on the three levels of health literacy

Level Potential impact
Functional People are able to process simple information that can help them find a specialist physician who understands medical procedures. Obviously, physician rating sites could promote this functional level of health literacy by providing a wide range of information; conversely, they could counteract it by disseminating false or at least biased information (for example, by putting only those physicians at the top if the list who bought a premium account offered by the specific physician rating sites).
Interactive Through the development of enhanced cognitive and social skills and structures, this level of health literacy allows patients to play a more active role in interactions with their health environments. Physician rating sites could improve this interactive level—for example, if physician rating sites were to serve as a navigational tool with which patients are better able to steer through the health care system and enhance their communication and exchange of knowledge about specific physicians (or hospitals) from peer to peer (for example, by offering open text options at physician rating sites that allow users to describe in a more narrative style how they experienced the performance of a certain physician).
Critical People have the ability to question so-called standards and to critically evaluate health-related information [29]. It would be practically relevant if, in general, the exchange of information between patients (peer-to-peer) regarding specific physicians and health care facilities would lead to the development of an increasingly higher and more discriminating level of quality assessment of health care through patients themselves. For example, physician ratings could cover aspects of health care quality that other patients have not sought before, thus providing the possibility to expand patients’ horizons in terms of quality assessment. Furthermore, users of physician rating sites could post ratings of which physician reviews were more or less helpful to them or may have even been false and misleading. See, for example, the concept of labeling—that is, to provide information about information (meta-information), which can be either descriptive or evaluative [32].

Physician Rating Sites and Physician Welfare

Alongside the consideration of potential benefits and harms of physician rating sites for patients, the process of ethical decision making should also address the possible side effects for physicians. In particular, it should take into account the possible emotional and psychological burdens for physicians, as well as potential financial damages to those physicians who are not rated as well. In addition to the concern of defamation of individual physicians, another broader concern arises that physicians are discussed and furthermore rated not only based on their professional skills but also as a person. Refer to the assessment from the President of the German Medical Association, Frank U Montgomery, that “The only people who speak up on the Internet are those with an extreme opinion” (translation by the author) [15]. Buckman (see above) pointed his arguments in the same direction. Whether the fears of physician representatives are justified is more than questionable. Recent evaluations of rating tendencies from the United States and Germany demonstrate that the majority of reviews in physician rating sites appear to be positive [1,33].

Nevertheless, the potential harm to physicians should be taken seriously. For example, making the ratings first available when they have reached a certain baseline number (eg, 5–10) reduces the impact of extreme opinions, and peer review allows for the differentiation and elimination of defamations. However, when the baseline number or the peer-review requirements are set too high, then potential benefits for patients could be hindered. An ethical and regulative challenge is depicting the differentiation between disproportionate defamation on the one hand and relevant critique on the other hand. The criteria that physician rating sites use for these differentiation tasks (including examples of ratings classified as defamation or relevant critique) should be made transparent to the users. Furthermore, eliminating overly negative ratings without eliminating overly positive ratings will create a general bias toward more positive ratings and will decrease the differentiation between physicians. See also the section below on legitimacy of decision making in the implementation and regulation of physician rating sites.

Physician Rating Sites and Social Justice

If we accept the assumption, as discussed above, that physician rating sites could have a positive effect on patient welfare as well as on the advancement of health literacy, then they could also have an impact on equal opportunity among the different socioeconomic groups within one society [30,34]. For reasons of equity, one should investigate whether all socioeconomic groups that could benefit from physician rating sites have unrestricted access to the Internet. The Internet as a source of information regarding the quality of physicians is relatively accessible in comparison with alternative forms of media (print media and personal contact). The relativity arises as a result of the contrast between possible effective alternatives. Arguably, one of the most effective available options to find a good physician is to ask friends or relatives in the medical profession to recommend a colleague. Thus, it is indisputable that physicians as a social group have structural advantages in the search for a new physician due to insider information received from colleagues.

Even though the Internet is widely accessible, one must consider remaining financial and nonfinancial access barriers, such as income, culture, gender, and age. Effective use of physician rating sites remains primarily dependent on the cognitive and intellectual capabilities of the users. This phenomenon could negatively affect the already-existing health inequalities between more- and less-educated groups (inequity).

A further issue to be considered are effects that have been observed in the context of public reporting of quality indicators [21]. If quality indicators such as satisfaction with care are correlated with race and socioeconomic status [35,36], then physicians may shy away from treating some groups of patients out of fear of being penalized by relatively poor ratings in physician rating sites.

Legitimacy of Decision Making in the Implementation and Regulation of Physician Rating Sites

Decisions regarding the implementation or regulation of physician rating sites through public institutions (eg, NHS Choices in the United Kingdom or statutory health insurers in Germany) are associated with inevitably complex issues. Such issues cannot be solved by reference to an ultimate ethical principle [16]. Whenever health care institutions are confronted with the challenges of ethical assessments, they increase the legitimacy of their decisions when certain minimal requirements for a fair decision-making process are met [34] (see Table 2 [37-39]).

Table 2.

Basic conditions for a fair decision-making process concerning the implementation and regulation of physician rating sites

Condition Implication
Transparency Empirical information and normative arguments that were relevant for the decision-making process on more- or less-restrictive regulation of physician rating sites should be made available to the public.
Justification Decisions should be based on a relevant rationale. Relevant reasons are especially those that explicitly and comprehensibly ascribe to the above-described ethical criteria: patient and physician welfare, autonomy, and justice.
Participation Subjective evaluations that are part of the decision-making process are inevitable due to the complexity of the question. The legitimacy of such subjective evaluations increases when the affected populations (here patients, physicians, and insurance agents) have been given the opportunity to participate and to provide relevant empirical information and normative arguments [37,38].
Minimizing conflicts of interest Decisions on the implementation or regulation of physician rating sites should be regulated in order to avoid as many conflicts of interest as possible [39]. Conflicts of interest exist, for example, if the decision maker him- or herself benefits from any financial advantages on decisions made for or against any particular forms of regulation of physician rating sites.
Openness for revision Every decision should be open for revision provided that better normative arguments or better evidence on the effects of physician rating sites is available.

Discussion

The previous sections specified fundamental ethical principles relevant to the discussion of the basic concept of physician rating sites (allowing patients to rate, comment on, and discuss physicians’ performance—online and visible to everyone). The specified ethical principles should be recognized when the various stakeholders in the field of physician rating sites (physicians, patients, politicians, insurance companies, owners of private physician rating sites, and others) develop their viewpoints toward the basic concept of physician rating sites. These principles should also play a crucial role when decisions on the implementation and (more- or less-restrictive) regulation of physician rating sites are made. Even when thorough empirical evaluations of potential unknown effects of physician rating sites are strongly required, drawing on plausibility and normative arguments is unavoidable for justifying (regulatory) decisions regarding physician rating sites. The aforementioned basic conditions for a fair decision-making process are particularly relevant under such conditions of normative complexity and insufficient evidence (uncertainty).

In the opinion of the author, two issues present a special challenge and should play an important role when deciding about more- or less-restrictive physician rating sites regulations. First, the potential psychological and financial harms to physicians need to be contained without limiting the potential health and health literacy benefits for patients. In many countries the medical profession enjoys privileges such as strong advocacy groups and special social facilities. Thus, the denial of transparency on patient experiences and satisfaction (with physician performance) requires a strong rationale. Second, in light of the unequal distribution of health opportunities, particularly due to discrepancies in health literacy, possible countermeasures (such as physician rating sites) are highly relevant. Measures to improve public physician rating sites (such as NHS Choices and the AOK website) should be specifically tailored to the needs of vulnerable subgroups of the population. Preferably, aspects such as accessibility and the clarity of information should be evaluated and further improved.

If more general health policy discussions on the acceptance or rejection of the basic ideas of physician rating sites have been settled, further analyses need to focus on the ethical aspects of adequate implementation and management of such websites. Issues such as data privacy and validity, as well as ethical guidelines such as the e-Health Code of Ethics, will then play an important role [19,20,32].

Abbreviations

NHS

National Health Service

Footnotes

None declared

References

  • 1.Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med. 2010 Sep;25(9):942–6. doi: 10.1007/s11606-010-1383-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Reimann S, Strech D. The representation of patient experience and satisfaction in physician rating sites. A criteria-based analysis of English- and German-language sites. BMC Health Serv Res. 2010;10:332. doi: 10.1186/1472-6963-10-332. http://www.biomedcentral.com/1472-6963/10/332.1472-6963-10-332 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Lagu T, Lindenauer PK. Putting the public back in public reporting of health care quality. JAMA. 2010 Oct 20;304(15):1711–2. doi: 10.1001/jama.2010.1499.304/15/1711 [DOI] [PubMed] [Google Scholar]
  • 4.Kadry B, Chu LF, Kadry B, Gammas D, Macario A. Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res. 2011;13(4):e95. doi: 10.2196/jmir.1960. http://www.jmir.org/2011/4/e95/v13i4e95 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Schaefer C, Schwarz S. [Doctor rating sites: which of them find the best doctors in Germany?] Z Evid Fortbild Qual Gesundhwes. 2010;104(7):572–7. doi: 10.1016/j.zefq.2010.09.002.S1865-9217(10)00243-6 [DOI] [PubMed] [Google Scholar]
  • 6.Lopez A, Detz A, Ratanawongsa N, Schillinger D, Sarkar U. What do your patients say about you on the internet? A thematic analysis of online reviews of primary care physicians. 33rd Annual Meeting of the Society of General Internal Medicine Meeting; April 28-May 1, 2010; Minneapolis, MN, USA. 2010. [Google Scholar]
  • 7.Black EW, Thompson LA, Saliba H, Dawson K, Black NM. An analysis of healthcare providers' online ratings. Inform Prim Care. 2009;17(4):249–53. doi: 10.14236/jhi.v17i4.744. [DOI] [PubMed] [Google Scholar]
  • 8.Bacon N. Will doctor rating sites improve standards of care? Yes. BMJ. 2009;338:b1030. doi: 10.1136/bmj.b1030. [DOI] [PubMed] [Google Scholar]
  • 9.McCartney M. Will doctor rating sites improve the quality of care? No. BMJ. 2009;338:b1033. doi: 10.1136/bmj.b1033. [DOI] [PubMed] [Google Scholar]
  • 10.Strech D. [Physician rating sites from an ethical viewpoint: a roadmap] Z Evid Fortbild Qual Gesundhwes. 2010;104(8-9):674–81. doi: 10.1016/j.zefq.2010.09.019.S1865-9217(10)00262-X [DOI] [PubMed] [Google Scholar]
  • 11.Carvel J. The Guardian. 2008. Dec 30, [2011-07-29]. Patients to Rate and Review Their GPs on NHS Website http://www.guardian.co.uk/society/2008/dec/30/doctors-rating-website-nhs.
  • 12.Jha AK, Orav EJ, Zheng J, Epstein AM. Patients' perception of hospital care in the United States. N Engl J Med. 2008 Oct 30;359(18):1921–31. doi: 10.1056/NEJMsa0804116. http://dx.doi.org/10.1056/NEJMsa0804116.359/18/1921 [DOI] [PubMed] [Google Scholar]
  • 13.Winefield HR, Murrell TG, Clifford J. Process and outcomes in general practice consultations: problems in defining high quality care. Soc Sci Med. 1995 Oct;41(7):969–75. doi: 10.1016/0277-9536(94)00403-g.027795369400403G [DOI] [PubMed] [Google Scholar]
  • 14.Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME Guide No. 7. Med Teach. 2006 Mar;28(2):117–28. doi: 10.1080/01421590600622665.P86M5X746N4L4128 [DOI] [PubMed] [Google Scholar]
  • 15.BÄK Newsletter Ärztlicher Kreisverband Ebersberg Körperschaft des öffentlichen Rechts. 2009. Jun 25, [2011-07-29]. Arztbewertungsportale: massive Kritik am AOK-Arzt-Navigator http://www.aekv-ebersberg.de/aktuelles/105-baek-newsletter-2009-004-vom-25062009.
  • 16.Dyer KA. Ethical challenges of medicine and health on the Internet: a review. J Med Internet Res. 2001;3(2):E23. doi: 10.2196/jmir.3.2.e23. http://www.jmir.org/2001/2/e23/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Medical Professionalism Project Medical professionalism in the new millennium: a physicians' charter. Lancet. 2002 Feb 9;359(9305):520–2. doi: 10.1016/S0140-6736(02)07684-5.S0140-6736(02)07684-5 [DOI] [PubMed] [Google Scholar]
  • 18.Beauchamp TL, Childress JF. Principles of Biomedical Ethics. 6th edition. New York, NY: Oxford University Press; 2009. [Google Scholar]
  • 19.Eysenbach G. Towards ethical guidelines for e-health: JMIR theme issue on eHealth ethics. J Med Internet Res. 2000;2(1):E7. doi: 10.2196/jmir.2.1.e7. http://www.jmir.org/2000/1/e7/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.e-Health Ethics Initiative e-Health Code of Ethics (May 24) J Med Internet Res. 2000;2(2):E9. doi: 10.2196/jmir.2.2.e9. http://www.jmir.org/2000/2/e9/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA. 2005 Mar 9;293(10):1239–44. doi: 10.1001/jama.293.10.1239.293/10/1239 [DOI] [PubMed] [Google Scholar]
  • 22.Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA. 2000 Apr 12;283(14):1866–74. doi: 10.1001/jama.283.14.1866. http://jama.ama-assn.org/cgi/pmidlookup?view=long&pmid=10770149.jrv90044 [DOI] [PubMed] [Google Scholar]
  • 23.Leatherman ST, Hibbard JH, McGlynn EA. A research agenda to advance quality measurement and improvement. Med Care. 2003 Jan;41(1 Suppl):I80–6. doi: 10.1097/00005650-200301001-00009. [DOI] [PubMed] [Google Scholar]
  • 24.Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care. 2003 Jan;41(1 Suppl):I30–8. doi: 10.1097/00005650-200301001-00004. [DOI] [PubMed] [Google Scholar]
  • 25.Schauffler HH, Mordavsky JK. Consumer reports in health care: do they make a difference? Annu Rev Public Health. 2001;22:69–89. doi: 10.1146/annurev.publhealth.22.1.69.22/1/69 [DOI] [PubMed] [Google Scholar]
  • 26.Omoigui NA, Miller DP, Brown KJ, Annan K, Cosgrove D, Lytle B, Loop F, Topol EJ. Outmigration for coronary bypass surgery in an era of public dissemination of clinical outcomes. Circulation. 1996 Jan 1;93(1):27–33. doi: 10.1161/01.cir.93.1.27. http://circ.ahajournals.org/cgi/pmidlookup?view=long&pmid=8616936. [DOI] [PubMed] [Google Scholar]
  • 27.Werner RM, Asch DA, Polsky D. Racial profiling: the unintended consequences of coronary artery bypass graft report cards. Circulation. 2005 Mar 15;111(10):1257–63. doi: 10.1161/01.CIR.0000157729.59754.09. http://circ.ahajournals.org/cgi/pmidlookup?view=long&pmid=15769766.111/10/1257 [DOI] [PubMed] [Google Scholar]
  • 28.Blendon RJ, DesRoches CM, Brodie M, Benson JM, Rosen AB, Schneider E, Altman DE, Zapert K, Herrmann MJ, Steffenson AE. Views of practicing physicians and the public on medical errors. N Engl J Med. 2002 Dec 12;347(24):1933–40. doi: 10.1056/NEJMsa022151. http://dx.doi.org/10.1056/NEJMsa022151.347/24/1933 [DOI] [PubMed] [Google Scholar]
  • 29.Nutbeam D. The evolving concept of health literacy. Soc Sci Med. 2008 Dec;67(12):2072–8. doi: 10.1016/j.socscimed.2008.09.050.S0277-9536(08)00457-7 [DOI] [PubMed] [Google Scholar]
  • 30.Kickbusch I. Health literacy: engaging in a political debate. Int J Public Health. 2009;54(3):131–2. doi: 10.1007/s00038-009-7073-1. [DOI] [PubMed] [Google Scholar]
  • 31.World Health Organization . Health Promotion Glossary. Geneva: WHO; 1998. [2011-11-18]. http://www.who.int/hpr/NPH/docs/hp_glossary_en.pdf. [Google Scholar]
  • 32.Eysenbach G, Diepgen TL. Towards quality management of medical information on the internet: evaluation, labelling, and filtering of information. BMJ. 1998 Nov 28;317(7171):1496–500. doi: 10.1136/bmj.317.7171.1496. http://bmj.com/cgi/pmidlookup?view=long&pmid=9831581. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Strech D, Reimann S. [German language physician rating sites: the status quo of evaluation criteria, evaluation tendencies and utilization] Gesundheitswesen. 2011;73:1–8. doi: 10.1055/s-0031-1297254. [DOI] [PubMed] [Google Scholar]
  • 34.Daniels N. Just Health: Meeting Health Needs Fairly. New York, NY: Cambridge University Press; 2008. [Google Scholar]
  • 35.Harpole LH, Orav EJ, Hickey M, Posther KE, Brennan TA. Patient satisfaction in the ambulatory setting. Influence of data collection methods and sociodemographic factors. J Gen Intern Med. 1996 Jul;11(7):431–4. doi: 10.1007/BF02600192. [DOI] [PubMed] [Google Scholar]
  • 36.Fiscella K, Franks P. Influence of patient education on profiles of physician practices. Ann Intern Med. 1999 Nov 16;131(10):745–51. doi: 10.7326/0003-4819-131-10-199911160-00005.199911160-00005 [DOI] [PubMed] [Google Scholar]
  • 37.Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev. 2006;(3):CD004563. doi: 10.1002/14651858.CD004563.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Thompson AG. The meaning of patient involvement and participation in health care consultations: a taxonomy. Soc Sci Med. 2007 Mar;64(6):1297–310. doi: 10.1016/j.socscimed.2006.11.002.S0277-9536(06)00577-6 [DOI] [PubMed] [Google Scholar]
  • 39.Lo B, Field MJ, Committee on Conflict of Interest in Medical Research, Education, and Practice, Institute of Medicine. Conflicts of Interest in Medical Research, Education, and Practice . Conflicts of Interest in Medical Research, Education, and Practice. Washington, DC: National Academies Press, Institute of Medicine; 2009. [PubMed] [Google Scholar]

Articles from Journal of Medical Internet Research are provided here courtesy of JMIR Publications Inc.

RESOURCES