Unhappy doctors: what are the causes and what can be done?

As the response to a recent BMJ editorial showed, unhappy doctors are a worldwide phenomenon. In this article, based on workshops with doctors in the United States and United Kingdom, Nigel Edwards and colleagues suggest that the cause is a breakdown in the implicit compact between doctors and society: the individual orientation that doctors were trained for does not fit with the demands of current healthcare systems. They outline what a new compact might look like

Richard Smith's editorial in the BMJ about unhappiness in the medical profession provoked a huge response and confirmed that this is an international and widespread problem.^1–4 We describe here the views of doctors themselves, gleaned from workshops in the United States and the United Kingdom, on the problem and what might be done about it.

Summary points

• Several reports from around the world describe declining morale among doctors, but little is known about the reasons

• Workload and pay, though important, do not fully explain the problem

• A key factor seems to be a change in the psychological compact between the profession, employers, patients, and society so that the job is now different from what doctors expected

• Developing a new compact that is more acceptable to the profession is important

• Clinical leaders have a potentially crucial role in developing a new compact

Evidence gathering

There is limited evidence to inform the debate, and therefore this article and our longer report⁵ are based on literature on the subject and seminars held in Massachusetts last August and in London in October. As with Smith's original article the ideas should be treated as tentative.

In the US seminar the participants were managers and medical executives from various healthcare systems including Kaiser Permanente. The UK participants were mostly senior members of the medical profession, educators, officers of the BMA, medical managers, and representatives of junior doctors, medical students, and a patients' group. Senior representatives from the Department of Health were present for part of the discussion. Participants were selected for their breadth of knowledge of the profession and the issues. We also asked individuals who were well placed to influence the implementation of some of the solutions.

The ideas that emerged from the workshop have been tested at events with members of the Royal College of Physicians and with senior members of the medical profession at a conference held by the Department of Health.

Potential causes of unhapiness

Pay and workload are obvious causes for unhappiness among doctors. However, evidence from systems with much higher pay and longer consultation times suggests that these are not enough by themselves to ensure high morale. Several of the causes are probably the result of changes in the expectations of patients, governments, and employers; and there may also be causes within medicine itself.

The developed world has seen significant reductions in medical autonomy and increases in accountability as a result of the growing evidence base and a long running attempt to bring medicine under managerial and cost control by governments, payers, and employers. This has resulted in the growing use of guidelines, protocols, audit, regulation, and inspection that many doctors perceive as eroding their control over their professional lives. Though there are benefits from these changes, having control over work is important for the job satisfaction of clinicians⁶,⁷ and can have implications for overall health of employees.⁸ A related change in the relationship with employers has been the increased emphasis on numerical targets, efficiency, and volumes of work that dates from the late 1970s.

The changes in relationships with patients and society are particularly important. There has been a decline in deference for all professions and a perceived loss of trust, coinciding with a feeling that the media has become much more hostile. In fact doctors are a highly trusted profession,⁹ and the proportion of negative news stories is fairly constant—although the total number of stories has increased.¹⁰ At the same time, patients are increasingly active consumers and they demand and have been encouraged to expect enhanced services, including extended hours and rapid access. The easy availability of health information coupled with a sense of entitlement is shifting the power in the doctor-patient relationship and causing unease. This is compounded by sometimes unrealistic expectations about the power of medicine to solve the ills of modern life. Smith described this as a bogus contract between the profession and patients but one which doctors have themselves sometimes encouraged.¹

Medical causes

Several causes relate to medicine itself. The job is difficult and emotionally demanding, and doctors are more likely to be self critical and have other personality traits associated with work related stress.¹¹ The poor record of the profession in giving mutual support or giving and receiving feedback aggravates this. Working in teams is also associated with being better able to cope with stress,¹² but skills in teamworking are not universal in the profession.¹³ The selection, training, and socialisation of doctors has tended to compound the problems of high workload, stress, and reaction to changes in the job.

Medicine has been based on a model in which doctors are trained to deal with individuals, not organisations; to take personal responsibility rather than delegate; and to do their best for each patient rather than make trade-offs in a resource constrained environment.¹⁴ These factors make high workloads and high levels of workplace stress all the harder to deal with. They also create a real problem in that professional values and training based on an individualistic orientation do not prepare doctors to function successfully as members of large, complex organisations. Little training is given to equip doctors for this, and the difficulty that many consequently experience leads to stress and frustration.

Failing to deliver what was promised

An important theme in all of these changes is the dissonance between what doctors might have reasonably expected the job to be and how it now is. The psychological contract or compact is a useful concept to explain this problem.

This is the implicit deal between doctors, patients, employers and society that defines what the parties to the relationship give and what they get in return.¹⁵ This seems to have changed without any explicit discussion with those involved and without being replaced with an equally meaningful or rewarding alternative.¹⁶

The type of compact that defined what doctors would give and what they would get in return and which has operated for most of the postwar period is characterised in outline in box B1. The mismatch between this promise and the imperatives facing organisations to find new approaches to delivering safe, high quality care and service is summarised in box B2.

Box 1.

The old compact

What doctors give	What they get in return
•Sacrifice early earnings and study hard	• Reasonable remuneration
•See patients	• Reasonable work/life balance later
•Provide “good” care as the doctor defines it	• Autonomy
	• Job security
	• Deference and respect

Box 2.

The old promise and new imperatives

Doctors promised	New imperatives
•Reasonable work/life balance	• Greater accountability (eg guidelines)
•Autonomy	• Patient centred care
•Job security	• Be more available to patients, provide personalised service
•Deference and respect	• Work collectively with other doctors and staff to improve quality
	• Evaluation by non-technical criteria and patients' perceptions
	• A growing blame culture

Rewriting the compact

Elements of the old promise to doctors are clearly unsustainable given the need to modernise the NHS and other healthcare systems and improve care. Indeed some elements of the old compact are a positive barrier to improvements in medicine and healthcare, particularly in as much as it perpetuates the ethos of what one physician speaker at the US workshop called “practising alone together.”¹⁷ A new and more sustainable compact is required.

The first step might be to re-evaluate the relationship between doctors and healthcare organisations such as the NHS to make the compact more explicit. A new compact might ask doctors to work within guidelines, be accountable for key objectives and improving quality, actively support and contribute to achieving the goals of the organisation, work within resource constraints, and engage in team and collaborative working. Many doctors are doing this already.

Doctors should have the opportunity to shape the goals of the organisation, participate in resource allocation, and have the resources to do the job expected. They will also need training and technical support to do this, including time to step off the “hamster wheel” to engage in improvement activities, high quality data, and a supportive culture that uses information for learning rather than judgment.

A new compact would require organisations to take a highly participative approach with high quality appraisal, personal development, and other modern human resource management techniques. There is emerging evidence that the environment that this creates has beneficial results for patients.¹⁸

The relationship with government also needs to change. The original settlement in which doctors were given the right to regulate themselves and a high level of autonomy has already been undermined. In a new deal doctors might agree to follow nationally agreed standards and guidelines, work to improve quality, and account for their work. In return doctors should expect to be instrumental in developing the standards. The targets they are asked to account for should be realistic and there needs to be a manageable number. Government has a right to ask the profession to continue to improve but should not expect to micromanage it in detail.

Mutual respect and restraint

Mutual respect is an important part of any new deal, including some restraint in each side's criticisms of the other. It also means recognising the legitimacy of the profession to regulate its affairs and the legitimacy of government to have a view on the key goals of the system and in setting standards.

One area where government often fails to meet the expectations of the profession is in not using evidence to design policy.¹⁹ There is an unavoidable conflict between the reductionist approach of medicine and the messy, political, and complex world of policy. Governments could do more to use what evidence there is and evaluating what is done, but the profession needs to accept that different standards of evidence apply in policy making.²⁰,²¹

The new compact with patients is perhaps the most problematic area. There are too many other pressures in society and messages from the media to turn back growing expectations, even if this were desirable. There are over 400 million interactions between patients and health professionals each year in the United Kingdom, and a relatively small number of patients generate a large number of these. At present the problem is that there is often not the time to have the conversation about expectations or to develop the relationship to use time in consultations most productively.²² More effective delegation to other professionals would assist in this.²³

At a more macro level the profession, NHS managers, and government could work together to engage the public and media in a dialogue about the limits of health care, the nature of medicine, its uncertainties, and the dangers of a blame culture. The NHS, government, and the profession could help with this and set realistic expectations by not announcing new services or innovations until they are actually in place.

Supporting the development of a new compact

The training and preparation for becoming a consultant or general practitioner principal needs to equip doctors better for taking leadership and management roles. Good progress has been made in this area, but much more needs to be done, starting much earlier in medical careers. Action is needed to ease the often difficult transition between different phases of careers. The new compact between doctors requires recognition of the contribution of other professionals to the healthcare system, being accountable to and supporting colleagues, offering and accepting constructive feedback, and active engagement in mentoring and guiding others. In return doctors need to receive greater personal support from colleagues and the work environment, help and support to improve their services, and opportunities to continue to develop personally and to have a portfolio of interests (clinical skills, research, education, personal interests, projects, etc.)

Two current tacit assumptions about medical careers need to change to support this change in the compact. Firstly, to encourage creativity, personal renewal, and learning, greater geographical and career mobility should be encouraged. Doctors should be able to restart career paths, retrain, and diversify without attracting criticism. Less controversially, the idea of greater diversity, portfolio careers, or different career stages (similar to that proposed for the consultant's contract) should be promoted. These changes will need to be supported by improved career advice and development and arrangements for re-entry after breaks.

The workshops identified the crucial role that medical leaders (both formal and informal) play in setting the tone of the organisation and being a role model for others. This pivotal position as leaders, managers, and opinion formers in their organisations means that medical leaders at all levels have the opportunity to be instrumental in developing the dialogue about the “gives” and “gets” needed for a new compact. This dialogue is also necessary to create a successful primary care or hospital trust.

Conclusions

We cannot return to the old compact, and something more meaningful and dynamic needs to be put in its place. Disillusionment and disappointment can be eased through honest discussion of the match between doctors' expectations and organisational and societal needs. It is also worth remembering that many doctors are happy and it will be important to ensure that they are fully engaged in helping to develop solutions.

More discussion and research are required to understand this problem and its possible solutions in more detail. In the meantime there is a key role for leaders in the medical profession nationally and in hospitals and primary care to work together with NHS managers to develop a new compact that improves care for patients, improves the effectiveness of the healthcare organisation, and helps create a happier workforce.

Figure.

PAXTON AND FARROW/SPL

Trained for this . . .