Does certification improve medical standards?

Short abstract

The English chief medical officer recently recommended certification of doctors to improve professional regulation. Could a system similar to that used in the United States, which is associated with better care, be the way forward?

Worldwide, the regulation of medical professionals is central to attempts at quality improvement in health care. The arguments for strengthening professional regulation come from evidence of systemic underperformance and isolated cases of egregious behaviour in individuals. We review data on the effect of certification in the United States on quality of care, and we consider the implications for the current debate on revalidation in the United Kingdom.

Problems in the UK

In the UK routine data continue to highlight uneven quality of care compared with other countries.^1-3 Good doctors, safer patients lists recent cases of exceptionally poor clinical practice or criminal conduct: Harold Shipman, Clifford Ayling, Richard Neale, William Kerr, Michael Haslam, Rodney Ledward, and the department of paediatric cardiac surgery at the Bristol Royal Infirmary.⁴ Although rare, such cases still occur and point to failures in underlying systems for detecting and preventing unsatisfactory performance at an early stage.

Furthermore, over time the skills and knowledge of medical professionals can erode, with potentially serious consequences for quality of care. In a systematic review of the relation between experience and quality of care, over half of the studies (32 of 62; 52%) reported an association between decreasing performance and increasing years in practice for all outcomes assessed. These results suggest that older doctors and those who have been practising for many years have less factual knowledge, are less likely to adhere to appropriate standards of care, and may also have poorer patient outcomes.⁵

Role of professional regulation

Professional regulation has three main purposes. The first is to ensure that minimally acceptable standards of care are being provided. The second is to provide accountability and reassure patients and payers that medical professionals are deserving of trust. The third is to improve quality of care by providing guidance about best practice and fostering improvements in performance through measurement and feedback.

In the UK the concept of revalidation was developed to answer concerns about controlling and maintaining professional standards and conduct. However, the core feature of the revalidation process proposed by the General Medical Council was a peer appraisal process, and in the fifth report of the Shipman inquiry Dame Janet Smith strongly criticised it for lacking sufficient rigour.⁶ The main problem with the proposed model was its over-riding concern with providing reassurance. It contained little in the way of objective measurement of performance or ways to assess compliance with scientifically derived standards of practice. Since Dame Janet Smith's criticisms, the Department of Health has been taking stock of professional regulation in the UK, and the chief medical officer's report contains 44 recommendations to strengthen professional regulation.⁴ A key recommendation is the introduction of renewable specialist certification within a framework of revalidation. Certification is a well established process in the US, and it is timely to review available evidence on the impact that certification has on quality of care.

Figure 1.

US doctors already have to get recertified

Credit: PAT SULLIVAN/AP/EMPICS

Box 1: Requirements for specialty certification in the US

• Three to six years (depending on specialty) of training in an accredited training programme

• Passed a rigorous cognitive examination

• Various combinations of:

  Satisfactory evaluations by the programme director on six competencies (patient care, medical knowledge, practice based learning and improvement, interpersonal and communications skills, professionalism, and systems based practice)

  Passed oral examinations

  Satisfactory audits of medical records

  Satisfactory review of case logs

  Satisfactory observed performance on real or standardised patients

Specialty based certification: what is it?

In the US certification is a voluntary process that allows doctors to demonstrate achievements and competencies beyond the minimum acceptable standards required for licensing purposes. The American Board of Medical Specialties (www.abms.org/) is the umbrella organisation for the 24 approved specialty boards that manage certification. In 2002, more than 85% of licensed doctors in the US held a valid certificate.⁷ Board certification is increasingly valued by doctors and is expected by payers, medical groups, hospitals, and patients. Box 1 shows the requirements for initial certification.⁸

Despite the similarities between certification in the US and membership examinations of the medical royal colleges in the UK, important differences exist. Certification in the US provides assurance that a specialist has completed an educational programme and passed an objective evaluation of their knowledge, skills, and experience. Successful candidates are certified as specialists. In the UK success in membership examinations, such as the MRCP (www.mrcpuk.org/history.html), does not usually confer specialist status; instead it is recognised as a qualification for higher specialist training.

In 2000, the American Board of Medical Specialties ratified a process of recertification (the maintenance of certification programme). This means that certification is “time limited” to six to 10 years, and to renew their certified status specialists must show evidence of:

• Professional standing

• Commitment to lifelong learning and involvement in periodic self assessment

• Cognitive expertise (results from a standardised examination)

• Standards based evaluation of performance in clinical practice.⁷

Box 2: Key strengths of certification

The key strengths of the certification approach lie in the way it encompasses:

• Objective, summative measures of knowledge

• Subjective ratings of approval and respect from peers

• Requirements for ongoing learning

• Methods for rigorous self auditing of clinical practice and improvement

• Information on the competence of individual doctors that is comprehensible to patients

This maintenance programme is similar in its intention to revalidation and may point to ways that professional regulation could be strengthened in the UK.

Certification and quality: the evidence

Evidence about the impact of certification on quality of care was gathered as part of a broader UK research initiative that collects and synthesises empirical evidence on a wide range of interventions used to improve quality of health care. Details of our search strategies and methodology are available in our full report on regulatory interventions (www.health.org.uk/aboutus/publications/research/QEI_regulation_report.pdf).

A systematic review of studies published between 1966 and 1999 found that over half (16 of 29 analyses reported in 11 articles) showed positive and statistically significant associations between certification and superior outcomes.⁹ Since 1999, four well conducted studies have concluded that board certification is associated with provision of higher quality care across a range of specialties.

Two studies evaluated whether the certification status of doctors treating patients with acute myocardial infarction was associated with greater compliance with recommended treatments or lower rates of mortality. One study investigated whether certification was associated with decreased patient mortality or length of stay.¹⁰ The study used data from 16 629 stays in hospital in 1993 in Pennsylvania and found that certification in internal medicine or cardiology was associated with a 19% reduction in mortality (after adjusting for hospital resources, comorbidities, and other variables). The other study used nationwide data from 101 251 patients (> 65 years old) in hospital in the US between January 1994 and February 1996.¹¹ This study found that board certified doctors provided better quality of care than non-certified ones, although no statistically significant difference was seen in mortality rates at 30 days.

In another study, certification by the American Board of Surgery was associated with reduced mortality (non-certified v certified odds ratio 1.4, 95% confidence interval 1.1 to 1.9) and complication rates (1.2, 1.0 to 1.4) after colon resection (although subspecialty certification in colorectal surgery was not related to outcomes).¹² A further study found that noncertified mid-career anaesthesiologists had higher rates of patient mortality (1.13, 1.00 to 1.26) and failure to rescue rates (1.13, 1.01 to 1.27) than other anaesthesiologists (although the type of hospital was not controlled for in this study).¹³

As well as correlations between certification and clinical processes and outcomes, recent studies have found that a lack of certification is associated with increased risk of disciplinary action. One Californian study compared 890 doctors who were disciplined between 1998 and 2001 with 2981 randomly selected, non-disciplined controls.¹⁴ Lack of certification was associated with an increased risk of disciplinary action (2.22; P < 0.001). The offences that resulted in disciplinary action included negligence (n = 335; 38%), unprofessional conduct (n = 88; 10%), substance misuse (n = 87; 10%), and inappropriate prescribing (n = 78; 9%). Another study in Ohio compared disciplined doctors (n = 340 who committed 477 offences) with a matched control group.¹⁵ Offenders were significantly less likely to be board certified (0.65, 0.46 to 0.92). In this case, the most common offences were impairment due to the use of alcohol or drugs (n = 100; 21%), inappropriate prescription or possession of drugs (n = 66; 14%), and negligence or incompetence (n = 34; 7%). A further study in Oklahoma found that non-certified doctors were at greater risk of being disciplined (univariate hazard ratio 3.3; P < 0.001).¹⁶

Thus, the association between certified status and higher quality of care is consistent across a range of clinical specialties, geographical locations, and permutations of applying and interpreting regulation.

Whither professional regulation in the UK?

The notion of professional ethos, supported by intensive training and peer led inquiries into “fitness to practise” has long been the foundation for assuring quality in the NHS. However, it is no longer acceptable to rely so heavily on opaque principles of professionalism. Renewable certification provides more transparency via validated processes for assessing skills, knowledge, and competence. Most of the available evidence on professional regulation is associative rather than clearly causal, but it seems to support rigorously conducted certification as a good method to improve quality of care.

Almost all of the published studies have been conducted in the US, where certification is coordinated and intellectually underwritten by the boards of medical specialties, which are broadly analogous to the royal colleges in the UK. Adopting certification as a key regulatory instrument in the UK will have important implications for the colleges, individually and collectively. Certification encompasses many complex tasks including standard setting, establishing valid processes for assessment and reporting, and ensuring uniformity of approach across various specialties. Although informed by existing processes such as membership examinations, these tasks will require ongoing investment in research and development.

The adoption of time limited certification in the UK will have many financial implications. In the US much of the cost is borne by doctors themselves who are likely to benefit from the process. Recertification is a way to reaffirm the commitment to patient care, and to reassure doctors themselves that—on an objective scale of knowledge and according to subjective accounts of their peers—they continue to practise well.¹⁷

However, there may be an argument for some of the costs to be offset by the NHS; particularly if certified status becomes a factor, as in the US, in “pay for performance.”¹⁸ We need a method of measuring costs and benefits to identify where potential gains will accrue and provide guidance towards appropriate sources of funding.

As the NHS strives to secure improvements in quality of care, it is important to consider the central part played by the professions. As we have argued elsewhere, individual professional conduct, along with the values and normative tools of the collective professions, will always provide a patient with the best quality assurance.¹⁹ Certification, or validation within the UK context, provides a way to strengthen and bolster that vital protection and reassurance (box 2).

Summary points

England's chief medical officer recently recommended the adoption of certification as one of several changes to strengthen professional regulation

Specialty certification is a well established process in the United States

Observational studies conducted in the US have found associations between professional certification status and quality of care