Revalidation and electronic cataract surgery audit: a Scottish survey on current practice and opinion

Abstract

Purpose

To determine current knowledge and opinion on revalidation, and methods of cataract surgery audit in Scotland and to outline the current and future possibilities for electronic cataract surgery audit.

Methods

In 2010 we conducted a prospective, cross-sectional, Scottish-wide survey on revalidation knowledge and opinion, and cataract audit practice among all senior NHS ophthalmologists. Results were anonymised and recorded manually for analysis.

Results

In all, 61% of the ophthalmologists surveyed took part. Only 33% felt ready to take part in revalidation, whereas 76% felt they did not have adequate information about the process. Also, 71% did not feel revalidation would improve patient care, but 85% agreed that cataract surgery audit is essential for ophthalmic practice. In addition, 91% audit their cataract outcomes; 52% do so continuously. Further, 63% audit their subspecialist surgical results. Only 25% audit their cataract surgery practice electronically, and only 12% collect clinical data using a hospital PAS system. Funding and system incompatibility were the main reasons cited for the lack of electronic audit setup. Currently, eight separate hospital IT patient administration systems are used across 14 health boards in Scotland.

Conclusion

Revalidation is set to commence in 2012. The Royal College of Ophthalmologists will use cataract outcome audit as a tool to ensure surgical competency for the process. Retrospective manual auditing of cataract outcome is time consuming, and can be avoided with an electronic system. Scottish ophthalmologists view revalidation with scepticism and appear to have inadequate knowledge of the process. However, they strongly agree with the concept of cataract surgery audit. The existing and future electronic applications that may support surgical audit are commercial electronic records, web-based applications, centrally funded software applications, and robust NHS connections between community and hospital.

Introduction

A series of high-profile failures of medical care and regulation in the 1990s led to public demand for a system to ensure that doctors remain fit to practise during their careers.^{1, 2, 3, 4} The process of revalidation is expected to commence in 2012 and will be overseen by the United Kingdom's General Medical Council (GMC). Doctors will be expected to produce supporting information to demonstrate that they are practising in accordance with Good Medical Practice (http://www.gmc-uk.org/guidance/good_medical_practice.asp) over a 5-year cycle in order for their Licence to Practise to be renewed. Although the GMC will define the core requirements for supporting information, the United Kingdom's medical Royal Colleges and Faculties will be responsible for issuing guidance on how these requirements can be met in a specialty context.⁵ Supporting information for revalidation will typically include structured peer and patient feedback,⁵ review of any complaints or critical events, evidence of continuing professional development, a declaration of health and probity, and evidence of quality improvement activity such as clinical audit, review of clinical outcomes, and case reviews (http://www.gmc-uk.org/Royal_Colleges_of_Surgeons.pdf_30594314.pdf).^{6, 7} Future developments might also include formal assessment of skills in certain situations in craft specialties.⁸

Revalidation is already taking place in some European countries. In the Netherlands, for example, clinicians are required to undergo 5-yearly assessments.⁹ Elsewhere, participation in Continuing Medical Education (CME) is becoming mandatory. In Germany, financial penalty or licence suspension is considered for those unable to meet the requirements. In Italy, CME is mandatory by law, but the regulation has not been adequately enforced. The picture is similar in France, whereas in Spain CME is voluntary.¹⁰ There have been calls to accept a single system to recognise CME activities.¹¹

The American health system has also commenced a recertification process. In 2000, all 24 member Boards of the American Board of Medical Specialties agreed to evolve their recertification programmes towards one of continuous professional development. In 2006, all boards had their Maintenance of Certification (MOC) approved, including the American Board of Ophthalmology (http://www.abms.org/Maintenance_of_Certification/ABMS_MOC.aspx).

US ophthalmologists' MOCs must be approved every 10 years. To be approved they must hold a full licence to practise, demonstrate CME, pass periodic ophthalmic review tests, demonstrate their ophthalmic cognitive knowledge, and pass Office Record Review modules (http://www.abop.org/maintain/req.asp). There does not appear to be a requirement for surgical audit.

In the United States, the American Medical Association (AMA) has responsibility for the rules of designation of credit for CME events.¹²

The Royal College of Ophthalmologists' (RCOphth) specialty framework for revalidation has been approved by the GMC. It comprises the 12 attributes that span the GMC's following four recommended Revalidation domains:

• Knowledge, Skills, and Performance

• Safety and Quality

• Communication, Partnership, and Teamwork

• Maintaining Trust

Recommended tools to demonstrate these attributes include Multisource Feedback, Patient Surveys, Continued Personal Development, Mandatory Training, and Documentation of Compliance with relevant Local Policies/Protocols.

The RCOphth has identified cataract surgery as one area of practice where there is sufficient data from large-scale audit and research on expected outcomes, and complication rates to justify a requirement that cataract surgeons should include such data in their supporting information for revalidation. Posterior capsular rupture is a complication of cataract surgery that is an important risk factor for a poor visual outcome, and also provides an index of surgical dexterity.^{13, 14, 15} However, patient-related risk factors are also an extremely important determinant of the probability of occurrence of posterior capsule rupture.¹⁴ A logistic regression model to adjust observed complication rates for the complexity of cases is currently being evaluated.¹⁶

Ophthalmologists who operate will be required to present one detailed audit of 50 consecutive cases, a record of the total number of procedures performed in the 5-year cycle, and a record of all cases of posterior capsular rupture (http://www.rcophth.ac.uk/page.asp?section=449&sectionTitle =Revalidation+%26+Appraisal). Those performing specialist surgical procedures will also be required to audit their outcomes (http://www.rcophth.ac.uk/page.asp?section=449&sectionTitle=Revalidation+%26+Appraisal).

Retrospective manual auditing of cataract outcome is time consuming and can be avoided with an electronic system, as recommended in the RCOphth's Cataract Surgery Guidelines.¹⁷ Some UK units already have such systems⁵ and the revalidation process may provide impetus for others to procure an electronic medical record. The alternative would be individual trusts designing their own open-source electronic application. Initially the AoMRC and Connecting for Health had hoped to have funding to support such ventures,¹⁵ but their main aim now is to develop systems at a national level. Indeed, the surgical colleges of Great Britain and Ireland are currently piloting such a data set (http://www.rcseng.ac.uk/revalidation/online-portfolio), and the NHS Information Standards Board has finally adopted the Cataract National Dataset^{13, 14, 15, 18, 19} as an NHS-approved full legacy standard. A major problem in the design of a uniform system, however, is the large variety of IT systems across hospitals in the United Kingdom, and thus wide variability in terms of clinical noting and reporting.

Within the background of impending revalidation, we designed this study to determine current knowledge and opinion on revalidation in Scotland. We also sought to establish current practice in cataract and specialist surgery audit throughout Scotland. Additionally, we evaluated the variety of hospital IT systems in place, the current use of electronic recording in cataract surgery, and the factors preventing more widespread practice of electronic recording. Finally we discuss the existing and future possibilities with regard to electronic data collection and cataract surgery.

Materials and methods

In April 2010 we conducted a prospective, cross-sectional, Scottish-wide survey of all senior NHS ophthalmologists (consultants, associate specialists, and staff grades). The mailing comprised a one-page paper questionnaire (Figure 1) with a covering letter and a stamped addressed envelope for return. The design of the questionnaire incorporated feedback and discussion we received at national Scottish meetings, such as the Scottish Ophthalmology and Glaucoma Club, in addition to a conference in Edinburgh on Electronic Connection in Ophthalmology with over 150 attendees. Results were anonymous and were recorded manually on Microsoft Excel for analysis.

Figure 1.

Revalidation questionnaire.

Results

Out of 123 ophthalmologists, 75 (61%) replied within 5 weeks of surveys being posted.

Revalidation knowledge and opinion

Of the 75 ophthalmologists who responded, 25 (33%) felt ready to take part in revalidation, 27 (36%) felt they were unsure, and 23 (31%) felt unprepared. In total, 57 (76%) respondents felt they did not have adequate information about the process, whereas 18 (24%) were happy with what they had received. Also, 53 (71%) respondents did not feel that revalidation would improve patient care, whereas 22 (29%) did. Further, 64 (85%) respondents agreed or strongly agreed that cataract surgery audit is essential for ophthalmic practice, with 7 (11%) being unsure and 4 (5%) indicating that they disagreed with the need for the process.

Cataract and specialist surgery audit practice

In all, 59 (91%) respondents who replied (and who operate) audited their cataract outcomes whereas 6 (9%) did not. Also, 29 (52%) respondents performed continuous audit and 27 (48%) performed sample audit. In total, 43 (90%) respondents audited their cataract surgery outcome in 2009 or 2010, with the remainder (6 (10%) respondents) having last performed cataract surgery audit between 2006 and 2008. Further, 42 (75%) respondents performed their own audit, with 9 (17%) getting their trainee to perform this task, and 4 (8%) respondents got their secretaries, nurses, or optometrists to perform the audit.

In all, 48 (63%) respondents audited their subspecialist surgical results. These were grouped into strabismus (10 respondents, 21%), glaucoma (10 respondents, 21%), macular treatments (8 respondents, 17%), vitreo-retinal (10 respondents, 21%), corneal grafts (6 respondents, 12%), oculoplastics (3 respondents, 6%), and oncological procedures (1 respondent, 2%).

Electronic data collection and existing hospital IT systems

In total, 17 (25%) respondents audited their cataract surgery practice electronically whereas 52 (75%) performed this manually. The electronic methods used were Medisoft, the hospital PAS (patient administration system), and personal databases. Also, 22 (48%) respondents had not changed to an electronic system because of lack of funding. In addition, 14 (30%) respondents cited IT system incompatibility with existing hospital IT systems as a reason for not having changed to the electronic system. There were 5 (11%) miscellaneous reasons. Further, 5 (11%) respondents felt that manual collection of data was adequate.

In total, 36 (48%) respondents knew the type of IT system used in their department, with a variety of seven different systems used across the 14 health boards in Scotland (Figure 2) These are Isoft, Hiss, Topaz, Trak, Meditech, Compass, and eOasis. Further, 9 (12%) respondents electronically collected clinical data other than surgical outcome data using the hospital PAS system.

Figure 2.

Hospital IT systems used in Scotland.

Discussion

Knowledge and attitude towards revalidation

The concept of regular review of the continuing fitness to practice of doctors was suggested in the early 1970s. However, it did not gain real momentum until 1998, following the excessive mortality associated with paediatric cardiac surgery in Bristol. Plans were in place for the introduction of revalidation by 2004, but were postponed following an enquiry into the activities of the serial killer and general practitioner Harold Shipman by Dame Janet Smith. A report by the Chief Medical Officer for England, Sir Liam Donaldson, was followed by a government white paper in 2007.⁴ The formal process of appraisal of doctors leading to revalidation is being piloted in various locations in the United Kingdom during 2010–2011 with a view to general introduction in the second half of 2012.

The RCOphth has decided that audit of cataract outcomes should be undertaken by all ophthalmologists who perform cataract surgery in the United Kingdom as part of their supporting information for revalidation. Our survey suggests that Scottish ophthalmic consultants and middle-grade staff view the general revalidation process with scepticism and appear to have inadequate knowledge of the process. However (and reassuringly), the vast majority strongly agree with the concept of cataract surgery audit. Indeed, approximately 90% performed cataract surgery audit in the year before the survey. Additionally, 63% also audited their subspecialty interest. We accept that one of the limitations of the above results is that this survey was performed in April 2010 when the revalidation process was in its embryonic phase. Since then there have been several publications and meetings on general revalidation concepts by the General Medical Council and the British Medical Association as well as ophthalmology-based revalidation by the RCOphth. We have attached a RCOphth appraisal and revalidation checklist that shows all the components required (Figure 3). This, we hope, will go some way in reassuring the readers that most of the domains are already covered in the current Scottish ophthalmology consultant appraisal. Indeed, we feel that had this checklist been available at the time of our survey, we would have had a more positive response towards revalidation concepts. Further GMC-approved guidance on revalidation is expected in the coming months, which will enable individual colleges to further simplify specialty requirements for revalidation.

Figure 3.

Royal College of Ophthalmology appraisal and revalidation checklist.

IT systems and current collection of cataract surgery data

Although 90% of surgeons audit their cataract surgery outcome, only 25% of these do so electronically. The electronic means have been the use of Hospital IT systems, Medisoft, and personal databases. Medisoft has gained considerable popularity, particularly in southern England. Indeed, it were Medisoft users whose data permitted a detailed analysis of over 55 000 cataract operations for the National Dataset.^{13, 14, 15} Personal databases, while providing a simple solution, are unlikely to be capable of meeting the requirements of NHS policy on Information Governance.

Several of our responders have commented on requests made to their health boards for procurement of commercial electronic medical records (EMR) applications that have been turned down because of funding requirements for both installation and continued IT support. Our survey shows a wide range of hospital IT systems in use across the 14 Scottish health boards that would each require separate episodes of software redesign to allow collection of patient outcome data. A more uniform hospital IT structure across Scotland would have potentially allowed for the design and collection of patient outcome data with easy link to existing patient identity within the hospital PAS system. This would have also incurred minimal finance resource and would be supported by existing hospital IT support systems. Only a very small group (5 respondents, 11%) were of the belief that electronic collection was unnecessary.

At a national level, the authors are currently aware of two cataract EMR systems with the potential for development to a revalidation tool. Medisoft is a commercially available EMR application specifically for use in ophthalmology, and contributed towards data collection for the Cataract National Dataset.¹⁴ The RCOphth is developing a web-based, open-source tool using a combination of a previously designed cataract EMR (Fife cataract EMR) and Moorfield's Eye Hospital's open-source application.²⁰

In September 2010, the Scottish government passed a business case to connect all community optometrists to hospital ophthalmology departments with an NHS connection such as N3 or Sci Gateway. This has the potential for electronically sending cataract post-operative data from community to hospital. It does not take into account cataract surgery case mix, but has potential in the future for this development.²¹

With the advent of revalidation, there is now a greater compulsion on the departments to look at electronic data collection methods to fulfil revalidation needs. The power of electronic data collection of cataract surgical outcome cannot be refuted, as shown by the several publications relating to the Cataract National Dataset.¹³ Electronic collection would also enable the pooling of pseudo-anonymous cataract outcome data against which outcomes of units or individual surgeons (appropriately adjusted for case mix) can be compared in the future.^{13, 14, 15, 18, 19} A National Ophthalmology Database has recently been set up as a repository for such data.

Conclusion

Our national survey shows scepticism and poor knowledge of the revalidation process. However, cataract surgery audit is recognised as essential and is widely carried out. Only 25% of this audit is carried out electronically and the principal reason behind this is a perceived lack of funding, preventing the installation of IT systems to make the process easier.

Revalidation and electronic collection of surgical data will be a process in evolution requiring new ways of thinking and practice within financial and IT constraints that are individual to each department. This survey has outlined current practice and opinion in connection with this process. A repeat survey following the commencement of revalidation may well highlight a change in opinion.

The authors declare no conflict of interest.