GP experiences of partner and external peer appraisal: a qualitative study

Abstract

Background

Appraisal is being adopted both in the UK and internationally as a means of aiding personal development for family doctors. However, it is not clear by whom they should be appraised.

Aim

To explore attitudes of GPs towards being appraised by externally appointed GP colleagues and by their own partners.

Design of study

Semi-structured interviews of GPs who had experienced both forms of appraisal.

Setting

Lothian, Scotland.

Method

Sixty-six GPs agreed to take part in a study of partner (n = 46) and external (n = 20) peer-based appraisal. Six months later this group was followed up by questionnaire to determine views of the process, in order to obtain a purposeful sample of 13 GPs who were interviewed in depth.

Results

We uncovered concern and a need for clarity about the linkage of appraisal to revalidation. Interviewees felt that the potentially charged nature of appraisal could lead to collusion between appraiser and appraisee, which may lead to a superficial engagement. Similarly, lack of local knowledge of an appraisee potentially enabled a strategy of avoidance. GPs opting for partner appraisal were less likely to undergo appraisal due to lack of protected time.

Conclusion

There are reported advantages and disadvantages to having an external peer or partner appraisal. The relationship between revalidation and appraisal needs to be clarified as this leads to collusion and avoidance strategies by both appraisers and appraisees. Good training is required to both recognise and address these strategies. Protected time is essential for effective appraisal.

INTRODUCTION

During the 1990s, the UK medical profession increasingly recognised the need to demonstrate that doctors are fit for practice.^1,2 However, this process accelerated following a series of national scandals,^3-5 and a subsequent need to ensure public confidence. In the 1990s evidence accumulated of low morale within general practice,⁶ and a need to support professional development of GPs throughout their careers.⁷ In 1997, the General Medical Council published its draft policy on revalidation.⁸ At its core was the requirement for GPs to have annual appraisal. Although UK guidance states that GP appraisal should be carried out by a practising GP, there is no indication if this should be carried out by a partner or an external colleague. A 2002 study in South East Scotland⁹ exploring the attitudes of GPs to annual appraisal indicated general support for its introduction. The most acceptable appraisers identified were externally appointed GPs from another practice, or a peer partner from the same practice. Two local healthcare cooperatives (LHCCs) in Lothian were interested in piloting these approaches; and so an opportunity arose to compare the experiences of both groups.

The aim of the study was to explore GPs' perceptions of the advantages, disadvantages, threats and opportunities of being appraised by externally appointed GP colleagues and by their own partners.

METHOD

Participants

GPs in the 2002 study⁹ who had agreed to further research were approached. A total of 46 GPs in nine practices in the LHCC opting for partner appraisal (where GPs would both appraise and be appraised), and 20 GPs from 10 practices in the LHCC opting for external appraisal, volunteered. Five GPs from a different LHCC were recruited (by advertisement) to appraise this latter group and were offered sessional fees for training and for each appraisal. The numbers in this group were limited by funding. All appraisers and appraisees undertook a 1-day training course, based on NHS Education for Scotland's appraisal system,¹⁰ including role-play of the appraiser, appraisee and observer roles. Appraisals were planned for the following 2–3 months. Six months later, participants were contacted by questionnaire, which explored the content and satisfaction with the process of their appraisal, and a sub-group of responders to this questionnaire agreed to an indepth, semi-structured interview.

How this fits in

In the UK and other countries, appraisal is being adopted as a means of aiding GP personal development. However, it is not clear if this is best carried out by an external appraiser or by a partner colleague. Responders in this study who had experienced one of these types of appraisal expressed concerns that the threat of a linkage with revalidation has created the risk of collusion between partners and avoidance strategies with external appraisers, who may lack local practice knowledge. Good training is required to both recognise and address these avoidance strategies.

Semi-structured interviews

Interviews took place of a purposive sample of 13 male and female GPs representing a range of ages and views as expressed in the questionnaire. Questions were used as a guide, and covered views about appraisal in general, training, organisation, suggested changes and experience of the process. Particular issues arising from the questionnaire concerning the management of difficult problems and the time set aside for and location of the appraisal were also included. Interviewees were encouraged to discuss what they felt was important with as little prompting as possible. Anonymity was guaranteed.

Analysis

The interviews, which lasted 45–75 minutes were recorded at the GP surgeries and transcribed verbatim. Analysis was carried out by hand using content analysis to look for broad emergent themes¹¹ and how these themes related to the type of appraisal undertaken. A random sample of transcripts was independently read by another researcher to verify the identification of emergent themes. Interviews were stopped after data saturation was achieved.¹²

RESULTS

Eighty-two per cent (n = 54) returned the questionnaires, and of these 59% (n = 32) had actually had an appraisal. Appraisals were more likely to have occurred in the external appraiser group (external, n = 17/20, 85%; partner, n = 15/32, 47%; p<0.05 χ² test). The most common reason cited for not carrying out appraisal was a lack of time.

Semi-structured interviews

Thirteen people were interviewed (five from the partner group, five from the external group and three external appraisers). Data saturation was reached after 10 interviews.

The major themes arising in the interviews were: time to prepare for and undertake appraisal; location; the nature of the appraiser (particularly in relation to peer versus external appraisal but also, practice dynamic, age and sex issues); the relationship with revalidation including the uses of the outcomes of appraisal and confidentiality; and the difficulties in relation to possible collusion and lack of challenge. Pervading these issues was a sense that the process was just beginning and that it would take time to bed down into a final form. As a number of these themes reiterate previous research,^9,14,15 we chose to focus here on: areas where the experience of partner and external appraisals differed; the difficulty in arranging partner appraisals; the influence of location; how the relationship of appraisal to revalidation, practice dynamic and choice of appraiser influenced collusion; and avoidance behaviour.

Time

While both groups discovered that finding protected time for reflection and to undertake the appraisal was challenging, those who felt most time constrained were from the partner appraisal group, and this led on occasion to appraisals not taking place:

‘I did do it [preparation], but at the 11th hour admittedly.’ (Partner appraisee [PA].)

‘I felt really uncomfortable about that [the timing] and I had to cut him short.’ (PA acting in appraiser role.)

Location

The issue of location was more important in the partner-appraised group, as the appraisal frequently took place in the appraiser's consulting room, creating feelings of a power imbalance:

‘I was appraised in his consulting room … I didn't feel it was the optimum … all the things about it being his territory and me sitting in the patients' chair and stuff like that.’ (PA.)

Most who commented preferred the appraisal to take place in a neutral environment, but this was not universal:

‘Sitting on low, comfy chairs with a low table has just the same problems as sitting in someone else's consulting room in front of a desk … there are pros and cons of doing it different ways.’ (External appraisee [EA].)

The purpose of appraisal and the consequences of its relationship to revalidation

Participants in both groups felt that the relationship of appraisal to revalidation affected the process, leading to caution in what was discussed or promised in the way of development plans:

‘However you sell appraisal, it's never going to be a comfortable thing for people because … it's going to be linked to revalidation … if you don't become revalidated then it's a big hassle.’ (EA.)

‘I was deeply suspicious of it and thought it might be some sort of stick to beat me with.’ (PA.)

One external appraisee admitted that she had set easily obtainable objectives so she was ‘going to look rosy at the next appraisal’.

There was also concern about how information that has been gathered during an appraisal should be shared:

‘Confidentiality was untidy. I wasn't sure if I had somebody that I was meant to be reporting back to … or if it remained with me … what happens if you find people who are failing … what is the appraiser meant to do? … That hasn't been spelled out in any way.’ (External appraiser.)

The impact of choice of appraiser and practice dynamic on collusion and avoidance

There was disagreement about whether a GP appraiser should ideally be from within the practice or external. Issues raised in relation to this included lack of local/personal knowledge; the impact of current practice dynamic, for example health, work efficiency or personality issues with or between partners; the particular difficulties surrounding appraising more senior partners; and the resultant impact of these issues on evasion or collusion in the process. Members of both groups had mixed feelings and were able to see advantages and disadvantages of both methods.

Some responders thought external appraisers who lacked local knowledge of appraisees (and any work-related problems they may have) were in danger of not realising that discussion of serious problems was being evaded. However, this was balanced by the belief that partner appraisers might collude with appraisees to avoid discussing problems that were too uncomfortable.

Partner appraisees thought local knowledge of the practice's problems was very important, and that feedback to partners by someone with local knowledge would be more likely to be effective:

‘They [external appraisers] don't know the problems that happen day to day in our practice … but my partners who know the day-to-day running of things have more constructive suggestions and I think will actively help me if they know what my problems are.’ (PA.)

However, in the partner appraisal group some thought tackling difficult areas with partners was too problematic:

‘Trying to deal with some of the behaviour (e.g. motivation and interaction within the practice) in the appraisal process is a bit tricky because that is personally threatening.’ (PA.)

‘Saying everything is wonderful because they are too terrified to say it is not.’ (PA.)

‘The first thing my appraiser said was “well we accept that … there are no problems or anything, so how do you feel?” and that's how we started.’ (PA.)

Collusion was not confined to partner appraisal. Here, an external appraiser realised that the appraisee did not wish to discuss health issues that she thought may have been affecting his work:

‘I suppose in an appraisal there are certain issues that we should have touched upon which you can avoid … I felt it was very important to respect his wishes but wondered afterwards … what mechanism there was or how one should deal with it.’ (External appraiser.)

Some external appraisers recognised that the appraisees were occasionally ‘playing the system’ and avoiding issues:

‘There was definitely intentional digression in a couple … the games people play, we were both doing it … both elements are just as able to manipulate the direction of appraisal.’ (External peer appraiser.)

For some (mentioned by both groups) there was an advantage in an external appraiser looking freshly at the appraisee with no preconceived ideas:

‘It's helpful if someone comes from outside, it takes it out with the practice and can be more reflective and they can give you ideas.’ (EA.)

Most in the partner group felt positive about appraising each other, and some that the presence of their appraiser made compliance with development plans more likely:

‘It empowered us to know a lot more about what's going on in each other's lives.’ (PA.)

‘Because my appraiser is here she actually is a constant reminder.’ (PA.)

Some suggested that external appraisers who appraised a whole practice would get more of a handle on individual problems of partners that would build up over time and could constructively feed into a practice plan. They could also be used for ‘bouncing ideas off’:

‘[The appraiser] would say, “Well you know, here are a few concerns I've picked up, what do you think of that?” … almost have a two-way process … I think that would be potentially more fruitful and I think it could be built on year on year.’ (PA.)

Age and sex

A few responders mentioned the difficulty of connecting with the appraiser. These experiences occurred in both groups when there was no choice of appraiser. They felt that there was a generational mismatch, not with regard to age but more to do with experience of general practice:

‘It was like talking to my Dad.’ (EA.)

‘… feeling of, “Well do you actually know my life or why I can't make the changes the way you might make changes?” or whatever.’ (EA female appraised by male.)

Another GP spoke of having a younger appraiser and the difficulties of realism versus idealism:

‘It's easy to tell everybody what they ought to be doing when you are young … [when you are older] you realise that you've got to temper your ideals with a bit of realism.’ (EA.)

Senior partners were a particular problem for partner appraisers:

‘The most junior partner drew the most senior partner … how is he appraising someone who knows volumes more? He felt really awkward.’ (PA.)

A system in evolution

Most appraisees recognised that the appraisal process was in its early stages and was likely to evolve into something more robust:

‘You can't push it too far too quickly, otherwise you put these things in jeopardy. But … once it is in … bedded down, then you start to screw the process a little bit.’ (PA.)

Many suggested that the first appraisal was the most difficult as it was new and required time to get to know the appraiser. They felt a second appraisal would start with previous information and objectives and therefore be more productive:

‘You do need to practise it to get good at it.’

DISCUSSION

Summary of main findings

This study compared the experiences of GPs undergoing peer partner and external appraisal. Both groups were confused as to the purpose of appraisal, particularly in relation to revalidation. They saw the revalidation link as confusing the purpose of the appraisal. They also believed it encouraged ‘gaming’ and evasion in externally appraised systems and collusion in internal systems. Participants perceived that there were advantages and disadvantages to both methods of appraisal. Partner appraisal had the potential to help strengthen relationships in partnerships as partners were felt to be more likely to be aware of the strengths and weaknesses of colleagues. However, it was recognised that, for some GPs, the opportunity to confide in someone outside the practice was more beneficial, particularly if the GP was being appraised by a partner with whom they had difficulties. There may be a case for both types of appraisal to occur alternately to obtain the benefits of both.

GPs believed it was important that they felt comfortable with their appraiser. Age and sex match for some is important and systems might do well to have some element of choice of appraiser built into them. However, having one appraiser appraise a whole partnership may give that person a more accurate handle on the strengths and weaknesses in the group.

There were also concerns about how difficult issues, such as workload distribution, might be dealt with once uncovered. The use of facilitated practice ‘away days’ (possibly by an appraiser), were seen as a solution by some. In terms of dealing with difficult appraisals, a support network for appraisers was also suggested.

Protected time for the appraisal was very important. It is worrying that in the partner appraised group, even in a volunteer population (all of whom had found time to attend training) many appraisals did not happen. There is a need, therefore, to ensure that all parties have both protected time and adequate funding to spend an appropriate amount of time on preparing and undertaking the appraisal.

Strengths and limitations of the study

This study reports the views of a small group of GPs who had volunteered to undergo appraisal. Many of them had already made use of opportunities to reflect on this practice and had identified personal development needs. A less enthusiastic group of GPs might have different views of the appraisal process. In addition, this was, for many, their first experience of appraisal. GPs were aware of the evolving nature of the appraisal process, recognising that as appraisers and appraisees become more comfortable with the process it is likely that more difficult territory will be covered. However, it is not clear if subsequent experience will diminish or promote avoidance and collusion strategies. Despite these limitations, the study identifies some strengths and weaknesses in partner and external peer appraisal, which are likely to be of general relevance.

Comparison with existing literature

The information gathered from the interviews reinforces the findings obtained in our previous questionnaire study on attitudes to appraisal⁹ and those of other researchers;¹⁴ in particular: that participants remain unsure as to what appraisal is for;¹⁵ that there should be clarification of the specific role of the appraisal interview in relation to revalidation; and that the appraisee must feel comfortable with the appraiser.¹⁵

Implications for research and practice

Appraisal is an issue that most healthcare systems in economically developed countries will need to grapple with. This UK-based study shows that there are concerns about the linkage of appraisal to revalidation. This linkage may lead to risks of collusion and ‘gaming’ by unengaged appraisers and appraisees, and might diminish any governance effect the process was meant to have. Good appraiser training and selection is required to overcome this, as is clarity over ownership of the appraisal process. GPs in systems based on partner appraisal were less likely to undergo appraisal, or for the appraisal to seem rushed, therefore particular care is required to ensure adequate preparation and appraisal time. Participants were aware that with experience the appraisal process is likely to improve, and were optimistic about its long-term future.

Further research into the experience of GPs undertaking different forms of appraisal is required to confirm the findings from this motivated sample, and to explore methods of preventing collusion and evasion during the appraisal process.

Funding body

Study supported by grants for West Lothian Primary Care Trust and North Lothian LHCC, and resources from NHS Education for Scotland. Brian McKinstry is supported by the Scottish Chief Scientist Office

Ethical approval

At the time of research, ethical approval in Lothian was not deemed necessary for interviewing staff members

Competing interests

All three authors were or are employed by NHS Education Scotland. This organisation, following commencement of this project, took on as one of its responsibilities the organisation of GP appraisal in Scotland. Johnstone Shaw has, since this research was carried out, become responsible for the implementation of GP appraisal in Scotland