Conducting randomised controlled trials in primary care: lessons from an obesity management trial

Abstract

Background

A UK trial ending in 2002 reported that a training intervention to improve the management of obesity in primary care had no impact. Process analysis showed that the intervention was taken up by very few of the practitioners in the participating practices.

Aim

The aim of the current study was to explore both the reasons behind low levels of implementation and the context in which the trial was delivered.

Design

In-depth qualitative interviews.

Setting

General practices in the North East of England.

Method

Interviews with 13 practitioners (GPs and practice nurses) and 10 patients, representing seven of the eight intervention practices in the largest centre of the original trial.

Results

While patients were clear that they had participated in a trial few of those interviewed had any recollection of the intervention. Most staff were positive about the training, resources to use with patients and the weight management model, but not all training needs were met. High initial expectations of the trial gave way to disillusionment, exacerbated by significant misunderstanding by some practice staff of their role in implementation.

Conclusions

Frustration among practitioners managing obesity in primary care combined with unrealistic expectations of and misunderstanding about an intervention designed to improve care in the field, appeared to have resulted in disillusionment with and consequent disengagement from the trial.

INTRODUCTION

Primary care has been cited as a suitable setting in which to facilitate the promotion of weight loss.^1,2 Between 1999 and 2002, a multicentre study in the north of England measured the impact of a brief intervention to improve obesity management in primary care. Results showed that the intervention — a training programme delivered to practice staff by dietitians — had no effect on the weight of patients.³

The training programme was a brief (3 × 1.5 hours) and was based on a standardised programme promoting an evidence-based model for obesity management described in detail elsewhere.³ It encouraged a whole practice approach and regular patient follow up and monitoring. Staff were responsible for implementing the model with patients.

Process evaluation at the time revealed that the intervention had been implemented with, at best, 14% of patients. This follow-up qualitative study was mounted in 2004 with the aims of exploring three research questions:

• reasons behind the low level implementation of the model within the practices;

• the practicalities of conducting a pragmatic trial on obesity management in primary care; and

• general feelings of staff and patients regarding the management of obesity in a primary care setting.

Findings presented in this paper add to the scant literature regarding the implementation of research in primary care.⁴

METHOD

We selected all eight intervention practices in the largest of the trial centres and set out to interview at least one practice nurse and one GP from each. Ten patients were selected reflecting the diversity of trial participants in terms of age group, sex, successful and unsuccessful dieters. Letters of invitation were sent to the patients from our sampling frame in batches of 15. Prepaid envelopes were supplied and patients willing to participate were asked to return their expression of interest. The process was repeated until we achieved our recruitment target.

A topic guide was developed following discussion by the research team, all of whom worked on the original trial (Boxes 1 and 2). Interviews were recorded and transcribed. Two of the researchers independently coded the data using N-Vivo software, and developed a framework as part of the analysis process, so that data could be explored by theme and case. Consensus about representation of the themes arising from the data was resolved by discussion between the two researchers. Explanations for the issues arising were developed by discussion with the third researcher.

Box 1. Interview guide for patients.

• ▸Introductory questions about you, background, how long have you lived in and the area. Ask questions about weight history
• ▸Ask questions about the study: relationships with staff, recollections of anything that changed during study period
• ▸For patients who lost weight ask about weight loss and how it was achieved
• ▸For patients who gained weight ask about support from primary care
• ▸Ask about current support with weight loss, services used in past and ideal place for support

Box 2. Interview topic guide for doctors and nurses.

• ▸Ask questions about the training programme itself: about attendance, scheduling and impact on them
• ▸Ask questions about the model of obesity management being promoted and its practicability in clinic time
• ▸Ask about the resources for patients
• ▸Ask why they think intervention failed to improve weight loss
• ▸Ask about obesity management in general: level of priority
• ▸Ask what they consider an ideal weight management model to be

RESULTS

Thirteen practice staff — nine practice nurses and four GPs — representing seven of the eight practices were interviewed. GPs were more reluctant to participate than practice nurses, explaining in most cases that they had played little or no part in the trial and therefore had no comment to make on it. Ten patients agreed to be interviewed. Data from the interviews confirmed that the obesity management model had indeed been implemented with very few patients and points to a number of inter-related factors which seem to have led to a progressive disengagement with the trial by practice staff.

Communication about the aim of the trial

This qualitative study uncovered a substantial degree of misunderstanding about whose responsibility it was to deliver the weight management model to patients in the original trial. Although the trial protocol stated that it was the responsibility of the practice staff, most of the staff interviewed believed that the research team would maintain some degree of responsibility. In two practices, nurses believed the research team would take complete responsibility for implementation and one nurse expressed her relief at ‘handing over’ the care of the patients at her practice to the research team.

Nurses at two practices described a moment some months into the trial when they received a reminder from the research team about the study protocol and realised that they should still have been implementing the model. In both cases, they believed the study to be far too far long and allowed it to ‘fizzle out.’

How this fits in

Follow-up studies that seek to understand the context and provide explanations for the success or failure of interventions under trial are rare. This qualitative study generates hypotheses concerning the practicalities of conducting research in primary care which extend far beyond those evident from findings of the original trial. It raises issues concerning the scope of the intervention and communication between researchers and practice staff. In addition it illustrates how a clinical topic can exert idiosyncratic and unexpected effects on the conduct and outcome of a study.

The perspective of participating patients

The majority of patients interviewed had no recollection of any change in treatment patterns that might have been associated with the trial: on prompting they did not recall having been seen or weighed more frequently, having behavioural targets set, nor did they recognise the dietary leaflets associated with the programme. Several patients demonstrated a good understanding of the trial protocol and — based on the perceived lack of action — assumed their practice to be in the control arm:

‘I understood that some practices would have some help and some support and other things like that, and some wouldn't have … I suppose like a placebo, you know like in a tablet study. I assume that our practice was one that didn't get an awful lot of support or help, because there wasn't a lot of feedback, or you know, “Come along on a more regular basis and get weighed”.’

Among patients that did notice a change in their care, some recalled short-term increased activity and some seeing the dietary leaflets associated with the intervention. Half of the patients interviewed lost weight during the trial follow-up period but they unanimously attributed this to reasons outside of the study.

In terms of the more general issues around weight loss, all the patients spoke of the difficulty they had trying to lose weight. When asked about the help they received from primary care they expressed concern with the seeming lack of new options for weight loss and a feeling that staff couldn't offer them anything new, concerns echoed by the practice staff (see below). The specific difficulty of understanding nutritional labelling on foods was a commonly raised issue.

Patients felt that discussion about diet was only useful when it was individualised — it was not useful to have very general healthy eating leaflets, or to discuss risk if it was not directly related to an individual. Patients sometimes felt that practice staff applied a very formulaic approach. One woman described the advice given to her overweight daughter by the GP as just ‘stop eating that, that and that’. She felt that childhood obesity has been assigned political importance but was left feeling that her practice was just not keeping up with current ideas about treatment. Another stated:

‘They just address straight away the food and give them [the patients] a diet, and that is just so wrong, it's like writing a prescription before somebody walks in, before they know their ailments.’

On the other hand, most patients seemed to have low expectations of external help with their weight; recognising that the responsibility ultimately lay with them. Many stated that they were very well aware of the principles of weight loss — about what to eat and the need to increase exercise. There was a strong consensus that unless you had the right state of mind, there was little a GP or nurse could do to help.

Staff perception of the training programme (intervention)

Almost all practice staff were positive about the format and style of the training. The resources given for use by both practice and patient received unanimous high praise, the only complaint coming from one practice which said that they had not received enough to meet patient demand.

The content of the training generated a broader range of responses. Most staff described the weight management strategies covered in the training to be a refresher of what they already knew. Two nurses from one practice were sufficiently disappointed by this that they immediately dismissed the study as a ‘waste of time’. Conversely another nurse described herself as having begun with so little knowledge that she ‘could have done with a bit more information.’ Another had found it confusing to have to apply a mathematical formula to work out an energy prescription for patients, admitting: ‘I don't think I understand it even now!’ Personal experience of weight loss also affected staff perceptions of the training, with one nurse acknowledging private doubts about whether the model could work, given her experience of ‘dieting’. She felt the ‘portions [of food] allowed were too generous’.

Significantly, a treatment algorithm — outlining the recommended steps for the implementation of the weight management model and given out at the end of the training package — was not recognised or remembered by any of those interviewed. ‘Well, that just never happened,’ was a comment representative of most participants.

Prevailing attitude of staff regarding weight management

It seems likely that practices participating in the original research had an interest in weight management. Despite this, interviews at most practices uncovered feelings of frustration towards weight management before, during, and indeed since the trial. Nurses frequently described feelings of exhaustion and hopelessness when dealing with overweight patients. Although several interviewees believed weight management should be a high priority, most felt that at their practice — and especially among GPs — it was not viewed as such. ‘Most doctors have no interest in obesity’, agreed one GP, because it is the patient's ‘personal problem’. Another GP said:

‘It's not a medical problem … we really haven't got time to address weight.’

None of the staff interviewed perceived they had any success in their efforts to help people lose weight. In the case of the practice nurses, it was clear that they aspired to respond to each new encounter positively, however, they commonly described short-term achievements with patients, relentlessly followed by weight regain. The frustration this can cause is clear in one nurse's account:

‘I'm the one that struggles with these people, trying … banging your head against a brick wall, seeing them coming back and back … lose three kilos, see them then 3 months later it's back on again.’

Staff felt they quite quickly reached the limits of their ability to help and acknowledged that the hardest thing of all was helping people to achieve long term behaviour change. In some cases they were left with negative feelings towards the patients:

‘… they'll insist they've really, really tried, they don't eat any fast food, they don't eat any sweets or chips or anything of that nature, but they don't lose any weight. So apart from insinuating that they're lying through their teeth, it becomes a stalemate, really.’

‘I probably see weight loss as a little bit tedious … I find the patients' attitudes a little bit — Give me a pill and make me thin! You come up against a brick wall. I find it quite annoying.’

Lack of time to invest in helping people lose weight has arisen in previous studies⁴ and was consistently raised by staff in this study. Nobody we interviewed said they'd be likely to raise the subject of weight loss opportunistically because of time constraints but also because of the fear of upsetting a patient. Similarly, the lack of options available when considering referring patients for more specialist help was commonly cited.

Several nurses mentioned the impact of the latest general medical services (GMS) contract, suggesting that it had reduced the priority assigned to weight management:

‘We've been running a weight management clinic and a slimming group … we've been seeing them here on a regular basis to weigh them and keep the support going. And the doctors have now decided it's a waste of our time — well, it's not a good use of our time. It's the GMS contract. Points do not make prizes where weight is concerned!’

This feeling that the new contract had adversely affected weight loss services was not unanimous with a nurse and GP from two different practices maintaining that it had not altered the care they offered to their patients.

Finally, around half of the staff mentioned that they found weight loss a difficult issue because they were overweight themselves:

‘I struggle at losing weight myself, so I feel such a hypocrite … and to be honest, when I have someone coming to me to lose weight, in the practice, my heart just sinks, cause I think … I'm not brilliant at this.’

Impact of clinical area on conduct of trial

Coming amid this low expectation of success, staff described high expectations of the weight loss trial. Almost all of the nurses described an initial response of enthusiasm and extremely high hopes for the intervention. However, for many this was followed by disappointment, which in some cases had a significant impact on the work of implementing the weight management model.

The main object of the nurses' hopes was for something new. Feeling, as one nurse had put it, that ‘they had nothing left up their sleeve with which to help the patients’, they hoped that the research project would provide them with different techniques with which to tackle obesity. ‘We thought it would be something new, revolutionary, to inspire us,’ said one. Others hoped a new tablet or some new weight loss advice was being developed, because in their experience:

‘… everything that's churned out in diets and weight loss always seems to basically come to the same conclusion.’

GPs seem to have had lower expectations of the intervention from the start. Some nurses described the impact of the GPs' negative attitude toward the study. One told of catching sight of her GP's face during a training session and realising that the doctor:

‘… wasn't that bothered about it, wasn't that interested in it. I remember her looking like “I can't be bothered to be here”, and maybe that influenced us.’

Even in practices where GPs endorsed the study in principle, the nurses said it was clear from the start that they never intended to be involved themselves. One said:

‘It was something for us to do.’

Desperation to help patients appeared to have affected the systematic recruitment of patients to the trial by practice staff: they grasped an opportunity to help patients for whom all else had failed. One GP stated that many of the patients put forward for the trial were the most difficult long-term cases, with very high BMIs, complicating health problems and difficulty exercising. In one practice the GP admitted that when she saw the list of patients who would be participating:

‘My heart sank. I thought, I don't think they'll get anywhere with these patients. I knew they wouldn't hear what was being said to them.’

Almost all the health professionals reported some degree of disappointment with the intervention. For some, this occurred during training, when they realised that the weight management model being offered was not radically different to the approaches they were familiar with. In the longer term, and in those practices that did seek to implement the model, disappointment in the study was more likely to come as a result of failure by the patients to lose weight or keep it off. A GP described her experience:

‘They were very keen to come and lose weight, but then it just slowly, slowly died down.’

Other factors

Additional circumstances that may affect any long trial — staff changes, illness, and a practice split — were all cited as additional reasons for the poor implementation of the model.

DISCUSSION

Summary of main findings

The aim of the intervention in the original trial was to improve weight management services in primary care by improving implementation of evidence-based weight loss strategies. However, the weight loss model promoted in the training was actually delivered to very few patients participating in the trial. Findings from this qualitative study describe a range of factors that contributed to the low implementation of the intervention, some of which are particular to the field of obesity management. These included problems of communication between researchers and practices, difficulties of meeting varied training needs with a standardised training programme, and frustration and disappointment of both staff and patients around the management of weight in primary care.

Strengths and limitations of the study

This study gives insights into the design limitations of the original trial and these are outlined below. We will go on to discuss the limitations of the qualitative follow-on study.

We identified definite problems of communication between researchers and practices resulting in serious misunderstandings about the central principles of the intervention being trialled. We were surprised by the degree of misunderstanding of who carried responsibility for delivering the weight loss model to patients and have examined our procedures in an attempt to explain this finding. On reflection we believe that our introductory written materials and practice recruitment strategy were implicated in this misunderstanding. The introductory flyer was circulated to all practices in the target PCTs and was used to stimulate interest in a trial that was not resourced to provide incentives to practices. On reflection it probably placed too much emphasis on the benefits of participating and underplayed the input of the practice staff, although it was only a minor element of a recruitment strategy that relied primarily on personal contact.

In an earlier trial,⁵ we found that involvement of local practitioners enhanced practice recruitment rates and so recruitment in this trial was conducted mainly by local dietitian researchers. This strategy may have inadvertently misled practice staff into thinking that researchers would be involved in the delivery of care to patients. Our recruitment strategy also allowed practices to sign up following a meeting with a single GP or nurse acting as practice gatekeeper. This may have increased the possibility of misunderstanding about the trial protocol. In some cases, following agreement to participate, researchers were invited back to inform the whole practice team about the study. In other cases the gatekeeper was left to cascade verbal and written protocol information to the rest of team.

It is more difficult for us to understand how intervention practices were left with the misunderstanding about implementation after completion of the training. It could in part be due to the fact that the outcome of the trial was measured using intention to treat which meant staff were not compelled to attend all (or indeed any) of the training and so may have missed the focus of it. It is clear that in some practices, researchers inadvertently raised hopes that other professionals would lighten the workload for a busy practice. This may have been magnified because the researchers in direct contact with the practices were local dietitians.

It is also clear from our findings that there were varied needs in terms of training for the multidisciplinary practice teams. Our standardised programme — while easier to evaluate via a trial — contributed to early disillusionment among some nurses, some finding it too difficult and some too basic. The fact that this intervention was attempting to ensure that best available evidence was being applied in a consistent systematic way and was not introducing a new treatment similarly led to disengagement by some nurses.

Achieving intentional weight loss is difficult and characterised by low levels of success. It appears that the health professionals helping people through this process suffer a sense of failure similar to that of the patients they are trying to help. This desperation led in some cases to practices abandoning the systematic recruitment plan outlined in the trial protocol³ — in favour of helping patients for whom all had else failed.

The qualitative study reported in this paper provides insights into the impact of the trial design and explanations for the low level of implementation of the intervention and go far beyond the findings of the randomised controlled trial. These were hitherto unrecognised by the research team. The study was, however, conducted in only one of the four sites of the trial and the findings may not be representative of all four centres.

In addition, the delay between the end of the trial and the interviews (trial ended 2002 and follow-up interviews were carried out in 2004) may have affected the recall of the research participants. As the main findings from the trial were published in 2003, it is possible that misunderstanding about implementation of the intervention may have been an attribution made by participants to account for their sense of failure.

Comparison with existing literature

Gill and Carter⁶ stated that obesity management occupies an uncertain position in general practice, with different practices, and even different individuals within practices, affording it differing levels of priority. Our findings echo this and the findings of other studies, which found that practitioners have serious doubts about the ultimate efficacy of their work in this field.^1,7,8 In common with other recent work,⁹ we describe a situation in which GPs in particular are reluctant to sign up to weight management — this is despite the fact that a recent study showed that obesity doubled prescribing in most drug categories.¹⁰ This GP reluctance was further evidenced by the fact that so few were willing to participate in this qualitative study. It seems their lack of endorsement impinged on the attitude of practice nurses.

A study by Epstein and Ogden reported that GPs felt patients were too ready to hand over responsibility for weight loss to the healthcare professions.⁹ This perception was prevalent among the staff in our study, however the patients were very clear that the responsibility ultimately lay with themselves.

Implications for future research and clinical practice

This qualitative study provides much needed information on how aspects of trial design can affect the outcome of randomised controlled trials conducted in primary care. Some of our findings are generic and some specific to the difficult clinical area of obesity management. This paper will assist both primary care practitioners in the interpretation of this trial based evidence and researchers designing randomised controlled trials to be conducted in primary care. It adds to the small body of work that uses qualitative research to help evaluate the impact of complex interventions.¹¹

Our findings raise the need for clarity in communication by researchers and perhaps the danger of using a positive spin to enhance recruitment of practices. At the time of practice recruitment it was possible for individual practitioners with a special interest to act as gatekeepers to practice participation in a study and we identify this as a weakness in practice-based research. It also describes the danger of applying a ‘one-size-fits-all’ approach to education in primary care: an approach we used to facilitate evaluation, but which appears to have led to disengagement with the research by some participants in our trial.

Specific to the clinical area, it appears that patients and staff (particularly nurses) feel frustrated by the lack of treatment options and in the case of nurses by the lack of endorsement by their GP colleagues. In our trial these factors appear to have impacted on selection and recruitment of patients and contributed to low levels of implementation of the intervention. There are wide implications here for future research — and for obesity management — in primary care.

Funding body

Funded by the Leeds Primary Care Trusts Research and Development Consortium. The funding body had no involvement in the design, data collection or analysis of this project

Ethics committee

Granted by Leeds West REC on 23 March 2004 (reference 04/039)

Competing interests

The authors have stated that there are none