A multidimensional conceptual framework for analysing public involvement in health services research

Abstract

Objective  To describe the development of a multidimensional conceptual framework capable of drawing out the implications for policy and practice of what is known about public involvement in research agenda setting.

Background  Public involvement in research is growing in western and developing countries. There is a need to learn from collective experience and a diverse literature of research, policy documents and reflective reports.

Methods  Systematic searches of research literature, policy and lay networks identified reports of public involvement in research agenda setting. Framework analysis, previously described for primary research, was used to develop the framework, which was then applied to reports of public involvement in order to analyse and compare these.

Findings  The conceptual framework takes into account the people involved; the people initiating the involvement; the degree of public involvement; the forum for exchange; and methods used for decision making. It also considers context (in terms of the research focus and the historical, geographical or institutional setting), and theoretical basis.

Conclusions  The framework facilitates learning across diverse experiences, whether reported in policy documents, reflections or formal research, to generate a policy‐ and practice‐relevant overview. A further advantage is that it identifies gaps in the literature which need to be filled in order to inform future research about public involvement.

Introduction

In 1999, the Health Technology Assessment (HTA) programme commissioned primary research and a literature review about the advantages and disadvantages of different methods of public involvement in identifying and prioritizing topics for research. This reflected a growing general interest in participatory research ¹ and ‘citizen science’, ² and the application of these approaches in countries such as the UK to patient or public involvement in research activities such as ethics committees ³ and randomized controlled trials. ⁴ The methods and substantive findings of the HTA‐commissioned systematic review of public involvement in research agenda setting have already been published. ⁵ The aim of this paper is to reflect on the development and utility of the conceptual framework we developed for that review. Framework analysis was originally developed as a method for analysing primary data in applied social research. ⁶ We describe here how the same principles were employed in a systematic review to label the ‘data’ of research reports in meaningful and manageable sections for subsequent retrieval and exploration. The result is both a general method for systematically reviewing diverse literature, and a framework for analysing public involvement in research in particular.

In their review of the literature on public involvement in health research, Boote et al. ⁷ pose four key questions (i) how can public involvement be conceptualized? (ii) how and why does public involvement influence health research? (iii) how can this influence be measured and evaluated? and (iv) what factors are associated with successful public involvement in health research?

This paper addresses all four questions and demonstrates how the development and implementation of a conceptual framework is a useful analytic tool in appraising what is known about public involvement in research agenda setting.

This paper focuses on the involvement of people whose primary interest in health‐care is their own health or that of their family, as users of services or carers; and people representing these groups through community organizations, networks, or campaigning and self‐help groups. These people have variously been called ‘patients’, ‘users’, ‘consumers’, ‘lay people’ and ‘citizens’. Each of these terms, when linked with participation, has implications for a service that claims to inform its decisions with evidence. ‘Consumerism’ implies customer choice, where information is required to inform that choice whether the customers are patients, service users or their families and carers. Public involvement invokes accountability and the need for evidence about performance to be available for public review. Consumer or public uses of evidence invoke a fundamental question about the nature of the evidence: who decides what evidence to seek and how to seek it? A discourse about ‘citizens’ invokes the right to be involved in the generation of evidence as well as the planning and review of services. ‘Patient’ and ‘public’ involvement in decisions about their own care, or reviewing services, is meaningless if, in an evidence‐informed culture, the evidence is irrelevant to either group. In this paper, we use ‘public involvement’ as an inclusive term, and the term ‘lay’ to distinguish people from research and service professionals when this is necessary for clarity.

Our interest is in the involvement of people, other than researchers and service professionals, in guiding what research is done, and how it is done. This is markedly different from the traditional, more passive, role of research subject who may or may not consent to participate by providing data. The distinction has been described by INVOLVE (established by the Department of Health to promote public involvement in research) as doing research with, or by, the public rather than to, about, or for the public. ⁸

Key concepts relating to public involvement in research

In an early attempt to characterize public involvement in developed countries, Arnstein ⁹ proposed a ‘ladder of participation’ with rungs on the ladder representing increasing degrees of participation: from non‐participation or manipulation and therapy; through the tokenism of informing, consulting and placating; to citizen power through partnership, delegated power and citizen control. A similar scale has been described drawing on participatory processes for research in the developing world. ¹⁰

Other authors have distinguished involvement methods by considering who took the initiative, or the underlying theories. A two‐dimensional representation of models of involvement combining degrees of involvement (from information to control) with a distinction between individuals and organized groups has recently been employed to describe public involvement in mental health services in England. ¹¹ Mullen et al. ¹² have noted that an important factor shaping how public involvement is viewed by health professionals and by different sectors of society is who initiates it. Theories about learning organizations highlight the importance of inclusion, reflection and managing diversity. ¹³ , ¹⁴ Characteristic features of participatory research, which probably requires the greatest changes in organizational structures and procedures, ¹⁵ include collaboration, mutual education, and acting on results developed from community‐relevant research questions.

Seeking a conceptual framework

The conceptual framework described in this paper built on our own experiences of public involvement, and on a detailed and sensitive search strategy to identify, then appraise and analyse relevant literature.

Personal, practical experience

All the authors came to this study with practical experience of either developing or researching public involvement in research in the statutory or voluntary sectors. Between us we had experience of lay‐controlled research, public consultation, and working collaboratively both as researchers and as lay people. We had invited public involvement in research, and accepted such invitations, both with and without support and training. These experiences necessarily affected our interpretation of the literature. In particular, we appreciated how the fine details of involvement methods can be crucial to the practicalities of implementation. For instance, formal methods for facilitating multidisciplinary groups generally perform better than informal ones, and aspects likely to be important are ensuring that everyone’s voice is heard, and that all options are discussed. ¹⁶

Literature searching and analysis

Our literature searches across medicine, nursing, social welfare and the social sciences covered reports published in academic journals and patient‐ and carer‐group publications, and unpublished reports accessible through professional and patient or public networks. ⁵ As we accumulated literature, we followed an iterative process similar to framework analysis as applied to primary research data. ⁶ This involved familiarization with the literature; gradually developing a conceptual framework using concepts embedded in the review question, and in the theoretical and empirical literature; applying the framework systematically to the data; and subsequently rearranging the data within the framework in order to construct a chart for each key dimension with distilled summaries from all relevant documents identified. The charts were then used to map the range and nature of involvement and find associations between themes with a view to providing explanations for the findings. We describe below the key concepts used to build our framework, and how these were combined so as to enable us to analyse and draw conclusions from the literature.

Categorizing the literature

As well as reports of public involvement, we found discussion papers about differing priorities, conceptual frameworks, community equipoise, power, democratic practice and advocacy. These papers drew on a wide range of theories, and described episodes of involvement in many different research areas. Research programmes have tended to employ organizational change theories, ¹⁷ while lay groups taking the initiative are better described in terms of collective social action and social movement theory. ¹⁸ The health topics covered in the literature included different health conditions (asthma, breastfeeding, cancer, cystic fibrosis, dental health, diabetes, disfigurement, HIV, hyperactivity, learning difficulties, mental health, physical and complex disabilities); populations (older people, young people); interventions (physiotherapy, organ transplants, wheelchairs and other assistive devices); and settings (homelessness, occupational health, school health, urban health). The papers came from different disciplines (agriculture, genetics, environmental studies) and reported different research methodologies (systematic reviews, technology assessment, participatory research).

Reports of public involvement were first characterized as ‘purposeful’, ‘opportunistic’ or ‘indirect’. Public involvement in identifying and/or prioritizing research topics was defined as ‘purposeful agenda setting’ to distinguish these from reports of public identification or prioritization of research topics while undertaking related work (e.g. health service development, voluntary sector development, campaigning). The latter we termed ‘opportunistic agenda setting’. ‘Indirect’ public involvement referred to situations where priorities were deduced from other activities, such as clinical encounters, where patients’ questions, decisions, use of services or treatments, or complaints, were used to inform research. In ‘direct’ public involvement, communities identified their own health needs and took action to address these. ¹⁹ , ²⁰

INVOLVE have condensed Arnstein’s ladder of participation ⁹ into three steps: ‘consultation’, ‘collaboration’ and ‘lay control’. ⁸ We anticipated finding examples of all three of these in the literature on research agenda setting and so adopted the categories for our framework. INVOLVE defined ‘consultation’ as asking lay people for their views and using those views to inform decision making. For example, funders of research have held one‐off meetings with people to ask them about their priorities for research, or written to people in accessible terms to invite their views. These views were not necessarily adopted, although they may have informed decisions. ‘Collaboration’ was interpreted to mean an active, on‐going partnership with lay people. For example, people who use services have been committee members or collaborated informally to complete a task. ‘Lay‐controlled research’ referred to people designing, undertaking and disseminating the results of a research project, in which professionals were only involved by invitation.

A further issue is that of the forum used for public involvement. Different forums for engaging people in collaborations or consultations fit different purposes; for example, qualitative methods, such as one‐to‐one interviews, focus groups, town meetings and citizens’ juries, are useful for eliciting ideas, whereas quantitative questionnaires for ranking or rating, are more appropriate for priority‐setting. ²¹

A conceptual framework of public involvement

Drawing on the concepts described above, our framework was based on three critical dimensions: whether lay people were involved as individuals or as members of organized groups; whether public involvement was at the invitation of the research programme or in response to action by the lay public; and the degree to which the public was involved (consultation, collaboration and lay control).

Figure 1 shows the relationship between these dimensions in shaping the framework we developed to describe public involvement in research agenda setting. The figure combines a simplified version of Arnstein’s ladder of involvement, ⁹ as operationalized by INVOLVE, ⁸ and Mullen et al.‘s ¹² distinction between ‘reactive’ and ‘proactive’ public involvement.

Figure 1.

 Framework for describing consumer involvement in research agenda setting, including forums for Type B.

Columns 1–4 represent (from left to right) decreasing public involvement. Rows 1–4 represent (from top to bottom) decreasing researcher commitment to public involvement. Some cells in the matrix shown in Fig. 1 remain empty because they represent scenarios that are illogical, imply minimal engagement between the two groups, or were simply not found. For instance, lay controlled activity was unlikely to be initiated by research programmes (column 1, row 1), and research programmes cannot consult or collaborate with other people if they are minor partners or absent (row 4, columns 2 and 3). Although one might envisage minimal public involvement resulting from invitations from researchers (rows 1 and 2, column 4), we found no reported examples.

Ideas can be elicited in various forums (e.g. committee membership, written consultation, focus groups or public meetings) with either formal or informal methods for building consensus or other decision making. Figure 1 includes the forums for type B (written consultations, face‐to‐face consultations such as interviews, multiple face‐to‐face consultations such as a series of meetings, or a combination of written and face‐to‐face consultations). We therefore needed to consider specific interactions between research programmes and the public in terms of both the approach to engagement and the operational methods of involvement. We also tabulated the context (the research focus, the historical, geographical or institutional setting), the underpinning theories, and the selection or self‐selection of lay participants. For instance, Fig. 2 tabulates examples of public involvement through participatory social research with Californian young people, ²² an Israeli academic training programme for older people ²³ and a Japanese opinion poll about in vitro fertilization, genetic engineering, biotechnology. ²⁴

Figure 2.

 Examples of public involvement.

Evaluating public involvement

Advantages and disadvantages

The main reasons for public involvement in research are political mandate and the pursuit of ‘better’ research. ²⁵ Political mandate may be satisfied by a tokenistic public presence during the planning process. Key measures of whether people are meaningfully involved are whether they feel empowered or influential, and whether their involvement leads to actual change. ²⁶ ‘Better research’ could mean methodologically or ethically superior research, or research that is more relevant and thus more likely to influence practice and improve health in ways seen as important by service users. These are final benefits or, if not met, final harms and costs (such as lack of research, poorer research, opportunity costs and bad publicity).

Public influence of research may be evident in the choice of research topics; the control of the research design, conduct and funding; the background and accountability of the research; and the potential benefits and subsequent spread, and use, of research findings. ²⁷ For instance, plans for a conventional survey by a researcher of disabled people were challenged by the service users obtaining funding and directing a piece of research themselves, with the help of a professional researcher as a consultant, thereby controlling the presentation of the findings and their use in future decisions about funding. ²⁸

On the basis of the collective experience of lay people, practitioners and researchers, INVOLVE has listed the advantages and disadvantages of different degrees of involvement. ⁸ For example, consultation has the advantage of obtaining lay people’s views quickly, but without necessarily a commitment to acting on them; it is regarded as a ‘safe’ way to initiate the process of working with them. A disadvantage is that some people find it frustrating to be asked their views without any commitment to subsequent action. Collaboration, as a type of public involvement, is seen as increasing the likelihood of accessing research participants; involving lay people to help with recruitment and informed consent and interpreting and understanding data; and of outcome measures and assessment criteria being relevant to lay participants. People involved in collaboration are more likely to feel ownership of research results and be more active in disseminating these. Disadvantages of collaboration include additional time and costs and some loss of control over the research on the part of researchers. Lay‐controlled research has the perceived advantage of reaching marginalized groups; addressing questions that researchers may not consider important; and developing research skills among lay people and the professionals with whom they work. With this kind of research, there is often greater commitment from public organizations to disseminate research findings, but this type of involvement requires researchers to hand over the ownership of a project to other people.

Methods for evaluating public involvement can draw on literatures about public engagement with science, community involvement and action research. These literatures evaluate involvement: organizationally (leadership, structures, resources and attitudes) ²⁹ within communities ³⁰ ; procedurally ³¹ , ³² ; interpersonally (fairness and competence) ³³ ; and in terms of impact (such as mutual learning and civic agency, ³³ or influencing the research agenda ³⁴ ). A systematic review of the impact of patient and public involvement on UK services collated evidence about the mechanisms and outcomes in terms of structure and resources, politics and discourse, and attitudes and culture. ³⁵ Addressing the newer field of public involvement in research agenda setting, where detailed reports were rare, we chose two measures of impact that related directly to the review question (records of lay priorities, and records of reflections and lessons learnt) and aligned our work with participatory approaches for mutual learning, reflection and change. ³⁶

Applying the framework

Formal research of public involvement was rare. The literature was replete with enthusiastic reports and reflections but with little or no detail about public involvement, and often little attempt at objectivity. In appraising reports, we noted the achievements and challenges of each type of involvement (consultation, collaboration and lay control), and considered their characteristics across the framework’s dimensions (who took the initiative; involvement of individuals or groups; the forum of exchange; methods for decision making) and whether these led to records of lay priorities or lay influence on ways of working.

Descriptions of all the reports, and the methods and findings of the review have been published in full elsewhere, ⁵ and are available free at http://www.ncchta.org. Key findings about public consultation and collaboration are summarized here alongside illustrative references. Analysis of reports, using the framework, found that none of its features guaranteed public influence of research agendas, but nor did any preclude it. Different methods had varying degrees of success in a range of contexts.

With consultation, more was learnt where lay people were more directly involved in the debate, for instance through an iterative Delphi study ³⁷ or face‐to‐face meetings, ¹⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² or encouraged to participate by an independent facilitator, ⁴³ or were even leading the debate. ³⁸ Least was learnt from simple written consultations. ⁴⁴ Reflections on the methods for agenda setting sometimes neglected to consider the methods for public involvement, and opportunities for shared learning were lost. ²⁰ , ⁴⁵ In particular, organizational change approaches captured lay people’s ideas and priorities or lessons about public involvement only when reflection on the agenda setting exercise specifically addressed public involvement. ¹⁷ , ³⁸ , ³⁹ , ⁴³ , ⁴⁴ Working with established community groups was feasible, although it could be time consuming. It provided a route for gauging local opinion from broad sections of the community, including people who were not activists and people who may be unfamiliar the English language. ⁴¹ Working with lay people whose shared interests were not health focused faced difficulties with lack of attendance, lack of understanding and lack of commitment. ⁴⁰ , ⁴¹

Little is known about how lay people view consultation of individuals because all of these documents have been written from the research perspective, with the exception of a report of consensus conferences, ⁴⁶ which has been written from both research and lay perspectives. Even less was learnt about consulting individual lay people in the UK as all these reports are from abroad. Commercial interests that involved lay partners were particularly successful in capturing and making use of their ideas and priorities. Their motivation for investing in consumer involvement was a greater market share through more advanced technologies, and a greater profit. ⁴⁷ , ⁴⁸ , ⁴⁹ , ⁵⁰ , ⁵¹ Least was learnt when research programmes perceived consumers primarily as users of research. ⁴⁶ , ⁵² Opinion surveys gave a broad but shallow picture of attitudes, perceptions of benefit and harm of research, and limited data about research priorities or reflection about the process of involvement owing to the ‘closed’ questions. ²⁴ , ⁵³ Patients recruited in clinical settings have been interviewed to identify interventions ⁵⁴ and outcomes ⁵² , ⁵⁴ to frame evaluation agenda. Involving individuals with personal experience of problems has sometimes added emotive and persuading language, and could be particularly thought provoking for researchers. ⁵⁰ , ⁵¹ , ⁵²

With respect to collaboration, collaborative committees, when working well, provided a structure for lay members to introduce a broader range of views by consulting their peers. ³⁸ , ³⁹ , ⁵⁵ Such committees facilitated democratic processes, openness, appropriate choice of members, and support and training for all involved. Lay members needed to be informed about the technical and organizational background of the work. ¹⁷ , ³⁸ Collaborations needed to confront tensions and benefited from careful management. ⁵⁶ Lay perspectives were reported with greater confidence when a range of methods had given consistent views. ⁵⁷ As with consultations, more was learnt about lay priorities and methods for identifying them when programmes were required to reflect on the methods for incorporating their perspectives. ⁵⁷ Public involvement was furthered where lay people were involved as partners in reflecting on working practices. ⁵⁵ , ⁵⁷ Other examples of committee membership were either reported with no reflection on the methods they employed, or with insufficient detail to be included in the review, thereby raising concerns about possible ‘tokenism’.

Involvement shared a number of common characteristics when research programmes invited individuals, rather than organized groups to collaborate. Training, education or ‘knowledge transfer’ was an integral part of all these examples. This exercise required investment of time, resources and personal relationships but there was the motivation of finding research‐based answers, not just identifying the questions. ²² , ²³ , ⁵⁸ , ⁵⁹ Topics for research were identified by lay people who were empowered to learn from the collaboration. ²³ , ⁵⁸ Participative training methods provided a learning experience for trainers too, and sometimes changed their attitudes. ²⁴ , ⁵⁹

The third model of lay control has been used in many different areas. Working independently, lay people have both called for and conducted research. However, our review suggests this is the least formally developed area of public involvement in research agenda setting.

Reflections on the framework and its value

The framework we developed and have described in this paper is comprehensive in its scope. It accommodates diverse methods for public involvement in research, operating at international, national, regional and local levels, in urban and rural areas, stretching across the developed and developing world, and in contexts that differ widely in terms of topic of interest, organizational structures and underpinning theories. The framework draws together examples of public involvement that share fundamental principles, but that have developed in very different contexts (such as participatory research with Californian young people and with farmers in developing countries). ²² , ⁶⁰ It distinguishes between variables operating at different levels; at initiation, and subsequent choice of participants, forum and decision‐making processes. It juxtaposes public involvement methods that share characteristics along one or more dimensions. Application of the framework facilitates the identification of general trends (such as collaborative relationships being more productive), and highlights exceptions (such as tokenistic committee involvement) to deepen our understanding of public involvement processes and increase progress towards more productive methods of public involvement.

Categorizing the different methods (A–H in Fig. 1) showed that top down type A methods alone (commonly operationalized by large research programmes as committee membership) achieved little, while bottom‐up type C (collaborative strategies with individual consumers) achieved a lot, but only for small scale research (typically participatory research). A particularly fruitful method for involving the public in setting large‐scale research agendas appears to be a combination of collaboration and consultation, with lay people taking leading roles in consulting their peers. Another advantage of our framework is that it exposes important gaps in the existing literature, most notably about methods for collective decision making, ¹⁶ which are rarely reported in detail.

The task of developing the framework called for new methods for systematically reviewing diverse literatures. As with framework analysis applied to primary research, ⁶ our framework was constructed from a combination of a priori issues, the information needs of the study set by the funders, and issues that emerged from the data. The dimensions of the framework emerged from our personal experience, from the background methodological literature, and from the empirical or reflective reports that provided the ‘data’ of the review. As with much primary qualitative research, the framework was not identified until we were familiar with the literature as data, and it was instrumental in helping us code and organize the data, then describe and interpret the concepts. This approach allowed comparative analysis of varied and complex methods and the identification of gaps, whether these were gaps in individual reports (such as little description of methods, even less of public views and influence) or gaps in our collective knowledge, in particular, about methods for reaching decisions with patients/publics.

Distinguishing between ‘purposive’ and ‘opportunistic’ methods for identifying and prioritizing research topics widened the scope of the review to include all relevant activities by lay people, not only those that related directly to the prior interests of professionals. This broad scope revealed that a key barrier is not the inability of lay people to identify or prioritize research topics, but the tendency of professional organizations not to grasp them. Our framework allowed the abstract concept of empowerment to be addressed in very practical terms: the numbers of people involved; whether they were individuals or networked group members; within one‐off or repeated opportunities for involvement; whether the forum allowed two‐way communication; whether members of the public had leading roles or played a part in decision making; and whether there was any training or other resources to support their involvement.

Framework analysis is designed so that it can be viewed and assessed by people other than the primary analyst. ⁶¹ Consequently, framework analysis in a systematic review displays the data in such a way that readers can navigate the charts to find evidence relevant to their circumstances (in this case, for instance, international or national programmes or individual projects), or to the options available to them (e.g. patient organizations responding to consultations or only entering collaborative partnerships or taking the initiative). Applying the framework to these data reveals the theories that have been applied to work constructively with individuals, ²³ , ²⁴ or within organizations. ¹⁷ It also reveals the achievements possible with investment within a commercial enterprise, ⁵¹ or the options available (collaboration, ²³ consultation ³⁷ or both ⁵⁷ ) and what has been learnt from them. Thus, readers can draw out for themselves relevant implications for their policy and practice from what is known about public involvement in research agenda setting.

Discussion

The framework we have described categorizes the engagement of participants in research agenda setting in terms of the types of people involved, the degree of public involvement, and the initiators of the engagement. It combines and extends concepts proposed by Arnstein ⁹ and Mullen et al. ¹² The framework is consistent with an eight‐dimensional framework described by Byrt and Dooher. ²⁶ Their definition of ‘formal’ participation (planned, with clear roles and responsibilities) and ‘informal’ participation (unplanned, spontaneous involvement of service users where roles may be blurred) is similar to our distinction between ‘purposeful’ and ‘opportunistic’ involvement. Williamson’s ⁶² distinction between ‘patients and carers’, ‘consumers’ and ‘consumerists’ is also relevant to the conceptual structures underling our framework. According to Williamson, patients and carers are interested in their personal interactions with health services, consumers with the quality of services for particular groups of people, and consumerists challenge structures more fundamentally by focusing on general principles such as access, information, choice, advocacy, equity, safety and redress. Williamson uses the distinction between ‘overt’ and ‘covert’ participation to differentiate between participation by individuals who are motivated explicitly by their own personal experience, problem, illness or disability, and that by those who bring other roles or skills. This dimension, that we did not consider, is a source of controversy. Some people hold a strong preference for being represented by their peers, rather than by professional advocates who do not have personal experience of the relevant health condition, but may have a paid position such as chief executive of a patient support charity.

Our review findings provide some answers to Boote et al.’s questions ⁷ : how can public involvement be conceptualized; how and why does public involvement influence health research; and what factors are associated with success? With respect to Boote et al.’s third question about how to measure and evaluate the influence of the public in health research, as part of our framework analysis, we recorded influence in terms of whether there was a record of the public’s priorities, a reflective record of the process, and public involvement in the authorship of these records. Our methods of appraisal, which took into account the presence or absence of research methods, authorship and setting, were crude measures of the appropriateness of methods and the reliability of conclusions in a newly emerging field.

In many ways our analysis drew similar conclusions to research addressing public participation in health services. We found that lay people tended to be less influential when they complied with involvement methods established by professionals, rather than sharing the task of developing methods for involvement. Willis ⁶³ has taken the analysis of controlling influences a step further in her study of District Health Councils in Tasmania, bodies established by professionals specifically in order to involve lay people. She concluded that, in a system of structured community participation, dominant bureaucratic interests may often shape the form this participation takes, rather than it being designed by community members themselves. However, she also points out that structured forms of community participation with appropriate links to the wider public may have advantages in allowing quick responses, and so become part of a good network to work with, rather than react to, health decision making. We found examples to support this conclusion where lay committee members were given time and resources to consult their peers more widely.

Particular success has been achieved by a research agenda setting exercise that addressed the key dimensions of the framework by combining the benefits of representative consultation with iterative collaboration and explicit decision making. ⁶⁴ A comprehensive evaluation of process and outcome concluded that the main participation objectives were achieved through a legitimate and rational process leading to patients’ influence of the agenda and mutual learning about the topic, ways of working together and each others’ perspectives and priorities.

Our framework has been since used in a systematic review of involvement in a broader range of activities: developing health‐care policy and research, clinical practice guidelines and patient information. ⁶⁵ Use of the framework in this review ensured that different methods of involvement were described in comparable terms, and it enabled the review to highlight areas where no evidence was available at all. The review showed clearly that no trials to date have evaluated different degrees of involvement, different forums of communication, lay involvement in decision making, or the provision of training or personal or financial support for lay involvement.

Conclusions

This review of involving the public in research agenda setting builds on the technique of framework analysis which has previously been described only for primary research. ⁶ We found this approach useful for developing a conceptual framework of public involvement in research based on accessing and reviewing a broad literature. Our framework is consistent with analyses in the literature about empowerment for public involvement in public services more broadly. It is potentially applicable to a wide range of reports of public involvement in research and research‐based activities. Use of the framework facilitates learning from many different strategies and reports of these, from informal reflections to formal research. Such a breadth can thus generate an overview of achievements to inform policies and practices in the area of public involvement in research. As with other systematic review methods, application of the framework also usefully identifies gaps in the literature which need to be filled in order to increase our understanding of how to promote public involvement and evaluate the effectiveness of different approaches.