Key items for reports of primary care research: an international Delphi study

Abstract

Objective

Reporting guidelines can improve dissemination and application of findings and help avoid research waste. Recent studies reveal opportunities to improve primary care (PC) reporting. Despite increasing numbers of guidelines, none exists for PC research. This study aims to prioritise candidate reporting items to inform a reporting guideline for PC research.

Design

Delphi study conducted by the Consensus Reporting Items for Studies in Primary Care (CRISP) Working Group.

Setting

International online survey.

Participants

Interdisciplinary PC researchers and research users.

Main outcome measures

We drew potential reporting items from literature review and a series of international, interdisciplinary surveys. Using an anonymous, online survey, we asked participants to vote on and whether each candidate item should be included, required or recommended in a PC research reporting guideline. Items advanced to the next Delphi round if they received>50% votes to include. Analysis used descriptive statistics plus synthesis of free-text responses.

Results

98/116 respondents completed round 1 (84% response rate) and 89/98 completed round 2 (91%). Respondents included a variety of healthcare professions, research roles, levels of experience and all five world regions. Round 1 presented 29 potential items, and 25 moved into round 2 after rewording and combining items and adding 2 new items. A majority of round 2 respondents voted to include 23 items (90%–100% for 11 items, 80%–89% for 3 items, 70%–79% for 3 items, 60%–69% for 3 items and 50%–59% for 3 items).

Conclusion

Our Delphi study identified items to guide the reporting of PC research that has broad endorsement from the community of producers and users of PC research. We will now use these results to inform the final development of the CRISP guidance for reporting PC research.

Strengths and limitations of this study.

• Research team from six nations to bring diversity to the analysis.

• We included a diverse Delphi panel to include the perspectives of researchers, clinicians, educators and patients.

• The Delphi surveys had a high response rate.

• Most participants had English as their first language, but a large proportion of the respondents were bilingual.

• There were only four clinicians who did not have another non-clinical professional role.

Introduction

Primary care (PC) is a distinct model of healthcare that can improve patient and population health,¹ and PC has its own set of research questions that are of interest to the PC community. PC research uses an array of research methods and has developed approaches that emphasise patient-centred, problem-oriented care of whole patients.² Despite the breadth in topics and approaches that are employed in PC research, there are underlying common elements that are always needed to make PC research useful for researchers, clinicians, patients and policy-makers.

Consensus Reporting Items for Studies in Primary Care (CRISP) is an international, interprofessional, interdisciplinary initiative to help improve the reporting of PC research (http://www.crisp-pc.org/). The goal of CRISP is to improve the quality and usefulness of reports of PC research so that the results may be appropriately applied to improve the process of care and health outcomes for patients and communities.

CRISP research has studied current practices, assessed needs and collated ideas for improvement through a scoping review² and surveys of PC researchers and clinicians.^{3 4} These studies have demonstrated the need to improve PC research reporting, documented a desire for research reporting guidelines tailored to the needs and characteristics of PC research and generated lists of specific suggestions for items that would make reports more useful.^2–4 Our prior work has emphasised that reports of PC research are not always useful to readers as the reports do not include contextual elements, nor the recognition of competing demands, nor the factors that impact on function such as multidisciplinary teams and therapeutic relationships.

Researchers across many fields recognise the need to improve research reporting.^{5 6} The Enhancing the QUAlity and Transparency Of Health Research (EQUATOR) network catalogues a growing number of guidelines for the reporting of health research (https://www.equator-network.org). Many have been widely adopted, with potential benefits including more effective dissemination, translation and implementation of new knowledge and reduction of research waste. Many well-known EQUATOR guidelines focus on standard research methods (eg, Preferred Reporting Items for Systematic Reviews and Meta-Analyses,⁷ Consolidated Standards of Reporting Trials and⁸ Strengthening the Reporting of Observational Studies in Epidemiology⁹), but the bulk of the 400-plus guidelines are discipline specific. However, no guideline focuses directly on the reporting needs of PC.

This study aims to reach a consensus around the potential items for the CRISP guidance statement based on the expertise of the international PC research community. We will use these results to inform the final CRISP guidance to improve the reporting of PC research.

Methods

We used a Delphi survey to reach a consensus on potential items for PC research reporting among the broad international community of producers and users of PC research. A Delphi survey is a consensus building method that gathers opinions from a select group of participants and allows participants to compare their opinion to others in the group via consecutive surveys.¹⁰ We chose the Delphi design as the most appropriate consensus-building method as it enables participation by people who are distant in place¹⁰ and we published our study protocol online.¹¹ Our reporting is informed by Guidance on Conducting and Reporting Delphi Studies¹² and the Checklist for Reporting Results of Internet E-Surveys Checklist for reporting internet e-surveys.¹³

Delphi panel

We sought to recruit a diverse panel to reflect the nature of the PC research community to represent the producers and users of PC research who bring unique expertise to the subject. This approach differs from most Delphi studies that typically use a small, homogeneous group of experts.^{10 12} We aimed for 100 participants to include practitioners, researchers, patients and policy-makers from high-income and lower-middle-income countries.

We recruited participants from a list of volunteers from our prior surveys and CRISP activities^{3 4} as well as our professional networks. We emailed volunteers, inviting them to complete a demographic survey and consent to participate in the online Delphi survey. Based on these survey returns, we identified target groups to guide our further invitation efforts.

We used a purposeful sampling procedure and developed a matrix to stratify targeted characteristics including world regions, demographic factors, healthcare professions, research disciplines, research roles and experience levels. Inclusion criteria required participants to be actively engaged in some aspect of PC, read English well enough to complete the survey, be able to access the online survey and give informed consent. We applied no exclusion criteria.

Delphi survey development

The round 1 survey presented a list of potential reporting items drawn from the results of our prior CRISP research: a needs assessment survey among the international PC research community,⁴ a survey focusing on the needs of practicing clinicians³ and a scoping review.² (table 1) In each of the surveys, we asked respondents for their views on what could be improved in PC research reporting and what items are important to include in research reports so that they are useful for their own research and/or clinical practice. We extracted the free-text comments from the two surveys, and EAS, WRP and PP synthesised the comments into an initial list of potential reporting items. The whole CRISP Working Group then reviewed the list, commented on each item and suggested new wording for clarity. We presented this aggregate list of potential items to the Delphi panel in round 1. We pilot tested the survey with the Working Group and colleagues who made suggestions to improve clarity of the potential items and survey instructions.

Table 1.

Source of potential reporting items and Delphi panellists in the Consensus Reporting Items for Studies in Primary Care programme of research

Source	Target group sampling method	Numbers	Items	Volunteers for Delphi
Online survey 2021*	International PC research community Producers and users of PC research Purposive and snowball† sampling	255 respondents 24 nations 60% physicians 56% North American	X	X
Online survey 2021‡	Practicing PC clinicians—sees patients≥half-time Physician, nurse/Nurse Practitioner, Physician Assistant, psychologist, social worker, pharmacist Purposive and snowball† sampling	252 respondents 29 nations 88% physicians 55% women	X	X
Literature review 2021§	Scoping review on PC research reporting Systematic review of seven databases	25 papers included/2847 titles screened	X
Editor interviews	Editors of journals regularly publishing PC research	9/11 journals	X
Academic meetings	Research presentations (oral, poster), open meetings Australian Association for Academic Primary Care—2019, 2021 North American Primary Care Research Group—2018, 2019, 2020, 2021 Society of Academic Primary Care—2020 WONCA Europe—2021		X	X

*Phillips et al.⁴

†Lewis-Beck et al.¹⁹

‡Phillips et al.³

§Phillips et al.²

PC, primary care; WONCA, World Organization of National Colleges, Academies and Academic Associations of General Practitioners/Family Physicians.

Participants received an email invitation between May to September 2021. Qualtrics XM software (Qualtrics, Seattle, Washington, USA) was used to provide respondents with online access to the closed Delphi survey using a unique survey code that allowed us to link participant responses between rounds.

The survey presented questions in the same order to all participants (online supplemental appendix 1 and 2). Round 1 presented 29 potential items over 32 pages and required the participant to respond within 4 weeks. Round 2 presented 25 potential items over 29 pages and required participants to respond within 6 weeks. Respondents could review and change answers, and no question forced response to advance the survey. We did not offer any financial incentive for participation, but participants could elect to be named in the acknowledgement of the manuscript (see below).

Analysis

Round 1 presented 29 potential reporting items to the Delphi participants. We calculated the percentage of participants that voted to include each item, exclude the item or indicated that they were unsure. Only participants who voted to include an item were asked about whether it should be required or recommended. For incomplete surveys, we included questions that were answered in the analysis.

Three investigators (EAS, WRP and PP) reviewed and summarised all comments and presented them to the Working Group, along with the descriptive statistics. Reworded items from round 1 were included in round 2 if they met the protocol criteria, which included at least 50% of participants agreeing that the item should be included (figure 1).¹¹ We chose this relatively low threshold so as not to prematurely exclude items that could be reconsidered by participants after rewording or reflection on the comments from other participants.¹⁴

Figure 1.

Flow chart of Delphi study to rank key items for reports of primary care research. ¹See table 1 for details of Consensus Reporting Items for Studies in Primary Care (CRISP) data sources. ²See table 2 for characteristics of round 2 respondents. ³See table 3 for Delphi results.

We prespecified that round 3 would only proceed if items had less than 50% of participants agreeing it should be included, plus suggestions for changes in wording.¹¹

Patient and public involvement

We included clinician-researchers in our research team. Researchers, clinicians, educators and patients were involved as participants in the Delphi survey.

Results

Following our prespecified sampling procedure,¹¹ we invited 116 respondents to participate in the Delphi study. Round 1 was completed by 98 (84%). Round 2 was completed by 89 of the original 98 (91%; 77% of those agreeing to participate) (table 2). Panellists were from all five world regions and demographics are detailed in table 2.

Table 2.

Characteristics of Delphi round 2 respondents; n=89

Characteristic	Category	Respondents, N (%)
Geographical region* (N=88)	African region
	Sub-Saharan Africa	3 (3)
	Region of the Americas
	Latin America	4 (5)
	Caribbean	2 (2)
	USA	26 (30)
	Canada	9 (10)
	Asia region
	Southern-Eastern Asia	4 (5)
	Western Asia	2 (2)
	European region
	Northern Europe	5 (6)
	Southern Europe	1 (1)
	Western Europe	4 (5)
	British Isles	9 (10)
	Pacific region
	Australia	18 (20)
	New Zealand	1 (1)
Age	<25 years	0 (0)
	26–35 years	8 (9)
	36–45 years	22 (25)
	46–55 years	20 (22)
	56–65 years	25 (28)
	>65 years	12 (13)
	Prefer not to say	2 (2)
Gender	Woman	53 (60)
	Man	36 (40)
	Non-binary/third gender	0 (0)
	Prefer not to say	0 (0)
Identifies as part of ethnic majority in country of residence†	Yes	64 (72)
	No	22 (25)
	Prefer not to say	3 (3)
Identifies as first nations or Indigenous person†	Yes	4 (4)
	No	79 (89)
	Prefer not to say	6 (7)
First language‡	English	66 (74)
	Spanish	4 (4)
	Dutch	4 (4)
	Swedish	2 (2)
	Chinese	2 (2)
	Portuguese	2 (2)
	French	2 (2)
	Other‡	7 (8)
Role(s) in primary care research§	Researcher/scientist	54 (61)
	Academic journal peer reviewer	44 (49)
	Primary investigator	41 (46)
	Clinician-researcher	40 (45)
	Methodologist	21 (24)
	Academic journal editor	21 (24)
	Patient	8 (9)
	Research manager	8 (9)
	Trainee/student	5 (6)
	Other—teacher/educator	3 (3)
	Community representative	2 (2)
	Other	4 (4)
Primary profession¶	Healthcare professional	39 (44)
	Scientist/researcher	37 (42)
	Educator	7 (8)
	Public health	1 (1)
	Other	5 (6)
Direct patient care of healthcare professionals (N=39)	≤50% of working week	27 (69)
	>50% of working week	12 (31)
Role/specialty (of healthcare professionals) (N=57)	Physician—General Practitioner/family doctor	41 (72)
	Nurse and nurse practitioner	6 (11)
	Pharmacist	3 (5)
	Physiotherapist	2 (4)
	Physician assistant	1 (2)
	Social worker	1 (2)
	Physician—paediatrician	1 (2)
	Podiatrist	1 (2)
	Other	1 (2)
Highest non-clinical research degree attained	No research degree	8 (9)
	Bachelor’s degree	2 (2)
	Master’s degree	18 (20)
	Doctoral degree	61 (69)
Level of primary care research experience	Novice	11 (12)
	Intermediate	40 (45)
	Advanced	38 (43)

*Geographical regions are presented using the United Nations M49 Standard. Regions without participants are not listed.

†Ethnicity data were collected but not presented, following current recommendations.

‡Other first languages spoken (N=1 for each): Arabic, Bahasa Malay, Catalan, German, Hebrew, Kuche and Phusto.

§Respondents may have identified more than one role in primary care research.

¶Respondents were required to select only one primary profession. Some healthcare professionals listed other primary professions, thus the total n of healthcare professionals in this item may exceed the responses in the role/specialty of healthcare professionals item.

Round 1 was completed by 98 people (84% of 116 volunteers), with 96% completing all questions. Tables 3 and 4 lists the levels of endorsement for inclusion, requirement and recommendation for each potential reporting item in rounds 1 and 2, respectively.

Table 3.

Potential reporting items in Delphi round 1 reporting respondent votes to include, require or recommend items

Potential reporting items Ranked by % voting to include item then by % voting to require item		Delphi round 1, N=98
		Include item	Among respondents Voting to include item*
			Require item*	Recommend item*
1	Discuss implications of study findings for research, patient care, education and policy with specific focus on primary care.	97% (88/91)	72% (63/88)	28% (25/88)
2	Include ‘primary care’ in the title, abstract and/or keywords or a discipline-specific term (eg, general practice, family medicine).	97% (95/98)	55% (52/95)	45% (43/95)
3	Describe the patients and populations in sufficient detail to allow comparison to other PC patient populations, including multimorbidities, inequity, vulnerability.	89% (84/94)	53% (43/84)	47% (38/84)
4	Describe clinical interventions in detail sufficient to allow readers to judge applicability to a variety of PC settings, including resources required, implementation, sustainability, challenges faced, solutions adopted.	88% (80/91)	55% (44/80)	43% (34/80)
5	Describe study interventions in detail sufficient to allow reader to replicate the research.	86% (78/91)	71% (55/78)	28% (22/78)
6	If a single disease is being studied, address the limitations and how multimorbidity might affect interpretation of study findings.	85% (77/91)	40% (31/77)	60% (46/77)
7	Describe if and how measures have been validated in PC settings.	85% (77/91)	41% (31/77)	59% (45/77)
8	Describe the research processes and how they might influence validity, transferability and scalability across PC settings (eg, recruitment, incentives, support for the intervention in a research setting).	83% (78/94)	65% (51/78)	34% (27/78)
9	Describe the importance of the issue under study (eg, problem, disease, diagnosis or illness) within the PC setting.	82% (79/96)	57% (45/79)	42% (33/79)
10	Use person-first language (eg, person with diabetes, not diabetic).	82% (80/98)	44% (35/80)	55% (44/80)
11	Describe the national and local healthcare system to allow comparison to other systems, such as: access to care, organisation of primary care, payment system, universal care or coverage, self referral to consultants, drug coverage, any characteristics likely different from the setting for most readers.	81% (74/91)	47% (35/74)	51% (38/74)
12	Describe how PC patients and/or community members were involved throughout the research process.	78% (75/96)	39% (29/74)	58% (43/74)
13	Describe how measures are relevant to PC patients and PC patient care.	73% (66/91)	42% (28/66)	58% (38/66)
14	Describe the clustering of patients, clinicians, teams and clinics and how it is addressed in the analysis.	71% (67/94)	55% (37/67)	42% (28/67)
15	Describe the origin of the research question and how it relates to patient care and primary care practice.	70% (69/98)	48% (33/69)	46% (32/69)
16	Describe implementation strategies used to encourage adoption of the intervention into routine PC clinical care.	70% (64/91)	30% (19/64)	64% (41/64)
17	Specify if the focus is on single clinical encounters or on a longitudinal course of care for a clinical problem (eg, episode of illness, episode of care).	66% (61/92)	44% (27/61)	54% (33/61)
18	If clinicians are an object of study, report separate groups separately or give rationale for aggregating different types of clinicians.	64% (60/94)	50% (30/60)	47% (28/60)
19	Describe how PC practicing clinicians were involved throughout the research process.	63% (60/96)	28% (17/60)	72% (43/60)
20	Discuss the impact of any recommendations on competing demands in PC practice.	63% (57/90)	23% (13/57)	75% (43/57)
21	Practice team—describe the composition and organisation of teams delivering patient care.	59% (55/94)	42% (23/55)	56% (31/55)
22	Describe the professional backgrounds of members of the research team and their experience in PC.	56% (54/96)	39% (21/54)	56% (30/54)
23	Practice context—describe PC clinicians by profession, specialty, training and certification.	57% (54/95)	33% (18/54)	65% (35/54)
24	Describe the relationships between study patients and clinicians/teams, including, the definition used and method for classifying patients. New or established patient. Continuity patient. (Does the patient receive most of their care over time from the clinician?) Referred or primary patient. (Was the patient referred to the clinician for a consultation?)	57% (52/91)	25% (13/52)	69% (36/52)
25	Report strength of recommendations.†	53% (48/91)	40% (19/48)	56% (27/48)
26	Describe the relationships between researchers and treating clinicians/team members and how relationships might influence the process and outcomes of the research.	50% (48/96)	40% (19/48)	50% (24/48)
27	Report if the clinical problem studied is a new, continuing, recurrent or chronic/persistent problem.	49% (46/94)	46% (21/46)	48% (22/46)
28	Specify the focus of study as: a problem (eg, headache) disease/diagnosis (eg, migraine) illness (eg, person living with migraines).†	47% (43/92)	33% (14/43)	60% (25/43)
29	Report or translate measures into forms useful in PC patient care.	26% (24/91)	33% (8/24)	67% (16/24)

Please note that this is not the final Consensus Reporting Items for Studies in Primary Care guidance.

Table lists reporting items presented to Delphi participants, not final recommendations for research reporting.

*For each question, some respondents did not answer or selected ‘unsure’, so the total require and recommend does not equal the total include.

†DNP—did not proceed to round 2.

PC, primary care.

Table 4.

Potential reporting items in Delphi round 2 reporting respondent votes to include, require or recommend items

Potential reporting items Ranked by % voting to include item then by % voting to require item		Delphi round 2, N=89
		Include item	Among respondents Voting to include item*
			Require item*	Recommend Iitem*
1	Discuss implications of study findings for research, patient care, education and policy with specific focus on primary care.†	99% (86/87)	88% (73/86)	12% (10/86)
2	Describe the patients and populations in sufficient detail to allow comparison to other primary care patient populations.†	97% (84/87)	70% (59/84)	29% (24/84)
3	Describe the research processes and how these might influence the validity, generalisability and applicability of the study findings for primary care practice. (Examples: recruitment, incentives, implementation, study supports not routinely available in practice.)	97% (84/87)	60% (50/84)	39% (33/84)
4	Use person-focused language to refer to populations and participants in the research.†	96% (85/89)	48% (41/85)	51% (43/85)
5	Include ‘primary care’ and/or discipline-specific terms in the title, abstract and keywords. (Discipline-specific terms include general practice, family medicine, nursing, general internal medicine, general paediatrics and other primary care team members.)	93% (83/89)	66% (55/83)	34% (28/83)
6	Describe interventions and their implementation in sufficient detail to allow readers to judge applicability to routine practice in a variety of primary care settings. (Examples: resources required, sustainability, challenges faced, solutions adopted.)‡	95% (83/87)	63% (52/83)	36% (30/83)
7	Describe the healthcare system in sufficient detail to allow comparisons to other systems. (Examples: access to care, organisation of primary care, payment system, universal care or coverage, patient self-referral to consultants, payment of medication, technology including electronic health records, others.)	94% (82/87)	56% (45/82)	44% (36/82)
8	Describe how primary care patients, practicing clinicians, community members and other stakeholders were involved in the research process.‡	91% (79/87)	46% (36/79)	54% (43/79)
9	Explain the rationale for the research question and how it relates to primary care.†	90% (79/88)	57% (45/79)	43% (34/79)
10	Describe if and how study measures have been validated in primary care populations or settings.	90% (78/87)	55% (43/78)	45% (35/78)
11	Describe how study outcome measures are meaningful to primary care patients and their care.†	90% (78/87)	40% (31/78)	58% (45/78)
12	If clinical research focuses on a single disease, report if multimorbidity is present and how it might affect interpretation of the study findings.	89% (77/87)	51% (39/77)	49% (38/77)
13	Specify if the study focus is single clinical encounters or longitudinal courses of care for a clinical condition (eg, episode of care vs episode of illness).	83% (72/87)	27% (19/72)	72% (51/72)
14	Describe the magnitude or importance of the topic under study in the primary care setting. (Examples: disease prevalence, burden of suffering, disability.)	80% (70/87)	56% (38/70)	43% (30/70)
15	Describe any grouping of patients, clinicians, teams or clinical settings and how it is addressed in the analysis.	75% (65/87)	45% (29/65)	55% (36/65)
16	Identify the theory, model or framework used and explain why it is appropriate to the research question in primary care.§	71% (62/87)	26% (16/62)	73% (45/62)
17	Report categories of clinicians separately or provide a rationale for aggregating different groups.	70% (61/87)	25% (15/61)	74% (45/61)
18	Report findings in forms useful to primary care clinicians and patients. (Examples: number needed to treat, absolute risks instead of just relative risks, etc.)	63% (55/87)	36% (20/55)	60% (33/55)
19	Describe the backgrounds of members of the research team and the team’s familiarity with primary care.	61% (53/87)	34% (18/53)	62% (33/53)
20	Discuss the impacts of any recommendations on other demands and priorities in primary care practice.†	52% (45/87)	22% (10/45)	78% (35/45)
21	When collecting and reporting personal characteristics of study participants, define, classify and identify the characteristics. Describe the source of data and the rationale for inclusion.†	51% (44/87)	52% (23/44)	45% (20/44)
22	Specify if study patients have existing relationships with the clinicians/teams or are new patients.†	48% (42/87)	36% (15/42)	62% (26/42)
23	In studies of patients with a clinical condition, specify if it is acute or chronic, new or recurrent.†	48% (42/87)	31% (13/42)	69% (29/42)
24	Describe the membership, roles and collaboration of the team delivering primary care to the patient.	46% (40/87)	18% (7/40)	80% (32/40)
25	For each category of clinician, report profession, specialty and qualifications. (Examples: physicians, certified family physicians, nurses, nurse practitioners, registered dieticians, masters degree social workers.)	40% (35/87)	20% (7/35)	80% (28/35)

Please note that this is not the final Consensus Reporting Items for Studies in Primary Care guidance.

Table lists reporting items presented to Delphi participants, not final recommendations for research reporting.

*For each question, some respondents did not answer or selected ‘unsure’, so the total require and recommend does not equal the total Include.

†Item substantially reworded in round 2 from round 1.

‡New item in round 2 combined from two items from round 1.

§New item added in round 2.

Respondents suggested rewording or combining for most items. In addition, they suggested two new items: reporting demographics of participants (table 4, item 21) and theory informing research (table 4, item 16).

For one item in round 1, ‘report or translate measures into forms useful in PC patient care’, 28% were unsure if it should be included (ie, they answered ‘unsure’ in the survey, note that this is not shown in table 3), and participant comments suggested that they did not understand the statement, so the item was substantially reworded for round 2 to be ‘report findings in forms useful to PC clinicians and patients (Examples: number needed to treat, absolute risks instead of just relative risks, etc.)’ (table 4, item 18). A majority (81%) voted to include ‘describe the national and local healthcare system to allow comparison to other systems…’ (table 3, item 11) and 14/15 of those who answered ‘no’ were from North America.

We invited the 98 people who completed round 1 to participate in round 2 and 89 (91%) responded. Round 2 presented 25 items, including the two new items (table 4). Some of the items from round 1 items were combined for round 2: (1) three items (table 3, items 4, 5 and 16) from round 1 were combined into: ‘describe how PC patients, practicing clinicians, community members and other stakeholders were involved in the research process’ and (2) and three items (table 3, items 12, 19 and 26) were combined into ‘describe interventions and their implementation in sufficient detail to allow readers to judge applicability to routine practice in a variety of PC settings’.

There were limited suggestions for rewording in round 2 and no suggestions about adding or combining items. Round 2 results showed only minor changes from round 1, demonstrating Delphi panel consensus on the list of items. Therefore, per protocol,¹¹ we did not proceed to a round 3 (figure 1).

A majority of round 2 respondents voted to include 23 items (90%–100% for 11 items, 80%–89% for 3 items, 70%–79% for 3 items, 60%–69% for 3 items and 50%–59% for 3 items). Among those voting to include items, over 50% voted to require reporting for 11 items. For many items, votes were relatively close between required and recommended (eg, 60:40 or less), with few items showing a strong preference.

Discussion

This Delphi study of the international PC research community reached consensus on potential items for guidance for the reporting of PC research. The study represents the first time that the PC research community has been consulted on this topic. These items highlight the unique needs of PC research and complement the items commonly listed in guidelines developed by other experts for specific research methods and other purposes.¹⁵ The process prioritised 21 reporting items for inclusion in reports of PC research, and there was limited support for making items mandatory.

While consensus was reached on all 21 reporting items, participants varied in the strength of their support for individual items. Greatest consensus (≥90% agreement to include) was reached with items relating to research implications, strategies to improve indexing and searching and transferability of study findings. Less support was directed at items that were seen as being more difficult to collect and report, including describing PC teams, relationships among patients–clinicians–researchers, and specific patient demographics. Potential items pertaining to research team background and experience in PC were less well supported. A few comments suggested this information might make some team members feel unwelcome or under-appreciated.

The preparatory programme of CRISP research and its international, interdisciplinary, inclusive approach gave this study particular strengths. Our Delphi panel engaged diverse participants reflecting the breadth of PC and its research enterprise. Consensus across these groups suggests broad agreement on what is important in PC and the research supporting practice, research, education and policy.

The Delphi process is a broadly accepted method for reaching consensus among expert groups.¹⁰ It is recommended for the development of research reporting guidelines, though not yet employed by most groups.¹⁶ There are multiple approaches to determining the ideal size and composition of Delphi panels.¹⁰ Most research reporting guidelines have relied on small homogenous groups of academic experts in research methodology. We elected to engage a large and diverse panel, as we recognise the complexity of PC research and the value of expertise contributed by researchers, clinicians, educators and patients.

Study limitations include the practical adaptations required by the current COVID-19 pandemic. Our authorship team could not meet in person, but a small group (WRP, EAS and PP) had frequent virtual meetings and communicated by email with the Working Group. Most respondents had English as their first language, but there are no indications that the needs differed between English and non-English speaking participants, and the global geographic spread of participants increases confidence that findings can be widely generalised. Also, a large proportion of the respondents were bilingual. There were only four clinicians who did not have another non-clinical professional role. This may represent a missing clinician perspective or more likely the commonality of portfolio careers.^{17 18}

This Delphi study is one of the final steps in the crystallisation of the CRISP reporting checklist for PC research. The Working Group will use the Delphi findings to inform the next steps, which will include a group discussion on the final wording and order of the items and pilot testing of the checklist with diverse groups of researchers with different levels of expertise and experience. The overall vision of CRISP is to improve research reporting in PC to ensure reports are as helpful as possible for researchers, patients, clinicians and policy-makers. This Delphi survey represents an important step on the CRISP journey to providing the support for PC researchers.

Data availability statement

Data are available upon reasonable request. Full data will be shared with researchers on appropriate request to the corresponding author.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study involves human participants and was approved by Monash University Human Ethics Research Committee (26508). Participants gave informed consent to participate in the study before taking part. We protected identified data within password-protected accounts accessible to EAS and PP.