Inclusion of participants from low-income and middle-income countries in core outcome sets development: a systematic review

Jamlick Karumbi; Sarah L Gorst; David Gathara; Elizabeth Gargon; Bridget Young; Paula R Williamson

doi:10.1136/bmjopen-2021-049981

. 2021 Oct 18;11(10):e049981. doi: 10.1136/bmjopen-2021-049981

Inclusion of participants from low-income and middle-income countries in core outcome sets development: a systematic review

Jamlick Karumbi ^1,^2,^✉, Sarah L Gorst ¹, David Gathara ^2,³, Elizabeth Gargon ¹, Bridget Young ¹, Paula R Williamson ¹

PMCID: PMC8527127 PMID: 34667005

Abstract

Objective

Our study aims to describe differences or similarities in the scope, participant characteristics and methods used in core outcome sets (COS) development when only participants from high-income countries (HICs) were involved compared with when participants from low-income and middle-income countries (LMICs) were also involved.

Design

Systematic review.

Data sources

Annual Core Outcome Measures in Effectiveness Trials systematic reviews of COS which are updated based on SCOPUS and MEDLINE, searches. The latest systematic review included studies published up to the end of 2019.

Eligibility criteria for selecting studies

We included studies reporting development of a COS for use in research regardless of age, health condition or setting. Studies reporting the development of a COS for patient-reported outcomes or adverse events or complications were also included.

Data extraction and synthesis

Data were extracted in relation to scope of the COS study, participant categories and the methods used in outcome selection.

Results

Studies describing 370 COS were identified in the database. Of these, 75 (20%) included participants from LMICs. Only four COS were initiated from an LMIC setting. More than half of COS with LMIC participants were developed in the last 5 years. Cancer and rheumatology were the dominant disease domains. Overall, over 259 (70%) of COS explicitly reported including clinical experts; this was higher where LMIC participants were also included 340 (92%). Most LMIC participants were from China, Brazil and South Africa. Mixed methods for consensus building were used across the two settings.

Conclusion

Progress has been made in including LMIC participants in the development of COS, however, there is a need to explore how to enable initiation of COS development from a range of LMIC settings, how to ensure prioritisation of COS that better reflects the burden of disease in these contexts and how to improve public participation from LMICs.

Keywords: epidemiology, public health, protocols & guidelines

Strengths and limitations of this study.

This is the first paper describing the differences and similarities in the core outcome sets (COS) development processes when there are participants from low-income and middle-income countries (LMICs) compared with when participants are exclusively from high-income country settings.
This paper describes COS published up to 2019. It is possible that more COS involving LMICs participants have been developed during 2020 or are in the process of being developed now.
Public participation has been described in only a few studies, this may introduce a possible bias in the classification of public participants categories thereby altering the conclusions of public participation.

Background

A key barrier to translating clinical research into practice is lack of conclusive evidence. This is due to different trials, of the similar interventions, being too heterogenous and reporting on outcomes that are often non comparable^{1 2} or not relevant to patients and users.^{3 4}

Development of core outcome sets (COSs) could ensure all future research in a field reports a common subset of outcomes. This would reduce research waste thereby enhancing comparability and improving research translation and use.^{5 6} COSs are agreed-on minimum standardised outcome sets that should be measured and reported in all research in a given health area.⁷ They consist of a core domain set (this defines what domains should be measured) and core outcome measurement set (defines the instruments which would be appropriate to measure the domain).⁸ In recent years, the use of COS has been promoted by journals, Cochrane review groups and funders.^{9 10} COSs have also become useful in routine clinical practice data collection for clinical audit and feedback.^{11 12}

Since 2010, the Core Outcome Measures in Effectiveness Trials (COMET) Initiative has been collating, stimulating and promoting the development and use of COS by maintaining an up to date publicly available searchable database.¹³ The COMET database includes published studies of COS development, as well as planned and ongoing work. The types of studies included in the database are those in which COS have been developed, as well as studies relevant to COS development, including systematic reviews of outcomes. Within the database, there are publications that describe a COS only (hereafter denoted specific for COS), and other publications that describe a COS as well as provides recommendations about other trial design aspects like eligibility criteria (hereafter denoted as COS as part of a wider trial design). Studies describing the development of a patient-reported outcome (PRO) COS (a core set of patients reported symptoms and health-related quality of life domains) or core event set (a core set of adverse events or complications). Initially, the database included studies that had been identified by ad hoc means, however, it is now systematically updated using the annually updated systematic reviews as the basis of the updates since 2013.^14–17 The use of this database should help minimise duplication of effort in the development of COS and potentially encourage the use of COS to tackle global health challenges.

How are COS developed?

There is no gold standard for development of COS, however minimum standards have been agreed in the last few years. These are the Core Outcome Set-STAndards for Development (COS-STAD) recommendations, comprising 11 minimum standards for all COS development projects.¹⁸ The recommendations focus on three key domains: (1) the scope of the COS; specific area of health or healthcare that the COS applies needs to be described, with details of health condition, population and types of interventions. The COS may be developed to encompass all stages or severity of a health condition or it may be focused on a particular sub-population, for example, a COS may be developed for all patients with COVID-19, or it may focus on patients with long COVID-19 only. (2) the stakeholders involved in the COS development; the stakeholder groups to be involved and the target number from each group is dependent on the scope of the COS and the existing knowledge and practicability. Decisions on stakeholders’ involvement should be documented and explained in the study protocol. (3) the consensus processes; this typically involves developing a consensus on ‘what’, ‘how’ and ‘when’ to measure. Some of the methods used to build consensus include: Delphi technique; nominal group technique, consensus development conference and semistructured group discussion. The Delphi technique with two or three rounds combined with a consensus conference has been most widely used.^19–21 Choice of method is informed by several factors including; the need to build a true consensus with methodological rigour, strategies to ensure that a diverse range of opinions are heard, and factors such as financial and carbon costs that might limit the practicality of face-to-face meetings.⁷

To date, COSs have been developed for various thematic or disease conditions and more are being developed.^{13 19} It has been opined that development and use of a COS would help reduce outcome heterogeneity and reporting bias, while ensuring that wide-ranging perspectives, including patients’ opinions, are incorporated, thus enhancing the value and quality of research.²² Most COSs have been developed from the perspectives of high-income countries (HICs), with over 70% of COS including participants from Europe and North America. There has been an increase in the proportion of COS that have included participants from Africa and South American countries in COS development, but this still remains low at around 25% as per COS published up to the end of 2018.¹⁹ A closer look shows that the proportion of COS participants residing in low-income and middle-income countries (LMICs) of these regions is even lower being just about 16% in 2016. Further, participants have mainly been those with research and clinical expertise in the various disease areas rather than as public participants with relevant lived experience.¹⁶

Previous work describing COS studies^{14–16 19} has not examined whether there is a difference in the scope, stakeholder participation and approaches to consensus between COS that have include had participants only from HICs compared with those that also included participants from LMICs. The differences in the global burden of diseases may influence which COS are developed, and available resources may determine the methods used in COS development. For example, LMICs have had a higher level of infectious diseases while non communicable diseases have been generally more prevalent in HICs. The hypothesis therefore is that COS with LMIC stakeholders are more likely to be developed for infectious disease conditions and those with predominately HIC stakeholders to are likely to be for non-communicable diseases. Additionally, different methodologies for consensus building require different level of resources, for example, a face-to-face Delphi workshop may be more expensive if it must include a wider range of stakeholders and online Delphi processes are also likely to be influenced by internet connectivity which may be lower in LMICs compared with HICs. The aim of this paper is to describe the differences or similarities in the scope, participant characteristics and methods used in COS developed with participants from HICs and LMICs as classified by the Organisation for Economic Co-operation and Development.²³

Methods

Study selection

Inclusion criteria

We included all COS studies, identified from the original COMET systematic review and annual updates.14–17 24 25 Studies were eligible for inclusion in the review if they developed or applied methodology for determining which outcome domains or outcomes should be measured in research clinical trials or other forms of health. Studies were eligible for inclusion if they reported the development of a COS, regardless of any restrictions by age, health condition or setting. Studies describing the development of a PRO COS (a core set of patients reported symptoms and health-related quality of life domains) or core event set (a core set of adverse events or complications) were also eligible for inclusion. Studies describing the update of an existing COS are included as linked papers to the original COS.¹⁷ Eligible studies are added to the database, as they are found with the annual update to the systematic review, ensuring the database is kept up to date.

Identification of relevant studies

The search strategy for identifying eligible COS development studies has previously been described^14–17 and is briefly described here. A comprehensive search is used to identify studies that had been published or indexed in SCOPUS and MEDLINE via Ovid from inception up to December 2019. The search strategy, which was developed for the original COMET systematic review in 2013, is provided in online supplemental appendix. Annual database searches were repeated in 2015–2020 to identify studies published up to the end of 2019. Hand searching of studies is also performed.

Supplementary data

bmjopen-2021-049981supp001.pdf^{(41.1KB, pdf)}

The review was conducted in accordance with the Preferred Reporting Items of Systematic Reviews and Meta-Analyses (PRISMA) guidelines and the included studies are shown in a PRISMA flow diagram (see figure 1).

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart of identification of eligible studies from the COMET database. Data were extracted from the COS systematic reviews.^14–17 COS, core outcome sets; COMET, Core Outcome Measures in Effectiveness Trial; HIC, high-income country; LMICs, low-income and middle-income countries.

Data extraction

Data were extracted in relation to scope of the COS study (aim of the study, intended use of the COS, year of publication, target population, intervention and disease or condition area), participant categories (clinicians, researchers etc, and their geographic location) and the methods used in outcome selection (Delphi, interviews, focus group discussions, etc, alone or in combination). Data on the burden of disease in LMICs for the last 5 years were obtained, from the Institute of Health Metrics and Evaluation database,²⁶ to help check its alignment to the COS developed over the same period.

Analysis

A description of the proportion of COS that included participants from LMICs and those which were initiated from the LMIC setting is provided. For a COS to be classified as being initiated from an LMIC, either ethical approval of the study had to have been sought in an LMIC, and/or the affiliation of the lead author had to be based in an LMIC; we also used the sites of participant recruitment as confirmation. The scope, participant categories and consensus methods used are described for COS that included participants from LMICs and those with exclusively HIC participants separately. Participation here is defined as being a study participant and contributing data to the COS study.

Patients and public involvement

Patients and the public were not involved in the development of the research question or the design and conduct of this systematic review.

Findings

General description

There was a total of 550 references published by end of 2019 in the COMET Initiative database describing 370 COS. Of these 370 COS, 295 (80%) included only HIC participants and 75 (20%) also included LMIC participants. Two COS had only LMIC participants, one in Sri Lanka²⁷ and another in China.²⁸ A flow diagram of the study selection process is provided in figure 1 as guided by PRISMA statement.²⁹

Year of publication

Between 2014 and 2019, there has been an almost twofold increase in COS that have included LMIC participants to 28% (40/141) from 15% (35/229) in preceding years. Figure 2 describes the absolute numbers of the COS developed up to 2019.

Year of COS publication. COS, core outcome sets; HIC, high-income country; LMIC, low-income and middle-income country.

Scope of included studies

The comparison of the scope of the included studies is shown in table 1. Overall, more than half (63%) of the studies were specifically designed for COS development rather than being part of a wider project or trial. However, a higher proportion of studies with LMIC participants were specific for COS (83%) compared with those with only HIC participants (58%). Over 80% of COS were developed for research only. More than half of studies (55%) did not explicitly report on the age group of the target population characteristics. During the interpretation of the data, this was assumed to be adults. This was based on the disease being predominantly associated with the adult population and the lack of reference to any literature on children.

Table 1.

Scope of included studies

	HICs n (%) (N=295)	LMICs n (%) (N=75)	Total (%) (N=370)
Scope of the COS study
Study aims
Part of wider trial design*	124 (42)	13 (17)	137 (37)
Specific for COS†	171 (58)	62 (83)	233 (63)
Intended use of recommendations
Research	264 (89)	61 (81)	325 (88)
Research and practice	31 (11)	14 (19)	45 (12)
Population characteristics
Neonates	4 (1)	1 (1)	5 (1)
Adults	72 (24)	16 (21)	88 (24)
Children	27 (9)	7 (9)	34 (9)
Children and adults	28 (9)	10 (13)	38 (10)
Not specified‡	164 (56)	41 (55)	205 (55)
Intervention characteristic
Any intervention	77 (26)	30 (40)	107 (29)
Device	5 (2)	1 (1)	6 (2)
Device and surgery	7 (2)	1 (1)	8 (2)
Pharmacological treatment	50 (17)	9 (12)	59 (16)
Procedure	9 (3)	5 (7)	14 (4)
Radiotherapy	2 (1)	0 (0)	2 (1)
Rehabilitation	9 (3)	2 (3)	11 (3)
Screening	2 (1)	0 (0)	2 (1)
Surgical	31 (11)	6 (8)	37 (10)
Vaccine	0 (0)	2 (3)	2 (3)
Both pharmacological and pharmacological treatment	3 (1)	0 (0)	3 (1)
Non-pharmacological treatment	4 (1)	2 (3)	6 (2)
Health system	5 (2)	2 (3)	7 (2)
Not specified	91 (31)	15 (20)	106 (29)

Open in a new tab

*Publications that describe a COS as well as provides recommendations about other trial design aspects like eligibility criteria.

†Publications that describe a COS only.

‡Interpretated to mean adults.

COS, core outcome set; HICs, high-income countries; LMICs, low-income and middle-income countries.

Cancer (16% for COS with HIC participants only and 12% for COS with LMIC participants) and rheumatology (9% for COS with HIC participants only and 16% for COS with LMIC participants) were the dominant disease domains across the COS developed either with LMIC or HIC participants. A slightly higher proportion of COS for rheumatology, pregnancy, skin, anaesthesia and child health included participants from LMICs compared with HICs. Five COS on health systems had LMIC participants. Figure 3 describes the disease categories that were covered by the COS.

Comparison of disease categories covered in the COS with exclusively HIC participants and those also with LMIC participants. COS, core outcome sets; HIC, high-income country; LMIC, low-income and middle-income country.

Of the 75 COS with LMIC participants, only four (5%) were initiated from LMIC setting. All were from Asia, with two from China^{28 30} and one each from Sri Lanka²⁷ and the Philippines.³¹ No COS has been initiated from Africa or South America, although COS for conditions which are more prevalent in LMIC settings had a higher number of participants from LMICs. For example, the malaria COS by Moorthy et al³² included participants from ten sub-Sahara African Countries but none from other LMICs. The COS for the prevention of preterm births had LMIC participants from Egypt, Nigeria, South Africa, China, Iran, Lebanon, Pakistan, Argentina and Brazil.³³

Characteristics of participants involved in consensus process

All COS that had LMIC participants explicitly reported characteristics of their sample compared with 92% of COS that had only HIC participants (see table 2). Overall, over 70% of COS explicitly reported including clinical experts; this was higher where LMIC participants were involved (92%). A slightly higher proportion of COS that included LMIC participants (57%) had included members of the public compared with those with HIC participants only (32%). A higher proportion of LMIC COS (23%) had carers as part of public participants compared with the HIC COS (9%). Of the 28 COS with LMIC participants, only 4 (all with HIC patient participants too) translated the COS development materials to non-English languages. These are COS for clinical trials in acute diarrhoea,³⁴ infant colic,³⁵ childhood constipation³⁶ and type 2 diabetes.³⁷ Non-clinical research expertise was more common in COS with LMIC participants (44%) compared with those with HIC participants only (31%).

Table 2.

Participant characteristics

Participant category	Subcategory (not mutually exclusive)	HIC n (%*)	LMIC n (%*)	Total (%)
Clinical experts		201/295 (68)	69/75 (92)	270/370 (73)
	Clinical experts	166 (56)	62 (83)	228 (62)
	Clinical research experts	92 (31)	41 (55)	133 (36)
	Members of a clinical trial network	8 (3)	7 (9)	15 (4)
	Clinical and non-clinical researchers	12 (4)	1 (1)	13 (4)
Public participation		94/295 (32)	43/75 (57)	137/370 (37)
	Patients	72 (24)	28 (37)	100 (27)
	Carers	26 (9)	17 (23)	43 (12)
	Patient support group representatives	21 (7)	9 (12)	30 (8)
	Service users	4 (1)	5 (7)	9 (2)
Non-clinical research expertise		92/295 (31)	33/75 (44)	125/370 (34)
	Researchers	55 (19)	21 (28)	76 (21)
	Statisticians	22 (7)	8 (11)	30 (8)
	Epidemiologists	13 (4)	9 (12)	22 (6)
	Academic representatives	5 (2)	0 (0)	5 (1)
	Methodologists	19 (6)	5 (7)	24 (6)
	Economists	7 (2)	2 (3)	9 (2)
Authorities		50/295 (17)	22/75 (29)	72/370 (19)
	Regulatory agency representatives	33 (11)	15 (20)	48 (13)
	Government agencies	14 (5)	5 (7)	19 (5)
	Policy makers	10 (3)	7 (9)	17 (5)
	Charities	4 (1)	0 (0)	4 (1)
	Service commissioners	3 (1)	1 (1)	4 (1)
Industry representatives		36/295 (12)	19/75 (25)	55/270 (20)
	Pharmaceutical industries	32 (11)	19 (25)	51 (14)
	Device manufacturers	4 (1)	1 (1)	5 (1)
	Biotech company representatives	1 (1)	0 (0)	1 (1)
Others		10/295 (3)	8/75 (11)	18/370 (5)
	Service providers	5 (2)	2 (3)	7 (2)
	Ethicists	1 (1)	1 (1)	2 (1)
	Journal editors	4 (1)	5 (7)	9 (2)
	Systematic review authors	13 (4)	2 (3)	15 (4)
Not stated	Not reported	23 (8)	0 (0)	23 (6)

Open in a new tab

*Since the categories are not mutually exclusive each percentage is based on the total for that category, that is, 295 for HIC, 75 for LMIC and 370 for the total.

HIC, high-income country; LMIC, low-income and middle-income country.

Participants from LMICs were drawn from the three continents of Asia (60%), South America (49%) and Africa (31%). However, there was a skewed representation with LMIC participants largely being from three countries, with South Africa representing 50% of participants from Africa, Brazil representing 46% of participants from South America and China representing 30% of the participants from Asia.

Methods for selection of outcomes

Methods used for outcome selection ranged from Delphi, consensus conferences and semistructured discussion as shown in table 3. Rarely was one method used for COS development. Semistructured discussion seemed to be the most commonly used single method, with 20% of COS with HIC participants only and 8% of COS with LMIC participants describing its usage.

Table 3.

Methods used in the selection of outcomes

Description	HIC n=295 (%)	LMIC n=75 (%)	Total n=370 (%)
Delphi only	10 (3)	5 (7)	15 (4)
Consensus conference only	11 (4)	4 (5)	15 (4)
Focus group discussions only	1 (<1)	0 (0)	1 (<1)
Nominal group techniques only	1 (<1)	0 (0)	1 (<1)
Semistructured discussions only	64 (22)	6 (8)	70 (19)
Survey only	4 (1)	1 (1)	5 (1)
Review of literature only	32 (4)	4 (5)	36 (10)
Interviews only	1 (<1)	0 (0)	1 (<1)
Mixed methods	171/295 (58)	55/75 (73)	226/370 (61)
Consensus conference and other(s)	8 (3)	1 (1)	9 (2)
Delphi and other(s)	73(25)	42 (56)	115 (31)
Focus group discussions and other(s)	7 (2)	7 (9)	14 (4)
Nominal group techniques and other(s)	15 (5)	11 (15)	26 (7)
Semi structured discussions and other(s)	88 (30)	33 (44)	121 (33)
Survey and other(s)	25 (8)	8 (11)	33 (9)
Review of literature and other(s)	108 (37)	45 (60)	153 (41)
Interviews and other(s)	20 (7)	7 (9)	27 (7)
Unstructured and other(s)	4 (1)	8 (11)	12 (3)
No methods described	6 (2)	1 (1)	7 (2)

Open in a new tab

HIC, high-income country; LMIC, low-income and middle-income country.

Most COS used a combination of methods. Literature review in combination with other methods was most common, with 37% of COS with HIC participants and 60% of COS with LMIC participants using this method as shown in table 3. Only one COS that included LMIC participants had no methods described compared with six that only had HIC participants.

A combination of the three most common methods was more likely to be used for LMIC COS (n=23, 30%) compared with the HIC COS (n=37, 13%) as shown in figure 4.

Combinations of methods for outcome selection. HIC, high-income country; LMIC, low-income and middle-income countries.

Discussion

This review provides the first comparison of COS that have been developed with participants from HICs and LMICs.

Out of the 370 published COS, 75 (20%) COS included participants from LMICs. Thirty-three COS have been published since the fifth update of COS systematic review with fifteen including participants from LMICs.¹⁹ The COS span a range of health areas, target populations and methods used. The methods for outcome selection are similar across the settings, perhaps reflecting the increased standardisation of methods of COS development generally.

Interestingly, even though participants from LMICs have been involved in earlier COS, for example, the cancer reporting recommendations by Miller and others in 1981,³⁸ the inclusion of LMIC participants has increased predominantly in the last 5 years. In this time, there has been an almost twofold increase in the proportion of COS that have had LMIC participants. Global connectivity is allowing people to conduct research without the need for physical meetings. A look at the development processes used is ‘older’ COS shows that most depended on consensus building during conferences and physical workshops.^{32 38 39} These kinds of consensus meetings tend to be more expensive and limit inclusion of participants from varied settings. Despite the increase, Brazil, China and South Africa have been the main drivers of participation from LMICs.

All the COS developed across the settings have mainly focused on research or research and practice. A slightly higher proportion (19%) of COS with LMIC participants are for research and practice compared with 11% of COS with HIC participants only. Given that COS with participants from LMICs are ‘newer’ this could reflect that more recent COS are being developed with the intention of spanning the continuum from research to routine practice in mind and also recent availability of guidance for COS development like the COMET handbook¹¹ and COS-STAD¹⁸ might have influenced the inclusion of wider groups of stakeholders.

Although the COS span a wide range of conditions, the top five conditions where COS exist are cancer, rheumatology, neurology, cardiovascular and orthopaedic conditions. These are mainly non communicable diseases, and even though these are now becoming more common in LMICs, the burden of these conditions has mainly been in HICs.²⁶ Of the COS with LMIC participants, pregnancy, lungs and health systems were among the top five conditions. LMICs are still facing infectious diseases like neonatal sepsis, lower respiratory tract infections, malaria and HIV among others,^{26 40} and one would therefore expect that COS with LMIC participation would reflect this. Only two COS, for malaria³² and leprosy³¹ represent conditions that are usually of high burden in LMICs. Thus, even though LMIC participants are being included in COS development, COS for research may not be reflecting the disease burden in LMICs. This may be because research funding priorities, for which COS are then being developed, are more likely to reflect an HIC perspective.

Interestingly, only four COS have been initiated from an LMIC setting, two from China (COS for hip-preserving treatment of osteonecrosis of the femoral head,²⁸ COS for infertility treatment research³⁰) and one each in Sri Lanka (COS for effectiveness of antiepileptic therapy in children²⁷) and the Philippines, (COS for tibialis posterior transfer surgery³¹). There is still no COS initiated from Africa or South America. Given the varying disease epidemiology and burden between HICs and LMICs there is a need to identify mechanisms to enable LMIC stakeholders to develop context relevant COS so as to improve applicability and adoptability of COS to these settings.¹⁴ These strategies could probably include awareness raising about COS in LMIC settings, provision of resources for COS development and dissemination of guidelines and methods for COS development. These are some of the barriers which have been described potentially affecting COS development from an HIC perspective (interviewees were mainly from Europe and North America).⁴¹ There is a need to assess which barriers and enablers are particularly relevant to COS development in LMIC settings.

Details of public participation are still suboptimally reported, and where reported, a higher proportion of COS with LMIC participants had some public participation compared with COS with HIC participants only. Of note however, is that public participants tended to be from HIC settings. For example, the type 2 diabetes COS included public participants from the UK and Greece,³⁷ the breast cancer COS included patients with breast cancer from USA⁴² and the COS for acute diarrhoea,³⁴ infant colic,³⁵ childhood constipation³⁶ all included public participants from Europe. Public participation from LMICs could potentially be improved by translation of consensus building materials into non-English languages. Harman et al translated Delphi materials into Portuguese language in a two times a day to improve public participation from Brazil.³⁷ Unfortunately, there still were no public participants from Brazil suggesting that there are additional reasons for the lack of public participants from LMIC settings.

The lack of public participation from LMICs could be due to numerous reasons some of which could include temporal, cultural and resources differences between LMICs and HICs. First, COS that have included participants from LMICs have mostly been developed in the last decade, during which guidance has been published on how to develop COS including patient participation. Second, the availability of organised patient groups in HIC settings for given conditions or diseases provides a means of identifying and inviting patients as public participants; such groups are generally lacking in LMIC settings. Third, information asymmetry between the public and patients versus clinicians or researchers could potentially limit the participation of the former in COS development. Although evidence is scarce, we suggest that information asymmetry may be a bigger problem in LMICs due to lower literacy levels when compared with HICs.⁴³ Additionally, some of the major research funders in HICs, for example, the National Institute for Health Research in the UK and the Patient-Centred Outcomes Research Institute in the USA, are increasingly championing public participation in research.

It is therefore likely that a multipronged approach is needed. Further research to explore what would work, in addition to language translation, to improve public participation from LMICs is also needed.

Additionally, there is need to describe the stage at which public participants are involved in COS development and whether it influences which outcomes are included in the COS in a given area. Previous work has shown that patients might sometimes be inhibited when put together with clinicians in a consensus building process⁴⁴ but recent experiences show this is changing, with high satisfaction reported, at least from HIC countries, where joint consensus meetings are held.^{37 45}

This is the first paper describing the differences and similarities in the COS development processes when there are participants from LMICs compared with when participants are exclusively from HIC settings. However, there are a few limitations. First, this paper describes COS published up to 2019. It is possible that more COS involving LMICs participants have been developed during 2020 or are in the process of being developed now. Second, public participation has been described in only a few studies, this may introduce a possible bias in the classification of public participants categories thereby altering the conclusions of public participation.

Conclusion

Progress has been made in including LMIC participants in the development of COS. Most COS are still being developed with an HIC perspective and as such, still reflect the priorities from HIC perspectives. Only four COS have been initiated from LMICs. Public participation is still low and is poorly documented. There is, therefore, a need to explore how to enable initiation of COS development from a range of LMIC settings, how to ensure prioritisation of COS that better reflects the burden of disease in these contexts and how to improve public participation from LMICs.

Supplementary Material

Reviewer comments

bmjopen-2021-049981.reviewer_comments.pdf^{(309.1KB, pdf)}

Author's manuscript

bmjopen-2021-049981.draft_revisions.pdf^{(1.9MB, pdf)}

Acknowledgments

This work is also published with the permission of the Director of KEMRI.

Footnotes

Twitter: @jkarumb1, @GorstSarah

Contributors: Study conception, design and planning of the study were by JK, PRW, DG, BY, SLG, EG as this was part of JK’s PhD. Data aggregation, analysis and writing the original draft were by JK. JK screened all title and abstracts supported by PRW and SLG who carried out double screening of these articles. Full-text articles were screened by JK and any uncertainties were discussed with the team. PRW is fully responsible for the overall content for the work and the conduct of the study, had access to the data, and controlled the decision to publish. All authors were involved with reviewing and editing the manuscript.

Funding: JK has been supported to undertake this work as part of a PhD studentship from the University of Liverpool within the MRC/NIHR Trials Methodology Research Partnership.

Competing interests: None declared.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

Ethics statements

Patient consent for publication

Not applicable.

References

1.Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials 2017;18:122. 10.1186/s13063-017-1870-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Duffy JMN, Ziebland S, von Dadelszen P, et al. Tackling poorly selected, collected, and reported outcomes in obstetrics and gynecology research. Am J Obstet Gynecol 2019;220:71.e1–71.e4. 10.1016/j.ajog.2018.09.023 [DOI] [PubMed] [Google Scholar]
3.Sinha IP, Williamson PR, Smyth RL. Outcomes in clinical trials of inhaled corticosteroids for children with asthma are narrowly focussed on short term disease activity. PLoS One 2009;4:e6276. 10.1371/journal.pone.0006276 [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Tong A, Crowe S, Gill JS, et al. Clinicians' and researchers' perspectives on establishing and implementing core outcomes in haemodialysis: semistructured interview study. BMJ Open 2018;8:e021198. 10.1136/bmjopen-2017-021198 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials 2007;8:1–3. 10.1186/1745-6215-8-39 [DOI] [PMC free article] [PubMed] [Google Scholar]
6.Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009;374:86–9. 10.1016/S0140-6736(09)60329-9 [DOI] [PubMed] [Google Scholar]
7.Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials 2012;13:132. 10.1186/1745-6215-13-132 [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Chevance A, Tran VT, Ravaud P. Improving the generalizability and credibility of core outcome sets (Coss) by a large and international participation of diverse stakeholders. J Clin Epidemiol 2020;125:1–8. 10.1016/j.jclinepi.2020.01.004 [DOI] [PubMed] [Google Scholar]
9.Hughes KL, Kirkham JJ, Clarke M, et al. Assessing the impact of a research funder's recommendation to consider core outcome sets. PLoS One 2019;14:e0222418. 10.1371/journal.pone.0222418 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.COMET Initiative . COS endorsments, 2020. Available: http://www.comet-initiative.org/COSEndorsement [Accessed 26 Sep 2020].
11.Williamson PR, Altman DG, Bagley H, et al. The comet Handbook: version 1.0. Trials 2017;18:280. 10.1186/s13063-017-1978-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Webbe JWH, Duffy JMN, Afonso E, et al. Core outcomes in neonatology: development of a core outcome set for neonatal research. Arch Dis Child Fetal Neonatal Ed 2020;105:fetalneonatal-2019-317501. 10.1136/archdischild-2019-317501 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.COMET Initiative . COMET_database n.d. Available: http://www.comet-initiative.org/Studies [Accessed 10 Feb 2020].
14.Gargon E, Gurung B, Medley N, et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One 2014;9:e99111. 10.1371/journal.pone.0099111 [DOI] [PMC free article] [PubMed] [Google Scholar]
15.Gorst SL, Gargon E, Clarke M, et al. Choosing important health outcomes for comparative effectiveness research: an updated review and user survey. PLoS One 2016;11:1–12. 10.1371/journal.pone.0146444 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Davis K, Gorst SL, Harman N, et al. Choosing important health outcomes for comparative effectiveness research: an updated systematic review and involvement of low and middle income countries. PLoS One 2018;13:1–14. 10.1371/journal.pone.0190695 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Gargon E, Gorst SL, Williamson PR. Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research. PLoS One 2019;14:1–18. 10.1371/journal.pone.0225980 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Kirkham JJ, Davis K, Altman DG, et al. Core outcome Set-STAndards for development: the COS-STAD recommendations. PLoS Med 2017;14:e1002447. 10.1371/journal.pmed.1002447 [DOI] [PMC free article] [PubMed] [Google Scholar]
19.Gargon E, Gorst SL, Williamson PR. Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research. PLoS One 2019;14:e0225980. 10.1371/journal.pone.0225980 [DOI] [PMC free article] [PubMed] [Google Scholar]
20.Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 2011;8:e1000393. 10.1371/journal.pmed.1000393 [DOI] [PMC free article] [PubMed] [Google Scholar]
21.Turnbull AE, Dinglas VD, Friedman LA, et al. A survey of Delphi panelists after core outcome set development revealed positive feedback and methods to facilitate panel member participation. J Clin Epidemiol 2018;102:99–106. 10.1016/j.jclinepi.2018.06.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
22.Williamson P, Altman D, Blazeby J, et al. Driving up the quality and relevance of research through the use of agreed core outcomes. J Health Serv Res Policy 2012;17:1–2. 10.1258/jhsrp.2011.011131 [DOI] [PubMed] [Google Scholar]
23.OECD . Development assistance Committee list of official development assistance recipients 2013. Available: http://www.oecd.org/dac/stats/49483614.pdf
24.Gorst SL, Gargon E, Clarke M, et al. Choosing important health outcomes for comparative effectiveness research: an updated review and identification of gaps. PLoS One 2016;11:1–14. 10.1371/journal.pone.0168403 [DOI] [PMC free article] [PubMed] [Google Scholar]
25.Gargon E, Gorst SL, Harman NL, et al. Choosing important health outcomes for comparative effectiveness research: 4th annual update to a systematic review of core outcome sets for research. PLoS One 2018;13:1–15. 10.1371/journal.pone.0209869 [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Institute of Health Metrics and Evaluation . Global burden of disease 2020. Available: https://vizhub.healthdata.org/gbd-compare/ [Accessed 1 Dec 2020].
27.Murugupillai R, Ranganathan SS, Wanigasinghe J, et al. Development of outcome criteria to measure effectiveness of antiepileptic therapy in children. Epilepsy Behav 2018;80:56–60. 10.1016/j.yebeh.2017.12.011 [DOI] [PubMed] [Google Scholar]
28.Xue Z, Sun J, Li T, et al. How to evaluate the clinical outcome of joint-preserving treatment for osteonecrosis of the femoral head: development of a core outcome set. J Orthop Surg Res 2019;14:317. 10.1186/s13018-019-1364-x [DOI] [PMC free article] [PubMed] [Google Scholar]
29.Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Harbin Consensus Conference Workshop Group, Conference Chairs, Legro RS, et al. Improving the reporting of clinical trials of infertility treatments (imprint): modifying the CONSORT statement†‡. Hum Reprod 2014;29:2075–82. 10.1093/humrep/deu218 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Cross H. A Delphi consensus on criteria for contraindications, assessment indicators and expected outcomes related to tibialis posterior transfer surgery. Int J Lepr Other Mycobact Dis 2005;73:13–21. 10.1489/1544-581X(2005)73[13:ADCOCF]2.0.CO;2 [DOI] [PubMed] [Google Scholar]
32.Moorthy V, Reed Z, Smith PG, et al. Measurement of malaria vaccine efficacy in phase III trials: report of a who consultation. Vaccine 2007;25:5115–23. 10.1016/j.vaccine.2007.01.085 [DOI] [PubMed] [Google Scholar]
33.van 't Hooft J, Duffy JMN, Daly M, et al. A core outcome set for evaluation of interventions to prevent preterm birth. Obstet Gynecol 2016;127:49–58. 10.1097/AOG.0000000000001195 [DOI] [PMC free article] [PubMed] [Google Scholar]
34.Karas J, Ashkenazi S, Guarino A, et al. A core outcome set for clinical trials in acute diarrhoea. Arch Dis Child 2015;100:359–63. 10.1136/archdischild-2014-307403 [DOI] [PubMed] [Google Scholar]
35.Steutel NF, Benninga MA, Langendam MW, et al. Developing a core outcome set for infant colic for primary, secondary and tertiary care settings: a prospective study. BMJ Open 2017;7:1–11. 10.1136/bmjopen-2016-015418 [DOI] [PMC free article] [PubMed] [Google Scholar]
36.Kuizenga-Wessel S, Steutel NF, Benninga MA, et al. Development of a core outcome set for clinical trials in childhood constipation: a study using a Delphi technique. BMJ Paediatr Open 2017;1:e000017. 10.1136/bmjpo-2017-000017 [DOI] [PMC free article] [PubMed] [Google Scholar]
37.Harman NL, Wilding JPH, Curry D, et al. Selecting core outcomes for randomised effectiveness trials in type 2 diabetes (SCORE-IT): a patient and healthcare professional consensus on a core outcome set for type 2 diabetes. BMJ Open Diabetes Res Care 2019;7:e000700. 10.1136/bmjdrc-2019-000700 [DOI] [PMC free article] [PubMed] [Google Scholar]
38.Miller AB, Hoogstraten B, Staquet M, et al. Reporting results of cancer treatment. Cancer 1981;47:207–14. [DOI] [PubMed] [Google Scholar]
39.van der Heijde D, Bellamy N, Calin A, et al. Preliminary core sets for endpoints in ankylosing spondylitis. assessments in ankylosing spondylitis Working group. J Rheumatol 1997;24:2225–9. [PubMed] [Google Scholar]
40.Vos T, Lim SS, Abbafati C, et al. Global burden of 369 diseases and injuries in 204 countries and territories, 1990-2019: a systematic analysis for the global burden of disease study 2019. Lancet 2020;396:1204–22. 10.1016/S0140-6736(20)30925-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
41.Gargon E, Williamson PR, Young B. Improving core outcome set development: qualitative interviews with developers provided pointers to inform guidance. J Clin Epidemiol 2017;86:140–52. 10.1016/j.jclinepi.2017.04.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
42.Ong WL, Schouwenburg MG, van Bommel ACM, et al. A standard set of value-based patient-centered outcomes for breast cancer: the International Consortium for health outcomes measurement (ICHOM) initiative. JAMA Oncol 2017;3:677–85. 10.1001/jamaoncol.2016.4851 [DOI] [PubMed] [Google Scholar]
43.Bloom G, Standing H, Lloyd R. Markets, information asymmetry and health care: towards new social contracts. Soc Sci Med 2008;66:2076–87. 10.1016/j.socscimed.2008.01.034 [DOI] [PubMed] [Google Scholar]
44.Young B, Bagley H. Including patients in core outcome set development: issues to consider based on three workshops with around 100 international delegates. Res Involv Engagem 2016;2:25. 10.1186/s40900-016-0039-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
45.Hall DA, Smith H, Hibbert A, et al. The COMiT'ID study: developing core outcome domains sets for clinical trials of Sound-, Psychology-, and Pharmacology-Based interventions for chronic subjective tinnitus in adults. Trends Hear 2018;22:1–16. 10.1177/2331216518814384 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2021-049981supp001.pdf^{(41.1KB, pdf)}

Reviewer comments

bmjopen-2021-049981.reviewer_comments.pdf^{(309.1KB, pdf)}

Author's manuscript

bmjopen-2021-049981.draft_revisions.pdf^{(1.9MB, pdf)}

Data Availability Statement

All data relevant to the study are included in the article or uploaded as online supplemental information.

[R1] 1.Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials 2017;18:122. 10.1186/s13063-017-1870-2 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R2] 2.Duffy JMN, Ziebland S, von Dadelszen P, et al. Tackling poorly selected, collected, and reported outcomes in obstetrics and gynecology research. Am J Obstet Gynecol 2019;220:71.e1–71.e4. 10.1016/j.ajog.2018.09.023 [DOI] [PubMed] [Google Scholar]

[R3] 3.Sinha IP, Williamson PR, Smyth RL. Outcomes in clinical trials of inhaled corticosteroids for children with asthma are narrowly focussed on short term disease activity. PLoS One 2009;4:e6276. 10.1371/journal.pone.0006276 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R4] 4.Tong A, Crowe S, Gill JS, et al. Clinicians' and researchers' perspectives on establishing and implementing core outcomes in haemodialysis: semistructured interview study. BMJ Open 2018;8:e021198. 10.1136/bmjopen-2017-021198 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R5] 5.Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials 2007;8:1–3. 10.1186/1745-6215-8-39 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R6] 6.Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009;374:86–9. 10.1016/S0140-6736(09)60329-9 [DOI] [PubMed] [Google Scholar]

[R7] 7.Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials 2012;13:132. 10.1186/1745-6215-13-132 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R8] 8.Chevance A, Tran VT, Ravaud P. Improving the generalizability and credibility of core outcome sets (Coss) by a large and international participation of diverse stakeholders. J Clin Epidemiol 2020;125:1–8. 10.1016/j.jclinepi.2020.01.004 [DOI] [PubMed] [Google Scholar]

[R9] 9.Hughes KL, Kirkham JJ, Clarke M, et al. Assessing the impact of a research funder's recommendation to consider core outcome sets. PLoS One 2019;14:e0222418. 10.1371/journal.pone.0222418 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R10] 10.COMET Initiative . COS endorsments, 2020. Available: http://www.comet-initiative.org/COSEndorsement [Accessed 26 Sep 2020].

[R11] 11.Williamson PR, Altman DG, Bagley H, et al. The comet Handbook: version 1.0. Trials 2017;18:280. 10.1186/s13063-017-1978-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R12] 12.Webbe JWH, Duffy JMN, Afonso E, et al. Core outcomes in neonatology: development of a core outcome set for neonatal research. Arch Dis Child Fetal Neonatal Ed 2020;105:fetalneonatal-2019-317501. 10.1136/archdischild-2019-317501 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R13] 13.COMET Initiative . COMET_database n.d. Available: http://www.comet-initiative.org/Studies [Accessed 10 Feb 2020].

[R14] 14.Gargon E, Gurung B, Medley N, et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One 2014;9:e99111. 10.1371/journal.pone.0099111 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R15] 15.Gorst SL, Gargon E, Clarke M, et al. Choosing important health outcomes for comparative effectiveness research: an updated review and user survey. PLoS One 2016;11:1–12. 10.1371/journal.pone.0146444 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R16] 16.Davis K, Gorst SL, Harman N, et al. Choosing important health outcomes for comparative effectiveness research: an updated systematic review and involvement of low and middle income countries. PLoS One 2018;13:1–14. 10.1371/journal.pone.0190695 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R17] 17.Gargon E, Gorst SL, Williamson PR. Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research. PLoS One 2019;14:1–18. 10.1371/journal.pone.0225980 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R18] 18.Kirkham JJ, Davis K, Altman DG, et al. Core outcome Set-STAndards for development: the COS-STAD recommendations. PLoS Med 2017;14:e1002447. 10.1371/journal.pmed.1002447 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R19] 19.Gargon E, Gorst SL, Williamson PR. Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research. PLoS One 2019;14:e0225980. 10.1371/journal.pone.0225980 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R20] 20.Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 2011;8:e1000393. 10.1371/journal.pmed.1000393 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R21] 21.Turnbull AE, Dinglas VD, Friedman LA, et al. A survey of Delphi panelists after core outcome set development revealed positive feedback and methods to facilitate panel member participation. J Clin Epidemiol 2018;102:99–106. 10.1016/j.jclinepi.2018.06.007 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R22] 22.Williamson P, Altman D, Blazeby J, et al. Driving up the quality and relevance of research through the use of agreed core outcomes. J Health Serv Res Policy 2012;17:1–2. 10.1258/jhsrp.2011.011131 [DOI] [PubMed] [Google Scholar]

[R23] 23.OECD . Development assistance Committee list of official development assistance recipients 2013. Available: http://www.oecd.org/dac/stats/49483614.pdf

[R24] 24.Gorst SL, Gargon E, Clarke M, et al. Choosing important health outcomes for comparative effectiveness research: an updated review and identification of gaps. PLoS One 2016;11:1–14. 10.1371/journal.pone.0168403 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R25] 25.Gargon E, Gorst SL, Harman NL, et al. Choosing important health outcomes for comparative effectiveness research: 4th annual update to a systematic review of core outcome sets for research. PLoS One 2018;13:1–15. 10.1371/journal.pone.0209869 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R26] 26.Institute of Health Metrics and Evaluation . Global burden of disease 2020. Available: https://vizhub.healthdata.org/gbd-compare/ [Accessed 1 Dec 2020].

[R27] 27.Murugupillai R, Ranganathan SS, Wanigasinghe J, et al. Development of outcome criteria to measure effectiveness of antiepileptic therapy in children. Epilepsy Behav 2018;80:56–60. 10.1016/j.yebeh.2017.12.011 [DOI] [PubMed] [Google Scholar]

[R28] 28.Xue Z, Sun J, Li T, et al. How to evaluate the clinical outcome of joint-preserving treatment for osteonecrosis of the femoral head: development of a core outcome set. J Orthop Surg Res 2019;14:317. 10.1186/s13018-019-1364-x [DOI] [PMC free article] [PubMed] [Google Scholar]

[R29] 29.Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. 10.1371/journal.pmed.1000097 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R30] 30.Harbin Consensus Conference Workshop Group, Conference Chairs, Legro RS, et al. Improving the reporting of clinical trials of infertility treatments (imprint): modifying the CONSORT statement†‡. Hum Reprod 2014;29:2075–82. 10.1093/humrep/deu218 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R31] 31.Cross H. A Delphi consensus on criteria for contraindications, assessment indicators and expected outcomes related to tibialis posterior transfer surgery. Int J Lepr Other Mycobact Dis 2005;73:13–21. 10.1489/1544-581X(2005)73[13:ADCOCF]2.0.CO;2 [DOI] [PubMed] [Google Scholar]

[R32] 32.Moorthy V, Reed Z, Smith PG, et al. Measurement of malaria vaccine efficacy in phase III trials: report of a who consultation. Vaccine 2007;25:5115–23. 10.1016/j.vaccine.2007.01.085 [DOI] [PubMed] [Google Scholar]

[R33] 33.van 't Hooft J, Duffy JMN, Daly M, et al. A core outcome set for evaluation of interventions to prevent preterm birth. Obstet Gynecol 2016;127:49–58. 10.1097/AOG.0000000000001195 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R34] 34.Karas J, Ashkenazi S, Guarino A, et al. A core outcome set for clinical trials in acute diarrhoea. Arch Dis Child 2015;100:359–63. 10.1136/archdischild-2014-307403 [DOI] [PubMed] [Google Scholar]

[R35] 35.Steutel NF, Benninga MA, Langendam MW, et al. Developing a core outcome set for infant colic for primary, secondary and tertiary care settings: a prospective study. BMJ Open 2017;7:1–11. 10.1136/bmjopen-2016-015418 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R36] 36.Kuizenga-Wessel S, Steutel NF, Benninga MA, et al. Development of a core outcome set for clinical trials in childhood constipation: a study using a Delphi technique. BMJ Paediatr Open 2017;1:e000017. 10.1136/bmjpo-2017-000017 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R37] 37.Harman NL, Wilding JPH, Curry D, et al. Selecting core outcomes for randomised effectiveness trials in type 2 diabetes (SCORE-IT): a patient and healthcare professional consensus on a core outcome set for type 2 diabetes. BMJ Open Diabetes Res Care 2019;7:e000700. 10.1136/bmjdrc-2019-000700 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R38] 38.Miller AB, Hoogstraten B, Staquet M, et al. Reporting results of cancer treatment. Cancer 1981;47:207–14. [DOI] [PubMed] [Google Scholar]

[R39] 39.van der Heijde D, Bellamy N, Calin A, et al. Preliminary core sets for endpoints in ankylosing spondylitis. assessments in ankylosing spondylitis Working group. J Rheumatol 1997;24:2225–9. [PubMed] [Google Scholar]

[R40] 40.Vos T, Lim SS, Abbafati C, et al. Global burden of 369 diseases and injuries in 204 countries and territories, 1990-2019: a systematic analysis for the global burden of disease study 2019. Lancet 2020;396:1204–22. 10.1016/S0140-6736(20)30925-9 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R41] 41.Gargon E, Williamson PR, Young B. Improving core outcome set development: qualitative interviews with developers provided pointers to inform guidance. J Clin Epidemiol 2017;86:140–52. 10.1016/j.jclinepi.2017.04.024 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R42] 42.Ong WL, Schouwenburg MG, van Bommel ACM, et al. A standard set of value-based patient-centered outcomes for breast cancer: the International Consortium for health outcomes measurement (ICHOM) initiative. JAMA Oncol 2017;3:677–85. 10.1001/jamaoncol.2016.4851 [DOI] [PubMed] [Google Scholar]

[R43] 43.Bloom G, Standing H, Lloyd R. Markets, information asymmetry and health care: towards new social contracts. Soc Sci Med 2008;66:2076–87. 10.1016/j.socscimed.2008.01.034 [DOI] [PubMed] [Google Scholar]

[R44] 44.Young B, Bagley H. Including patients in core outcome set development: issues to consider based on three workshops with around 100 international delegates. Res Involv Engagem 2016;2:25. 10.1186/s40900-016-0039-6 [DOI] [PMC free article] [PubMed] [Google Scholar]

[R45] 45.Hall DA, Smith H, Hibbert A, et al. The COMiT'ID study: developing core outcome domains sets for clinical trials of Sound-, Psychology-, and Pharmacology-Based interventions for chronic subjective tinnitus in adults. Trends Hear 2018;22:1–16. 10.1177/2331216518814384 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Inclusion of participants from low-income and middle-income countries in core outcome sets development: a systematic review

Jamlick Karumbi

Sarah L Gorst

David Gathara

Elizabeth Gargon

Bridget Young

Paula R Williamson

Series information

Abstract

Objective

Design

Data sources

Eligibility criteria for selecting studies

Data extraction and synthesis

Results

Conclusion

Strengths and limitations of this study.

Background

How are COS developed?

Methods

Study selection

Inclusion criteria

Identification of relevant studies

Figure 1.

Data extraction

Analysis

Patients and public involvement

Findings

General description

Year of publication

Figure 2.

Scope of included studies

Table 1.

Figure 3.

Characteristics of participants involved in consensus process

Table 2.

Methods for selection of outcomes

Table 3.

Figure 4.

Discussion

Conclusion

Supplementary Material

Acknowledgments

Footnotes

Data availability statement

Ethics statements

Patient consent for publication

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases