Evaluation of the reporting quality of clinical practice guidelines on pancreatic cancer using the RIGHT checklist

Abstract

Background

The International Reporting Items for Practice Guidelines in Health Care (RIGHT) statement is a set of recommendations for the reporting in clinical practice guidelines (CPGs). We aimed to assess the reporting quality of CPGs for pancreatic cancer following the RIGHT checklist.

Methods

Guidelines for pancreatic cancer were identified by searching electronic databases, guideline databases, and medical society websites. The reporting quality was evaluated by calculating the adherence to the items of the RIGHT checklist and summarizing them over the seven domains and the entire checklist. We also present results stratified by selected characteristics.

Results

A total of 22 guidelines were found eligible. Mean overall adherence to the RIGHT items was 60.0%. All guidelines adhered to the RIGHT items 3, 7a, 13a, while no guidelines reported the items 14c or 18b, which are some of the topics dealing with rationale for recommendations and funding source, respectively. Of the seven domains of the RIGHT checklist, “Review and quality assurance” and “Funding and declaration and management of interests” had the lowest reporting rates (25.0% and 43.2%, respectively); the remaining five domains had reporting rates >50%. CPGs that reported funding support, were published in higher-impact journals, and that applied a grading system for the quality of evidence, tended to have higher reporting rates.

Conclusions

Our results show that reporting quality of pancreatic cancer CPGs still needs to be improved. The use of the RIGHT statement should be encouraged when developing new guidelines.

Introduction

Pancreatic cancer is a highly lethal disease, over 430,000 deaths—4.5% of all cancer-related deaths—in 2018 worldwide (1). The current five-year survival rate for pancreatic cancer has increased from 2.5% in the 1970s to 10.5% in 2017, representing a modest improvement in survival when compared with other cancers reported in the SEER database (2). Because of nonspecific symptoms, approximately 80% of patients are typically diagnosed at advanced stage, when surgical resection is no longer likely to be beneficial (3,4). At present, the most accepted therapeutic approach for advanced disease is conventional cytotoxic chemotherapy, providing only months of additional survival benefit (5).

Although clinical progress has been slow, the sequencing of cancer genomes has advanced our understanding of the biology of pancreatic cancer. The complexity of the genome indicates that pancreatic cancer is a heterogeneous disease and individualized therapy is required (6). As a consequence, new potentially practice-changing evidence is emerging, such as the addition of pembrolizumab as an option for mismatch repair-deficient or microsatellite instability-high tumors (7), and targeted therapy for BRCA mutation or NTRK fusions (8). In addition, palliative and supportive care, follow-up after treatment, and the management of adverse events need better consideration (9). Of note, the existing evidence regarding these novel therapies is still limited in terms of the number of studies and sample size.

Many clinical practice guidelines (CPGs) have been published to address these issues, balance the benefits and harms, and assist clinicians individualize clinical decision making for patients with pancreatic cancer. As the number of guidelines increases, concerns have risen about their reliability, which may be affected by the validity of the recommendations and the quality in reporting and methodology. Explicit reporting of the methods and procedures ensures high quality of guidelines, which in turn leads to reliable recommendations (10). Thus, the quality of CPGs should be reviewed critically before recommendations are implemented into clinical practice.

Several instruments have been developed to assess the quality of CPGs. Among them, the International Reporting Items for Practice Guidelines in Health Care (RIGHT) checklist focuses on guideline reporting. The tool contains seven domains (11). RIGHT was published in 2016, and has been widely accepted for the reporting quality assessment of guidelines. To our knowledge, the reporting quality of CPGs for pancreatic cancer has not yet been assessed. Therefore, we aimed to identify guidelines for pancreatic cancer published in the past three years and use the RIGHT tool to critically evaluate their reporting quality. This study provides data for guideline developers to improve their guideline reporting process for pancreatic cancer.

Methods

Study design

We conducted a critical appraisal of the reporting quality of CPGs for pancreatic cancer using the RIGHT checklist.

Literature search

We searched Medline (via PubMed), Chinese Biomedical Literature Database (CBM), Wan Fang Database and Chinese National Knowledge Infrastructure (CNKI) from their inception until November 27, 2020 to identify potentially relevant records. The specific search terms included pancreatic cancer, practice guideline, guidance, and recommendation. We also searched the website of The National Institute for Health and Care Excellence (NICE, https://www.nice.org.uk/), The National Comprehensive Cancer Network (NCCN, https://www.nccn.org/), World Health Organization guidelines (WHO, https://www.who.int/publications/guidelines/year/en/), Scottish Intercollegiate Guidelines Network (SIGN, https://www.sign.ac.uk/our-guidelines/) and Guidelines International Network (GIN, https://guidelines.ebmportal.com/) as well as Google Scholar to identify additional literature. Language was restricted to Chinese and English.

Inclusion and exclusion criteria

We included the most recent versions of CPGs focusing on screening, testing, diagnosis, treatment, or management of pancreatic cancer, published between 2018 and 2020, with full-text accessible. We excluded translations, summaries and interpretations of guidelines, as well as draft or unpublished guidelines.

Screening

CPG selection was performed in two stages. First, titles and abstracts were screened; then, the full texts of potentially relevant articles and related supplementary materials were collected to identify eligible CPGs according to the criteria. The selection was performed independently by two authors (MY Sun and XX Chen). Conflicts were settled by consensus, consulting a third reviewer (YF Ma) if necessary.

Data extraction of guidelines

Data extraction was conducted by two authors independently (XH Ma and YJ Yang). In case of inconsistent interpretation, consensus was reached by consultation with a third author (XJ Zhang). A standardized electronic extraction form was used to obtain the relevant data from each guideline. Extracted data included first author, publication year, publication language, developers, region/country where the CPG was developed, format of publication (peer-reviewed journal, or website only), impact factor (IF) of the journal according to the Science Citation Index (SCI) journal, and the scope/purpose and target population of the CPG.

Reporting quality assessment using RIGHT checklists

The RIGHT tool was used to assess the reporting quality of include CPGs. This instrument includes seven domains with a total of 35 items: basic information (six items), background (eight items), evidence (five items), recommendations (seven items), review and quality assurance (two items), funding and conflicts of interest statements and management (four items), and other information (three items). Each of these items was assessed for every guideline independently by two authors (QW Zhang and MM Jia). Disagreements were settled by consulting a third reviewer (JL Lu). The authors performing the assessment were trained by a member of the RIGHT tool working group, and were familiar with the use of this tool. Most items were rated as “Reported” or “Not reported”, based on the protocol of RIGHT instrument. “Reported” means that the relevant information was fully reported, whereas “Not reported” means that some relevant information was missing. Items that were not appropriate for evaluation in a specific guideline were assigned as “not applicable”.

Statistical analysis

We report the absolute number and percentage of guidelines reporting each item, and the mean percentages of items reported over all guidelines for the RIGHT checklist overall, and for each domain separately. We also present the mean overall reporting rates stratified by the year of publication, funding support, grading system and format of publication (journal categorized by IF, or website only).

Results

Identification of guidelines

Figure 1 shows the flowchart of the search and selection procedure. The electronic searches yielded 1,548 potentially relevant studies, of which 124 duplicate records were excluded. After exclusion of references by the evaluation of title and abstract, we retrieved 35 references for full-text review. Finally, 22 reports qualified for inclusion in our analysis.

Figure 1.

The flow chart of the selection process.

Characteristics of selected guidelines

Table 1 shows the characteristics of all included CPGs. Five CPGs were published in 2018, seven in 2019, and ten in 2020. Nineteen CPGs were in English, and three in Chinese. Of the 22 guidelines, 14 targeted pancreatic cancer in general, two PDCA exclusively, one chronic pancreatitis, one targeted patients with potentially curable pancreatic cancer, and two targeted individuals with increased risk of pancreatic cancer based on family history or germline mutation status (high-risk individuals). The included guidelines covered a broad range of aspects of pancreatic cancer, including screening, diagnosis, treatment and management. Six CPGs were originally from the United States, four from China, two from Europe (multinational), two from France, and one each from Japan, Poland, Australia, UK and Spain; three guidelines were developed globally. Eleven guidelines were supported by government or institutional funds, and 11 did not report their funding sources.

Table 1. Characteristics of eligible CPGs.

No	Year	Country/region	Language	Developer	Grading system	IF	Funding source
1 (12)	2020	China	Chinese	CRHA, CMA (CSS. CPSA)	GRADE	None	Unreported
2 (13)	2020	Europe	English	ESTRO, ACROP	Unreported	4.856	Society
3 (14)	2020	China	Chinese	NCRC-RT, CIMB (CSIP), CIM (CFC)	Unreported	None	Unreported
4 (15)	2019	Japan	English	JPS	GRADE	2.92	Society
5 (16)	2020	Global	English	IAP, APA, JPS, EPC	GRADE	3.629	Unreported
6 (17)	2020	France	English	INCA	GRADE	4.848	Government
7 (18)	2019	Australia	English	AGITG	Unreported	2.948	Government
8 (19)	2020	Spain	English	SEOM, AEC, SEOR, SEEN, SEPD, SERAM, SEAP	GRADE	2.737	Society
9 (20)	2020	USA	English	ASCO	GRADE	32.956	Society
10 (21)	2020	Europe	English	ESMO	Unreported	5.329	Society
11 (22)	2019	USA	English	ASCO	GRADE	32.956	Society
12 (23)	2020	Global	English	CAPS consortium	Unreported	19.819	Government
13 (24)	2019	Poland	English	Working group of PCC	Self-defined	None	Unreported
14 (25)	2019	USA	English	ASTRO	ASTRO	2.948	Unreported
15 (26)	2019	USA	English	USPSTF	USPSTF	45.54	Government
16 (27)	2019	USA	English	ASCO	GRADE	32.956	Society
17 (28)	2018	France	English	Several French medical and surgical societies	GRADE	3.57	Unreported
18 (29)	2018	Global	English	Working group	GRADE	3.629	Unreported
19 (30)	2018	China	Chinese	PCCCAA	Unreported	None	Unreported
20 (31)	2018	China	English	NHC of the PRC	Unreported	4.135	Unreported
21 (32)	2018	UK	English	NICE	Unreported	Website	Unreported
22 (33)	2020	USA	English	NCCN	Self-defined	Website	Unreported

CPGs, clinical practice guidelines; AEC, the Spanish Association of Surgeons; AGITG, The Australasian Gastrointestinal Trials Group; APA, the American Pancreatic Association; ASCO, The American Society of Clinical Oncology; ASTRO, American Society for Radiation Oncology; CAPS, international Cancer of the Pancreas Screening; CIM (CFC), Committee of Interventional Medicine (Cancer Foundation of China); CIMB (CSIP), Committee of Interventional Medicine and Bioengineering (Chinese Society of Interventional Physicians); CPSA (CSS.CMA), Chinese Pancreatic Surgery Association (Chinese Society of Surgery. Chinese Medical Association); CRHA, Chinese Research Hospital Association; EPC, European Pancreatic Club; ESMO, European Society for Medical Oncology; IAP, the International Association of Pancreatology; INCA, The French National Cancer Institute; JPS, Japan Pancreas Society; NCRC-RT, National Clinical Research Center for Radiology and Therapy; NHC of the PRC, The National Health Commission of the People’s Republic of China; PCC, the Polish Pancreatic Club; PCCCAA, Pancreatic cancer committee of Chinese anti-cancer association; SEOM, the Spanish Society of Medical Oncology; SEOR, the Spanish Society of Radiation Oncology; SEEN, the Spanish Society of Endocrinology; SEPD, the Spanish Society of Digestive Pathology; SERAM, the Spanish Society of Medical Radiology; SEAP, the Spanish Society of Pathology; USPSTF, US Preventive Services Task Force.

Analysis of overall reporting quality

Of the seven domains, the “Background” domain had the highest mean reporting rate of 76.7%; the “Basic information”, “Evidence”, “Recommendations” and “Other information” domains had also reporting rates greater than 50% (Figure 2). The reporting rate of “Funding and declaration and management of interests” domain was 43.2%. The “Review and quality assurance” domain had the lowest reporting quality, with a reporting rate of 25.0%.

Figure 2.

The reporting rates of each RIGHT checklist domain in the included clinical practice guidelines. RIGHT, International Reporting Items for Practice Guidelines in Health Care.

The mean overall reporting rate was 60%; 17 of the 35 sub-items were reported by at least 60% of the CPGs (Table 2). All guidelines adhered to the items 3 (abbreviations and acronyms), 7a (target population), and 13a (recommendations). No guideline reported the items 14c (other factors that may affect recommendations) or 18b (the role of funders).

Table 2. The reporting rates of each RIGHT checklist item in the included clinical practice guidelines (11).

Section/topic	No.	Item	Reported, n (%)	Not reported, n (%)	Not applicable, n (%)
Basic Information
Title/subtitle	1a	Identify the report as a guideline, that is, with ‘guideline(s)’ or ‘recommendation(s)’ in the title	20 (90.9)	2 (9.1)	0 (0)
	1b	Describe the year of publication of the guideline	7 (31.8)	15 (68.2)	0 (0)
	1c	Describe the focus of the guideline, such as screening, diagnosis, treatment, management, prevention, or others	18 (81.8)	4 (18.2)	0 (0)
Executive summary	2	Provide a summary of the recommendations contained in the guideline	9 (40.9)	13 (59.1)	0 (0)
Abbreviations and acronyms	3	Define new or key terms, and provide a list of abbreviations and acronyms if applicable	22 (100.0)	0 (0)	0 (0)
Corresponding developer	4	Identify at least 1 corresponding developer or author who can be contacted about the guideline	18 (81.8)	4 (18.2)	0 (0)
Background
Brief description of the health problem(s)	5	Describe the basic epidemiology of the problem, such as the prevalence/incidence, morbidity, mortality, and burden (including financial) resulting from the problem	21 (95.5)	1 (4.5)	0 (0)
Aim(s) of the guideline and specific objectives	6	Describe the aim(s) of the guideline and specific objectives, such as improvements in health indicators (e.g., mortality and disease prevalence), quality of life, or cost savings	18 (81.8)	4 (18.2)	0 (0)
Target population(s)	7a	Describe the primary population(s) that is affected by the recommendation(s) in the guideline	22 (100.0)	0 (0)	0 (0)
	7b	Describe any subgroups that are given special consideration in the guideline	20 (90.9)	2 (9.1)	0 (0)
End users and settings	8a	Describe the intended primary users of the guideline (such as primary care providers, clinical specialists, public health practitioners, program managers, and policymakers) and other potential users of the guideline	18 (81.8)	4 (18.2)	0 (0)
	8b	Describe the setting(s) for which the guideline is intended, such as primary care, low- and middle-income countries, or inpatient facilities	5 (22.7)	17 (77.3)	0 (0)
Guideline development groups	9a	Describe how all contributors to the guideline development were selected and their roles and responsibilities (e.g., steering group, guideline panel, external reviewers, systematic review team, and methodologists)	12 (54.5)	10 (45.5)	0 (0)
	9b	List all individuals involved in developing the guideline, including their title, role(s), and institutional affiliation(s)	19 (86.4)	3 (13.6)	0 (0)
Evidence
Health care questions	10a	State the key questions that were the basis for the recommendations in PICO (population, intervention, comparator, and outcome) or other format as appropriate	10 (45.5)	12 (54.5)	0 (0)
	10b	Indicate how the outcomes were selected and sorted	8 (36.4)	14 (63.6)	0 (0)
Systematic reviews	11a	Indicate whether the guideline is based on new systematic reviews done specifically for this guideline or whether existing systematic reviews were used	13 (59.1)	9 (40.9)	0 (0)
	11b	If the guideline developers used existing systematic reviews, reference these and describe how those reviews were identified and assessed (provide the search strategies and the selection criteria, and describe how the risk of bias was evaluated) and whether they were updated	14 (63.6)	0 (0)	8 (36.4)
Assessment of the certainty of the body of evidence	12	Describe the approach used to assess the certainty of the body of evidence	14 (63.6)	8 (36.4)	0 (0)
Recommendations
Recommendations	13a	Provide clear, precise, and actionable recommendations	22 (100.0)	0 (0)	0 (0)
	13b	Present separate recommendations for important subgroups if the evidence suggests that there are important differences in factors influencing recommendations, particularly the balance of benefits and harms across subgroups	20 (90.9)	2 (9.1)	0 (0)
	13c	Indicate the strength of recommendations and the certainty of the supporting evidence	14 (63.6)	0 (0)	8 (36.4)
Rationale/explanation for recommendations	14a	Describe whether values and preferences of the target population(s) were considered in the formulation of each recommendation. If yes, describe the approaches and methods used to elicit or identify these values and preferences. If values and preferences were not considered, provide an explanation	12 (54.5)	10 (45.5)	0 (0)
	14b	Describe whether cost and resource implications were considered in the formulation of recommendations. If yes, describe the specific approaches and methods used (such as cost-effectiveness analysis) and summarize the results. If resource issues were not considered, provide an explanation	12 (54.5)	10 (45.5)	0 (0)
	14c	Describe other factors taken into consideration when formulating the recommendations, such as equity, feasibility, and acceptability	0 (0)	22 (100.0)	0 (0)
Evidence to decision processes	15	Describe the processes and approaches used by the guideline development group to make decisions, particularly the formulation of recommendations (such as how consensus was defined and achieved and whether voting was used)	13 (59.1)	9 (40.9)	0 (0)
Review and quality assurance
External review	16	Indicate whether the draft guideline underwent independent review and, if so, how this was executed and the comments considered and addressed	8 (36.4)	14 (63.6)	0 (0)
Quality assurance	17	Indicate whether the guideline was subjected to a quality assurance process. If yes, describe the process	3 (13.6)	19 (86.4)	0 (0)
Funding and declaration and management of interests
Funding source(s) and role(s) of the funder	18a	Describe the specific sources of funding for all stages of guideline development	11 (50.0)	11 (50.0)	0 (0)
	18b	Describe the role of funder(s) in the different stages of guideline development and in the dissemination and implementation of the recommendations	0 (0)	11 (50.0)	11 (50.0)
Declaration and management of interests	19a	Describe what types of conflicts (financial and nonfinancial) were relevant to guideline development	16 (72.7)	6 (27.3)	0 (0)
	19b	Describe how conflicts of interest were evaluated and managed and how users of the guideline can access the declarations	11 (50.0)	11 (50.0)	0 (0)
Other information
Access	20	Describe where the guideline, its appendices, and other related documents can be accessed	10 (45.5)	12 (54.5)	0 (0)
Suggestions for further research	21	Describe the gaps in the evidence and/or provide suggestions for future research	14 (63.6)	8 (36.4)	0 (0)
Limitations of the guideline	22	Describe any limitations in the guideline development process (such as the development groups were not multidisciplinary or patients’ values and preferences were not sought), and indicate how these limitations might have affected the validity of the recommendations	11 (50.0)	11 (50.0)	0 (0)

RIGHT, Reporting Items for Practice Guidelines in Healthcare. (Details of the RIGHT checklist is available on: http://www.right-statement.org/right-statement/checklist).

Thirteen guidelines had an overall reporting rate above the mean (>60%). The two CPGs with the highest reporting rate (85.7%) were developed by The American Society of Clinical Oncology (ASCO). The guideline that had the lowest overall reporting rate (31.4%) was published in Chinese and developed by the National Clinical Research Center for Radiology and Therapy.

Subgroup analyses of reporting quality

The mean overall reporting rate was 50.3% in CPGs published in 2018, 67.3% in 2019, and 61.1% in 2020. We noted a higher reporting proportion in guidelines that were published in journals with IF ≥10 (73.1%); while guidelines published in Chinese journals were poorly reported (47.6%). The reporting quality was better for guidelines using an evidence grading system (69.2%) compared to 45.7% for those not using any grading system. Funding support also seemed to influence the reporting quality of the guidelines: reporting proportion was 64.9% in guidelines with funding support and 56.4% for those without, or not reporting funding support (Figure 3).

Figure 3.

The reporting proportions of included clinical practice guidelines stratified by selected characteristics. IF, impact factor.

Discussion

Pancreatic cancer is a highly fatal disease that requires high-quality health care to improve survival. Clinical guidelines offer timely clinical recommendations on how to optimize patient management (34). Concerns have been raised that the low quality of reporting guidelines may negatively affect patients. This is to our knowledge the first systematic review of reporting quality in pancreatic cancers guidelines using RIGHT tool.

Most guidelines included in our analysis had several deficiencies in reporting. Items pertaining to the external review as well as the funding and declaration and management of interests were poorly reported. In particular, no guideline described the role of funder(s) in the different stages of guideline development and in the dissemination and implementation of the recommendations. The reasons are difficult to ascertain. Both two items contain several aspects that may have contributed to the reluctance of developers to report them. For example, the funding for CPGs is likely to be primarily used for the publication and dissemination, and the authors may feel that it is not related to the quality of the evidence behind the recommendations, and thus see it unnecessary to report it in detail.

Although the lack of some basic information does not automatically mean that the recommendations are of poor quality, complete and transparent reporting nevertheless facilitates the acceptance and effective use of the guideline. For example, a well-constructed executive summary enables a clear presentation of the different management options and makes key recommendations easily identifiable. The year of publication in the title allows to assess immediately whether the guideline is up-to-date, helping readers find the latest recommendations (35). However, in our analysis, both the year of publication in the title and a summary of recommendations were rarely reported. To ensure high quality, reporting of basic information should not be neglected.

The reporting quality of the “Background” domain tended to be good, except for one item that describes the target setting and facility type of the guideline. The setting should be defined, because a guideline may be applicable only, for example, in selected geographic regions, primary care, or referral facilities. Most guidelines did not report the information about the health care questions the guideline intends to answer. These items of the RIGHT checklist highlight the importance of PICO (patient, intervention, comparison, and outcome) questions, and how the outcomes are selected and sorted. The PICO framework ensures the clinical questions cover the whole scope of the guideline (36). The lack of this information may be one explanation to the variable quality in CPGs.

We did not find any evidence that the overall reporting quality of CPGs for pancreatic cancer would have improved over time. This result may reflect the fact that the RIGHT tool is not yet widely used in the development of guidelines. In fact, the Appraisal of guidelines for research & evaluation II (AGREE II) instrument, which is a similar tool for methodological quality and was first published in 2010, six years before RIGHT (37,38), was also used infrequently: only one of 22 included CPGs reported the use of this tool for methodological adaptation (15). Therefore, to improve the overall quality of CPGs, the developers should use appropriate tools for guideline reporting and methodology, such as the RIGHT tool and AGREE II.

We found several factors correlating with reporting quality. First, a good quality of guideline reporting was seen in leading journals, which might reflect the transparent and high quality of these journals. Second, we found more complete reporting in guidelines that declared funding support. The reason may be that guideline development is costly and time-consuming, and therefore adequate funding and resources can promote the quality of the guideline (39). Third, the guidelines using evidence grading system such as GRADE presented a better quality in reporting. The use of a grading system may reflect a more comprehensive training or experience in CPG development methodology, which may also improve the quality of reporting.

We acknowledge our study has some limitations. The characteristics of the CPGs were heterogeneous, which may partly explain the observed variations in reporting completeness; thus, the differences between the subgroups we observed may be confounded by a range of other factors. The quality of our analysis is also limited by the inclusion criteria. We only included guidelines that were published in Chinese and English. Inclusion of the Chinese guidelines tended to be poorly reported, decreasing the overall estimates substantially. Therefore, the high proportion of English-language guidelines, often published in high-impact scientific journals, may have led to overestimation of the overall reporting quality.

Questions to be further discussed and considered

Question 1: What impact do you think the low reporting quality of clinical practice guidelines on pancreatic cancer will have on clinicians and clinical practices?

Expert opinion: Dr. Kunihiro Tsuchida

We think that the low reporting quality of clinical practice guidelines on pancreatic cancer will hinder internationally equal accessibility of clinical data and results. In this manuscript, we report the low reporting rates of “review and quality assurance” and “funding and declaration and management of interests”. In some occasions, society and committee are involved in making guidelines. This might be one reason for the low reporting rate of funding and management. Low rate of review and quality assurance is problematic. Guidelines should be reviewed more widely by independent review board, although we understand that standardization of intractable pancreatic cancers is very difficult.

Expert opinion: Dr. Mark Schattner

Guidelines must always be judicially applied. Unlike “standards of practice” which can be very broadly and often uniformly applied, “guidelines” need to be interpreted to make sure that they are being used in the correct clinical context. For example, they may only be appropriate for one subset of patients with a given disease. It is critical that the guidelines specify how and when they are intended to be used. Lower quality clinical practice guidelines may not recognize and call out this limitation which could result in their use in populations that were not well studied. It is critical that the clinician understand not only what the guidelines recommend but more importantly where the guidelines are intended to be used, the source data for the guidelines, and the inherent limitations of the guidelines. It is only with knowledge of all these factors that the CPG can be correctly applied.

Expert opinion: Dr. Matthew H. G. Katz

Unfortunately, there is a significant amount of bias in the way patients with pancreatic cancer are managed. That is due to several factors. Data for patients with localized disease is particularly biased because the amount and quality of the data that exist to inform the management of these patients is quite low. Therefore, many decisions are made on the basis of “experiences” and “anecdotes”. Without rigorous methodology to the guidelines, management of patients will continue to occur based on preference, bias and experience rather than on data. It is also important to note that existing guidelines to a very poor job of describing value. So patients may be subject to treatments that may improve outcomes statistically but not provide significant value.

Expert opinion: Dr. Salvatore Paiella

CPGs have been desired because of the general perception that medical assistance could have improved, providing end-users (clinicians) the best and updated scientific knowledge for the best patient’s care. On the one side, it is hard for the end-users to ascertain whether the guideline recommendations are trustworthy since the knowledge and the application of the various grading systems are time-consuming and complex. It is difficult to determine if the low reporting quality of the CPGs found in this manuscript is somehow impactful on managing pancreatic cancer patients. It has been reported that physicians are not prone to incorporate and follow precisely CPGs if they think they are based on opinions or poor evidence or if they are unbalanced toward the physician’s point of view (40). It has been reported that adherence to guidelines may positively impact survival, even for pancreatic cancer (41). On the opposite, it may be argued that non-adherence may produce a late diagnosis, an inappropriate treatment, and cost issues. The lowest reporting quality, the highest (apparent) untrustworthiness.

Question 2: What do you think the most important aspects needed for developing high-quality clinical practice guidelines on pancreatic cancer are?

Expert opinion: Dr. Kunihiro Tsuchida

Internationally recognized and open guidelines are important for developing high-quality clinical practice guidelines on pancreatic cancers for internationally unified therapies for future. For this purpose, translation in English and availability in other nations are important. For patients, explanation of important aspects of guidelines by clinicians and social workers will be of some help.

Expert opinion: Dr. Mark Schattner

I think the instruments (such as RIGHT checklist) developed to systemically critique guidelines are critical. These instruments remind a clinician of all of the factors that should be included in a high-quality practice guideline and if reported should allow one to quickly identify high quality clinical practice guidelines. Guidelines should also be reviewed and updated on an ongoing basis to make sure that they remain current in this rapidly changing field.

Expert opinion: Dr. Matthew H. G. Katz

More high quality data is needed. More rigorous methodology in guidelines is necessary. Attention to healthcare value is critical (so what if a drug improves survival by a couple of weeks—erlotinib—what about cost? And descriptions of how guidelines affect certain patient populations. What about low resource patients or settings, for example. Certain drugs or procedures may not be available in every clinical setting, and guidelines need to be tailored to the specific setting—or at least describe limitations of the recommendations in certain situations.

Expert opinion: Dr. Salvatore Paiella

High-quality CPGs should focus on several aspects; some of them are discussed as follows. First, an irreproachably documented systematic review has to be performed, involving methodologists since the first phase of the production. Second, they should use a grading system to increase trustworthiness. Third, they should include stakeholders and patients. This will ensure proper balancing. Fourth, they should be updated constantly. Fifth, they should have development, dissemination, and implementation strategies. Sixth, they should state the limitations. Seventh, they should receive and convey the feedback of both end-users and patients. Eight, they should have editorial independence and no sponsorship.

Question 3: How do you think conflicts of interest in the guidelines should be handled?

Expert opinion: Dr. Kunihiro Tsuchida

Conflict of interest (COI) is in 19b. Its reporting rate is 47.8%. Since new drugs such as immune checkpoint inhibitors are becoming available, COI could exist for clinical studies. However, proper management of COI is possible in each hospital and/or medical universities. Declaration of COI is important for clinical studies not only in pancreas cancers but also in other intractable diseases.

Expert opinion: Dr. Mark Schattner

Conflicts of interest can clearly result in some bias (either intentional or unintentional) in the development and wording of guidelines. It is critical that these conflicts be recognized and any influence they have on the guidelines be minimized. It is also important that these conflicts are completely and accurately reported to the reader so they can develop fully informed opinions about the guidelines.

Expert opinion: Dr. Matthew H. G. Katz

At ASCO, people with conflict of interest are not allowed to participate in development of guidelines. That is important. However, it does not solve all conflict of interest. For example, the ASTRO guidelines, which really only address one methodology, obviously interpret ambiguous clinical data through the lens of individuals who practice in that specialty. Guidelines need to get a broad range of input from providers of different specialties, who practice in different resource settings, to get the most generalizable and accurate recommendations.

Expert opinion: Dr. Salvatore Paiella

CPGs’ Authors commonly report conflicts of interest. A recent systematic review noted that about a half of CPGs claim at least one financial conflict of interest, especially in the oncology field, and that many are undisclosed. Payments included speaking engagements, travel and accommodation expenses coverage, consulting honoraria, research payments, investments, ownership stakes, and royalties (42). Conflicts of interest must be reported for each of the Authors involved, using a separate consultable document. A clear example to follow has been provided by the Association of Scientific Medical Societies in Germany (AWMF) (43).

Conclusions

In conclusion, the reporting quality of CPGs on pancreatic cancer has remained stable on a suboptimal level over the past three years. The low reporting quality of Chinese-language guidelines was particularly worrying. As adequate reporting is essential for the effective dissemination and implementation of the recommendations, the use of the RIGHT tool should be encouraged when developing CPGs.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.