Completeness of reporting of case reports in high‐impact medical journals

Abstract

Introduction

Case reports represent a relevant, timely and important study design in advancing medical scientific knowledge. They allow integration between clinical practice and clinical epidemiology. We aimed to assess the completeness of reporting (COR) of case reports published in high‐impact journals. We assessed the COR of case reports using the CARE guidelines.

Materials and methods

We selected three high‐impact journals and one journal specialized in publishing case reports, in which we included all published case reports from July to December 2017. Median COR score was calculated per study, and CORs were compared between journals with and without endorsement of CARE guidelines.

Results

One hundred and fourteen case reports were included. Overall median COR was 81%, IQR [63%‐96%]. Sections with the highest COR (84%‐100%) were patient information, clinical findings, therapeutic intervention, follow‐up and outcomes, discussion and informed consent. Sections with the lowest COR were title, keywords, timeline and patient perspective (2%‐34%). COR was higher in journals endorsing in comparison to those not endorsing CARE guidelines (77% vs 65%), respectively, median difference = −12% 95% CI [−16% to −7%].

Discussion

Overall completeness of case reports in included journals is high especially for CARE endorsing and dedicated journals but reporting of some items could be improved. Ongoing and future evaluations of endorsement status of reporting guidelines in medical journals should be assessed to improve completeness and reduce waste of clinical research, including case reports.

1. INTRODUCTION

According to the International Epidemiological Association, case reports are detailed descriptions of a few patients or clinical cases (frequently, just one sick person) with an unusual disease or complication, uncommon combinations of diseases, an unusual or misleading semiology, cause or outcome (maybe a surprising recovery). They often are preliminary observations that are later refuted and they cannot estimate disease frequency or risk (eg for lack of a valid denominator). ¹

However, case reports have a time‐honoured and rich tradition in medicine and scientific publication. Case reports represent a relevant, timely and important study design in advancing medical scientific knowledge, and they allow integration between clinical practice and clinical epidemiology. ² Sir William Osler used to outline that medical professionals should write and register the unusual about their clinical practice and reported it in a short and clear way. ³ Mainly, case reports have historically been important in (a) recognizing new, rare and unknown diseases, (b) evaluating therapeutic effects, adverse events, surveillance and costs of interventions and (c) improving problem‐based medical education. ⁴

Case report findings are not generalizable, do not address causal inference or explanatory mechanisms, and emphasize low‐probability events. They do not provide strong causal evidence in comparison with other designs, such as analytical observational studies or randomized controlled trials. ⁵ Nevertheless, they should be reported as complete as possible in order to ensure their appropriate assessment and potential usefulness.

Case reports comprise a significant proportion of the articles in many indexed medical journals. Case report volume, indexed on Embase and MEDLINE, increased by 45% from 49 918 in 2000 to 72 388 in 2010. ⁵ , ⁶ This amount of publication explains why it is important and pertinent to study the completeness and the proper writing of these articles. It could be a potential and wide way to reduce “waste in research”. ⁷

In 2013, the CARE guidelines (Consensus‐based Clinical Case Reporting Guideline Development) were developed to improve that quality and established a systematic tool of completeness of a case report. Statisticians, methodologists and several clinical professionals form the CARE group. Initially, they created a checklist that consisted in 13 topics required to classify a case report as “complete.” In the last 2016 update, a fourteenth topic was added accomplishing with a total of 31 items (Table 1). ⁸

Table 1.

CARE guideline 2016 checklist^*

Topic	Item	Question
Title	1	The words “case report” should be in the title along with what is of greatest interest in this case
Keywords	2	The key elements of this case in 2‐5 keywords
Abstract	3a	Introduction—What is unique about this case? What does it add to the medical literature?
	3b	The main symptoms of the patient and the important clinical findings
	3c	The main diagnoses, therapeutics interventions and outcomes
	3d	Conclusion—What are the main “take‐away” lessons from this case?
Introduction	4	Brief background summary of this case referencing the relevant medical literature
Patient Information	5a	Demographic information (such as age, gender, ethnicity, occupation)
	5b	Main symptoms of the patient (his or her chief complaints)
	5c	Medical, family and psychosocial history including co‐morbidities and relevant genetic information
	5d	Relevant past interventions and their outcomes
Clinical Findings	6	Describe the relevant physical examination (PE) findings
Timeline	7	Depict important milestones related to your diagnoses and interventions (table or figure)
Diagnostic Assessment	8a	Diagnostic methods (such as PE, laboratory testing, imaging, questionnaires)
	8b	Diagnostic challenges (such as financial, language or cultural)
	8c	Diagnostic reasoning including other diagnoses considered
	8d	Prognostic characteristics (such as staging in oncology) where applicable
Therapeutic Intervention	9a	Types of intervention (such as pharmacologic, surgical, preventive, self‐care)
	9b	Administration of intervention (such as dosage, strength, duration)
	9c	Changes in intervention (with rationale)
Follow‐up and Outcomes	10a	Clinician‐assessed outcomes and when appropriate patient‐assessed outcomes
	10b	Important follow‐up test results
	10c	Intervention adherence and tolerability (How was this assessed?)
	10d	Adverse and unanticipated events
Discussion	11a	Discussion of the strengths and limitations in the management of this case
	11b	Discussion of the relevant medical literature
	11c	The rationale for conclusions (including assessment of possible causes)
	11d	The main “take‐away” lessons of this case report
Patient Perspective	12	Did the patient share his or her perspective or experience? (Include when appropriate)
Informed Consent	13	Did the patient give informed consent? Please provide if requested
Additional Information	14	Acknowledgement section; Competing Interests; IRB approval when required

^*Source: http://www.care-statement.org/care-checklist.html.

To the best of our knowledge, completeness of case reports has not been widely evaluated and it has only addressed to specific clinical areas. We found few reports focused on assessment of completeness of case reports. Most of them were from Asia regarding topics as acupuncture. ⁹ , ¹⁰ Eldawlatly et al evaluated completeness of case reports in the Saudi Journal of Anesthesia between 2013 and 2017. They concluded that the main topics they needed to make an improvement were patients' perspective and obtained consent. ¹¹ Ravi et al ¹² measured the completeness in case reports from Indian journals, acknowledging that endorsement to CARE guidelines was low, because of the lack of information about them.

Other previous articles have studied the completeness of case reports in surgery and dermatology, ¹³ , ¹⁴ which shows that not only the awareness about the guidelines, but also the analysis of their use and consequent completeness of the articles, are topics that need more diffusion and communication. Overall, completeness of reporting (COR) (according to CARE guidelines) was considered acceptable but presenting a wide variability among studies. Kim et al as Eldawlatly et al found a mean adherence around 75%. ³ , ¹¹ On the other hand, An and Ravi et al found that COR was between low and acceptable (below 50%). ¹⁰ , ¹²

Our primary objective was to assess the COR of case reports published in high‐impact medical journals. In secondary analyses, we evaluated whether reporting was better for journals that explicitly endorsing the CARE statement in their instructions for authors. We hypothesized that completeness of case reports published in high‐impact medical journals is high and above 75%.

2. MATERIALS AND METHODS

We searched MEDLINE to find case reports published from July to December 2017 in three general medical journals: JAMA, The Lancet and The New England Journal of Medicine (NEJM). Those journals were selected based on their relevance and high‐impact factor (Journal Impact Factor—Clarivate Analytics Impact Factor 2017). In addition, we included The BMJ Case Reports as a journal explicitly dedicated to publish case reports only.

Of the included journals, The BMJ Case Reports and JAMA refer explicitly to the CARE statement in their instructions for authors, whereas The Lancet and The NEJM journals do not mention the CARE statement. The Lancet requires in their submission guidelines to follow several EQUATOR guidelines (http://www.equator-network.org) without direct mention of the CARE guideline.

From the ethical perspective, this paper was classified as nonrisk observational research. The abstract, introduction, certain sections of methods and results, discussion and funding proposed by the adapted PRISMA guidelines for reporting of meta‐epidemiological methodology research were fulfilled. ¹⁵

2.1. Data selection

Once journals were selected, two researchers (MV, DO) hand‐searched in each published issue during the mentioned period. All reports of a single case were included. Case reports articles that included more than one patient were excluded of this study. Disagreements during selection were solved by consensus with a third researcher (JAC).

Title and abstract screening were performed including all possibly relevant evaluations for further review. Full texts of all remaining studies were retrieved and appraised for eligibility. Finally, we exported retrieved citations to Mendeley (Elsevier). ¹⁶

2.2. Data extraction

The CARE checklist was used to evaluate all case reports. In order to train research members (MV and DO), a pilot test was performed using 15 case reports not included in this study and individual concerns about specific items were discussed and solved by consensus with a third researcher (JAC) until agreement.

Completeness of included case reports was assessed independently by two blinded (each other's individual assessment) researchers (MV and DO), and data were organized using a pre‐specified Excel spreadsheet (Microsoft Corp). Each of 29 items considered by CARE statement was categorized as “yes” (reported) or “no” (did not report or could not tell something about reporting). Basic study characteristics were journal of publication and whether the journal explicitly endorsing CARE statement.

2.3. Data analysis

Analysis was completed using Excel and R statistical software package. ¹⁷ Descriptive analysis included the number and proportion of manuscripts reporting each of the CARE items.

The “COR score” for each manuscript was calculated as the “yes” answers as a proportion of the “yes + no” answers: COR score (%) = [yes/(yes + no)] × 100. To calculate COR, we used the best scenario assessment of the two blinded evaluators. In addition, we reported the proportion of agreement for each pair of evaluations and the kappa coefficient that was interpreted as minimal agreement (0.21‐0.39), weak (0.40‐0.59), moderate (0.60‐0.79), strong (0.80‐0.90) and almost perfect agreement (above 0.90).

Normality of COR scores was checked using distribution and probability plots. Median COR score among all questions of the CARE statement was calculated, as well as per study section. Two secondary analyses were performed. First, we compared journals that explicitly endorsed the CARE statement (BMJ Case Reports and JAMA) to journals that did not endorse CARE statement (The Lancet and NEJM) by using Mann‐Whitney U test. A normal approximation of the 95% confidence interval of the difference was provided. Second, we compared COR scores among the four journals by using the Kruskal‐Wallis test and multiple post hoc pairwise comparisons with the Dunn test. P values and effect sizes are presented.

3. RESULTS

One hundred and fifty‐four papers were screened. Two papers were classified as case series reports and were excluded. Of these, 114 papers were classified as case reports from July 1 to December 31 of 2017. They were distributed in BMJ Case Reports n = 73 (64%), JAMA n = 17 (15%), The LANCET n = 5 (4%) and NEJM n = 19 (17%).

The median COR among all questions was 81%, IQR [63%‐96%]. Sections with highest median COR were patient information (93%), clinical findings (100%), therapeutic intervention (96%), follow‐up and outcomes (90%), discussion (88%), informed consent (84%) and additional information (96%). On the other hand, the sections with lowest mean completeness were title (21%), keywords (2%), timeline (30%) and patient perspective (17%). At individual questions, completeness was below 60% to: question 2, 3a, 3d, 7, 8d, (Table 2).

Table 2.

Median completeness of reporting (COR) per topic and proportion of individual COR score of each question of the CARE statement in included case reports

Topic	Item	Question	Median COR (%)	Proportion of individual COR (%)	Proportion of agreement (%)	Kappa coefficient
Title	1	The words “case report” should be in the title along with what is of greatest interest in this case	21	21	93	0.757
Keywords	2	The key elements of this case in 2‐5 key words	2	2	98	**
Abstract	3a	Introduction—What is unique about this case? What does it add to the medical literature?	59	55	64	0.231
	3b	The main symptoms of the patient and the important clinical findings		63	69	0.385
	3c	The main diagnoses, therapeutics interventions, and outcomes		66	76	0.523
	3d	Conclusion—What are the main “take‐away” lessons from this case?		54	56	0.000
Introduction	4	Brief background summary of this case referencing the relevant medical literature	71	71	79	0.560
Patient Information	5a	Demographic information (such as age, gender, ethnicity, occupation)	93	76	79	0.534
	5b	Main symptoms of the patient (his or her chief complaints)		100	97	*
	5c	Medical, family, and psychosocial history including co‐morbidities, and relevant genetic information		89	71	0.246
	5d	Relevant past interventions and their outcomes		96	75	0.075
Clinical Findings	6	Describe the relevant physical examination (PE) findings	100	100	84	*
Timeline	7	Depict important milestones related to your diagnoses and interventions (table or figure)	30	30	75	0.104
Diagnostic Assessment	8a	Diagnostic methods (such as PE, laboratory testing, imaging, questionnaires)	74	100	96	*
	8b	Diagnostic challenges (such as financial, language, or cultural)		68	68	0.369
	8c	Diagnostic reasoning including other diagnoses considered		81	62	0.201
	8d	Prognostic characteristics (such as staging in oncology) where applicable		52	64	0.197
Therapeutic Intervention	9a	Types of intervention (such as pharmacologic, surgical, preventive, self‐care)	96	100	90	*
	9b	Administration of intervention (such as dosage, strength, duration)		91	61	0.032
	9c	Changes in intervention (with rationale)		96	70	0.008
Follow‐up and Outcomes	10a	Clinician‐assessed outcomes and when appropriate patient‐assessed outcomes	90	100	84	*
	10b	Important follow‐up test results		96	71	0.022
	10c	Intervention adherence and tolerability (How was this assessed?)		76	54	0.067
	10d	Adverse and unanticipated events		84	57	0.081
Discussion	11a	Discussion of the strengths and limitations in the management of this case	88	71	68	0.365
	11b	Discussion of the relevant medical literature		98	89	0.179
	11c	The rationale for conclusions (including assessment of possible causes)		96	76	0.089
	11d	The main “take‐away” lessons of this case report		80	87	0.664
Patient Perspective	12	Did the patient share his or her perspective or experience? (Include when appropriate)	17	17	83	*
Informed Consent	13	Did the patient give informed consent? Please provide if requested	84	84	89	0.683
Additional Information	14	Acknowledgement section; Competing Interests; IRB approval when required	96	96	89	0.334

^**Kappa coefficient not quantifiable (two cells with zero).

^*Kappa coefficient not quantifiable (one cell with zero).

Kappa coefficient presented poor or fair agreement for 18 items. Ten items showed high proportion of agreement and a kappa coefficient at least moderate (items 1, 2, 5b, 6, 8a, 9a, 10a, 11d, 12 and 13). Mean proportion of agreement among 31 items was 77% (Table 2).

Case reports published in journals with an explicit adherence to CARE Statement in their instructions to authors presented a median COR score higher than journals without explicit endorsing (median = 77.4% [71‐87.1] vs median = 64.5% [60.5‐71]), median difference = −12.8% 95% CI [−16.1% to −6.5%], P < .0001.

The distribution of median COR scores of the CARE statement in included case reports stratified by journal of publication is presented in Figure 1 and detailed COR scores for each item of the CARE Statement in Figure 2.

Figure 1.

Distribution of completeness of reporting (COR) scores of the CARE statement in included case reports stratified by journal of publication. Data presented as n, median [IQR]

Figure 2.

Completeness of reporting (COR) score of the included case reports detailed by items of CARE statement

There were differences among COR scores (P < .0001). Pairwise comparisons showed that the median COR score for BMJ Case Reports was higher than JAMA (median difference = 3.9%, P < .0001), The Lancet (median difference = 2.4%, P < .0001) and NEJM COR scores (median difference = 5.3%, P < .0001). We were unable to demonstrate differences in other pairwise comparisons.

4. DISCUSSION

Although there is still room for improvement in quality of reporting of case reports, our study showed that overall completeness in reporting is high for CARE endorsing journals (median COR above 70%). Sections with highest COR were patient information, clinical findings, therapeutic intervention, follow‐up and outcomes, discussion and informed consent. Sections with lowest COR were title, keywords, timeline and patient perspective. Journals with explicit endorsing of CARE statement in their instructions to authors present a higher COR, which leads to generate recommendations to improve the completeness of case reports studies in the current medical literature.

There is a growing interest in problems affecting the validity and reliability of published healthcare research. ¹⁸ , ¹⁹ Inadequate reporting is a widespread problem and has been frequently observed in publications of in vivo and in vitro reports as well and a broad range of clinical reports. ²⁰ , ²¹ In 2009, Iain Chalmers and Paul Glasziou showed that at least 50% of research reports were not usable because of incomplete reporting. ²⁰ , ²² This waste in research has severe consequences for researchers, clinicians, decision‐makers and patients. Interestingly, we did not find many published reports about completeness of case reports in the literature.

Older evidence‐based medicine paradigms have led us to rank study designs based on the level of evidence that they represent, with case reports being the lowest. However, newer paradigms became less dependent on study design and allowed on occasions stronger inferences from well‐done observational studies. ²³ Systematic synthesis of case reports and case series, when they are the only or best available evidence, is possible ²⁴ and may produce adequate inferences. ²³

Are case reports still useful and should be published? John P. Vandenbroucke and Manfred Hauben arguing that case reports are still valuable in recognition of new diseases, describing new interventions, detection of drug side effects (being the pillar for pharmacovigilance), study the mechanisms of the disease, and audit and recognition of rare manifestations of diseases. ²⁵ In addition, case reports may have a valuable role in medical education (teaching and learning from a narrative). The remaining importance of case reports reveals a need for improvement of the completeness of their reports, being the CARE guidelines the standard for this purpose. ⁸ , ²⁶ , ²⁷

CARE guidelines are aimed to enhance a complete and transparent reporting. Without adequate reporting, it is almost impossible to assess reliability and validity of study findings, and the strengths and weaknesses of the case, and to use its information in practice. A recent initiative addressed both the methodological quality (validity) and adequate reporting by using a tool organized in four domains: selection, ascertainment, causality and reporting. ²⁴ In the last one, their authors asked: is the case(s) described with sufficient details to allow other investigators to replicate the research or to allow practitioners make inferences related to their own practice? While CARE guidelines encourage to publish a complete and meaningful exposition of medical information, this effort advance further into the validity analysis of a case report which is only possible having a report as complete as possible.

Completeness, quality and reporting guidelines' adherence has been widely addressed in clinical trials ²⁸ , ²⁹ , ³⁰ and observational studies describing ranges of completeness from 30 up to 85%. ³¹ , ³² , ³³ , ³⁴ Considering the limited external validity of our results, our estimate of COR for case reports in included journals is high specially for CARE endorsing journals.

We demonstrated that COR is above 70% and this is the first study evaluating major high‐impact general medical journals. There are several differences between the analyses of each item in the checklist, being the day‐to‐day practice items the ones with the highest COR. These include patient information, clinical findings, therapeutic intervention, follow‐up and outcomes, discussion; exposing the main interest in publication of these articles, and also, illustrating that clinicians are still relying on case reports to help orientate challenging cases, mostly due to the lack of patients with such diseases. Additionally, we acknowledge that according to ethics declarations, like Helsinki and Nuremberg, written and informed consent of the patient is one of the items with high COR. Each included journal has clear policies about informed consents for case reports but a few percentages of cases do not explain this in detail in the text. Probably, informed consent has been documented towards the editorial board of the journal but has not been mentioned in the study report.

Items such as inclusion of the words “case report” in the title, keywords, timeline and patient perspective had the lowest COR. First, title and keywords provide an initial overview of the manuscript content and could increase the retrieval by electronic searches. ³⁵ Articles with short titles describing the results (related directly to the clinical case) are cited more often in literature with two classical examples: (a) “A preliminary communication on extensively disseminated Kaposi's sarcoma in a young homosexual man” and (b) “An essay on the shaking palsy”. ³⁶ , ³⁷ , ³⁸ Second, low COR scores to patient perspective may reflect a lack of inclusion of the patient in the reporting process, outside of the consent to be published. As a source of evidence, case reports must include patient perspective as an opportunity to discover patient‐reported outcomes measures useful for further research. ³⁹

Journals with explicit endorsement of CARE statement in their instructions to authors present higher COR. A 2012 systematic review indicated that for some items of the CONSORT checklist, trials published in journals that endorse CONSORT were more completely reported than trials published before the time of endorsement or in nonendorsing journals. ²⁸ , ⁴⁰ , ⁴¹ However, Stevens et al assessed if the COR was related to journals' endorsement of reporting guidelines including other checklist than previous studies. ³⁴ There was insufficient evidence to determine the relation between journals' endorsement and the COR.

This is one of the few studies that evaluate completeness of case reports using CARE guidelines and the first one to consider three high‐impact international journals. Since 1990s, PubMed has shown more than 30 000 hits per year with the free text “case report” and in 2017 there was more than 50 000. Since nothing suggests this diminishes, this is an enormous opportunity of improvement in order to reduce waste in research and improve utility. ⁴²

This study has some limitations. We included three high‐impact journals and one more dedicated only to publish case reports (which accounted for the most of the included studies). This fact reduces the external validity of our results to the overall literature. Also, selecting well‐ranked included journals more likely to have adequate reporting may overestimate the COR scores in comparison with other journals. Finally, our results are not generalizable to other publication languages.

We chose only on 6 months' calendar after the CARE guidelines publication and this type of time‐period selection may be limited. We tried to address this restriction by addressing an up‐to‐date time period but this still could be insufficient; expanding our study to covering a longer period of inclusion, two separate periods, or including some specialty journals (journals with lower bibliometric indicators) may provide valuable information regarding the distribution of case report completeness across medical literature and strength external validity.

Two journals referring explicitly to the CARE statement in their instructions for authors were included. While this may be considered as an endorsing of reporting guidelines, the process of full implementing is more challenging and it is not limited to ask for its use only. ⁴³ We hope that our results generate discussions regarding the wording of endorsement and encourage journals to be clearer in their requests regarding reporting guideline use only.

Our aim was to assess the completeness without considering other important characteristics of the studies that may influence the COR (ie specialty of the clinical case, author's details and professional context). Finally, judgements about completeness clearly have a subjective component. Completeness about certain topic may be influenced by several factors like clinical experience, prior training or practicality and transparency of each CARE question. In the explanation and elaboration of the CARE guidelines, original author's present several examples and each item from the checklist is explained and accompanied by published examples. We took this reference to prior training but our findings showed a high variability in the proportion of agreement and kappa coefficients between two evaluators. This finding has also been described in observational studies during assessment of COR. ⁴⁴ Studying determinants of COR and increasing the agreement among evaluators (as a way to reduce information bias) remain as potential areas of research in terms of completeness evaluation studies.

Reduce waste and improve quality of clinical research and its following publication is still one of the main goals in our current practice. This study aims at providing tools for the future publishing completeness of case reports, leading us to support the recommendations made by Stevens et al. ³⁴ Ongoing and future evaluations of endorsement status of reporting guidelines in medical journals should be assessed to improve completeness and reduce waste of clinical research, including case reports.

CONFLICT OF INTERESTS

The authors have declared none conflict of interests.

AUTHOR CONTRIBUTIONS

All authors have made substantive intellectual contributions to the development of the protocol and this article. JAC conceptualized the study and led the writing of the article. AVH led the supervision of the article preparation. MV and DO performed the data extraction for the study. JAC and MV performed all analyses. DA and DM provided support and guidance from the early protocol stage. JAC, AVH, MV, DO and DM read and approved the final article. DA reviewed the final version of the protocol.

Calvache JA, Vera‐Montoya M, Ordoñez D, Hernandez AV, Altman D, Moher D. Completeness of reporting of case reports in high‐impact medical journals. Eur J Clin Invest. 2020;50:e13215 10.1111/eci.13215