Assessing the compliance of systematic review articles published in leading dermatology journals with the PRISMA statement guidelines: A systematic review

Abstract

Background

Reporting quality of systematic reviews and meta-analyses is of critical importance in dermatology because of their key role in informing health care decisions.

Objective

To assess the compliance of systematic reviews and meta-analyses in leading dermatology journals with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement guidelines.

Methods

This review was carried out in accordance with PRISMA guidelines. Included studies were reviews published across 6 years in the top 4 highest-impact-factor dermatology journals of 2017. Records and full texts were screened independently. Data analysis was conducted with univariate multivariable linear regression. The primary outcome was to assess the compliance of systematic reviews and meta-analyses in leading dermatology journals with the PRISMA statement.

Results

A total of 166 studies were included and mean PRISMA compliance across all articles was 73%. Compliance significantly improved over time (β = .016; P = <.001). The worst reported checklist item was item 5 (reporting on protocol existence), with a compliance of 15% of articles.

Conclusion

PRISMA compliance within leading dermatology journals could be improved; however, it is steadily improving.

Capsule Summary.

• •Reporting quality of systematic reviews in dermatology is of critical importance because they form the reference standard for summarizing evidence.
• •Compliance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines in leading dermatologic journals has improved, but there are still gaps in reporting. Authors should ensure they are fully reporting their reviews.

Introduction

Systematic reviews and meta-analyses are methodologically rigorous studies that form the reference standard for creating evidence in health care.¹ They are an important part of dermatologic research because they guide health care decision within clinical dermatology and also work to minimize bias.² It is critical now more than ever to ensure reviews are sufficiently reported, given the continual increase in systematic reviews within dermatology.³ Reviews that are well reported allow clinicians and policy makers alike to make transparent and informed judgments to guide decisions within dermatology. The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement is a 27-item checklist that ensures reporting transparency of a review.⁴

Previous work has demonstrated that systematic reviews and meta-analyses within dermatology are less likely to evaluate publication bias,⁵ which is one of the 27 items in the PRISMA reporting guidelines. Further work published by the Cochrane Skin Group assessed 38 reviews on selected dermatologic topics and demonstrated a higher methodological quality in their reviews compared with non–Cochrane Skin Group reviews.⁶ A recent systematic review assessing the adherence of 136 dermatology reviews to the PRISMA guidelines highlighted that reporting quality of dermatology articles is often inadequate, but has been improving.⁷ To our knowledge, our review represents the largest study assessing reporting quality of systematic reviews and meta-analyses published within dermatology to date and the largest assessment comparing reporting quality of Cochrane versus non-Cochrane reviews within dermatology.

Methods

Our systematic review was carried out in accordance with PRISMA guidelines; the Cochrane Handbook for Systematic Reviews and Interventions and a search technique similar to that of previous work were used.⁸ The protocol for this review was published a priori and also registered with the international prospective register of systematic reviews, Research Registry.⁹

The top 4 highest-impact-factor dermatology journals of 2017 were identified with the Thomson Reuters InCites Journal Citation Reports. In order of impact factor (from highest to lowest), these were identified as JAMA Dermatology, Journal of the American Academy of Dermatology, Journal of Investigative Dermatology, and the British Journal of Dermatology. MEDLINE PubMed was searched on August 26, 2018, for reviews for the following years: 2017, 2016, 2012, 2011, 2007, and 2006. These years were chosen because 2016/2017 was the most contemporaneous 2-year period at the start of the search, and 5-year periods preceding these years were chosen to allow for comparators.

The search strategy was developed in line with an information search specialist. It included the use of Boolean logical operators such as “OR” to improve sensitivity of the search. The search terms were “systematic review” OR “meta-analysis” OR “meta-analyses” AND “JOURNAL NAME” (Fig 1). The search was performed on August 26, 2018. PubMed's search filters for systematic reviews and meta-analyses were also used to refine the search.

Fig 1.

Example of the electronic search strategy on PubMed MEDLINE to search for articles within the British Journal of Dermatology.

The inclusion and exclusion criteria were applied. Inclusion criteria were systematic reviews and meta-analyses within dermatology, reviews only published within the top 4 highest-impact-factor journals in 2017, reviews published in 2016/2017, 2011/2012, and 2006/2007, and English-language studies. Exclusion criteria were articles that were not systematic reviews and meta analyses, articles outside of the dates and journals previously mentioned, historical reviews, narrative literature reviews, unpublished reviews, and gray literature.

Seven independent researchers (N.D., S.R., M.R.B., M.M., C.I., Y.Z.U., and G.M.) screened the titles and abstracts against inclusion and exclusion criteria. After this, the full text was assessed for inclusion and exclusion criteria. Any discrepancies between articles were resolved by discussion or senior author decision (B.G.) and the reasons for exclusion was noted. Articles that passed both stages of screening were included for data extraction.

Data extraction took place with a standard extraction database created on Google Forms (Alphabet Inc, Mountain View, CA). Duplicates were removed before extraction. A training session took place before data extraction, focusing on accurate marking of included studies against the PRISMA guidelines to ensure consistency. This training session was conducted by senior researchers in our group and it involved “practice” marking of dermatology systematic reviews against the PRISMA checklist. Any discrepancies between researchers were reported to the team and discussed in training among the whole research team.

The following data items were extracted: compliance of each article with the 27-item PRISMA checklist, study authors, date of publication, journal name, dermatology subtopic assessed by the review, country where the review took place as assessed by first author, commercial or non-commercial funding, protocol existence, whether it was a Cochrane review, and whether it was registered. Dermatology subtopics were divided by using the Cochrane Skin Group titles categorized by the British Association of Dermatologists Diagnostic Index.¹⁰ The following 2 subtopics were also added for this review: educational and meta-research. Overall outcome of the review was assessed according to whether the review showed a positive, equivocal, or negative outcome with respect to the intervention being studied.

Articles were scored for compliance with the 27-item PRISMA checklist. A score of 1, or “adequate,” was given for an article that met all the criteria for a particular item. A score of zero was given for those that did not meet or partially met the item requirements. Some items were not applicable for the article in question; in such cases, a maximum possible PRISMA score was calculated per article to avoid articles being penalized for not reporting irrelevant items. To give a richer analysis for PRISMA items deemed to have more than 1 component, in addition to a score of zero, a score of either inadequate or not described was given. If an item did not meet all components within the PRISMA item, it was deemed not described. If an item partially met the requirements of all components of the item, then it was deemed inadequate. A compliance score expressed as a percentage was calculated for each article.

The data were analyzed with Microsoft Excel (Microsoft Corporation, Albuquerque, NM). Continuous variables are presented as a mean and range. Categoric variables are presented as percentage values. Summary measures were used to calculate the mean number of PRISMA items adequately reported for all studies and the percentage of studies compliant with each element of the PRISMA guidelines. Univariate multivariable linear regression was used to assess the associations between PRISMA compliance against the publication year, Cochrane versus non-Cochrane studies, studies with protocols versus those without them, and registered studies versus those that were not registered. Associations are reported as β coefficients with 95% confidence intervals.

The primary outcome was to assess the compliance of systematic reviews and meta-analyses in leading dermatology journals with the PRISMA statement. Secondary outcomes were conducted to assess whether reporting quality improved over time, there was a difference in compliance between Cochrane and non-Cochrane reviews, certain items were consistently poorly reported, and studies were registered and whether they had a protocol.

The PRISMA score across all articles was compared according to the following factors: year published, subtopics, presence of a protocol, and a priori registration status.

The examination of bias was limited to the assessment of whether each systematic review had remained compliant with PRISMA items 12 (assessment of risk of bias in individual studies), 15 (assessment of risk of bias across studies), 19 (present data on the risk of bias within studies), and 22 (present data on the risk of bias across studies).

Results

A total of 575 records were identified in the initial literature search. After the removal of duplicates and full-text screening, a total of 166 systematic reviews and meta-analyses were included for data extraction. Fig 2 shows record selection.

Fig 2.

PRISMA flow diagram showing article selection process.

The study characteristics are shown in Table I, including journal, year of publication, theme of article, country of first author, registration status, whether it was Cochrane, protocol existence, funding type, and study outcome. The journal with the most studies was the British Journal of Dermatology, with 49% of the total articles, followed by Journal of the American Academy of Dermatology (39%), Journal of the American Medical Association Dermatology (9%), and Journal of Investigative Dermatology (3%). The number of articles increased each year, with 2017 accounting for 40% of all articles. “Psoriasis and other keratinizing disorders” was the most-studied subtopic, at 15% of all articles. The United States contributed 30% of all studies, with the United Kingdom contributing 17% of articles. Only 19% of studies were registered, 24% had a protocol, and 8% were Cochrane articles. The majority of studies (72%) were not funded. The majority of funded studies received non-commercial funding. In regard to outcome of the study, this was not applicable in most cases (48%) and there were positive outcomes in 32%.

Table I.

Study characteristics

	No of studies	% of studies
Journal
British Journal of Dermatology	82	49
Journal of the American Academy of Dermatology	64	39
JAMA Dermatology	15	9
Journal of Investigative Dermatology	5	3
Date of publication
2017	67	40
2016	47	28
2012	20	12
2011	16	10
2007	8	5
2006	8	5
Theme
Psoriasis and other keratinizing disorders	25	15
Dermatitis and eczema	22	13
Tumors and cysts of the skin and appendages	18	11
Disorders of skin appendages (hair, nails, sweat glands)	13	8
Skin disease resulting from drugs or treatment	13	8
Connective tissue and immunobullous and related	12	7
Infectious diseases affecting the skin	9	5
Other	9	5
Disorders of skin color	7	4
Benign and malignant infiltrations of the skin	7	4
Meta-research (e.g., reporting quality)	6	4
Urticaria and other inflammatory skin disorders	5	3
Skin conditions caused by environmental or physical injury	4	2
Psychologic, psychiatric, and related disorders of the skin	4	2
Disorders involving the skin's blood and lymphatic vessels	3	2
Disorders of the dermis and subcutaneous tissue	3	2
Education	3	2
Papulosquamous disorders including lichen planus	2	1
Genetic and chromosomal disorders affecting the skin	1	1
Metabolic and nutritional disorders affecting the skin	0	N/A
Birthmarks and developmental abnormalities of the skin	0	N/A
Country
US	50	30
UK	28	17
Netherlands	17	10
Germany	12	7
Australia	9	5
France	7	4
Denmark	5	3
Taiwan	5	3
Spain	4	2
Israel	4	2
Canada	4	2
China	3	2
South Korea	3	2
Italy	3	2
Belgium	2	1
Brazil	2	1
Norway	1	1
Iran	1	1
South Africa	1	1
Peru	1	1
Hungary	1	1
Greece	1	1
Ireland	1	1
Japan	1	1
Registration
Not registered	135	81
Registered	31	19
Cochrane
Not Cochrane	153	92
Cochrane	13	8
Protocol
No protocol	126	76
Protocol	40	24
Funding
None	120	72
Non-commercial	40	24
Commercial	6	4
Outcome
N/A	80	48
Positive	53	32
Equivocal	24	14
Negative	9	5

N/A, Not applicable; UK, United Kingdom; US, United States.

Each item in the PRISMA checklist from item 1 to 27 was assessed for overall compliance across all studies (Table II). These were marked as adequate, inadequate, not described, and not applicable, as described earlier. A number of items were not applicable for the study; for example, item 23 (presenting the results of any additional analyses conducted) was not applicable in 67% of studies. The items of highest compliance, at 100%, were item 3 (describing the rationale for the review) and item 26 (providing a conclusion). Items with the poorest compliance were item 5 (reporting protocol and registration), at 15% reporting, and item 12 (reporting the risk of bias in individual studies), at 23% reporting.

Table II.

Compliance of each item of the Preferred Reporting Items for Systematic Reviews and Meta-analyses checklist

PRISMA item	Adequate	Adequate, %	Inadequate	Inadequate, %	Not described	Not described, %	N/A	N/A, %	Total possible minus N/A	Studies adequately reporting each item when applicable, %
3	166	100	0	0	0	0	0	0	166	100
26	166	100	0	0	0	0	0	0	166	100
14	69	42	0	0	3	2	94	57	72	96
6	154	93	0	0	12	7	0	0	166	93
9	153	92	0	0	13	8	0	0	166	92
1	150	90	0	0	16	10	0	0	166	90
25	150	90	0	0	16	10	0	0	166	90
11	149	90	0	0	17	10	0	0	166	90
20	148	89	9	5	9	5	0	0	166	89
27	147	89	0	0	18	11	1	1	165	89
23	48	29	0	0	6	4	112	67	54	89
2	147	89	0	0	19	11	0	0	166	89
7	147	89	0	0	19	11	0	0	166	89
21	64	39	0	0	9	5	93	56	73	88
17	140	84	0	0	26	16	0	0	166	84
13	114	69	0	0	29	17	23	14	143	80
4	130	78	36	22	0	0	0	0	166	78
18	129	78	0	0	37	22	0	0	166	78
10	126	76	0	0	40	24	0	0	166	76
8	111	67	0	0	55	33	0	0	166	67
15	45	27	0	0	27	16	94	57	72	63
16	44	27	0	0	28	17	94	57	72	61
24	88	53	78	47	0	0	0	0	166	53
19	54	33	0	0	112	67	0	0	166	33
22	44	27	58	35	63	38	1	1	165	27
12	39	23	28	17	99	60	0	0	166	23
5	25	15	21	13	120	72	0	0	166	15

PRISMA items 3 and 26 had 100% compliance, whereas PRISMA items 5 and 12 had the least compliance.

PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analyses.

The compliance of all 166 studies against the PRISMA statement was assessed (Table III). The number of items that were adequately reported of the theoretic maximum number of relevant, applicable items gave a percentage PRISMA score for each systematic review. The mean compliance of all articles was 73%. The maximum compliance was 100% and minimum compliance was 23%, demonstrating a 77% range in compliance between studies. The average compliance of the journals to PRISMA was also calculated, with Journal of Investigative Dermatology at 85%, British Journal of Dermatology at 76%, JAMA Dermatology at 76%, and Journal of the American Academy of Dermatology at 68%.

Table III.

Compliance of each article with the Preferred Reporting Items for Systematic Reviews and Meta-analyses guideline

Author	Adequate	Adequate, %	Inadequate	Inadequate, %	Not described	Not described, %	N/A	N/A, %	Total no. applicable	Total no. applicable, %	Adequately reported, %
Adil A11	18	67	2	7	6	22	1	4	26	96	69
Adler BL12	16	59	1	4	5	19	5	19	22	81	73
Agbai O13	13	48	2	7	7	26	5	19	22	81	59
Ali FM14	18	67	1	4	3	11	5	19	22	81	82
Atwan A15	23	85	2	7	0	0	2	7	25	93	92
Atzmony L16	25	93	1	4	1	4	0	0	27	100	93
Bae JM17	24	89	0	0	2	7	1	4	26	96	92
Bae JM18	21	78	2	7	4	15	0	0	27	100	78
Baillie L19	16	59	2	7	4	15	5	19	22	81	73
Balak DM20	18	67	2	7	2	7	5	19	22	81	82
Barbarot S21	16	59	2	7	3	11	6	22	21	78	76
Bath-Hextall F22	19	70	1	4	2	7	5	19	22	81	86
Bauer A23	21	78	1	4	4	15	1	4	26	96	81
Bertolotti A24	19	70	2	7	5	19	1	4	26	96	73
Bjerre RD25	18	67	1	4	2	7	6	22	21	78	86
Bobotsis R26	18	67	2	7	2	7	5	19	22	81	82
Brewer JD27	23	85	1	4	3	11	0	0	27	100	85
Broeder JA28	23	85	2	7	1	4	1	4	26	96	88
Brown G29	15	56	2	7	5	19	5	19	22	81	68
Brunssen A30	20	74	0	0	1	4	6	22	21	78	95
Burden-Teh E31	14	52	2	7	6	22	5	19	22	81	64
Callander J32	14	52	2	7	6	22	5	19	22	81	64
Callen J33	10	37	1	4	11	41	5	19	22	81	45
Chahoud J34	25	93	0	0	2	7	0	0	27	100	93
Chang YS35	22	81	1	4	4	15	0	0	27	100	81
Charrow A36	14	52	2	7	6	22	5	19	22	81	64
Charrow A37	14	52	1	4	8	30	4	15	23	85	61
Chasset F38	25	93	2	7	0	0	0	0	27	100	93
Chen X39	21	78	4	15	2	7	0	0	27	100	78
Chi CC40	17	63	4	15	4	15	2	7	25	93	68
Connolly KL41	5	19	1	4	16	59	5	19	22	81	23
Courtenay M42	9	33	2	7	10	37	6	22	21	78	43
Crijns HJ43	12	44	1	4	9	33	5	19	22	81	55
Dai J44	18	67	2	7	6	22	1	4	26	96	69
Delaplace M45	13	48	2	7	7	26	5	19	22	81	59
Devillers AC46	7	26	1	4	14	52	5	19	22	81	32
Dommasch ED47	23	85	1	4	3	11	0	0	27	100	85
Durbec F48	15	56	1	4	6	22	5	19	22	81	68
Eady, EA49	17	63	0	0	5	19	5	19	22	81	77
Eleftheriadou V50	10	37	2	7	9	33	6	22	21	78	48
Eminovic N51	11	41	2	7	9	33	5	19	22	81	50
Ernst E52	17	63	2	7	3	11	5	19	22	81	77
Falagas ME53	7	26	3	11	14	52	3	11	24	89	29
Finnane A54	18	67	1	4	3	11	5	19	22	81	82
Frew JW55	16	59	2	7	3	11	6	22	21	78	76
Futamura M56	14	52	1	4	6	22	6	22	21	78	67
Gantz M57	13	48	1	4	7	26	6	22	21	78	62
Garden BC58	18	67	2	7	6	22	1	4	26	96	69
Gerami P59	17	63	1	4	5	19	4	15	23	85	74
Gerami P60	13	48	1	4	7	26	6	22	21	78	62
Gerbens LA61	17	63	1	4	4	15	5	19	22	81	77
Glick ZR62	19	70	1	4	7	26	0	0	27	100	70
Gomez-Garcia F63	24	89	1	4	1	4	1	4	26	96	92
Gomez-Garcia F64	24	89	0	0	3	11	0	0	27	100	89
Gonzalez-Lopez G65	24	89	1	4	2	7	0	0	27	100	89
Guillen-Aguinaga S66	26	96	1	4	0	0	0	0	27	100	96
Gunaratne DA67	14	52	2	7	6	22	5	19	22	81	64
Gupta AK68	12	44	2	7	8	30	5	19	22	81	55
Gupta AS69	13	48	2	7	6	22	6	22	21	78	62
Haddad C70	15	56	3	11	4	15	5	19	22	81	68
Hadley G71	18	67	1	4	7	26	1	4	26	96	69
Haedersdal M72	14	52	2	7	6	22	5	19	22	81	64
Hague A73	15	56	4	15	3	11	5	19	22	81	68
Halling-Overgaard AS74	15	56	3	11	4	15	5	19	22	81	68
Hamann CR75	22	81	1	4	4	15	0	0	27	100	81
Harfmann KL76	13	48	1	4	8	30	5	19	22	81	59
Heinl D77	19	70	0	0	3	11	5	19	22	81	86
Hill MK78	15	56	2	7	4	15	6	22	21	78	71
Hoorens I79	7	26	2	7	12	44	6	22	21	78	33
Huang YC80	23	85	1	4	3	11	0	0	27	100	85
Huang YC81	20	74	3	11	4	15	0	0	27	100	74
Ingram JR82	16	59	1	4	5	19	5	19	22	81	73
Ingram JR83	22	81	2	7	2	7	1	4	26	96	85
Jabbar-Lopez Zarif K84	27	100	0	0	0	0	0	0	27	100	100
Jacobsen Audrey A85	18	67	1	4	4	15	4	15	23	85	78
Jang YH86	9	33	3	11	10	37	5	19	22	81	41
Jascholt I87	12	44	2	7	8	30	5	19	22	81	55
Kantor R88	24	89	0	0	3	11	0	0	27	100	89
Karia Pritesh S89	17	63	0	0	5	19	5	19	22	81	77
Katugampola RP90	10	37	2	7	10	37	5	19	22	81	45
Khatami A91	18	67	2	7	6	22	1	4	26	96	69
Kim A92	18	67	2	7	6	22	1	4	26	96	69
Kim JP93	17	63	4	15	6	22	0	0	27	100	63
Kouwenhoven TA94	16	59	2	7	3	11	6	22	21	78	76
Kramer ON95	12	44	2	7	7	26	6	22	21	78	57
Krengel S96	14	52	2	7	6	22	5	19	22	81	64
Kwok CS97	20	74	1	4	5	19	1	4	26	96	77
Lai YC98	25	93	0	0	2	7	0	0	27	100	93
Langan SM99	13	48	3	11	6	22	5	19	22	81	59
Lavda AC100	22	81	0	0	5	19	0	0	27	100	81
Lebrun-Vignes B101	22	81	0	0	5	19	0	0	27	100	81
Lee YH102	18	67	2	7	7	26	0	0	27	100	67
Li AW103	14	52	3	11	5	19	5	19	22	81	64
Liu LY104	15	56	1	4	6	22	5	19	22	81	68
Lodi G105	23	85	0	0	2	7	2	7	25	93	92
Lomas A106	17	63	0	0	5	19	5	19	22	81	77
Lowe GC107	13	48	1	4	7	26	6	22	21	78	62
Lubeek SF108	18	67	2	7	2	7	5	19	22	81	82
Lubeek SF109	16	59	2	7	4	15	5	19	22	81	73
Martin LK110	19	70	2	7	6	22	0	0	27	100	70
Mazaud C111	26	96	1	4	0	0	0	0	27	100	96
Muranushi C112	24	89	1	4	2	7	0	0	27	100	89
Nankervis H113	20	74	2	7	0	0	5	19	22	81	91
Ng CY114	24	89	2	7	1	4	0	0	27	100	89
Nguyen J115	12	44	0	0	10	37	5	19	22	81	55
Nilsen LT116	18	67	1	4	3	11	5	19	22	81	82
Ogunsanya ME117	16	59	1	4	4	15	6	22	21	78	76
Olsen JR118	20	74	2	7	4	15	1	4	26	96	77
Opel D119	18	67	1	4	2	7	6	22	21	78	86
Pampena R120	24	89	1	4	2	7	0	0	27	100	89
Patterson AT121	6	22	3	11	13	48	5	19	22	81	27
Petrelli F122	26	96	0	0	1	4	0	0	27	100	96
Pickett K123	20	74	1	4	1	4	5	19	22	81	91
Plasmeijer EI124	17	63	2	7	3	11	5	19	22	81	77
Pope V125	13	48	1	4	7	26	6	22	21	78	62
Quirke M126	23	85	2	7	0	0	2	7	25	93	92
Reich K127	20	74	1	4	6	22	0	0	27	100	74
Rencz F128	20	74	0	0	6	22	1	4	26	96	77
Ridd MJ129	20	74	1	4	1	4	5	19	22	81	91
Riemer CA130	18	67	1	4	3	11	5	19	22	81	82
Robinson A131	13	48	3	11	7	26	4	15	23	85	57
Rodriguez-Zuniga MJM132	27	100	0	0	0	0	0	0	27	100	100
Roozeboom MH133	25	93	0	0	2	7	0	0	27	100	93
Rotta I134	23	85	1	4	3	11	0	0	27	100	85
Rungapiromnan W135	25	93	1	4	1	4	0	0	27	100	93
Saleem MD136	18	67	2	7	2	7	5	19	22	81	82
Schiller M137	13	48	1	4	9	33	4	15	23	85	57
Schlager JG138	25	93	1	4	0	0	1	4	26	96	96
Schmitt J139	24	89	1	4	2	7	0	0	27	100	89
Schoch D140	14	52	1	4	7	26	5	19	22	81	64
Seidler EM141	7	26	3	11	15	56	2	7	25	93	28
Shahwan KT142	11	41	1	4	12	44	3	11	24	89	46
Shapiro S143	16	59	4	15	6	22	1	4	26	96	62
Shaw J144	15	56	1	4	8	30	3	11	24	89	63
Shreberk-Hassidim R145	12	44	1	4	9	33	5	19	22	81	55
Shreberk-Hassidim R146	8	30	2	7	12	44	5	19	22	81	36
Simonart T147	15	56	0	0	8	30	4	15	23	85	65
Simonsen AB148	18	67	0	0	8	30	1	4	26	96	69
Singh S149	25	93	0	0	2	7	0	0	27	100	93
Snast I150	18	67	1	4	3	11	5	19	22	81	82
Snoswell C151	17	63	2	7	3	11	5	19	22	81	77
Stranzenbach R152	10	37	2	7	11	41	4	15	23	85	43
Tang H153	25	93	0	0	2	7	0	0	27	100	93
Thandar Y154	17	63	3	11	3	11	4	15	23	85	74
Thompson AK155	22	81	1	4	3	11	1	4	26	96	85
Totté JEE156	25	93	0	0	2	7	0	0	27	100	93
van Zuuren EJ157	17	63	3	11	6	22	1	4	26	96	65
van Zuuren EJ158	26	96	1	4	0	0	0	0	27	100	96
van Zuuren EJ159	20	74	1	4	2	7	4	15	23	85	87
van Zuuren EJ160	20	74	1	4	3	11	3	11	24	89	83
van Zuuren EJ161	21	78	1	4	4	15	1	4	26	96	81
Vrijman C162	20	74	1	4	1	4	5	19	22	81	91
Vrijman C163	18	67	0	0	5	19	4	15	23	85	78
Wan MT164	8	30	4	15	9	33	6	22	21	78	38
Warshaw EM165	19	70	1	4	3	11	4	15	23	85	83
Whitton M166	23	85	1	4	3	11	0	0	27	100	85
Williams K167	11	41	1	4	9	33	6	22	21	78	52
Wu SZ168	17	63	0	0	5	19	5	19	22	81	77
Xu H169	20	74	1	4	6	22	0	0	27	100	74
Xu T170	23	85	0	0	4	15	0	0	27	100	85
Yamauchi PS171	15	56	1	4	6	22	5	19	22	81	68
Yen H172	25	93	0	0	2	7	0	0	27	100	93
Yiu ZZ173	27	100	0	0	0	0	0	0	27	100	100
Zidorio APC174	17	63	2	7	3	11	5	19	22	81	77
Zimmermann S175	26	96	1	4	0	0	0	0	27	100	96
Zwischenberger BA176	14	52	1	4	7	26	5	19	22	81	64

The average PRISMA compliance tended to increase with each year across all articles (Table IV). The lowest compliance was the earliest year assessed, which was 2006, with an average compliance of 53%. Multivariate regression analysis demonstrated that PRISMA compliance significantly improved over time (β = .016; P < .001).

Table IV.

Compliance across different subgroups

Compliance	%
Vs year
2017	77
2016	75
2012	75
2011	69
2007	60
2006	53
Vs Cochrane
Cochrane	83
Not Cochrane	73
Vs protocol
Protocol	86
No protocol	69
Vs registration
Registered	86
Not registered	71

Cochrane studies had an average compliance of 83% versus non-Cochrane studies of 73% (Table IV). Analysis showed Cochrane reviews were significantly more compliant than non-Cochrane studies (β = .10; P = .03). Studies with protocols had a much higher average compliance than those without them (86% versus 69%, respectively), with the difference being significant on comparison (β = .17; P < .001). Furthermore, there was a significant difference in PRISMA compliance between registered studies (86%) versus nonregistered ones (71%), with β = .15 (P < .001).

Studies were then grouped according to subtopic, year of publication, Cochrane versus non-Cochrane, protocol existence, and registration. In regard to the dermatology subtopic, “psoriasis and other keratinizing disorders” demonstrated the most compliance with PRISMA, at 81% (Table V).

Table V.

Average compliance of each subtopic with the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines

Subtopic	Compliance, %
Psoriasis and other keratinizing disorders	81
Papulosquamous disorders including lichen planus	80
Skin conditions caused by environmental or physical injury	78
Disorders of skin color	78
Benign and malignant infiltrations of the skin	78
Infectious diseases affecting the skin	77
Psychologic, psychiatric, and related disorders of the skin	77
Education	76
Disorders involving the skin's blood and lymphatic vessels	75
Meta-research (eg, reporting quality)	75
Dermatitis and eczema	73
Urticaria and other inflammatory skin disorders	73
Skin disease resulting from drugs or treatment	71
Connective tissue, immunobullous and related	70
Disorders of the dermis and subcutaneous tissue	70
Tumors and cysts of the skin and appendages	69
Disorders of skin appendages (hair, nails, sweat glands)	66
Genetic and chromosomal disorders affecting the skin	62
Other	66

Discussion

This review assessed 166 systematic reviews and meta-analyses published in 4 leading dermatology journals against the PRISMA statement. To our knowledge, this is the largest review assessing PRISMA compliance of articles published in leading dermatology journals to date. We found that although compliance with PRISMA guidelines improved over time, there remain large gaps in reporting. In 2017, the mean compliance with the PRISMA statement was 73%. We found a wide range in the levels of compliance with the PRISMA statement, ranging from 23% to 100% between individual studies, and we compared compliance with the PRISMA statement in relation to several variables.

The PRISMA compliance of systematic reviews and meta-analyses published in dermatologic journals assessed here compares favorably to that reported in other studies. The quality of reporting of systematic reviews and meta-analyses in other fields has also indicated room for improvement. An assessment of systematic reviews published between 2012 and 2013 in leading otolaryngology journals found that a median 54% of items of the PRISMA statement were adequately reported.¹⁷⁷ Another systematic review assessing the compliance with the PRISMA statement relating to articles published between 2010 to 2015 in 5 major ophthalmology journals found the median compliance per article to be 56%.¹⁷⁸ Furthermore, in 2016, Pidgeon et al⁸ reviewed systematic reviews published in 3 major craniofacial surgery journals within a 5-year period ending in 2015 and found a mean PRISMA score of 72.5%. As such, subtotal PRISMA compliance is not limited to dermatology and is observed across several medical and surgical specialties.

Within the field of dermatology itself, Croitoru et al⁷ reviewed systematic reviews published between 2013 and 2017 in 5 major dermatology journals, including the same 4 journals investigated here. As we did, the authors reported room for greater compliance with the PRISMA statement, but noted an increase in the proportion of PRISMA items that are fully reported yearly. The authors found that in 2017, 65% of evaluated PRISMA items were fully reported compared with 53% in 2013.

Similar items appear to be underreported across dermatology, as well as other fields. In our study, the most underreported items were item 5 (protocol and registration), item 12 (risk of bias in individual studies), and item 22 (risk of bias across studies). The same items were also similarly poorly reported by other studies; for example, across otolaryngology journals, craniofacial journals, and other larger-scale studies of reporting quality.^8,177,179 The reporting of protocols, publishing protocols, and registering studies is an important step in robust medical research.¹⁸⁰ They not only allow independent peer review of the study at hand, which allows improvements in study methodology, but also ensure complete transparency in differences between the protocol and the final study.¹⁸⁰ Moreover, protocol registration may help prevent study duplication, in which researchers are able to access databases to ascertain whether there are similar studies in progress.¹⁸¹ Assessments of risk of bias within individual studies and across studies such as missing studies (known as publication bias) and missing data in included studies (known as selective reporting bias) are also important because if results are missed, it can lead to in overestimation or underestimation of treatment effects.¹⁸²

There are a number of limitations to this study. First, our analysis is limited to 4 major dermatology journals. Although these journals represent a large proportion of dermatology-related research output, critical systematic reviews and meta-analyses published in other dermatologic journals were not included. Our analysis is also limited by the subjectivity of assessing compliance with individual PRISMA items. We attempted to minimize this bias by training data collectors, using independent duplicate scoring, and using independent review of differences by another author, as detailed in our protocol. Our assessment of registration and protocol entirely depended on whether this detail was included by authors within the articles; we did not conduct searches of registries for this information. Finally, we acknowledge that in some instances failure to report compliance with individual PRISMA items within the article does not necessarily mean noncompliance; for example, word count limits may result in curtailing of detail, which in turn may underreport actual PRISMA compliance. Moreover, it is possible that certain items were not reported simply because the assessment was not carried out; for example, risk of bias was not assessed and hence not reported or protocols were not carried out and therefore not reported.

Conclusions

The reporting of systematic reviews and meta-analyses in the top 5 dermatologic journals is currently suboptimal, with scope for improvement in compliance with the PRISMA statement, most notably in relation to protocol and registration, as well as the assessment of bias. We demonstrate here that overall compliance with the PRISMA statement has improved, but more vigilance is required to improve compliance further. Researchers should attempt to familiarize themselves with the PRISMA statement both before conducting research and at article creation. Journals may encourage reporting in accordance with PRISMA guidelines through a variety of means (eg, a mandatory completed PRISMA checklist that is provided at submission, the provision of software to “screen for compliance”). Future work may also include investigation into the reasons behind incomplete reporting, aiming to identify barriers that authors may face in producing articles that are maximally compliant, and providing assistance to overcome such barriers.