Diagnostic tests for oral cancer and potentially malignant disorders in patients presenting with clinically evident lesions

Tanya Walsh; Richard Macey; Alexander R Kerr; Mark W Lingen; Graham R Ogden; Saman Warnakulasuriya

doi:10.1002/14651858.CD010276.pub3

. 2021 Jul 20;2021(7):CD010276. doi: 10.1002/14651858.CD010276.pub3

Diagnostic tests for oral cancer and potentially malignant disorders in patients presenting with clinically evident lesions

Tanya Walsh ^1,^✉, Richard Macey ¹, Alexander R Kerr ², Mark W Lingen ³, Graham R Ogden ⁴, Saman Warnakulasuriya ⁵

Editor: Cochrane Oral Health Group

PMCID: PMC8407012 PMID: 34282854

Abstract

Background

Squamous cell carcinoma is the most common form of malignancy of the oral cavity, and is often proceeded by oral potentially malignant disorders (OPMD). Early detection of oral cavity squamous cell carcinoma (oral cancer) can improve survival rates. The current diagnostic standard of surgical biopsy with histology is painful for patients and involves a delay in order to process the tissue and render a histological diagnosis; other diagnostic tests are available that are less invasive and some are able to provide immediate results. This is an update of a Cochrane Review first published in 2015.

Objectives

Primary objective: to estimate the diagnostic accuracy of index tests for the detection of oral cancer and OPMD, in people presenting with clinically evident suspicious and innocuous lesions.

Secondary objective: to estimate the relative accuracy of the different index tests.

Search methods

Cochrane Oral Health's Information Specialist searched the following databases: MEDLINE Ovid (1946 to 20 October 2020), and Embase Ovid (1980 to 20 October 2020). The US National Institutes of Health Ongoing Trials Register (ClinicalTrials.gov) and the World Health Organization International Clinical Trials Registry Platform were also searched for ongoing trials to 20 October 2020. No restrictions were placed on the language or date of publication when searching the electronic databases. We conducted citation searches, and screened reference lists of included studies for additional references.

Selection criteria

We selected studies that reported the diagnostic test accuracy of the following index tests when used as an adjunct to conventional oral examination in detecting OPMD or oral cavity squamous cell carcinoma: vital staining (a dye to stain oral mucosa tissues), oral cytology, light‐based detection and oral spectroscopy, blood or saliva analysis (which test for the presence of biomarkers in blood or saliva).

Data collection and analysis

Two review authors independently screened titles and abstracts for relevance. Eligibility, data extraction and quality assessment were carried out by at least two authors, independently and in duplicate. Studies were assessed for methodological quality using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS‐2). Meta‐analysis was used to combine the results of studies for each index test using the bivariate approach to estimate the expected values of sensitivity and specificity.

Main results

This update included 63 studies (79 datasets) published between 1980 and 2020 evaluating 7942 lesions for the quantitative meta‐analysis. These studies evaluated the diagnostic accuracy of conventional oral examination with: vital staining (22 datasets), oral cytology (24 datasets), light‐based detection or oral spectroscopy (24 datasets). Nine datasets assessed two combined index tests. There were no eligible diagnostic accuracy studies evaluating blood or salivary sample analysis. Two studies were classed as being at low risk of bias across all domains, and 33 studies were at low concern for applicability across the three domains, where patient selection, the index test, and the reference standard used were generalisable across the population attending secondary care.

The summary estimates obtained from the meta‐analysis were: ‐ vital staining: sensitivity 0.86 (95% confidence interval (CI) 0.79 to 0.90) specificity 0.68 (95% CI 0.58 to 0.77), 20 studies, sensitivity low‐certainty evidence, specificity very low‐certainty evidence; ‐ oral cytology: sensitivity 0.90 (95% CI 0.82 to 0.94) specificity 0.94 (95% CI 0.88 to 0.97), 20 studies, sensitivity moderate‐certainty evidence, specificity moderate‐certainty evidence; ‐ light‐based: sensitivity 0.87 (95% CI 0.78 to 0.93) specificity 0.50 (95% CI 0.32 to 0.68), 23 studies, sensitivity low‐certainty evidence, specificity very low‐certainty evidence; and ‐ combined tests: sensitivity 0.78 (95% CI 0.45 to 0.94) specificity 0.71 (95% CI 0.53 to 0.84), 9 studies, sensitivity very low‐certainty evidence, specificity very low‐certainty evidence.

Authors' conclusions

At present none of the adjunctive tests can be recommended as a replacement for the currently used standard of a surgical biopsy and histological assessment. Given the relatively high values of the summary estimates of sensitivity and specificity for oral cytology, this would appear to offer the most potential. Combined adjunctive tests involving cytology warrant further investigation. Potentially eligible studies of blood and salivary biomarkers were excluded from the review as they were of a case‐control design and therefore ineligible. In the absence of substantial improvement in the tests evaluated in this updated review, further research into biomarkers may be warranted.

Keywords: Humans; Bias; Biomarkers, Tumor; Biomarkers, Tumor/analysis; Biomarkers, Tumor/blood; Carcinoma, Squamous Cell; Carcinoma, Squamous Cell/diagnosis; Carcinoma, Squamous Cell/pathology; Coloring Agents; Early Detection of Cancer; Light; Lip Neoplasms; Lip Neoplasms/diagnosis; Lip Neoplasms/pathology; Mouth; Mouth/pathology; Mouth Neoplasms; Mouth Neoplasms/diagnosis; Mouth Neoplasms/pathology; Saliva; Saliva/chemistry; Sensitivity and Specificity

Plain language summary

What are the most accurate tests for finding cancer of the mouth (oral cancer) and conditions that may lead to oral cancer?

Why is it important to improve the detection of oral cancer?

Persistent abnormal patches or sores in the mouth may represent mouth cancer or oral potentially malignant disorders (OPMDs). OPMDs can sometimes turn into mouth cancer, but if identified early enough patient outcomes can be improved.

What is the aim of this review?

Diagnosing mouth cancer involves the surgical removal of a piece of affected tissue (biopsy) that is then sent to a laboratory for examination of the cells using a microscope. This is painful for patients and involves a delay in finding out the results. The aim of this Cochrane Review review was to find out the accuracy of less invasive diagnostic tests that may provide more timely results. Researchers in Cochrane included 63 studies to answer this question.

What was studied in the review?

Three tests used in addition to a visual examination were evaluated.

‐ Vital stain: a liquid that can be used as a mouthrinse or applied directly to a suspected abnormal area of the mouth. It is thought that any area that is coloured after applying this liquid has a high chance of being mouth cancer or an OPMD. ‐ Oral cytology: a brush is used to remove cells from the suspected abnormal area that are sent to a laboratory for microscopic examination. ‐ Light‐based detection: a special light shone in the mouth that is believed to make cancerous areas appear different to healthy areas.

A small number of studies evaluated a combination of these tests. No studies evaluated the accuracy of tests of blood or saliva.

What are the main results of the review?

The review included 63 studies involving 7942 abnormal patches or mouth sores. Each study participant underwent one or more diagnostic tests as well as a surgical biopsy.

The proportion of people in the included studies with mouth cancer or OPMDs identified through surgical biopsy ranged widely from 4% to 97%. Based on adults attending general dental practices in the UK, in a sample of 1000 lesions 25 would be mouth cancer or an OPMD, and 975 lesions would not be mouth cancer or an OPMD.

‐ Of the 1000 lesions that are tested with vital staining: 22 will be correctly identified as having mouth cancer or an OPMD (true positives), but three lesions that truly are mouth cancer or an OPMD will remain undetected; their 'negative' test results will be incorrect (false negatives). 663 lesions will be correctly identified as not having mouth cancer or an OPMD (true negatives), but 312 people will be incorrectly identified; their 'positive' test results will suggest they have mouth cancer or an OPMD (false positives).

‐ For oral cytology: 23 lesions will be correctly identified as having mouth cancer or an OPMD (true positives), but two lesions that truly are mouth cancer or an OPMD will remain undetected (false negatives). 917 lesions will be correctly identified as not having mouth cancer or an OPMD (true negatives), but 58 lesions will be incorrectly identified (false positives).

‐ For light‐based tests: 22 lesions will be correctly identified as having mouth cancer or an OPMD (true positives), but three lesions that truly are mouth cancer or an OPMD will remain undetected (false negatives). 488 lesions will be correctly identified as not having mouth cancer or an OPMD (true negatives), but 487 lesions will be incorrectly identified (false positives).

There was considerable variability in the accuracy of the vital staining and light‐based tests which meant that the results of a future study could take a broad range of values.

How reliable are the results of the studies in this review?

There were shortcomings in many of the studies that put them at high risk of bias. We rated the certainty of the evidence as moderate for oral cytology and low or very low for the remaining tests.

Who do the results of this review apply to?

Studies included in the review were carried out in many different countries but no studies originated in Africa. Most studies were completed in hospitals. Studies were published between 1980 and 2020.

What are the implications of this review?

Although cytology was the most accurate of all the tests, none can be recommended as a replacement for the currently used standard of a surgical biopsy and pathology assessment. Most studies investigated patients that had been referred to a hospital clinic for further investigation and so we have only limited information on how accurate they would be when used by a general dentist or frontline medical provider.

How up‐to‐date is this review?

Review authors searched for and used studies published up to 20 October 2020.

Summary of findings

Summary of findings 1. Summary of findings table ‐ test comparisons.

Question	What is the diagnostic accuracy of tests for the diagnosis of oral cancer and potentially malignant disorders?
Population	Patients referred to a secondary care facility for further investigation of a clinically evident lesion
Index test	Intra‐ and extra‐oral conventional visual and tactile examination and adjunctive tests (vital staining, oral cytology, light‐based examination, alone or in combination)
Comparator test	Estimates were compared across different adjuncts
Target condition	Oral cancer and potentially malignant disorders
Reference standard	Biopsy and histology
Action	If potentially malignant disorders can be correctly identified at an early stage then remedial action can be taken to improve morbidity and mortality
Diagnostic stage	Lesion detection or assessment of patients presenting with clinically evident suspicious or innocuous lesions
Quantity of evidence	63 studies providing 79 datasets. 4006 lesions with dysplasia or OSCC from a total of 7942 examined lesions (50% prevalence)
Findings: analysis comparing oral cytology, light‐based examination, vital staining, combined tests (staining plus other adjunct) Consequences in a cohort of 1000 lesions
					Pre‐test probability 2.5%^a		Certainty of the evidence
Test	Datasets	Total lesions	Sensitivity (95% CI)	Specificity (95% CI)	False negatives (missed lesions)	False positives	Sensitivity	Specificity
Oral cytology	24	2892	0.90 (0.82 to 0.94)	0.94 (0.88 to 0.97)	2 (1 to 4)	58 (29 to 117)	⊕⊕⊕⊝ MODERATE^b	⊕⊕⊕⊝ MODERATE^b
Light‐based examination	24	2587	0.87 (0.78 to 0.93)	0.50 (0.32 to 0.68)	3 (2 to 5)	487 (312 to 663)	⊕⊕⊝⊝ LOW^c	⊕⊝⊝⊝ VERY LOW^c
Vital staining	22	1780	0.86 (0.79 to 0.90)	0.68 (0.58 to 0.77)	3 (2 to 5)	312 (224 to 409)	⊕⊕⊝⊝ LOW^d	⊕⊝⊝⊝ VERY LOW^d
Combined tests	9	683	0.78 (0.45 to 0.94)	0.71 (0.53 to 0.84)	5 (1 to 14)	283 (156 to 458)	⊕⊝⊝⊝ VERY LOW^e	⊕⊝⊝⊝ VERY LOW^e
Using vital staining as the reference category: difference in sensitivity for oral cytology ‐0.04 (‐0.12 to 0.04) P = 0.33, light‐based ‐0.02 (‐0.11 to 0.08) P = 0.70, combined 0.08 (‐0.18 to 0.33); difference in specificity for oral cytology ‐0.25 (‐0.36 to ‐0.14) P < 0.001, light‐based 0.18 (‐0.03 to 0.39) P = 0.10, combined ‐0.03 (‐0.21 to 0.16)

Test	Characteristics	Classification of response	Other information
Vital staining (e.g. toluidine blue, tolonium chloride)	Vital staining refers to the use of dyes such as toluidine blue or tolonium chloride to stain oral mucosa tissues for OPMD or malignancy (Leston 2010; Lingen 2008; Patton 2008). The procedure is as follows: pre‐rinse with acetic acid rinse with water apply toluidine blue post‐rinse with acetic acid rinse with water observe mucosa to check for staining	The result of the test is classified as positive if tissue is stained and negative if no tissue is stained, or equivocal if no definitive result can be obtained	Advantages: ability to define areas that could be malignant or abnormal but cannot be seen; assess the extent of the OPMD for excision Disadvantages: benign inflammatory lesions are subject to stain; possibility of failure of some cancerous lesions to stain; possibility of failure of some dysplastic lesions (particularly those with a lower grade or with a thick keratotic surface) to stain; variation in test performance depending on how thorough the test procedures are followed; contraindicated in those who are known to be allergic to iodine
Brush cytology (e.g. OralCDx brush biopsy)	Brush cytology refers to the microscopic assessment and interpretation of cell samples from OPMD that are flaked off from the oral mucosa by the brushing, smearing, scraping or lavage to collect cell samples, which are then sealed on glass slides. They are then analysed using an imaging system that assesses the sampled cells (Leston 2010; Lingen 2008; Patton 2008)	Following analysis, cytopathologists classify test results as positive, atypical, or negative.	Advantages: include the ability to collect information from, and detect large or multiple lesions and to access "the basement membrane collecting cells from all three epithelial layers of the oral mucosa. The liquid‐based cytology reduces the problems relating to sampling and fixation and presents a better cytological morphology" (Divani 2009) Disadvantages: smaller or less obvious lesions may be overlooked; difficulties in detecting lesions when there is necrosis or coagulated blood; inadequate training of operators (Divani 2009); cells are potentially seen out of context
Light‐based detection (chemiluminescence e.g. ViziLite plus, tissue fluorescence imaging e.g. ViziLite, Microlux DL; VELscope, Identafi 3000; tissue fluorescence spectroscopy)	Light‐based systems to identify malignant and potentially malignant lesions and to highlight their presence through tissue reflectance (Leston 2010; Lingen 2008; Patton 2008) e.g. using Microlux DL, the procedure is as follows (Lingen 2008): pre‐rinse with acetic acid use blue‐light source to visually assess the oral cavity ViziLite Plus also provides a tolonium chloride solution (toluidine blue) to aid in the marking of the lesion for biopsy once the light source is removed	The result of the test is classed as negative if the appearance of the epithelium is lightly bluish white and positive if the appearance of the epithelium is distinctly white (acetowhite)	Advantages: simple to use; non‐invasive; do not require consumable reagents; provide real‐time results; can be performed by a wide range of operators after a short training period Disadvantages: the necessity of a dark environment; high initial set up (for VELscope) or recurrent costs (for ViziLite in low‐income countries); lack of permanent record unless photographed; inability to objectively measure visualisation results
Blood and saliva analysis	These novel technologies are at an early stage of development and evaluation. Analysis of blood or saliva samples which tests for the presence of biomarkers of OPMD and oral cancer (Brinkmann 2011; Lee 2009; Li 2006)	Cut‐off probabilities vary widely and are dependent on the individual biomarker or combination of biomarkers examined	Advantages: non‐invasive (saliva tests) or minimally invasive (blood tests) Disadvantages: there is a tendency for the estimated diagnostic accuracy of new health technologies to decline over time as evidence from independent evaluations accumulate (Wyatt 1995). This bias, which can be substantial, has been demonstrated in other domains, e.g. acute abdominal pain (Liu 2006) and clinical decision support systems (Garg 2005). Promising biomarker tests in several clinical areas were eventually been shown to be disappointing (Buchen 2011). It remains to be seen whether this is the case with oral cancer and OPMDs

Domain	Patient selection	Index test	Reference standard	Flow and timing
Description	Describe methods of patient selection. Describe included patients (characteristics, prior testing, presentation and severity of the target condition (class), intended use of index test and setting)	Describe the index test(s) and how it was conducted and interpreted. Describe the sequence of tests, any training or calibration of clinicians (levels of agreement should be reported; where this is measured by the kappa statistic, acceptable values range from 0.61 (moderate agreement) to 1.00 (almost perfect agreement) (Landis 1977)), any procedures taken to ensure blinding of examiners, post‐hoc or a priori threshold specification, any conflict of interest or commercial funding. Methods of site selection should be clearly documented	Describe the reference standard and how it was conducted and interpreted. Ideally, the biopsied tissue should be examined by more than 1 pathologist. If there is a lack of agreement any methods for reaching consensus should be clearly documented. Any measures taken to ensure pathologists were blinded to the results of the index tests should be documented, along with the sequence of reference and index tests. Methods of site selection should be clearly documented	Describe the characteristics and proportion of patients who did not receive the index test(s) and/or reference standard, who received a reference standard other than the biopsy, or who were excluded from the 2 x 2 table (refer to flow diagram). Describe the time interval and any interventions between index test(s) and reference standard. The length of time between the index test and reference standard should be short in most cases. If the period elapsed between index test and reference standard is greater than 2 weeks then this will be considered an unacceptable delay
Signalling questions (Yes/No/Unclear)	Was a consecutive or random sample of patients enrolled? Classify as 'Yes' if consecutive patients or a random sample of individuals were recruited Classify as 'No' if non‐consecutive patients or a non‐random sample of individuals were recruited Classify as 'Unclear' if patient selection was not clearly described	Was calibration of examiners undertaken and results reported? Classify as 'Yes' if the examiners participated in dedicated training and calibration was reported to an acceptable standard Classify as 'No' if the examiners did not participate in dedicated training or was not assessed, or training was undertaken but calibration was not to an acceptable standard Classify as 'Unclear' if the information on training and calibration was not stated	Is the reference standard likely to correctly classify the target condition? Classify as 'Yes' if the biopsy was independently confirmed by at least 2 qualified pathologists Classify as 'No' if the biopsy was not independently confirmed by at least 2 qualified pathologists, or there was lack of agreement between pathologists Classify as 'Unclear' if the study does not state who confirmed the biopsy	Was there an appropriate time interval between the index test(s) and reference standard? Classify as 'Yes' if the delay between the index test(s) and reference standard is considered acceptable for most participants Classify as 'No' if the delay between the index test(s) and reference standard is considered unacceptable for most participants Classify as 'Unclear' if the delay between the index test(s) and reference standard is not explicitly stated
‐	Did the study avoid inappropriate exclusions? Classify as 'Yes' if patients with either class I or class II lesions were recruited Classify as 'No' if only patients with class I lesions were recruited Classify as 'Unclear' if class of lesions was not clearly described	Were the index test results interpreted without knowledge of the results of the reference standard? Classify as 'Yes' if interpreters of index test results clearly do not know results of biopsy/histopathology Classify as 'No' if interpreters of index test results clearly know results of biopsy/histopathology Classify as 'Unclear' if study did not provide any information on whether interpreters of index tests were blinded to biopsy/histopathology	Were the reference standard results interpreted without knowledge of the results of the index test? Classify as 'Yes' if pathologists clearly do not know the index test results when interpreting biopsied tissues Classify as 'No' if pathologists know the results of index test results when interpreting biopsied tissues Classify as 'Unclear' if the study did not provide any information on whether the pathologists were blinded to the index test results	Did all patients receive the same reference standard? Classify as 'Yes' if the same reference standard was used in all participants Classify as 'No' if the same reference standard was not used in all participants Classify as 'Unclear' if it is unclear whether different reference standards were used
‐	‐	Where multiple index tests were used, were the results of the second index test interpreted without knowledge of the results of the first index test? Classify as 'Yes' if index test results were interpreted without knowledge Classify as 'No' if the index test results were interpreted with knowledge Classify as 'Unclear' if it is unclear whether the results of the second index test were interpreted without knowledge of the results of the first index test?	‐	Were all patients included in the analysis? Classify as 'Yes' if all patients were included in the analysis Classify as 'No' is only some patients were included in the analysis Classify as 'Unclear' if it is unclear whether all patients were included in the analysis
‐	‐	If a threshold was used, was it pre‐specified? Classify as 'Yes' if the threshold was pre‐specified Classify as 'No' if the threshold was not pre‐specified Classify as 'Unclear' if it is unclear whether the threshold was pre‐specified	‐	‐
‐	‐	Were any conflicts of interest stated? Classify as 'Yes' if the study declared no conflict of interest Classify as 'No' if the study if the study declared a conflict of interest Classify as 'Unclear' there was no information on conflict of interest	‐	‐
Risk of bias: High/Low/Unclear	Could the selection of patients have introduced bias?	Could the conduct or interpretation of the index test have introduced bias?	Could the reference standard, its conduct, or its interpretation have introduced bias?	Could the patient flow have introduced bias?
Concerns regarding applicability: High/Low/Unclear	Are there concerns that the included patients do not match the review question?	Are there concerns that the index test, its conduct, or interpretation differ from the review question?	Are there concerns that the target condition as defined by the reference standard does not match the review question?	‐

Date	Event	Description
29 March 2021	New search has been performed	Searches updated to 20 October 2020
29 March 2021	New citation required but conclusions have not changed	Review update including 23 new studies bringing the total to 63 included studies. Changes to author byline. Conclusions remain unchanged

Test	No. of studies	No. of participants
1 All	64	7942
2 Vital staining with application method covariate	21	1780
3 Cytology with covariate	20	2892
4 Light‐based with covariate	23	2587
6 Vital staining plus adjunct (light or cytology)	9	683

*Study characteristics*
Patient Sampling	Method of patient selection: "ninety patients with tobacco‐associated hyperkeratotic red and white lesion, ulcerative lesion, and frank malignancy. Suspicious‐looking red and white lesions without any history of tobacco were also included in this study. Patients who presented with hypermelanotic lesion associated with tobacco habit were excluded"
Patient characteristics and setting	Age: 22 to 70 years Sex: 15 female, 75 male SES: not reported Ethnicity: not reported Stated risk factors: 41% smokers, alcohol consumption not reported Number of patients/lesions: 90/90 Lesion site: buccal mucosa 70 cases, tongue 7, labial mucosa 6, palatal mucosa 4, and floor of mouth 1 Severity: "Out of 56 potentially malignant lesions, 34 were speckled leukoplakia, 18 were homogeneous leukoplakia, 3 were verrucous leukoplakia, and 1 was erythroplakia" Country: India Type of facility: secondary care facility assumed Prevalence: 82/90
Index tests	Category: light‐based ‐ VELscope, staining ‐ toluidine blue Description: quote: "the lesion was subjected to standard 1% toluidine blue staining" Positivity threshold: quote: "Greenish fluorescence emanating from site was considered as indicative of normal and healthy mucosa. Dark/brownish color fluorescence observed through VELscope was considered as positive for dysplastic changes within the mucosa." Quote: "Dark blue (royal or navy) staining of lesion was considered positive for dysplasia [Figure 3]. No staining or light blue staining of lesion was considered as negative for dysplasia" Sequence of tests: index (VELscope, toluidine blue) followed by reference Training or calibration of clinicians: not reported Blinding of examiners: not reported Multiple tests: no Method of site selection: based on clinical diagnosis Conflict of interests: authors declare no conflict of interest
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: incisional biopsy Positivity threshold: quote: "oral potentially malignant lesions or malignant lesions" Sequence of tests: index then reference Training or calibration of pathologists: not reported Blinding of examiners: not reported Multiple tests: no Method of site selection: quotes: "After scrutinizing photographic evidence of VELscope finding and toluidine blue testing, sites for surgical biopsy were decided" "biopsy sites were selected based on fluorescence loss and/or toluidine blue stain test. In case both the VELscope and toluidine blue test were negative, the site for biopsy was dictated by clinical examination" Target condition: quote: "The dysplastic lesions were graded according to the number of dysplastic features exhibited into mild, moderate, and severe dysplasia"
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: not reported but assumed following index tests Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	Results from the toluidine blue test could possibly be excluded from the meta‐analysis in a sensitivity analysis. The 2 tests may not be considered to be independent and so only the first test (VELscope) could be used. However, nothing is explicitly stated in the manuscript to say that lesions identified by VELscope were subsequently given toluidine blue staining
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "The study focuses on 45 oral mucosa lesions from 32 patients (13 female, 19 male, mean age 59 years, range 42‐82), coming under observation at the Department of Otolaryngology ‐ Head and Neck Surgery, at the University of Catanzaro"
Patient characteristics and setting	Age: mean age 59 years (range 42 to 82) Sex: 13 female, 19 male SES: not reported Ethnicity: not reported Stated risk factors: none stated Number of patients/lesions: 32/45 Lesion site: quote: "The site of the lesion was tongue in 11 cases, buccal mucosa in 9, floor of the mouth in 8 and hard or soft palate in 4" Severity: non‐neoplastic (hyper‐keratoses, hyper‐para‐keratoses, etc.), mild dysplasia, moderate dysplasia, severe dysplasia, in situ carcinoma, invasive carcinoma Country: Italy Type of facility: Department of Otolaryngology ‐ Head and Neck Surgery, at the University of Catanzaro Prevalence: 30/45 lesions were premalignant or malignant
Index tests	Category: vital staining ‐ toluidine blue Description: quote: "Patients rinsed the oral cavity with water for 20 sec. to remove debris prior to rinsing with 1% acetic acid for 20 sec. Toluidine blue (1% W/W) was applied as an oral rinse for 20 sec. and then 1% acetic acid was used for 20 sec to eliminate mechanically retained stain" Positivity threshold: quote: "Lesions that showed dark blue staining were considered to be positive for premalignant or malignant tissue, while those with light staining, or totally not coloured, were considered negative" Sequence of tests: staining followed by reference standard Training or calibration of clinicians: no training reported Blinding of examiners: index test completed before reference standard Multiple tests: no Method of site selection: toluidine blue rinse ‐ no site selection Conflict of interests: not stated
Target condition and reference standard(s)	Category: biopsy (punch) with histopathologic assessment Description: quote: "The biopsies were performed under local anaesthesia by punch biopsy, all specimens were labelled with a progressive number and in a separate book, for each specimen the clinical examination and the result of the toluidine blue staining were reported" Positivity threshold: quote: "Histopathologic diagnoses were referred as: non‐neoplastic (hyper‐keratoses, hyper‐para‐keratoses, etc.), mild dysplasia, moderate dysplasia, severe dysplasia, in situ carcinoma, invasive carcinoma" Sequence of tests: index test followed by reference standard Training or calibration of pathologists: not stated Blinding of examiners: index test completed before reference standard. Quote: "The pathologist examining all the biopsies was not informed regarding the clinical or staining evaluation of each sample" Multiple tests: no Method of site selection: unclear Target condition: precancerous and cancerous oropharyngeal and oral cavity lesions
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: within same appointment so minimal interval Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: consecutive sampling suggested: "All patients over 18 years of age with oral soft tissue lesions who required incisional or excisional biopsy for further diagnosis were included in the study" Exclusions: quote: "Patients with contraindications for biopsy sampling, such as hemorrhagic diseases or uncontrolled systemic diseases, or patients with a confirmed diagnosis of dysplasia or malignancy in a previous biopsy were excluded from the study"
Patient characteristics and setting	Age: 52.3 years (SD 14.8) Sex: 24 female, 21 male SES: not reported Ethnicity: not reported Stated risk factors: tobacco use recorded Number of patients/lesions: 50 patients invited but only 45 eligible, 54 lesions Lesion site: oral cavity Severity: benign, dysplastic, and malignant oral lesions Country: Iran Type of facility: "Oral and Maxillofacial Medicine Department of the School of Dentistry, Mashhad University" Prevalence: 21/54 lesions with dysplasia or malignant carcinoma
Index tests	Category: light‐based ‐ VELscope Description: conventional oral cavity examination was performed prior to the fluorescence visualisation of the oral cavity. Quote: "the location and approximate loss of fluorescence areas and regions irradiating red/orange light were drawn as a schematic illustration with different colors" Positivity threshold: quote: "Regions with LOF or that were seen as red/orange were considered to be suspicious sites (positive VEL)" Sequence of tests: index followed by reference standard Training or calibration of clinicians: not reported Blinding of examiners: not reported Multiple tests: no Method of site selection: any lesion in the oral cavity Conflict of interests: authors declare that they have no conflicts of interest
Target condition and reference standard(s)	Category: incisional biopsy and histopathology Description: biopsies taken from sites identified by conventional examination and VELscope Positivity threshold: with dysplasia or malignant carcinoma Sequence of tests: followed index test Training or calibration of pathologists: quote: "experienced specialist in oral and maxillofacial pathology" Blinding of examiners: not reported Multiple tests: no Method of site selection: quotes: "the selection of biopsy sites were done after the consensus of two specialists in oral and maxillofacial medicine" "For premalignant lesions/conditions without any evidence of dysplasia (COE negative), the biopsy sample was taken from locations that are surgically accessible" Target condition: mild moderate and severe dysplasia or malignant carcinoma
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: not reported, no reason to expect a time interval between tests Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	The reported results used in the meta‐analysis are those in which the specialist conventional examination and the fluorescence have been combined
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "One hundred and sixty‐four consecutive patients aged over 16 years presenting in oral medicine clinics at two London hospitals with white, red, and mixed white and red patches were invited to participate in the study. One hundred and twenty‐six patients (76.8%) consented and were investigated by a standard protocol that involved clinical visual examination and autofluorescence examination followed by biopsy." Of the 126 patients 116 received the reference test
Patient characteristics and setting	Setting: secondary care, oral medicine clinics Age: over 16 years Sex: male 70 (55.6%), female 56 (44.4%) SES: not reported Ethnicity: white 76 (60.3%), non‐white 50 (39.7%) Stated risk factors: smokers 61 (48.4%); ex‐smoker 28 (22.2%); never smoked 37 (29.4%) alcohol: current user 92 (73%); ex‐user 8 (6.4%); never used 26 (20.6%) Number of patients/lesions: 126/126, 10 did not receive reference test Lesion site: buccal mucosa 54 (42.9%), tongue 40 (31.7%), floor of mouth 14 (11.1%), palate 11 (8.7%), alveolar ridge 7 (5.5%) Severity: unclear prior to testing, described as "with white, red, and mixed white and red patches" Country: UK Type of facility: oral medicine clinic at 2 hospitals Prevalence: dysplastic: 44/116 (VELscope and ViziLite)
Index tests	Index test 1 ‐ VELscope Category: light‐based Description: the main lesion was selected and diagnosed by consensus of 2 experienced examiners after comprehensive clinical examination, photographed then examined with the VELscope; quote: "under dimmed room light, with protective eye wear worn by the patient throughout the procedure" Positivity threshold: quote: "The possible outcome of the autofluorescence examination was determined by the manufacturer’s literature i.e. FVL – fluorescence visualization loss, FVR – fluorescence visualization retained and FVI – fluorescence visualization increased" Sequence of tests: VELscope then ViziLite then toluidine blue, followed by reference standard Training or calibration of clinicians: calibrated by an experienced professional from the manufacturer, no results reported Blinding of examiners: index test examiners blinded to reference test results Multiple tests: yes; conducted independently during the same session, by the same examiners. Order: VELscope, ViziLite, toluidine blue Method of site selection: principle area of morphology decided by consensus after visual examination Conflict of interests: equipment provided by LED Diagnosis who manufacture and market the VELscope Index test 2 ‐ ViziLite Category: light‐based Description: chemiluminescent light source adjunct to oral examination; quote: "The oral cavity was rinsed with 1% acetic acid solution prior to examination with the ViziLite" Positivity threshold: quote: "lesions that showed an aceto‐white appearance under the chemiluminescent light" Sequence of tests: VELscope then ViziLite then toluidine blue, followed by reference standard Training or calibration of clinicians: 2 clinicians, 1 of whom was an experienced examiner. No reporting of training or calibration by demonstration of kappa score, although does report that COE findings, the area for further examination identified, and findings using ViziLite were all verified and consensus found between the 2 clinicians Blinding of examiners: index test examiners blinded to reference test results Multiple tests: yes; conducted independently during the same session, by the same examiners. Order: VELscope, ViziLite, toluidine blue Method of site selection: quote: "The principal area (site) of morphologically altered mucosa excluding any ulcerated areas (by consensus of both examiners) was selected and photographed. All further investigations were performed on this clinically detected area of mucosal abnormality" Conflict of interests: equipment provided by manufacturers and marketers of ViziLite; quote: "ViziLite kits for the study were supplied by Zila Inc."
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "A surgical biopsy was performed for histopathological assessment... the presence or absence of dysplasia was further reviewed by an experienced oral pathologist" Positivity threshold: quote: "The presence or absence of dysplasia in the biopsy specimen was recorded by an experienced oral pathologist" Sequence of tests: index tests followed by reference standard Training or calibration of pathologists: only 1 experienced pathologist Blinding of examiners: quote: "A surgical biopsy of the clinically altered area was performed for histopathological assessment, and after formal diagnostic reporting by two pathologists (blinded to ViziLite data), the presence or absence of dysplasia was further reviewed by an experienced oral pathologist." Each index test reported separately, quote taken from ViziLite paper, but not clarified in other paper Multiple tests: no Method of site selection: the index test results were used to inform the selection of biopsy site Target condition: dysplasia
Flow and timing	Patients receiving index test but not reference test: 10 Patients receiving reference test but not index test: 0 Time interval: the tests were conducted independently (though on the same session for patient convenience) Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	Index test ‐ VELscope & ViziLite: only able to report on 116 patients that received reference test. Papers report data on leukoplakia and erythroplakia (through clinical diagnosis only), only data reporting dysplastic or non‐dysplasia from pathology used Results published for ViziLite are misprinted in the journal article, 72 classed as "other", made up of 52 aceto‐white and 20 normal. Confirmed by author
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "Subjects were recruited and screened at walk‐in clinics," no details on methods of recruitment. The study separately included people with suspicious oral lesions (> 14 days), with no prior pathology and a second group with clinically normal oral mucosa (not considered further in this review). Suspicious clinical lesions included leukoplakia, erythroplakia, speckled leukoplakia, ulcerated lesions or masses present for > 14 days Exclusions: 5 participants due to methodological differences, 1 inconclusive biopsy, 6 did not consent to biopsy, 1 had a diagnosis which did not require biopsy, and 1 had a recent biopsy result
Patient characteristics and setting	Age: over 18 years Sex: 51 male, 18 female SES: not reported Ethnicity: not reported Stated risk factors: 20 tobacco smokers, 15 tobacco chewers, 4 both Number of patients/lesions: 69 participants enrolled, 55 included ‐ 64 lesions (53 suspicious lesions and 11 normal) Lesion site: oral cavity Severity: suspicious lesions Country: India Type of facility: walk‐in centre Prevalence: 36/53 lesions
Index tests	Category: vital staining ‐ Wheat Germ Agglutinin‐fluorescein isothiocyanate Description: quotes: "before WGA‐FITC application, tissue autofluorescence was documented and photographed utilizing the excitation light source"... "The oral cavity was then cleansed via an oral rinse", followed by a distilled water rinse, "1 mL of 20 lM WGA‐FITC solution was applied with a swab for approximately 10 seconds to observable oral lesions, including a 2‐cm margin".. "subject then rinsed with approximately 30 mL of distilled water to remove any residual solution. After rinsing, the clinician used the excitation light 173 source and custom filter glasses to scan the oral cavity" Positivity threshold: level of fluorescence was graded on a scale of 0 to 4 (0 = no WGA‐FITC fluorescence; 4 = very bright WGA‐FITC fluorescence). Quote: "any aberrant staining between the suspicious lesion(s) and surrounding normal tissue was considered a 'positive' test result" Sequence of tests: index test followed by reference standard Training or calibration of clinicians: 2 study personnel graded the autofluorescence but no training or calibration was reported Blinding of examiners: not reported Multiple tests: no Method of site selection: observable oral lesions Conflict of interests: 2 employees of Visual Solutions
Target condition and reference standard(s)	Category: incisional or punch biopsy and histopathology Description: quote: "formalin‐fixed, embedded in paraffin, microtome sectioned, and stained with hematoxylin ‐ 205 eosin for histopathological evaluation" Positivity threshold: cancer, dysplasia, benign Sequence of tests: followed index test Training or calibration of pathologists: quote: "board certified examiner" Blinding of examiners: quote: "All specimens were analyzed by a blinded oral pathologist" Multiple tests: no Method of site selection: oral lesion Target condition: cancer and dysplasia
Flow and timing	Patients receiving index test but not reference test: 8 (6 did not consent to biopsy, 1 had a diagnosis which did not require biopsy, and 1 had a recent biopsy result) Patients receiving reference test but not index test: 0 Time interval: not reported, no reason to expect a time interval between tests Patients receiving both index and reference test but excluded from analysis: 1 ‐ inconclusive
Comparative
User defined 1
Notes	We contacted study authors, they will confirm numbers for suspicious lesions only, without the normal mucosa which are included in the results of the paper
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quotes: "One hundred and sixty patients referred to the Department of Oral Medicine, Faculty of Dentistry, Complutense University of Madrid, Spain were selected for this study" "patients presented with 160 mucosal lesions, which required biopsy evaluation"
Patient characteristics and setting	Age: 55.3 +‐ 16.1 years, range 13 to 100 years Sex: 77 male, 83 female SES: not reported Ethnicity: all Caucasian Stated risk factors: 34% smokers, 27% consumed alcohol regularly Number of patients/legions: 160/160 Lesion site: oral cavity Severity: quote: "subjects with benign lesions or clinically suspicious premalignant or malignant lesions that were either white or red, exophytic or presenting as non‐healing ulcers' Country: Spain Type of facility: secondary Prevalence: 29/160
Index tests	Category: vital staining ‐ toluidine blue Description: quote: "Toluidine Blue was applied as a mouth rinse using the protocol described by Mashberg, with 1% aqueous acetic acid applied initially as a mucolytic agent and after Toluidine Blue rinsing to remove excess stain" Positivity threshold: quote: "The stain was considered positive when the surface mucosa took on a blue colour, either if the entire lesion was stained or just a portion of it. Those that do not took colouration or with equivocal findings were considered negatives" Sequence of tests: index followed by reference Training or calibration of clinicians: quotes: "a working clinical diagnosis was established using WHO Criteria 1980" "The test outcome was subjected to clinical evaluation, by four experienced oral pathologists previously calibrated in pairs" Blinding of examiners: index completed prior to reference Multiple tests: no Method of site selection: toluidine blue rinse so no site selection Conflict of interests: not reported
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: biopsy method unclear Positivity threshold: quote: "Following histopathological diagnosis, all lesions were classified in two groups: non‐dysplastic/nonmalignant lesions, when there were no signs of dysplasia or histological malignancy and dysplastic/malignant lesions, when dysplasia or invasion was present" Sequence of tests: index followed by reference Training or calibration of pathologists: quote: "To avoid any inter examiner variability, the biopsies from this study were evaluated by the same pathologist to determine the presence and degree of dysplasia, or malignancy" Blinding of examiners: unclear whether the same pathologist evaluates the index and reference test Multiple tests: no Method of site selection: quote: "For lesions with a positive toluidine test, the biopsy was taken from the stained area" Target condition: dysplasia or histological malignancy
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: quote: "biopsy was taken from the stained area" so during the same appointment Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quotes: "Patients presenting to the tertiary oral medicine referral clinic at UCSF for initial or follow‐up evaluations, who were diagnosed with oral leukoplakia, erythroleukoplakia, or erythroplakia, were enrolled" "A consecutive sample of eligible, consenting participants was enrolled between November 2006 and March 2008"
Patient characteristics and setting	Age: 61 +‐ 10.6 years, range 42 to 90 years Sex: 23 male, 20 female SES: not reported Ethnicity: all Caucasian Stated risk factors: 64% current or past smokers, alcohol consumption not reported Number of patients/legions: 43/77 Lesion site: oral cavity Severity: quote: "patients with oral leuko(erythro)plakia, lesions clinically identified as 'premalignant' or 'suspicious for carcinoma'" Country: USA Type of facility: tertiary Prevalence: 70/77
Index tests	Category: light‐based ‐ ViziLite Description: quote: "patients were asked to rinse with a 1% acetic acid mouthrinse. In a darkened room, a chemiluminescent light was then used for visual inspection of the oral cavity" Positivity threshold: quote: "ViziLite examination that demonstrated increased brightness in comparison with the visual examination was described as ViziLite positive" Sequence of tests: index followed by reference Training or calibration of clinicians: quote: "the results recorded were based on agreement between both of these investigators" Blinding of examiners: index completed prior to reference Multiple tests: yes Method of site selection: not reported Conflict of interests: quote: "This study was funded by Zila Corporation" Category: light‐based (ViziLite) plus staining (toluidine blue) Description: quote: "A preswab containing 1% acetic acid was applied over any visible lesions, and patients were then asked to rinse and expectorate using tap water. This was followed by application of a swab with 1% toluidine blue (tolonium chloride) on the lesions and then a postswab containing 1% acetic acid, and patients were again asked to rinse and expectorate using tap water" Positivity threshold: quotes: "The toluidine blue uptake in the lesions was recorded as positive or negative" "Any staining including a dark or pale blue mucosal uptake, including uptake in fissures, was classified as a positive response." For this combined test, a positive test result was indicated when both the ViziLite and the toluidine blue tests were positive Sequence of tests: index followed by reference Training or calibration of clinicians: quote: "the results recorded were based on agreement between both of these investigators" Blinding of examiners: index completed prior to reference Multiple tests: yes Method of site selection: quote: "application of a swab with 1% toluidine blue (tolonium chloride) on the lesions" Conflict of interests: quote: "This study was funded by Zila Corporation"
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quotes: "An incisional biopsy was carried out under local anesthesia, using either a punch or a scalpel depending on the site and extent of the lesion" "All biopsies were carried out in areas of clinically apparent white, red, or red‐white change (these changes were noted on visual examination before toluidine blue application)" Positivity threshold: quote: "Following histopathological diagnosis, all lesions were classified in two groups: non‐dysplastic/non‐malignant lesions, when there were no signs of dysplasia or histological malignancy and dysplastic/malignant lesions, when dysplasia or invasion was present" Sequence of tests: index followed by reference Training or calibration of pathologists: quote "To avoid any inter examiner variability, the biopsies from this study were evaluated by the same pathologist to determine the presence and degree of dysplasia, or malignancy" Blinding of examiners: quote: "Information regarding toluidine blue staining was not provided to the pathologist" Multiple tests: no Method of site selection: quotes: "All biopsies were carried out in areas of clinically apparent white, red, or red‐white change (these changes were noted on visual examination before toluidine blue application)" "the site of the incisional biopsies was determined based on identification of the most suspicious site within a lesion(s) based on clinical appearance and/or toluidine blue uptake by agreement between these two investigators" Target condition: dysplasia or histological malignancy
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: quote: "These procedures were all performed at a single clinic visit and were completed in approximately 1–2 h" Patients receiving both index and reference test but excluded from analysis: 7 benign lesions (3 non‐specific ulcers, 1 lupus, 3 lichen planus) were excluded from the calculation of sensitivity and specificity
Comparative
User defined 1
Notes	Study used 3 tests, ViziLite, toluidine blue, and combined ViziLite and toluidine blue. This meta‐analysis included results for ViziLite alone, or ViziLite and toluidine blue in combination. The results of toluidine blue alone were not included, as this test was not independent of the ViziLite test
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

*Study characteristics*
Patient Sampling	Method of patient selection: quotes: "Phase II: Of the study all the cases detected positive by chief investigator and expert examiner were considered" "82 participants consented for staining procedure and biopsy procedure"
Patient characteristics and setting	Age: 34.98 years (SD 12.65), range 19 to 69 years Sex: all male SES: not reported Ethnicity: all Caucasian Stated risk factors: 353 (13.7%) smoked mostly Bidis, 702 (27.3%) were tobacco chewers, 1415 (55.01%) were both smokers as well as tobacco chewers; alcohol consumption not reported Number of patients/legions: 43/77 Lesion site: oral cavity Severity: all dysplasia classed as positive Country: India Type of facility: male prison Prevalence: 76/82 Note: these demographic details refer to the screening study (sample size 2572) from which the participants for the diagnostic study were drawn
Index tests	Category: staining (toluidine blue, Lugol's iodine) Description: quote: "1% toluidine blue and 2% Lugol’s iodine were applied as illustrated by Epstein JB10" Positivity threshold: not explicitly stated, reported as + or ‐ Sequence of tests: index followed by reference Training or calibration of clinicians: quote: "Training and calibration of the investigator (AC) in examination process was carried in the Department of Oral medicine and Radiology of the institution" Blinding of examiners: independent assessments of index tests. Quote: "Out of 82 participants, 41 participants were first screened with toluidine blue using above procedure and interpretation. After application of toluidine blue the 41 participants underwent Lugol’s iodine application after 24 hours of washout period. All the procedure was interchanged for 41 participants i.e. 41 participants were first screened with Lugol’s iodine and subsequently with toluidine blue. Lugol’s Iodine was applied on the same region 24 hrs after toluidine blue application and subjected for biopsy" Multiple tests: yes Method of site selection: based on clinical examination Conflict of interests: no conflict of interest statement
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "After taking biopsy, specimen was immediately stored in 10% Formalin solution and subjected to histopathological analysis" Positivity threshold: quote: "For study purpose histological interpretation was limited to the dysplasia present (positive) or absent (negative)" Sequence of tests: index followed by reference Training or calibration of pathologists: quote: "All histopathological interpretations were carried out by the Oral Pathologist (SN)." Training and calibration not reported Blinding of examiners: not reported Multiple tests: no Method of site selection: not reported Target condition: dysplasia or histological malignancy
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: quote: "Toluidine blue application and subjected to biopsy" Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "Fifty‐eight patients who presented with suspected oral lesions were recruited into the study"
Patient characteristics and setting	Age: not reported Sex: not reported SES: not reported Ethnicity: not reported Stated risk factors: quote: "About two‐thirds of cases consisted of smokers or people who chewed betel nuts" Number of patients/lesions: 58 patients and lesions Lesion site: oral cavity Severity: quote: "The lesions were characterised as homogeneous leukoplakia, heterogeneous leukoplakia, erythroplakia, and ulceration" Country: Taiwan Type of facility: secondary Prevalence: (cancer or precancerous) 29/58
Index tests	Category: vital staining ‐ methylene blue Description: quote: "A standard staining procedure included gargling with rinsing solution (1% lactic acid in purified water flavoured with raspberry) for 20 s, subsequently after power air spray on the lesion, the dye was given by gargling methylene blue (1% malachite, 1% eosin, 0.5% glycerol and dimethyl sulphoxide) for 20 s followed by an additional 20 s of gargling with the rinsing solution" Positivity threshold: unclear Sequence of tests: index followed by reference Training or calibration of clinicians: not reported. Quote: "The pattern and intensity of the staining was recorded and supervised by an experienced oral surgeon" Blinding of examiners: index completed prior to reference Multiple tests: no Method of site selection: methylene rinse so no site selection Conflict of interests: quote: "grant supported by NSC‐94‐2314B075 and VGH94242C"
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: biopsy method unclear Positivity threshold: quote: "Following histopathological diagnosis, all lesions were classified in two groups: non‐dysplastic/non‐malignant lesions, when there were no signs of dysplasia or histological malignancy and dysplastic/malignant lesions, when dysplasia or invasion was present" Sequence of tests: index followed by reference Training or calibration of pathologists: quote: "The pathological diagnosis was examined and verified independently by two specialists." Unclear Blinding of examiners: examinations were independent Multiple tests: no Method of site selection: quote: "The most obviously stained area or the most suspicious looking area, if there was no update of dye, was biopsied" Target condition: dysplasia and squamous cell carcinomas
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: completed at the same time Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: any patients with mucosa lesion referred to College of Stomatology, Sun Yat‐sen University between October 2000 and March 2001 were included
Patient characteristics and setting	Age: 58.3 years (range 7 to 76) Sex: male 11%, female 89% SES: not reported Ethnicity: not reported Stated risk factors: not reported Number of patients/lesions: 60/60 Lesion site: mainly on buccal and lingual areas Severity: not stated Country: China Type of facility: College of Stomatology, Sun Yat‐sen University Prevalence: 54/60
Index tests	Category: vital staining ‐ Oratest Description: Oratest staining ‐ similar to toluidine blue. Use the pre‐exam mouthwash fluid for 20 seconds, wash the mouth twice with water, then use 1% toluidine blue to wash the mouth for 20 seconds and then wash the mouth with water twice, then use another mouthwash fluid (not clearly described) for 30 seconds. For the topical application group, 1% toluidine blue was used topically without mouth washing Positivity threshold: blue staining of the lesion predict a positive outcome, blurred blue staining which could not be washed out by the mouthwash fluid was also considered as positive Sequence of tests: before pathological test Training or calibration of clinicians: no training reported Blinding of examiners: index test completed before reference standard Multiple tests: no Method of site selection: not reported Conflict of interests: not reported
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: scalpel biopsy‐based pathology, no further description provided Positivity threshold: not reported Sequence of tests: index then reference test Training or calibration of pathologists: not reported Blinding of examiners: not reported Multiple tests: no Method of site selection: not reported Target condition: oral cancers and epithelial dysplasia
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: not clear Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Unclear risk

*Study characteristics*
Patient Sampling	Second entry created for Cheng 2003, the 60 patients were split into 2 groups, one used a topical application and the other a rinse. To allow for covariate analysis a second entry is necessary to split the data for the results
Patient characteristics and setting
Index tests
Target condition and reference standard(s)
Flow and timing
Comparative
User defined 1
Notes

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "150 consecutive patients with mucosal disorders," 10 excluded as they refused to consent to biopsy, 8 were unable due to racial protection law, 6 had lesions in oropharynx
Patient characteristics and setting	Age: 55.9 years (SD 12.7) Sex: 110 male, 16 female SES: not reported Ethnicity: not reported Stated risk factors: none stated Number of patients/lesions: 126 participants Lesion site: oral cavity, 71 buccal mucosa Severity: suspected lesions Country: Taiwan Type of facility: secondary care Prevalence: 63 dysplasia, 6 cancer, total 69/126
Index tests	Category: light‐based ‐ Hours UOC 100™ digital autofluorescence camera Description: quotes: "handheld 1920 x 1080‐pixel camera designed to display mucosal changes on a 3.5‐in. full‐color TFT–LCD screen" "emits light in the 400–460‐nm spectrum" Positivity threshold: quote: "findings were listed as fluorescence visualization loss (FVL) or fluorescence visualization retained (FVR)" Sequence of tests: index test followed by reference standard Training or calibration of clinicians: quote: "course taught by an experienced professional" Blinding of examiners: not reported, but index test performed first so assumed Multiple tests: no Method of site selection: oral lesions following COE carried out by a certified experienced examiner specialized in oral and maxillofacial surgery Conflict of interests: authors declare that they have no conflict of interest
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "A biopsy was performed for histopathological assessment" Positivity threshold: quote: "presence or absence of dysplasia or oral cancer" Sequence of tests: followed index test Training or calibration of pathologists: quote: "reports were confirmed and graded by a certified pathologist" Blinding of examiners: assume different examiners Multiple tests: no Method of site selection: quote: "any area at the abnormal mucosa with autofluorescence loss or retained" Target condition: dysplasia (oral cancer cases were excluded)
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: completed at the same time Patients receiving both index and reference test but excluded from analysis: 6 patients with oral cancer were excluded from the final analysis as the remit of the paper was to detect/diagnose OPMDs/dysplasia
Comparative
User defined 1
Notes	Numbers taken from Table 3 dysplasia rows, but does not include the 6 cancer diagnoses. We contacted authors for this information on 11th February 2020
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quotes: "study group consisted of 25 patients with 26 lesions which had been visited from Oct 2005 to Jan 2007, at Oral Medicine of Mashhad Faculty of Dentistry and Otorhinolaryngology Department of QAEM, IMAM, REZA and OMID hospitals, Mishhad, Iran" "Inclusion criterion was: lesions clinically diagnosed as oral potentially malignant (leukoplakia, OLP) or malignant lesions (OSCC and verrucous carcinoma) and requiring an incisional biopsy for definite diagnosis"
Patient characteristics and setting	Age: 54.00 ± 17.38 years Sex: 13 male, 12 female SES: not reported Ethnicity: not reported Stated risk factors: unclear Number of patients/lesions: 25/26 Lesion site: oral cavity Severity: quote: "lesions clinically diagnosed as oral potentially malignant (leukoplakia, OLP) or malignant lesions (OSCC and verrucous carcinoma) and requiring incisional biopsy for definite diagnosis" Country: Iran Type of facility: secondary Prevalence: (dysplasia/malignancy) 8/26 Exclusions: history of any treatment for the lesion, systemic contraindication for scalpel biopsy
Index tests	Category: cytology ‐ OralCDx (laboratory processing not performed at OralCDx laboratory) Description: quote: "After determination of site biopsy, under local anaesthesia, needed for scalpel biopsy, the OralCDx brush was placed in the selected area and turned 5 to 10 times until appearing pinpoint bleeding‐upon manufacture's recommendation" Positivity threshold: quote: "The pathological findings were categorized as 3 groups: 1) Positive: dysplastic epithelial changes, 2) Negative: absence of any evidence suggesting dysplasia, 3) Inadequate sampling" Sequence of tests: index followed by reference Training or calibration of clinicians: unclear, quote: "They were examined by a pathologist informed about clinical diagnosis" Blinding of examiners: index completed prior to reference, quote: "blind to the histopathological results" Multiple tests: no Method of site selection: quote: "The most impressive site of biopsy was determined upon one if these criteria: 1) The most probable site of dysplasia/malignancy OR 2) High risk area for dysplasia/malignancy OR 3) The most surgically accessible site" Conflict of interests: university support acknowledged
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: scalpel biopsy method unclear Positivity threshold: quote: "The Pindborg criteria for detecting dysplasia and malignancy were used and the histopathologic diagnosis was made. The presence of dysplasia/malignancy in histopathology was classified as normal, mild, moderate and severe dysplasia (level 1 to 3), carcinoma in situ (level 4) and carcinoma (level 5)" Sequence of tests: index followed by reference Training or calibration of pathologists: unclear Blinding of examiners: quote: "The histopathologic preparations were observed by the same pathologist blind to the cytopathical study and informed about clinical diagnosis" Multiple tests: not applicable Method of site selection: quote: "The scalpel biopsy was done immediately in the site of pin‐point bleeding" Target condition: quote: "The Pindborg criteria for detecting dysplasia and malignancy were used and the histopathologic diagnosis was made. The presence of dysplasia/malignancy in histopathology was classified as normal, mild, moderate and severe dysplasia (level 1 to 3), carcinoma in situ (level 4) and carcinoma (level 5)"
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: not discussed Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	Assumed figures in Table 2 should be reversed: 'Gold Standard' Normal and Disease should be reversed. This has been done for data entry
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Unclear risk

*Study characteristics*
Patient Sampling	Method of patient selection: 132 patients from School and Hospital of Stomatology, Wuhan University, China between July 2002 and November 2003 Inclusion criteria for patients: (i) superficial ulceration suspicious of malignancy (ii) oral leukoplakia (iii) oral lichen planus (iv) oral leukokeratosis Excluded: (i) benign oral lesion (ii) lesion without histological result after clinical diagnosis
Patient characteristics and setting	Age: mean 49.4 (SD 12.7) years Sex: 67 female, 61 male SES: not specified Ethnicity: not specified Stated risk factors: not discussed Number of patients/lesions: 128/128 Lesion site: oral cavity Severity: all patients suspected of malignancy, leukoplakia, lichen planus, leukokeratosis Country: China Type of facility: secondary Prevalence: 33/128
Index tests	Category: vital staining ‐ rose bengal Description: quote: "(i) oral examination of the location, size, morphology and surface characteristics of the lesions, (ii) mouth rinse with distilled water (reagent A) to clean the lesions for 1 minute, (iii) applications of solution of RB (reagent B) with cotton tip for 2 minutes, (iv) mouth rinse with distilled water (reagent C) to remove excess RB solution for 1 minutes, (v) oral examination of the location, size, morphology and surface characteristics of sites stained" Positivity threshold: quote: "Staining result of a lesion was classified as 1, 2, 3 or 4 according to the shade tabs. In the present study, staining results of 3 and 4 were regarded as RB positive staining, while staining results of 1 and 2 were regarded as RB negative staining" Sequence of tests: unclear regarding ordering, implied that histology performed after index Training or calibration of clinicians: single clinician training not reported Blinding of examiners: assume given implied sequence of tests Multiple tests: no Method of site selection: oral examination Conflict of interests: grant sponsor, Science and Technology Bureau of Wuhan City, People’s Republic of China; grant number: 20026002084, assume no conflict
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: scalpel used, methods unclear Positivity threshold: unclear reporting, but outcomes reported as either malignant (including dysplasia or squamous cell carcinoma) or benign Sequence of tests: unclear, but assumed after index from reporting structure Training or calibration of pathologists: none, only 1 experienced oral pathologist used Blinding of examiners: blinded to index test Multiple tests: no Method of site selection: unclear but each patient had 1 lesion, assume biopsy taken from this lesion Target condition: malignant or benign
Flow and timing	Patients receiving index test but not reference test: 4 Patients receiving reference test but not index test: 0 Time interval: not specified Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		Unclear risk

*Study characteristics*
Patient Sampling	Method of patient selection: patients with oral mucosal lesions, possibly consecutive but not clearly reported
Patient characteristics and setting	Age: not reported Sex: not reported SES: not reported Ethnicity: not reported Stated risk factors: none stated Number of patients/lesions: 200 patients/lesions Lesion site: oral cavity Severity: quote: "Patients with oral mucosal lesions" Country: India Type of facility: Department of Oral Pathology and Microbiology Prevalence: 25/200 were malignant, the remainder were benign
Index tests	Category: light‐based ‐ VELscope Description: not reported Positivity threshold: loss of autofluorescence, quotes: "appeared dark compared to the surrounding unaltered tissue", versus "lesions that exhibited retention of autofluorescence" "Only a complete FVL was rated as malignant or dysplastic ...and lesions that demonstrated autofluorescence patterns other than a complete FVL were included in the FVR group" Sequence of tests: index followed by reference standard Training or calibration of clinicians: not reported Blinding of examiners: not reported, although ordering ensured blinding to histology Multiple tests: no Method of site selection: unclear Conflict of interests: authors declared no conflict of interest
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "Hematoxylin and eosin staining of the formalin fixed paraffin embedded tissue sections was carried out" Positivity threshold: benign/malignant ‐ no dysplasia labelled in results Sequence of tests: index test followed by reference standard Training or calibration of pathologists: quote: "two experienced oral pathologists" Blinding of examiners: quote: "blinded to the VELscope findings" Multiple tests: no Method of site selection: oral lesions detected on COE Target condition: malignant/dysplastic lesions
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: not reported, no reason to expect a time interval between tests Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	Uncertain over other conditions which authors classify as benign but may be categorised as dysplasia in other studies, needs clarification (table 3)
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "Thirty‐five patients with oral mucosal lesions identified by the Orofacial Lesions Council of Ege University, Izmir, Turkey, were seen for further evaluation"
Patient characteristics and setting	Age: mean 56.2 years Sex: 13 male, 22 female SES: not stated Ethnicity: not stated Stated risk factors: 47% smokers Number of patients/lesions: 35/43 Lesion site: buccal mucosa 56%, tongue 19%, hard palate 14% Severity: quote: "Lesions selected for further examination with Tblue staining and brush cytology were homogenous and non‐homogenous leukoplakia, reticular erosive/ulcerated lichenoid lesions, and superficial ulcerations suspicious of malignancy" Country: Turkey Type of facility: university clinic Prevalence: 15/43
Index tests	Index test 1: toluidine blue Category: vital staining Description: quote: "head, neck and intraoral examinations were completed before Tblue application, oral brush cytology and scalpel biopsy" Toluidine blue: quote: "The oral rinsing protocol was: 20 s pre‐rinse with 30 ml of 1% acetic acid; 20 s water rinse; 20 s rinse/gargle with 10 ml of the 1% tolonium chloride solution; 20 s post‐rinse with 30 ml of 1% acetic acid (twice); a final water rinse" Positivity threshold: quote: "The pattern of dye retention and the intensity of stain retention were recorded (2, dark blue staining; 1, minimal blue staining; 0, no blue staining). Occasionally, normal mucosa also appeared light blue, but this staining was not interpreted as positive" Sequence of tests: staining, brush biopsy, followed by reference standard Training or calibration of clinicians: not reported, although performed by 1 examiner Blinding of examiners: not specified Multiple tests: yes. Combined test results used (Table 1) as tests not carried out independently; results of first index test used to inform second index test Method of site selection: not reported Conflict of interests: stated no conflict and funded by university Index test 2: brush cytology Category: cytology ‐ cytobrush Description: quote: "Brush cytology was performed using a Cytobrush Plus GT which was rotated on the lesion site with pressure, until pinpoint bleeding was observed. The harvested cells were transferred to a slide by a 3608 turning and rolling motion with the brush and the slides were washed rapidly with ethyl alcohol for fixation. Cytology specimens were stained with hematoxylin–eosin, and examined by an oral pathologist" Positivity threshold: quote: "The brush cytology results were classified as malignant, atypical (suspicious), benign tissues or inadequate sample" Sequence of tests: toluidine blue, brush biopsy, followed by reference standard Training or calibration of clinicians: not reported Blinding of examiners: not specified Multiple tests: yes. Combined test results used (Table 1) as tests not carried out independently; results of first index test used to inform second index test Method of site selection: not reported Conflict of interests: stated no conflict and funded by university
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quotes: "All lesions were subject to scalpel biopsy with selection based on the clinical appearance of the lesion" "The scalpel biopsy specimens were submitted in formalin for hematoxylin‐eosin staining. After embedding, 5 mm thick sections were prepared" Positivity threshold: quotes: "Pathologic interpretation was based on established criteria and classified as squamous cell carcinoma, epithelial dysplasia, hyperkeratosis, lichen planus, and other benign" "biopsies confirmed as severe dysplasia, carcinoma‐in‐situ or SCC were considered 'positive'; no dysplasia, mild and moderate dysplasia were 'negative'" Sequence of tests: index then reference Training or calibration of pathologists: quote: "Surgical biopsies were performed by an experienced oral and maxillofacial surgeon" Blinding of examiners: not reported Multiple tests: 1 reference standard Method of site selection: quote: "All areas retaining Tblue were biopsied; in sites with no retention of staining, clinical judgment guided the biopsy procedure" Target condition: quote: "Pathologic interpretation was based on established criteria (WHO) and classified as squamous cell carcinoma, epithelial dysplasia, hyperkeratosis, lichen planus, and other benign lesions"
Flow and timing	Patients receiving index tests but not reference test: 0 Patients receiving reference test but not index tests: 0 Time interval: not more than 2 weeks between the 3 methods of investigation Patients receiving both index and reference test but excluded from analysis: 1 for brush biopsy considered a false positive because of inadequate cell sampling
Comparative
User defined 1
Notes	Toluidine blue prepared as an oral rinse since there is no pharmaceutical grade toluidine blue available in Turkey
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		Low risk

PERMALINK

Diagnostic tests for oral cancer and potentially malignant disorders in patients presenting with clinically evident lesions

Tanya Walsh

Richard Macey

Alexander R Kerr

Mark W Lingen

Graham R Ogden

Saman Warnakulasuriya

Abstract

Background

Objectives

Search methods

Selection criteria

Data collection and analysis

Main results

Authors' conclusions

Plain language summary

Summary of findings

Summary of findings 1. Summary of findings table ‐ test comparisons.

Background

Target condition being diagnosed

Index test(s)

1. Index tests for oral cancer and OPMDs.

Clinical pathway

Alternative test(s)

Rationale

Objectives

Secondary objectives

Methods

Criteria for considering studies for this review

Types of studies

Participants

Index tests

Target conditions

Reference standards

Search methods for identification of studies

Electronic searches

Searching other resources

Data collection and analysis

Selection of studies

Data extraction and management

Assessment of methodological quality

2. Indicators for the assessment of quality (QUADAS‐2).

Statistical analysis and data synthesis

Investigations of heterogeneity

Sensitivity analyses

Assessment of reporting bias

Summary of findings and assessment of the certainty of the evidence

Results

Results of the search

1.

Methodological quality of included studies

2.

3.

Vital staining

Oral cytology

Light‐based detection

Combined tests

Findings

4.

5.

Investigations of heterogeneity

Vital staining

Oral cytology

6.

Light‐based detection

7.

Combined tests

Discussion

Summary of main results

Strengths and weaknesses of the review

Applicability of findings to the review question

Authors' conclusions

Implications for practice.

Implications for research.

What's new

History

Acknowledgements

Appendices

Appendix 1. MEDLINE Ovid search strategy

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "96 patients with suspicious oral lesions who attended the outpatients clinics of Otorhinolaryngology Department, Swaroop Rani Nehru Hospital Allahabad, were screened"
Patient characteristics and setting	Age: (benign and premalignant) 38, range 19 to 75 years; (SCC) 52, range 35 to 74 years Sex: (benign and premalignant) 42 male, 22 female; (SCC) 24 male, 8 female SES: not reported Ethnicity: not reported (India) Stated risk factors: (benign and premalignant) 62% smokers, 66% chewed tobacco, 44% chewed pan, 47% consumed alcohol regularly; (SCC) 23% smokers, 80% chewed tobacco, 83% chewed pan, 19% consumed alcohol regularly Number of patients/lesions: 96 Lesion site: oral cavity Severity: quote: "suspicious premalignant or malignant lesions of the oral cavity irrespective of site, stage and sex were selected" Country: India Type of facility: secondary Prevalence: (premalignant and malignant) 46/96
Index tests	Index test 1: toluidine blue Category: vital staining Description: quote: "One percent aqueous toluidine blue was applied to suspicious lesion for 30 seconds. It was followed by a tap water or normal saline rinse. Then it was rinsed by 1% acetic acid for 30 seconds to reduce background staining" Positivity threshold: participants' results classed as positive or negative but no thresholds or inadequate/equivocal results reported. Sequence of tests: staining, brush biopsy, followed by reference standard Training or calibration of clinicians: not reported Blinding of examiners: iIndex completed prior to reference Multiple tests: yes. Combined test results used (Table 3 ) results of first index test used to inform second index test Method of site selection: not reported Conflict of interests: not reported Index test 2: brush cytology Category: cytology Description: quote: "Utilizing a small, hard toothbrush, a transepithelial biopsy was taken with minimum discomfort to the patient. To obtain an adequate specimen, moderate pressure was applied on the lesion by the brush until pinpoint bleeding was noted signalling entry into lamina propria and thus signalling entry into the full thickness of the epithelium" Positivity threshold: quote: "The following parameters were analysed in the smear: enlarged nuclei, variation in nuclear size and shape (pleomorphism), nuclear borders, nuclear/cytoplasmic ratio, number of nuclei, hyperchromatism, chromatin patter and distribution, and discrepancy in maturation." Participants' results classed as positive or negative but no thresholds or inadequate/equivocal results reported Sequence of tests: toluidine blue, brush biopsy, followed by reference standard Training or calibration of clinicians: not reported Blinding of examiners: index completed prior to reference. Not reported whether results of toluidine blue staining were known prior to biopsy Multiple tests: yes. Combined test results used (Table 3 ) results of first index test used to inform second index test Method of site selection: not reported Conflict of interests: not reported
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "For histology, after routine processing and paraffin embedding, several sections (3‐4 mm in thickness) were cut from each case. At least 1 section of each case was stained with hematoxylin‐eosin and examined for histological diagnosis" Positivity threshold: results reported as SCC, dysplasia or benign Sequence of tests: both index tests were followed by reference standard Training or calibration of pathologists: not reported Blinding of examiners: not reported Multiple tests: no Method of site selection: not reported Target condition: benign (including oral submucous fibrosis and angiomatous malformation), premalignant (dysplasia) and squamous cell carcinoma
Flow and timing	Patients receiving index tests but not reference test: 0 Patients receiving reference test but not index tests: 0 Time interval: not reported Patients receiving both index and reference test but excluded from analysis: 0 All patients had toluidine blue staining, oral brush biopsy, and scalpel biopsy
Comparative
User defined 1
Notes	Excluded brush biopsy as an independent index test (blue staining present) and only used results for toluidine blue staining and combined index test of staining and brush biopsy. Numbers for true positive, false positive, true negative, and false negative have been calculated from raw data presented in table 3 sensitivity and specificity values. Results have been entered for both premalignant and malignant
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Unclear
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Unclear risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "Patients presenting in the department of oral and maxillofacial surgery in the University Medical Center Hamburg Eppendorf to rule out invasive squamous cell carcinoma were recruited for this study in a prospective single blinded design"
Patient characteristics and setting	Age: 41 to 76 years Sex: 25 male, 35 female SES: not reported Ethnicity: not reported Stated risk factors: 88% current or previous smokers, 68% consumed > 20 g per day alcohol regularly Number of patients/lesions: 60/60 Lesion site: oral cavity Severity: dysplastic and malignant oral mucosa lesions Country: Germany Type of facility: secondary Prevalence: (premalignant and malignant) 48/60
Index tests	Category: light‐based Description: quote: "The device was used under a dimmed room light ... The autofluorescence excitation device uses visible light in the 430 nm wave length" Positivity threshold: quote: "the complete loss of the normal tissue fluorescence (fluorescence visualization loss) was rated as malignant or dysplastic" Sequence of tests: index followed by reference standard Training or calibration of clinicians: quote: "Additionally both examiners were calibrated for the device in advance" Blinding of examiners: index completed prior to reference Multiple tests: no Method of site selection: not reported Conflict of interests: the authors declare that they have no competing interest
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "All specimens were placed in 4% buffered formalin solution for fixation. Paraffin embedded material was cut into 4 μm thick sections and stained with hematoxylin + eosin (H.E.; MERCK Germany). After color staining all mucosa lesions underwent a histo‐ pathological evaluation by two independent experienced pathologists" Positivity threshold: results reported as SCC, dysplasia or benign Sequence of tests: index test followed by reference standard Training or calibration of pathologists: quote: "two independent experienced pathologists" Blinding of examiners: quote: "two independent experienced pathologists" Multiple tests: no Method of site selection: not reported Target condition: dysplasia, carcinoma
Flow and timing	Patients receiving index tests but not reference test: 0 Patients receiving reference test but not index tests: 0 Time interval: not reported but assumed at the same appointment Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: not reported, reports recruitment at referral clinic and walk‐in centres but no further details Exclusions: 2 had previous biopsy, 9 did not consent, 6 for "methodologic variances by study personnel", 3 had inconclusive pathology reports
Patient characteristics and setting	Age: over 18 years Sex: 62 male, 35 female SES: not reported Ethnicity: not reported Stated risk factors: tobacco users 27%, alcohol 9%, both 23% Number of patients/lesions: 117 patients recruited, 97 included ‐ 86 had 100 clinically suspicious lesions Lesion site: oral cavity Severity: quote: "different suspicious oral lesions at various stages" Country: US and India Type of facility: quote: "a multisite clinical trial was performed at the North Memorial Hospital (Minneapolis, MN) and 2 walk‐in clinics, the Mazumdar‐Shaw Cancer Center and the KLES Dental College (Bangalore, India)" Prevalence: 66/100 were cancerous or dysplastic
Index tests	Category: combined system of light and stain ‐ nuclear stain with UV‐A fluorescence. Where positive results were observed wheat germ agglutinin conjugated to fluorescein isothiocyanate (WGA‐FITC) was applied as a second test Description: quotes: "an ethanol based solution, with citric acid and surfactants, was used to clean the suspicious lesion(s) with a swab, including a 2‐cm margin, followed by a water rinse" "a UV‐excitable fluorescent nuclear stain was applied to the suspicious lesion(s) and margin using a swab, followed by another water rinse." A UVA light was used to assess staining Positivity threshold: quote: "If the nuclear stain’s fluorescent pattern was uniform between the lesion(s) and the margin, then the system’s result was 'negative'" Sequence of tests: clinical exam with UV‐A autofluorescence, nuclear stating, WGA‐FITC, then reference standard Training or calibration of clinicians: not reported Blinding of examiners: not reported Multiple tests: no (combined test as described above) Method of site selection: oral lesions Conflict of interests: 2 authors employed by Inter‐Med, Inc, Racine, WI
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment (brush biopsies used for healthy control) Description: quotes: "scalpel or 5‐mm punch biopsy" "fixed in formalin, embedded in paraffin, sectioned by microtome, and stained with hematoxylin and eosin (H&E) for diagnosis" Positivity threshold: cancerous, dysplastic, or benign Sequence of tests: index tests followed by reference standard Training or calibration of pathologists: quote: "board‐certified oral pathologist" Blinding of examiners: yes Multiple tests: no Method of site selection: oral lesions Target condition: cancerous/dysplastic
Flow and timing	Patients receiving index test but not reference test: 9 Patients receiving reference test but not index test: 0 Time interval: not reported, no reason to expect a time interval between tests Patients receiving both index and reference test but excluded from analysis: 3 had inconclusive pathology reports
Comparative
User defined 1
Notes	Demographics reported for Groups 1 and 2 combined ‐ not reported separately in the manuscript for Group 1 only. For the purposes of meta‐analysis we have taken the results of Table 2 and removed the results for the 11 normal controls that all tested negative
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		High risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "From 01/2005 to 01/2009, 70 patients (45 male, 25 female; mean age 62 years (range 27–88)) with 88 oral lesions of uncertain dignity were included in the study"
Patient characteristics and setting	Age: mean age 62 years, range 27 to 88 years Sex: 45 male, 25 female SES: not reported Ethnicity: not reported Stated risk factors: not reported Number of patients/lesions: 70/88 Lesion site: oral cavity Severity: quote: "clinically suspicious but not obvious malignant lesions" Country: Germany Type of facility: secondary Prevalence: (premalignant and malignant) 37/51
Index tests	Index test 1 ‐ brush cytology Category: cytology ‐ cytobrush Description: quote: "From each lesion, four slides from a separate brush biopsy were taken without prior mucosal disinfection. A proprietary brush (Cytobrush Plus GT; Medscan, Malmo, Sweden) was used to obtain epithelial cells. The small brush was rotated for 10 revolutions over an approximately 10 x 10 mm (100 mm2) area, resulting in most cases in petechial bleeding points. Immediately after harvesting, the cells were transferred onto four pre‐charged glass microscope slide (SuperFrost Plus, Menzel, Braunschweig, Germany) in a rotating manner and fixed by ethyl alcohol (Merckofix, Merck, Darmstadt, Germany). One slide was sent to the local Department of Pathology for hematoxilineosin (H&E) staining and cytopathologic interpretation" Positivity threshold: quotes: "generally accepted cytopathologic criteria" "Insufficient, tumor cell negative, suspicious for tumor cells, tumor cells positive" "grouped as 'negative' in cases with benign changes or with the finding of mild dysplastic epithelial cells (SIN 1) only, and as 'positive' if cells with moderate or severe dysplasia (SIN 2, SIN 3) or malignant tumor cells were present" Sequence of tests: index followed by reference standard Training or calibration of clinicians: quotes: "two experienced pathologists" "In cases of false‐negative or contradictory results for histology, cytology and DNA‐ICM, respective slides were re‐assessed by three experienced persons (BG, BS, KPW). If there was an disagreement, the respective slides were reviewed together until a mutual agreement was obtained" Blinding of examiners: quote: "All investigators were blinded against the results of the complementary method and against the histological results which served as the gold standard" Multiple tests: yes Method of site selection: quote: "From each lesion" Conflict of interests: the authors declare that they have no conflict of interest. Unfunded study Index test 2 ‐ brush cytology plus DNA‐image cytometry Category: cytology ‐ cytobrush plus DNA‐image cytometry Description: quotes: "From each lesion, four slides from a separate brush biopsy were taken without prior mucosal disinfection. A proprietary brush (Cytobrush Plus GT; Medscan, Malmo, Sweden) was used to obtain epithelial cells. The small brush was rotated for 10 revolutions over an approximately 10 x 10 mm (100 mm2) area, resulting in most cases in petechial bleeding points. Immediately after harvesting, the cells were transferred onto four pre‐charged glass microscope slide (SuperFrost Plus, Menzel, Braunschweig, Germany) in a rotating manner and fixed by ethyl alcohol (Merckofix, Merck, Darmstadt, Germany)" "For each specimen, 250–300 randomly chosen cells of the suspicious cell population were measured. The mean DNA content of > 30 non‐neoplastic nuclei (lymphocytes) was taken as internal reference. Coefficients of variation of reference cells were below 5%. Reference extinction was normalized to the 2c (2‐fold DNA content)‐value of the reference cells. The standards and guidelines of the European Society for Analytical Cellular Pathology (ESACP) for DNA‐ICM were followed" Positivity threshold: quotes: "a slide was classified as DNA‐diploid, if only one DNA stemline (STL) between 1.80 c and 2.20 c was detected. DNA‐polyploidy was assumed, if there were DNASTLs between 1.80 c and 2.20 c and between 3.60 c and 4.40 c. A lesion was characterized as DNA‐aneuploid, if an abnormal DNASTL < 1.80 c and > 2.20 c or < 3.60 c and > 4.40 c and/or 9 c exceeding events were detected" "For DNA‐ICM, a case was classified as 'negative' if DNA‐aneuploidy could not be proven, and as 'positive' for the finding of DNA‐aneuploidy" Sequence of tests: index followed by reference standard Training or calibration of clinicians: quotes: "All DNA‐ICM measurements were conducted in duplicates by two investigators (SM, KPW) with experience in the method" "In cases of false‐negative or contradictory results for histology, cytology and DNA‐ICM, respective slides were re‐assessed by three experienced persons (BG, BS, KPW). If there was an disagreement, the respective slides were reviewed together until a mutual agree‐ ment was obtained" Blinding of examiners: quote: "DNA‐ICM was conducted without knowledge of the nature of the lesion or the histopathological grading" Multiple tests: yes Method of site selection: quote: "From each lesion" Conflict of interests: the authors declare that they have no conflict of interest. Unfunded study
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "After harvesting cells, a scalpel incision biopsy (approximately 0.1 ??? 0.1 cm) was taken from exact the same area. The obtained tissue was formalin‐fixed, paraffin‐embedded and diagnosed at the Department of Pathology" Positivity threshold: quotes: "The tissue was classified as normal (optionally with inflammatory alterations), dysplastic (subdivided in squamous intraepithelial neoplasia (SIN) 1 and > 120), or invasive carcinoma" "The histological diagnoses of the scalpel biopsies were grouped as 'negative' in cases with benign changes or with the finding of mild dysplastic epithelial cells (SIN 1) only, and as 'positive' if cells with moderate or severe dysplasia (SIN 2, SIN 3) or malignant tumor cells were present" Sequence of tests: index test followed by reference standard. Training or calibration of pathologists: quote: "two independent experienced pathologists" Blinding of examiners: quote: "two independent experienced pathologists" Multiple tests: no Method of site selection: quote: "After harvesting cells, a scalpel incision biopsy (approximately 0.1 x 0.1 cm) was taken from exact the same area. The obtained tissue was formalin‐fixed, paraffin‐embedded and diagnosed at the Department of Pathology" Target condition: moderate or severe dysplasia, carcinoma
Flow and timing	Patients receiving index tests but not reference test: 0 Patients receiving reference test but not index tests: 0 Time interval: not reported but assumed at the same appointment Patients receiving both index and reference test but excluded from analysis: 1 (insufficient cell yield for DNA‐IC)
Comparative
User defined 1
Notes	Multiple tests but considered independent. Quote: "All investigators were blinded against the results of the complementary method and against the histological results which served as the gold standard." Authors also report results for the "combined" accuracy of the tests, the combination of cytology and DNA‐ICM was considered as positive if either 1 or both of the tests showed positive results. In line with the remit of the review, the combined results were not used as they did not meet the inclusion criteria 'combinations of tests would be included if 1 test informed the use of the second adjunct.' In this case, brush cytology did not inform DNA‐IC so the results for the combined test were not included in the review. However, the results for brush cytology and DNA‐IC alone were used. Results in meta‐analysis are those for all cases
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "100 consecutive patients presenting with clinically suspicious lesions of the oral cavity"
Patient characteristics and setting	Age: mean age 50.4 years (SD 15.2), range 19 to 90 years Sex: 78 male, 22 female SES: not reported Ethnicity: not reported Stated risk factors: 72 smokers, 19 tobacco chewing, 12 betel nut consumers, 10 alcohol Number of patients/legions: 100/100 Lesion site: oral cavity Severity: suspicious for malignancy, malignant Country: India Type of facility: not reported Prevalence: 48/96
Index tests	Category: cytology ‐ brush, DNA‐IC Description: quotes: "The cytology material was collected with the help of a toothbrush. The brush was rotated under slight pressure several times on the suspicious looking area without local anesthesia. Brush was then immediately rolled on glass slides. A minimum of four slides were made per case. The smears were processed for cytologic examination and image analysis. For routine cytology, the smears were stained with MayGrunwald giemsa (May–Grunwald–Giemsa), hematoxylin and eosin (H & E), and Papanicolaou stain (Pap)" "Air‐dried smears were stained by Feulgen method.... If the peak was narrow and the value was between 1.8 c and 2.2 c, it was taken to represent a diploid DNA content. If the peak fell less than 1.8 c or over 2.2 c, it was considered as aneuploid DNA content. Multiple peaks with distribution outside diploid range was also taken as aneuploid" Positivity threshold: suspicious for malignancy, malignant Sequence of tests: index followed by reference Training or calibration of clinicians: not reported Blinding of examiners: not reported Multiple tests: yes Method of site selection: collected by the pathologist from the most suspicious region of the lesion without prior disinfection Conflict of interests: no conflict of interest statement
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: surgical biopsy Positivity threshold: dysplasia, malignancy Sequence of tests: index then reference Training or calibration of pathologists: not reported Blinding of examiners: not reported Multiple tests: no Method of site selection: not reported Target condition: dysplasia, malignancy
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: reference test biopsy immediately followed the index test, quote: "Following the sampling for cytologic analysis, surgical biopsy of the same site was performed in all the cases concomitantly" Patients receiving both index and reference test but excluded from analysis: sampling considered inadequate for 4 cases
Comparative
User defined 1
Notes	The manuscript reports the results for standard brush cytology, DNA‐IC and the results of the test combined. In line with the remit of the review, the combined results were not used as they did not meet the inclusion criteria 'combinations of tests would be included if 1 test informed the use of the second adjunct.' In this case, brush cytology did not inform DNA‐IC so the results for the combined test were not included. However, the results for brush cytology and DNA‐IC alone were used
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "All patients attended the Maxillofacial Surgery Clinic at the University Hospital in Mainz, Germany and were examined between September 2005 and December 2007"
Patient characteristics and setting	Age: 62.8 +/‐ 18.3 years Sex: approximately 2:1 ratio SES: not reported Ethnicity: unclear Stated risk factors: not reported Number of patients/lesions: 135/182 Lesion site: oral cavity Severity: quote: "clinically diagnosed as squamous cell carcinoma (SCC) or suspicious epithelial lesions" Country: Germany Type of facility: secondary Prevalence: 113/182
Index tests	Category: cytology ‐ cytobrush Description: quote: "The cytologic samples were obtained using the Cytobrush Plus GT. The brushes were obtained from the lesions in a rotating manner without local anaesthetic so that petechial bleeding points occurred in the majority of cases" Positivity threshold: quote: "subgroups of benign hyperplasia or hyperkeratosis, mild, moderate or severe dysplasia, and SCC were used to classify the cytologic diagnoses" Sequence of tests: index followed by reference Training or calibration of clinicians: unclear Blinding of examiners: quote: "All cytologic smears were examined blindly, i.e. without information on the patient or the result of the histological examination, by an independent, experienced cytopathologist" Multiple tests: no Method of site selection: most suspicious region of the lesion was sampled Conflict of interests: not reported
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: scalpel, quote: "Following the sampling for cytologic analysis, all oral lesions were examined by taking a conventional biopsy by excision" Positivity threshold: 2 classification systems: SCC and carcinoma in situ; high risk lesions of SIN II/III and SCC Sequence of tests: index followed by reference Training or calibration of pathologists: unclear but 2 pathologists for each sample ‐ 3 pathologists in total Blinding of examiners: analysed by different specialists Multiple tests: no Method of site selection: most suspicious region of the lesion was sampled Target condition: dysplasia and carcinoma
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: same time Patients receiving both index and reference test but excluded from analysis: 4
Comparative
User defined 1
Notes	Low attrition
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "78 patients participating in the study attended the outpatient clinic of the Oral and Maxillofacial Surgery clinic of the Mainz University Medical Centre and suffered from suspicious oral mucosal lesions"
Patient characteristics and setting	Age: mean 61.7 years Sex: 59% male, 41% female SES: not reported Ethnicity: not reported Stated risk factors: not reported Number of patients/lesions: 78/78 Lesion site: cheek 26, gingival 22, floor of the mouth 7, sulcus glossoalv. 2, tongue lower side 2, tongue dorsum 3, palate 8, arcus palatoglossus. 5, inner lips 3 Severity: 41% red, like erythroplakia (17%) or erythroleukoplakia (24%); 21% white, like leukoplakia 21% ulcerous; 17%, a speckled, including fibrin‐covered lesions Country: Germany Type of facility: university medical centre Prevalence: 33/78
Index tests	Category: light‐based Description: quote: "Two different investigation methods were applied: the standard examination by white light and the examination by a 400‐nm wavelength light source that is supposed to trigger a green light emission (> 500 mm) in normal mucosa." Documented by digital reflex photography Positivity threshold: SCC, and dysplasia [identified] depending on 2 different autofluorescence features: 1) a black or dark green aspect, as well as red indicating dysplasia or SCC (positive). Also, a speckled, heterotopic aspect of both green and autofluorescence negative or reddish regions indicated a positive finding; 2) the presence of red mucosal autofluorescence was evaluated as a separate indicator for dysplasia or SCC (positive) Sequence of tests: index then reference Training or calibration of clinicians: not clearly discussed Blinding of examiners: 1 experienced examiner performed the visual examination, 2 examiners examined the photographs, they were blinded and independent Multiple tests: visual then light, class as 1 index test only, as an adjunct Method of site selection: not discussed Conflict of interests: none
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "a biopsy by incision was performed. Then, the biopsies were fixed with formaldehyde 4.5% and processed for light microscopy via paraffin‐embedded, haematoxylin–eosin‐stained slices" Positivity threshold: mucosal hyperkeratosis, lichen planus, SCC, inflammation, dysplasia, healthy mucosa Sequence of tests: index then reference Training or calibration of pathologists: performed by 1 experienced examiner Blinding of examiners: not described Multiple tests: no Method of site selection: biopsy of photographed lesion Target condition: SCC and dysplasia
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: biopsy taken immediately after index test Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	Results calculated from sensitivity and specificity in table 2
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: unreported
Patient characteristics and setting	Age: 52.44 years (SD 13.55) Sex: 98 males, 105 females SES: not reported Ethnicity: not reported Stated risk factors: none stated Number of patients/lesions: 203 patients Lesion site: unclear (subjects with OSCC, OPMDs and other oral mucosa disease without dysplasia according to clinical and pathologic diagnosis) Severity: unclear (no specific criteria on severity) Country: China Type of facility: secondary care (university affiliated dental hospital) Prevalence: 86/203
Index tests	Category: cytology DNA‐IC Description: the DNA cytometry test was conducted with the MotiCytometer system provided by a diagnostic facility company from Xiamen, China. This system uses an automatic microscope and a camera to scan cells in the specimen slices. It uses normal cells in the specimen as internal standard, and performs automatic classifications and counting of cell nucleus based on parameters, among which DNA index (DI) is the key parameter Positivity threshold: standards for a positive result: 1) ≥ 10% hyperplasia, 2) 3 or more cells with DI ≥ 2.5, or 3) aneuploid peak visible Sequence of tests: index followed by reference Training or calibration of clinicians: not stated Blinding of examiners: examiners are blinded from results of the histopathologic assessment Multiple tests: no Method of site selection: not stated Conflict of interests: statement not provided. Funded by the Program for New Clinical Techniques and Therapies of Peking University School and Hospital of Stomatology (PKUSSNCT‐15A05)
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: surgical biopsy. After HE staining, 2 pathologists performed assessments according to the WHO classifications on epithelium dysplasia Positivity threshold: not clearly specified but possibly the WHO classifications on epithelium dysplasia Sequence of tests: index then reference Training or calibration of pathologists: unreported Blinding of examiners: examiners are blinded from results of DNA cytometry Multiple tests: no Method of site selection: unreported Target condition: OSCC, OPMDs and other oral mucosa disease without dysplasia
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: unreported Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	Translation provided by Fang Hua on 2 March 2020
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "52 subjects who sought examination and remedy of oral lesions at the Department of Oral Medicine"
Patient characteristics and setting	Age: mean age 58 years, range 19 to 84 Sex: 15 male, 37 female SES: not reported Ethnicity: not reported Stated risk factors: not reported Number of patients/legions: 52/52 Lesion site: oral cavity Severity: severe dysplasia or neoplasia Country: China Type of facility: secondary care Prevalence: 22/52
Index tests	Category: cytology ‐ DNA‐IC Description: quote: "To obtain smears, we used a cytobrush cell collector, which was rolled at the location of the mucosal lesion at least three times using gentle pressure. The exfoliative cells were transferred to two slides, which were immediately fixed with absolute ethanol. Nuclear DNA contents (ploidy) were measured after Feulgen staining using an automated DNA image cytometer" Positivity threshold: DNA‐aneuploid Sequence of tests: index followed by reference Training or calibration of clinicians: not reported Blinding of examiners: not explicitly stated but "The smears and biopsy tissues were examined at different institutions" Multiple tests: no Method of site selection: suspicious lesions Conflict of interests: the authors declare that they have no conflicts of interests
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: surgical biopsy Positivity threshold: severe dysplasia or malignancy Sequence of tests: index then reference Training or calibration of pathologists: not reported Blinding of examiners: assumed based on quote "The smears and biopsy tissues were examined at different institutions" Multiple tests: no Method of site selection: suspicious lesion Target condition: severe dysplasia or malignancy
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: not reported Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	No information provided in report to enable re‐analysis at mild+ level
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of the index tests?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Unclear
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk

*Study characteristics*
Patient Sampling	Method of patient selection: quote: "A thorough examination of the oral soft tissue was conducted for most patients if asymptomatic mucosal alterations were observed then patients were referred for an evaluation, then rescheduled 10‐14 days later for re‐evaluation and TB testing." Implied that the 14‐day period is part of recruitment process and patients are not deemed part of the study if lesions do not persist. Conditions recorded squamous cell carcinoma (invasive), carcinoma in situ, atypia, or benign (hyperplasia, keratosis, inflammation, etc.)
Patient characteristics and setting	Age: not reported Sex: not reported SES: not reported Ethnicity: not reported Stated risk factors: not reported Number of patients/lesions: 178/235 Lesion site: oral cavity Severity: asymptomatic mucosal alterations Country: USA Type of facility: secondary ‐ Veteran Administration Medical Center so concern over wider applicability Prevalence: 105/235
Index tests	Category: vital staining ‐ toluidine blue Description: photographed before and after direct application of stain, twice mouthrinse with water (20 seconds each), 1 rinse with 1% acetic acid solution (20 seconds), area dried with gauze, '1% toluidine blue solution containing acetic acid and alcohol' applied with cotton swab, rinse with acetic acid solution (1 minute), rinse with water Positivity threshold: quote: "positive for malignancy if the lesion stains dark blue (royal or navy); either the entire lesion or a portion of it may stain solidly or stippled. Occasional equivocal stains are considered positive unless proven otherwise" Sequence of tests: toluidine blue then biopsy Training or calibration of clinicians: not discussed, possibly only 1 examiner Blinding of examiners: not explicitly stated but implied as biopsy results are returned for comparison with toluidine blue results Multiple tests: no Method of site selection: not described, assumed sites of lesions after examination Conflict of interests: not reported
Target condition and reference standard(s)	Category: biopsy with histopathologic assessment Description: quote: "A biopsy of the stained areas was performed" or "a biopsy of the most suspicious area as defined by our erythroplastic criteria was performed" Positivity threshold: invasive carcinoma or carcinoma in situ Sequence of tests: index then reference Training or calibration of pathologists: not discussed Blinding of examiners: not discussed Multiple tests: no Method of site selection: quote: "A biopsy of the stained areas was performed" or "a biopsy of the most suspicious area as defined by our erythroplastic criteria was performed" Target condition: invasive carcinoma, carcinoma in situ, atypia, benign
Flow and timing	Patients receiving index test but not reference test: 0 Patients receiving reference test but not index test: 0 Time interval: description implied biopsy taken immediately after rinsing Patients receiving both index and reference test but excluded from analysis: 0
Comparative
User defined 1
Notes	Re‐analysed data to include atypia as positive (atypia defined in text as epithelial dysplasia)
*Methodological quality*
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Unclear
Were the reference standard results interpreted without knowledge of the results of the index tests?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Was there an appropriate interval between index test and reference standard?	Yes
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Could the patient flow have introduced bias?		Low risk