Standards for Reporting PET Clinical Trials

The report by Kamal et al. in this issue of The Journal of Nuclear Medicine is a seminal contribution to the literature with several timely messages for imaging (1). The message regarding the benefit of surveillance imaging is clear, but this paper also demonstrates the challenges in analysis of pooled clinical data from the literature, as well as special issues that pertain to consistency and standards in clinical imaging. The authors of this paper are appropriately rigorous about the definitions of terms used and assumptions made in their analysis. The study goals were to assess the diagnostic accuracy, clinical impact, impact on therapeutic decisions (including use of other diagnostic tests), and effect on patient outcome of scans obtained during the surveillance timeframe. In this thorough review describing a process of analysis, many limitations in the imaging literature were uncovered.

An initial issue relates to the definition of the term surveillance in patients with solid tumors. The authors define it as “imaging performed at least 6 mo after completion of treatment with curative intent among patients who were considered to be disease-free by clinical examination or other imaging at the time of PET.” This definition seems clear enough, but how practical is a consistent schedule for repeated scans for surveillance in clinical practice? How does this lack of standard protocols affect data analysis from several studies? The study numbers themselves tell a tale. The literature search using the inclusion criteria of surveillance, monitoring, follow-up, and PET imaging for the subject diseases in this report yielded 1,813 citations; the number of papers yielding analyzable data totaled only 12. There is an apparent lack of high-quality literature for this particular use of ¹⁸F-FDG PET or PET/CT for some of the cancers that have the greatest public health impact.

These authors describe in detail how the impact on clinical decision making and clinical outcome can be assessed in a way that will be convincing to the oncology community. The investigators used 2 standardized tools for study quality assessment. These were the Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS) and the Cochrane Risk of Bias tools that apply to diagnostic studies. The guidelines listed in PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (www.prisma-statement.org), were adhered to for the review, and appropriate statistical methods for data analysis were applied. The most disturbing message from this analysis is that the data available to address significant questions about surveillance imaging using ¹⁸F-FDG PET were inadequate for their analysis. Patient groups were often small and included variable tumor histology; some had mixed retrospective and prospective enrollment. Understandably, the results for sensitivity and specificity for diagnosis of unsuspected disease were widely variable for each cancer histology that was assessed. It was difficult to discern the value of these scans, which loosely fell into the category of cancer surveillance. An analysis such as this also supposes that the scans were consistently performed with the same imaging method and image analysis. Numerous groups have emphasized the need to perform ¹⁸F-FDG PET in a consistent and reproducible manner in clinical cancer imaging. However, significant variations existed in dose, uptake time, field-of-view parameters, bed positions, reconstruction parameters, region-of-interest analysis, and partial-volume corrections, not to mention days after treatment.

In the past, payers have considered reimbursement for diagnostic examinations based on literature findings. Criteria for consideration included number of patients enrolled in the study, retrospective or prospective design, and disease type. Today, these criteria are even more stringent. Assessments are standardized and require quantitative information. It is difficult to assess clinical impact, impact on care choices, and impact on clinical outcome without rigorous standardized imaging methods. The most common type of review, metaanalysis studies, can provide misleading conclusions if the data analyzed are of poor quality.

Surveillance imaging of any type in the absence of other clinical indicators of recurrent disease is difficult to defend on many levels. Increased cost in the absence of a clear benefit to the patient is always an obvious consideration. This cost is more than dollars and includes anxiety associated with incidental findings. Perhaps more important is the contribution to lifetime radiation exposure from diagnostic scans, and this dose is considerably higher for a PET/CT scan than for PET using radionuclide attenuation sources. Can the low probability of finding occult disease overcome the risks associated with increased body radiation, even in patients who already have a cancer diagnosis?

The findings reported in this paper demand a constructive response from the diagnostic imaging community, and the authors are to be complimented for so clearly spelling out this message in an unbiased way. We need to design prospective clinical studies that meet QUADAS and other similar criteria for analysis. In clinical cancer imaging, this implies consistent imaging practices, well-defined patient groups, and study questions with clear hypotheses. Metrics to report the impact of the study on patient outcome, clinical practice, medical economics, decision making, and use of other diagnostic tests need to be standardized so that meta-analyses can appropriately compare data. We need to engage members from non-imaging clinical specialties who know the patient group and care practices, imagers, and biostatisticians in joint efforts to ensure that as we go forward, any imaging study provides substantial findings that will make a useful contribution to the imaging literature.

The community of authors, reviewers, and editors should be put on notice by this report so that they may better serve the larger medical community. Why is it that our literature includes so many publications of weak quality? The proliferation of commercial journals, and increasing competition from online formats as well, clearly provides an expanding number of ways in which biomedical study results can be disseminated. We need to strengthen review criteria for submitted manuscripts and train reviewers to follow rigorous guidelines, especially as they relate to study impact. These measures would include a revisiting of the definition of a sound study design, including imaging protocols with testable hypotheses and standards and well-described means of data analysis. Editors should be supported in upholding these principles, for which they assume a major responsibility by their job definition. In many instances, nothing more may be required than that the reviewer indicate the need for a statistical reviewer. Is the imaging community up to this challenge? Of course we must be.

An enormous amount of effort was put into conducting, analyzing, and reporting the 1,813 published studies that were reviewed for inclusion in the analysis of Patel et al., but nearly all of them were judged inadequate for this analysis. We should not expose our patients to radiation with protocols that do not provide useful information, and we cannot afford to waste the efforts of members of our community in performing and publishing weak-quality studies. We all work toward the common goal of providing the highest-quality cost-effective care in our clinical imaging and clinical research imaging. Learning how to perform an effective clinical study is essential for building credibility with the physicians who want to rely on nuclear medicine imaging, indeed any imaging, for making well-informed treatment decisions.