Skip to main content
BMJ Open Access logoLink to BMJ Open Access
. 2021 Oct 6;32(9):536–545. doi: 10.1136/bmjqs-2021-013311

Patient-centred outcomes of imaging tests: recommendations for patients, clinicians and researchers

Matthew J Thompson 1,, Monica Zigman Suchsland 1, Victoria Hardy 2, Danielle C Lavallee 3, Sally Lord 4, Emily Beth Devine 5, Jeffrey G Jarvik 6, Steven Findlay 7, Thomas A Trikalinos 8, Fiona M Walter 9, Roger Chou 10, Beverly B Green 11, Karen J Wernli 11, Annette L Fitzpatrick 12, Patrick M Bossuyt 13
PMCID: PMC10447372  PMID: 34615733

Abstract

Background

Imaging tests are one of the most frequently used diagnostic modalities in healthcare, but the benefits of their direct impacts on clinical decision-making have been countered by concerns that they can be overused. Assessing the relative value of imaging tests has largely focused on measures of test accuracy, which overlooks more comprehensive benefits and risks of imaging tests, particularly their impact on patient-centred outcomes (PCOs). We present the findings of the Patient Reported Outcomes of Diagnostics (PROD) research study in response to a methodological gap in the area of diagnostic test comparative effectiveness research.

Methods

Over a 3-year period, the PROD Study engaged with multiple stakeholders to identify existing conceptual models related to PCOs for imaging testing, conducted primary research and evidence synthesis, and developed consensus recommendations to describe and categorise PCOs related to imaging testing.

Results

The PROD framework categorises PCOs from imaging studies within four main domains: information or knowledge yielded, physical impact, emotional outcomes and test burden. PCOs interact with each other and influence effects across domains, and can be modified by factors related to the patient, clinical situation, healthcare team and the testing environment.

Conclusions

Using PCOs to inform healthcare decision-making will require ways of collating and presenting information on PCOs in ways that can inform patient–provider decision-making, and developing methods to determine the relative importance of outcomes (including test accuracy) to one another.

Keywords: comparative effectiveness research, decision support, clinical, evaluation methodology, patient-centred care, quality improvement methodologies

Introduction

Multiple frameworks have been developed to evaluate diagnostic tests, which typically include generating evidence across phases of technical efficacy, test accuracy, diagnostic efficacy, therapeutic efficacy, patient outcome and societal aspects.1–4 While test accuracy plays a pivotal role for clinical outcomes and regulatory approval, there have been repeated demands for more comprehensive methods to evaluate the benefits and risks of diagnostic tests in terms of patient-centred outcomes (PCOs).5

The concept that tests can impact patient well-being, in addition to ‘clinical’ outcomes,6 has mainly been explored in the context of impacts of false positive screening test results.7–9 There have been few attempts to systematically determine which PCOs are important to patients undergoing testing, nor the extent to which these outcomes are shared across different types of tests, and how these outcomes can be used as part of shared decision-making. Patient-centred care is based on the understanding that PCOs include topics that patients themselves identify as important,10 which in turn can be used to drive service improvements by comparing performance on outcome metrics that matter to patients.11 12 While this concept has been applied to comparative effectiveness research of interventions, it has rarely been applied to diagnostic tests.5

The Patient Reported Outcomes of Diagnostics (PROD) Study13 aimed to develop consensus-based recommendations to guide methods for incorporating PCOs within comparative effectiveness research of diagnostic tests. We focused on imaging testing, given that this is one of the most frequently used modalities of testing in healthcare, yet faces concerns of overuse and rising costs.9 14 15 We used primary research, evidence syntheses and input from multiple stakeholders to describe and categorise PCOs from imaging tests with a goal of informing clinical care, research and policy.

Methods

Overview of approach

Our approach was based on the multistep processes used to develop consensus methods and research reporting guidelines.12 16–18 Over a 3-year period, we: (1) confirmed the methodology gap and identified relevant conceptual models and frameworks, (2) recruited and engaged stakeholders, (3) identified PCOs currently used in clinical recommendations for imaging testing, (4) conducted qualitative research with patients and healthcare providers to identify PCOs, (5) conducted a scoping review of PCOs from existing qualitative literature, and (6) developed consensus-based recommendations on PCOs of imaging testing (figure 1).19 We considered that consensus methods provided two advantages over other methods (such as Delphi or nominal group technique): first, allowing synthesis of the best available information; and second, allowing a process of consensus and validation between key stakeholders.20

Figure 1.

Figure 1

Data sources and methods used to develop final recommendations. Note: This figure was created by the authors and no persmission is required.

Confirmation of methodology gap and identification of relevant conceptual models and frameworks

We used an iterative process to identify published literature that had attempted to address the methodological gap, including an extensive search of existing research on diagnostic test evaluation and imaging tests specifically. This was used to provide additional justification for the proposed research and identify any additional evaluation frameworks for diagnostic tests, conceptual models outlining the range of PCOs that may occur with testing, and literature specific to PCOs from imaging tests. We used a non-systematic scan of the literature search, including existing systematic reviews,1–8 input from methods experts and focused hand-searching.

Stakeholder recruitment and engagement

We adopted the Six-Stage Model for Patient-Centered Outcomes Research and Comparative Effectiveness Research to guide stakeholder engagement.21 Stakeholders were selected from the following: (1) patients and patient advocates with support from a patient advisory network (http://becertain.org/partnerships/patient-advisory-network); (2) primary care clinicians and radiology staff from the Washington, Wyoming, Alaska, Montana and Idaho Practice and Research Network; (3) the American College of Radiology (ACR); (4) consumer advocates; (5) imaging industry and (6) diagnostic methods experts. Recruitment was based on a combination of outreach through national, regional and local organisations. The stakeholders guided research direction, informed data collection instruments, interpreted findings and contributed to research outputs.22 23 Additionally, stakeholders attended a 2-day meeting to develop the final recommendations.

Identification of PCOs currently used in clinical recommendations

In order to identify the frequency and type of PCOs reported in studies of imaging testing that are used to inform clinical recommendations, we conducted a secondary analysis of studies included in the ACR’s Diagnostic Imaging Appropriateness Criteria which are used by referring physicians and radiologists to guide imaging test decisions and are incorporated into clinical decision support mechanisms. We used a broad definition of PCOs used by Patient Centered Outcomes Research Institute (PCORI) and modified by other researchers in this field.5 6 24 We searched for PCOs reported in articles published across all clinical areas relevant to the PROD Study (ie, excluding paediatric and obstetric imaging, interventional radiology), and used systematic methods to extract and synthesise data.25 26

Qualitative research with patients and healthcare providers to identify PCOs

We conducted semistructured interviews with 45 patients who had undergone imaging studies across a variety of conditions and imaging modalities, 16 primary care providers, and 16 radiologists and radiology technologists to seek their experiences and perceptions of PCOs.25 27 Patients and clinicians were recruited from primary care clinical sites and radiology offices affiliated with the Washington, Wyoming, Alaska and Montana Practice and Research Network (WPRN). This research also aimed to identify factors that could influence the perceived importance of these outcomes for patients.28

Scoping review of PCOs from existing qualitative literature

A scoping review of qualitative studies reporting PCOs from imaging studies was used to broaden the evidence base of PCOs beyond those identified from our primary research. We searched for studies reporting PCOs across multiple imaging modalities and clinical settings. The review aimed to identify relevant studies that had explored patients’ emotions, knowledge, and physical preferences in relation to imaging tests either before, during, or after imaging testing. The scoping review, described in full in online supplemental appendix 1, identified and synthesised qualitative research that had reported PCOs from any type of imaging modality, clinical setting and patient group.29–31

Supplementary data

bmjqs-2021-013311supp001.pdf (723.2KB, pdf)

Development of consensus-based recommendation on PCOs for imaging testing

At the end of the 3-year period, 28 stakeholders (online supplemental appendix 2) participated in a 2-day meeting held in Seattle, Washington, which aimed to define and categorise PCOs related to imaging testing, and provide recommendations for next steps needed to implement PCOs in decision-making. We followed the five key elements of consensus methods as outlined by Black et al. 32 (1) Approach to the task: the open-ended study goals were chosen to avoid influencing judgement or selectivity of the stakeholders, and our process included research evidence, experience of consumers (in this case, patients/caregivers) and clinical expertise (in this case, the primary care, radiologists and other stakeholders).33 (2) Participant selection: one of the most important components of our consensus method was engaging multiple stakeholders, as described above, representing different potential viewpoints and expertise on PCOs. We considered this would provide a range of values, beliefs and experiences.20 (3) Presentation of scientific data: we used the regular stakeholder meetings to engage stakeholders across the entire lifespan of this research. This included direct involvement with developing the primary research studies, input to findings from the primary research (including as coauthors), and presenting emerging descriptions and details of PCOs as they emerged. (4) Structure of the interaction: stakeholder meetings were facilitated by research staff with experience in patient engagement and occurred approximately quarterly teleconference for the initial study period of 2 ½ years. We held additional meetings only attended by patients and caregivers, with the same frequency over this period, to allow their voices to be fully heard. (5) Method of synthesising data: we aimed to achieve conclusions regarding the definition and categorisation of PCOs using a reflexive and iterative process.34 Over the initial 2 ½ years of the research period, we synthesised findings from the emerging research at quarterly stakeholder meetings, developing and publishing in peer-reviewed literature the emerging findings. The 2-day in-person conference aimed to debate and review proposed final definitions and categorisation of PCOs, and provide recommendations for next steps needed to implement PCOs in decision-making. We did not intend to use this meeting to eliminate or rank importance of PCOs. Prior to the meeting, stakeholders received preparatory material including descriptions of PCOs that had emerged during the previous 2 ½ years, and lay summaries of additional publications. During the meeting, the research team presented the draft materials and used small group breakout sessions, to seek input on both the clarity of these definitions and whether any PCOs had been overlooked or missed out. We also used small groups to attempt to identify the best way to categorise the PCOs. Following the meeting, multiple written drafts of the consensus recommendations were distributed to stakeholders, and agreement was reached on the final document from all stakeholders.

Role of funding source

The work was funded by the PCORI, which approved the research plan submitted by the research team, but had no input to the research methods, findings, development of consensus recommendations, nor in preparation or approval for any manuscripts submitted for publication.

Human subjects approval

All primary qualitative studies conducted by the authors, which are referenced in this manuscript, were approved by the University of Washington Division of Human Subject as documented in those publications.25 27 28 The activities of this consensus manuscript (including scoping review and systematic review) were determined to not involve human subjects and did not require additional IRB approval.

Results

Confirmation of the methodology gap and identification of relevant conceptual models and frameworks

We identified several recommendations from groups such as guideline development organisations from the USA and Europe, and the Grading of Recommentations Assessment, Development and Evaluation (GRADE) Working Group that supported a need for methods to measure the effects of tests on PCOs.5 18 35 The need for methods to broaden the evaluation of imaging testing was identified in several publications, including limitations in reporting of imaging test research.9 36 Several studies had suggested a range of potential PCOs from diagnostic tests, but none had defined these in a systematic way.6–9 37 This step of the process therefore confirmed that the methods gap that PCORI had identified had not been addressed in other publications.

Identification of PCOs currently used in clinical recommendations for imaging testing

The secondary analysis of the ACR Appropriateness Criteria identified 89 eligible studies; these covered a wide range of clinical areas and imaging modalities.26 The most frequent PCOs identified were: concerns about radiation exposure (n=37), the need for additional testing following an initial test (n=20), test complications (n=19), and indeterminate or incidental findings (n=10). Other PCOs included quality of life (n=7), physical discomfort (n=5), patient values and experiences (n=4), patient financial and time costs (n=4), psychosocial outcomes (eg, depression, anxiety, claustrophobia) (n=4) and test duration (n=2) This analysis highlighted that relatively few PCOs are included in studies that underpin this set of clinical recommendations. The outcomes identified were mainly related to immediate or short-term health complications from the test process itself, and rarely reported from patients themselves.

Qualitative research with patients and healthcare providers to identify PCOs

Analysis of interviews with 45 patients, 16 primary care providers, and 16 radiologists and radiology technologists identified four themes related to PCOs.25 27 38 These were: (1) information or knowledge gained from the test to address patients’ questions and to facilitate next steps in their healthcare; (2) physical experiences during the test procedure, such as discomfort or potential adverse effects; (3) positive and negative impacts of the testing process on patients’ emotions; and (4) the direct and indirect financial burden of testing. This research also highlighted factors that might influence outcomes, such as the effectiveness and content of patient–provider communication, impact of radiology staff, and patients’ previous testing experience, underlying health, level of knowledge, expectations of the imaging test, insurance status, and cultural background.

Scoping review of PCOs from existing qualitative literature

We identified 25 qualitative studies that described PCOs, mainly focusing on mammography and MRI scanning, and most related to cancer screening, conducted in multiple countries (online supplemental appendix 1). We identified PCOs in three main domains, namely: (1) knowledge or information yielded by the imaging test including the desire to know what is wrong, irrespective of the finding, and a desire to know what individuals might experience both during test preparation and the procedure itself; (2) the emotional impact of the test both during preparatory stages and during the test, and the impact of compassion and empathy from radiology staff; and (3) physical discomfort associated with the testing procedure.

Development of consensus-based recommendation on PCOs of imaging testing

The stakeholder meeting facilitated discussion and feedback on proposed PCOs, domains and recommendations. The PROD team developed a matrix which proposed PCOs occurring across a range of domains, before, during and after an imaging test. This matrix was shared with stakeholders to develop consensus regarding the full scope of potential PCOs and categorisation into potential domains. The main outcomes, described below, categorised PCOs within four domains, outlined how PCOs can potentially interact with each other and identified factors that can modify how they are experienced.

Domain 1: information or knowledge yielded by an imaging test

PCOs in this domain included test information that contributes to determining an underlying cause for patients’ symptoms or concerns, or information that led to reducing the likelihood of a condition (table 1). In addition, patients described the value of knowing or seeing these results, regardless to some extent of what the imaging had revealed. Additional PCOs related to the impact of test results on decision-making, such as facilitating access to a higher level of care or a particular treatment course. There were also several negative outcomes, including: misleading information, particularly false positive tests prompting the need for additional testing to confirm/rule out a condition; inconclusive or indeterminate results that did not provide a definitive diagnosis and could lead to further testing; and unexpected or incidental findings that might or might not have clinical significance, but require additional testing or investigations with associated burdens and impact on PCOs.

Table 1.

Patient-centred outcomes within domain of information or knowledge yielded by imaging tests

Outcomes Definition/explanation
Finding cause of symptoms
  • Finding out what is causing symptoms

  • With a known diagnosis, information that leads to finding out how serious it is

  • The desire for a definitive diagnosis, to reduce uncertainty

Reducing the probability of a condition that patient worried about
  • Excluding a serious condition based on the test results

Value of just knowing or finding out more, whatever the outcome
  • The desire to know, to find answers or just to find out/see something in their own body

Decision-making around the information given by the test, leading to action
  • Test results facilitating access to higher level of care or proceed with a particular treatment course

False information from test results
  • Initial positive results leading to need for further tests (or a sequence of tests) to further confirm/rule out a condition (false positives)

  • False reassurance from test results (false negatives)

Incidental and indeterminate findings
  • Indeterminate or inconclusive results leading to further downstream testing (‘testing cascade’) in attempts to arrive at a definitive diagnosis

  • Unexpected findings that may or may not have clinical significance, but can lead to additional testing

This table was created by the authors and no permission is required.

Domain 2: physical effects of the test or testing process

Preparing for imaging tests was associated with specific unpleasant experiences, such as undergoing bowel preparation prior to colonography (table 2). However, more prominent outcomes were pain and physical discomfort while undergoing the imaging procedure (eg, mammography), as well as its immediate consequences. In some instances, the discomfort from an imaging modality depended on the area of the body being examined, for example, transvaginal versus abdominal ultrasound. Other physical outcomes included the effects of ionising radiation, which was mainly cited by patients undergoing frequent imaging.

Table 2.

Patient-centred outcomes within domain of physical outcomes from imaging tests

Outcomes Definition/explanation
Preparation for the test
  • Undergoing preparation such as bowel preparation, or fasting, or ensuring full bladder

Physical discomfort, tolerability during the test
  • Pain from interventions or manipulation needed as part of testing procedure

  • Bruising from body parts being compressed or held in certain positions

Longer term physical effects
  • Reactions to contrast material

  • Cumulative effects of ionising radiation

This table was created by the authors and no permission is required.

Domain 3: emotional impact of the test or testing process

Tests used to evaluate new and/or concerning symptoms provided reassurance and relief when results ruled out certain conditions (table 3). Even in situations when results indicated an underlying condition, some patients reported feelings of relief that they had found the cause of their concerns. However, testing also produced anxiety and negative emotional outcomes that could occur before the test (anticipation anxiety), and afterwards while waiting for results, and as a result of test findings. During the imaging procedure, the physical constraints, confined spaces, noise and unfamiliarity of the procedures could lead to feelings of claustrophobia and distress, as well as embarrassment from having to undress. On some occasions, negative emotional outcomes resulted from disappointment or regret about initiating testing, from test results that did not answer concerns or revealed information that did not advance their care.

Table 3.

Patient-centred outcomes within domain of emotional outcomes from imaging tests

Outcomes Definition/explanation
Reassurance, relief
  • Relief or reassurance after finding symptoms not caused by serious condition

Anxiety, worry
  • Fear or anxiety waiting for testing to be performed and in anticipation of results

  • Stress and anxiety while waiting to get test result

  • Distress and other negative emotional impacts when test shows a serious condition, including false positive results

Claustrophobia, embarrassment
  • Claustrophobia, distress from imaging testing process (eg, narrowness of scanner, noise)

  • Embarrassment or loss of modesty from exposing private body parts

Lack of control
  • Perceived lack of control over the test ordering, its conduct and the next steps the results lead to

  • Feeling abandoned, isolated or helpless during the imaging test itself

Decisional regret, mismatch with expectations
  • Frustration or regret about uncertain test results leading to further testing

  • Disappointment in test results that do not provide the information or findings expected, or that leave residual uncertainty

This table was created by the authors and no permission is required.

Domain 4: test burden

Outcomes related to the direct or indirect burdens of imaging tests (table 4) included financial costs to patients, but these varied with the healthcare/health insurance system. These costs were particularly noted for complex imaging (MRI, CT, etc), and in situations where an initial, less costly, imaging test revealed findings that required more complex, and more expensive, subsequent imaging. Direct costs were not always expected or considered prior to the test itself, and few patients were aware of costs. Similarly, healthcare providers recognised the importance of cost, but lacked information about actual costs. Other test burdens included time off work and travel times particularly for patients living far from imaging centres. Occasionally, the results of imaging testing led to financial benefits to patients, for example, when imaging testing revealed conditions related to occupational or other injuries.

Table 4.

Patient-centred outcomes within domain of burden from imaging tests

Outcomes Definition/explanation
Financial costs of the test
  • Costs that the patient experiences directly or indirectly from the testing process itself

  • Costs that arise from the clinical actions that the test leads to

  • Potential financial benefits from workplace injuries or insurance or disability claims

Disruption to work or social life
  • Disruption to work, school, social activities from waiting for test to be performed, and undergoing the test itself, and waiting for test results (eg, time off work)

This table was created by the authors and no permission is required.

Interactions between outcomes

In addition to the outcomes categorised within these four domains, PCOs interacted with, and influenced, outcomes in other domains. These interactions included, for example, knowledge or information provided by a test result (domain 1) influencing patients’ emotions (domain 3), or, a test that is more physically unpleasant (domain 2) that provides more valuable information (domain 1) than one that is less invasive. The pattern of interactions appeared to be complex and varied with factors such as test modality and clinical situation. Recognising these interactions exist suggests that weighing of risks and benefits across domains is likely to be challenging to incorporate into research or clinical care.

Modifiers of outcomes

Multiple factors potentially impacted or modified whether or not a PCO occurred, its severity, its relative importance and its impact on the patient (table 5).28 These ranged from the characteristics of the individual patient, the type of test they are undergoing, the clinical situation, as well as the healthcare providers involved, the physical environment of the testing suite and communication of results. Embedded within several of these modifiers was the concept of a patient’s prior (pretest) probability of a particular outcome or condition. For example, a patient with a known history of a given condition may be at higher risk of recurrence (higher prior probability), whereas an asymptomatic individual being screened for that condition would be at a lower prior probability. These prior probabilities can influence outcomes following the test result (such as greater anxiety in the emotion domain).

Table 5.

Factors that may modify patient-centred outcomes (PCOs) related to imaging tests

Modifier Definition/examples
Individual patient characteristics
  • Sociocultural factors may impact response to multiple components of the testing process

  • Prior experience of the test or testing process may influence knowledge and expectations of the test’s purpose and acceptance of related risks

  • Prior probability of the condition being tested for, whether actual or perceived by the patient

Test type (ie, screening, diagnostic, monitoring)
  • Screening tests can lead to false positive results given large numbers of asymptomatic people being tested with relatively low prior probability

  • Monitoring or surveillance for known condition (eg, cancer recurrence) can lead to anxiety in test intervals, and/or reassurance if results are negative

Clinical situation
  • Nature of the clinical condition being evaluated and its potential significance for that person’s healthcare may affect the balance of PCOs

  • Testing in high acuity (eg, emergency department) settings may present different balance of benefits and risks (and prior probability of a given condition) compared with lower acuity (eg, primary care) setting

Clinicians and healthcare team
  • Perception of size and importance of test benefits and risks, based on experience, relationship with patient, healthcare setting, and knowledge or perception of an individual patient’s prior probability

  • Ability (including time) to communicate indication for test is being used, relative risks and benefits

  • Medical culture(s) may impose norms around test utilisation, acceptable levels of risk, patient expectations

Physical environment of imaging suite
  • Location in the clinic/hospital, visual appearance can influence patient emotions around the testing process

  • Radiology staff can modify outcomes such as emotions and physical experiences through communication, trust and empathy

Communication of test results
  • Methods and timing of communicating results may impact the knowledge or information or emotional impact of the test

This table was created by the authors and no permission is required.

Discussion

The Institute of Medicine highlighted that patient-centred practices are needed to address the psychological and social dimensions of patients’ healthcare concerns, in order to close quality gaps in healthcare.39 40 This study provides the first comprehensive attempt to define PCOs of imaging tests. We found that PCOs from imaging tests can be categorised into four domains: (1) information or knowledge yielded, (2) physical effects, (3) emotional outcomes and (4) test burden. We also noted that PCOs interact and influence each other in ways that are complex. Moreover, PCOs are not experienced identically; instead, they are influenced by factors related to the patient, the clinical environment and the physical environment of the test.

Our research advances an earlier concept that tests have more than ‘medical value’.6 7 41 42 While certain outcomes related to testing have been previously described, for example, emotional distress from false positive screening tests,43–45 a clear description and categorisation of PCOs related to imaging tests have largely been overlooked.46 47

A strength of our recommendations is that they were informed by a wide body of primary and secondary research, and relied also on extensive input from a range of stakeholders, and are applicable to imaging testing generally, rather than one type of test (eg, screening) or a single imaging modality. We used consensus methods, following key steps that have been recommended for this type of process.32 Furthermore, a key area of guidance from our stakeholders was to focus on PCOs more directly related to the test and the testing process itself, rather than less direct (or indirect) impacts of the test on ‘downstream’ clinical management decisions and outcomes. We acknowledge there is limited literature to evaluate the validity, reliability and rigour of consensus methods,32 but we believe the methods used fulfil the criteria proposed by Hasson et al, namely credibility, applicability, auditability and confirmability.48 We used a process of prolonged engagement, with ongoing reflection of research findings both from research conducted by our team as well as wider literature, and based draft PCOs on evidence, and iterated on these over a period of 3 years including teleconference and a face-to-face conference with a heterogeneous group of stakeholders, representing patient, caregiver, clinician, researcher and industry perspectives. However, we acknowledge that there may now be value in further research to prioritise or rank the PCOs that we have described, using methods such as Delphi or nominal group technique.

The current focus on accuracy in evaluating imaging tests (and diagnostic tests in general) risks ignoring outcomes that may be meaningful from patients’ perspectives; it is the balance of outcomes that are important to patients, their caregivers and clinicians.49 However, patients and their clinicians currently lack information to make reasoned choices on the benefits and harms of diagnostic tests.50 Incorporating PCOs into diagnostic test evaluations has implications for guideline developers and policymakers. Measures of patient satisfaction typically focus on patient experience of services provided, overlooking PCOs. Fulfilling outcomes that are important to patients may lead to greater satisfaction. This can only be achieved for diagnostic testing if the outcomes that are important to patients are known and can be measured.25 51 52 Regulatory approval of new tests also focuses on test accuracy, although there is growing interest by the US Food and Drug Administration to consider additional impacts of tests in such decisions.

In order to advance the use of PCOs in research and implementation of imaging testing, we propose several next steps (box 1). First, validated measurement instruments exist for only some PCOs we identified, and few outcomes are collected using information from patients; there is a need to develop instruments to measure the range of outcomes we describe.53 This may be challenging when measuring the value of information (knowledge) from patients’ perspectives, and in considering the implications of the modifying factors we identified. Second, we need to determine the relative importance of outcomes to each other (and particularly to test accuracy). Evaluating risks and benefits of a given test is likely to vary with the severity, prominence or impact of the PCO related to that test. Balancing beneficial and harmful, as well as short-term and long-term impacts, may require more quantitative or discrete choice methods.54 Third, improved reporting of PCOs by extending current reporting standards for diagnostic accuracy studies55 could facilitate reporting of additional test outcomes.

Box 1. Next steps needed to advance the use of patient-centred outcomes (PCOs) for imaging testing.

Measurement

  • Validated measurement instruments for the full range of PCOs.

  • Outcomes collected from patients themselves.

Relative importance of outcomes

  • Methods to rank the importance of individual PCOs, and to balance positive (beneficial) and negative (harmful) outcomes, considering their severity, impact, timing of outcome.

Reporting

  • Expanded standards for reporting PCOs in evaluation of imaging tests.

Impacting care

  • Information on PCOs that can be collected and meaningfully shared with patients and the healthcare team to inform decision-making.

This box was created by the authors and no permission is required.

Finally, measuring PCOs is of little value if it fails to inform healthcare decision-making and quality of care.51 If the outcomes of imaging testing that are important to patients are known and can be measured, attempts to achieve these outcomes may lead to greater engagement with subsequent clinical management.51 52 Currently, however, for most tests and testing situations, patients, caregivers and providers lack information on PCOs. Efforts will be needed to collect information of PCOs and present this in ways that can be used to guide decision-making.35 56 We anticipate that this expansion of methods for test evaluation will stimulate new standards for research, reporting and use of PCOs, across the wider field of diagnostic testing beyond imaging tests.

Footnotes

Contributors: MJT conceived the study, obtained funding and supervised the research programme, drafted the manuscript and is the guarantor of the submitted manuscript. MZS, VH, EBD, DL, SL, SF, TAT, FMW, RC, BBG, KJW, AF and PMB assisted in drafting and editing the manuscript. All authors reviewed and approved the final version of the manuscript prior to submission.

Funding: This study was funded by Patient Centered Outcomes Research Institute (PCORI) (ME-1503-29245).

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of PCORI

Competing interests: JGJ reports grants from NIH/NIAMS, during the conduct of the study, in particular he was supported by the UW Clinical Learning, Evidence And Research (CLEAR) Center for Musculoskeletal Disorders, NIH/NIAMS P30AR072572 and the content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. JGJ also resports grants from Springer Publishing: royalties as a book co-editor; GE-Association of University Radiologists Radiology Research Academic Fellowship (GERRAF): travel reimbursement for Faculty Board of Review; Wolters Kluwer/UpToDate: royalties as a chapter author.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

No data are available. We do not have data to share for this manuscript.

Ethics statements

Patient consent for publication

Not required.

References

  • 1. Lijmer JG, Leeflang M, Bossuyt PMM. Proposals for a phased evaluation of medical tests. Med Decis Making 2009;29:E13–21. 10.1177/0272989X09336144 [DOI] [PubMed] [Google Scholar]
  • 2. Van den Bruel A, Cleemput I, Aertgeerts B, et al. The evaluation of diagnostic tests: evidence on technical and diagnostic accuracy, impact on patient outcome and cost-effectiveness is needed. J Clin Epidemiol 2007;60:1116–22. 10.1016/j.jclinepi.2007.03.015 [DOI] [PubMed] [Google Scholar]
  • 3. Lin JS, Thompson M, Goddard KAB, et al. Evaluating genomic tests from bench to bedside: a practical framework. BMC Med Inform Decis Mak 2012;12:117. 10.1186/1472-6947-12-117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Walter FM, Thompson MJ, Wellwood I, et al. Evaluating diagnostic strategies for early detection of cancer: the CanTest framework. BMC Cancer 2019;19:586. 10.1186/s12885-019-5746-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5. Gatsonis C. Standards in the design, conduct and evaluation of diagnostic testing for use in patient centered outcomes research Washington, DC: PCORI, 2012. Available: http://www.pcori.org/assets/Standards-in-the-Design-Conduct-and-Evaluation-of-Diagnostic-Testing-for-Use-in-Patient-Centered-Outcomes-Research1.pdf
  • 6. Bossuyt PMM, McCaffery K. Additional patient outcomes and pathways in evaluations of testing. Med Decis Making 2009;29:E30–8. 10.1177/0272989X09347013 [DOI] [PubMed] [Google Scholar]
  • 7. Lee DW, Neumann PJ, Rizzo JA. Understanding the medical and nonmedical value of diagnostic testing. Value Health 2010;13:310–4. 10.1111/j.1524-4733.2009.00597.x [DOI] [PubMed] [Google Scholar]
  • 8. Vis JY, van Zwieten MCB, Bossuyt PMM, et al. The influence of medical testing on patients' health: an overview from the gynecologists' perspective. BMC Med Inform Decis Mak 2013;13:117. 10.1186/1472-6947-13-117 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Carlos RC, Buist DSM, Wernli KJ, et al. Patient-centered outcomes in imaging: quantifying value. J Am Coll Radiol 2012;9:725-8. 10.1016/j.jacr.2012.06.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Snyder CF, Jensen RE, Segal JB, et al. Patient-reported outcomes (pros): putting the patient perspective in patient-centered outcomes research. Med Care 2013;51:S73-9. 10.1097/MLR.0b013e31829b1d84 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Black N. Patient reported outcome measures could help transform healthcare. BMJ 2013;346:f167. 10.1136/bmj.f167 [DOI] [PubMed] [Google Scholar]
  • 12. Center for Medical Technology Policy (CMTP) . Effectiveness guidance documents Baltimore, MD2012. Available: http://www.cmtpnet.org/our-work/providing-methodological-guidance/effectiveness-guidance/
  • 13. Helfand M, Berg A, Flum D. Draft Methodology Report:“Our Questions, Our Decisions: Standards for Patient-centered Outcomes Research”. 23. Patient-Centered Outcomes Research Institute, 2012. [Google Scholar]
  • 14. American’s Health Insurance Plans (AHIP) . Ensuring quality through appropriate use of diagnostic imaging. Washington, DC, 2008. [Google Scholar]
  • 15. American Board of Internal Medicine (ABIM) . Unnecessary tests and procedures in the health care system: what physicians say about the problem, the causes, and the solutions. The ABIM Foundation, 2014: 1–13. [Google Scholar]
  • 16. Perry S, Kalberer JT. The NIH consensus-development program and the assessment of health-care technologies: the first two years. N Engl J Med 1980;303:169-72. 10.1056/NEJM198007173030334 [DOI] [PubMed] [Google Scholar]
  • 17. Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med 2010;7:e1000217. 10.1371/journal.pmed.1000217 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Schünemann HJ, Schünemann AHJ, Oxman AD, Brozek J, et al. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ 2008;336:1106-10. 10.1136/bmj.39500.677199.AE [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Halcomb E, Davidson P, Hardaker L. Using the consensus development conference method in healthcare research. Nurse Res 2008;16:56–71. 10.7748/nr2008.10.16.1.56.c6753 [DOI] [PubMed] [Google Scholar]
  • 20. Campbell SM, Cantrill JA. Consensus methods in prescribing research. J Clin Pharm Ther 2001;26:5–14. 10.1046/j.1365-2710.2001.00331.x [DOI] [PubMed] [Google Scholar]
  • 21. Concannon TW, Meissner P, Grunbaum JA, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med 2012;27:985-91. 10.1007/s11606-012-2037-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA 2012;307:1587–8. 10.1001/jama.2012.442 [DOI] [PubMed] [Google Scholar]
  • 23. Deverka PA, Lavallee DC, Desai PJ, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res 2012;1:181–94. 10.2217/cer.12.7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Gazelle GS, Kessler L, Lee DW, et al. A framework for assessing the value of diagnostic imaging in the era of comparative effectiveness research. Radiology 2011;261:692–8. 10.1148/radiol.11110155 [DOI] [PubMed] [Google Scholar]
  • 25. Zigman Suchsland ML, Witwer E, Truitt AR, et al. Patient-Centered outcomes related to imaging testing in US primary care. J Am Coll Radiol 2019;16:156-163. 10.1016/j.jacr.2018.08.021 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Thompson MJ, Hardy V, Zigman Suchsland M, et al. A Secondary Analysis to Identify Patient-Centered Outcomes in the ACR's Appropriateness Criteria. J Am Coll Radiol 2019;16:1645–55. 10.1016/j.jacr.2019.05.016 [DOI] [PubMed] [Google Scholar]
  • 27. Zigman Suchsland ML, Hardy V, Zhang Y, et al. Provider perspectives of patient experiences in primary care imaging. J Am Board Fam Med 2019;32:392–7. 10.3122/jabfm.2019.03.180288 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. J Pers Soc Psychol 1986;51:1173–82. 10.1037//0022-3514.51.6.1173 [DOI] [PubMed] [Google Scholar]
  • 29. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005;8:19–32. 10.1080/1364557032000119616 [DOI] [Google Scholar]
  • 30. Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010;5:69. 10.1186/1748-5908-5-69 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Colquhoun HL, Levac D, O'Brien KK, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol 2014;67:1291–4. 10.1016/j.jclinepi.2014.03.013 [DOI] [PubMed] [Google Scholar]
  • 32. Black N, Murphy M, Lamping D, et al. Consensus development methods: a review of best practice in creating clinical guidelines. J Health Serv Res Policy 1999;4:236–48. 10.1177/135581969900400410 [DOI] [PubMed] [Google Scholar]
  • 33. Rycroft-Malone J. Formal consensus: the development of a national clinical guideline. Qual Health Care 2001;10:238-44. 10.1136/qhc.0100238 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Campbell SM, Braspenning J, Hutchinson A, et al. Research methods used in developing and applying quality indicators in primary care. BMJ 2003;326:816-9. 10.1136/bmj.326.7393.816 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Gopalakrishna G, Leeflang MMG, Davenport C, et al. Barriers to making recommendations about medical tests: a qualitative study of European guideline developers. BMJ Open 2016;6:e010549. 10.1136/bmjopen-2015-010549 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Mathers SA, Chesson RA, Proctor JM, et al. The use of patient-centered outcome measures in radiology: a systematic review. Acad Radiol 2006;13:1394–404. 10.1016/j.acra.2006.08.010 [DOI] [PubMed] [Google Scholar]
  • 37. Ferrante di Ruffano L, Hyde CJ, McCaffery KJ, et al. Assessing the value of diagnostic tests: a framework for designing and evaluating trials. BMJ 2012;344:e686. 10.1136/bmj.e686 [DOI] [PubMed] [Google Scholar]
  • 38. Zigman Suchsland M, Cruz MJ, Hardy V, et al. Qualitative study to explore radiologist and radiologic technologist perceptions of outcomes patients experience during imaging in the USA. BMJ Open 2020;10:e033961. 10.1136/bmjopen-2019-033961 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Institute of Medicine . Crossing the quality chasm, 2001. Available: https://www.iom.edu/~/media/Files/Report%20Files/2001/Crossing-the-Quality-Chasm/Quality%20Chasm%202001%20%20report%20brief.pdf
  • 40. Rathert C, Wyrwich MD, Boren SA. Patient-centered care and outcomes: a systematic review of the literature. Med Care Res Rev 2013;70:351–79. 10.1177/1077558712465774 [DOI] [PubMed] [Google Scholar]
  • 41. van Ravesteijn H, van Dijk I, Darmon D, et al. The reassuring value of diagnostic tests: a systematic review. Patient Educ Couns 2012;86:3–8. 10.1016/j.pec.2011.02.003 [DOI] [PubMed] [Google Scholar]
  • 42. Whiting P, Toerien M, de Salis I, Salis de, et al. A review identifies and classifies reasons for ordering diagnostic tests. J Clin Epidemiol 2007;60:981–9. 10.1016/j.jclinepi.2007.01.012 [DOI] [PubMed] [Google Scholar]
  • 43. DeFrank JT, Barclay C, Sheridan S, et al. The psychological harms of screening: the evidence we have versus the evidence we need. J Gen Intern Med 2015;30:242-8. 10.1007/s11606-014-2996-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Brewer NT, Salz T, Lillie SE. Systematic review: the long-term effects of false-positive mammograms. Ann Intern Med 2007;146:502–10. 10.7326/0003-4819-146-7-200704030-00006 [DOI] [PubMed] [Google Scholar]
  • 45. Harris RP, Sheridan SL, Lewis CL, et al. The harms of screening: a proposed taxonomy and application to lung cancer screening. JAMA Intern Med 2014;174:281–5. 10.1001/jamainternmed.2013.12745 [DOI] [PubMed] [Google Scholar]
  • 46. Slatore CG, Sullivan DR, Pappas M, et al. Patient-centered outcomes among lung cancer screening recipients with computed tomography: a systematic review. J Thorac Oncol 2014;9:927–34. 10.1097/JTO.0000000000000210 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47. Broderick JE, DeWitt EM, Rothrock N, et al. Advances in Patient-Reported Outcomes: The NIH PROMIS(®) Measures. EGEMS 2013;1:1015. 10.13063/2327-9214.1015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32:1008–15. [PubMed] [Google Scholar]
  • 49. Thompson M, Weigl B, Fitzpatrick A, et al. More than just accuracy: a novel method to incorporate multiple test attributes in evaluating diagnostic tests including point of care tests. IEEE J Transl Eng Health Med 2016;4:2800208. 10.1109/JTEHM.2016.2570222 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. Hoffmann TC, Del Mar C. Patients' expectations of the benefits and harms of treatments, screening, and tests: a systematic review. JAMA Intern Med 2015;175:274–86. 10.1001/jamainternmed.2014.6016 [DOI] [PubMed] [Google Scholar]
  • 51. Gleeson H, Calderon A, Swami V, et al. Systematic review of approaches to using patient experience data for quality improvement in healthcare settings. BMJ Open 2016;6:e011907. 10.1136/bmjopen-2016-011907 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52. Solberg LI, Asche SE, Butler JC. It is time to ask patients what outcomes are important to them. Am J Accountable Care 2015;3. [Google Scholar]
  • 53. Solberg LI, Asche SE, Butler J, et al. Patient-Centered outcomes measurement: does it require information from patients? J Patient Cent Res Rev 2017;4:221-229. 10.17294/2330-0698.1456 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54. Qaseem A, Alguire P, Dallas P, et al. Appropriate use of screening and diagnostic tests to foster high-value, cost-conscious care. Ann Intern Med 2012;156:147–9. 10.7326/0003-4819-156-2-201201170-00011 [DOI] [PubMed] [Google Scholar]
  • 55. Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015;351:h5527. 10.1136/bmj.h5527 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Seidenwurm D, Rosenberg R. Quality of life and diagnostic imaging outcomes. J Am Coll Radiol 2010;7:265–8. 10.1016/j.jacr.2010.01.001 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjqs-2021-013311supp001.pdf (723.2KB, pdf)

Data Availability Statement

No data are available. We do not have data to share for this manuscript.


Articles from BMJ Quality & Safety are provided here courtesy of BMJ Publishing Group

RESOURCES