Monitoring quality of care in hepatocellular carcinoma: A modified Delphi consensus

Abstract

Although there are several established international guidelines on the management of hepatocellular carcinoma (HCC), there is limited information detailing specific indicators of good quality care. The aim of this study was to develop a core set of quality indicators (QIs) to underpin the management of HCC. We undertook a modified, two‐round, Delphi consensus study comprising a working group and experts involved in the management of HCC as well as consumer representatives. QIs were derived from an extensive review of the literature. The role of the participants was to identify the most important and measurable QIs for inclusion in an HCC clinical quality registry. From an initial 94 QIs, 40 were proposed to the participants. Of these, 23 QIs ultimately met the inclusion criteria and were included in the final set. This included (a) nine related to the initial diagnosis and staging, including timing to diagnosis, required baseline clinical and laboratory assessments, prior surveillance for HCC, diagnostic imaging and pathology, tumor staging, and multidisciplinary care; (b) thirteen related to treatment and management, including role of antiviral therapy, timing to treatment, localized ablation and locoregional therapy, surgery, transplantation, systemic therapy, method of response assessment, and supportive care; and (c) one outcome assessment related to surgical mortality. Conclusion: We identified a core set of nationally agreed measurable QIs for the diagnosis, staging, and management of HCC. The adherence to these best practice QIs may lead to system‐level improvement in quality of care and, ultimately, improvement in patient outcomes, including survival.

INTRODUCTION

Hepatocellular carcinoma (HCC) is a leading cause of premature morbidity and cancer‐related death worldwide; it is projected to account for 60% of all global deaths from chronic liver disease by 2030.^{[ 1 ]} Increasing in incidence at a greater rate than all other malignancies,^{[ 2 ]} HCC is now one of the most common global causes of cancer‐related deaths,^{[ 1 ]} with an estimated yearly burden of 830,000 deaths in 2020.^{[ 3 ]} Over the past decade, significant improvements have been made in the treatment of HCC across all stages of disease. Nevertheless, variability in the management of patients with HCC is not infrequent and can lead to suboptimal patient outcomes. For example, adherence to surveillance in at‐risk populations, such as cirrhosis, remains highly variable with adherence rates of only 52% in some series.^{[ 4 ]} In addition, adherence to key therapeutic guidelines, such as the Barcelona Clinic Liver Cancer (BCLC) staging system, has been shown in the international literature to be particularly poor in those patients with intermediate (36%) and advanced disease (46%).^{[ 5 ]} Moreover, the likelihood that management decisions are made within the framework of a multidisciplinary team (MDT) may be as low as 11% despite this being widely recommended.^{[ 6 ]}

Monitoring evidence‐based practice across the disease trajectory through prevention and screening, diagnosis and staging, and treatment with respect to their impact on clinical and patient outcomes, including survival, is pivotal to identifying opportunities for improvement initiatives and optimizing care in HCC. Clinical quality registries (CQRs) are recognized as important tools for monitoring and evaluating quality of cancer care. CQRs achieve this by not only measuring variation from what is considered evidence‐based optimal practice but also through the provision of insights into the extent of variation between institutions (through benchmarking performance between centers), thereby driving practice improvements with the potential for subsequent benefits in clinical outcomes. A vital step in measuring quality of care is the development of a core set of quality indicators (QIs) defined a priori by a consensus involving clinical experts and consumers. Such QIs reflect compliance with processes of care that represent optimal practice; they should be evidence based and supported by expert opinion as well as being acceptable to a wide range of stakeholders and, importantly, feasible to measure.^{[ 7 ]}

The aim of this study was to develop a core set of nationally agreed QIs for the diagnosis, staging, and management of HCC by using a modified Delphi (mDelphi) consensus method. These QIs will be critical in measuring quality of care in HCC and understanding the extent of variation in the provision of high‐quality care as well as the impact of this variation on clinical outcomes.

MATERIALS AND METHODS

The QIs were derived from an extensive review of the literature, including national and international guidelines, reviewed by an HCC working group, followed by a two‐round mDelphi survey. An overview of the process for developing the HCC QIs is provided in Figure 1.

FIGURE 1.

Developing the HCC quality indicator set. DI, disagreement index; HCC, hepatocellular carcinoma; mDelphi, modified Delphi; QI, quality indicator.

Participants

HCC working group

The HCC working group comprised 21 members from five mainland Australian states and included 14 hepatologists, two medical oncologists, a radiation oncologist, a consumer representative, a palliative care specialist, a hepatopancreatobiliary (HPB) surgeon, and an interventional radiologist.

Expert mDelphi participants

Seventeen additional experts in the management of HCC accepted an invitation to participate in the mDelphi. These included seven hepatologists, two medical oncologists, a radiation oncologist, three surgeons (two HPB and a transplant surgeon), two radiologists, a transplant physician, and a palliative care nurse practitioner. The experts were joined by two consumer representatives with lived experience of HCC. Commitment was sought for all rounds when agreeing to participate in the mDelphi.

Selection of proposed QIs

An extensive literature search limited to English was undertaken using the Medline database to identify high‐quality guidelines and studies that had evaluated quality of care in HCC (Figure 1). The studies were imported into the COVIDENCE online software (www.covidence.org) for study selection. Following screening and review, four guidelines or consensus statements^{[ 8} , ⁹ , ¹⁰ , ^{11 ]} and 11^{[ 5} , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ^{21 ]} full‐text articles were used to determine a preliminary list of potential QIs. Comparable QIs were amalgamated, referenced, and reviewed (AM, JL, SR) and then presented to the HCC working group for assessment. The HCC working group was asked to compare similar QIs grouped by the same indicator reference number and taking the nuances into consideration and to score their preferred choice using the following options: include, exclude, or choose if undecided/indifferent to the proposed QIs. A reason for exclusion was requested by a dropdown menu with the options of: duplication/similar; not specific/too loose; or inaccurate/not standard practice. A 70% consensus among the HCC working group was required for a QI to progress to the mDelphi consensus rounds.

mDelphi consensus rounds

A two‐round mDelphi was undertaken with the HCC working group, the expert mDelphi participants, and consumer representatives. Round 1 consisted of an online survey generated using Qualtrics software, version 2021, (https://www.qualtrics.com) and was accompanied with a detailed supplementary document (Figure 1). Before assigning a score, participants were asked to consider the literature and grade of evidence supporting the QI and its ability to draw attention to a quality of care issue. Participants who had agreed to participate in the mDelphi but had not responded within the given time frame were sent two reminders to complete the survey.

The second round consisted of an online meeting using the Zoom video‐conferencing platform (https://zoom.us/). The meeting was chaired by a nonvoting registry expert (JZ) and facilitated by two clinical expert hepatologists (JL, SR). Each participant received a summary document that included the group results as well as individual scores to support the objectives of this meeting (Figure 1). Participants were involved in real‐time scoring through the audience response system Poll Everywhere (https://www.polleverywhere.com/) following a detailed discussion on each proposed QI. In round 2, participants rated not only the importance but also the feasibility of collecting data associated with the indicator by using the nine‐point scale. Real‐time results were only shared with the group once every participant had completed voting to keep the process blinded.

Analysis and final QI set

The median score for importance, the disagreement index (DI), and the percentage score ≥7 were analyzed for both rounds using Microsoft Excel 2019. In addition, the median score for feasibility was evaluated following round 2. An “unable to comment” response was excluded from these analyses. The DI was calculated using the interpercentile range adjusted for symmetry, which indicates the variation in ratings and was based on the RAND method.^{[ 22 ]} A DI ≥1 indicates significant variation or lack of consensus on the proposed indicator. To capture the most important QIs while reducing the burden of data collection, the following threshold was used to determine the final set of QIs: median score for importance and feasibility ≥7; DI ≤1; and percentage score ≥70% for ratings 7–9.

RESULTS

Summary of indicators

A total of 173 QIs was derived from the literature, amalgamated into 94 proposed measures, and presented to the group (Appendix S1). Of these, 40 QIs progressed to the mDelphi consensus rounds (Appendix S2). The majority of the 54 QIs excluded by the HCC working group were due to similarities with other QIs or the proposed QI was nonspecific.

mDelphi participant characteristics

A summary of invitations and participants is provided in Appendix S3. Thirty‐five (HCC working group, consumer representatives, and experts) members provided their input in the first round to rate the importance of the 40 proposed QIs. Round 2 was conducted in two sessions over a total duration of 7 hours, with 32 participants overall. A summary of the participant characteristics over the two rounds is provided in Table 1.

TABLE 1.

mDelphi participant characteristics

Participant characteristics	Round 1 n = 35 (%)		Round 2 n = 32 (%)
Sex
Female	12	−34	9	−28
Male	23	−66	23	−72
State
Victoria	9	−26	8	−25
New South Wales	8	−23	10	−31
Queensland	7	−20	5	−16
Western Australia	6	−17	5	−16
Other a	5	−14	4	−13
Speciality
Hepatology	20	−57	17	−53
Surgery	4	−11	3	−9
Medical Oncology	3	−9	2	−6
Radiology	3	−9	3	(9)
Consumer representative	1	−3	3	−9
Other b	4	−11	4	−13

Abbreviation: mDelphi, modified Delphi.

^a Australian Capital Territory, South Australia, Tasmania.

^b Radiation oncology, palliative care, transplant physician.

mDelphi rounds

Based on a median score ≥7 and DI < 1, 29/40 (72.5%) indicators were rated very important and 11/40 (27.5%) had disagreement (DI > 1) on importance in the round 1 online survey. No indicators were rated as unimportant (Appendices S4–S6). In round 2, the rating of the indicators took place following an in‐depth discussion among the participants. Of the 40 proposed QIs, 27 (68%) had their wording modified to reflect the diagnosis, staging, or management of HCC.

Final QIs

Final QIs were determined both on the importance and the feasibility to collect data by an HCC registry. Detailed results on the rating and wording changes are provided in Appendices S4–S6. Ten of the proposed 18 diagnostic or staging QIs were initially rated as important and feasible indicators. These included QIs applicable to presentation and initial investigations (n = 2); surveillance (n = 1); imaging and pathology (n = 2); biopsy (n = 1); staging (n = 3); and multidisciplinary care (n = 1). For treatment and management QIs, 15 of the 20 proposed QIs were rated important and feasible. These were antiviral therapy (n = 1); localized therapy (n = 2); liver transplantation (n = 2); locoregional therapy (n = 2); systemic therapy (n = 3); supportive care (n = 3); and other management QIs (n = 2). Although feasibility was an important factor, some indicators moved from important to not important in round 2 as others in the same section were updated and captured similar information. For example, indicator 2.4.1 (transarterial chemoembolization [TACE] is offered as first‐line therapy in BCLC‐B disease) was deemed important in round 1 but unimportant in round 2 as wording was updated and amalgamated to capture 2.4.1 (TACE or transarterial radioembolization [TARE] or other therapies with intent to delay progression and prolong survival is offered to patients with BCLC‐B intermediate stage HCC not suitable for curative treatment). Indicator 2.6.4b (best supportive care offered to patients with stage D disease) was later revised and excluded due to significant overlap with indicator 2.6.1 (patients with BCLC‐D are offered symptom management in conjunction with supportive care services). Two additional QIs related to biopsy (liver biopsy attempted where diagnostic uncertainty remains after adequate imaging with multiphase computed tomography [CT] and/or magnetic resonance imaging [MRI]) and systemic therapy (documented that systemic therapy was offered for patients with Child‐Pugh A cirrhosis or well‐selected patients with Child‐Pugh B cirrhosis plus advanced HCC with macrovascular invasion and/or metastatic disease) were excluded because five or fewer participants scored these QIs despite high ratings for importance and feasibility. Only one outcome indicator, 3.2.1 (postoperative 90‐day mortality of liver resection in patients with cirrhosis should be less than 3%), was rated highly for importance and feasibility (Appendix S6).

At the conclusion of round 2, 23 (58%) of the 40 QIs were proposed for inclusion into an HCC CQR, having met the criteria for importance and feasibility by having a median score ≥7, low DI of <1, and a 7–9 rating from ≥70% of the participants. From the initial 94 proposed QIs, nine diagnostic and staging QIs, 13 treatment and management QIs, and one outcome QI were included in the final set (Table 2).

TABLE 2.

Final consensus quality indicator set

Quality indicator		Valid	Feasible
Diagnosis and staging
1.1	Presentation, initial investigations, and referral
1.1.3	Specialist investigations to achieve a diagnosis completed within 4 weeks of referral	9	9
1.1.4	Documented evaluation at baseline of underlying liver disease etiology, presence/absence of cirrhosis, alpha‐fetoprotein level, extent of liver dysfunction (e.g., Child‐Pugh score or MELD and presence or absence of portal hypertension), and comorbidities (that impact on management)	9	8
1.2	Surveillance
1.2.1	Documented regular surveillance with ultrasound and/or appropriate alternative liver imaging performed within 6–12 months before the first detection of HCC in patients with known cirrhosis and who are at risk and managed though a specialist center	8	8
1.3	Imaging modalities and pathology
1.3.1	Diagnosis of HCC is confirmed either by established imaging criteria (e.g., LI‐RADS, using multiphase CT or MRI or CEUS) or histologically	9	8
1.3.2	Documented diagnosis in patients without cirrhosis (not known to be at increased risk of HCC) is confirmed by histopathology	8	8
1.5	Staging
1.5.1a	Tumor stage is clearly defined and documented using the BCLC staging system (stage 0, stage A–D)	8	9
1.5.1b	All patients with cirrhosis are stratified according to Child‐Pugh score and/or MELD and BCLC staging system	8	9
1.5.2	Documented staging parameters include radiological imaging (tumor size, number and location of lesions, metastases, and vascular invasion), Eastern Cooperative Oncology Group performance status and cirrhosis status, and assessment of liver function (e.g., Child–Pugh score or MELD)	9	8
1.6	Multidisciplinary care
1.6.1	Diagnosis, staging, and treatment planning of a patient with suspected or proven HCC is managed by an MDT	9	9
Treatment and management
2.1	Antiviral therapy
2.1.1	Patients with HCC (with or without cirrhosis) and viral hepatitis B or C should receive antiviral therapy	9	9
2.2	Localized therapy
2.2.1	Patients with early stage HCC should receive therapy with curative intent	9	9
2.2.2	Liver resection offered as first‐line therapy in patients with preserved liver function, sufficient liver remnant, and absence of significant portal hypertension (or a valid reason for not undergoing treatment)	8	8
2.3	Liver transplantation
2.3.1	Documented discussion of liver transplantation in patients within overall transplant criteria who are not suitable for curative hepatic resection or ablative therapy	8	8
2.3.2	Patients with HCC on the waiting list for liver transplantation should be monitored for HCC progression	8.5	8
2.4	Locoregional therapy
2.4.2	Transarterial chemoembolization or transarterial radioembolization or other therapy with intent to delay progression and prolong survival is offered to patients with BCLC‐B HCC not suitable for curative treatment	8	8
2.4.3	Documented response to HCC‐directed treatment with contrast‐enhanced CT or MRI every 3–6 months	9	9
2.5	Systemic therapy
2.5.1a	Documented that systemic therapy was offered to suitable patients with advanced HCC not amenable to curative or locoregional therapies with intent to prolong survival	9	8
2.5.2	Currently approved first‐line systemic therapy was offered to eligible patients with HCC	9	9
2.6	Supportive care
2.6.1	Patients with BCLC‐D are offered symptom management in conjunction with supportive care services	9	9
2.6.3	Patients referred to palliative care in advanced disease	8	9
2.7	Other
2.7.2	Clinical and radiological (multiphase CT or MRI using standardized criteria) assessment completed to monitor treatment response	9	9
2.7.3	Treatment commences within 4 weeks of decision to treat (from MDT)	8	8.5
Outcome
3.2.1	Postoperative 90‐day mortality after liver resection in patients with cirrhosis should be less than 3%	9	9

Abbreviations: BCLC, Barcelona Clinic Liver Cancer; CEUS, contrast enhanced ultrasound; CT, computed tomography; HCC, hepatocellular carcinoma; LI‐RADS, Liver Imaging and Reporting Data System; MDT, multidisciplinary team; MELD, Model for End‐Stage Liver Disease; MRI, magnetic resonance imaging.

DISCUSSION

Our overarching aim is to improve health outcomes of patients with HCC through the routine monitoring of compliance with an agreed set of QIs that reflect optimal evidence‐based practice. We used a rigorous consensus method to develop a core set of evidence‐based QIs to monitor the quality of care received by patients diagnosed with HCC. Our final QI set is especially useful in settings with universal health coverage, including countries within the Asia Pacific regions. Prior studies have unequivocally demonstrated that the provision of regular audit and feedback reports to clinicians and hospitals as well as benchmarking performance against their peers improves clinical practice and, potentially, survival and/or patient‐reported outcomes.^{[ 23} , ^{24 ]}

In Australia, there is good agreement on how HCC should be managed, with a national consensus statement identifying 31 key recommendations across surveillance, use of multidisciplinary meetings, staging using the BCLC system, diagnosis, treatment options, and patient management. The consensus statement bears strong similarity with American Association for the Study of Liver Diseases guidelines.^{[ 8} , ^{10 ]} Further, an international study identified additional themes, including resection, ablation, and transplantation or other locoregional therapies as a bridge to transplant, as appropriate modalities for early or recurrent HCC.^{[ 25 ]} However, these studies did not develop a set of specific QIs to reflect compliance with evidence‐based optimal practice.

Our study used a systematic consensus method and robust discussions with a wide multidisciplinary group, including consumer representatives, to recommend 23 of 40 proposed QIs for inclusion as part of an HCC CQR. This will enable monitoring and assessment of the appropriateness of treatment and the compliance with optimal care recommendations. This is especially important for QIs with expected variations in care, such as the feasibility of discussing every lesion within a multidisciplinary setting. Whether or not it is appropriate for all patients to be routinely discussed remains controversial, but reporting of such measures will provide stakeholders valuable insights and the opportunity to discuss potential improvements in clinical practice. Further, process measures, such as the documentation of the Child‐Pugh score and/or Model for End‐Stage Liver Disease (MELD) and BCLC staging system, ensure there is clarity on the appropriate treatment pathway for the MDT and serves as a surrogate parameter.

Participants rated 17 QIs as low to moderate importance and/or feasibility, with some disagreement due to factors such as reliability of measurement, burden of data collection, and the related setting of the QI. For example, QIs in section 1.4 pertaining to biopsy for indeterminate lesions were excluded from the core QI set due to disagreement on importance and feasibility, despite reaching high median scores. Participants discussed issues with knowing the true denominator as indeterminate lesions ultimately characterized as other than HCC would not be captured in most HCC databases or registries that included only confirmed cases of HCC. This predicament presented a challenge with feasibility, and ultimately this QI did not reach consensus for inclusion. A further two QIs were proposed in the case of initial investigations and referrals to specialists with documented liver imaging (e.g., ultrasound), blood test results, and underlying liver disease. Following deliberation in round 2, these QIs were considered difficult to collect by a specialist unit and better suited to a primary care setting.

A recently published study by Asrani and colleagues^{[ 26 ]} from North America identified 29 QIs that covered similar areas of surveillance, diagnosis and staging, treatment, and outcomes by using a 13‐member panel. Our study validated 20 QIs developed by Asrani and colleagues (Table 3). However, 13 QIs in our study did not overlap with their published report (Table 3). These included indicators measuring access to investigations and treatment in a timely manner; staging indicators, including explicit documentation using the BCLC criteria, or the stratification of patients with cirrhosis according to Child‐Pugh and/or MELD scores; and treatment with antivirals in viral hepatitis‐mediated HCC or the use of TACE or TARE in intermediate‐stage disease. Nine QIs developed by Asrani et al. did not map to our mDelphi, such as margin negative resections and clinical decompensation, hospice care and length of stay, or the utilization of intensive care unit support in the last 2 weeks of life. They also recommended four patient‐reported outcomes addressing pain, anxiety, fear of treatment, and uncertainty about the future (Table 3). However, whether these patient‐reported outcomes were psychometrically assessed for reliability, feasibility, and validity is unclear.^{[ 26 ]} Participants in our study noted that there were a number of important determinants of quality of care, including patient‐centered indicators to monitor pain control, psychosocial support, and symptom burden. Participants agreed that the use of multidimensional patient‐reported outcome measures assessed for psychometric properties in an HCC population was a more feasible approach to capture patient‐centered care within the registry.

TABLE 3.

Comparison of quality indicators developed by Asrani et al.^{[ 26 ]} and this study

This study	Asrani et al.
Specialist investigations to achieve a diagnosis completed within 4 weeks of referral a	Overlaps with developed quality indicators below
Documented evaluation at baseline of underlying liver disease etiology, presence/absence of cirrhosis, AFP level, extent of liver dysfunction (e.g., Child‐Pugh score or MELD and presence or absence of portal hypertension), and comorbidities (that impact management) b
Documented regular surveillance with US and/or appropriate alternative liver imaging performed within at least 6–12 months before the first detection of HCC in patients with known cirrhosis or who are at risk and managed though a specialist center	Patients with cirrhosis should undergo surveillance for HCC with US of the liver every 6 months, with or without AFP b
	Patients with cirrhosis and cured hepatitis C infection should continue to undergo HCC surveillance b
Demographic information collected by the HCC registry will cover four out of five of the indicators	Regardless of cirrhosis status, Asian men infected with hepatitis B should undergo HCC surveillance beginning at age > 40 years b
	Regardless of cirrhosis status, Asian women infected with hepatitis B should undergo HCC surveillance beginning at age > 50 years b
	Regardless of cirrhosis status, patients with chronic hepatitis B who were born in sub‐Saharan Africa should undergo HCC surveillance beginning at age 20 years b
	Regardless of cirrhosis status, adults infected with hepatitis B who have a family history of HCC should undergo HCC surveillance c
	Patients with underlying chronic liver disease and new AFP > 20 ng/ml should undergo diagnostic evaluation for HCC with dynamic CT or MRI b
Documented diagnosis in patients without cirrhosis (not known to be at increased risk of HCC) is confirmed by histopathology a
Diagnosis of HCC confirmed either by established imaging criteria (e.g., LI‐RADS using documented multiphase CT or MRI or CEUS) or histologically	LI‐RADS should be used by the interpreting radiologist to describe liver lesions found by dynamic CT or MRI in patients with cirrhosis or chronic hepatitis B b
	Among patients who undergo dynamic imaging to diagnose HCC, arterial phase enhancement and portal venous or delayed venous phase washout should be recorded c
	For patients who undergo tumor biopsy, pathological diagnosis of HCC should be based on the International Consensus Group for Hepatocellular Neoplasia recommendations using the required histologic and immune–histologic analyses c
	Patients with LI‐RADS 3 lesions should undergo repeat dynamic imaging (with the same or different imaging modality) within 6 months c
Diagnosis, staging and treatment planning of a patient with suspected or proven HCC is managed by an MDT	Patients with LI‐RADS 4 lesions should be reviewed by an MLTB b
Documented staging parameters include radiological imaging (tumor size, number and location of lesions, metastases, and vascular invasion), Eastern Cooperative Oncology Group performance status and cirrhosis status and assessment of liver function (e.g., by Child–Pugh score or similar scoring system or MELD)	Patients with HCC should undergo cross‐sectional imaging of the chest at the time of HCC diagnosis to evaluate for pulmonary metastases b
	Tumor burden, liver function, and performance status or score reflective thereof should be documented at the time of diagnosis of HCC b
Tumor stage is clearly defined and documented using the BCLC staging system (stage 0, stage A–C, stage D) a
All patients with cirrhosis are stratified according to Child‐Pugh score +/ or MELD and BCLC staging system a
Patients with HCC (with or without cirrhosis) and viral hepatitis B or C should receive antiviral therapy a
Patients with early stage HCC should receive therapy with curative intent a
Liver resection offered as first‐line therapy in patients with preserved liver function, sufficient liver remnant, and absence of significant portal hypertension (or a valid reason for not undergoing treatment)	In patients with BCLC 0–A HCC without portal hypertension, surgical resection should be performed when anatomically possible b
Documented discussion of LT in patients within overall transplant criteria who are not suitable for curative hepatic resection or ablative therapy	Patients with HCC without extrahepatic disease who are not resection candidates and without absolute contraindications for LT should undergo evaluation for LT b
	Patients with HCC should have LT candidacy documented in the medical record b
Patients with HCC on the waiting list for LT should be monitored for HCC progression a
Transarterial chemoembolization or transarterial radioembolization or other therapies with intent to delay progression and prolong survival is offered to patients with BCLC‐B stage HCC not suitable for curative treatment a
Documented response to HCC‐directed treatment surveillance with contrast‐enhanced CT or MRI every 3–6 months a
Documented that systemic therapy was offered to suitable patients with advanced (BCLC‐C) HCC or unresectable disease not amenable to curative or locoregional therapies with intent to prolong survival	Patients with HCC who are not candidates for resection, LT, or locoregional therapy should be offered systemic therapy b
	Patients with HCC that progresses after locoregional therapy and who are not candidates for resection or LT should be offered systemic therapy b
	Patients with well‐preserved liver function (Child‐Pugh A), good performance status, and BCLC stage C HCC should be offered systemic therapy b
Currently approved first‐line systemic therapy agents was offered to eligible patients with HCC a
Patients with BCLC‐D are offered symptom management in conjunction with supportive care services	Patients with cirrhosis and BCLC stage D HCC who are not candidates for LT should receive palliative support b
Patient referred to palliative care in advanced disease	Advance care planning should be documented in patients with BCLC‐C or D HCC b
	Patients with HCC and symptomatic bone metastases should be offered palliative radiotherapy c
Clinical and radiological (multiphase CT or MRI using standardized criteria) assessment completed to monitor treatment response a
Treatment commences within 4 weeks of decision to treat (from MDT) a
Diagnosis, staging, and treatment planning of a patient with suspected or proven HCC is managed by an MDT	MLTB recommendations should be documented in the medical record b
HCC registry will collect death data	3‐year survival b
	Percent of margin‐negative resections c
	Percent clinical decompensation (ascites, hepatic encephalopathy, spontaneous bacterial peritonitis, jaundice, portal hypertension–related gastrointestinal bleed) within 30 days following locoregional therapy c
	Hospice, length of stay c
	Intensive care unit use in the last 2 weeks of life c
Postperioperative 90‐day mortality of liver resection in patients with cirrhosis should be less than 3% a

Abbreviations: AFT, alpha‐fetoprotein; BCLC, Barcelona Clinic Liver Cancer; CEUS, contrast enhanced ultrasound; CT, computed tomography; HCC, hepatocellular carcinoma; LI‐RADS, Liver Imaging and Reporting Data System; LT, liver transplantation; MDT, multidisciplinary team; MELD, Model for End‐Stage Liver Disease; MLTB, Multidisciplinary Liver Tumor Board; MRI, magnetic resonance imaging; US, ultrasound.

^a Only Maharaj Lubel et al. (n = 13).

^b Comparable between both studies (n = 20).

^c Only Asrani et al. (n = 9).

An important consideration in the choice of QIs in our study was the feasibility of collecting the necessary data. By contrast, similar QIs recommended by Asrani et al.^{[ 26 ]} in their core data set failed the feasibility test in our mDelphi process. For example, documented surveillance by ultrasound at 6‐month intervals was considered. However, participants in our study rated these QIs as highly important but logistically not feasible due to the inability to capture these data given current provisions and resourcing of existing HCC institutional databases.

Our study had several strengths, including an in‐depth literature review using recently published studies and international best practice guidelines and supporting documents that enabled robust discussions to identify highly important and feasible QIs. This was especially useful in providing clarity and contextualizing the QIs to the management of HCC across the country. In addition, participants from varying multidisciplinary clinical backgrounds as well as consumer representatives were engaged in the HCC working group consensus and both rounds of the mDelphi process. The inclusion of the core QI set in a registry setting will allow the implementation of these QIs using structured retrospective and prospective data collection. The implementation and subsequent evaluation of these QIs will draw greater awareness to the opportunity to improve quality of care for patients diagnosed with HCC by facilitating clinician and stakeholder engagement.^{[ 7 ]} This will be achieved by providing risk‐adjusted benchmarked reports to participating hospital sites that will highlight variations in care and clinical outcomes at a health‐service level.^{[ 27 ]} For participating sites to meet the optimal care QIs, all data variables/points will need to be met to comply with the overall QI. However, a minimum data set accompanied by an in‐depth data dictionary will be developed for each QI, allowing the registry to also report on the different components of the indicator.

A limitation of our study was the reduction in participant numbers rating some Qis, especially toward the end of the first session in round 2. However, all participants had the opportunity to provide comments in round 1 that were taken into consideration when discussing each QI in round 2. Further, the intent was to hold the second session in person rather than as an online meeting to maximize the opportunities for discussion, but given the corona virus disease 2019 pandemic and travel restrictions, round 2 was an online virtual meeting. Whether this had an impact on the discussions in comparison to an in‐person meeting is unknown. However, more time was allowed over two sessions to discuss each indicator, and the initial consensus within the HCC working group decreased the burden of discussing a large number of QIs. In addition, facilitators (JL, SR, JZ) and study lead (AM) met on a regular basis before the second round to discuss topics, such as technology, presentation of material, and inclusion of participants, as supported by recent best practice recommendations on ways to maximize a virtual meeting.^{[ 28 ]}

The developed HCC QIs and associated capture of a concise data set will allow quality of HCC care to be assessed. Over time, adherence to these best practice QIs may lead to system‐level improvements in quality of care and, ultimately, patient outcomes, such as survival.

AUTHOR CONTRIBUTIONS

Ashika Maharaj, John Lubel, John Zalcberg, and Stuart Roberts designed the study. Eileen Lam, Elysia Greenhill, and Liane Ioannou provided operational support; John Lubel, Stuart Roberts, and all other authors participated in the voting rounds for the mDelphi. Ashika Maharaj prepared the first draft of the manuscript. All authors interpreted the data, edited and reviewed the manuscript, and gave final approval for submission of the final manuscript.

FUNDING INFORMATION

Astra Zeneca Australia; Eisai Australia; Ipsen. The funding organizations had no role in the study design and development, participation in the modified Delphi, or manuscript review.

CONFLICTS OF INTEREST

Stuart Roberts advises AstraZeneca and Eisai. Katherine Stuart advises Gilead. Paul Clark is on the speakers' bureau of Eisai. Christos Karapetis advises Roche, Astra Zeneca, Eisai, BMS, and MSD. Alexander Thompson has received grants from Gilead and Roche; he advises Assembly Biosciences; he consults for, advises, and is on the speakers' bureau of AbbVie, Antios, Janssen, and Dicerna. David Pryor advises Roche and is on the speakers' bureau of Bayer. The other authors have nothing to report.

ETHICAL APPROVAL

This study was approved by the Monash University Human Research Ethics Committee (MUHREC #29179).

Supporting information

Appendix S1 Summary list of potential HCC clinical quality indicators ‐ References.

Appendix S2 HCC working group review.

Appendix S3 Summary of invitations and participants.

Appendix S4 mDelphi Results ‐ Diagnosis and Staging QIs.

Appendix S5 mDelphi Results ‐ Treatment QIs.

Appendix S6 mDelphi – Outcomes QI.

Maharaj AD, Lubel J, Lam E, Clark PJ, Duncan O, George J, et al. Monitoring quality of care in hepatocellular carcinoma: A modified Delphi consensus. Hepatol Commun. 2022;6:3260–3271. 10.1002/hep4.2089