Responsiveness of quality of life instruments for the comparison of minimally invasive cholecystectomy procedures

Abstract

Background

Traditional metrics of postoperative outcomes (morbidity and mortality) are not useful to compare minimally invasive procedures with each other. Patient reported outcomes, such as quality of life (QOL) scores, offer an alternative approach. Compared with a large body of data in cancer treatment, the responsiveness of these instruments for minimally invasive surgery is not well described. To better define expected differences, we analyzed the reported QOL outcomes in randomized, controlled trials (RCTs) comparing single and four-port laparoscopic cholecystectomy.

Methods

Searching Medline, Embase, Psychinfo, Scopus, and the Cochrane Library (1946 to Jan 2012), two independent reviewers identified RCTs comparing single with four-port cholecystectomy in adult patients using perioperative QOL assessments. The quality of the studies was assessed regarding trial design and QOL reporting. Rev-Man was used for mathematical analysis of the pooled outcome data using a random-effects model. Standardized mean difference estimation was utilized when pooling studies reporting different QOL tools. Statistical heterogeneity was assessed using χ² and I².

Results

Of 743 citations, 37 RCTs were identified. Five studies with a total of 502 patients compared single with four-port cholecystectomy on QOL and were included. Pooled analysis was performed using preoperative and 1-month postoperative outcomes. At 1 month postoperatively, the reported effect size of perioperative QOL changes was up to 5 points (~1/2 SD) on the global SF 12 score. The largest difference in change of perioperative physical functioning was 9.9 points (~1 SD). No difference between the treatments was demonstrated.

Conclusions

Reporting of QOL may improve the comparison of minimally invasive surgical procedures. This systematic review reports clinically important changes and did not demonstrate a difference between treatments at 1 month postoperatively. The optimal timing and trial design for QOL tools in this setting needs to be defined further.

The introduction of laparoscopic gastrointestinal surgery 25 years ago provided a paradigm shift for the burden of treatment suffered by patients requiring surgical approaches for abdominal diseases. Novel technologies for abdominal surgery continue to evolve, including robotics, flexible and semi-rigid instrumentation, and reduced access sites laparoscopy. An example for this is the continued development of approaches to cholecystectomy, including single-port and transvaginal flexible endoscopic procedures.

Morbidity and mortality are very low and many procedures are performed on an outpatient basis; thus, these traditional metrics of postoperative outcomes are not feasible to compare minimally invasive procedures with each other.

Patient reported outcomes, such as pain and quality of life (QOL) offer an alternative approach for the comparison of these techniques. Many quality of life tools were established to investigate and manage chronic diseases, such as asthma or cancer. Calibration of the QOL instruments by way of identifying a clinically significant, minimally important difference is a critical psychometric characteristic. Minimally important differences in chronic disease settings have been well described. The responsiveness of QOL tools in the perioperative period is less well known.

Guidelines for use of QOL instruments to compare laparoscopic surgery with open surgery were published in 2004 as the result of a consensus conference convened by the European Association for Endoscopic Surgery (EAES) [1]. The group assessed a multitude of surgical conditions (colon cancer, reflux, hernia, gallstone disease) and reviewed the use of validated and ad hoc instruments in two randomized and eight nonrandomized trials, comparing laparoscopic to open surgery. Recommendations were given to use the SF-36 or the Psychological General Well Being Index (PGWB) in conjunction with the Gastrointestinal Quality of Life index (GIQLI) or the GIQLI alone for the evaluation of gallbladder disease based on the validity and change of the overall scores compared to baseline (responsiveness). Minimally important differences for the comparison of open versus laparoscopic surgical procedures were not included in the consensus recommendations.

Additional minimally invasive methods for cholecystectomy have been developed since the consensus guidelines were published. Examples are Natural Orifice Translumenal Endoscopic Surgery (NOTES), single-port surgery (also called LESS, SILS, and other acronyms), and needle-port laparoscopy using 2-mm incisions. In addition, the assessment of health-related QOL has undergone further development with the introduction of shorter questionnaires and an interest in single-item domains. We wish to investigate the use and responsiveness of QOL tools for the comparison of two minimally invasive cholecystectomy procedures with each other, including the assessment of the overall score and specific domains in the perioperative period.

Review question

What is the reported effect size of quality of life instruments for the comparison of minimally invasive cholecystectomy procedures?

Methods

With the assistance of an experienced librarian, the subject headings cholecystectomy, gallbladder, gallstone disease, quality of life, QOL, HRQOL, health status, well-being, activities of daily living, social functioning, fatigue, recovery of function, return to activity, postoperative pain, pain measurement, body image, beauty, esthetics, cicatrix, port, patient satisfaction, laparoscopy, SF-36, SF 12, GI-QLI, ASIS, and PGWB were used to search Medline (1946 to Jan Week 2, 2012), Embase (1988 to Week 3, 2012), PsychINFO, Scopus, and the Cochrane Library. In addition, we reviewed the reference sections of eligible studies and requested potentially eligible studies from content experts.

Studies were eligible for inclusion if the design was a randomized, controlled trial of adult patients undergoing single-port laparoscopic cholecystectomy versus four-port laparoscopic cholecystectomy using quality of life as a postoperative outcome. All languages were included.

Studies investigating other operative procedures or other study designs were excluded, as were studies of pediatric patients. Studies also were excluded if the patient reported outcomes included only pain as measured by the visual analog scale (VAS), cosmesis, or satisfaction scores without a validated QOL tool.

Two independent reviewers (a surgeon and a quality of life researcher) considered the potential eligibility of each of the abstracts and titles that resulted from executing the search strategy. Reviewers requested the full text versions of all potentially eligible studies, including disagreements. Two reviewers working independently and blindly considered the full text reports (all available versions of each study, including abstracts) for eligibility. The reviewers calibrated their judgments using a smaller set of reports (n = 20). Subsequently, disagreements were harmonized by consensus; agreement was measured using the kappa statistics.

Data extraction included full description of participants enrolled, the interventions they received (single port cholecystectomy), the control interventions (standard cholecystectomy), and the measure of outcome (pain scale, quality of life tools). The outcomes of interest were: pain (VAS) scores, quality of life scores, total score and single-item responses where available, along with the time points for the ascertainment of this outcome.

Quality

To assess the methodological quality of the randomized trials, we determined how the randomization sequence was generated, how allocation was concealed, whether there were important imbalances at baseline; which groups were blinded (patients, caregivers, data collectors, outcome assessors, data analysts); what was the loss to follow-up; whether the analysis was by intention to treat and how missing outcome data was dealt with. A Jadad score [2] was determined for each study and the risk of bias was assessed using the Cochrane Collaboration risk assessment tool [3]. In addition, we evaluated the quality of the health-related quality of life (HRQOL) reporting based on the checklist for minimum standards for evaluation of HRQOL outcomes as proposed by Efficace [4].

Pooling

When possible, we generated meta-analytic estimates of treatment effect for the postoperative outcome variables pain (VAS), bodily pain (QOL), physical functioning, vitality/fatigue, and emotional functioning. Additional analyses were performed for the change from baseline. We used the random-effects meta-analyses to test the hypothesis that the treatment effect was significantly different from zero. We measured inconsistency for each outcome by estimating the I² test and its associated confidence interval [5]. We used RevMan and StatsDirect software to conduct the analyses. Standard mean difference is reported for outcomes that were measured with difficult metrics (SF-36 vs. other QOL). Weighted mean difference is reported for outcomes with the same metric (e.g., VAS).

Reporting bias

We contacted authors with a summary form of their data for verification and completion of missing data if possible. The protocol for author contact included email or personal conversation to the corresponding author with statement of purpose, data collected and missing, and opportunity to email or fax back the results.

Results

A query of the Cochrane database of systematic reviews did not identify any prior systematic reviews or meta-analysis for the use of QOL tools with cholecystectomy or minimally invasive cholecystectomy. A January 2012 search of MEDLINE (1946 to January Week 2, 2012), Embase (1988 to 2012 Week 3), EBM Reviews-Cochrane Central Register of Controlled Trials (January 2012), PsycINFO (1987 to January Week 3 2012), and Scopus resulted in 743 abstracts following the search strategy outlined in the methods section.

After independent review by two blinded reviewers, 143 abstracts were considered eligible for full text review, because they included both cholecystectomy and terms used to elicit patient reported outcomes relating to quality of life outcomes (Fig. 1). Of these, 39 papers describing randomized controlled trials were identified. Thirty reports compared either laparoscopic cholecystectomy with open cholecystectomy or different interventions during laparoscopic cholecystectomy (inpatient vs. outpatient, medication for nausea, pain medication regimen). Eight trials (nine references) [6–14] compared minimally invasive cholecystectomy methods to each other. One compared needle trocar access with standard cholecystectomy [6], one compared high versus low pressure cholecystectomy [7], and seven reports described the comparison of single-port versus four-port cholecystectomy [8–14]. One of those papers [8] contained the description of the trial design only, leaving six publications reporting on five trials of interest. The kappa for inclusion of these trials for the independent reviewers was 0.65 [15]. No additional studies were encountered upon review of the reference sections of eligible studies, available reviews, or from content experts.

Fig. 1.

Search strategy and selection process. PRO patient-reported outcomes, RCT randomized controlled trial, QOL quality of life

The five randomized, controlled trials included in the meta-analysis reported outcomes of 502 patients (Table 1). All were initially published in 2011; an additional report of one trial was published in May of 2012. The studies varied in size (n = 40 vs. n = 200), design (single-center vs. multicenter), and blinding (unblinded vs. double blinded). All studies were of moderate to low study design quality (Jadad score 1–3) [2]. One report included a consort flow diagram and a statement about patients included in the analysis [13]. One study reported the overall number of patients who received treatment after randomization and the percentage of patients available for follow-up at each time point [10]. The remaining studies did not report on withdrawals, dropouts, or missing data. Assessment of bias is depicted in Table 2.

Table 1.

Included study description

Author	Years	Setting	Total	Quality of life tools	Source
Bucher	2011	Switzerland, single institution	150	SF 12	Manuscript
Leung	2011	Chicago, 3 hospitals	79	Surgical outcomes measurement system	Abstract, unpublished manuscript
Lirci	2011	Italy 2 hospitals	40	SF 36	Pilot, dual publication
Ma	2011	Portland, single institution	44	SF 36	Manuscript
Marks/Phillips	2011/2012	Multicenter (9 centers, US, Italy, UK) Industry sponsor	200	SF 8, SF 12	Initial and interim manuscripts

Table 2.

Assessment of bias

Issue (yes/no)	Bucher	Lirici	Ma	Marks/Phillips	Leung
Concealment	NR	Sealed envelopes	Computerized algorithm	NR	NR
Missing data	NR	NR	NR	NR	NR
Baseline difference	No difference	NR for QOL	Age, no difference in global QOL but in PF	No difference	NR
ITT	No, treatment received	Yes	NR	NR	NR

NR not reported, ITT intention to treat, PF physical functioning domain, QOL quality of life

Quality of life as measured by a validated QOL instrument was a secondary outcome in all included trials. One trial defined QOL as a composite of length of stay (LOS), pain score, and QOL scores using the composite measure as primary outcome. Perioperative quality of life was measured with short-form (SF) 36, 12, or 8 in four of the five trials; one used the “surgical outcomes measurement system” [14]. We used two tools from the cancer literature to assess the appropriateness of the quality of life reporting in the trials [4, 16] as detailed in Table 3. For these tools, the presence of 8 of 11 items in the reporting was deemed to denote a high-quality report. Each trial utilized a previously validated QOL instrument; one trial discussed whether the chosen tool was appropriate for the setting. None of the studies reported if the 4-week (chronic) or 1-week recall (acute) version of the SF-36 based tools was administered. No study reported how the QOL tools were administered, if they were self-reported by a (blinded) patient or elicited by a blinded or unblinded evaluator. In view of the listed data (ranging from score 41–89), we assume that raw domain scores rather than norm-based QOL scores (general population mean ~50) [17] were reported in the studies utilizing the SF-36–based tools. Three of the five studies measured preoperative and postoperative outcomes. Postoperative outcome assessment varied between 3 days to 1 year. The most frequently reported assessment time point was at approximately 1 month postoperatively and served as comparison point for this analysis. One trial [13] reported on individual patient changes in quality of life, and one trial reported on the proportion of patients with clinically significantly pain outcomes [12], assessments considered important for palliative care trials [16]. The largest study (n = 200) reported a physical functioning summary score; unfortunately, scores for mental functioning or single-domain scores were not provided. Mental functioning scores were listed as “not different” [10, 11].

Table 3.

Check list for minimum standards for evaluation of HRQOL outcomes

Issue (yes/no)	Bucher	Lirici	Ma	Marks/Phillips	Leung
Quality of life related a priori-hypothesis stated	No	Yes	No	No	No
Rationale for instrument reported	No	No	Yes	No	No
Psychometric properties reported	No	No	No	No	No
Cultural validity verified	Yes	Yes	Yes	Yes	Yes
Adequacy of domains covered	No	No	No	No	No
Instrument administration reported (how)	No	No	No	No	No
Baseline compliance reported	Yes	No	No	No	No
Timing of assessments documented	Yes	Yes	Yes	Yes	Yes
Missing data documented	No	No	No	No	No
Clinical significance addressed	No	No*	No	No	No
Presentation of results in general	Yes	Yes	Yes	Yes	Yes

The three components regarded as centrally important in the quality of life reporting are noted in italics

HRQOL health-related quality of life

^*Clinical significance of measuring quality of life was addressed, not however the clinical significance of the findings

Because only five trials were available for analysis, the quality assessment did not lead to the exclusion of any study. Pooled analysis was performed using preoperative and 1-month postoperative outcomes using a random effects model. Clinical pain outcomes (VAS) were analyzed for comparison with other studies and systematic reviews. Preoperative pain scores were reported in one study. The QOL summary score was reported in one study. Four QOL domains were reported in more than one study at similar postoperative time points: bodily pain, vitality/fatigue, physical functioning, and emotional functioning. Two studies provided data for preoperative and postoperative assessment of vitality/fatigue, physical functioning and bodily pain. To allow for computation, standard deviations for incompletely reported data had to be estimated from the range and from other published resources. No statistically significant difference between the treatments was demonstrated (Figs. 2, 3, 4, 5, and 6). Significant statistical heterogeneity between the trials was demonstrated for the outcomes postoperative pain within 24 h after surgery (VAS), bodily pain, and the perioperative change in physical functioning (Figs. 2, 3, and 7). The inconsistency between the trials was largely explained by this heterogeneity as expressed in the I². One could postulate that the pain and physical function outcomes are closely related and therefore may exhibit similar properties. The heterogeneity can be explained by the significant clinical differences between the trials: there are multi-institution and single-institution studies included, one study spanning three different countries with different health care systems. The research methodology differed in blinded or unblinded studies, and possibly regarding missing data, although that is not reported. In addition, the different trials used different instrumentation (different single-port devices, requiring different incision sizes). They also may have used different pain regimen, and the data were not collected at the identical time interval from surgery.

Fig. 2.

Forrest plot and data table of the outcome postoperative pain as measured by visual analog scale (VAS) on postoperative day 1. Weighted mean difference

Fig. 3.

Forrest plot and data table of the outcome physical functioning, 1 month postoperative. Standardized (STD) mean difference

Fig. 4.

Forrest plot and data table of the outcome postoperative vitality, 1 month postoperative. STD mean difference

Fig. 5.

Forrest plot and data table of the outcome bodily pain, 1 month postoperative. STD mean difference

Fig. 6.

Forrest plot and data table of the outcome emotional functioning, 1 month postoperative. Weighted mean difference

Fig. 7.

Forrest plot of the physical functioning from preoperative to 1 month postoperative. STD mean difference

At 1 month, the reported effect size of perioperative QOL changes was up to 5 points on the global SF 12 score in one trial [13] with a difference of global scores between surgical treatment arms of 3 points. One trial recorded an 18-point difference in physical functioning summary score (~1.8 × standard deviations) from preoperative to postoperative day 1; however, at 1 month the difference was 0.5 points.

The change from baseline for bodily pain and vitality/fatigue from preoperative to postoperative (1 month) was reported as 1 point (~0.1 × standard deviation). The largest reported change from baseline was −6.5 (~2/3 × standard deviation) in the physical function domain scale (Fig. 7), and the difference in change between the treatment arms was 9.9 (~1 standard deviation) [12]. This has not been replicated so far and its clinical significance is uncertain.

Discussion

This review describes the considerable interest of the surgical community in patient-reported outcomes in the perioperative period, with more than 700 citations encountered. The interest in comparing novel minimally invasive procedures using quality of life tools is evidenced by five randomized, controlled trials published in 1 year using these outcomes. In our meta-analysis, no significant difference between treatment options at the time points measured was encountered. The confidence in the result of the analysis is tempered by a number of limitations in the trials that contributed data, and the missingness thereof. The trials exhibited significant clinical and some statistical heterogeneity. The overall number of patients is small and so is the number of studies. To allow for computation, standard deviations for incompletely reported data had to be estimated from the range and from other published resources. Fortunately, in the case of SF-36 based tools, this is quite reasonable to do and well validated [17]. The multitude of studies reporting on VAS also permits for estimation of standard deviations. The overall QOL reporting in the included manuscripts suggests that the experience with QOL tools in the perioperative period is still limited compared with cancer or asthma research. The reporting of QOL data in these trials could certainly be improved. Incorporating recommended reporting guidelines will facilitate comparison across studies. In addition, the optimal timing of QOL tool administration relative to the events of interest (MIS surgery) requires further delineation. A number of studies comparing open and laparoscopic surgery described that up to 2 weeks but not at 1 month postoperatively any differences could be found. Other differences were encountered much later, at the 4–12 months’ timeframe [1, 18].

The main purpose of our study was to evaluate the responsiveness of the QOL tools in the perioperative period. Given the limitations of the studies noted above, this was best accomplished by measuring the change from baseline within each group. Although pooling was again limited due to variable data collection and reporting, measurable differences of up to 1.8 standard deviations were encountered. This supports that QOL tools are responsive enough that they can be used to measure statistically and clinically significant differences in the perioperative period for minimally invasive surgery. Consideration of effect size gives initial guidance for future study design. When procedures are very similar and the expected effect size is small, a study will have to be appropriately powered to detect this. Improved data reporting will enable the minimally invasive surgical community to use QOL tools more effectively in the design of future studies.