Skip to main content
Medicine logoLink to Medicine
. 2017 Sep 1;96(35):e7633. doi: 10.1097/MD.0000000000007633

Robotic versus thoracoscopic lung resection

A systematic review and meta-analysis

Alexander Emmert a,, Carmen Straube b, Judith Buentzel b, Christian Roever c
Editor: Monica Casiraghi
PMCID: PMC5585477  PMID: 28858083

Abstract

Background:

Robotic video-assisted surgery (RVATS) has been reported to be equally effective to video-assisted surgery (VATS) in lung resection (pneumonectomy, lobectomy, and segmentectomy). Operation time, mortality, drainage duration, and length of hospitalization of patients undergoing either RVATS or VATS are compared in this meta-analysis.

Methods:

A systematic research for articles meeting our inclusion criteria was performed using the PubMed database. Articles published from January 2011 to January 2016 were included. We used results of reported mortality, operation time, drainage duration, and hospitalization length for performing this meta-analysis. Mean difference and logarithmic odds ratio were used as summary statistics.

Results:

Ten studies eligible were included into this analysis (5 studies for operation time, 3 studies for chest in tube days, 4 studies for length of hospitalization, and 6 studies for mortality). We were able to include 3375 subjects for RVATS and 58,683 subjects for VATS. Patients were mainly treated for lung cancer, metastatic foci, and benign lesions. We could not detect any difference between operation time; however, we found 2 trends showing that drainage duration and length of hospitalization are shorter for following RVATS than for following VATS. Mortality also is lower in patients undergoing RVATS.

Conclusions:

Therefore, we conclude that RVATS is a suitable minimal-invasive procedure for lung resection and suitable alternative to VATS. RVATS is as time-efficient as VATS and shows a trend to reduced hospital stay and drainage duration. More and better studies are required to provide reliable, unbiased evidence regarding the relative benefits of both methods.

Keywords: lung cancer, robot-assisted minimally invasive surgery, video-assisted minimally invasive surgery

1. Introduction

Surgery is a pre-requisite for successful cancer management, both for diagnostics and treatment.[1,2] During the last years, minimal-invasive surgery procedures such as video-assisted thoracic surgery (VATS) or robot video-assisted thoracic surgery (RVATS) have become increasingly refined and are meanwhile commonly used for lung resection instead of an open thoracotomy approach.[3]

Patients undergoing VATS suffer from fewer complications, have less pain and blood loss, and recover faster than patients subjected to open thoracotomy.[4,5] Furthermore, VATS lobectomy is associated with shorter chest tube duration, hospitalization, lower morbidity, and improved survival.[6]

The da Vinci robotic surgical (RVATS-system) has been established in several different disciplines and has found application in urologic, gynecologic, and rectal surgery. It appears to be especially advantageous of surgery of deep and narrow spaces such as the pelvis or the mediastinum.[7] The da Vinci system was introduced to thoracic surgery as RVATS.[8] It offers several technical advantages such as 3-dimensional high-definition field of view, tremor filtration, augmented dexterity, or the capability of tele-surgery.[9] The application of RVATS underwent various improvements and upgrades since the first case-series report in 2002, whereas different techniques have been described and developed for performing robotic lobectomy.[1012] Patients treated with a robotic approach show a lower morbidity and mortality than patients undergoing open thoracotomy.[13]

Both VATS and RVATS are superior to open thoracotomy in terms of survival, morbidity, and mortality.[2,4,6,13] Both approaches were recently compared by Ye et al[14], whose meta-analysis mainly focuses on morbidity and mortality. We additionally included parameters such as operating time, hospitalization, and drainage duration. Since Ye et al published their meta-analysis, 2 more comparative studies have been published, showing the issue to be topical.[15,16] We included several new studies[1719] in addition to those by Ye et al.

2. Methods

2.1. Literature review and data extraction

A systematic literature review was performed by searching PubMed on 26 January 2016, using the search terms ([“surgery” OR “resection” OR “lobectomy”] AND [“thoracic” OR “thoracoscopic” OR “lung” OR “pulmonary”] AND [“robotic” OR “robot assisted” OR “da Vinci” OR “daVinci”]). No language restriction and no filters were applied. A total of 990 records were identified by the search. Only data of already published studies found through online research were used for meta-analysis, and we did not require the approval of the local ethics committee. Ten studies were selected for meta-analysis (listed in Table 1), all reporting lung resection (pneumonectomy, lobectomy, and segmentectomy) for either malign (lung cancer and metastatic foci) or benign lesions. Inclusion criteria were reporting of operation time, length of hospitalization, data on drainage duration and mortality. Exclusion criteria were: (i) data not suitable for statistical analysis methods used for our analysis, (ii) reviews.

Table 1.

Study characteristics (VATS/RVATS).

2.1.

Two people independently extracted data on number of cases, age and gender of patients, operation time, length of hospitalization, drainage duration, and mortality. Two studies used the same historical data for comparison. If not explicitly quoted, mean differences and P-values (based on t-tests) were used to obtain standard errors.[17,18,26]

2.2. Statistical analysis

Random-effects models were used to combine data from different studies.[27,28] For continuous endpoints (operating time, hospitalization duration, and drainage duration), effect estimates and their standard errors were used, and mortality effects were compared by considering logarithmic odds ratios. In the case of zero counts in a contingency table, a value of 0.5 was added to all cells.[20] Heterogeneity between studies was estimated using the restricted maximum likelihood (REML) method, and combined effect estimates and associated confidence intervals were derived using the modified Knapp-Hartung approach.[29] Correlations between effect estimates due to the use of common data were accounted for by considering their covariance in the analysis. Computations were performed using R and the metafor package.[30,31]

3. Results

3.1. Study characteristics

Nine retrospective, observational studies and 1 prospective cohort study published between 2011 and January 2016 were included in this analysis (see also Fig. 1). Study characteristics are listed in Table 1. A total of 3758 patients undergoing RVATS were compared with 58,677 patients experiencing VATS. Mean age of patients varied between 61 and 71 years for RVATS and 57 and 77 years for VATS. The number of patients included into these studies ranged from 17 to 2498 for RVATS and from 28 to 37,595 for VATS. If specified usually one surgeon or in the case of Adams et al 6 surgeons treated patients. Furthermore, Adams et al, Jang et al, Lee et al reported that the cases of RVATS published in these studies were first case series while establishing RVATS as the new operation method. Patients were treated for lung cancer, metastatic foci, and benign lesions (Table 3). The mortality endpoint definitions of studies included varied (refer also to Table 2). Although operation time, length of hospitalization, and chest were analyzed by these studies, we found no study reporting and evaluating pain or quality of life. Indication for operating patients was lung cancer, metastatic foci, or benign lesions. Only 2 studies[20,22] reported the number of lymph nodes removed and the number of lymph node stations dissected for both RVATS and VATS. Overall, the number of lymph nodes removed and lymph node stations dissected was similar (please refer to Table 3).

Figure 1.

Figure 1

Flow diagram of papers screened and included.

Table 3.

Entities, tumor stage, and number of lymph nodes removed and lymph node stations dissected.

3.1.

Table 2.

Mortality data and endpoint definitions.

3.1.

Two studies utilized the same cohort of VATS patients (from a national database),[17,18] which common underlying data induces a positive correlation between the resulting estimates. This was accordingly accounted for which can be derived based on the group-specific standard errors.

3.2. Operation time

Six studies reported data on durations of surgery. The estimated mean differences in operating time are shown in Fig. 2. For RVATS, there are inconsistently reported longer operation times as well as shorter operation times. The combined effect estimate is at +8.97 minutes (95% confidence interval [−28.12,+46.07]), indicating a slightly longer duration for RVATS. But, it is not significantly different from zero (P = .56). The corresponding estimate of the between-study heterogeneity is at τ=34.7.

Figure 2.

Figure 2

Duration of surgery (mean difference).

3.3. Duration of hospitalization

Although we could not observe a significant difference in duration of hospital stay between the RVATS and the VATS group, at least a trend of shorter hospitalization became apparent in patients undergoing RVATS in the 6 studies analyzed. Figure 3 shows the data along with the combined estimate. The estimated difference in hospitalization time is at –1.08 days (95% CI [–2.33,+0.17], P = .078) for RVATS. The between-study heterogeneity is estimated as τ=1.06.

Figure 3.

Figure 3

Duration of hospitalization (mean difference).

3.4. Drainage duration

A trend to shortened drainage duration was reported in patients experiencing RVATS compared to VATS in all 3 studies (see Fig. 4). The combined estimate is at an average of –0.71 days (95% CI [−1.50,+0.10], P = .064) for RVATS. The between-study heterogeneity τ is estimated as zero.

Figure 4.

Figure 4

Drainage duration (mean difference).

3.5. Mortality

Table 2 lists the data on mortality along with the corresponding mortality endpoint definitions. Six studies reported on mortality; 5 of these show fewer deaths in the RVATS group than for VATS, whereas 1 did not observe any deaths in either group. The effect estimates and the combined estimate are illustrated in Fig. 5. The combined effect on the odds ratio scale is 0.52 (95% CI [0.29, 0.92]), that is, an estimated almost 2-fold decrease in mortality. Despite the concerns regarding the comparability of estimates from different studies due to differing endpoint definitions, the between-study heterogeneity (τ) here is estimated at zero. Note that although the direction of effect is consistent between studies, the joint estimate is to some extent driven by the large study of Paul et al.[19]

Figure 5.

Figure 5

Mortality (odds ratio).

4. Discussion

Increasing evidence suggests that perioperative outcomes of minimally invasive thoracic surgery are better than those of conventional open thoracotomy. The overall incidence of complications such as arrhythmia, pneumonia, pain, and inflammatory markers was reduced in several previous studies.[3235]

VATS lobectomy has been associated with highly satisfactory results and has become the most exciting technical development in thoracic surgery over the past 5 years. Compared with open lobectomy, VATS lobectomy appears to have improved long-term outcomes and is supported by evidence-based treatment guidelines.[32,36,37]

RVATS lobectomy or segmentectomy is not, at this time, widely performed because of its technical difficulty. Furthermore, the availability of the DaVinci system is still limited due to the substantial acquisition and running costs.[38] Nonetheless, robotic pulmonary resections prove to be safe and effective even at the initial learning experience. The duration of operations is considered to be acceptable. The effect of a steep learning curve for RVATS lobectomy has been well documented.[39,40]

Evolving operation procedures require continuous assessment if these new methods are equal or even superior to standard operation techniques. So far, research focused on learning curves and costs of RVATS compared to other minimal invasive techniques.[12,23,41] Although the procedures of RVATS and open thoracotomy in pulmonary surgery have been compared in systematic reviews and meta-analyses previously, only 1 publication yet investigated the outcome of patients undergoing either RVATS or VATS.[13] The authors concluded that there was no difference in mortality and morbidity between the 2 minimally invasive techniques.[14] However, there are also several other parameters that may influence the choice of the operation technique applied: operation time, drainage duration, and length of hospitalization were additionally investigated in our meta-analysis. We included more recent studies and 4 studies not part of the formerly published meta-analysis by Ye et al. The size of the studies was variable; we included 62,435 patients altogether, of whom 3758 underwent RVATS and 58,677 VATS.[14] The size of the studies included was variable. We could not detect a difference in operation duration; therefore, we conclude that RVATS is about as efficient as VATS. RVATS showed a tendency toward shorter hospitalization time and drainage duration compared to VATS of 1 and 0.7 days, respectively, which are clinically relevant effects. Interestingly, we observed a 2-fold decrease of mortality in patients undergoing RVATS, which was not detected by Ye et al in 2015. However, this result should be interpreted with caution, as different studies used differing endpoint definitions (see also Table 2). The effect's direction was consensual over all studies. There are several limitations to our study. First, as previously observed, any meta-analysis of observational studies is affected by the same biases present in the original studies contained in it.[42,43] The studies included in this publication were not randomized, but either retrospective observational (9) or prospective studies (1). It is in the nature of the current problem that options for randomization and especially for blinding are very limited. However, the quality of observational studies may be greatly improved by careful design; the potential gain in validity goes beyond a mere increase in the sample size.[44] For example, among the studies investigated here, efforts made to enhance comparability of treatment groups included comparing initial patients for both procedures[20] or the use of propensity matching.[21,24] Second, various operative factors related to the procedure itself, such as surgical instruments (e.g., no distinction between the generations of the DaVinci system used was made), sutures, and drugs, may have influenced the results. Furthermore, the surgeon's experience might influence the operative outcome. Some of the studies included specified, that only 1 surgeon operated patients undergoing RVATS[20,22,23] or that the cases of RVATS reported in these studies were the first series using this new technique.[17,18,20,22] Tumor entity and staging may influence which technique is chosen for surgery and included into the studies we analyzed (an overview of staging and entity can be found under Table 3). None of the studies included into this meta-analysis reported data on pain or quality of life as end-points. Both would be interesting markers for patient outcome and should be considered when designing new studies. However, we could show that outcome of patients undergoing RVATS is not worse than those undergoing VATS in the investigated endpoints. The costs for pulmonary lobectomy by RVATS are still higher than those of VATS,[20] but our finding of shorter hospitalization time in favor of RVATS should be economically counterbalanced in further considerations.

Summing our results up, we conclude that RVATS lobectomy is a suitable surgical procedure in pulmonary surgery with a potential to prove beneficial to patients even when compared to VATS lobectomy.

From our result we are able to conclude that RVATS is suitable for thoracic surgery. However, future clinical research is needed to investigate suitable indications and contraindications of RVATS lung resection to institutionalize training programs to standardize the systems, and to reduce procedure related costs and limitations to widen its area of application. By improving and implementing robotic techniques during routine clinical practice, we believe that in the near future RVATS will become a standard procedure when applying minimally invasive surgical techniques. However, more well-designed studies are required to provide reliable and less biased evidence regarding the relative benefits of both RVATS and VATS.

Footnotes

Abbreviations: RVATS = robot-assisted minimally invasive surgery, VATS = video- assisted minimally invasive surgery.

JB and CR contributed equally to this study.

Funding: The authors acknowledge support by the German Research Foundation and the Open Access Publicsation Funds of Göttingen University.

The authors have no conflicts of interest to disclose.

References

  • [1].Ye B, Tantai J-C, Li W, et al. Video-assisted thoracoscopic surgery versus robotic-assisted thoracoscopic surgery in the surgical treatment of Masaoka stage I thymoma. World J Surg Oncol 2013;11:157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [2].Sullivan R, Alatise OI, Anderson BO, et al. Global cancer surgery: delivering safe, affordable, and timely cancer surgery. Lancet Oncol 2015;16:1193–224. [DOI] [PubMed] [Google Scholar]
  • [3].Park BJ, Heerdt PM. Minimally invasive surgical techniques in the treatment of lung cancer. Minerva Chir 2009;64:573–88. [PubMed] [Google Scholar]
  • [4].Falcoz P-E, Puyraveau M, Thomas P-A, et al. Video-assisted thoracoscopic surgery versus open lobectomy for primary non-small-cell lung cancer: a propensity-matched analysis of outcome from the European Society of Thoracic Surgeon database. Eur J Cardiothorac Surg 2016;49:602–9. [DOI] [PubMed] [Google Scholar]
  • [5].Higuchi M, Yaginuma H, Yonechi A, et al. Long-term outcomes after video-assisted thoracic surgery (VATS) lobectomy versus lobectomy via open thoracotomy for clinical stage IA non-small cell lung cancer. J Cardiothorac Surg 2014;9:88. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [6].Whitson BA, Groth SS, Duval SJ, et al. Surgery for early-stage non-small cell lung cancer: a systematic review of the video-assisted thoracoscopic surgery versus thoracotomy approaches to lobectomy. Ann Thorac Surg 2008;86:2008–18. [DOI] [PubMed] [Google Scholar]
  • [7].Lee SH, Lim S, Kim JH, et al. Robotic versus conventional laparoscopic surgery for rectal cancer: systematic review and meta-analysis. Ann Surg Treat Res 2015;89:190–201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [8].Wei B, D’Amico TA. Thoracoscopic versus robotic approaches: advantages and disadvantages. Thorac Surg Clin 2014;24:177–88. vi. [DOI] [PubMed] [Google Scholar]
  • [9].Xu H, Li J, Sun Y, et al. Robotic versus laparoscopic right colectomy: a meta-analysis. World J Surg Oncol 2014;12:274. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [10].Melfi FMA, Fanucchi O, Davini F, et al. Robotic lobectomy for lung cancer: evolution in technique and technology. Eur J Cardiothorac Surg 2014;46:626–30. discussion 630–631. [DOI] [PubMed] [Google Scholar]
  • [11].Melfi FMA, Menconi GF, Mariani AM, et al. Early experience with robotic technology for thoracoscopic surgery. Eur J Cardiothorac Surg 2002;21:864–8. [DOI] [PubMed] [Google Scholar]
  • [12].Cao C, Manganas C, Ang SC, et al. A systematic review and meta-analysis on pulmonary resections by robotic video-assisted thoracic surgery. Ann Cardiothorac Surg 2012;1:3–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [13].Zhang L, Gao S. Robot-assisted thoracic surgery versus open thoracic surgery for lung cancer: a system review and meta-analysis. Int J Clin Exp Med 2015;8:17804–10. [PMC free article] [PubMed] [Google Scholar]
  • [14].Ye X, Xie L, Chen G, et al. Robotic thoracic surgery versus video-assisted thoracic surgery for lung cancer: a meta-analysis. Interact Cardiovasc Thorac Surg 2015;21:409–14. [DOI] [PubMed] [Google Scholar]
  • [15].Rinieri P, Peillon C, Salaün M, et al. Perioperative outcomes of video- and robot-assisted segmentectomies. Asian Cardiovasc Thorac Ann 2016;24:145–51. [DOI] [PubMed] [Google Scholar]
  • [16].Mahieu J, Rinieri P, Bubenheim M, et al. Robot-assisted thoracoscopic surgery versus video-assisted thoracoscopic surgery for lung lobectomy: can a robotic approach improve short-term outcomes and operative safety? Thorac Cardiovasc Surg 2015;64:354–62. [DOI] [PubMed] [Google Scholar]
  • [17].Farivar AS, Cerfolio RJ, Vallières E, et al. Comparing robotic lung resection with thoracotomy and video-assisted thoracoscopic surgery cases entered into the Society of Thoracic Surgeons database. Innovations (Phila) 2014;9:10–5. [DOI] [PubMed] [Google Scholar]
  • [18].Adams RD, Bolton WD, Stephenson JE, et al. Initial multicenter community robotic lobectomy experience: comparisons to a national database. Ann Thorac Surg 2014;97:1893–8. discussion 1899–1900. [DOI] [PubMed] [Google Scholar]
  • [19].Paul S, Jalbert J, Isaacs AJ, et al. Comparative effectiveness of robotic-assisted vs thoracoscopic lobectomy. Chest 2014;146:1505–12. [DOI] [PubMed] [Google Scholar]
  • [20].Jang H-J, Lee H-S, Park SY, et al. Comparison of the early robot-assisted lobectomy experience to video-assisted thoracic surgery lobectomy for lung cancer: a single-institution case series matching study. Innovations (Phila) 2011;6:305–10. [DOI] [PubMed] [Google Scholar]
  • [21].Kent M, Wang T, Whyte R, et al. Open, video-assisted thoracic surgery, and robotic lobectomy: review of a national database. Ann Thorac Surg 2014;97:236–42. discussion 242–244. [DOI] [PubMed] [Google Scholar]
  • [22].Lee BE, Korst RJ, Kletsman E, et al. Transitioning from video-assisted thoracic surgical lobectomy to robotics for lung cancer: are there outcomes advantages? J Thorac Cardiovasc Surg 2014;147:724–9. [DOI] [PubMed] [Google Scholar]
  • [23].Deen SA, Wilson JL, Wilshire CL, et al. Defining the cost of care for lobectomy and segmentectomy: a comparison of open, video-assisted thoracoscopic, and robotic approaches. Ann Thorac Surg 2014;97:1000–7. [DOI] [PubMed] [Google Scholar]
  • [24].Swanson SJ, Miller DL, McKenna RJ, et al. Comparing robot-assisted thoracic surgical lobectomy with conventional video-assisted thoracic surgical lobectomy and wedge resection: results from a multihospital database (Premier). J Thorac Cardiovasc Surg 2014;147:929–37. [DOI] [PubMed] [Google Scholar]
  • [25].Demir A, Ayalp K, Ozkan B, et al. Robotic and video-assisted thoracic surgery lung segmentectomy for malignant and benign lesions. Interact Cardiovasc Thorac Surg 2015;20:304–9. [DOI] [PubMed] [Google Scholar]
  • [26].Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions [Internet]. [Cited April 28, 2016]. Available at: http://handbook.cochrane.org/.
  • [27].Academic Press, Hedges LV, Olkin I. Statistical Methods for Meta-Analysis. 2014;369. [Google Scholar]
  • [28].John Wiley & Sons, Inc., Hartung J, Knapp G, Sinha BK. Bayesian meta-analysis [Internet]. Statistical Meta-Analysis with Applications 2008. [Google Scholar]
  • [29].Röver C, Knapp G, Friede T. Hartung-Knapp-Sidik-Jonkman approach and its modification for random-effects meta-analysis with few studies. BMC Med Res Methodol 2015;15:99. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [30].R: The R Project for Statistical Computing [Internet]. [Cited April 28, 2016]. Available at: https://www.r-project.org/.
  • [31].Conducting Meta-Analyses in R with the Metafor Package | Viechtbauer | Journal of Statistical Software [Internet]. [cited April 28, 2016]. Available at: https://www.jstatsoft.org/article/view/v036i03.
  • [32].Yang C, Mo L, Ma Y, et al. A comparative analysis of lung cancer patients treated with lobectomy via three-dimensional video-assisted thoracoscopic surgery versus two-dimensional resection. J Thorac Dis 2015;7:1798–805. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [33].Yim AP, Wan S, Lee TW, et al. VATS lobectomy reduces cytokine responses compared with conventional surgery. Ann Thorac Surg 2000;70:243–7. [DOI] [PubMed] [Google Scholar]
  • [34].Muraoka M, Oka T, Akamine S, et al. Video-assisted thoracic surgery lobectomy reduces the morbidity after surgery for stage I non-small cell lung cancer. Jpn J Thorac Cardiovasc Surg 2006;54:49–55. [DOI] [PubMed] [Google Scholar]
  • [35].Walker WS, Carnochan FM, Pugh GC. Thoracoscopic pulmonary lobectomy. Early operative experience and preliminary clinical results. J Thorac Cardiovasc Surg 1993;106:1111–7. [PubMed] [Google Scholar]
  • [36].Taioli E, Lee D-S, Lesser M, et al. Long-term survival in video-assisted thoracoscopic lobectomy vs open lobectomy in lung-cancer patients: a meta-analysis. Eur J Cardiothorac Surg 2013;44:591–7. [DOI] [PubMed] [Google Scholar]
  • [37].Ettinger DS, Akerley W, Bepler G, et al. Non-small cell lung cancer. J Natl Compr Cancer Netw JNCCN 2010;8:740–801. [DOI] [PubMed] [Google Scholar]
  • [38].Park BJ, Flores RM, Rusch VW. Robotic assistance for video-assisted thoracic surgical lobectomy: technique and initial results. J Thorac Cardiovasc Surg 2006;131:54–9. [DOI] [PubMed] [Google Scholar]
  • [39].Toker A, Özyurtkan MO, Kaba E, et al. Robotic anatomic lung resections: the initial experience and description of learning in 102 cases. Surg Endosc 2016;30:676–83. [DOI] [PubMed] [Google Scholar]
  • [40].Veronesi G. Robotic lobectomy and segmentectomy for lung cancer: results and operating technique. J Thorac Dis 2015;7(suppl 2):S122–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [41].Park BJ, Flores RM. Cost comparison of robotic, video-assisted thoracic surgery and thoracotomy approaches to pulmonary lobectomy. Thorac Surg Clin 2008;18:297–300. [DOI] [PubMed] [Google Scholar]
  • [42].Sanderson S, Tatt ID, Higgins JPT. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol 2007;36:666–76. [DOI] [PubMed] [Google Scholar]
  • [43].Egger M, Schneider M, Davey Smith G. Spurious precision? Meta-analysis of observational studies. BMJ 1998;316:140–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [44].Rubin D. For objective causal inference, design trumps analysis. Ann Appl Stat 2008;2:808–40. [Google Scholar]

Articles from Medicine are provided here courtesy of Wolters Kluwer Health

RESOURCES