Skip to main content
Elsevier Sponsored Documents logoLink to Elsevier Sponsored Documents
. 2013 Nov 2;382(9903):1496–1506. doi: 10.1016/S0140-6736(13)60992-7

Percutaneous vesicoamniotic shunting versus conservative management for fetal lower urinary tract obstruction (PLUTO): a randomised trial

Rachel K Morris a,c, Gemma L Malin a, Elisabeth Quinlan-Jones a, Lee J Middleton d, Karla Hemming b, Danielle Burke b, Jane P Daniels d, Khalid S Khan a, Jon Deeks b,d, Mark D Kilby a,c,*; for the Percutaneous vesicoamniotic shunting in Lower Urinary Tract Obstruction (PLUTO) Collaborative Group
PMCID: PMC3898962  PMID: 23953766

Summary

Background

Fetal lower urinary tract obstruction (LUTO) is associated with high perinatal and long-term childhood mortality and morbidity. We aimed to assess the effectiveness of vesicoamniotic shunting for treatment of LUTO.

Methods

In a randomised trial in the UK, Ireland, and the Netherlands, women whose pregnancies with a male fetus were complicated by isolated LUTO were randomly assigned by a central telephone and web-based randomisation service to receive either the intervention (placement of vesicoamniotic shunt) or conservative management. Allocation could not be masked from clinicians or participants because of the invasive nature of the intervention. Diagnosis was by prenatal ultrasound. The primary outcome was survival of the baby to 28 days postnatally. All primary analyses were done on an intention-to-treat basis, but these results were compared with those of an as-treated analysis to investigate the effect of a fairly large proportion of crossovers. We used Bayesian methods to estimate the posterior probability distribution of the effectiveness of vesicoamniotic shunting at 28 days. The study is registered with the ISRCTN Register, number ISRCTN53328556.

Findings

31 women with singleton pregnancies complicated by LUTO were included in the trial and main analysis, with 16 allocated to the vesicoamniotic shunt group and 15 to the conservative management group. The study closed early because of poor recruitment. There were 12 livebirths in each group. In the vesicoamniotic shunt group one intrauterine death occurred and three pregnancies were terminated. In the conservative management group one intrauterine death occurred and two pregnancies were terminated. Of the 16 pregnancies randomly assigned to vesicoamniotic shunting, eight neonates survived to 28 days, compared with four from the 15 pregnancies assigned to conservative management (intention-to-treat relative risk [RR] 1·88, 95% CI 0·71–4·96; p=0·27). Analysis based on treatment received showed a larger effect (3·20, 1·06–9·62; p=0·03). All 12 deaths were caused by pulmonary hypoplasia in the early neonatal period. Sensitivity analysis in which non-treatment-related terminations of pregnancy were excluded made some slight changes to point estimates only. Bayesian analysis in which the trial data were combined with elicited priors from experts suggested an 86% probability that vesicoamniotic shunting increased survival at 28 days and a 25% probability that it had a large, clinically important effect (defined as a relative increase of 55% or more in the proportion of neonates who survived). There was substantial short-term and long-term morbidity in both groups, including poor renal function—only two babies (both in the shunt group) survived to 2 years with normal renal function. Seven complications occurred in six fetuses from the shunt group, including spontaneous ruptured membranes, shunt blockage, and dislodgement. These complications resulted in four pregnancy losses.

Interpretation

Survival seemed to be higher in the fetuses receiving vesicoamniotic shunting, but the size and direction of the effect remained uncertain, such that benefit could not be conclusively proven. Our results suggest that the chance of newborn babies surviving with normal renal function is very low irrespective of whether or not vesicoamniotic shunting is done.

Funding

UK National Institute of Health Research, Wellbeing of Women, Hannah Eliza Guy Charity (Birmingham Children's Hospital Charity).

Introduction

Fetal lower urinary tract or bladder outflow obstruction (LUTO) can lead to abnormal renal development, the results of which persist into childhood. The two most common congenital malformations to cause LUTO are posterior urethral valves1 and urethral atresia.2 Severe prenatal renal impairment is often associated with clinically significant oligohydramnios. Such an ultrasound presentation, between 16 and 24 weeks, is associated with a high prevalence of pulmonary hypoplasia, resulting in high perinatal mortality and morbidity.3–5

LUTO is usually diagnosed at 20 weeks of gestation, when most pregnant women in developed countries have a routine detailed fetal anomaly scan. Typical ultrasound features in the fetus are megacystis (enlarged bladder with a dilated proximal urethra) and bilateral hydronephrosis with or without renal parenchymal cystic appearances (cystic kidney disease). Such ultrasound features are generally associated with clinically significant oligohydramnios. The occurrence of both renal cystic change and oligohydramnios with megacystis is highly predictive of a urethral obstructive origin.6 Such prenatal ultrasound findings are of little value in the differentiation of posterior urethral valves from other causes of LUTO,6,7 and the final underlying pathological diagnosis is often not confirmed until the postnatal period. Some researchers have advocated fetal urinalysis to allow fetal triage and the allocation of risk of postnatal renal damage prospectively.8–10 However, investigators of a systematic review11 noted that the overall diagnostic accuracy of such testing is low and could not be relied on in case selection for treatment.

Bladder drainage by serial vesicocentesis or by continuous drainage into the amniotic cavity by placement of a vesicoamniotic shunt has been used to relieve fetal LUTO by bypassing the urethral blockage. Prenatal vesicoamniotic shunting attempts to reduce or avoid renal parenchymal damage and chronic oligohydramnios that can adversely affect pulmonary development.12–14 Our group's previous systematic review15 to assess the effectiveness of bladder drainage (vesicoamniotic shunting or vesicocentesis) showed that fetal bladder drainage increased survival (pooled odds ratio [OR] for perinatal survival 2·53, 95% CI 1·08–5·93). However, none of the studies identified in the systematic review were randomised trials and so the potential for bias in these results is substantial. Some investigators have suggested that tremendous improvements in fetal selection for vesicoamniotic shunting have led to increased survival.16 The results of our review15 suggested that the poor outlook group might benefit more from shunting than would babies who have a better outlook, and that a false-positive diagnosis of LUTO is made in 25% of those babies with a better outlook.17

We aimed to assess the effect on survival of vesicoamniotic shunting compared with conservative management in fetuses with LUTO.

Methods

Study design and participants

We undertook a randomised, international, multicentre trial (Percutaneous vesicoamniotic shunting for fetal Lower Urinary Tract Obstruction [PLUTO]). Parallel to the trial were a register of eligible pregnancies with LUTO for which randomisation was not possible because of either patient or clinician preference, and an anonymous register of terminations of eligible pregnancies. Inclusion criteria were consenting pregnant women who had a singleton male fetus with an ultrasound diagnosis of LUTO (diagnosed on the basis of the visualisation of an enlarged bladder and dilated proximal urethra, bilateral or unilateral hydronephrosis, and cystic parenchymal renal disease6,18) about whom the clinician was uncertain as to the optimum management.

Fetuses with additional major structural or chromosomal anomalies were excluded. Female fetuses were also excluded from the trial because they are more likely than male fetuses to have a complex cause behind their diagnosis and a very poor outlook. No eligibility criteria related to gestational age or volume of amniotic fluid. All participants provided written informed consent. The participants were counselled and given written information about the risks of the procedure: miscarriage, prelabour rupture of membranes and premature labour, chorioamnionitis, and shunt blockage or migration. The PLUTO trial received ethics approval from Nottingham Research Ethics Committee 2 (Jan 21, 2005; reference 04/Q2404/89). The protocol was published19 and no substantial changes were made during the course of the study.

Randomisation and masking

21 fetal medicine centres in England, Scotland, Ireland, and the Netherlands participated in the study, seven of which (from England, Scotland, and the Netherlands) recruited women to the trial. Pregnancies were allocated to intervention (placement of vesicoamniotic shunt) or conservative management by a central telephone and web-based randomisation service provided by the University of Birmingham Clinical Trials Unit (Birmingham, UK). Allocations were concealed until all baseline characteristics for the individual participant had been recorded. We used a computerised minimisation procedure to achieve balance between groups for gestational age at diagnosis (<24 or ≥24 weeks), volume of amniotic fluid (≤5th centile or >5th centile), and age of mother (<20, 20–35, or >35 years). Because of the nature of the intervention, clinician and participant masking was not possible.

Procedures

For vesicoamniotic shunting, participants were offered sedation, and prophylactic antibiotics were given 2 h before the procedure. The fetus and fetal bladder were visualised continuously by high-quality, high-resolution, real-time 2D ultrasound (Siemens S2000 or equivalent). With a sterile, minimum-touch technique, a vesicoamniotic pigtail catheter was inserted percutaneously with either the King's College/Rocket introducer or the Harrison shunting set, according to clinician preference. Optimum placement of the shunt (with the distal end in the fetal bladder and the proximal end in the amniotic cavity) and fetal viability were confirmed immediately and at several hours post-procedure.

Follow-up ultrasound scans were arranged at the clinicians' discretion, but were usually no less frequent than every 4 weeks. All investigators had special training in fetal medicine (recognised by the UK Royal College of Obstetrics and Gynaecology or international equivalent). UK and Ireland investigators without demonstrable competence in ultrasound-guided vesicoamniotic shunt insertion referred women to the Fetal Medicine Centre at Birmingham Women's Hospital (Birmingham, UK). Leiden University Medical Centre (Leiden, Netherlands) received referrals from other hospitals across the Netherlands. All vesicoamniotic shunts were inserted within 7 days of random assignment. Conservative management was standard care (no intervention) with follow-up ultrasound scans arranged at the clinicians' discretion, but usually no less frequently than every 4 weeks.

The primary outcome measure was survival to 28 days after birth. Secondary outcomes were survival at 1 and 2 years, and renal function at 28 days, 1 year, and 2 years (measured by serum creatinine, renal ultrasound appearance, and evidence of renal impairment [need for medical treatment, dialysis, or transplantation]). Data for serious adverse events were obtained, including pregnancy loss before 24 weeks, premature rupture of membranes, chorioamnionitis, damage to maternal uterus or other organs, damage to fetal organs, and migration of stent.

Statistical analysis

The planned sample size for the trial was based on our group's previous meta-analysis of observational studies,15 reported at the time we were designing the trial, in which we noted an improvement in survival with vesicoamniotic shunting from 39% to 61% (equivalent to a relative risk [RR] of 1·55). At 80% power and α=0·05, 75 pregnancies in each study group would be sufficient to detect such a difference.

We calculated RRs and their 95% CIs by standard normal approximation methods for the primary outcome of survival at 28 days, with significance assessed by two-sided Fisher's exact test. All primary analyses were done on an intention-to-treat basis, but these were compared with an as-treated analysis to investigate the effect of a fairly large proportion of crossovers. Intrauterine deaths and terminations of pregnancy were included in the analysis and classed as a death in the primary analysis, although we did a sensitivity analysis in which non-treatment-related terminations (parent choice) were excluded. Analyses of livebirths and survival to 1 year and 2 years were done as per the primary outcome. Formal statistical analysis was not attempted for the other outcome measures, nor were subgroup analyses or covariate adjustments, because of the small number of participants recruited. We have reported summary statistics (means with SDs or medians with IQRs) or individual values for these outcomes as appropriate.

Using a questionnaire, we elicited expert opinions on benefits of vesicoamniotic shunting relative to conventional treatment from fetal medicine specialists, paediatric nephrologists, and paediatric urologists to provide informative Bayesian priors for survival to 28 days. The elicited opinions were pooled across experts to create an informative prior distribution. We used Bayesian methods to estimate the posterior probability distribution of the effectiveness of vesicoamniotic shunting at 28 days (a logistic model) with both the informative experts' prior distribution and a non-informative prior (centred at the value of no effect, with very large variance [1002]).20 The analysis with non-informative priors replicates the frequentist analysis to allow for additional probabilistic interpretation of the results. We also used a Bayesian Cox proportional hazards method to model survival to Dec 1, 2012, with non-informative priors (informative priors were not elicited for survival beyond 28 days). The Cox proportional hazards model was stratified from conception to 36·5 weeks (around the time of birth) and from 36·5 weeks onwards. To make the estimate on the hazard ratio (HR) scale consistent with the RR estimates of relative survival (insofar as both HR>1 and RR>1 favour vesicoamniotic shunting), the HR compares the hazard on conservative management with the intervention.

The probability of vesicoamniotic shunting increasing survival (RR or HR >1) and the probability of such an intervention substantially increasing survival at 28 days after birth (RR or HR >1·55, the difference the trial was powered to detect had recruitment targets been met) were calculated from posterior distributions. All Bayesian analyses were done with WinBUGS,20 with 200 000 iterations after allowing for a 10 000-iteration burn-in and checking for convergence with several common measures. Summary estimates reported are medians with 95% credible intervals (CrIs). All Bayesian analyses were done on an intention-to-treat basis. Additional Bayesian analyses (eg, as treated) are reported elsewhere.21

This trial is registered with the ISRCTN Register, number ISRCTN53328556.

Role of the funding source

The funders had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results

Seven of the 21 participating centres recruited women to the trial. 31 women with singleton pregnancies complicated by LUTO agreed to be randomly assigned to a trial group between Oct 2, 2006, and Oct 11, 2010. Follow-up for the 2-year outcome was completed on Sept 21, 2012. The planned sample size of the trial was 150, but enrolment was stopped after 31 women had been assigned because of difficulties with recruitment. The main barriers were issues with achievement of sponsorship and indemnity for international centres, slow approval for UK centres, a higher proportion of parents than expected choosing for termination of pregnancy (n=68), lower prevalence of disease than reported in the scientific literature,22 and a high proportion of parents and clinicians choosing to enter a registry rather than be randomly assigned to a trial group (n=45). The registry data will be reported elsewhere.21 Figure 1 shows the trial profile. Complete follow-up for all participants was available to 2 years (range 2·2–5·6).

Figure 1.

Figure 1

Trial profile

Normal renal function is defined as serum creatinine less than 50 μmol/L; mild renal impairment is defined as serum creatinine of 50 μmol/L or more, not requiring medical treatment; moderate renal impairment is defined as serum creatinine of 50 μmol/L or more, requiring medical treatment; end-stage renal failure is defined as need for transplant or dialysis. TOP=termination of pregnancy. *Two treatment related TOPs occurred secondary to spontaneous rupture of membranes after shunt insertion at 17 and 22 weeks. † Parental decision at 18–25 weeks. ‡After spontaneous rupture of membranes at 16 weeks. §Due to pulmonary hypoplasia.

Baseline characteristics of participants were well balanced between trial groups (table 1). The median age of mothers in the trial was 28 years (IQR 23–33), with presentation at a median gestation of 21 weeks (18–22), with 27 of 31 at less than 24 weeks' gestation at the time of LUTO diagnosis. Most fetuses had oligohydramnios (table 1; maximum pool depth range in shunt group 0·0–4·4 cm and in conservative management group 0·0–4·9 cm). Seven fetuses underwent vesicocentesis and fetal urinalysis before shunt insertion, five or which had a prospective designation of good outlook and two a poor outlook.26 Four fetuses in the shunt group and five in the conservative management group were prospectively karyotyped and all had normal male karyotype. All 31 were confirmed as male fetuses postnatally and had no dysmorphic features of chromosomal anomaly. 15 women underwent a shunt procedure; six were done at Birmingham Women's Hospital, one at Newcastle Royal Victoria Infirmary (Newcastle, UK), three at Liverpool Women's Hospital (Liverpool, UK), one at St George's Hospital (London, UK), and four at Leiden University Medical Centre. Five operators inserted the shunts; 10 of the 15 insertions used the Harrison shunt and five the Rocket shunt.

Table 1.

Baseline characteristics of mothers and fetuses

Vesicoamniotic shunt(n=16) Conservative management (n=15)
Maternal age (years) 27 (23–33) 28 (26–33)
Maternal age group*
<20 years 2 0
20–35 years 12 12
>35 years 2 3
Ethnic origin
White 13 13
Asian 2 1
Black 1 1
Gestational age (weeks) 20 (16–22) 21 (19–22)
Gestational age group*
<24 weeks 13 14
≥24 weeks 3 1
Amniotic fluid volume maximum pool depth (cm) 1·6 (0–2·9) 1·0 (0·2–2·9)
Amniotic fluid volume, by centile*
<5th centile 10 9
≥5th centile 6 6
Renal pelvis dilatation, left (mm) 7·4 (5·0–14) 7·3 (5·0–9·0)
Renal pelvis dilatation, right (mm) 8 (4·8–10) 8·6 (5·0–11)
Anteroposterior renal pelvis dilatation diameter >90th centile
Bilateral 13 12
Unilateral 2 0
Neither 0 1
Not recorded 1 2
Renal pelvis severe hydronephrosis >1·5 cm§
Bilateral 1 0
Unilateral 1 1
Neither 13 13
Not recorded 1 1
Macrocystic renal appearance
Bilateral 0 1
Unilateral 2 4
Neither 14 10
Renal echogenicity
Bilateral 2 4
Unilateral 3 3
Neither 7 5
Not recorded 4 3
Bladder wall thickness >3 mm
Yes 6 9
No 8 5
Not recorded 2 1

Data are n or median (IQR).

*

Stratification variable and predefined subgroup.

See reference 23.

See reference 24.

§

See reference 25.

Three of the 16 pregnancies randomly assigned to vesicoamniotic shunting did not receive the intervention, and two of the 15 randomly assigned to conservative management had shunting. Indications for not receiving the allocated shunt were that parents withdrew consent after assignment (to treatment, but not follow-up), that the woman or the fetus had a contraindication to shunting after assignment, or that the clinician decided that insertion of a shunt was inappropriate because of poor fetal condition. The two cases in which women randomly assigned to conservative management received a shunt were both due to deterioration in the clinical situation (principally anhydramnios) and the clinician believing that shunting was appropriate. In three cases (one in the shunt group and two in the conservative management group) parents decided on termination of the pregnancy between 18 and 25 weeks because of perceived poor outlook by the health-care professionals and parents. These decisions were believed to be unrelated to the treatment.

12 livebirths occurred in each trial group. In the shunt group there was one intrauterine death at 16 weeks after shunt insertion and three pregnancies were terminated (two of which were related to treatment and followed spontaneous rupture of membranes after shunt insertion, and one of which was by parental choice; figure 1). In the conservative management group three pregnancies resulted in fetal loss, with one intrauterine death at 18 weeks and two terminations of pregnancy (by parental choice) at 18 and 24 weeks.

The median interval between random assignment and delivery was 93 days (IQR 69–118) in the shunt group and 104 days (94–112) in the conservative management group. The median interval for both groups combined was 101 days (89–112). No significant difference in gestational age of delivery was seen between the two groups (34·6 weeks [IQR 33·4–37·2] in the shunt group and 36·4 weeks [34·5–37·4] in the conservative management group). Two babies (both in the shunt group) were delivered preterm between 24 and 32 weeks.

Of the fetuses assigned to the shunt group, half survived to 28 days compared with about a quarter of those assigned to conservative management (table 2). All 12 neonatal deaths were caused by pulmonary hypoplasia; 11 occurred within the first 24 h after birth, and the other at 3 days. In the as-treated analysis, about two-thirds of the participants given a vesicoamniotic shunt survived to 28 days compared with about a fifth given conservative management (table 2). Sensitivity analysis that excluded non-treatment-related terminations of pregnancy made some slight changes only to point estimates (table 3).

Table 2.

Survival outcomes

Intention-to-treat analysis
As-treated analysis*
Vesicoamniotic shunt Conservative management RR (95% CI) p value Vesicoamniotic shunt Conservative management RR (95% CI) p value
Livebirths 12/16 12/15 0·94 (0·64–1·37) >0·99 11/15 13/16 0·90 (0·61–1·33) 0·69
28 days 8/16 4/15 1·88 (0·71–4·96) 0·27 9/15 3/16 3·20 (1·06–9·62) 0·03
1 year 7/16 3/15 2·19 (0·69–6·94) 0·25 8/15 2/16 4·27 (1·07–16·96) 0·02
2 years 7/16 3/15 2·19 (0·69– 6·94) 0·25 8/15 2/16 4·27 (1·07–16·96) 0·02

Data are n/N, unless otherwise indicated. Primary outcome was survival to 28 days. Terminations of pregnancy were included as treatment failures. RR=relative risk.

*

Three pregnancies were allocated to vesicoamniotic shunt but received conservative management, and two pregnancies were allocated conservative management but received vesicoamniotic shunt (non-randomised comparison).

Two-sided Fisher's exact test.

Table 3.

Sensitivity analysis excluding non-treatment-related terminations of pregnancy*

Intention-to-treat analysis
As-treated analysis
Vesicoamniotic shunt Conservative management RR (95% CI) p value Vesicoamniotic shunt Conservative management RR (95% CI) p value
Livebirths 12/15 12/13 0·87 (0·64–1·17) 0·60 11/14 13/14 0·85 (0·62–1·15) 0·60
28 days 8/15 4/13 1·73 (0·68–4·45) 0·28 9/14 3/14 3·00 (1·02–8·80) 0·05
1 year 7/15 3/13 2·02 (0·65–6·26) 0·25 8/14 2/14 4·00 (1·03–15·60) 0·05
2 years 7/15 3/13 2·02 (0·65–6·26) 0·25 8/14 2/14 4·00 (1·03–15·60) 0·05

Data are n/N, unless otherwise indicated. Primary outcome was survival to 28 days. RR=relative risk.

*

Two non-treatment-related terminations of pregnancy were allocated to the conservative management group, one to the vesicoamniotic shunt group.

Three pregnancies were allocated to vesicoamniotic shunt but received conservative management, and two pregnancies were allocated to conservative management but received vesicoamniotic shunt (non-randomised comparison).

Two-sided Fisher's exact test (the CIs and the p value seem incompatible because of the sparse data having a different effect under the different statistical assumptions made in their calculation [normal approximation vs exact method]).

One baby in each trial group subsequently died before age 1 year because of renal failure (figure 1). RR estimates for survival at 1 year were close to those seen at 28 days (tables 2, 3). No deaths occurred between age 1 year and age 2 years, so the RR estimates for survival at 2 years are the same as those for 1 year.

Most livebirths were preterm (less than 37 weeks' gestational age) and most were admitted to a neonatal intensive care unit or children's hospital (table 4). Some neonates required ventilation or immediate treatment for renal impairment (table 4). Of the 12 babies who survived to 28 days, only two did not have any renal impairment (serum creatinine <50 μmol/L and did not need medical treatment, dialysis, or transplantation). Both babies were among those who received a shunt and were also the only two babies without renal impairment at the 1-year follow-up. No difference in survival with normal renal function was seen in the as-treated analysis at 28 days or at 1 year (table 4). At age 2 years one baby in the shunt group had progressed from mild to moderate impairment and one baby in the conservative management group had progressed from moderate impairment to end-stage renal failure (requiring dialysis and awaiting transplantation). At 2 years, one baby in the shunt group had serious cognitive and motor impairment.

Table 4.

Other outcomes

Vesicoamniotic shunt Conservative management
Livebirths
n 12 12
Days from randomisation to delivery 93 (69–118) 104 (94–112)
Gestational age at delivery (days) 249 (234–263) 255 (242–262)
Preterm labour (<37 weeks) 7 8
Vaginal delivery 8 7*
Mean birthweight, kg (SD) 2·8 (0·5) 2·8 (0·4)
Birthweight <10th centile 5 4
Admitted to neonatal ICU or children's hospital 10 10
Required ventilation 6 7
Required treatment for renal impairment 4 3
Perinatal (about 28 days)
n 8 4
Required surgery in perinatal period 5 3
Still an inpatient 3 2
Serum creatinine, μmol/L 29, 29, 88, 96, 105, 108, 119, 342 70, 126, 449, 620
Renal function§
Normal 2 0
Mild impairment 2 1
Moderate impairment 4 2
End-stage renal failure 0 1
1 year
n 7 3
Required surgery from perinatal period to 1 year 6 0
Days in hospital 0, 1, 3, 20, 25, 84, 102 22, 39, NR
Weight <10th centile 4 2
Serum creatinine, μmol/L 34, 37, 58, 64, 81, 88, 226 60, 60, 501
Renal function§
Normal 2 0
Mild impairment 1 0
Moderate impairment 4 3
2 years
n 7 3
Required surgery between 1 and 2 years 4 1
Days in hospital 0, 1, 5, 19, 30, 37, 116 23, 37, 40
Weight <10th centile 3 2
Serum creatinine, μmol/L 65, 34, 87, 227, 60, 74, NR 502, 61, 72
Renal function§
Normal 2 0
Mild 0 0
Moderate 5 2
End-stage renal failure 0 1
Cognitive impairment 1 serious None reported abnormal

Data are n, median (IQR), or a list of individual values, unless otherwise indicated. ICU=intensive care unit. NR=not recorded (where individual values shown).

*

Includes one vaginal breech.

One baby did not have weight recorded.

Two babies did not have weight recorded.

§

Normal renal function is defined as serum creatinine less than 50 μmol/L; mild impairment is defined as serum creatinine of 50 μmol/L or more, not requiring medical treatment; moderate impairment is defined as serum creatinine of 50 μmol/L or more, requiring medical treatment; end-stage renal failure is defined as need for transplant or dialysis.

On transplant register.

Not all infants were investigated for cognitive impairment.

Abnormal changes were diagnosed in all 12 perinatal survivors, and a post mortem was done for three of the neonatal deaths (one in the shunt group and two in the conservative management group). The diagnoses were nine cases of posterior urethral valves (five in the shunt group and four in the conservative management group), five cases of urethral atresia (four in the shunt group and one in the conservative management group), and one case of urethral syrinx that caused obstruction (in the conservative management group). Of seven cases that were pregnancy losses (terminations or intrauterine deaths), three had a post-mortem examination, which confirmed urethral atresia in two fetuses and of posterior urethral valves in one. Thus in 13 cases no such changes were seen; four of these were miscarriages and nine were neonatal deaths attributed to pulmonary hypoplasia.

All babies with posterior urethral valves underwent urethral valve resection in the perinatal period (up to 6 weeks postnatally), and all those with urethral atresia had a vesicostomy. Six of the ten survivors (all in the shunt group) required some sort of renal surgery at 1 year (table 4). One baby required both valve resection and a vesicostomy in the perinatal period. At age 1 year, three babies in the shunt group had required orchidopexy and one a vesicostomy; two underwent repeat valve resection, one a nephrostomy, and one a nephrectomy. Table 5 lists the complications of the vesicoamniotic shunt intervention. Total pregnancy losses were four of 15, although one of these losses was thought to be unrelated to treatment.

Table 5.

Complications of vesicoamniotic shunting

Proportion affected* Outcome
Spontaneous rupture of membranes after shunt insertion 3/15 Intrauterine death at 16 weeks' gestation (n=1); pregnancy affected by chorioamnionitis and parents chose to terminate at 17–22 weeks' gestation (n=2)
Dislodged 3/15 Spontaneous rupture of membranes with chorioamnionitis and parents chose to terminate at 17 weeks' gestation (n=1); subsequent pregnancy urinary ascites and neonatal death due to pulmonary hypoplasia (n=1); shunt reinserted, subsequent preterm labour at 26 weeks' gestation, alive at 28 days (n=1)
Blocked 1/15 Parents chose to terminate at 19 weeks' gestation (not related to treatment; persistent fetal bradycardia; n=1)

Four of 15 pregnancies were lost in the vesicoamniotic shunt group (three from spontaneous rupture of membranes and one after blockage).

*

Six were affected in total, with some having more than one complication.

For our Bayesian analysis, the survey questionnaire was emailed to 248 experts, from whom 59 replies were received, seven of which were too incomplete to be included. Thus 52 experts provided information about their beliefs for change in perinatal mortality from intrauterine vesicoamniotic shunting. Although the experts generally supported the notion that shunting improved 28-day survival (the elicited probability of survival increasing was 79%), they were collectively sceptical about the ability of shunting to provide a treatment effect as large as that anticipated by the trial (ie, an RR of 1·55). The elicited probability that the treatment effect would be this large was only 15% (figure 2A).

Figure 2.

Figure 2

Results of Bayesian analysis

Bayesian prior and posterior estimates of relative risk of survival to age 28 days. The prior distribution (A) was obtained by eliciting prior distributions from 52 experts, averaging the distributions, and fitting a normal distribution. The posterior distribution (B) is based on combining the elicited prior with the intention-to-treat results. The posterior distribution (C) is based on combining a non-informative prior with the intention-to-treat results.

Uncertainty about the size of the treatment effect remained when the elicited opinions were combined with the trial results (figure 2B). The probability of survival increasing with vesicoamniotic shunting rose to 86% and the probability that VAS had a large, clinically important effect (an increase in the proportion of babies who survived to 28 days of 55% or more) rose to 25%. Some uncertainty remained (probability 14%) about whether shunting might decrease survival. Analysis with the non-informative prior showed a larger uncertainty about the treatment effect than did analysis with the informative prior (figure 2C), but with a much higher probability (67%) of survival improving by more than 55%.

Survival analysis of outcomes and partitioning by survival to 36·5 weeks and from 36·5 weeks to 5·7 years suggest that vesicoamniotic shunting could have harmful effects from conception to 36·5 weeks (HR 0·90, 95% CrI 0·25–3·06), with only a 19% probability of the HR being greater than 1·55. However, from 36·5 weeks onwards some evidence suggests that shunting might increase survival outcomes, conditional on survival to 36·5 weeks (4·20, 0·86–33·07), with a probability of 88% of the HR being greater than 1·55 (figure 3).

Figure 3.

Figure 3

Kaplan-Meier estimates of survival from conception to end of follow-up

Vertical dashes on the lines represent censored observations, either through parental choice to terminate pregnancy (before 36·5 weeks) or end of follow-up. Hazard ratios (HRs) were calculated for the period 0–36·5 weeks and from 36·5 weeks to the end of follow-up (conditional on survival to 36·5 weeks). These HRs are Bayesian estimates based on non-informative priors. The hazard rate comparison is of conservative management versus vesicoamniotic shunt, such that the interpretation of HRs greater than 1 as being indicative of benefit is consistent with the interpretation of the relative risk estimates. CrI=credibility interval.

Discussion

Survival to 28 days, 1 year, and 2 years seems to be higher in fetuses that received vesicoamniotic shunting, but uncertainty remains about the direction and size of the effect such that benefit cannot be conclusively proven. However, overall outlook in both trial groups at 2 years was poor, with only two babies surviving to that age without renal impairment. This finding reinforces the fact that the natural pathogenesis of this fetal disease is severe and that mortality and morbidity are substantial independent of treatment (panel).17

Panel. Research in context.

Systematic review

We used our previous systematic reviews15,27 to asssess the existing evidence about vesicoamniotic shunting for fetal lower urinary tract obstruction. The results of the most recent review27 suggest an odds ratio for perinatal survival with vesicoamniotic shunting of 3·86 (95% CI 2·00–7·45) and for survival with normal renal function of 0·50 (0·13–1·90). The evidence from observational studies suggests that vesicoamniotic shunting improves perinatal survival, but the effect on long-term renal function is unclear.

Interpretation

The results of the PLUTO randomised controlled trial are consistent with the findings of the observational evidence for perinatal survival, but suggest that the chance of newborn babies surviving with normal renal function is very low irrespective of whether or not vesicoamniotic shunting is done.

Even if perinatal survival is increased, vesicoamniotic shunting might not have a long-term benefit. Some of the women in the trial did not receive the allocated treatment because of clinician choice or a changing clinical situation. Few women were willing to consider participation in the trial and chose either entry onto the registry or termination of pregnancy. Among the trial participants, all perinatal survivors had abnormalities confirmed as secondary to congenital bladder outflow obstruction (posterior urethral valves and atresia). A high proportion of fetuses were at a gestational age of less than 24 weeks and had ultrasound features consistent with an overall poor outlook.10 This subgroup seemed to benefit most from vesicoamniotic shunting in our systematic review and subsequent meta-analysis of perinatal outcomes.15

The trial was stopped early because of poor recruitment. Results are available for all babies until at least age 2 years. After 4 years, 144 women overall had been involved in the study (randomly assigned in the trial, registered, or terminations of pregnancy). The prevalence of LUTO overall was as previously reported,17 but the prevalence of isolated, antenatally detected LUTO amenable to intervention was reduced.

A high proportion of pregnancies were lost (termination or intrauterine death) in both study groups and pregnancy outcomes (livebirths) did not differ significantly between the groups. However, survival to 28 days (perinatal and neonatal survival) seemed to be higher with vesicoamniotic shunting, but this result was not significant. On the basis of the expert (informative) and vague (non-informative) priors respectively, the Bayesian analysis gave estimated probabilities of 86% and 92% that shunting increased survival, and of 25% and 67% that the increase is greater than 55%. These findings are therefore in keeping with the results of our group's systematic review of outcomes from case-cohort studies.27 Because all neonatal deaths in the trial were from pulmonary hypoplasia, the reduction in perinatal mortality noted with shunting is probably due to prevention or amelioration of the oligohydramnios at the crucial time of lung development (canalicular phase, between 16 and 24 weeks' gestation).

Clinical outcomes at 1 and 2 years show an overall poor outlook in both groups, but again with a possible improved survival with shunting. The likelihood of surviving with normal renal function was very small in both groups, which suggests that the damage to the renal parenchyma had already taken place at the time of diagnosis and was irreversible. Again, these findings are consistent with the meta-analysis results27 for postnatal survival with normal renal function from small cohort studies.

Another important finding of the PLUTO trial was the high proportion of women who did not receive their allocated treatment. This crossover occurred either because of clinician choice or because of changing clinical features as the pregnancy progressed. Analysis on the basis of treatment received showed larger differences in outcome between the shunt and conservative management groups. For survival at 28 days and at 1 and 2 years post-delivery, the outcomes favoured shunting and the differences were significant. No difference in survival with normal renal function was seen at 28 days or at 1 or 2 years in the as-treated analysis. However, because treatment selection was partly informed by outlook in this analysis, these findings could be confounded by selection bias.

The PLUTO trial showed that in the context of an intervention for a rare, fetal disease, few women (or their families) were willing to take part in a randomised trial of treatment and chose either entry onto the registry (in which case they or their health professional could choose treatment) or, more often, chose to terminate the pregnancy. Entry onto the registry was also commonplace because clinicians were not in clinical equipoise (ie, they were not substantially uncertain with respect to the best management option). The findings from the PLUTO registry will be reported in the UK National Institute of Health Research Health Technology Assessment monograph.21

An important limitation of this trial is that the small number of pregnant women recruited made the study underpowered. For this reason, the trial was only sufficiently powered to detect very large differences in the primary outcome. However, inclusion of a preplanned Bayesian analysis did allow direct estimation of the probability of benefit to assist decision making, even though conventional methods did not reach significance. One of the major barriers to recruitment was the high proportion of women entered onto the registry. Such a decision was often due to the treating clinician apparently not being in equipoise with respect to the benefits of the intervention, which contrasts with the findings of the expert opinions elicited at the start of the study (wherein diverse opinions and apparent equipoise was seen, despite the findings of our group's earlier systematic review15). This issue suggests a divergence of opinion between specialists and established clinical equipoise. A strength in this respect is that all children completed follow-up assessment and no data were missing for the primary and secondary endpoints. All endpoints were assessed independently by paediatric neonatologists and nephrologists as part of routine practice. The primary analysis was done on an intention-to-treat basis with no exclusions.

Allocation could not be masked from fetal medicine specialists, parents, or endpoint assessors because of the nature of the intervention. Because the primary endpoint was survival, the results for the primary outcome would not have been affected by the absence of masking. However, this feature of the study design might have had an effect on the number of women who crossed over from the conservative management to the shunt group because of a change in the assessment of the clinical situation as the pregnancy progressed. This selection bias will have been a confounder and the size and direction of its effect cannot be assessed. Thus, the number of women who crossed over from conservative management to shunting limits the conclusions that can be drawn from the study.

Some researchers consider Bayesian analyses to be subjective, since they are dependent on the experts who agree to participate and whose opinions form the basis of the informative priors. The best way in which to elicit prior opinions so as to accurately reflect beliefs is also a heavily debated topic. Such analyses are also prone to convergence issues, although these issues tend to occur in more complicated models than those used in our analysis.

PLUTO is the only randomised controlled trial to investigate the effectiveness of vesicoamniotic shunting in LUTO. We therefore cannot directly compare our results with previous work in terms of strengths and limitations. Although the number of participants recruited to PLUTO was small with respect to the prospective power calculation for the trial, it is similar to the numbers included in previously reported cohort studies. These retrospective case-cohort studies and their results have been rigorously assessed by our group via systematic review and meta-analysis as part of the PLUTO study, reported during the recruitment period.15,27 The results from this systematic review suggest an OR for perinatal survival with vesicoamniotic shunting of 3·86 (95% CI 2·00–7·45) and for survival with normal renal function of 0·50 (0·13–1·90). The conclusion from the observational evidence is that shunting improves perinatal survival, but the effect on long-term renal function is unclear. As an additional qualification, the outcome measures used for renal function in these observational studies were heterogeneous.

Because the findings of the PLUTO trial were limited by the small numbers of participants recruited, any recommendations should be interpreted with this limitation in mind. However, the results of the PLUTO trial are consistent with the observational evidence, which suggests that vesicoamniotic shunting improves overall perinatal survival, but that the long-term outlook for these babies is poor, with high mortality and morbidity. Parents should be counselled about the risks of pregnancy loss with or without shunt insertion. In the UK, the relevant National Institute for Health and Care Excellence interventional procedure guidance28 should be updated to take account of this new evidence.

Since PLUTO was a multicentre, international trial and did not recruit the necessary number of women to achieve adequate statistical power, a further randomised controlled trial is unlikely to be feasible. The babies recruited into the PLUTO trial must be prospectively followed up throughout childhood to attempt to assess the effects of vesicoamniotic shunting on outcomes such as renal function, incontinence, cognitive development, and quality of life. However, the number of children who survive to a suitable age (2–5 years) is likely to be small.

Further research is needed to try to overcome the barriers to recruitment identified in this study—namely, the methods of randomised controlled trials in rare diseases (especially diseases related to pregnancy). Improvements have already been made in the past few years in the UK with respect to ethics and research and development approval for studies recruiting from several sites. These changes greatly reduced the time that centres took to recruit their first participants in the later period of the PLUTO trial. However, the study was limited in its ability to recruit by difficulties in obtaining indemnity and thus sponsorship for international centres. This issue is something that the international academic community, higher education institutions, and funders must work hard to resolve, so that research questions can be adequately addressed in the study of rare disease.

Acknowledgments

Acknowledgments

The study sponsor was the University of Birmingham and the study was funded by the UK National Institute of Health Research (NIHR) Health Technology Assessment (HTA) programme (project grant 07/01/44), Wellbeing of Women, and the Hannah Eliza Guy Charity (Birmingham Children's Hospital Charity). The views expressed in this Article are those of the authors and do not necessarily reflect those of the NIHR HTA programme, the NIHR, or the UK Department of Health. We thank the members of the trial steering committee and data monitoring committee for their assistance throughout the project. The PLUTO trial was coordinated by the Birmingham Clinical Trials Unit and we acknowledge the hard work of all of the staff involved in the study. We thank the fetal medicine midwives at the Fetal Medicine Centre at Birmingham Women's Hospital and all the recruiting centres. We acknowledge the contribution of Celia Brown and Richard Lilford in doing the original data collection and analysis of Bayesian priors.

Contributors

RKM was a research fellow for the trial (2005–12), managed the trial, and was involved in data collection and analysis for the systematic reviews, data collection and analysis of trial and registry data, and data collection for the Bayesian priors. GLM was a research fellow for the trial, participated in management of the trial, and was involved in data collection and analysis for the systematic reviews, and data collection of trial and registry data. EQ-J was the coordinating midwife for the trial and was involved in data collection and analysis for the systematic reviews and data collection and analysis of the trial and registry data. LJM was the trial statistician and contributed to the design of the trial and registry study for which he also did the statistical analysis. KH undertook and wrote the Bayesian analysis of outcomes. DB did the Bayesian analysis of survival. JPD was involved in the trial design and management. KSK was involved in the study design and management and in obtaining grant funding. JD supervised the analysis of the results of the trial, registry study, and the Bayesian analysis. MDK was the chief investigator for the study and was involved in the study design, management, analysis of results, and writing of the report. He was the principal investigator on the UK National Institute of Health Research Health Technology Assessment and Wellbeing of Women grants. As the corresponding author he had full access to all the data in the study, had final responsibility for the decision to submit for publication, and takes final responsibility for the contents of the report. All authors were involved in the writing of the report.

PLUTO Collaborative Group

Z Alfirevic (Liverpool Women's NHS Foundation Trust, Liverpool, UK); D K James (Nottingham University Hospitals NHS Trust, Nottingham, UK); G Tydeman (Fife Forth Park Hospital, Kirkcaldy, UK); S Sturgiss (Newcastle Royal Victoria Infirmary, Newcastle, UK); B Thilaganathan (St George's Healthcare NHS Trust, London, UK); D Oepkes, P N Adama van Scheltema (Leiden University Medical Centre, Netherlands).

Conflicts of interest

We declare that we have no conflicts of interest.

References

  • 1.Quintero RA, Johnson MP, Romero R. In-utero percutaneous cystoscopy in the management of fetal lower obstructive uropathy. Lancet. 1995;346:537–540. doi: 10.1016/s0140-6736(95)91381-5. [DOI] [PubMed] [Google Scholar]
  • 2.Steinhardt G, Hogan W, Wood E, Weber T, Lynch R. Long-term survival in an infant with urethral atresia. J Urol. 1990;143:336–337. doi: 10.1016/s0022-5347(17)39952-4. [DOI] [PubMed] [Google Scholar]
  • 3.Nakayama DK, Harrison MR, de Lorimier AA. Prognosis of posterior urethral valves presenting at birth. J Pediatr Surg. 1986;21:43–45. doi: 10.1016/s0022-3468(86)80651-0. [DOI] [PubMed] [Google Scholar]
  • 4.Barker AP, Cave MM, Thomas DF. Fetal pelvi-ureteric junction obstruction: predictors of outcome. Br J Urol. 1995;76:649–652. doi: 10.1111/j.1464-410x.1995.tb07796.x. [DOI] [PubMed] [Google Scholar]
  • 5.Hutton KA, Thomas DF, Arthur RJ, Irving HC, Smith SE. Prenatally detected posterior urethral valves: is gestational age at detection a predictor of outcome? J Urol. 1994;152:698–701. doi: 10.1016/s0022-5347(17)32684-8. [DOI] [PubMed] [Google Scholar]
  • 6.Kaefer M, Peters CA, Retik AB, Benacerraf BB. Increased renal echogenicity: a sonographic sign for differentiating between obstructive and nonobstructive etiologies of in utero bladder distension. J Urol. 1997;158:1026–1029. [PubMed] [Google Scholar]
  • 7.Abbott JF, Levine D, Wapner R. Posterior urethral valves: inaccuracy of prenatal diagnosis. Fetal Diagn Ther. 1998;13:179–183. doi: 10.1159/000020834. [DOI] [PubMed] [Google Scholar]
  • 8.Johnson MP, Bukowski TP, Reitleman C, Isada NB, Pryde PG, Evans MI. In utero surgical treatment of fetal obstructive uropathy: a new comprehensive approach to identify appropriate candidates for vesicoamniotic shunt therapy. Am J Obstet Gynecol. 1994;170:1770–1776. doi: 10.1016/s0002-9378(94)70353-1. [DOI] [PubMed] [Google Scholar]
  • 9.Freedman AL, Bukowski TP, Smith CA. Use of urinary beta-2-microglobulin to predict severe renal damage in fetal obstructive uropathy. Fetal Diagn Ther. 1997;12:1–6. doi: 10.1159/000264415. [DOI] [PubMed] [Google Scholar]
  • 10.Khalek N, Johnson MP. Fetal urinary obstruction: prenatal assessment and prognosis. In: Kilby MD, Oepkes D, Johnson A, editors. Fetal therapy: scientific basis and critical appraisal of clinical benefits. Cambridge University Press; Cambridge: 2012. pp. 246–252. [Google Scholar]
  • 11.Morris RK, Quinlan-Jones E, Kilby MD, Khan KS. Systematic review of accuracy of fetal urine analysis to predict poor postnatal renal function in cases of congenital urinary tract obstruction. Prenat Diagn. 2007;27:900–911. doi: 10.1002/pd.1810. [DOI] [PubMed] [Google Scholar]
  • 12.Manning FA, Harrison MR, Rodeck C, members of the International Fetal Medicine and Surgery Society Catheter shunts for fetal hydronephrosis and hydrocephalus. Report of the International Fetal Surgery Registry. N Engl J Med. 1986;315:336–340. doi: 10.1056/NEJM198607313150532. [DOI] [PubMed] [Google Scholar]
  • 13.Elder JS, Duckett JW, Jr, Snyder HM. Intervention for fetal obstructive uropathy: has it been effective? Lancet. 1987;330:1007–1010. doi: 10.1016/s0140-6736(87)92567-0. [DOI] [PubMed] [Google Scholar]
  • 14.Crombleholme TM, Harrison MR, Golbus MS. Fetal intervention in obstructive uropathy: prognostic indicators and efficacy of intervention. Am J Obstet Gynecol. 1990;162:1239–1244. doi: 10.1016/0002-9378(90)90026-4. [DOI] [PubMed] [Google Scholar]
  • 15.Clark TJ, Martin WL, Divakaran TG, Whittle MJ, Kilby MD, Khan KS. Prenatal bladder drainage in the management of fetal lower urinary tract obstruction: a systematic review and meta-analysis. Obstet Gynecol. 2003;102:367–382. doi: 10.1016/s0029-7844(03)00577-5. [DOI] [PubMed] [Google Scholar]
  • 16.Biard JM, Johnson MP, Carr MC. Long-term outcomes in children treated by prenatal vesicoamniotic shunting for lower urinary tract obstruction. Obstet Gynecol. 2005;106:503–508. doi: 10.1097/01.AOG.0000171117.38929.eb. [DOI] [PubMed] [Google Scholar]
  • 17.Malin G, Tonks AM, Morris RK, Gardosi J, Kilby MD. Congenital lower urinary tract obstruction: a population-based epidemiological study. BJOG. 2012;119:1455–1464. doi: 10.1111/j.1471-0528.2012.03476.x. [DOI] [PubMed] [Google Scholar]
  • 18.Morris RK, Malin GL, Khan KS, Kilby MD. Antenatal ultrasound to predict postnatal renal function in congenital lower urinary tract obstruction: systematic review of test accuracy. BJOG. 2009;116:1290–1299. doi: 10.1111/j.1471-0528.2009.02194.x. [DOI] [PubMed] [Google Scholar]
  • 19.Kilby MD, Khan K, Morris K. PLUTO trial protocol: percutaneous shunting for lower urinary tract obstruction randomised controlled trial. BJOG. 2007;114:904. doi: 10.1111/j.1471-0528.2007.01382.x. [DOI] [PubMed] [Google Scholar]
  • 20.Lunn DJ, Thomas A, Best N, Spiegelhalter DJ. WinBUGS— a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. 2000;10:325–337. [Google Scholar]
  • 21.Morris RK, Malin GL, Quinlan-Jones E, et al. The PLUTO study and randomised controlled trial: evaluation of the effectiveness, cost-effectiveness and acceptability of percutaneous vesicoamniotic shunting for lower urinary tract obstruction. Health Technol Assess (in press). [DOI] [PMC free article] [PubMed]
  • 22.Anumba DO, Scott JE, Plant ND, Robson SC. Diagnosis and outcome of fetal lower urinary tract obstruction in the northern region of England. Prenat Diagn. 2005;25:7–13. doi: 10.1002/pd.1074. [DOI] [PubMed] [Google Scholar]
  • 23.Magann EF, Sanderson M, Martin JN, Chauhan S. The amniotic fluid index, single deepest pocket, and two-diameter pocket in normal human pregnancy. Am J Obstet Gynecol. 2000;182:1581–1588. doi: 10.1067/mob.2000.107325. [DOI] [PubMed] [Google Scholar]
  • 24.Chitty LS, Altman DG. Charts of fetal size: kidney and renal pelvis measurements. Prenat Diagn. 2003;23:891–897. doi: 10.1002/pd.693. [DOI] [PubMed] [Google Scholar]
  • 25.Fernbach SK, Maizels M, Conway JJ. Ultrasound grading of hydronephrosis: introduction to the system used by the Society for Fetal Urology. Pediatr Radiol. 1993;23:478–480. doi: 10.1007/BF02012459. [DOI] [PubMed] [Google Scholar]
  • 26.Johnson MP, Corsi P, Bradfield W. Sequential urinalysis improves evaluation of fetal renal function in obstructive uropathy. Am J Obstet Gynecol. 1995;173:59–65. doi: 10.1016/0002-9378(95)90170-1. [DOI] [PubMed] [Google Scholar]
  • 27.Morris RK, Malin GL, Khan KS, Kilby MD. Systematic review of the effectiveness of antenatal intervention for the treatment of congenital lower urinary tract obstruction. BJOG. 2010;117:382–390. doi: 10.1111/j.1471-0528.2010.02500.x. [DOI] [PubMed] [Google Scholar]
  • 28.NICE . Fetal vesico–amniotic shunt for lower urinary tract outflow obstruction (interventional procedure guidance 202) National Institute of Health and Clinical Excellence; London: 2006. http://www.nice.org.uk/IPG202 (accessed Dec 22, 2012). [Google Scholar]

RESOURCES