Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Feb 27.
Published in final edited form as: JAMA. 2015 Jan 20;313(3):255–263. doi: 10.1001/jama.2014.17985

Trends in Use of and Reproductive Outcomes Associated With Intracytoplasmic Sperm Injection

Sheree L Boulet Dr 1, Akanksha Mehta 1, Dmitry M Kissin 1, Lee Warner 1, Jennifer F Kawwass 1, Denise J Jamieson 1
PMCID: PMC4343214  NIHMSID: NIHMS661797  PMID: 25602996

Abstract

IMPORTANCE

Intracytoplasmic sperm injection (ICSI) is increasingly used in patients without severe male factor infertility without clear evidence of a benefit over conventional in vitro fertilization (IVF).

OBJECTIVE

To assess national trends and reproductive outcomes for fresh IVF cycles (embryos transferred without being frozen) following the use of ICSI compared with conventional IVF with respect to clinical indications for ICSI use.

DESIGN, SETTING, AND POPULATION

Retrospective cohort study using data on fresh IVF and ICSI cycles reported to the US National Assisted Reproductive Technology Surveillance System during 1996-2012.

MAIN OUTCOMES AND MEASURES

Trends in ICSI use during 1996-2012 with respect to male factor infertility, unexplained infertility, maternal age 38 years or older, low oocyte yield, and 2 or more prior assisted reproductive technology cycles; reproductive outcomes for conventional IVF and ICSI cycles during 2008-2012, stratified by the presence or absence of male factor infertility.

RESULTS

Of the 1 395 634 fresh IVF cycles from 1996 through 2012, 908 767 (65.1%) used ICSI and 499 135 (35.8%) reported male factor infertility. Among cycles with male factor infertility, ICSI use increased from 76.3% (10 876/14 259) to 93.3% (32 191/34 506) (P < .001) during 1996-2012; for those without male factor infertility, ICSI use increased from 15.4% (4197/27 191) to 66.9% (42 321/63 250) (P < .001). During 2008-2012, male factor infertility was reported for 35.7% (176 911/494 907) of fresh cycles. Among those cycles, ICSI use was associated with a lower multiple birth rate compared with conventional IVF (30.9% vs 34.2%; adjusted relative risk [RR], 0.87; 95% CI, 0.83-0.91). Among cycles without male factor infertility (n = 317 996), ICSI use was associated with lower rates of implantation (23.0% vs 25.2%; adjusted RR, 0.93; 95% CI, 0.91-0.95), live birth (36.5% vs 39.2%; adjusted RR, 0.95; 95% CI, 0.93-0.97), and multiple live birth (30.1% vs 31.0%; adjusted RR, 0.93; 95% CI, 0.91-0.95) vs conventional IVF.

CONCLUSIONS AND RELEVANCE

Among fresh IVF cycles in the United States, ICSI use increased from 36.4% in 1996 to 76.2% in 2012, with the largest relative increase among cycles without male factor infertility. Compared with conventional IVF, ICSI use was not associated with improved postfertilization reproductive outcomes, irrespective of male factor infertility diagnosis.


The introduction of intracytoplasmic sperm injection (ICSI) in 1992 revolutionized the treatment of couples with male factor infertility and made paternity possible for a large proportion of men with nonobstructive azoospermia, or no measurable sperm count.1,2 Over the past 2 decades, the use of ICSI for patients with borderline or even normal semen characteristics has increased,3 without clear evi dence of a benefit to using ICSI over conventional in vitro fertilization (IVF).4-6 The Practice Committees of the American Society for Reproductive Medicine and the Society for Assisted Reproductive Technology concluded that there is insufficient evidence to support the routine use of ICSI in patients without male factor infertility.7 Although ICSI may have a role in IVF cycles using preimplantation genetic testing, in vitro maturation, or previously cryopreserved oocytes, the routine use of ICSI for these indications requires further investigation.7

In contrast to conventional IVF, ICSI bypasses natural barriers to fertilization, thereby increasing the possibility of the transmission of genetic defects from one generation to the next. Pregnancies resulting from the use of ICSI have been associated with 1.5 to 4 times increased incidences of chromosomal abnormalities,8,9 imprinting disorders,10 autism,11 intellectual disabilities,11 and birth defects12,13 compared with pregnancies resulting from conventional IVF. These increased risks may be related to the effects of underlying male or female sub-fertility, other medical factors present in couples who are candidates for ICSI, or the ICSI procedure.

Intracytoplasmic sperm injection is also considerably more expensive than conventional IVF and adds to financial burdens already experienced by many couples undergoing fertility treatment.14,15 The higher reimbursement associated with ICSI has been postulated as one possible reason for the increasing use of this technology.

The aim of this study was to assess national trends and reproductive outcomes of fresh IVF cycles associated with the use of ICSI compared with conventional IVF with respect to clinical indications for ICSI use.

Methods

All data used in this study were derived from the National Assisted Reproductive Technology Surveillance System (NASS), a data reporting system for the federally mandated collection of information on all assisted reproductive technology (ART) cycles performed in the United States.16 In NASS, ART cycles are defined as fertility treatments in which eggs and sperm or embryos are handled (manipulated) for the purpose of establishing a pregnancy. NASS includes cycle-level information on patient characteristics, clinical characteristics of the ART procedure, and pregnancy outcomes. Multiple cycles among individual patients are not linked. NASS captures information from an estimated 97% of ART cycles performed annually.17 Each year, 7% to 10% of reporting clinics are randomly selected for validation and their reported data are compared with medical records. Discrepancy rates are calculated and are less than 5% with the exception of the following infertility diagnoses: diminished ovarian reserve (8.4%), other factor (9.5%), and unknown factor (6.5%).17

Because information on ICSI use is not consistently collected across clinics for frozen embryo cycles or cycles canceled prior to oocyte retrieval (ovarian stimulation or monitoring was initiated but cycle did not proceed to oocyte retrieval), we restricted our analysis to all fresh (embryos transferred without being frozen) conventional IVF and ICSI cycles performed from 1996 through 2012 in which oocyte retrieval was attempted. We used linear regression models to assess trends in the use of ICSI for all fresh cycles and for those with the following indications: male factor infertility (infertility due to abnormal semen characteristics, abnormal sperm function, or surgical sterilization), unexplained infertility (infertility with unidentified etiology), female patient aged 38 years or older, 2 or more prior ART cycles and no prior live birth, low oocyte yield (<5 oocytes retrieved), and use of preimplantation genetic testing. Annual ICSI rate was the dependent variable and year was the continuous predictor. Data collection for preimplantation genetic testing was implemented in 2004; thus, we evaluated this factor only for 2004 through 2012.

To account for advances in ICSI techniques and technology, we subsequently restricted the analysis to the 5 most recent years (2008-2012) and evaluated the association between ICSI and reproductive outcomes. We compared the distribution of patient and clinical characteristics between cycles using conventional IVF and ICSI, stratified by male factor and non– male factor infertility. The characteristics assessed in this study included female patient age, race/ethnicity (as reported by clinics), infertility diagnosis, number of prior live births, number of prior spontaneous abortions, number of prior ART cycles (includes prior fresh and frozen cycles), use of donor egg or embryo, use of donor sperm (including cycles using only donor sperm or mixed patient and donor sperm), number of oocytes retrieved, number of embryos transferred, embryo stage at transfer, number of embryos cryopreserved, use of assisted hatching (the purposeful disruption of an embryo's zona pellucida by laser, mechanical, or chemical means), and use of genetic testing. Race/ethnicity was assessed for reported variations in IVF birth outcomes. NASS does not collect information on fertilization rates; therefore, we indirectly assessed rates of failed fertilization by calculating the percentage of cycles cancelled between retrieval and transfer for cycles using conventional IVF or ICSI, stratified by male factor and non–male factor infertility. We also compared rates of implantation, clinical intrauterine pregnancy, live birth, miscarriage, multiple live birth, preterm delivery (<37 weeks’ gestation), and low birth weight (<2500 g) for each strata.

To account for potential confounding by factors associated with ICSI use, we estimated propensity scores using logistic regression models with ICSI as the outcome and included all baseline covariates that may predict probability of treatment selection (age, infertility diagnosis, number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, use of donor egg or embryo, use of donor sperm, and number of oocytes retrieved). Backward selection with a significance level of P<.05 was used to determine the final models. Separate propensity score models were estimated for cycles with and without male infertility. Because covariate adjustment using propensity scores produces unbiased estimates of rate ratios, we included the estimated propensity scores in all outcome models.18 We used robust Poisson regression models with generalized estimating equations for clustering by clinic to estimate unadjusted and adjusted risk ratios for the association between the use of ICSI and reproductive outcomes. The multivariable models included the aforementioned patient and clinical characteristics except race/ ethnicity because of a high percentage (39.7%) of missing information. The models for cycle cancellation did not include assisted hatching, number of embryos transferred, and embryo stage at transfer because this information is not available for canceled cycles. Data were missing for less than 2% of all other covariates.

We also compared reproductive outcomes for conventional IVF and ICSI for subgroups with selected indications including unexplained infertility, age 38 years or older, 2 or more prior ART cycles and no prior live birth, low oocyte yield, and use of genetic testing. All models included propensity scores derived from indication-specific logistic regression models using backward selection with a significance level of P<.05.

For bivariable comparisons, we used Pearson χ2 tests and applied the Bonferroni method to control the familywise error rate due to multiple comparisons. We considered each stratum a “family” and multiplied the P values by 20. For the multivariate models, we also used the Bonferroni method to adjust the P values for the 8 outcomes assessed within each indication. A2-tailed P<.05 was considered statistically significant. SAS version 9.3 was used for all analyses. The study was approved by the Centers for Disease Control and Prevention's institutional review board. A waiver of informed consent was obtained.

Results

NASS captured data on a total of 1 395 634 fresh IVF cycles from 1996 through 2012. Of these, 908 767 cycles (65.1%) used ICSI and 486 867 cycles (34.9%) used conventional IVF. Male factor infertility was reported for 499 135 cycles (35.8%) while no male factor diagnosis was reported for 896 499 cycles (64.2%). Overall, ICSI was used in 90.0% of cycles with male factor infertility and in 51.2% of cycles without male factor infertility.

The proportion of fresh IVF cycles using ICSI increased from 36.4% (15 073/41 450) in 1996 to 76.2% (74 512/97 756) in 2012 (Figure 1). Among cycles with a diagnosis of male factor infertility, ICSI use increased from 76.3% (10 876/14 259) to 93.3% (32 191/34 506) (P<.001). Among cycles with non–male factor infertility, ICSI use increased from 15.4% (4197/27 191) to 66.9% (42 321/63 250) (P<.001) during the study period.

Figure 1.

Figure 1

Use of ICSI Among Fresh IVF Cycles With and Without Male Factor Infertility, 1996-2012

ICSI indicates intracytoplasmic sperm injection; IVF, in vitro fertilization.

During 1996 through 2012, ICSI use increased significantly for all selected non–male factor indications, as well as for cycles without any indication (Figure 2). From 2004 onward, when data on preimplantation genetic testing was available, the use of ICSI was highest when preimplantation genetic testing was used.

Figure 2.

Figure 2

Use of ICSI Among Fresh IVF Cycles With Non–Male Factor Infertility by Type of Indication, 1996-2012

ART indicates assisted reproductive technology; ICSI, intracytoplasmic sperm injection; IVF, in vitro fertilization.

During 2008-2012, there were 494 907 fresh IVF cycles, 74.6% of which used ICSI (Table 1). Male factor infertility was identified in 35.7% of the cycles. Intracytoplasmic sperm injection was used in 92.9% of cycles with male factor infertility and in 64.5% of cycles without male factor infertility.

Table 1.

Characteristics of Fresh IVF Cycles With and Without ICSI by Male Factor Infertility Diagnosis, 2008-2012

Characteristics Male Factor Infertility, No. (%) Non-Male Factor Infertility, No. (%)
Conventional IVF (n = 12 648) ICSI (n = 164 263) Conventional IVF (n = 112 877) ICSI (n = 205 119)
Female patient age, y
    <30 1432 (11.3) 25 116 (15.3)a 11 658 (10.3) 19 170 (9.4)a
    30-34 3644 (29.8) 54 455 (33.2) 31 923 (28.3) 52 372 (25.5)
    35-39 4554 (36.0) 55 589 (33.8) 39 016 (34.6) 68 988 (33.6)
    ≥40 3018 (23.9) 29 103 (17.7) 30 280 (26.8) 64 589 (31.5)
Race/ethnicity
    Non-Hispanic white 5800 (45.9) 79 142 (48.2)a 48 165 (42.7) 90 757 (44.3)a
    Non-Hispanic black 489 (3.9) 6902 (4.2) 5202 (4.6) 9662 (4.7)
    Hispanic 796 (6.3) 9975 (6.1) 5341 (4.7) 12 661 (6.2)
    Asian/Pacific Islander 1100 (8.9) 12 719 (7.7) 8746 (7.8) 18 353 (9.0)
    Other 20 (0.2) 224 (0.1) 176 (0.2) 373 (0.2)
    Missing 4413 (34.9) 55 301 (33.7) 45 247 (40.1) 73 313 (35.7)
Infertility diagnosis
    Tubal factor 1731 (13.7) 14 239 (8.7)a 24 700 (21.9) 33 998 (16.6)a
    Endometriosis 1164 (9.2) 13 335 (7.5)a 13 884 (12.3) 23 744 (11.6)a
    Uterine factor 636 (5.0) 6426 (3.9)a 5819 (5.2) 10 650 (5.2)
    Ovulatory disorder 1822 (14.4) 19 820 (12.1) 17 273 (15.3) 28 321 (13.8)a
    Diminished ovarian reserve 2945 (23.3) 32 981 (20.1)a 30 570 (27.1) 70 680 (34.5)a
    Unexplained 0 0 25 253 (22.4) 38 820 (18.9)a
    Other 1074 (8.5) 13 371 (7.5)a 15 783 (14.0) 39 497 (19.3)a
No. of prior live births
    0 9007 (71.6) 121 049 (74.0)a 77 326 (68.8) 143 533 (70.5)a
    1 2747 (21.8) 33 363 (20.4) 25 501 (22.7) 42 171 (20.7)
    ≥2 827 (6.6) 9142 (5.6) 9571 (8.5) 18 029 (8.9)
No. of prior spontaneous abortions
    0 8490 (67.7) 121 786 (74.6)a 71 580 (63.8) 132 756 (65.3)a
    1 2720 (21.7) 29 151 (17.9) 24 230 (21.6) 42 391 (20.9)
    ≥2 1336 (10.7) 12 265 (7.5) 16 472 (14.7) 28 085 (13.8)
No. of prior ART cycles
    0 7210 (57.0) 89 783 (54.7)a 66 891 (59.3) 109 477 (53.4)a
    1 2447 (19.4) 32 998 (20.1) 20 922 (18.5) 40 546 (19.8)
    ≥2 2984 (23.6) 41 448 (25.2) 25 022 (22.2) 55 017 (26.8)
Oocyte/embryo source
    Nondonor 11 737 (92.8) 155 777 (94.8)a 102 156 (90.5) 176 795 (86.2)a
    Donor 911 (7.2) 8486 (5.2) 10 721 (9.5) 28 324 (13.8)
Sperm donor usedb
    Yes 2089 (16.6) 6898 (4.2)a 5191 (4.6) 10 986 (5.4)a
    No 10 535 (83.5) 157 330 (95.8) 107 630 (95.4) 194 038 (94.6)
No. of oocytes retrieved
    0-4 1823 (14.6) 16755 (10.3)a 13 711(12.3) 29 233 (14.4)a
    5-9 3308 (26.4) 42 870 (26.2) 30 530 (27.4) 53 358 (25.9)
    10-20 5559 (44.4) 77 200 (47.3) 50 153 (44.9) 86 853 (42.9)
    ≥21 1826 (14.6) 26 535 (16.2) 17 200 (15.4) 33 964(16.8)
No. of embryos transferred
    0-1 1629 (14.9) 21 544 (14.0)a 16 353 (15.7) 30 998 (16.5)a
    2 6066 (55.5) 87 545 (56.9) 57 138 (55.0) 99 253 (52.7)
    3 2192 (20.1) 31 732 (20.6) 20 387 (19.6) 38 306 (20.4)
    ≥4 1047 (9.6) 13 170 (8.6) 10 021 (9.6) 19 682 (10.5)
    Missingc 1714 10 272 8978 16 880
Embryo stage
    Day 3 5902 (54.0) 76 110 (49.4)a 55 466 (53.4) 88 525 (47.0)a
    Day 5 4029 (36.9) 62 768 (40.8) 40 001 (38.5) 80 808 (42.9)
    Other 1002 (9.2) 15 090 (9.8) 8425 (8.1) 18 874 (10.0)
    Missingc 1715 10 295 8985 16 912
No. of embryos cryopreserved
    0 7981 (63.1) 95 579 (59.4)a 67 363 (59.7) 121 045 (59.0)a
    1-2 1744 (13.8) 25 792 (15.7) 16 614 (14.7) 30 602 (14.9)
    ≥3 2923 (23.1) 40 892 (24.9) 28 900 (25.6) 53 472 (26.1)
Assisted hatching usedd
    Yes 3306 (30.2) 61 552 (40.0)a 32 721 (31.5) 86 039 (45.7)a
    No 7627 (69.8) 92 416 (60.0) 71 178 (68.5) 102 200 (54.3)
    Missingc 1714 10 272 8978 16 880
Preimplantation genetic testing performed
    Yes 269 (2.2) 5022 (3.1) 3235 (2.9) 15 958 (7.9)a
    No 12 150 (97.8) 157 758 (96.9) 108 375 (97.1) 186 261 (92.1)

Abbreviations: ART, assisted reproductive technology; ICSI, intracytoplasmic sperm injection; IVF, in vitro fertilization.

a

P<.05 by χ2 test. P values are adjusted for the 2O characteristics assessed in the stratum using Bonferroni correction.

b

Sperm donor includes cycles using only donor sperm or mixed patient and donor sperm.

c

Missing percentages greater than 8% due to cycles that were canceled between retrieval and transfer and therefore did not report this information.

d

Assisted hatching defined as the purposeful disruption of an embryo's zona pellucida by laser, mechanical, or chemical means.

Among couples with male factor infertility, the female partners among those undergoing ICSI were younger, were less likely to have concomitant female factor infertility diagnoses, and had fewer prior live births and more prior ART cycles compared with those undergoing conventional IVF (Table 1). Cycles using ICSI had a larger number of oocytes retrieved, more day 5 transfers, more embryos cryopreserved, and higher rates of assisted hatching than those using conventional IVF. The proportion of cycles using donor sperm was much lower for ICSI compared with conventional IVF (4.2% vs 16.6%).

Among couples with non–male factor infertility, the female partners of those undergoing ICSI were more likely to be older than 40 years, to have diminished ovarian reserve, to have undergone 1 or more prior ART cycles, and to be nulliparous compared with those undergoing conventional IVF (Table 1). Use of donor oocytes, day 5 embryo transfer, use of assisted hatching, and use of preimplantation genetic testing was higher among cycles with ICSI than with conventional IVF.

Table 2 shows reproductive outcomes following conventional IVF and ICSI, stratified by the presence or absence of male factor infertility and adjusted for maternal factors and ART treatment characteristics. We tested models with and without the inclusion of a race/ethnicity covariate and found that the magnitude and direction of association did not change significantly when the variable was included. Among couples with male factor infertility, the percentage of cycles canceled between oocyte retrieval and embryo transfer was lower for cycles using ICSI than for those using conventional IVF (6.3% vs 13.6%; adjusted relative risk [RR], 0.50; 95% CI, 0.45-0.56). However, there were no differences in pregnancy, miscarriage, or live birth rates for transfers using ICSI compared with conventional IVF. The adjusted relative risks for implantation (25.5% vs 25.6%; adjusted RR, 0.95; 95% CI, 0.91-0.98) and multiple live birth (30.9% vs 34.2%; adjusted RR, 0.87; 95% CI, 0.83-0.91) were significantly lower in those undergoing ICSI compared with conventional IVF.

Table 2.

Reproductive Outcomes for Conventional IVF and ICSI Among Fresh Cycles With and Without Male Factor Infertility, 2008-2012a

Outcomes Conventional IVF ICSI Relative Risk (95% CI) P Valueb
Total No. No. (%) With Outcome Total No. No. (%) With Outcome Unadjusted Adjusted
Male Factor Infertility
Cycle canceled before transferc 12 648 1715 (13.6) 164 263 10 295 (6.3) 0.47 (0.42-0.54) 0.50 (0.45-0.56)d <.001
Among transfers
    Implantation ratee 22 886 5863 (25.6) 321 419 82 006 (25.5) 0.94 (0.88-1.00) 0.95 (0.91-0.98)f .02
    Clinical intrauterine pregnancy 10 933 5232 (47.9) 153 968 73 850 (48.0) 0.97 (0.93-1.02) 0.98 (0.95-1.01)f >.99
    Live birth 10 933 4296 (39.3) 153 968 61 450 (39.9) 0.97 (0.92-1.03) 0.98 (0.95-1.02)f >.99
Among pregnancies
    Miscarriage 5232 839 (16.0) 73 850 10 946 (14.8) 1.00 (0.92-1.08) 0.97 (0.91-1.04)f >.99
Among live births
    Multiple live birth 4296 1469 (34.2) 61 450 19 002 (30.9) 0.88 (0.84-0.94) 0.87 (0.83-0.91)f <.001
    Preterm delivery 4287 1250 (29.2) 61 347 16 822 (27.4) 0.94 (0.88-1.01) 0.93 (0.88-0.98)f .06
    Low birth weight in any infant 4230 1242 (29.4) 60 273 16 936 (28.1) 0.95 (0.89-1.00) 0.93 (0.88-0.98)f .06
Non-Male Factor Infertility
Cycle canceled before transferc 112 877 8985 (8.0) 205 119 16 911 (8.2) 1.03 (0.93-1.13) 0.88 (0.81-0.97)d .06
Among transfers
    Implantation ratee 216 125 54 362 (25.2) 395 054 90 987 (23.0) 0.92 (0.88-0.97) 0.93 (0.91-0.95)f <.001
    Clinical pregnancy 103 892 49 732 (47.9) 188 208 84 578 (44.9) 0.94 (0.91-0.97) 0.95 (0.93-0.97)f <.001
    Live birth 103 892 40 703 (39.2) 188 208 68 735 (36.5) 0.93 (0.90-0.97) 0.95 (0.93-0.97)f <.001
Among pregnancies
    Miscarriage 49 732 7921 (15.9) 84 578 13 978 (16.5) 1.03 (0.99-1.07) 1.03 (1.00-1.06)f .97
Among live births
    Multiple live birth 40 703 12 633 (31.0) 68 735 20 671 (30.1) 0.97 (0.93-1.00) 0.93 (0.91-0.95)f <.001
    Preterm delivery 40 642 11 490 (28.3) 68 533 19 324 (28.2) 1.00 (0.96-1.03) 0.97 (0.95-1.00)f .30
    Low birth weight in any infant 39 999 11 415 (28.5) 67 178 19 087 (28.4) 1.00 (0.96-1.03) 0.96 (0.94-0.99)f .03

Abbreviations: ICSI, intracytoplasmic sperm injection; IVF, in vitro fertilization.

a

All models included generalized estimating equations to account for clustering by clinic. Conventional IVF was the reference for all comparisons.

b

P values are adjusted for the 8 outcomes assessed in the stratum using Bonferroni correction.

c

Cycle canceled between retrieval and transfer.

d

Male factor infertility model was adjusted for maternal age, number of prior live births, number of prior spontaneous abortions, number of prior assisted reproductive technology cycles, number of oocytes retrieved, number of embryos cryopreserved, donor egg/embryo, donor sperm, use of preimplantation genetic testing, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, and diminished ovarian reserve). Non–male factor infertility model was adjusted for all of the above and for unexplained infertility.

e

Calculated as the number of embryos implanted divided by the total number of embryos transferred; if number of fetal heartbeats and number of live and stillborn infants was missing, then implantation rate was considered missing.

f

Male factor infertility models were adjusted for maternal age, number of prior live births, number of prior spontaneous abortions, number of prior assisted reproductive technology cycles, number of oocytes retrieved, number of embryos transferred, embryo stage at transfer, number of embryos cryopreserved, use of assisted hatching, donor egg/embryo, donor sperm, use of preimplantation genetic testing, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, and diminished ovarian reserve).

Among couples without male factor infertility, adjusted RRs for implantation (23.0% vs 25.2%; adjusted RR, 0.93; 95% CI, 0.91-0.95), pregnancy (44.9% vs 47.9%; adjusted RR, 0.95; 95% CI, 0.93-0.97), and live birth (36.5% vs 39.2%; adjusted RR, 0.95; 95% CI, 0.93-0.97) were significantly lower among cycles using ICSI compared with conventional IVF. The adjusted RRs for multiple birth (30.1% vs 31.0%; adjusted RR, 0.93; 95% CI, 0.91-0.95) and low birth weight (28.4% vs 28.5%; adjusted RR, 0.96; 95% CI, 0.94-0.99) were also lower in those undergoing ICSI compared with conventional IVF (Table 2).

When reproductive outcomes were examined among selected non–male factor indications for ICSI, the direction and magnitude of the adjusted RRs for the association between ICSI and conventional IVF followed patterns similar to those for the non–male factor infertility group as a whole (Table 3). Overall, cycles using ICSI tended to have lower cancellation, implantation, pregnancy, live birth, and multiple birth rates compared with cycles using conventional IVF, irrespective of the underlying indication.

Table 3.

Reproductive Outcomes for Conventional IVF and ICSI Among Fresh IVF Cycles With Selected Non-Male Factor Infertility Indications, 2008-2012a

Outcomes Conventional IVF ICSI Relative Risk (95% CI) P Valueb
Total No. No. (%) With Outcome Total No. No. (%) With Outcome Unadjusted Adjusted
Unexplained Infertility
Cycle canceled before transferc 25 253 1570 (6.2) 38 820 2160 (5.6) 0.91 (0.80-1.04) 0.84 (0.75-0.94)d .04
Among transfers
    Implantation rate 49 325 12 467 (25.2) 78 208 18 674 (23.9) 0.93 (0.85-1.00) 0.94 (0.90-0.98)e .02
    Clinical pregnancy 23 683 11 392 (48.1) 36 660 17 064 (46.6) 0.96 (0.90-1.01) 0.95 (0.91-0.99)e .09
    Live birth 23 683 9467 (40.0) 36 660 14 176 (38.7) 0.95 (0.89-1.01) 0.95 (0.92-0.99)e .09
Among pregnancies
    Miscarriage 11 392 1711 (15.0) 17 064 2537 (14.9) 1.02 (0.96-1.08) 1.01 (0.96-1.07)e >.99
Among live births
    Multiple live birth 9467 2758 (29.1) 14 176 4211 (29.7) 1.00 (0.95-1.06) 0.93 (0.90-0.97)e .001
    Preterm delivery 9455 2313 (24.5) 14 158 3614 (25.5) 1.04 (0.98-1.10) 0.98 (0.93-1.03)e >.99
    Low birth weight in any infant 9320 2398 (25.7) 13 879 3743 (27.0) 1.04 (0.99-1.10) 0.98 (0.94-1.02)e >.99
≥2 Prior ART Cycles and No Prior Live Births
Cycle canceled before transferc 14 077 1160 (8.2) 33 512 2808 (8.4) 1.01 (0.85-1.19) 0.92 (0.81-1.03)f >.99
Among transfers
    Implantation rate 30 621 5798 (18.9) 70 590 12 850 (18.2) 0.92 (0.86-0.98) 0.92 (0.89-0.95)g <.001
    Clinical pregnancy 12 917 5402 (41.8) 30 704 12 262 (39.9) 0.93 (0.89-0.97) 0.94 (0.92-0.97)g <.001
    Live birth 12 917 4292 (33.2) 30 704 9647 (31.4) 0.92 (0.87-0.96) 0.93 (0.90-0.96)g <.001
Among pregnancies
    Miscarriage 5402 985 (18.2) 12 262 2379 (19.4) 1.09 (1.01-1.17) 1.09 (1.02-1.17)g .10
Among live births
    Multiple live birth 4292 1371 (31.9) 9647 2967 (30.8) 0.94 (0.87-1.01) 0.93 (0.88-0.99)g .14
    Preterm delivery 4286 1296 (30.2) 2869 2869 (29.8) 0.98 (0.91-1.05) 0.97 (0.91-1.04)g >.99
    Low birth weight in any infant 4184 1285 (30.7) 9406 2822 (30.0) 0.98 (0.90-1.05) 0.96 (0.90-1.02)g >.99
Advanced Maternal Age (≥38 y)
Cycle canceled before transferc 46 058 4349 (9.4) 93 488 8747 (9.4) 0.98 (0.87-1.11) 0.86 (0.78-0.94)h .01
Among transfers
    Implantation rate 99 496 16 057 (16.1) 193 476 31 480 (16.3) 0.93 (0.86-1.00) 0.92 (0.90-0.95)i <.001
    Clinical pregnancy 41 709 16 606 (39.8) 84 741 32 333 (38.2) 0.93 (0.88-0.97) 0.94 (0.92-0.96) <.001
    Live birth 41 709 12 375 (29.7) 84 741 24 427 (28.8) 0.92 (0.87-0.98) 0.93 (0.91-0.96)i <.001
Among pregnancies
    Miscarriage 16 606 3794 (22.9) 32 333 7146 (22.1) 01.02 (0.98-1.07) 1.02 (0.99-1.07)i >.99
Among live births
    Multiple live birth 12 375 3428 (27.7) 24 427 6606 (27.0) 0.93 (0.88-0.98) 0.92 (0.89-0.96)i <.001
    Preterm delivery 12 350 3234 (26.2) 24 359 6718 (27.6) 1.02 (0.97-1.07) 1.01 (0.97-1.06)i >.99
    Low birth weight in any infant 12 087 3140 (26.0) 23 840 6340 (26.6) 0.99 (0.94-1.04) 0.98 (0.94-1.02)i >.99
Low Oocyte Yield (<5 Oocytes)
Cycle canceled before transferc 13 711 3430 (25.0) 29 233 5624 (19.2) 0.76 (0.69-0.83) 0.74 (0.66-0.83)j <.001
Among transfers
    Implantation rate 17 905 2476 (13.8) 38 982 4401 (11.3) 0.85 (0.79-0.92) 0.86 (0.81-0.91)k <.001
    Clinical pregnancy 10 281 2778 (27.0) 23 609 5264 (22.3) 0.85 (0.80-0.91) 0.89 (0.84-0.93)k <.001
    Live birth 10 281 2085 (20.3) 23 609 3758 (15.9) 0.82 (0.77-0.88) 0.86 (0.81-0.91)k <.001
Among pregnancies
    Miscarriage 2778 627 (22.6) 5264 1319 (25.1) 1.06 (0.98-1.15) 1.06 (0.98-1.15)k >.99
Among live births
    Multiple live birth 2085 362 (17.4) 3758 568 (15.1) 0.88 (0.76-1.01) 0.90 (0.79-1.02)k .78
    Preterm delivery 2077 412 (19.8) 3743 715 (19.1) 0.95 (0.85-1.07) 0.97 (0.86-1.08)k >.99
    Low birth weight in any infant 2048 375 (18.3) 3676 678 (18.4) 1.01 (0.90-1.12) 1.02 (0.92-1.14)k >.99
Preimplantation Genetic Testing Used
Cycle canceled before transferc 3235 727 (22.5) 15 958 3458 (21.7) 1.02 (0.88-1.19) 1.01 (0.87-1.17)l >.99
Among transfers
    Implantation rate 4190 1188 (28.4) 20 723 6148 (29.7) 0.99 (0.88-1.11) 0.97 (0.88-1.07)m >.99
    Clinical pregnancy 2508 1132 (45.1) 12 500 5807 (46.5) 0.99 (0.91-1.07) 0.98 (0.90-1.06)m >.99
    Live birth 2508 949 (37.8) 12 500 4855 (38.8) 0.97 (0.88-1.08) 0.96 (0.87-1.06)m >.99
Among pregnancies
    Miscarriage 1132 155 (13.7) 5807 828 (14.3) 1.13 (0.91-1.39) 1.13 (0.91-1.40)m >.99
Among live births
    Multiple live birth 949 220 (23.2) 4855 1213 (25.0) 1.07 (0.93-1.22) 1.04 (0.93-1.17)m >.99
    Preterm delivery 949 193 (20.3) 4841 1154 (23.8) 1.15 (1.02-1.30) 1.13 (1.00-1.27)m .37
    Low birth weight in any infant 930 194 (20.9) 4723 109 (23.2) 1.09 (0.96-1.25) 1.07 (0.93-1.23)m >.99

Abbreviations: ART, assisted reproductive technology; ICSI, intracytoplasmic sperm injection; IVF, in vitro fertilization.

a

All models included generalized estimating equations to account for clustering by clinic. Conventional IVF was the reference for all comparisons.

b

P values are adjusted for the 8 outcomes assessed in the stratum using Bonferroni correction.

c

Cycle canceled between retrieval and transfer.

d

Model adjusted for maternal age, number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, number of oocytes retrieved, number of embryos cryopreserved, donor egg/embryo, donor sperm, and use of preimplantation genetic testing.

e

Models adjusted for maternal age, number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, number of oocytes retrieved, number of embryos transferred, embryo stage at transfer, number of embryos cryopreserved, use of assisted hatching, donor egg/embryo, donor sperm, and use of preimplantation genetic testing.

f

Model adjusted for maternal age, number of prior spontaneous abortions, number of oocytes retrieved, number of embryos cryopreserved, donor egg/embryo, donor sperm, use of preimplantation genetic testing, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, diminished ovarian reserve, and unexplained).

g

Models adjusted for maternal age, number of prior spontaneous abortions, number of oocytes retrieved, number of embryos transferred, embryo stage at transfer, number of embryos cryopreserved, use of assisted hatching, donor egg/embryo, donor sperm, use of preimplantation genetic testing, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, diminished ovarian reserve, and unexplained).

h

Model adjusted for number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, number of oocytes retrieved, number of embryos cryopreserved, donor egg/embryo, donor sperm, use of preimplantation genetic testing, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, diminished ovarian reserve, and unexplained).

i

Models adjusted for number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, number of oocytes retrieved, number of embryos transferred, embryo stage at transfer, number of embryos cryopreserved, use of assisted hatching, donor egg/embryo, donor sperm, use of preimplantation genetic testing, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, diminished ovarian reserve, and unexplained).

j

Model adjusted for maternal age, number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, number of embryos cryopreserved, donor egg/embryo, donor sperm, use of preimplantation genetic testing, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, diminished ovarian reserve, and unexplained).

k

Models adjusted for maternal age, number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, number of embryos transferred, embryo stage at transfer, number of embryos cryopreserved, use of assisted hatching, donor egg/embryo, donor sperm, use of preimplantation genetic testing, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, diminished ovarian reserve, and unexplained).

l

Model adjusted for maternal age, number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, number of oocytes retrieved, number of embryos cryopreserved, donor egg/embryo, donor sperm, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, diminished ovarian reserve, and unexplained).

m

Models adjusted for maternal age, number of prior live births, number of prior spontaneous abortions, number of prior ART cycles, number of oocytes retrieved, number of embryos transferred, embryo stage at transfer, number of embryos cryopreserved, use of assisted hatching, donor egg/embryo, donor sperm, and infertility diagnosis (tubal factor, endometriosis, uterine factor, ovulatory disorder, diminished ovarian reserve, and unexplained).

Discussion

The results of this analysis demonstrate a steady increase in the proportion of ART cycles involving ICSI performed in the United States from 1996 through 2012. The use of ICSI doubled during the study period, from 36.4% to 76.2% of all fresh IVF cycles, with the greatest increase occurring in cycles without male factor infertility. When male factor infertility was present, reproductive outcomes—including pregnancy, miscarriage, and live birth rates—were comparable for cycles that used ICSI vs conventional IVF after adjustment for maternal factors.Notably,the likelihood of cycle cancellation between retrieval and transfer, a surrogate measure of failed fertilization, was markedly decreased for cycles where ICSI was used compared with those using conventional IVF, thereby confirming that ICSI increases the likelihood of fertilization in the context of male factor infertility. In contrast, implantation rates were also lower when ICSI was used and likely contributed to the significantly lower rates of multiple live births. In the absence of male factor infertility, ICSI use was associated with small but statistically significant decreases in implantation, pregnancy, live birth, multiple live birth, and low birth weight rates compared with conventional IVF. Although such differences may be a function of the large sample size and thus not clinically relevant, our findings suggest that use of ICSI may improve fertilization rates but not implantation or pregnancy rates in the setting of unexplained infertility, advanced maternal age, and low oocyte yield.

The findings of this study are consistent with results of a previous study showing an increase in the use of ICSI from 1995 through 2004 despite stable rates of male factor infertility diagnosis.3 The increasing trend in ICSI use in the United States parallels the trend seen globally, although the relative frequency with which ICSI is used varies markedly among countries.19

We found that ICSI use increased in the absence of any indication,thereby suggesting that the decision to perform ICSI instead of conventional IVF was likely being influenced by other unmeasured factors. The Practice Committees of the American Society for Reproductive Medicine and the Society for Assisted Reproductive Technology statement that the use of ICSI for unexplained infertility, low oocyte yield, and advanced maternal age did not improve clinical outcomes was released only recently, in 20127; it remains to be seen how this statement might affect the use of ICSI for non–male factor indications in the future.

Our results demonstrated no improvement in postfertilization reproductive outcomes with use of ICSI over conventional IVF in the absence of male factor infertility, regardless of the underlying indication for use. On the contrary, reproductive outcomes were slightly poorer when ICSI was used in non–male factor cases. We found that use of ICSI was associated with lower rates of multiple birth regardless of whether male factor infertility was present; however, this finding may be due to lower implantation rates in cycles where ICSI was used. Our findings were consistent with those of a previous randomized clinical trial, which found only marginal increases in implantation and pregnancy rates in couples undergoing conventional IVF compared with ICSI in the absence of male factor infertility, leading the authors to conclude that ICSI offered no clinical advantage over conventional IVF in cases of non–male factor infertility.6

Statistically significant findings reported in the current study might have reflected the large sample size and might lack clinical relevance. It has been suggested that relative risks of 0.5 or less in observational studies warrant further investigation while those between 0.5 and 1 are likely due to selection bias and residual confounding.20 Alternatively, by bypassing natural selection, ICSI use could have resulted in poorer-quality embryos compared with conventional IVF, leading to poorer reproductive outcomes. Another possibility is that ICSI use was warranted in some of the non–male factor cases for reasons that were not captured by this study—for example, poor sperm or egg quality—which could have led to poorer reproductive outcomes. Last, ICSI could be used as a rescue measure for fertilizing oocytes that have failed to be fertilized with conventional IVF.21 Because primary failure of fertilization might be secondary to poor oocyte quality, the use of ICSI in this clinical setting could lead to an association with poorer reproductive outcomes.

The primary limitation of this study was that NASS does not collect information on fertilization rates, for which ICSI is expected to be advantageous compared with conventional IVF. Although we were able to indirectly assess fertilization failure using the number of cycles canceled following oocyte retrieval, this measure assumes that cycles not proceeding to embryo transfer represent those in which all oocytes failed to fertilize, which may not necessarily be true. Second, although it has been suggested that ICSI might be beneficial compared with conventional IVF for cryopreserved oocytes, NASS does not collect this information. Third, there is no direct measure of embryo quality in NASS data, which could have modified the reproductive outcomes of interest. Furthermore, we lacked details on male factor evaluation. Although the diagnosis of male factor infertility typically includes men with fewer than 500 000 total motile sperm present in the ejaculate or less than 4% of normal morphologic forms based on strict criteria, it is unknown whether all centers uniformly adhere to this definition. In addition, NASS data are cycle based, and we were unable to link cycles for patients undergoing more than 1 cycle. We also restricted the analysis to fresh cycles; however, it is unlikely that findings for frozen-thawed embryos would differ from those for fresh cycles since ICSI rates would be similar. Finally, our observational, retrospective cohort study may be subject to bias because of uncontrolled confounding.

An important strength of this study is the use of NASS, a large, comprehensive, national-level database with a sufficient sample size to allow for subgroup analyses. To our knowledge, trends in the use of ICSI in the context of the indications for ICSI and outcomes following its use have not been previously investigated or reported.

Conclusions

Among fresh-embryo IVF cycles in the United States, the use of ICSI increased from 36.4% in 1996 to 76.2% in 2012, with the largest relative increase noted among cycles without a diagnosis of male factor infertility. Compared with conventional IVF, use of ICSI was not associated with improved reproductive outcomes irrespective of male factor infertility diagnosis.

Acknowledgments

Funding/Support: This study was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under award UL1TR000454 (Dr Mehta).

Role of the Funder/Sponsor: The National Institutes of Health had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Glossary

ART

assisted reproductive technology

ICSI

intracytoplasmic sperm injection

IVF

in vitro fertilization

NASS

National Assisted Reproductive Technology Surveillance System

Footnotes

Author Contributions: Dr Boulet had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: Boulet, Mehta, Kissin, Warner, Kawwass.

Drafting of the manuscript: Boulet, Mehta, Warner.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Boulet, Mehta, Warner.

Obtained funding: Mehta.

Administrative, technical, or material support: Mehta, Jamieson.

Study supervision: Kissin, Jamieson.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Disclaimer: The content, findings, and conclusions of this report are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health or the official position of the Centers for Disease Control and Prevention.

REFERENCES

  • 1.Palermo G, Joris H, Devroey P, Van Steirteghem AC. Pregnancies after intracytoplasmic injection of single spermatozoon into an oocyte. Lancet. 1992;340(8810):17–18. doi: 10.1016/0140-6736(92)92425-f. [DOI] [PubMed] [Google Scholar]
  • 2.Schlegel PN, Palermo GD, Goldstein M, et al. Testicular sperm extraction with intracytoplasmic sperm injection for nonobstructive azoospermia. Urology. 1997;49(3):435–440. doi: 10.1016/S0090-4295(97)00032-0. [DOI] [PubMed] [Google Scholar]
  • 3.Jain T, Gupta RS. Trends in the use of intracytoplasmic sperm injection in the United States. N Engl J Med. 2007;357(3):251–257. doi: 10.1056/NEJMsa070707. [DOI] [PubMed] [Google Scholar]
  • 4.Kim HH, Bundorf MK, Behr B, McCallum SW. Use and outcomes of intracytoplasmic sperm injection for non-male factor infertility. Fertil Steril. 2007;88(3):622–628. doi: 10.1016/j.fertnstert.2006.12.013. [DOI] [PubMed] [Google Scholar]
  • 5.Luna M, Bigelow C, Duke M, et al. Should ICSI be recommended routinely in patients with 4 or fewer oocytes retrieved? J Assist Reprod Genet. 2011;28(10):911–915. doi: 10.1007/s10815-011-9614-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Bhattacharya S, Hamilton MP, Shaaban M, et al. Conventional in-vitro fertilisation vs intracytoplasmic sperm injection for the treatment of non-male-factor infertility: a randomised controlled trial. Lancet. 2001;357(9274):2075–2079. doi: 10.1016/s0140-6736(00)05179-5. [DOI] [PubMed] [Google Scholar]
  • 7.Practice Committees of the American Society for Reproductive Medicine and Society for Assisted Reproductive Technology Intracytoplasmic sperm injection (ICSI) for non-male factor infertility: a committee opinion. Fertil Steril. 2012;98(6):1395–1399. doi: 10.1016/j.fertnstert.2012.08.026. [DOI] [PubMed] [Google Scholar]
  • 8.Bonduelle M, Van Assche E, Joris H, et al. Prenatal testing in ICSI pregnancies: incidence of chromosomal anomalies in 1586 karyotypes and relation to sperm parameters. Hum Reprod. 2002;17(10):2600–2614. doi: 10.1093/humrep/17.10.2600. [DOI] [PubMed] [Google Scholar]
  • 9.Gjerris AC, Loft A, Pinborg A, Christiansen M, Tabor A. Prenatal testing among women pregnant after assisted reproductive techniques in Denmark 1995-2000: a national cohort study. Hum Reprod. 2008;23(7):1545–1552. doi: 10.1093/humrep/den103. [DOI] [PubMed] [Google Scholar]
  • 10.Amor DJ, Halliday J. A review of known imprinting syndromes and their association with assisted reproduction technologies. Hum Reprod. 2008;23(12):2826–2834. doi: 10.1093/humrep/den310. [DOI] [PubMed] [Google Scholar]
  • 11.Sandin S, Nygren KG, Iliadou A, Hultman CM, Reichenberg A. Autism and mental retardation among offspring born after in vitro fertilization. JAMA. 2013;310(1):75–84. doi: 10.1001/jama.2013.7222. [DOI] [PubMed] [Google Scholar]
  • 12.Davies MJ, Moore VM, Willson KJ, et al. Reproductive technologies and the risk of birth defects. N Engl J Med. 2012;366(19):1803–1813. doi: 10.1056/NEJMoa1008095. [DOI] [PubMed] [Google Scholar]
  • 13.Zhu JL, Basso O, Obel C, Bille C, Olsen J. Infertility, infertility treatment, and congenital malformations: Danish national birth cohort. BMJ. 2006;333(7570):679. doi: 10.1136/bmj.38919.495718.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Wu AK, Odisho AY, Washington SL, III, Katz PP, Smith JF. Out-of-pocket fertility patient expense: data from a multicenter prospective infertility cohort. J Urol. 2014;191(2):427–432. doi: 10.1016/j.juro.2013.08.083. [DOI] [PubMed] [Google Scholar]
  • 15.Wu AK, Elliott P, Katz PP, Smith JF. Time costs of fertility care: the hidden hardship of building a family. Fertil Steril. 2013;99(7):2025–2030. doi: 10.1016/j.fertnstert.2013.01.145. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Fertility Clinic Success Rate and Certification Act of 1992. Pub L 102-493, 1063 Stat 146-3152. [Google Scholar]
  • 17.Centers for Disease Control and Prevention; American Society for Reproductive Medicine; Society for Assisted Reproductive Technology . 2011 Assisted Reproductive Technology National Summary Report. US Dept of Health and Human Services; Atlanta, GA: 2013. [Google Scholar]
  • 18.Austin PC. The performance of different propensity-score methods for estimating relative risks. J Clin Epidemiol. 2008;61(6):537–545. doi: 10.1016/j.jclinepi.2007.07.011. [DOI] [PubMed] [Google Scholar]
  • 19.Nyboe Andersen A, Carlsen E, Loft A. Trends in the use of intracytoplasmatic sperm injection marked variability between countries. Hum Reprod Update. 2008;14(6):593–604. doi: 10.1093/humupd/dmn032. [DOI] [PubMed] [Google Scholar]
  • 20.Grimes DA, Schulz KF. False alarms and pseudo-epidemics: the limitations of observational epidemiology. Obstet Gynecol. 2012;120(4):920–927. doi: 10.1097/AOG.0b013e31826af61a. [DOI] [PubMed] [Google Scholar]
  • 21.Beck-Fruchter R, Lavee M, Weiss A, Geslevich Y, Shalev E. Rescue intracytoplasmic sperm injection: a systematic review. Fertil Steril. 2014;101(3):690–698. doi: 10.1016/j.fertnstert.2013.12.004. [DOI] [PubMed] [Google Scholar]

RESOURCES