Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jan 1.
Published in final edited form as: Support Care Cancer. 2020 May 2;29(1):311–322. doi: 10.1007/s00520-020-05472-3

A single-arm, prospective trial investigating the effectiveness of a non-hormonal vaginal moisturizer containing hyaluronic acid in postmenopausal cancer survivors

Jeanne Carter 1,2,3, Raymond E Baser 4, Deborah J Goldfrank 1,5, Barbara Seidel 6, Lisania Milli 6, Cara Stabile 7, Jocelyn Canty 1, Sally Saban 1, Shari Goldfarb 4,8,9, Maura N Dickler 10, Ginger J Gardner 1,5, Elizabeth L Jewell 1,5, Yukio Sonoda 1,5, Marisa A Kollmeier 11, Kaled M Alektiar 11
PMCID: PMC7606252  NIHMSID: NIHMS1590243  PMID: 32358778

Abstract

Purpose:

To assess the feasibility and efficacy of a non-hormonal hyaluronic acid (HLA) vaginal gel in improving vulvovaginal estrogen-deprivation symptoms in postmenopausal women with a history of hormone receptor-positive (HR+) cancer.

Methods:

For this single-arm, prospective longitudinal trial, we identified disease-free patients with a history of HR+ breast cancer treated with aromatase inhibitors or HR+ endometrial cancer treated with surgery and postoperative radiation. Participants used HLA daily for the first 2 weeks, and then 3x/week until weeks 12–14; dosage was then increased to 5x/week for non-responders. Vulvovaginal symptoms and pH were assessed at 4 time points (baseline [T1]; 4–6 weeks [T2]; 12–14 weeks [T3]; 22–24 weeks [T4]) with clinical evaluation, the Vaginal Assessment Scale (VAS), Vulvar Assessment Scale (VuAS), Female Sexual Function Index (FSFI), and Menopausal Symptom Checklist (MSCL).

Results:

Of 101 patients, mean age was 55 years (range, 31–78), 68% (n=69) were partnered, and 60% (n=61) were sexually active. In linear mixed models, VAS/VuAS scores significantly improved at all assessment points (all p<.001). MSCL scores similarly improved (all p<.001). FSFI scores significantly improved from T1 to T2 (p<.03), T3 (p<.001), and T4 (p<.001). Severe vaginal pH (>6.5) decreased from 26% at T1 to 19% at T4 (p=.18).

Conclusions:

HLA moisturization improved vulvovaginal health/sexual function of cancer survivors. While HLA administration 1–2x/week is recommended for women in natural menopause, a 3–5x/week schedule appears to be more effective for symptom relief in cancer survivors.

Keywords: endometrial cancer, breast cancer, hyaluronic acid, sexual function, vaginal health, cancer survivorship

INTRODUCTION

With the number of cancer survivors nearing 16.9 million in the United States, the number of women struggling with the side effects of cancer therapy is also increasing. Chemotherapy, radiation therapy, and endocrine therapy can negatively affect vulvovaginal and sexual health by inducing menopause or worsening menopausal symptoms. Menopause triggered by cancer treatment is qualitatively different from natural menopause; it is typically abrupt, with more intense and prolonged estrogen-deprivation symptoms, often resulting in more chronic side effects to the vagina and surrounding tissues than with a gradual menopausal decline. Breast and endometrial cancer are two of the most prevalent cancers diagnosed in women in the United States [13]. Breast cancer is the most common malignancy in women, accounting for 30% of all female cancers and representing 22.1% of all survivors [1, 2]. Endometrial cancer is the most common gynecologic cancer in the United States [1, 2]. Approximately 61,880 women are diagnosed with endometrial cancer each year in the United States, and 95% of early-stage endometrial cancer patients will be long-term survivors [3]. Breast and endometrial cancers can be hormone-receptor positive, and systemic hormonal supplements for menopausal symptoms are contraindicated. In addition, many women tend to decline intravaginal hormonal therapies due to their fear of systemic absorption and the potential risk of cancer recurrence [4, 5]. Effective, natural, non-hormonal therapies are needed to treat vulvovaginal dryness and discomfort in female cancer survivors.

Non-hormonal vaginal gels containing hyaluronic acid (HLA) have been studied in Europe, with encouraging results in non-cancer, post-menopausal populations. HLA sodium salt retains high amounts of water due to its high molecular weight and has a moisturizing effect on the epithelium, thereby promoting epithelial elasticity of the tissues. Findings from recent studies support HLA vaginal gel as an effective alternative to estrogen-based treatments in relieving menopausal symptoms in women [611]. However, there are limited data regarding the physical and psychosexual impact of HLA. Formal investigation is needed to fully evaluate HLA’s benefits and impact on sexual function in women with a history of cancer. We sought to target these existing gaps in the literature by comprehensively investigating the feasibility and efficacy of sustained HLA use in female cancer survivors.

The study aims were (1) to investigate the feasibility of conducting a 12-week HLA treatment regimen, indicated by the percentage of women who completed the 12-week assessment and by patient-reported satisfaction with the treatment; (2) to evaluate the efficacy of 12 weeks of treatment with an HLA vaginal gel on improving vaginal and vulvar health, as well as sexual function; and (3) to explore whether a more frequent schedule of HLA administration during weeks 12–24 improved the vaginal and vulvar health of women who showed no improvement at the 12-week assessment point (i.e., non-responders).

METHODS

Study Design

This study was a Memorial Sloan Kettering Cancer Center (MSK) Institutional Review Board (IRB)-approved single-arm, prospective longitudinal clinical trial investigating the feasibility, and secondarily, the efficacy of a hyaluronic acid (HLA)-based vaginal gel to improve estrogen deprivation symptoms of vulvovaginal dryness and discomfort in post-menopausal women with a history of hormone-receptor positive breast or endometrial cancer. After obtaining informed consent, participants were instructed to apply the gel for 24 weeks. Treatment outcomes were assessed by clinical exam and patient-reported outcome (PRO) measures at baseline (pre-treatment) and at 4–6 weeks, 12–14 weeks, and 22–24 weeks (Table 1). Data collection occurred from June 2013 through December 2017.

Table 1.

Study Timeline and Assessment Points

Parameter Pre-Tx Baselin e Visit 2 wks 4–6 wks 8 wks 12–14 wks 20 wks 22–24 wks
Patients identified and referred to FSM Program by clinician X
Moisturizer Diary Reminder X X X
PhD and NP/GYN visit:
  • Clinical Assessment Form
    • VAS
    • VuAS
  • Targeted Pelvic Exam
    • pH
    • Epithelial atrophy
    • Vulvar atrophy
    • Vulvar/vestibular irritation
X X X X
Patient Questionnaires:
  • Menstrual Status (baseline only)

  • PROMIS

  • MSCL

Clinical Assessment Form
  • FSFI

  • SAQ

  • SSS

Moisturizer Diary
X X X X
Patient Satisfaction Questionnaire X

Tx, treatment; FSMP, Female Sexual Medicine Program; NP, nurse practitioner; VAS, Vaginal Assessment Scale; VuAS, Vulvar Assessment Scale; MSCL, Menopausal Symptom Checklist; FSFI, Female Sexual Function Index; SAQ, Sexual Activity Questionnaire; SSS, Sexual Self-Schema Scale

Participants and Study Setting

Eligible participants included the following: (1) post-menopausal women with a history of breast cancer who had completed primary cancer treatment consisting of surgery, chemotherapy or radiation therapy 3–60 months prior to study enrollment and were currently on an adjuvant aromatase inhibitor (AI), and (2) post-menopausal women with a history of endometrial cancer who had completed chemotherapy and/or maintenance therapy 3–60 months prior to study enrollment, at least 4 weeks post radiation therapy (external-beam radiation therapy [EBRT] or intravaginal radiation therapy [IVRT]), who underwent surgical treatment (total hysterectomy/bilateral salpingo-oophorectomy [TH/BSO]). All participants reported being bothered by vulvovaginal symptoms of estrogen deprivation (i.e., vulvovaginal dryness or discomfort with intercourse or examination) during or post-cancer treatment and were referred to the Female Sexual Medicine Program (FSMP) by their clinician. At baseline, all participants had no clinical evidence of disease or other cancer history (excluding non-melanoma skin cancer).

Potential participants were screened for eligibility at the FSMP and at various radiation oncology and gynecologic surgery outpatient clinics at MSK, were given a description of the study, and were invited to participate. All participants received the standard-of-care intervention at the FSMP, which consisted of education about common changes in vaginal/vulvar symptoms and sexual health post-cancer treatment, provision of psychosexual education regarding the sexual response, and recommendations for vulvovaginal health promotion. Baseline data were collected on the day of study consent, and participants began HLA-based treatment within 1 week of their baseline exam.

Study Intervention

The product used in the study, Hyalo GYN®, was approved by the United States Food and Drug Administration in 2010. The principal component of the hydrating gel is Hydeal-D® (a hyaluronic acid derivative). Other components include propylene glycol, carbomer (Carbopol 974P), methyl p-hydroxy-benzoate, propyl p-hydroxybenzoate, sodium hydroxide, and purified water. Participants were instructed to measure a full application of the HLA-based vaginal gel and insert it into the vagina (using the applicator provided) and to measure another full application and apply it on the vulva (manually) every day at bedtime for the first 2 weeks and then 3 times per week for 10–12 weeks. At 12–14 weeks, all participants were re-assessed for improvement in vaginal pH and Vaginal Assessment Scale (VAS) score, as well as improvement in vulvar atrophy and/or vulvar/vestibular irritation and Vulvar Assessment Scale (VuAS) score. The FSMP Psychologist (PhD) and/or FSMP Nurse Practitioner (NP) or gynecology clinician recorded these symptoms on the Sexual Medicine Clinical Assessment Form as part of routine clinical care. At 24 weeks on study, participants completed an additional questionnaire about their experience with the HLA-based gel and their acceptability of and satisfaction with the product.

Participants who experienced improvement in vaginal and/or vulvar symptoms from the HLA-based gel (i.e., responders) were instructed to continue to apply the product to both the vaginal and vulvar areas 3 times per week for the duration of the study (22–24 weeks). Those who did not experience a benefit (no change or worsening of vaginal and/or vulvar symptoms) with the HLA-based gel (i.e., non-responders) at 12–14 weeks, were instructed to increase application frequency to 5 times per week for the remainder of the study. Participants showing improvement in only one area (i.e., vagina or vulva), were instructed to continue with an application frequency of 3 times per week for the improved area and 5 times per week for the unaffected/worsened area.

Feasibility Endpoints

Patient satisfaction with and acceptability of the treatment and HLA gel product were assessed at 12–14 weeks with additional questions regarding overall symptom improvement and at 22–24 weeks with questions about product satisfaction. Retention rates were also assessed.

Efficacy Endpoints

Patients completed PRO measures and were assessed for vulvovaginal symptoms by a clinical gynecologic exam at baseline and at 4–6 weeks, 12–14 weeks, and 22–24 weeks (Table 1). Objective vaginal symptoms (vaginal pH and degree of epithelial atrophy) and vulvar symptoms (vulvar atrophy and/or vulvar/vestibular irritation) on clinical gynecologic exam and subjective vaginal symptoms (Vaginal Assessment Scale; VAS) and vulvar symptoms (Vulvar Assessment Scale; VuAS) were monitored closely for the study duration (baseline to 24 weeks). The validated VAS and VuAS measures [12] were our primary efficacy outcome measures, and changes in VAS and VuAS scores from baseline to T3 (12–14 weeks) were our primary efficacy endpoints. These instruments measure patients’ perceptions of their vulvovaginal tissue quality. The VAS assesses vaginal symptoms of dryness, soreness, irritation, and dyspareunia. The VuAS evaluates vulvar symptoms of dryness, soreness, irritation, and external pain post-stimulation. Symptoms are graded on a 4-point scale (0=none; 1=mild; 2=moderate; 3=severe) and are averaged to create VAS and VuAS composite scores. Higher scores indicate more symptoms.

Patient-Reported Outcomes:

Sexual function was measured by the Female Sexual Function Index (FSFI), the Sexual Activity Questionnaire (SAQ), and PROMIS sexual function items (PROMIS SxF). The FSFI is a 19-item self-report measure of female sexual function [13] that has been validated for use in female cancer survivors [14]. Higher FSFI scores indicate better functioning, with a total score ≤26.0 indicating sexual dysfunction [14]. The SAQ is a 14-item screening tool used to assess whether women are sexually active, to evaluate sexual function, and to identify reasons for any reported inactivity [15, 16]. The PROMIS SxF items address sexual interest, pain, orgasm, lubrication, and subjective arousal [1722]. Thirteen items from the PROMIS SxF measure were selected for our study. The Sexual Self-Schema Scale (SSS) was used to assess components of sexuality. The SSS is a self-rated assessment of the cognitive view of one’s sexual aspects [23]. Menopausal symptoms were assessed by the Menopausal Symptom Checklist (MSCL), a 36-item questionnaire querying a range of physical and psychological symptoms associated with estrogen deprivation, with higher scores indicating bothersome menopausal symptoms [24, 25]. Participants also were asked one question regarding their length of time in menopause on the baseline questionnaire.

Clinical Gynecologic Exam Outcomes:

Pelvic examination findings were recorded by a sexual medicine NP or general gynecologist on a standardized clinical checklist with descriptors of each vaginal and vulvar category to reduce interobserver variability. The pelvic exam checklist on the clinical assessment form was developed based on the Common Terminology Criteria for Adverse Events (CTCAE) and multidisciplinary discussions (psychologist, NPs, general gynecologist, radiation oncologist, and gynecologic oncologists) and from patient feedback. The pelvic exam checklist assesses physical vaginal characteristics (agglutination, scarring/adhesions, pH, moisture, rugosity, elasticity, length, thickness, epithelial integrity, vascularity, and irritation) and physical vulvar characteristics (vulvar atrophy, irritation, and vestibular irritation), with lower scores indicating fewer symptoms. Vaginal pH was measured using litmus paper and coded into clinically meaningful categories (<5, 5–6.5, >6.5). Normal vaginal pH is less than 5. Higher vaginal pH indicates greater vaginal atrophy. Four additional indicators of vaginal health were assessed (epithelial integrity, thickness, vaginal secretions, and vascularity) and rated on a 4-point scale (0, 1, 2, 3). These ratings were summed and rescaled to range from 0 to 100 to create a vaginal exam composite index, with higher scores indicating greater degrees of vaginal symptoms. Vulvar and vestibular health was assessed by examination (for vulvar atrophy and/or vulvar/vestibular irritation. Vulvar atrophy and vulvar irritation are both rated on a 4-point scale (0, 1, 2, 3). A score of 0 indicates normal vulvar characteristics or no presence of symptoms. A rating of 1 indicates some concern for symptoms or presence of vestibular irritation, and a rating of 3 indicates severe symptoms. These ratings are summed and rescaled to range from 0 to 100 to create a vulvar exam composite index. Higher scores indicate greater degrees of symptoms.

Statistical Analysis Methods

Demographic/medical characteristics and categorical endpoints were summarized using descriptive statistics (e.g., means and standard deviations for continuous variables, and frequencies and percentages for categorical variables). For continuous endpoints, linear mixed models (LMMs) controlling for assessment time, cancer diagnosis, months since treatment completion, and age were used to estimate the adjusted means and 95% confidence intervals (CIs) at each assessment time. LMM-based contrasts were used to test for statistically significant changes in the adjusted means from baseline to each follow-up assessment, as well as between the T3 and T4 assessments. For categorical endpoints, McNemar’s tests were used to test for significant changes between pairs of time points. Significance tests with a p<.05 were considered statistically significant. All statistical analyses were conducted in R version 3.6.1 [26].

RESULTS

Study Participants

Demographic and Medical Characteristics

The mean age of the entire cohort was 55 years (range, 31–78 years). Patients in the breast cohort had a mean age of 51.8 years (range, 31–75 years), and patients in the endometrial cohort had a mean age of 59.3 years (range, 38–78 years) (p<.001). Time in menopause for all patients was: less than 1 year, 7% (n=7); 1–5 years, 47% (n=48); more than 5 years, 46% (n=46). Sixty-eight percent (n=69) were partnered, 24% (n=24) were single, and 8% (n=8) were either widowed or divorced. Sixty percent (n=61) indicated current sexual activity (Table 2).

Table 2.

Participants Demographic and Medical Characteristics

Age
 Overall: Mean (SD), [Range] 55.0 (10.1), [31 – 78]
 Breast Cancer Group: Mean (SD), [Range] 51.8 (10.4), [31 – 75]
 Endometrial Cancer Group: Mean (SD), [Range] 59.3 (8.1), [38 – 78]
n (%)
Marital Status (N=101)
 Partnered 69 (68.3%)
 Single 24 (23.8%)
 Divorced/Widowed 8 (7.9%)
Race (N=101)
 White 78 (77.2%)
 Asian 9 (8.9%)
 Patient Declined/Other 8 (7.9%)
 Black 6 (5.9%)
Disease Site (N=101)
 Breast Cancer Group 58 (57.4%)
 Endometrial Cancer Group 43 (42.6%)
Total Sample - Sexually Active (N=101)
 Yes 61 (60.0%)
 No 40 (40.0%)
Breast Cancer Group (n=58)
 Yes 40 (69.0%)
 No 18 (31.0%)
Endometrial Cancer Group (n=43)
 Yes 21 (48.4%)
 No 22 (51.2%)
Total Sample - Time in Menopause (N=101)
 <1 Year 7 (7.0%)
 1–5 Years 48 (47.5%)
 >5 Years 46 (45.5%)
Breast Cancer Group (n=58)
 <1 Year 5 (8.6%)
 1–5 Years 34 (58.6%)
 >5 Years 19 (32.8%)
Endometrial Cancer Group (n=43)
 <1 Year 2 (4.6%)
 1–5 Years 14 (32.6%)
 >5 Years 27 (62.8%)
Time Since Treatment Completion (months)
 Overall: Mean (SD), [Range] 19.7 (16.4)
 Breast Cancer Group: Mean (SD), [Range] 22.6 (17.7)
 Endometrial Cancer Group: Mean (SD), [Range] 15.9 (13.7)

Intervention, Evaluability, and Attrition

A total of 146 women met the study’s eligibility criteria and were approached for study participation. Within this group, 107 women enrolled on the study and 39 declined (73% acceptance rate; Figure 1). All 107 of the consented participants completed the baseline assessment (61 breast and 46 endometrial cancer patients). After completing the baseline assessment, 6 of the eligible participants (3 breast and 3 endometrial cancer patients) withdrew for reasons unrelated to the study product or design (i.e., started tamoxifen, new evidence of disease, etc.), leaving 101 (58 breast and 43 endometrial cancer patients) evaluable patients for treatment initiation. Eighty-four participants completed the T2 assessment (49 breast and 35 endometrial cancer patients), 80 completed the T3 assessment (47 breast and 33 endometrial cancer patients), and 66 completed the T4 assessment (39 breast and 27 endometrial cancer patients). Twenty-one (11 breast and 10 endometrial) of 101 patients withdrew from the study before completing the primary efficacy evaluation timepoint (T3 assessment) for reasons related to the study product or design (i.e., lost to follow-up, noncompliant to dosing schedule, urinary tract infection, irritation/discharge etc.). All 101 participants who were enrolled and eligible to initiate treatment at baseline are included in the feasibility analysis. Women removed from study within 12 weeks of starting protocol treatment due to reasons related to the study product or design were considered evaluable for the feasibility objective. However, as these patients were evaluable for the feasibility objective, and their removal was related to the study product/design, they were not replaced. These participants were not evaluable for the secondary objective (effectiveness), which required completion of the T3 assessment. Fourteen participants (8 breast and 6 endometrial cancer patients) withdrew before study completion (T4).

Figure 1.

Figure 1.

Study accrual and attrition

Feasibility

Retention rates for the study indicated feasibility in conducting a 12-week HLA-based treatment regimen (baseline through T2), with rates of 83% at T2, 79% at T3, and 65% at T4. Of those who completed the T3 assessment, 98% (78/80) answered most or all of the items on a questionnaire about overall symptom improvement at 12–14 weeks (T3); the questionnaire included queries on whether the patient felt their vaginal and vulvar symptoms had improved on study, and whether they believed the improvement was due to the HLA-based product. Seventy-two percent (n=56/78) of the patients indicated improvements in vaginal symptoms and 21% (n=16 /78) indicated a partial benefit. Similar rates were reported regarding improvements in vulvar symptoms (70%, 35/50) and partial vulvar benefit (18%, 9/50). Ninety-two percent of all patients (72/78 patients with vaginal symptoms and 46/50 patients with vulvar symptoms) attributed the improvement to the HLA-based treatment or felt the treatment was partly responsible for symptom relief.

Satisfaction was assessed by participant expectations for the product, ease of use, and helpfulness in addressing tissue quality issues. Of those who completed the final assessment (T4), 98% (65/66) responded to most or all of the items of a satisfaction questionnaire; of those who responded, 69% (45/65) were treated for both vaginal and vulvar symptoms and 65% (42/65) reported sexual activity at T4. Eighty-nine percent (57/64) indicated the product moderately to extremely met their expectations. They found the product quite to extremely easy to use both in the vagina (87%, 55/63) and on the vulva (93%, 41/44), and viewed it to be somewhat to very helpful for vaginal tissue (98%, 64/65) and vulvar tissue (100%, 45/45) quality. By study completion, 94% (61/65) of the women felt their vaginal symptoms and 91% (41/45) felt their vulvar symptoms had been somewhat to very resolved. Ninety-eight percent (63/64) indicated they would recommend the HLA-based product to another female cancer survivor. Fifty-four percent (35/65) of the patients had previously never used a moisturizer for their vagina and 82% (37/45) had never used a moisturizer for their vulva. Of those who previously had tried a vaginal moisturizer, 86% (25/29) felt the HLA-based product was more helpful than the previously used over-the-counter vaginal moisturizer product.

Efficacy

The endometrial cancer patients, compared with the breast cancer patients, were older (59.3 vs. 51.8 years, respectively), had been in menopause longer (8.5 vs. 5.9 years, respectively), and had completed treatment more recently (15.9 vs. 22.6 months, respectively). The study endpoints did not differ by cancer diagnosis, and both groups of patients had similar trajectories of improvement over time. We therefore are presenting the study results for the two groups combined.

We estimated means and 95% CIs for each continuous endpoint at each of the 4 assessment time points using LMMs adjusted for assessment time, cancer diagnosis, months since treatment completion, and age (Table 3). None of the adjustment covariates were statistically significant predictors of our primary vulvar efficacy endpoint (VuAS). For our primary vaginal efficacy endpoint (VAS), older women (age >55) tended to have slightly better VAS scores across all visits (p=.038), but the interaction between age and visit was not statistically significant (p=.56), suggesting parallel patterns of VAS scores over time for the two age groups. Among breast cancer patients, we also assessed the effect of time on AIs on VAS and VuAS scores over time. Breast cancer patients on an AI for longer than 12 months had slightly higher VAS scores across all visits (p=.02) compared with breast cancer patients on an AI for 12 months or less. The interaction between time on AI and visit was not significant (p=.73), suggesting parallel patterns of VAS scores over time for the two groups.

Table 3.

Adjusted Means* of Vulvovaginal, Sexual Function, and Menopausal Outcomes Over Time

Variable Timepoint Mean (95% CI) p value**
VAS BL 1.14 (1.05, 1.23)
Total Score T2 0.5 (0.4, 0.6) <0.001
T3 0.47 (0.36, 0.57) <0.001
T4 0.32 (0.21, 0.43) <0.001
VuAS BL 0.89 (0.78, 0.99)
Total Score T2 0.43 (0.32, 0.54) <0.001
T3 0.44 (0.33, 0.56) <0.001
T4 0.29 (0.17, 0.42) <0.001
Vaginal Exam BL 29.99 (26.74, 33.24)
Composite T2 22.61 (19, 26.21) 0.001
T3 23.38 (19.76, 27.01) 0.003
T4 21.5 (17.55, 25.45) 0.0003
Vulvar Exam BL 32.88 (29.08, 36.67)
Composite T2 22.05 (17.88, 26.21) <0.001
T3 19.23 (15.04, 23.42) <0.001
T4 16.35 (11.83, 20.87) <0.001
Vaginal pH BL 6.26 (6.1, 6.41)
T2 6.35 (6.19, 6.51) 0.163
T3 6.24 (6.08, 6.4) 0.789
T4 6.2 (6.03, 6.37) 0.470
FSFI BL 12.26 (10.39, 14.13)
Total Score T2 14.26 (12.28, 16.25) 0.024
T3 15.78 (13.78, 17.78) 0.001
T4 16.63 (14.52, 18.73) <0.001
FSFI BL 2.55 (2.32, 2.78)
Desire T2 2.9 (2.66, 3.15) 0.003
T3 2.99 (2.74, 3.24) 0.0004
T4 3.11 (2.85, 3.38) <0.001
FSFI BL 2.02 (1.65, 2.39)
Arousal T2 2.43 (2.03, 2.82) 0.028
T3 2.69 (2.29, 3.09) 0.0004
T4 2.81 (2.39, 3.24) 0.0001
FSFI BL 1.6 (1.22, 1.98)
Lubrication T2 2.17 (1.76, 2.58) 0.003
T3 2.29 (1.88, 2.71) 0.0003
T4 2.62 (2.18, 3.05) <0.001
FSFI BL 2.21 (1.77, 2.66)
Orgasm T2 2.42 (1.95, 2.9) 0.331
T3 2.66 (2.18, 3.13) 0.043
T4 2.75 (2.25, 3.25) 0.021
FSFI BL 2.61 (2.28, 2.95)
Satisfaction T2 2.87 (2.5, 3.23) 0.178
T3 3.37 (3, 3.74) 0.0001
T4 3.13 (2.74, 3.52) 0.012
FSFI BL 1.39 (0.98, 1.79)
Pain T2 1.62 (1.19, 2.05) 0.280
T3 2.1 (1.67, 2.54) 0.001
T4 2.44 (1.98, 2.91) <0.001
PROMIS SxF BL 46.51 (45.02, 48.01)
Interest in Sexual Activity T2 48.55 (46.96, 50.14) 0.004
T3 49.49 (47.89, 51.1) <0.001
T4 49.79 (48.1, 51.48) <0.001
PROMIS SxF BL 70.49 (68.51, 72.46)
Vaginal Discomfort T2 61.54 (59.2, 63.87) <0.001
T3 60.35 (58.16, 62.55) <0.001
T4 58.62 (56.41, 60.83) <0.001
PROMIS SxF BL 67.06 (64.88, 69.23)
Vulvar Discomfort (Labial) T2 60.14 (57.63, 62.64) <0.001
T3 60.66 (58.22, 63.1) <0.001
T4 58.6 (56.14, 61.06) <0.001
PROMIS SxF BL 60.71 (57.82, 63.61)
Vulvar Discomfort (Clitoral) T2 52.62 (49.46, 55.78) 0.0001
T3 53.13 (50.06, 56.2) 0.0002
T4 52.79 (49.81, 55.77) 0.0001
SAQ BL 5.76 (4.78, 6.74)
Pleasure T2 6.88 (5.84, 7.92) 0.017
T3 7.85 (6.79, 8.91) <0.001
T4 8.11 (6.99, 9.23) <0.001
SAQ BL 1.33 (0.94, 1.73)
Discomfort T2 3.06 (2.6, 3.52) <0.001
T3 3.19 (2.74, 3.64) <0.001
T4 3.36 (2.89, 3.84) <0.001
SSS BL 56.19 (53.36, 59.02)
Total Score T2 57.7 (54.79, 60.61) 0.086
T3 56.85 (53.93, 59.77) 0.456
T4 56.14 (53.09, 59.19) 0.962
SSS BL 40.28 (39.12, 41.44)
Passion-Romantic T2 40.68 (39.46, 41.89) 0.421
T3 40.48 (39.26, 41.7) 0.687
T4 39.96 (38.65, 41.28) 0.572
SSS BL 37.24 (35.72, 38.75)
Openness-Directness T2 37.89 (36.33, 39.45) 0.197
T3 36.74 (35.17, 38.31) 0.333
T4 37.13 (35.48, 38.77) 0.847
SSS BL 21.32 (20.06, 22.58)
Embarrassed-Conservative T2 20.92 (19.61, 22.23) 0.408
T3 20.39 (19.07, 21.71) 0.058
T4 20.8 (19.4, 22.2) 0.343
MSCL BL 23.56 (21.26, 25.85)
Total Score T2 17.13 (14.74, 19.51) <0.001
T3 17.15 (14.75, 19.54) <0.001
T4 17.57 (15.09, 20.05) <0.001
MSCL BL 10.13 (9.04, 11.22)
# of Symptoms T2 7.33 (6.19, 8.48) <0.001
T3 7.33 (6.18, 8.48) <0.001
T4 7.47 (6.27, 8.66) <0.001
*

Means are from linear mixed models adjusting for time point, cancer diagnosis, months since treatment completion, and age. The endpoint marginal means were calculated from these models for each time point holding the other variables constant at their sample means.

**

P values are from tests of differences between the baseline mean and each of the follow-up means.

The T3 and T4 means for the VAS and VuAS were significantly different (p<0.05, see Figure 2). The T3 and T4 means did not significantly differ for any of the other endpoints.

CI, Confidence Interval; BL, Baseline; T2, Timepoint 2; T3, Timepoint 3; T4, Timepoint 4; VAS, Vaginal Assessment Scale; VuAS, Vulvar Assessment Scale; FSFI, Female Sexual Function Index; PROMIS SxF, PROMIS Sexual Function Items; SAQ, Sexual Activity Questionnaire; SSS, Sexual Self Schema Scale; MSCL, Menopausal Symptom Checklist

Compared to baseline, VAS and VuAS scores were significantly better (lower) at each of the three follow-up assessments (all p<.001, Figure 2 and Table 3). VAS and VuAS scores continued to improve between the T3 and T4 assessments (both p<.05). The vaginal exam composite and vulvar exam composite scores similarly improved at each follow-up assessment compared to baseline (all p<.01) but did not improve significantly between T3 and T4. Most of the other endpoints had similar patterns of improvement at each follow-up relative to baseline (Table 3), including sexual function as measured by the PROMIS and SAQ scales, and menopausal symptoms as measured by the MSCL. The FSFI sexual function scales also generally improved at each follow-up, although the orgasm, satisfaction, and pain subscales did not show significant improvement at T2 relative to baseline. Vaginal pH and the SSS did not reliably improve over the course of the study.

Figure 2.

Figure 2.

Estimated Vaginal Assessment Scale (VAS) and Vulvar Assessment Scale (VuAS) means scores and 95% confidence intervals by time point

Patient-reported perception of vaginal and vulvar dryness substantially improved from baseline to T3 and T4 (Table 4). At baseline, approximately 97% of patients reported mild to severe vaginal dryness, which dropped to 45% at T3 and 26% at T4 (both p<.001). Mild to severe vulvar dryness was reported by approximately 83% of patients at baseline, dropping to 51% at T3 and 33% at T4 (both p<.001).

Table 4.

Change in Vulvovaginal Health Outcomes from Baseline to T3 and from Baseline to T4

Baseline to T3 Comparison1 Baseline to T4 Comparison2
T1 n (%) T3 n (%) p value3 T1 n (%) T4 n (%) p value3
Vaginal Dryness
 None 2 (2.50%) 44 (55.0%) <0.001 2 (3.03%) 49 (74.2%) <0.001
 Mild 17 (21.2%) 24 (30.0%) 13 (19.7%) 9 (13.6%)
 Moderate 21 (26.2%) 11 (13.8%) 18 (27.3%) 6 (9.09%)
 Severe 40 (50.0%) 1 (1.25%) 33 (50.0%) 2 (3.03%)
Vulvar Dryness
 None 14 (17.5%) 39 (48.8%) <0.001 11 (16.7%) 44 (66.7%) <0.001
 Mild 22 (27.5%) 21 (26.2%) 16 (24.2%) 11 (16.7%)
 Moderate 28 (35.0%) 17 (21.2%) 27 (40.9%) 11 (16.7%)
 Severe 16 (20.0%) 3 (3.75%) 12 (18.2%) 0 (0.00%)
Vulvar Irritation
 None 40 (50.0%) 52 (65.0%) <0.001 34 (51.5%) 48 (72.7%) <0.001
 Mild 18 (22.5%) 23 (28.7%) 13 (19.7%) 16 (24.2%)
 Moderate 18 (22.5%) 3 (3.75%) 15 (22.7%) 2 (3.03%)
 Severe 4 (5.00%) 2 (2.50%) 4 (6.06%) 0 (0.00%)
Vulvar Atrophy
 None 26 (32.5%) 42 (52.5%) <0.001 20 (30.3%) 36 (54.5%) 0.0 04
 Mild 41 (51.2%) 36 (45.0%) 36 (54.5%) 27 (40.9%)
 Moderate 11 (13.8%) 2 (2.50%) 8 (12.1%) 3 (4.55%)
 Severe 2 (2.50%) 0 (0.00%) 2 (3.03%) 0 (0.00%)
Vestibular Irritation
 No 35 (43.8%) 48 (60.0%) 0.0 12 27 (40.9%) 42 (63.6%) 0.003
 Yes 45 (56.2%) 32 (40.0%) 39 (59.1%) 24 (36.4%)
Vaginal pH
 pH <5 7 (8.75%) 9 (11 .2%) 0.1 44 6 (9. 2%) 8 (12 .3%) 0.1 78
 pH 5–6.5 50 (62.5%) 54 (67.5%) 42 (64.6%) 45 (69.2%)
 pH >6.5 23 (28.7%) 17 (21.2%) 17 (26.2%) 12 (18.5%)
1.

Baseline (T1) frequencies and percentages are presented for the 80 patients who completed the T3 assessment.

2.

Baseline (T1) frequencies and percentages are presented for the 66 patients (65 for vaginal pH) who completed the T4 assessment.

3.

p values are from McNemar’s tests for paired proportions. Significant p values (p<0.05) indicate significant changes in the category percentages between the pair of assessment times.

On clinical exam, presence of vulvar irritation improved from approximately 50% at baseline to 35% at T3 and 27% at T4 (both p<.001; Table 4). Prevalence of vestibular irritation decreased from approximately 56% at baseline to 40% at T3 and 36% at T4 (both p<.001). Presence of vulvar atrophy on clinical exam significantly improved from approximately 68% at baseline to 48% at T3 (p<.001) and 46% at T4 (p=0.004). For vaginal pH, severe vaginal pH (>6.5) decreased from approximately 26% at baseline to 21% at T3 (p=.144) and 19% at T4 (p=.178).

Effect of More Frequent HLA Administration from T3 to T4

At T3 (12–14 weeks), participants were assessed for benefit from the HLA-based treatment. Among patients evaluable for vaginal endpoints, 60 (75%) of 80 patients were considered “non-responders” at 12 weeks (T3) and were instructed to increase their vaginal administration of HLA from 3 to 5 times per week for the remainder of the study. While vulvovaginal and sexual function significantly improved at each timepoint after treatment, only participants who showed improvement on both the VAS score and vaginal pH at her 12-week assessment could be considered a vaginal “responder”. Participants who improved on only one outcome or stabilized were still considered “non-responders”. The mean change (standard deviation, SD) in VAS between T3 and T4 was −0.26 (0.65) among women instructed to increase vaginal application, which indicates significant improvement (Wilcoxon rank-sum test p=.004) compared to a mean change of 0.16 (0.35) among women instructed to continue vaginal application 3 times per week. The changes in the vaginal exam composite between T3 and T4 were also significantly better (Wilcoxon rank-sum test p = 0.045) for women instructed to increase vaginal application (mean change, −5.0; SD, 15.0) compared to those instructed to continue vaginal application 3 times per week (mean change, 7.0; SD, 17.0).

Among patients evaluable for vulvar endpoints, 32 (68%) of 47 patients at T3 met the criteria for a dose increase and were instructed to increase their vulvar administration of the HLA-based treatment from 3 to 5 times per week for the remainder of the study. The mean change (SD) in VuAS between T3 and T4 was −0.34 (0.51) among women instructed to increase vulvar application, which indicates significant improvement (Wilcoxon rank-sum test p=.002) compared to a mean change of 0.08 (0.24) among women instructed to continue vulvar application 3 times per week. The changes in the vulvar exam composite between T3 and T4 were also significantly better (Wilcoxon rank-sum test p=.007) for women instructed to increase vulvar application (mean change, −11.0; SD, 21.0) compared to those instructed to continue vulvar application 3 times per week (mean change, 4.0; SD, 6.0).

DISCUSSION

Key Results

Compared to baseline, symptoms of vaginal and vulvar tissue quality significantly improved, including decreased vulvar and vestibular irritation, which matched what was seen on gynecologic exam. Although vaginal pH did not statistically improve from baseline, we did not expect significant decreases (towards pH<5) since the study patients were not administering vaginal estrogens and they also presented with a cancer history impacting pH at baseline; however, we were pleased to see a decrease in severely elevated pH (6.5 of greater) with consistent non-hormonal moisturizer use. With improvement in vulvovaginal tissue quality and comfort, our participants reported enhancement in all domains of the sexual response and were less bothered by menopausal symptoms. They also expressed increased confidence about future sexual activity using these simple non-hormonal strategies. Seventy-five percent of the patients were non-responders at T3 (no change or worsening) and were instructed to increase moisturizer application to 5 times per week. In hindsight, stabilization of symptoms could have been viewed as a positive response; however, our concern was to not undertreat any of the participants. In the women who required a dose increase to 5 times per week, however, the results showed their symptoms continued to improve at the higher frequency of moisturizer application.

Limitations

A limitation of our study was the lack of a placebo-controlled arm or comparison group. However, this intervention was implemented within a female sexual medicine program with the goal to address the symptoms of our patients. This study was offered as a treatment option, and patients could withdraw at any time if not satisfied with the study or product. Additionally, we did not recognize the importance of vulvar outcomes when first developing the study, and we added this as an objective later based on participant feedback and clinical exam outcomes. This contributed to uneven numbers for the vaginal and vulvar outcomes. The need to address vulvar health independently was a clinically significant, indirect benefit of conducting this study. Fortunately, we were able to accrue additional participants for statistical power.

Interpretation

Non-hormonal vaginal gels containing HLA have been investigated with encouraging results in non-cancer, post-menopausal populations [611, 27]. A recent study found HLA to be beneficial for vaginal dryness in cervical cancer patients and emphasized the need for additional studies supporting HLA benefits in other cancer populations [28]. Our study similarly concluded that HLA-based vaginal moisturizing gel applied intravaginally and topically to the vulva improved vulvovaginal health and sexual function in cancer survivors both in self-perceived symptoms and on clinical outcomes. The study was able to demonstrate that female cancer survivors need to moisturize at a higher frequency (3–5 times per week) than what is recommended (1–2 times per week) for women in natural menopausal to achieve adequate symptom relief. We also demonstrated that this product, which is used in the vagina to hydrate tissue quality, was also effective on vulvar tissue quality. Participants were highly satisfied with the product in improving vulvovaginal symptoms. Participants generally found that the non-hormonal moisturizer was helpful, easy to use, and would recommend it to others; however, satisfaction was assessed only among the patients who completed the T3 and T4 assessments, and it is possible that satisfaction was lower among the patients who left the study early This study identified an effective, acceptable, and beneficial non-hormonal option for vulvovaginal health and sexual function in female cancer survivors, and we hope these findings will be a valuable clinical contribution to the oncology field.

Generalizability

This study tested an HLA-based moisturizer in two of our most challenging patient populations with regard to tissue quality after cancer treatment (endocrine therapy or radiation therapy). The clinical benefit to vaginal and vulvar tissue quality from this non-hormonal moisturizer and its positive impact on sexual function can easily be translated into the clinical setting with other female cancer survivors experiencing vulvovaginal health concerns due to estrogen deprivation or pelvic radiation therapy.

FUTURE DIRECTIONS

Our study participants found it feasible to use an HLA-based gel internally and externally 3–5 times per week; however, suggestions for a vaginal suppository option to minimize discharge and to facilitate application were noted and could be tested in future studies. Although we had improvements in vulvovaginal health using a non-hormonal moisturizer, clinically, we have noted off study that some women can benefit from the addition of a low-dose estrogen, if medically appropriate. Future studies should test the benefit of non-hormonal moisturizers in combination with low-dose estrogens to standardize treatment guidelines for female cancer survivors experiencing persistent vulvovaginal symptoms.

Funding/Disclosures:

Fidia Pharmaceuticals supported the scientific aims of this clinical trial through donation of product (Hydeal-D©) and a small research grant to MSK to cover the time and effort of an MSK statistician, research assistant, and pharmacy staff member for product distribution. This research was also funded in part through the National Institutes of Health Cancer Center Support Grant P30 CA008748. Outside the submitted work, Dr. Jewell reports personal fees from Covidien/Medtronic.

Footnotes

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of a an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

Research Involving Human Participants: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional review board and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent: Informed consent was obtained from all individual participants included in the study.

REFERENCES

  • [1].Bluethmann SM, Mariotto AB, Rowland JH. Anticipating the “Silver Tsunami”: prevalence trajectories and comorbidity burden among older cancer survivors in the United States. Cancer Epidemiol Biomarkers Prev. 2016;25:1029–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [2].Siegel RL, Miller KD, Jemal A. Cancer statistics, 2019. CA Cancer J Clin. 2019;69:7–34. [DOI] [PubMed] [Google Scholar]
  • [3].National Cancer Institute: Surveillance, Epidemiology and End Results Program. SEER Cancer Statistics Review (CSR) 1975–2016. Updated September 5, 2019. https://seer.cancer.gov/csr/1975_2016/.
  • [4].Biglia N, Cozzarella M, Cacciari F, Ponzone R, Roagna R, Maggiorotto F, et al. Menopause after breast cancer: a survey on breast cancer survivors. Maturitas. 2003;45:29–38. [DOI] [PubMed] [Google Scholar]
  • [5].Ganz PA, Greendale GA, Kahn B, O’Leary JF, Desmond KA. Are older breast carcinoma survivors willing to take hormone replacement therapy? Cancer. 1999;86:814–20. [PubMed] [Google Scholar]
  • [6].Chen J, Geng L, Song X, Li H, Giordan N, Liao Q. Evaluation of the efficacy and safety of hyaluronic acid vaginal gel to ease vaginal dryness: a multicenter, randomized, controlled, open-label, parallel-group, clinical trial. J Sex Med. 2013;10:1575–84. [DOI] [PubMed] [Google Scholar]
  • [7].Costantino D, Guaraldi C. Effectiveness and safety of vaginal suppositories for the treatment of the vaginal atrophy in postmenopausal women: an open, non-controlled clinical trial. Eur Rev Med Pharmacol Sci. 2008;12:411–6. [PubMed] [Google Scholar]
  • [8].Ekin M, Yasar L, Savan K, Temur M, Uhri M, Gencer I, et al. The comparison of hyaluronic acid vaginal tablets with estradiol vaginal tablets in the treatment of atrophic vaginitis: a randomized controlled trial. Arch Gynecol Obstet. 2011;283:539–43. [DOI] [PubMed] [Google Scholar]
  • [9].Jokar A, Davari T, Asadi N, Ahmadi F, Foruhari S. Comparison of the hyaluronic acid vaginal cream and conjugated estrogen used in treatment of vaginal atrophy of menopausal women: a randomized controlled clinical trial. Int J Community Based Nurs Midwifery. 2016;4:69–78. [PMC free article] [PubMed] [Google Scholar]
  • [10].Markowska J, Madry R, Markowska A. The effect of hyaluronic acid (Cicatridine) on healing and regeneration of the uterine cervix and vagina and vulvar dystrophy therapy. Eur J Gynaecol Oncol. 2011;32:65–8. [PubMed] [Google Scholar]
  • [11].Tea MKM, Priemer V, Kubista E. Wirksamkeit und Sicherheit von Hyaluron-Säure-Zäpfchen (Cikatridina(R)) beider Behandlung hormon-oder chemotherapieinduzierter vaginalerAtrophie bei Mammakarzinompatientinnen. Fertilität und Reproduktion 2006;16:17–9. [Google Scholar]
  • [12].Eaton AA, Baser RE, Seidel B, Stabile C, Canty JP, Goldfrank DJ, et al. Validation of clinical tools for vaginal and vulvar symptom assessment in cancer patients and survivors. J Sex Med. 2017;14:144–51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [13].Rosen R, Brown C, Heiman J, Leiblum S, Meston C, Shabsigh R, et al. The Female Sexual Function Index (FSFI): a multidimensional self-report instrument for the assessment of female sexual function. J Sex Marital Ther. 2000;26:191–208. [DOI] [PubMed] [Google Scholar]
  • [14].Baser RE, Li Y, Carter J. Psychometric validation of the Female Sexual Function Index (FSFI) in cancer survivors. Cancer. 2012;118:4606–18. [DOI] [PubMed] [Google Scholar]
  • [15].Thirlaway K, Fallowfield L, Cuzick J. The Sexual Activity Questionnaire: a measure of women’s sexual functioning. Qual Life Res. 1996;5:81–90. [DOI] [PubMed] [Google Scholar]
  • [16].Vistad I, Fossa SD, Kristensen GB, Mykletun A, Dahl AA. The sexual activity questionnaire: pychometric properties and normative data in a norwegian population sample. J Womens Health (Larchmt). 2007;16:139–48. [DOI] [PubMed] [Google Scholar]
  • [17].DeWalt DA, Rothrock N, Yount S, Stone AA. Evaluation of item candidates: the PROMIS qualitative item review. Med Care. 2007;45:S12–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [18].Flynn KE, Jeffery DD, Keefe FJ, Porter LS, Shelby RA, Fawzy MR, et al. Sexual functioning along the cancer continuum: focus group results from the Patient-Reported Outcomes Measurement Information System (PROMIS(R)). Psycho-oncology. 2011;20:378–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [19].Flynn KE, Reeve BB. Progress on the PROMIS (R) Sexual Function Measure. Qual Life Res. 2010;19:98–9. [Google Scholar]
  • [20].Fortune-Greeley AK, Flynn KE, Jeffery DD, Williams MS, Keefe FJ, Reeve BB, et al. Using cognitive interviews to evaluate items for measuring sexual functioning across cancer populations: improvements and remaining challenges. Qual Life Res. 2009;18:1085–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [21].Jeffery DD, Tzeng JP, Keefe FJ, Porter LS, Hahn EA, Flynn KE, et al. Initial report of the cancer Patient-Reported Outcomes Measurement Information System (PROMIS) sexual function committee: review of sexual function measures and domains used in oncology. Cancer. 2009;115:1142–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [22].Reeve BB, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45:S22–31. [DOI] [PubMed] [Google Scholar]
  • [23].Andersen BL, Woods XA, Copeland LJ. Sexual self-schema and sexual morbidity among gynecologic cancer survivors. J Consult Clin Psychol. 1997;65:221–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • [24].Day R, Ganz PA, Costantino JP, Cronin WM, Wickerham DL, Fisher B. Health-related quality of life and tamoxifen in breast cancer prevention: a report from the National Surgical Adjuvant Breast and Bowel Project P-1 Study. J Clin Oncol. 1999;17:2659–69. [DOI] [PubMed] [Google Scholar]
  • [25].Robson M, Hensley M, Barakat R, Brown C, Chi D, Poynor E, et al. Quality of life in women at risk for ovarian cancer who have undergone risk-reducing oophorectomy. Gynecol Oncol. 2003;89:281–7. [DOI] [PubMed] [Google Scholar]
  • [26].R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: The R Foundation; Updated September 6, 2019. https://www.r-project.org/. [Google Scholar]
  • [27].Pitsouni E, Grigoriadis T, Douskos A, Kyriakidou M, Falagas ME, Athanasiou S. Efficacy of vaginal therapies alternative to vaginal estrogens on sexual function and orgasm of menopausal women: A systematic review and meta-analysis of randomized controlled trials. Eur. J Obstet Gynecol Reprod Biol 2018;229:45–56. [DOI] [PubMed] [Google Scholar]
  • [28].Delia P, Sansotta G, Pontoriero A, Iati G, De Salvo S, Pisana M, et al. Clinical evaluation of low-molecular-weight hyaluronic acid-based treatment on onset of acute side effects in women receiving adjuvant radiotherapy after cervical surgery: a randomized clinical trial.” Oncol Res Treat. 2019;42(4):217–23. [DOI] [PubMed] [Google Scholar]

RESOURCES