Validation of the Activities Assessment Scale in Women undergoing Pelvic Reconstructive Surgery

Abstract

Objective

The Activities Assessment Scale (AAS) is a 13-item postoperative functional activity scale validated in men undergoing hernia surgery. We evaluated the psychometric characteristics of the AAS in women undergoing vaginal surgery for prolapse (POP) and stress incontinence (SUI).

Methods

Participants included 163 women with POP and SUI enrolled in a randomized trial comparing sacrospinous ligament fixation to uterosacral vault suspension with and without perioperative pelvic floor muscle training. Participants completed the AAS and SF-36 at baseline and 2-weeks and 6-months post-operatively. Internal reliability of the AAS was evaluated using Cronbach’s alpha. Construct validity and responsiveness were examined in cross-sectional and longitudinal data using Pearson’s correlation coefficient and ANOVA. The AAS is scored from 0–100 (higher scores = better function).

Results

Mean baseline AAS score was 87± 17.3 (range 25 to 100). Functional activity declined from baseline to 2-weeks post-operatively (mean change −4.5; 95% CI −7.6 to −1.42) but improved above baseline at 6-months (mean change +10.9; 95% CI 7.8 to 14.0). Internal reliability of the AAS was excellent (Cronbach’s Alpha = 0.93). Construct validity was demonstrated by a correlation of 0.59–0.60 between the AAS and SF-36 Physical Functioning Scale (p<0.0001) and lower correlations between the AAS and other SF-36 scales. Patients who improved in physical functioning based on the SF-36 between 2-weeks and 6-months postoperatively showed an effect size of 0.86 for change in the AAS over the same time period.

Conclusions

The AAS is a valid, reliable and responsive measure for evaluation of physical function in women after pelvic reconstructive surgery.

Introduction

Improving physical activity is an important goal for many women undergoing surgery for pelvic floor disorders.(1) As such, change in functional activities after surgery is an important measure of patient experience. While a number of studies have evaluated changes in physical activity after general surgery and orthopedic procedures, data is limited on the changes in physical activity after gynecologic surgery, particularly pelvic reconstructive surgery.(2–5) The Activity Assessment Scale (AAS) is a measure of functional activity designed for use in the perioperative period.(6) The AAS measures a full spectrum of low-intensity to high-intensity activities, references a more immediate time interval than many other measures, and requires less time to complete than most health status assessment scales.(6) This instrument was originally constructed to assess patient-level outcomes in a multi-center randomized clinical trial of open versus laparoscopic inguinal herniorrhaphy.(7) The AAS is reliable, valid and responsive using longitudinal data from 2,164 men enrolled in this trial. Women were not included in this herniorrhaphy trial.

Several studies show clinically important gender differences in response to pain and functional recovery after surgery.(8–11) Thus, the psychometric properties demonstrated in men in the original validation of the AAS may not be applicable to women. The objective of this study is to evaluate the validity, reliability and responsiveness of the AAS in women undergoing vaginal surgery for pelvic organ prolapse (POP) and stress urinary incontinence (SUI). Our goal was to determine if the AAS is a useful clinical instrument to track postoperative recovery from surgery as well as to assess changes in functional outcome postoperatively compared to baseline in women with pelvic floor disorders.

Materials and Methods

This is a planned supplementary study of the Operations and Pelvic Muscle Training in the Management of Apical Support Loss (OPTIMAL) trial, a multi-center randomized trial conducted by the Pelvic Floor Disorders Network (PFDN).(12) The principal aims of the OPTIMAL trial are 1) to compare surgical outcomes after sacrospinous ligament fixation (SSLF) versus uterosacral vault suspension (ULS) and 2) to assess the role of perioperative behavioral and pelvic floor muscle training versus usual care in women undergoing vaginal surgery for apical POP and SUI using a 2x2 randomized factorial design.(12) The PFDN is sponsored by The Eunice Kennedy Shriver National Institute of Child Health and Human Development and consists of seven clinical sites and a data coordinating center. A detailed description of the OPTIMAL trial methods is published elsewhere.(12)

All OPTIMAL trial participants are adult women with Stage II–IV POP and coexisting SUI symptoms, who have prolapse at the vaginal apex (with or without a uterus) and have opted for vaginal prolapse surgery. All PFDN sites obtained local institutional review board approval, and all participants completed informed consents. Enrolled subjects were randomized to receive either a SSLF or ULS; in addition all subjects received a tension-free vaginal tape (TVT). Subjects underwent a second randomization to either a perioperative behavioral and pelvic muscle training (PMT) program or usual care. Randomization was stratified by site with separate randomization schedules generated by the data-coordinating center using permuted blocks. Participants and evaluators remained blinded to treatment assignment. The perioperative behavioral therapy program began two to four weeks prior to surgery and continued for three months after surgery. The OPTIMAL trial started recruiting patients in February 2008 and follow-up of the trial is ongoing, so for purposes of this analysis, comparisons within and between the surgical and behavioral interventions were not performed and blinding of the participants, investigators and statistician was maintained. The study sample for this analysis includes the first 169 consecutive women enrolled in the OPTIMAL trial.

Demographic data, medical history and standardized pelvic examination were completed at enrollment. All participants completed several self-administered questionnaires that included the AAS and the SF-36 at baseline and 2-weeks and 6-months after surgery.(2, 9) At 2-weeks patients compared their functional ability to that before surgery on a 5-point Likert scale (“much worse” to “much better”). Postoperatively, all subjects received a standardized set of postoperative instructions, including instructions on resumption of physical activities. Subjects were instructed that light physical activities including stretching, inside and outside walking, climbing stairs, cooking, dusting, clerical work and visiting friends were acceptable and encouraged as soon as they felt comfortable with the activity. They were instructed to refrain from activities which caused them to feel pressure in the pelvic or vaginal area for 6-weeks after the surgery. Light jogging and aerobic machines were acceptable if not accompanied by a feeling of vaginal or pelvic pressure. Postoperative pain management was administered according to the standard at each clinical site. Beyond 6-weeks, subjects were instructed to resume all normal activities without restrictions.

The AAS includes 13 items covering a broad sample of sedentary, movement-related and graded-intensity physical activities.(6) (Figure 1) Respondents are asked to rate the degree of difficulty performing each of these activities in the previous 24 hours on a 5-point scale from “No difficulty” to “Not able to do it.” A “did not perform for other reasons” response item is also included but not scored. The AAS has three subscales: sedentary activities (items 1–4); ambulatory activities (items 6–8); work/exercise activities (items 11–13). The AAS total and subscale scores are transformed to produce a range of 0–100, with higher values indicating greater functional activity.

Figure 1.

Activities Assessment Scale

Internal consistency, validity and responsiveness of the AAS were assessed for this analysis. Internal consistency, which represents the extent to which the items on a scale are related to one another was assessed for the AAS with Cronbach’s alpha. Values of ≥ 0.70 are considered acceptable. The test-retest reliability was not assessed in this analysis. The validity of the AAS was assessed using the same general approach as used by McCarthy et al.(6) Construct validity considers the extent to which a scale or sub-scale produces scores consistent with those of another instrument with known characteristics. Two of the variants of construct validity measures, the convergent and discriminant validity, will be examined using a series of correlational analyses between the AAS and the eight dimensions of the SF-36.(13) Specifically, it was expected that the AAS and its subscales would have larger correlations with the physical functioning scale of the SF-36 and smaller correlations with other scales such as the mental health scale. Convergent and discriminant validity was assessed at baseline and 6-months postoperatively. Correlations between the AAS and the SF-36 physical functioning scale were evaluated at the 2-week visit as this is the point of assessment when physical functioning was expected to be the lowest. Validity of the AAS was also assessed by known group comparisons. Groups of subjects expected to have higher or lower functional status relative to other groups were compared using t-tests or ANOVA. Specifically, the AAS of subjects who indicated that they perform physical activities that require major effort (i.e. lifting furniture, shoveling snow or lifting people or objects more than 25lbs) at least once per week were compared to the scores of those who perform such activities less often. In a similar fashion, the AAS scores were compared between subjects with and without major co-morbidities (diabetes, cardiovascular, respiratory, hepatic, renal, neurologic or other endocrine-metabolic diseases).

As with validity, the responsiveness of the AAS was assessed using the same general approach as used in the original development of the instrument.(6) The effect of POP on functional activity is not well studied, but clinical experience would suggest that its impact is widely variable. In OPTIMAL we anticipated that the major factor impacting activity level is the surgery itself and not the subject’s pelvic floor condition. Therefore, we hypothesized that for an individual subject, functional activity is variable at baseline. At 2-weeks after surgery, functional activity should be lower than at baseline. By 6 months, functional activity should approach or improve beyond baseline levels. To examine the responsiveness of the AAS, we evaluated change over two periods: baseline to 2-weeks and 2-weeks to 6-months. Within each time frame subjects were classified into one of three groups (improvement, no change, or worsening) for functional activity based on the change in the SF-36 functional activity scales. The minimum clinically important difference (MCID) of the SF-36 physical functioning scale (range 0–100) is 2 points for physical functioning scores below 40 and 3 points for physical functioning scores above 40.¹³ Subjects were considered to have “no change” during the specific interval if the change in the specific scale score was within the interval of ± the MCID of that domain. Subjects were considered as “improving” if the increase in the score was greater than the MCID and “worsening” if the decrease in score was greater than the MCID. The direction and amount of change of the AAS was determined for each of these groups (improving, no change and worsening) for each of the two specified periods and compared using ANOVA.

Additionally, subjects were asked to compare their ability to perform typical daily activities to before they underwent surgery on a 5-point scale from “much better” to “much worse” at the 2-week visit. Based on the their response to these external anchors, subjects were classified as “improved,” “unchanged” or “worse.” As described above, the direction and amount of change of the AAS was determined for each of these groups for each of the two specified periods and compared using ANOVA. Another measure of responsiveness that was evaluated for AAS is effect size, or the change in mean values between two time points divided by the standard deviation of the baseline value. An effect size of 0.2–0.49 is considered small, 0.5–0.7 is considered moderate, and 0.8 and above is considered large.(14)

For convergent and divergent validity, it was determined that a sample size of 85 subjects was sufficient to detect correlations of 0.30 or greater at 80% power (alpha 0.05, two-sided test). In the assessment of responsiveness, 150 subjects provided at least 80% power (alpha 0.05, two-sided test) to detect differences using one-way ANOVA among the groups defined by improvement, no change and worsening in functioning during normal activity of the same magnitude and variability of those seen by McCarthy et al.(6), with the additional assumption that the proportion of subjects in the 3 categories is roughly 1/2:1/4:1/4, respectively.

Results

Of the 169 study participants, 163 (96%) completed the AAS and SF-36 at baseline and 2-weeks and 145 (86%) completed both questionnaires at 6-months. Participants had a mean ± SD age of 58 ± 11. The majority was Caucasian (89%) while 6% were African American and 5% were Asian or other; 21.5% of the sample identified themselves as Hispanic. At enrollment, 37% of the study sample had Stage II pelvic organ prolapse, 57% were Stage III and 6% were Stage IV. At baseline, the mean±SD AAS was 87± 17 (range 25 to 100). Functional activity declined from baseline to 2-weeks (mean change −4.5; 95% CI −7.6 to −1.42), but significantly improved above baseline levels at 6-months (mean change +10.9; 95% CI 7.8 to 14.0). The median number of items that subjects indicated that they “did not perform for other reasons” was 1 (intraquartile range [IQR] 0–2) at baseline and 6 months and 3 (IQR 2–4) at 2 weeks after surgery. Internal consistency of the AAS was excellent (Cronbach’s Alpha = 0.93). The three AAS subscales demonstrated moderate correlations with each other (r = .56 to .69, p <0.0001 for each).

Table 1 shows correlations between the AAS and SF-36 subscales at baseline, 2-weeks, and 6-months after surgery. The AAS demonstrated good construct validity. Convergent validity is evident with correlations of 0.6, 0.6, and 0.59 between AAS and SF-36 Physical Functioning scale at baseline, 2-weeks, and 6-month respectively. The Bodily Pain scale also demonstrated moderate correlations with the AAS at each time point (r = 0.55–0.51). Divergent validity is demonstrated by lower correlations between the AAS and other SF-36 scales, which are not expected to measure functional activity such as mental health (r = 0.29–0.40). Similar patterns of relationships were seen with the AAS subscales. The correlations between the sedentary, ambulatory, work/exercise subscales and the SF-36 Physical Functional scale were 0.5, 0.56 and 0.5 at baseline and 0.45, 0.65, and 0.5 at 6 months, p<0.0001 for each.

Table 1.

Correlation between SF-36 Subscales and Total AAS Score

Subscale	Baseline			2 week follow-up			6 month follow-up
	N	Pearson Correlation (95% CI)	p-Value*	N	Pearson Correlation (95% CI)	p-Value*	N	Pearson Correlation (95% CI)	p-Value*
PHYSICAL FUNCTIONING	162	0.60 (0.48, 0.69)	<.0001	158	0.60 (0.49, 0.69)	<.0001	151	0.59 (0.48, 0.69)	<.0001
ROLE PHYSICAL	162	0.60 (0.49, 0.69)	<.0001	157	0.39 (0.25, 0.52)	<.0001	151	0.55 (0.43, 0.65)	<.0001
BODILY PAIN	159	0.55 (0.43, 0.65)	<.0001	159	0.57 (0.45, 0.66)	<.0001	151	0.51 (0.38, 0.62)	<.0001
GENERAL HEALTH	161	0.50 (0.37, 0.61)	<.0001	159	0.26 (0.11, 0.4)	0.0009	150	0.42 (0.28, 0.54)	<.0001
VITALITY	161	0.41 (0.27, 0.53)	<.0001	159	0.23 (0.08, 0.37)	0.0031	151	0.37 (0.22, 0.5)	<.0001
SOCIAL FUNCTIONING	161	0.51 (0.38, 0.61)	<.0001	159	0.43 (0.29, 0.55)	<.0001	151	0.39 (0.25, 0.52)	<.0001
ROLE EMOTIONAL	160	0.47 (0.33, 0.58)	<.0001	157	0.31 (0.16, 0.45)	<.0001	151	0.37 (0.22, 0.5)	<.0001
MENTAL HEALTH	161	0.40 (0.26, 0.52)	<.0001	159	0.36 (0.21, 0.49)	<.0001	151	0.29 (0.14, 0.43)	0.0003

^*testing whether the correlation is different from zero.

However, baseline AAS scores and subscale scores did not differ amongst women performing physical activities requiring major effort more than once a week and those performing such activities less than once a week (AAS total score: 82.6+/−17.4 vs. 81.9+/−17.4, p =0.80). The AAS total and subscale scores were also not significantly different between women with and without medical comorbidities (AAS total score: 85.5+/−16.8 vs. 81.1+/−17.5, p = 0.12).

The mean change in AAS scores for subjects whose physical functioning was classified as improved, unchanged or worse based on the SF-36 Physical Functioning scale from baseline to 2-weeks and 2-weeks to 6-months are shown in Tables 2 and 3, respectively. During both time periods, the mean change scores between the three groups were significant (p<0.0001 and p=0.02 respectively). From baseline to 2-weeks postoperatively, the largest change in the AAS score was for subjects whose physical functioning was “worse” (106/162; 65%) with an average decrease of −10.8±22.5 (effect size −0.71) and the smallest for those who were classified as “unchanged” (23/162; 14%) with a mean change of −2.2±29.0 (effect size −0.12). From 2-weeks to 6-months, mean AAS scores improved in all three SF-36 Physical Functioning subgroups. Eighty-one percent (123/151) were classified as having improved physical functioning during this time period and this group had the greatest improvements in AAS scores (mean change +17.4±19; effect size 0.86). The mean AAS change was lowest on the 14 women who were classified as worse (mean change +4.2±24.5; effect size 0.21). The patterns shown for the change in the AAS score over each time period were similar for each of the AAS subscales (data not shown).

Table 2.

Mean change and responsiveness in Activity Assessment Scale score from baseline to 2-weeks postoperatively by change in SF-36 Physical Function

SF-36 Physical Function	N Total=161	mean (SD) at baseline	mean (SD) at 2 week follow-up	Mean AAS Change (SD)	Effect size	P-value*
Improved	32	72.0 (19.5)	82.7 (14.0)	+ 8.2 (25.5)	0.42	<0.0001
No Change	23	84.0 (18.1)	85.5 (19.4)	−2.2 (29.0)	−0.12
Worse	106	85.5 (15.2)	75.2 (21.1)	−10.8 (22.5)	−0.71

^*F-statistics testing whether AAS change differs among the three categories according to SF-36 physical functioning subscale.

^#For baseline < 40: +/− 2 points for “no change” category, > 2 for “improving”, < −2 for “worsening”;

For baseline ≥ 40: +/− 3 points for “no change” category, > 3 for “improving”, < −3 for “worsening”.

Table 3.

Mean change and responsiveness in Activity Assessment Scale score from 2-weeks to 6-months postoperatively by change in SF-36 Physical Function

SF-36 Physical Functioning	N Total = 148	mean (SD) at 2-weeks	mean (SD) at 6 month follow-up	Mean AAS Change (SD)	Effect size	P- value*
Improved	123	76.77 (20.24)	94.19 (12.08)	17.42 (19)	0.86	0.020
No Change	11	85.57 (13.78)	93.15 (9.34)	7.57 (12.4)	0.55
Worse	14	81.4 (19.66)	85.55 (17.82)	4.15 (24.5)	0.21

^*F-statistics testing whether AAS change differs among the three categories according to SF-36 physical functioning subscale.

^#For 2-week < 40: +/− 2 points for “no change” category, > 2 for “improving”, < −2 for “worsening”;

For 2-week ≥ 40: +/− 3 points for “no change” category, > 3 for “improving”, < −3 for “worsening”

Two weeks after surgery subjects were asked to compare their ability to do activities that they might do on a typical day to their ability before surgery on a 5-point scale form “much better” to “much worse.” Fifty-two participants (32%) rated their ability to do physical activities 2-weeks after surgery as “a little worse” or “much worse”. This group had the greatest change in AAS scores during this time period (mean change −19.6±24.3; effect size −1.57). In contrast, those who said they had “no change” or were “a little better” or “much better” had significantly smaller changes in the AAS scores (mean change 2.1 ±17.5 and 3.6 ±25.1 and effect sizes of 0.11 and 0.18, respectively).

Discussion

A study of women with stage II–IV POP planning sacrocolpopexy reported that prior to surgery 27% felt that their POP interfered substantially with their ability to exercise or perform recreational activities and 19% felt that it interfered with housework or yard work.(15) One year after surgery, few had increased their major effort activities, but one-third had increased exercise and most reported that POP no longer interfered with their activities.(16) Epidemiologic studies suggest a relationship between heavy occupational work and prolapse.(17, 18) In one study of women planning surgery for SUI, almost three-quarters of women stated that they limited their physical activities because of the urinary leakage.(19) Beyond these few studies, however, little is known about how POP, SUI or their treatment affects physical activity.

Physical functioning is an important aspect of the day-to-day experience of patients before and after surgery. In spite of its obvious importance, few studies of gynecologic surgery have assessed functional status in the perioperative period. The AAS measures functional status in the perioperative period covering a spectrum of activities from sedentary to graded intensity physical activities. The scale is validated in men undergoing herniorrhaphy, but has not been validated in women.(6) Overall our findings provide further support that the AAS is a valid, reliable and responsive measure that can be used to evaluate physical functioning in women undergoing pelvic reconstructive surgery. Similar to the work by McCarthy et al., we found that the AAS is a coherent scale with good internal reliability and with evidence of convergent and discriminant validity.(6) The internal reliability as measured by Cronbach’s alpha was excellent. As predicted at each time point, AAS scores were highly correlated with the SF-36 physical functioning scale while the correlations were appreciably lower with the SF-36 mental health scale and other subscales not directly related to physical functioning. Unlike findings from McCarthy’s study, this study did not find AAS scores prior to surgery to be significantly different based on the presence or absence of co-morbidities, or based on the regularity subjects reported performing activities requiring major effort. However, the effect size of the difference between groups in this study was very similar to that in the McCarthy’s study (0.25 vs 0.31); it did not achieve statistical significance due to a substantially smaller sample size in our study.

Another important consideration is the ability of a measure to be responsive to clinically meaningful changes over time. As was predicted, mean physical functioning as measured by the AAS declined significantly from baseline to 2-weeks after surgery then improved above baseline by 6-months after surgery. In the 2-week postoperative period, the AAS mirrored the changes seen in the SF-36 physical functioning scale. The majority of patients had a decline in physical functioning during this time period with an appropriate decline in AAS score (effect size of 0.71). By 6-month postoperatively, 85% of subjects had an improvement in their physical functioning based on the SF-36, and the responsiveness of the AAS in this group was excellent with an effect size of 0.86. Surprisingly, AAS scores also improved in the small number of patients who had “no change” (n=11) or worsened (n = 14) based on SF-36, albeit to a lesser degree than those who improved. McCarthy et al reported a similar pattern when evaluating changes 3 months after hernia surgery which they interpreted as artifact.(6)

Unlike that seen for the SF-36, we did not observe good discrimination of mean change in AAS scores as defined by patient responses 2 weeks after surgery to the single item that asked subjects to describe their current ability to do activities compared to before surgery (better/no change/worse). Large changes in the AAS were noted for the group who considered themselves “worse” based on this item but little change was noted for those who felt they had “no change” or were “better.” There are several factors that may have contributed to this apparent discrepancy. First, this item was only assessed at 2-weeks postoperatively and it seems unlikely that many women would have resumed normal physical functioning by this time period, particularly given the standard postoperative instructions. Study participants may have interpreted the wording of the single item external anchor (i.e., activities you do in a typical day) to include activities beyond physical functioning, such as social activities. In addition, external anchors asking for comparisons before a treatment event like the one used in this study are subject to recall bias. Other investigators have suggested they are potentially problematic because patients’ assessment of their past health might be influenced by their current health status.(20, 21)

Some potential advantages of the AAS for evaluating physical function includes that it is specifically designed for the perioperative period and as such measures a more immediate time interval (previous 24 hours) than the other commonly used measure (e.g., the SF-36 uses 4 weeks). This allows clinicians and researchers the possibility of monitoring more immediate changes in physical functioning corresponding to changes in treatment and/or tracking multiple data points to describe a trajectory of improving or worsening physical functioning over time. The AAS measure is also relatively short, taking three to five minutes to complete, an attractive feature when considering patient burden. Its length also allows the AAS to be paired with other measures depending on the specific aims of a study.

The strengths of this study include a well-characterized sample of women with pelvic floor disorders from multiple centers in the United States and the longitudinal design that assessed the AAS at baseline and at two post-surgery follow-up intervals. All subjects underwent surgery for POP and SUI via a vaginal approach using a standard protocol. This standardization provides some advantages, but may also have limited the variability of the postoperative experience of study participants. One of the principal advantages of vaginal surgery is its quicker recovery and lower postoperative pain relative to other approaches particularly surgeries requiring laparotomy. As such there is a greater risk of a “ceiling effect” in this study than one that included procedures likely to result in a greater decline in functional activity after surgery such as abdominal sacral colpopexy. Additional research in different populations and with larger samples would be valuable to further evaluate the psychometric properties of the AAS. Another potential limitation of our study is that in the 4–6 week postoperative period subjects were instructed not to participate in strenuous physical activities. Some of the decline in physical activity seen in the 2-week postoperative interval may not have been a result of the surgery itself but a self-imposed decrease in activity based on the postoperative instructions. Finally, we did not assess the test-retest reliability of the AAS in this study, however, these properties were excellent in McCarthy’s study.(6)

In summary, the Activities Assessment Scale is a valid, reliable and responsive measure that researchers and clinicians may find useful in evaluating physical functioning in women after pelvic reconstructive surgery. It can be used to compare different groups at the same time point or the same group of patients at multiple time points.