A factor analysis approach for clustering patient reported outcomes

Summary

Background

In the field of radiation oncology, the use of extensive patient reported outcomes is increasingly common to measure adverse side effects after radiotherapy in cancer patients. Factor analysis has the potential to identify an optimal number of latent factors (i.e., symptom groups). However, the ultimate goal of treatment response modeling is to understand the relationship between treatment variables such as radiation dose and symptom groups resulting from FA. Hence, it is crucial to identify clinically more relevant symptom groups and improved response variables from those symptom groups for a quantitative analysis.

Objective

The goal of this study is to design a computational method for finding clinically relevant symptom groups from PROs and to test associations between symptom groups and radiation dose.

Methods

We propose a novel approach where exploratory factor analysis is followed by confirmatory factor analysis to determine the relevant number of symptom groups. We also propose to use a combination of symptoms in a symptom group identified as a new response variable in linear regression analysis to investigate the relationship between the symptom group and dose-volume variables.

Results

We analyzed patient-reported gastrointestinal symptom profiles from 3 datasets in prostate cancer patients treated with radiotherapy. The final structural model of each dataset was validated using the other two datasets and compared to four other existing FA methods. Our systematic EFA-CFA approach provided clinically more relevant solutions than other methods, resulting in new clinically relevant outcome variables that enabled a quantitative analysis. As a result, statistically significant correlations were found between some dose-volume variables to relevant anatomic structures and symptom groups identified by FA.

Conclusion

Our proposed method can aid in the process of understanding PROs and provide a basis for improving our understanding of radiation-induced side effects.

1. Introduction

Patient reported outcomes (PROs) are becoming more common as a means of understanding the details of patient experience following treatment (1–3). In particular, in the field of radiation oncology, extensive questionnaires to measure side effects after radiotherapy (RT) in cancer patients from PROs have been used to supplement objective assessments by healthcare professionals. While they may be more comprehensive than observer-based approaches to measuring response, it remains challenging to efficiently group symptoms shown in a questionnaire and to understand the relationship between radiation dose and adverse side effects. To tackle these issues, we propose a novel method to find clinically relevant symptom groups using factor analysis (FA) and show that a linear combination of symptoms in a group can be used as a new outcome variable to investigate the relationship between radiation dose and side effects after RT.

FA is a statistical method to investigate the relationship between items in a dataset (4). In FA, the variance of a factor is represented by an eigenvalue that is computed as the sum of its squared factor loadings for all the items. FA has previously been deemed useful to examine the relationship amongst different adverse effects after RT for various cancers (1, 5–8). In these studies, FA was typically applied to a limited number of items related to treatment characteristics (1, 7, 9), quality of life (6, 8, 9), or symptoms reflecting specific RT-induced injuries (1, 5–7). FA is likely to be even more useful as the dimensionality of the data increases such as in questionnaires using a clinicmetric atomized approach to document patient-reported outcomes, in which some of the included symptoms are also assumed to be related (10–12). Hence, identifying common characteristics from the co-variation between symptoms, as obtained from FA, is appealing for such extensive questionnaires.

Identifying a number of likely meaningful factors is one of the key challenges using FA (13–16). Despite many efforts to design robust methods to determine the optimal number of factors in FA, a conclusive gold standard method has not yet been proposed. In this study, we propose a novel method to choose the clinically relevant number of factors and factor loading cutoffs. In this method, factor structures obtained in exploratory FA (EFA), including a various number of factors and factor loading cutoffs, are submitted to confirmatory FA (CFA). The number of factors and factor loading cutoffs are assessed by each model’s comparative fit index (CFI) or root mean square error of approximation (RMSEA) as obtained in the CFA. We applied this method to 27 patient-reported questions on gastrointestinal (GI) symptoms following RT for prostate cancer in three different datasets. We used the clinical symptom categorization, previously described in (17), to compare the ability of our method in determining a clinically relevant number of factors and factor loading cutoffs with that of the aforementioned four existing methods. We also investigated associations between radiation dose to various segments of the GI tract and latent factor-based outcome variables for the identified symptom groups using linear regression analysis.

2. Related studies

Several objective and subjective methods have been developed to find the optimal number of factors in FA, primarily by focusing on the pattern of eigenvalues. This is typically done through a graphical representation where eigenvalues are shown in descending order against the number of factors, the so-called ‘Cattell’s scree plot’ (18). One of the frequently applied methods is Kaiser’s eigenvalue-greater-than-one rule (Kaiser’s rule) where an eigenvalue threshold of one (the arithmetic mean for all eigenvalues) is used to decide the number of factors to retain (19). In parallel analysis (PA), random datasets with the same number of samples and items as in the original dataset are simulated. FA performed on each dataset generates a distribution of eigenvalues of which the eigenvalues in the original dataset that are greater than the 95^th percentile eigenvalues derived from the random datasets are retained (20). Optimal coordinates (OC) and acceleration factor (AF) are two additional methods that are less subjective than the Kaiser’s rule (21). For OC, in a scree plot, an eigenvalue i is extrapolated with a linear equation connecting two points, the eigenvalue i + 1 and the last eigenvalue. The factors whose observed eigenvalues exceed the extrapolated eigenvalues are retained. A cut-off point for the number of factors in AF is determined at the most abrupt change of the slope in the scree plot using the second derivative. If an i^th point is identified, the number of factors to retain is i − 1. Another key challenge, when using FA, is how to decide an appropriate factor loading cutoff for which items whose loadings are larger than the cutoff are considered to be meaningful and have high correlation with their latent variable. Previously suggested factor loading cutoffs are recommended based on individual experience rather than any specific method and range from 0.3 to 0.5 (5, 9).

3. Materials and methods

3.1 Factor analysis

Suppose that X = (X₁, ···, X_p) is a set of p observable random variables with mean vector μ= (μ₁, ···, μ_p). The FA model is defined as

$$X=\mu +\wedge F+\epsilon ,$$ (1)

where F = (F₁, ···, F_k) is a set of k common factors, Λ = (λ_ij) is a p × k matrix of factor loadings, and ε = (ε₁, ···, ε_p) is a set of p specific factors. With respect to F and ε, we have a list of assumptions for FA: (1) the common factors have a mean equal to zero and a variance equal to one, i.e., E(F) = 0 and Var(F) = 1; (2) the specific factors have a mean equal to zero and the variance of a specific factor ε_i (i = 1,2, ···, p) is ψ_i that is called specific variance, i.e., E(ε) = 0 and Var(ε_i) = ψ_i; (3) the common factors are independent of each other, i.e., Cov(F_i,F_j) = 0 for i ≠ j = 1,2, ···, k where Cov indicates covariance; (4) the specific factors are independent of each other, i.e., Cov(ε_i, ε_j) = 0 for i ≠ j = 1,2, ···, p; (5) F_i and ε_j are independent of each other, i.e., Cov(F_i, ε_j) = 0 for i = 1,2, ···, k and j = 1,2, ···, p. From the above assumptions, we have Cov(F) = I where I is an identity matrix of order k and Cov(ε) = Ψ = diag(ψ₁, ···, ψ_p).

Under these assumptions, let Cov(X − μ) denote Σ that can be expressed as

$$\mathrm{\sum }=\wedge {\wedge }^{\prime }+\mathrm{\Psi },$$ (2)

where Cov(X_i, F_j) = λ_ij, $\mathit{Cov}\left(X_i,X_j\right)={\sigma }_{ij}={\mathrm{\sum }}_{m=1}^k{\lambda }_{im}{\lambda }_{jm}$, and $\mathit{Var}(X_i)={\sigma }_i={\mathrm{\sum }}_{j=1}^k{\lambda }_{ij}^2+{\psi }_i$. ${\mathrm{\sum }}_{j=1}^k{\lambda }_{ij}^2$ is called Communality of item X_i that represents the variance of X_i shared with the other items through the common factors. The parameters of the FA model can be estimated using either the maximum likelihood or the eigen-decomposition method. We implemented scripts for EFA and CFA using the R language packages psych (22) and lavaan (23), respectively.

3.2 Our suggested EFA-CFA method

A key difference between EFA and CFA is that EFA is conducted to determine a factor structure by exploring the data without prior hypothesis, whereas CFA is conducted to test a hypothesized structure that is already known (24). In our proposed method, we first performed EFA with a different number of factors. Structural models, including items whose factor loadings are larger than a cutoff and factors with ≥ 2 items per factor after applying the cutoff (6), were submitted to CFA, where a CFI and an RMSEA were calculated as performance metrics that indicate the extent to which the factor model fits the data (goodness-of-fit). This procedure was systematically carried out with a range of factors from 1 to 9 and a range of factor loading values from 0.30 to 0.50 in steps of 0.05. A final structural solution was identified for interpretation with a specific number of factors and a specific factor loading value, which results in the highest CFI (> 0.9 denotes a very good fit) along with the lowest RMSEA (< 0.1 denotes a very good fit). The following equation shows the underlying idea of the proposed method:

$$k^{\ast }\text{and}{\lambda }^{\ast }=\underset{\mathit{CFI}}{\text{argmax}}{f}_{\mathit{CFA}}(f_{\mathit{EFA}}(k,\lambda ))\text{or}\underset{\mathit{RMSEA}}{\text{argmin}}{f}_{\mathit{CFA}}(f_{\mathit{EFA}}(k,\lambda ))$$ (3)

where k^* and λ^* indicate the optimal number of factors and a factor loading cutoff to obtain the highest CFI along with the lowest RMSEA. In this study, a direct oblimin rotation was used in the EFA (6, 7).

Based on these results, we investigated the association between the identified symptom groups and radiation dose to three segments of the GI tract (anatomically inferiorly to superiorly: the anal sphincter region, the rectum, and the most caudal 4 cm of the sigmoid colon; Figure 1) following the structure definitions and delineation procedures in (25). For each symptom group, an outcome variable was derived from the following equation:

$$f_{ij}={\mathrm{\sum }}_{k=1}^{r_i}{\lambda }_{k}I(q_{jk}),$$ (4)

where f_ij indicates a value (factor score) of the outcome variable for the i^th symptom group (consisting of r questions) of the j^th patient and I(·) denotes the indicator function which is equal to 1 for the presence of a symptom in the same patient for the corresponding question q, otherwise 0. In this calculation, the factor loading λ is used as a weighting value, indicating the contribution of the corresponding question to the symptom group. Then, the factor score was modeled using simple linear regression:

$$f_j={\beta }_0+{\beta }_1{x}_j+e_j,j=1,2,\cdots ,n$$ (5)

where β₀ and β₁ are regression parameters to be determined from the data with n samples, x_j are dosimetric values and e_j are random errors with e_j~N(0, σ²). Therefore, f_j~N(β₀+β₁x_j,σ²).

Figure 1.

The anal sphincter (red), rectum (green), and the most caudal 4 cm of the sigmoid colon (blue), displayed in a treatment plan for radiotherapy of a representative prostate cancer patient with the dose distribution overlaid (for a slice in sagittal view).

3.3 Datasets

All five FA methods including our proposed method were investigated in each of the three datasets, which included men previously treated with RT for localized prostate cancer at the Sahlgrenska University Hospital in Gothenburg, Sweden from 1993 to 2006 (primary external beam RT [EBRT]): n = 277, median follow-up = 6.4 years; surgical prostatectomy followed by salvage EBRT (POSTOP): n = 191, median follow-up = 3.2 years; a combination of primary EBRT and brachytherapy (EBRT+BT): n = 344, median follow-up = 4.9 years). The total prescribed dose to the tumor/postoperative prostatic region was typically 70 Gy in 2 Gy fractions (50 Gy in 2 Gy fractions/20 Gy in 10 Gy fractions for EBRT/BT in EBRT+BT) delivered by 15 MV photon beam quality (17). The men answered a study-specific questionnaire surveying symptoms after pelvic RT with 27 questions directly reflecting potential RT-related gastrointestinal injuries (questions and symptoms will be used interchangeably in this paper; supplementary Online Appendix A). The degree of each symptom was scored from 0 (no symptom) to 1 or 3 ~ 6 depending on the question. The original study was approved by the regional ethical review board in Gothenburg and the informed consent of all participating subjects was obtained.

The suitability of applying FA to these datasets was evaluated using the Kaiser-Meyer-Olkin (KMO) test and the internal consistency of symptoms was assessed using Cronbach’s alpha with acceptable values ≥ 0.60 (5) and ≥ 0.70 (1), respectively. Symptoms with low prevalence in any dataset were excluded from analysis to prevent spurious results (26).

4. Results

4.1 Data quality test

For EBRT, POSTOP, and EBRT+BT, KMO was 0.80, 0.70, and 0.78, and Cronbach’s alpha was 0.87, 0.85, and 0.84, respectively, which were considerably higher than the recommended values. Since one symptom (question ID = 16; supplementary Online Appendix A) occurred very rarely in POSTOP (< 2%), we excluded this question from our analysis.

4.2 Determination of the number of clinically meaningful factors

With our EFA-CFA method separately tested using each dataset, 3 or 4-factor models with a factor loading cutoff of 0.5 produced the highest CFI along with the lowest RMSEA across all three datasets (Table 1). In particular, in both EBRT and EBRT+BT, CFI was > 0.9 and RMSEA was < 0.1. For all three cohorts, AF, OC, and PA yielded 1, 5, and 5-factor models, respectively, whereas Kaiser’s rule resulted in 8-factor models in EBRT and POSTOP and a 9-factor model in EBRT+BT (Figures 2A–2C). Most of the factor models with 5 or larger factors did not fulfill the selection criterion (i.e., ≥ 2 questions/factor), and some of the exploratory solutions that met the selection criterion failed to converge in CFA due to poorly specified models. In fact, this was expected since our questionnaire consists of only 27 questions.

Table 1.

CFA results for exploratory solutions with different factor loading cutoffs.

		POSTOP		EBRT		EBRT+BT
No. of factors	Loading	CFI	RMSEA	CFI	RMSEA	CFI	RMSEA
1	0.30	0.47	0.16	0.62	0.12	0.61	0.13
	0.35	0.55	0.16	0.67	0.12	0.63	0.13
	0.40	0.58	0.18	0.74	0.12	0.65	0.15
	0.45	0.68	0.19	0.83	0.13	0.70	0.19
	0.50	0.70	0.19	0.93	0.10	0.68	0.25
3	0.30	0.66	0.13	0.75	0.10	0.76	0.10
	0.35	0.66	0.13	0.81	0.09	0.76	0.10
	0.40	0.69	0.14	0.85	0.09	0.78	0.10
	0.45	0.78	0.14	0.90	0.08	0.93	0.09
	0.50	0.86	0.13	0.94	0.07	1.00	0.02
4	0.30	0.65	0.12	0.79	0.09	0.81	0.09
	0.35	0.65	0.13	0.82	0.08	0.79	0.09
	0.40	0.66	0.13	0.84	0.09	0.91	0.08
	0.45	0.75	0.12	0.90	0.07	0.93	0.08
	0.50	0.83	0.11	0.94	0.06	0.96	0.08
5	0.30	0.73	0.11	NA	NA	NA*	NA*
	0.35	NA	NA	NA	NA	NA*	NA*
	0.40	NA	NA	NA	NA	NA*	NA*
	0.45	NA	NA	NA	NA	NA	NA
	0.50	NA	NA	NA	NA	NA	NA

NA means that the exploratory model did not fulfill ≥2 symptoms in any symptom group;

^*denotes that the model failed to converge in CFA.

Figure 2.

The number of factors identified by Kaiser’s rule (8–9 factors: black open circles), PA (5 factors; green triangles), OC (5 factors; red lines), and AF (1 factor; black closed circles) in (A) POSTOP, (B) EBRT, and (C) EBRT+BT. Note: in AF, the final number of factors is i − 1, if an i^th point is identified.

4.3 Model validation

From our EFA-CFA method, 3 or 4-factor models resulted in the best goodness-of-fit in all three datasets. Therefore, we hypothesized that the symptom profiles for the patients, although being treated with three different treatment modalities, were ‘similar enough’ to be captured by the factors of a single model and thereby enable a validation of models across datasets.

Using a 4-factor model with a loading threshold of 0.5 as a final structural solution for all three cohorts, we validated the final exploratory model in each cohort with the other two datasets. For example, a 4-factor model constructed using EBRT was validated using POSTOP and EBRT+BT, separately, without retraining (Table 2 shows the question ID, original symptom category, loading, and communality for these final models (EBRT: CFI = 0.94, RMSEA = 0.06; EBRT+BT: CFI = 0.96, RMSEA = 0.08; POSTOP: CFI = 0.83, RMSEA = 0.11); the original symptom category that was previously defined based on our clinical experience was brought from the supplementary Online Appendix A). The structural models in EBRT and EBRT+BT obtained relatively high CFI (0.87, 0.97) and low RMSEA values (0.06, 0.09) for the other two datasets (Figure 3), but the performance of the structural model in POSTOP was lower (CFI: 0.75, 0.77; RMSEA: 0.10, 0.11).

Table 2.

Four-factor solutions with a factor loading cutoff of 0.50 for the three investigated cohorts.

Symptom group	Cohort	Question ID		Symptom category	Loading	Communality
Defecation urgency	EBRT		3	Urgency	0.78	0.61
			4	Urgency	0.72	0.52
	EBRT+BT		3	Urgency	0.85	0.73
			4	Urgency	0.83	0.69
Fecal leakage	POSTOP	A	27	Leakage	0.64	0.42
			34	Leakage	0.55	0.32
		B	14	Leakage	0.72	0.57
			26	Leakage	0.63	0.47
			29	Leakage	0.89	0.79
			32	Leakage	0.53	0.31
	EBRT		25	Sensory symptoms	0.57	0.35
			26	Leakage	0.87	0.77
			28	Leakage	0.54	0.31
			29	Leakage	0.62	0.39
			30 c	Leakage	0.59	0.37
			32	Leakage	0.67	0.49
	EBRT+BT		26	Leakage	0.88	0.77
			29	Leakage	0.76	0.59
			30 c	Leakage	0.67	0.46
Pain	POSTOP		5	Abdominal pain and cramps	0.64	0.45
			6	Abdominal pain and cramps	0.85	0.73
			24	Flatulence	0.52	0.30
	EBRT		5	Abdominal pain and cramps	0.74	0.56
			6	Abdominal pain and cramps	0.71	0.50
			24	Flatulence	0.58	0.38
	EBRT+BT		5	Abdominal pain and cramps	0.75	0.56
			6	Abdominal pain and cramps	0.89	0.80
Mucous	POSTOP		18	Stool content (Mucous)	0.94	0.89
			19	Leakage (Mucous)	0.73	0.57
			20	Leakage (Mucous)	0.50	0.32
			21	Stool content (Mucous)	0.61	0.53
	EBRT		18	Stool content (Mucous)	0.62	0.43
			19	Leakage (Mucous)	0.87	0.78
			20	Leakage (Mucous)	0.59	0.35
	EBRT+BT		18	Stool content (Mucous)	0.95	0.90
			19	Leakage (Mucous)	0.54	0.42
			21	Stool content (Mucous)	0.64	0.44

Figure 3.

Validation of the 4-factor models from each dataset (left) in the other two datasets (right).

As shown in Table 2, the questions in all EFA-CFA symptom groups overall agreed very well with the original symptom categories (supplementary Online Appendix A). However, in POSTOP, the symptom group for defecation urgency was not identified and two symptom groups (A and B) for fecal leakage were identified. The correlation coefficient between symptom groups A and B was low (0.21), implying the unique interpretation of each symptom group.

4.4 Association test between radiation dose and symptom groups

Two dose-volume metrics were extracted from each of the three GI tract segments: the maximum dose (D_max) and mean dose (D_mean) for POSTOP and EBRT (supplementary Online Appendix B). Owing to uncertainties regarding the BT dose contribution, we did not analyze EBRT+BT. Using Eq. (4), factor scores of the 4 symptom groups with a loading threshold of 0.5 in POSTOP and EBRT were computed. Using linear regression analysis, D_max and D_mean in the anal sphincter and rectum significantly predicted both fecal leakage groups for POSTOP (Table 3). For EBRT, rectum D_max and D_mean, and rectum D_max were associated with defecation urgency and mucous, respectively. The relationship between rectal D_mean and one of the generated outcome variables (‘Fecal leakage A’ in POSTOP) is illustrated in Figure 4. In 27 out of 191 patients with mean dose less than 30 Gy in rectum, no patient experienced fecal leakage. Beyond 30 Gy, as mean dose increased, the risk of fecal leakage increased linearly as well.

Table 3.

Simple linear regression analysis using maximum dose (D_max) and mean dose (D_mean) in sigmoid colon, anal sphincter and rectum associated with symptom groups. Bold indicates p ≤ 0.05. In the column of Symptom group, the number indicates each symptom group’s averaged factor score (and standard deviation in parenthesis).

			Sigmoid colon (the most caudal 4 cm)		Anal sphincter		Rectum
Cohort	Symptom group	Predictor	Coefficient (β1)	p	Coefficient (β1)	p	Coefficient (β1)	p
POSTOP	Fecal leakage (A) 0.05(0.20)	Dmax	0.001	0.26	0.003	0.41	0.022	0.032
		Dmean	0.001	0.67	−0.003	0.039	0.003	0.020
	Fecal leakage (B) 0.26(0.57)	Dmax	−0.0004	0.86	0.024	0.045	0.035	0.24
		Dmean	−0.005	0.37	0.013	0.002	0.006	0.13
	Mucous 0.47(0.78)	Dmax	0.001	0.80	0.030	0.07	0.014	0.73
		Dmean	0.003	0.72	0.009	0.09	0.002	0.70
	Pain 0.43(0.61)	Dmax	0.003	0.25	−0.013	0.30	−0.005	0.89
		Dmean	0.007	0.21	−0.002	0.71	0.010	0.031
EBRT	Defecation urgency 0.98(0.72)	Dmax	0.002	0.28	0.004	0.68	0.037	0.016
		Dmean	−0.0002	0.96	0.004	0.43	0.011	0.021
	Fecal leakage 0.57(0.99)	Dmax	0.002	0.39	−0.012	0.42	−0.015	0.47
		Dmean	−0.001	0.79	0.003	0.61	0.009	0.17
	Mucous 0.33(0.64)	Dmax	0.001	0.38	−0.001	0.92	0.028	0.040
		Dmean	−0.003	0.42	0.002	0.56	0.007	0.11
	Pain 0.44(0.64)	Dmax	−0.001	0.42	−0.006	0.54	0.013	0.35
		Dmean	−0.003	0.44	−0.001	0.88	0.004	0.35

Figure 4.

For a symptom group “fecal leakage (A)” in POSTOP, the factor score estimated using Eq. (4) was plotted as a function of mean dose (D_mean) in rectum with 10 Gy interval. The circle indicates the position of averaged mean dose and averaged factor score at each bin. The numbers above the circles represent the number of patients falling into each bin. The error bar represents standard error.

4.5 Simulations

To further validate our model, we designed simulation models based on the observed loadings (Λ̂) and correlation coefficients ( $\widehat{\mathit{Cov}}(F)$) between factors for POSTOP, EBRT, and EBRT+BT cohorts (supplementary Online Appendix C). Simulation data were generated using the following equation:

$$X~N{(0,1)}_{n\times k}\times \widehat{\mathit{Cov}}(F)\times {\widehat{\wedge }}^{\mathrm{T}}+N{(0,1)}_{n\times p}\times \mathit{diag}\left(\sqrt{1-{\mathrm{\sum }}_{j=1}^k{\lambda }_{ij}^2}\right)$$ (6)

where n indicates the number of samples simulated. For each cohort, we generated n = 500 samples based on the loadings and correlation coefficients between factors observed from 4-factor models with the same size of questionnaire (p = 26 for POSTOP and p = 27 for EBRT and EBRT+BT) as that used in our factor analysis. Note that for our proposed method, we performed EFA with a different number of factors from 1 to 6, and determined the number of factors with which the best CFI was obtained in CFA using different loading thresholds from 0.3 to 0.5 in steps of 0.05. We iterated this procedure 100 times and averaged the number of factors determined. Table 4 shows the mean (standard deviation) of the number of factors determined by each method. Note that we generated the simulation datasets from 4-factor models. Therefore, as the mean value is closer to 4, the method is more likely to be accurate. As shown in the table, our proposed method produced very accurate results with 4.1 and 3.9 factors on average for EBRT and EBRT+BT, respectively. However, for POSTOP, both the PA and OC produced the most accurate results with 3.9 factors, whereas the proposed method produced 3.4 factors. This appears to be consistent with the results shown in Table 1 where for POSTOP, CFI was the best when the number of factors was 3. AF produced the worst results with 1 factor for all the simulation datasets.

Table 4.

Mean (standard deviation) of the number of factors determined by each method after 100 simulations.

	Our method	Parallel analysis	Optimal coordinates	Kaiser’s rule	Acceleration factor
POSTOP	3.4(1.1)	3.9(0.2)	3.9(0.4)	4.5(0.5)	1.0(0.0)
EBRT	4.1(0.6)	2.9(0.4)	2.9(0.4)	4.2(0.5)	1.0(0.0)
EBRT+BT	3.9(0.5)	3.0(0.0)	3.0(0.3)	5.3(0.6)	1.0(0.0)

5. Discussion

Our systematic exploratory and confirmatory factor analysis (FA) method to determine the number of clinically relevant factors and factor loading cutoffs demonstrated overall high performance in the three investigated datasets, performed better than four previously proposed methods, and the groups of identified symptoms corresponded well to a clinical symptom categorization. In addition, we demonstrated that latent factor-based outcome variables could be used as future normal tissue morbidity endpoints.

In the field of radiation oncology, extensive questionnaires to assess toxicity after RT from patient-reported outcomes are becoming more readily available to supplement objective toxicity scores by healthcare professionals. Grouping questions in a questionnaire according to a shared underlying mechanism is likely to reflect domain-specific injuries (27). We applied FA to a large number of patient-reported gastrointestinal symptoms to identify clinically meaningful symptom groups. The four identified groups of symptoms included the majority of the clinically relevant symptoms for each domain. Furthermore, these groups of symptoms agreed well with the original clinical symptom categorization. Across the cohorts, the CFI for the POSTOP model was much lower than the CFI for the other two cohorts. Unlike EBRT and EBRT+BT patients, POSTOP patients received RT combined with surgery, making it likely that the latter regimen contributed to decreased robustness across the cohorts. On the other hand, the robustness between EBRT and EBRT+BT models suggests that the identified factors primarily are consequences of the same radiation-induced injuries in the gastrointestinal region.

An important perspective of the FA-identified symptom groups is to investigate their dose-response relationship. To test associations between radiation dose and the symptom groups, we performed linear regression analysis by using a combination of factor loadings of symptoms in a symptom group as a response variable. As a result, nine statistically significant correlations were found between maximum or mean dose to the rectum or anal sphincter region and the identified four symptom groups, implying the plausibility of the proposed idea.

A straightforward comparison of our identified GI domains to those identified in previous studies is difficult for various reasons including less detailed and fewer number of symptoms investigated (5, 6) or symptoms from various domains (6, 7, 9), inconsistency in methods applied (clustering technique (1, 5), factor loading cutoff (1, 6), and method used to determine the meaningful number of clusters (1, 5)), limited follow-up (9), other tumor sites (6–8) and mixed datasets including various tumor sites and regimens (5, 16). Despite these differences, our KMO and Cronbach’s alpha values were considerably higher than those in the studies by Kuku, et al. (5), Farnell, et al. (1), and Kim, et al. (7), respectively, indicating higher suitability of applying FA to our datasets. The latter statement is also in agreement with Skerman, et al. (16) who suggested that FA is the most suitable method to identify symptom clusters in clinical data.

Our EFA-CFA method generated a more relevant number of factors than the four existing methods as judged by a clinical symptom categorization. As investigated by Khan, et al. (8) where three different clustering methods consistently generated a different number of clusters, consensus is not expected since each analytical method clusters symptoms in slightly different ways. Therefore, all clustering methods applied to clinical data should be judged according to their clinical relevance (8, 16). An exploratory structure that achieves the best goodness-of-fit in CFA, as suggested by our method, is likely to give a valuable and clinically relevant description of the observed data.

6. Conclusions

The methodological novelty of the method is that we proposed a novel method to determine the optimal number of factors in FA by integrating EFA and CFA approaches, and also proposed the concept of a factor score that was modeled using dose-volume variables to investigate the dose-response relationship.

Our proposed exploratory and confirmatory factor analysis method, which uses the goodness-of-fit of the exploratory models in CFA, generated clinically more relevant groups of symptoms in various domains of gastrointestinal toxicity after RT for prostate cancer, and showed robustness across the investigated datasets in terms of several aspects: 1) better performance compared with other computational methods, 2) similarity of symptoms in the symptom groups identified across datasets, 3) validation of a cohort’s model with other two datasets, and 4) results in a simulation study. Our results demonstrated the importance of using a logically structured clustering approach followed by a judgment of clinical relevance when applying FA to these types of data. We also addressed an important problem of estimating a more relevant outcome variable for each symptom group. In our analysis, this enabled better modeling of associations between the symptom groups identified and dose-volume variables to various underlying anatomic structures. We believe the methodology has wide applicability to better understand dose-response relationships based on PROs. In future studies, we will apply this method to a questionnaire collected from head and neck cancer patients treated with RT to find relevant symptom groups and dose-response relationships in terms of trismus.