Construct Validity and Invariance of Four Factors Associated with Colorectal Cancer Screening across Gender, Race, and Prior Screening

Abstract

Understanding individuals' perceptions of colorectal cancer screening (CRCS) is important for developing effective interventions to increase adherence to screening guidelines. Theory-based cognitive and psychosocial constructs have been associated with CRCS in the literature, but few studies have evaluated the psychometric properties of such measures. We hypothesized a correlated four-factor model including CRCS perceived pros, cons, social influence, and self-efficacy. We also examined measurement invariance across subgroups based on gender, race (white; African American), and prior CRCS experience (never; overdue for repeat screening). We used baseline (n=1,250) and 2-week (n=1,036) follow-up survey data from participants in a behavioral intervention trial designed to increase CRCS. Only minor modifications were made to the hypothesized model to improve fit, and the final model was confirmed with a random half of the sample, as well as with follow-up data. Results support the hypothesized unidimensional construct measures and suggest that the items may be appropriate for all subgroups examined. Greater variance in responses to items assessing the perceived cons of CRCS was found among African Americans compared with whites, suggesting that race may moderate the association between perceived cons and CRCS in this sample. Pros, cons, social influence, and self-efficacy are associated with CRCS; therefore, using scales with known psychometric properties strengthens researchers' ability to draw conclusions about group differences and changes over time and to compare their results with other studies. Replication studies in other populations are needed to provide further evidence of construct validity for the scales reported here.

Introduction

Colorectal cancer (CRC) is the second leading cause of cancer deaths in the U.S. ¹ Regular CRC screening (CRCS) is recommended beginning at age 50 for average-risk adults. ^{1, 2} Despite the availability of multiple tests for the early detection and prevention of CRC, ³ screening rates are less than optimal. ^{4, 5}

Understanding individuals' perceptions of CRCS is important for developing effective interventions to increase adherence to screening guidelines. While demographic predictors of CRCS may be used to segment populations for delivering interventions, psychosocial constructs are usually chosen as intervention targets because they are more amenable to change. Consistent positive associations have been reported between CRCS and perceived benefits and low barriers, self-efficacy, physician recommendation, and intention. ⁶ Affect and social influence have seldom been studied in relation to CRCS, but observed positive associations suggest that these variables deserve greater attention in future research. ⁶ Many of these constructs are represented in various conceptual models; ^7-10 however, they often are labeled differently (e.g., pros and cons in the Transtheoretical Model are called benefits and barriers in the Health Belief Model). In addition, even when the same construct is used, the items used to measure the construct differ across studies.

Because conceptual and operational definitions of psychosocial constructs differ and because these constructs cannot be directly observed, studies to assess the reliability and validity of scales used to measure them are important. Unreliable or invalid measures may produce null or inconsistent associations with CRCS behavior. Further, measures must be suitable for different subgroups of the population in order to make valid comparisons across groups. Without valid measures of psychosocial constructs, our efforts to develop and evaluate interventions, to assess changes over time, and to compare results across studies are limited.

Despite the importance of using validated measures of psychosocial factors, only three published studies have examined the psychometric properties of scales measuring constructs associated with CRCS. ^11-13 Rawl et al. ¹¹ conducted exploratory factor analyses to confirm the unidimensionality of scales assessing perceived benefits and barriers for specific CRCS tests among people with adenomatous polyps and first-degree relatives of patients with CRC. Using data from non-Hispanic white male automotive workers participating in a worksite behavioral intervention trial to increase CRCS, Vernon et al. ¹³ reported on five scales measuring unidimensional constructs relevant to the intervention: salience and coherence, perceived susceptibility, cancer worries, response efficacy, and self-efficacy. Model fit for the five factors was similar for men with and without a history of polyps. Using Vernon et al.'s original items, Tiro et al. ¹² confirmed the factor structure of five constructs (salience and coherence, perceived susceptibility, cancer worries, response efficacy, and social influence) with data from patients attending an urban primary care clinic, and found the scales to be largely invariant across gender and race (white and African American).

The purpose of this study is to add to the scant literature that has examined the validity of psychosocial measures related to CRCS. Although the survey instrument was not designed to include all psychosocial CRCS determinants, the data collected provided the opportunity to validate four core constructs associated with CRCS: perceived benefits/pros and barriers/cons, social norms/influence, and self-efficacy. Only these four constructs were assessed with multi-item scales; other constructs such as perceived risk and cancer worry were assessed with single-item measures. We examined the unidimensionality of and associations between these four factors in order to provide evidence of factorial validity, which demonstrates that a set of items measures the hypothesized latent variables. Factorial validity is one way to provide support for construct validity; construct validity is important for making inferences from scale scores about the underlying construct of interest. We examined measurement invariance (i.e., equivalence) across subgroups in our sample because social or cultural factors might influence perceptions of CRCS (i.e., males vs. females, whites vs. African Americans) and because prior CRCS test use could alter perceptions of and attitudes toward CRCS (i.e., never vs. lapsed screeners).

METHOD

Setting

This research was conducted as part of an intervention trial to increase CRCS among patients at a multi-specialty clinic in Houston, Texas. The trial was approved by the institutional review board at the University of Texas-Houston School of Public Health (UT).

Participants and Procedures

We used the Kelsey-Seybold Clinic (KSC) administrative database to identify potential participants with the following characteristics: having received primary care through KSC within the last year, were between 50-70 years of age, never had CRC or polyps, had never been screened or were due for CRCS according to American Cancer Society guidelines, and had not had a physical exam within the past year. Additional eligibility criteria included no prior diagnosis of Crohn's disease or ulcerative colitis and ability to speak English.

Between January 2005 and May 2007, invitation letters were mailed with a telephone number if recipients wished to decline participation. KSC research staff telephoned all others to introduce the study, confirm eligibility, and enroll patients. Contact information for enrolled participants was sent to UT research staff who administered baseline and follow-up telephone surveys. Approximate time to complete each survey was 25 and 15 minutes, respectively. Eligible participants who completed the baseline survey were randomized to one of three study groups. Of those in the control group, half were randomly assigned to complete a 2-week follow-up survey.

Measures

The scales evaluated in this study were developed for the intervention trial based on our prior work, reviews of the literature ^{6, 13-15} and focus group findings. ¹⁶ Items assessing pros, cons, social influence, and self-efficacy constructs are listed in the Appendix.

Data analysis

Data were analyzed using confirmatory factor analysis (CFA) in AMOS 7.0 (Chicago, IL). Full-information maximum likelihood estimation was used to include respondents with missing data to test the final model.

Measurement Model

Using half of the baseline sample (n = 543), we built the measurement model sequentially, starting with cons, the construct with the most items. We allowed for improvements and modifications in the form of error covariances and removal of poor-performing items (i.e., low factor loading or cross-loading). After making modifications, other constructs were added successively and correlations between factors were examined. The final correlated four-factor model was then confirmed with the second half of the sample (n = 558), as well as with the full sample at baseline (n = 1,250) and 2-week follow-up (n = 1,036).

Factorial validity is indicated when a set of items correlates strongly with the hypothesized latent constructs. Using CFA, we assessed factorial validity by examining model fit and the significance of factor loadings. A high factor loading indicates that a specific item is a good indicator or measure of the latent factor/construct. We used multiple fit indices to evaluate model fit: the chi-square/degrees of freedom test, comparative fit index (CFI), Root Mean Square Error of Approximation (RMSEA) and its associated 90% confidence interval, and the Akaike Information Criterion (AIC). CFI values between 0.90-0.95 or above suggest adequate to good fit ^{17, 18} and RMSEA values <.06 suggest good model fit. ¹⁷ The AIC adjusts the chi-square statistic to account for model complexity and is recommended for comparisons between non-nested models. Models with the lowest AIC value are preferred. ¹⁹ We did not expect to obtain non-significant chi-square test statistics due to our large sample size.

Invariance Across Groups

We had a sufficient number of males (n=448) and females (n=633), whites (n=388) and African Americans (n=497), and never screened (n=528) and lapsed (n=553) participants for invariance testing across these groups. Invariance testing involves a series of analyses to assess the equivalence of specific model parameters across groups. First, we conducted independent, group-specific CFAs without modification to assess overall model fit within each group (configural invariance). Second, using the chi-square difference test, we examined measurement (equal factor loadings and factor variances) and structural (equal factor covariances and error covariances) invariance across groups. For each pair of comparisons, we began with a fully constrained model in which all parameters were specified as equal across groups. If the fully constrained model was not equivalent, we tested for partial invariance through a series of nested models. Following the process recommended by Byrne, ²⁰ we successively applied more stringent equality constraints in which we first tested the invariance of factor loadings, followed by factor variances, factor covariances, and error covariances. Once a fully or partially invariant model was identified, multiple fit indices were used to assess the overall fit for both groups simultaneously. We did not test for the equality of item intercepts or factor means because we expected substantive differences between groups. ²¹

Results

Participants

Among the baseline respondents, the mean age was 56 (SD = 4.4) years old and most (61%) were married or living with a partner. The majority had some education post high-school (75%).

Measurement Model

In the development phase, we dropped one cons item because the factor loading was <0.30 and we dropped three more cons items when modification indices suggested they should load on the pros factor (Table 1). The hypothesized correlated four-factor model with minor modifications (Figure 1) provided acceptable fit to the data (Table 1). Factor loadings ranged from 0.41 to 0.86 demonstrating strong indicators, but with some room for improvement. To improve overall model fit in the final model, we correlated the error variance of three items, two social influence and one pro. Each of these items referred to close others including friends and family, and the additional correlations improved model fit by capturing the shared variance related to these close others. All factor correlations were in the expected direction, but because two factors (pros and social influence) were highly correlated (r >.80), we also tested two alternative post-hoc models to avoid confirmatory bias. ²² First, we combined them into one factor and found that it was correlated with cons. Second, we created a second-order factor for pros and social influence and found that it was correlated with cons. Neither alternative significantly improved model fit based on AIC and consideration for parsimony, and so we accepted the hypothesized model supported by theory and previous empiric findings (Table 1). Similar fit for the final model was observed with the 2-week follow-up confirmation sample (Table 1).

Table 1.

Model development and validation; PCCaSO, 2005

Model	χ2	df	p	CFI	RMSEA	90% CI	AIC	Notes
Model Development: First random half of sample (n = 543)
All 14 cons	289.06	77	<.001	0.843	0.071	0.063-0.080	345.06	Dropped MDrec loading <.30
13 cons + 8 pros	542.04	188	<.001	0.837	0.059	0.053-0.065	670.04	Dropped 3 cross-loading cons
10 cons + 8 pros	314.73	134	<.001	0.894	0.050	0.043-0.057	424.73
10 cons, 8 pros + 4 norms	621.72	206	<.001	0.832	0.061	0.056-0.067	759.72	High correlation between norms and pros .88, p < .001
10 cons + combined   pros & norms factor	654.33	208	<.001	0.820	0.063	0.058-0.068	788.33	Not a significant improvement
2nd order factor of pros   & norms correlated   with 10 cons	640.79	207	<.001	0.825	0.062	0.057-0.068	776.79	Not a significant improvement
Correlated 4-factor model: cons, pros, norms, self- efficacy	1136.11	399	<.001	0.864	0.58	0.054-0.62	1268.11	Examined modification indices
Added 3 error covariances to correlated 4-factor model	960.10	396	<.001	0.896	0.051	0.047-0.055	1098.10	Final model. Significant improvement; χ2Δ = 176.01 (dfΔ = 3)
Final Model Validation
Second random half of sample (n = 558)	1000.77	396	<.001	0.893	0.052	0.048-0.056	1138.77
Full sample (n = 1,250)	1507.51	396	<.001	0.909	0.047	0.045-0.050	1705.51	See Figure 1
Full sample at 2-week follow-up (n = 1,036)	1014.93	396	<.001	0.917	.039	0.036-0.042	1212.93

CFI = comparative fit index; RMSEA = root mean square error of approximation; CI = confidence interval; AIC = Akaike information criterion

Figure 1.

Final correlated four-factor model using full baseline sample (N=1,250); PCCaSO, 2005

All estimates shown are standardized and statistically significant, p < .001

Invariance by Group

The correlated four-factor model was reanalyzed for each pair of comparison groups separately. Unless otherwise noted, all factor loadings, variances, covariances, and error covariances were significantly different from zero and equivalent across groups. When comparing males and females, three self-efficacy items and one pros item were not equivalent (Table 2). Although all loadings were in a positive direction and significantly different from zero, the self-efficacy items were stronger indicators of the construct for males, (i.e., larger factor loadings) whereas the pros item was a stronger indicator of the construct for females.

Table 2.

Non-invariant factor loadings, variances, and covariances across groups; PCCaSO, 2005

	B	SE	Beta	B	SE	Beta	CMIN	df	p =
	Never Screened (n = 528)			Prior CRCS (n = 553)
SE decision	1.56	0.09	0.80	1.31	0.09	0.71	16.10	7	.024
	Males (n = 448)			Females (n = 633)
SE Embarrass	1.19	0.05	0.90	1.09	0.05	0.82	12.81	5	.025
SE Prep	1.05	0.05	0.82	0.89	0.05	0.70	14.88	4	.005
Pro Early detection	0.74	0.07	0.59	0.94	0.07	0.69	14.74	7	.039
	Whites (n = 388)			African Americans (n = 497)
Con MD embarrass	0.52	0.09	0.40	0.91	0.09	0.64	28.31	8	.001
Con pain	1.66	0.19	0.66	0.89	0.11	0.45	34.59	8	.001
Variance pro	0.20	0.03	--	0.15	0.02	--
Variance con	0.17	0.03	--	0.37	0.06	--
Cons <-> Influence	0.02	0.01	0.11 NS	0.07	0.02	0.38
Pros <-> Cons	<−0.01	0.01	−0.01 NS	0.03	0.01	0.13
Pros <->Influence	0.17	0.02	0.96	0.13	0.02	0.90

B = unstandardized estimate; SE = standard error; Beta = standardized estimate

^NSp > .05

When comparing whites and African Americans, two cons factor loadings, factor variances for pros and cons, and factor covariances between cons and social influence, pros and cons, and pros and social influence were not equivalent (Table 2). Being embarrassed to talk to a doctor about CRCS was a stronger indicator of the cons factor for African Americans, whereas perceived pain during the test was a stronger indicator for whites (Table 2). Variance in the cons factor was greater among African Americans compared with whites, whereas the opposite was true for the variance in the pros factor (Table 2).

When comparing individuals with and without prior CRCS, only one factor loading was not invariant across groups; self-efficacy for making the decision to be screened for CRC was a stronger indicator of the construct for those who had never been screened (Table 2). Cronbach's alpha estimates for each scale are included in the Appendix. For unidimensional scales, alpha >0.70 indicates good internal reliability.

Discussion

In general, the results supported the hypothesized assignment of items to factors. Although we accepted the a priori model of four correlated but conceptually independent constructs, high covariances and error covariance modifications suggest some measurement overlap between pros and social influence. These associations may be due to the fact that the question stem was the same for all items and because all of the social influence items reflect positive influences on CRCS. The general construct of perceived benefits may subsume other positive-consequences constructs. ^{10, 23} For example, the three items from Vernon et al.'s ¹³ screening efficacy scale are reflected in our benefits items 1, 2, and 7 (see appendix). Several cons items were dropped from the model because of poor loading or cross-loading, which may have been due to misinterpretation of the item. Research staff who administered the telephone surveys indicated that several cons items were interpreted by participants as pros. Future use of the excluded cons items will require rewriting the items in simpler, more direct language and conducting cognitive interviews to determine if the new language improves comprehension. The final model was confirmed with follow-up data and the structure (i.e., correlated unidimensional constructs) is supported by theory and empirical research. Specifically, the Health Belief and Transtheoretical models hypothesize independent constructs for pros, cons, and self-efficacy. ^{9, 10} The Preventive Health Model includes social influence as a unidimensional predictor of intentions and behavior, and has been applied to CRCS. ^24-26 Validated measures of these constructs are important for future tests of causal models predicting CRCS behavior. The Transtheoretical model and Fishbein's ²⁷ integrated model allow for the integration of other constructs from the cancer screening literature such as perceived susceptibility and cancer worry. Tests of such integrated theoretical models with validated measures are important for moving the field forward in terms of explaining CRCS and for developing effective interventions to increase CRCS. The choice of survey items and response scales also has important implications for the interpretation of both measurement (i.e., factorial validity) and structural (i.e., factor correlations) models. For example, the positive association between pros and cons in this study is similar to other studies using response scales that range from ‘very important’ to ‘not at all important’ ^{28, 29} whereas negative associations are found with ‘strongly agree’ to ‘strongly disagree’ response scales. ^{23, 30, 31}

Comparing the underlying latent constructs across groups suggests that they are generally similar for males and females, whites and African Americans, and never and lapsed screeners. When non-invariance was observed, all factor loadings were significantly different from zero and in the same direction indicating that differences were only a matter of degree. Model fit was not substantively reduced when these loadings were constrained to be equal, which suggests that we had sufficient power to detect very small differences. Because complete measurement invariance is generally unobtainable, researchers must evaluate the amount and acceptability of potential bias when small group differences are detected. ³² We conclude that our scales are appropriate for all subgroups examined here and for future comparisons of mean group differences. However, we recommend that investigators using these scales examine latent group mean differences with and without the invariant items to determine whether their inclusion produces substantive differences in the results and conclusions. Between African Americans and whites, differences in factor variances and covariances may be due to the fact that African Americans' responses to cons items were more widely distributed across the response scale compared with whites. Because greater variance in the cons factor was observed for African Americans, future analyses using this data to examine the association between cons and other variables may need to include race as a potential moderator. If the association between cons and CRCS intention differed by race, it may be due to the measurement differences, the true association between constructs, or both. Future studies need to examine the equivalence of cons measures across race/ethnic groups to determine whether the differences reported here are sample-specific or whether they reflect true social or cultural differences in how individuals perceive the cons of CRCS.

A limitation of our study is our volunteer sample of patient participants, which may decrease generalizability of our findings to other populations, a limitation shared by most health promotion studies of cancer screening. Future studies will have to determine the scales' suitability for other populations (i.e., medically-underserved, other race/ethnic minorities) and modes of survey administration (i.e., mail). Although some cons items referenced a test-specific barrier (i.e., doing the stool blood test might be disgusting), they loaded on a general cons latent factor. A scale of CRCS test-specific cons, such as the one developed by Rawl et al.¹¹, may be preferred in future interventions that focus on a particular recommended test (i.e., colonoscopy) rather than a choice among multiple CRCS alternatives. Although the pattern of inter-item covariances suggests a unidimensional factor, which is a necessary condition for alpha to be an unbiased estimator of reliability, the small number of items and the size of the average inter-item covariances may produce less-than-desirable reliability estimates for the social influence scale.³³ Previous studies also have reported poor reliability estimates for social influence scales. ^{12, 13} Although additional research is needed for scale improvement, future use of these theory-based measures of social influence as predictors of CRCS will improve comparisons of results across studies.

Pros, cons, self-efficacy, and social influence are important constructs associated with CRCS. ⁶ Researchers depend on reliable and valid measures of these psychological constructs, which are not directly observable. Using scales with known psychometric properties strengthens researchers' ability to draw conclusions and compare their results to other studies. Using equivalent measures across groups is necessary for determining whether observed between-group mean differences are the result of true attitudinal differences regarding the underlying constructs. ³⁴ Our measures can be used in future studies along with validated measures of other constructs shown to be important for CRCS. Our measures appear to be appropriate for average-risk clinic samples of white and African American males and females aged 50 and older with and without prior CRCS; however, replication studies are needed to strengthen the evidence for construct validity provided here.

Appendix

Scales from final model and Cronbach's alpha from full baseline sample (n = 1,250); PCCaSO, 2005

I will read you a list of reasons and I'd like you to tell me how important each of them is in your decision of whether or not to get colon cancer testing. Response options: (4) very important – important – somewhat important – (1) not important
Pros	1	Believing finding cancer early gives you a better chance for a cure
α = .75	2	Having peace of mind after receiving clear findings
	3	Believing that colon cancer testing is safe with few serious complications
	4	Believing that those close to you will be relieved if you get tested for colon cancer
	5	Believing that getting tested is part of taking care of yourself
	6	Believing that colon cancer testing can find cancer early
	7	Understanding that when polyps are found and removed, cancer can be prevented
	8	Realizing that the benefits of having colon cancer testing outweigh any difficulty that you might have doing the test
Cons	1	Being worried or concerned that if you do the test you might find something wrong
α = .78	2	Being too embarrassed
	3	Believing that colon cancer testing takes too much time
	4	Thinking that doing the stool blood test might be disgusting
	5	Expecting pain during a screening test
	6	Believing that following a special diet or emptying your colon is too much trouble (prep)
	7	Believing that the lack of symptoms means that you don't need colon cancer testing (no sx)
	8	Having to arrange transportation to and from a colon cancer screening test (no ride)
	9	Concern about the cost of the test or lack of insurance coverage
	10	Being too embarrassed to talk to your doctor about colon cancer testing (MD emb)
Social influence	1	Believing that members of your immediate family think you should get tested for colon cancer
α = .65	2	Believing that most people your age are being told they should get tested for colon cancer (age grp)
	3	Believing that your friends think you should get tested for colon cancer
	4	Believing that your doctor thinks you should get tested for colon cancer
This set of questions asks about how confident you are about certain aspects of colon cancer testing. Response options: (4) very confident – confident – not very confident – (1) not at all confident
Self- Efficacy	1	How confident are you that you can make a decision about whether or not to get colon cancer testing?
α = .91	2	How confident are you that you can complete colon cancer testing?
	3	How confident are you that you can complete colon cancer testing even if you are nervous about it?
	4	How confident are you that you can complete colon cancer testing even if you are embarrassed about it?
	5	How confident are you that you can find the time to complete colon cancer testing?
	6	How confident are you that you can talk to your doctor about colon cancer testing?
	7	How confident are you that you can carry out any necessary preparation for colon cancer testing?
	8	How confident are you that you can get support from family or friends to help you complete colon cancer testing?
Items dropped from the final model; cons items misinterpreted by participants
	1	Needing/Having a recommendation from your doctor (MDrec)
	2	Feeling uncertain about what will be done during a colon cancer screening test
	3	Realizing that you may need to adjust your schedule to do colon cancer testing
	4	Believing that it would be upsetting to your family and friends if you had colon cancer