A psychometric evaluation of the Indonesian version of the collaborative practice assessment tool (CPAT) for assessing interprofessional education and collaborative practice among health practitioners and students

Abstract

Research focus has transitioned from interprofessional collaborative practice among qualified health practitioners to the involvement of pre-qualifying students in practicing interprofessional education. It is essential to establish outcome measures to enhance the seamless integration of interprofessional education and collaborative practice. The aim of this study was to develop a culturally appropriate quality measure for assessing interprofessional education and collaborative practice for health practitioners and students in Indonesia by performing cross-cultural validation of the collaborative practice assessment tool (CPAT). The consensus-based standards for the selection of health measurement instruments (COSMIN) standards of psychometric properties were used to guide the study. The evaluation of the psychometric properties was conducted, involving meticulous structural validity evaluation based on a three-step factorial analysis (exploratory factor analysis, confirmatory factor analysis, and multi-group confirmatory factor analysis) and measurement invariance. The parameters analyzed were related to the design requirements of a measure (i.e., targeted population, study sample, and size), the internal structure (structural validity, internal consistency, and measurement invariances), and hypotheses testing for construct validity based on a validated conceptual framework. This study involved 266 practitioners and 232 students. The COSMIN standards for general design requirements were fulfilled. Structural validity confirmed the 7-factor of 48-item structure; measurement invariances indicated configural, metric, and scalar invariants in both practitioner and student cohorts. Construct validity was confirmed by meeting the COSMIN requirement, with over 75% of the tested hypotheses accepted. In conclusion, the findings suggest the newly validated Indonesian CPAT has good psychometric properties concerning internal structure (i.e., structural validity, internal consistency, and measurement invariance) and hypotheses testing, and is therefore a quality measure for assessing interprofessional education and collaborative practice with health practitioners and students in Indonesia.

Introduction

Emerging evidence indicates that interprofessional education leads to interprofessional collaborative practice, strengthens health systems, and improves health outcomes [1-4]. Accordingly, the world health organization (WHO) strongly recommends interprofessional education (IPE) and interprofessional collaborative practice (IPCP) as a strategic measure to overcome the global health workforce shortage and strengthen the healthcare system [2]. When interprofessional education is introduced early in health professional education, it facilitates the development of the collaborative competencies needed for future practice [4,5]. However, disparity exists because many educational institutions currently provide only uniprofessional learning experiences with students from different professional backgrounds trained separately [6,7]. This uniprofessional education nurtures fragmented provision of healthcare that limits the scope and integration of services, leading to an increase in the overall cost and duration but a decrease in the quality of healthcare [2].

As research has shifted its focus from interprofessional education for qualified health practitioners to include pre-qualifying students [4,5,8], outcome measures developed for both practitioners and students are needed to support greater integration between interprofessional education and interprofessional collaborative practice (i.e., greater integration between health professional education and healthcare). Indonesia recognizes the urgent need to embed interprofessional education within health education programs [9-11]. Unfortunately, a limited number of instruments are available to assess interprofessional education and collaborative practice outcomes at both the education level (students) and in the workplace (practitioners). Moreover, those that are available were all developed in English [12].

One outcome measure, the collaborative practice assessment tool (CPAT), has been validated in Bahasa Indonesia for practitioners [13]. The other available Indonesian measures, the readiness for interprofessional learning scale (RIPLS) and Chiba interprofessional competency scale (CICS29), were validated for students [14,15]. To the best of the authors’ knowledge, equivalent interprofessional outcome measures for use by practitioners and students are limited, with none available for use in the Indonesian context.

Consensus-based standards for the selection of health measurement instruments (COSMIN) require instrument validation with the intended user's population and relevant settings, thus making it crucial to assess the psychometric properties of the instruments in each targeted population and setting [16-19]. In other words, measures validated for health practitioners are not recommended for use by health students, and vice versa. Furthermore, following cross- cultural translation, evaluation is required to assess whether the measure's translated items adequately reflect the performance of the items in the original version, specifically concerning the content validity and internal structure of the translated measure [16,18,19]. Therefore, the current study adds one significant psychometric evaluation to the CPAT: to carry out measurement invariance tests as part of the structural validity evaluation to enable CPAT Indonesia to be used equivalently for health practitioners and students in Indonesia. Having an invariant measure for practitioners and students makes the scores related to relevant outcomes comparable [20,21]. This makes it possible to identify the development and improvement of students’ interprofessional identities and compare them with the interprofessional characteristics of practitioners in the workplace.

Methods

The study comprises three phases: (A) cross-cultural validation of the CPAT; (B) evaluation of the internal structure of the measure (i.e., structural validity, internal consistency, and measurement invariances); and (C) hypotheses testing for construct validity using a validated conceptual framework for interprofessional collaboration. The COSMIN taxonomy and standards of psychometric properties were used to guide the study procedures [16,18,19].

Procedures

Before the validation process, the necessary permission to use the instrument was obtained from the corresponding authors of the original English instrument [22] and the previously validated Indonesian CPAT [13], henceforth referred to as the previous Indonesian CPAT. A total of 53 items in the previous Indonesian CPAT were validated only (i.e., not re-piloted) in the current study. The data obtained were then used to analyze the instrument's psychometric properties in terms of the general design requirements, internal structure for structural validity, and hypotheses testing for construct validity. The CPAT version developed in this study will be referred to as the newly validated Indonesian CPAT. An overview of the psychometric properties evaluated in this study is presented in Figure 1.

Figure 1.

Study procedures for evaluating psychometric properties.

Participants

This research was conducted in Indonesia. Participants were purposively sampled, targeting Indonesian health practitioners and students from various health backgrounds to ensure the representation of the intended users [18]. Participants must have met the following inclusion criteria: (A) Indonesian health practitioners and students from any health professional/educational background; and (B) experience in health-team collaboration with another practitioner or student with a different professional/educational background from their own. Participation in the study was voluntary, and all responses were anonymous. Potential participants were sent an information sheet providing details of the study, including the consent process and an invitation to complete the survey online using Qualtrics [23], generated in Australia.

Collaborative practice assessment tool (CPAT)

The CPAT is a self-assessment measure developed in English to measure interprofessional collaborative practice among health practitioners. The original version includes 56 items, eight subscales, and three open-ended questions [22]. Henceforth, this CPAT version will be referred to as the original CPAT. The Indonesian version of the CPAT consists of 53 items (three items were removed based on exploratory factor analysis), eight subscales, and three open-ended questions [22]. Both versions of the CPAT were validated for use with healthcare practitioners.

The original instrument subscales are: (1) mission, meaningful purpose, and goals (Cronbach's α= 0.88; 8 items); (2) general relationships (Cronbach's α= 0.89; 8 items); (3) team leadership (Cronbach's α= 0.80; 8 items); (4) general roles responsibilities, autonomy (Cronbach's α= 0.81; 10 items); (5) communication and information exchange (Cronbach's α= 0.84; 6 items); (6) community linkages and coordination of care (Cronbach's α= 0.76; 4 items); (7) decision-making and conflict management (Cronbach's 87 α= 0.67; 6 items); and (8) patient involvement (Cronbach's α= 0.87; 5 items). Six of the items needed reverse coding. Responses were based on a 7-point Likert scale (1 = strongly disagree; 2 = mostly disagree; 3 = somewhat disagree; 4 = neither agree nor disagree; 5 = somewhat agree; 6 = mostly agree; and 7 = strongly agree).

The CPAT was chosen in this study because of its comprehensiveness in covering the most important aspects of interprofessional collaboration [22,24], such as team collaboration and communication, leadership, role clarity and understanding, team conflict management, and patient involvement in their care. The CPAT is widely used to measure team performance, recommended as the best instrument to assess interprofessional teamwork [25] and has been translated into several languages and used in many countries, including Japan, Taiwan, Singapore, the USA and Canada [26-32], as well as in Indonesia [22]. All studies validated the tool with health practitioners.

Structural validity

Structural validity reflects the extent to which conclusions drawn from observations on measurements (in terms of scores) adequately represent the dimensionality of the measured construct [16]. Structural validity evaluation in this study was a three-step process of factor analysis, starting with exploratory factor analysis (EFA), followed by confirmatory factor analysis (CFA) and multi-group confirmatory factor analysis (MG-CFA). EFA was conducted to determine if the 8-factor 53-item model for the newly validated Indonesian CPAT should be retained to reflect the findings of the original version and previous Indonesian CPAT. An initial structural model with the most suitable indices was then constructed to conduct the CFA. The CFA started by independently testing the initial model (EFA results) to the practitioner dataset, which was then verified to the student dataset; the practitioner dataset was used as a calibrator. The factor structures were set as equal across studies to enable the rating of the quality of the summary score [16]. Missing data were treated with listwise deletion, so the analysis was run only on observations with complete datasets. Factorial analyses were conducted using IBM SPSS 26 and AMOS 26 [33].

While confirming the best-fit model for the two datasets, item deletion was treated cautiously by removing one item at a time and combining the error term covariance, which was created based on the modification index (MI) > 20 [34]. There is no definitive cut-off for the application of model fit indices; for this study, the COSMIN good fit indices criteria were used (i.e., comparative fit index (CFI) > 0.95; or root mean square error of approximation (RMSEA) < 0.06; or standard root mean square of the residual (SRMR) < 0.08) [16].

Internal consistency reliability

Internal consistency refers to the interrelationship of the observed construct and how well these constructs are correlated in measuring the same general concept [16,18,19]. The common measurements of correlation reliability were represented with a Cronbach's α score and were calculated for each subscale to confirm its unidimensionality. To complement Cronbach's α score, the composite reliability (CR) for each factor was calculated for CFA. Composite reliability calculations are based on standard loadings and error variances; thus, unequal factor loadings of items associated with the analysis are weighted and accounted for [35]. A Cronbach's α score of 0.70 is acceptable, with a score >0.80 considered high. A score >0.95 is undesirable, as a very high score may suggest item redundancy rather than homogeneity [5,16]. The average variances extracted (AVE) for each factor were also calculated [36].

Measurement invariance

Next, the generated CPAT model was tested for configural, metric, and scalar invariances to assess the equivalence of the model when used for cross-group testing [37,38]. The application of restraints to the model is expected to cause a decrease in the fit indices. Thus, a reduction in the value of the CFI can be expected, but this decrease should not be more than 0.01 to confirm invariances [37]. The higher the CFI, the better the data fits the model. As previously described, the COSMIN criterion for a good fit index was used as the cut-off. The targeted chi-square was between 1 and 3; values less than 5 are acceptable [37].

Configural invariance tests a model by comparing the structure of tested groups based on the number of latent and observable variables that were estimated freely (i.e., testing the model without constraints). A good model fit indicates that the data passes configural invariance across groups and serves as confirmation to continue testing metric invariance [37]. Metric invariance analyses were performed with the observable variables’ factor loadings constrained to be equal across groups. Scalar invariance imposes the same constraints as metric invariance, but with the additional constraint that the item thresholds (τ) are equated across groups [38]. Metric and scalar invariance is achieved when the difference in CFI (ΔCFI) is less than 0.01, and the fit indices difference between the two models is not significantly reduced or remains the same across groups despite the imposing constraints [37].

Hypotheses testing for construct validity

The CFA is a confirmatory, theory-driven test [24,39]. Therefore, an analytical planning model estimates the population covariance matrix based on a tested theoretical, conceptual framework for interprofessional collaboration. The model suggested by Stutsky and Spence Laschinger [24] was used as the basic causal model to compare the estimated and the observed matrix in the participants’ responses. This conceptual framework for interprofessional collaboration covers relevant aspects of interprofessional practice and is conceptualized based on a validated process. The model organized constructs as antecedent factors, mediators, and consequences. Based on this model, an assumption path was designed to evaluate construct validity; the “leadership and communication” domain comprised the antecedent factor, the “shared goals and roles understanding” domain was a mediator, and the domains of “members’ relationship”, “barriers to team collaboration”, “conflict management and decision-making”, “patient involvement”, and “community empowerment” were consequence factors. The hypotheses proposed for each dataset were grouped into mediated assumptions (Figure 2) and direct assumptions (Figure 3).

Figure 2.

Mediated assumptions. HDirect: direct hypothesis.

Figure 3.

Direct assumptions. Hdir: direct hypothesis.

The mediated hypothesis consists of five sub-hypotheses for both practitioner and student cohorts. (1) “shared goals and roles understanding” mediates the positive effects of “leadership and communication on member relationships” (HMed.1); (2) “shared goals and roles understanding” mediates the positive effects of “leadership and communication on barriers to team collaboration” (HMed.2); (3) “shared goals and roles understanding” mediates the positive effects of “leadership and communication on conflict management and decision-making” (HMed.3); (4) “shared goals and roles understanding” mediates the positive effects of “leadership and communication on patient involvement” (HMed.4); and (5) “shared goals and roles understanding” mediates the positive effects of “leadership and communication on community empowerment” (HMed.5). The mediated assumptions are illustrated in the path analysis depicted in Figure 3.

Direct assumptions include hypotheses related to: (1) “leadership and communication have positive effects on shared goals and roles understanding” (HDir.1); (2) “shared goals and roles understanding” has positive effects on “member relationships” (HDir.2); (3) “shared goals and roles understanding” has positive effects on “barriers to team collaboration” (HDir.3); (4) “shared goals and roles understanding” has positive effects on “conflict management and decision-making” (HDir.4); (5) “shared goals and roles understanding” has positive effects on “patient involvement” (HDir.5); and (6) “shared goals and roles understanding” have positive effects on “community empowerment” (HDir.6). The direct assumptions are illustrated in the path analysis depicted in Figure 4.

Figure 4.

Construct models of the newly validated Indonesian collaborative practice assessment tool. (A) The initial construct for CFA (7-Factor 53-Item model); and (B) the final model for multi- group CFA (7-Factor 48-Item model).

In addition, assumption analysis was also used to determine the function of shared goals and roles understanding as a mediator. Partial mediation is indicated when the direct paths (non- mediated relationships) and indirect paths (mediated relationships) are significant for the assumptions being tested; shared goals and roles understanding is indicative of full mediation if only the indirect path (mediated relationships) is significant. The direct causal effects were calculated using AMOS 26, while the mediated assumptions were calculated separately using a user-defined estimand, a custom function for AMOS analysis that provides additional calculations and outputs to existing applications [40]. The estimand allowed the isolation of two or more parameters of interest and created an indirect causal relationship between them.

Results

The results of this study are presented in Figure 1, following the procedural flow, which includes reporting findings regarding the instrument's general design requirements, the internal structure (structural validity, internal consistency, and measurement invariances of configural, metric, and scalar), and hypotheses testing on the direct and indirect assumptions.

Sample description

The participants included 266 individuals, with 188 (70.7%) being women aged between 21 and 60 years (mean = 36.1; standard deviation = 8.2). The top three participating professions were doctors, nurses, and physiotherapists. The student sample included 232 participants, with 174 (75%) females aged between 16 and 35 years (mean = 22.5; standard deviation = 3.7). The three most represented subjects were nursing, medicine, and dietetics. A detailed breakdown of participant characteristics is presented in Table 1.

Table 1. Participants characteristics.

Practitioners (n = 266)		Students (n = 232)
Characteristics	n (%)	Characteristics	n (%)
Sex		Sex
Male	78 (29.3)	Male	58 (25)
Female	188 (70.7)	Female	174 (75)
Age (years)		Age (years)
21–30	69 (2.3)	16–20	55 (26.7)
31–40	123 (46.2)	21–25	127 (61.7)
41–50	62 (23.3)	26–30	13 (6.3)
51–60	12 (4.5)	31–35	11 (5.3)
Professional backgrounds		Study course
Dentist	9 (3.4)	Dentistry	16 (7.8)
Nurse	39 (14.7)	Dietetics	30 (14.6)
Pharmacist	14 (5.3)	Health promotion	25 (12.1)
Physician	172 (64.7)	Medicine	50 (24.3)
Physiotherapist	19 (7.1)	Nursing	67 (32.5)
Public health expert	7 (2.6)	Pharmacy	18 (8.7)
Radiographer	6 (2.2)	Length of study (years)
Length of work (years)		3–4	151 (73.3)
1–10	157 (59)	5–6	39 (18.9)
11–20	89 (33.5)	7–8	16 (7.8)
21–30	20 (7.5)

Structural validity

The sample size for both datasets was within the 5 to 1 ratio of respondents to the number of tested items [18,19]. The suitability of the two datasets for factor analysis was confirmed with Kaiser-Meyer-Olkin (KMO) indexes of 0.92 (practitioner) and 0.91 (student), and Bartlett's test of sphericity for both data sets indicated values of p < 0.001. The structure of the two 8-factor models was used as the basis for EFA modeling for this study, with items based on the original CPAT and the previous Indonesian CPAT version. The EFA was rigorously used to optimize the model for regrouping items on relevant latent factors and verified by Cronbach's α for the total score and each subscale until a suitable model was obtained.

The EFA results indicated that the original CPAT version of the factorial structure was not the best model for the current study. The previous Indonesian CPAT version of the instrument was more suitable, although not entirely defensible. Three subscales in the original CPAT version were entirely preserved in the previous Indonesian CPAT. Together with the contributory items, these three subscales (F1, F2, and F6 in the original CPAT version were retained as F1, F2, and F5 in the previous Indonesian CPAT) were decisive factors in both versions and confirmed by good Cronbach's α scores (see Underlying data). One subscale was weak in both versions: F7 in the original CPAT version (Cronbach's α=0.67) and F6 in the previous Indonesian CPAT version (Cronbach's α=0.70). One new subscale was registered in the previous Indonesian CPAT, which was not a derivative of any of the subscales in the original CPAT version (F8, Cronbach's α=0.61). Information related to the factorial structure of the instrument with the internal consistency score of each subscale and the change in item positioning in both versions is presented in the Underlying data. A final EFA model with good internal consistency for each subscale in both datasets was generated with 53 items specifying a 7-factor conformation and was used as the initial factorial structure for CFA (Table 2).

Table 2.

Comparison of the factorial structure between the previously validated Indonesian collaborative practice assessment tool (CPAT) and the original English CPAT

Subscales	Domains	Remarks
F1	Members relationships (9 items)	All nine items from Ind.F2 relationships among team members (derived from Ori.F2 general relationships).
F2	Antecedent factor: Leadership and communication (14 items)	All five items from Ind.F3 leadership (derived from Ori.F3 leadership). Nine items from Ind.F4 team coordination and organization (derived from Ori.F5 communication and information exchange).
F3	Interprofessional collaborative practice: Shared goals & roles understanding (14 items)	All nine items from Ind.F1 mission, goals, and objectives derived from Ori.F5 (derived from Ori.F1 mission, meaningful, purpose, goals). Five items from Ind.F4 (derived from Ori.F4 general roles responsibilities, autonomy).
F4	Community empowerment (4 items)	All four items from Ori.F5 Team Relationship with the Community (derived from Ori.F6 community linkages and coordination of care)
F5	Conflict management and decision- making (4 items)	All two items from Ind.F6 decision-making and conflict management (derived from Ori.F7 decision-making and conflict management) Two items from Ind.F7 (derived from Ori.F4 general roles responsibilities, autonomy).
F6	Patient involvement (3 items)	All three items from Ind.F7 patient involvement, responsibility, and autonomy (derived from Ori.F8 patient involvement).
F7	Barriers to team collaboration (5 items)	All five items from Ind.F8 barriers to team collaboration (derived from three different Ori factors).

Ind: the previous Indonesian CPAT; Ori: Original CPAT

Because this study used an assumption test based on a validated conceptual framework to synchronize with subsequent analyses, the subscale names were modified to match the labels used in the reference conceptual framework while retaining the main elements of each factor's name from the previous two versions. In the 7-factor 53-item model, five subscales (F1, F4, F5, F6, and F7) and their respective items almost entirely maintained the previous Indonesian CPAT constructs. Factor 2, antecedent factors of “leadership and communication”, was generated by combining the two complete latent factors of “team leadership” (Ori.F3) and “communication and information exchange” (Ori.F5) in the original CPAT version. Factor 3, “interprofessional collaborative practice: shared goals and roles understanding”, was also a merged version of two complete factors in the original CPAT version: “mission, meaningful, purpose, goals” (Ori.F1), and “general roles responsibilities, autonomy” (Ori.F4). EFA results for dimension reduction confirmed the pair's unidimensionality; when run independently and as MG-CFA for each dataset (trial testings), “team leadership” (Ori.F3) and “communication and information exchange” (Ori.F5) were highly linearly dependent (CFA confirmed bivariate correlation >0.85). This multicollinearity issue impacted the results, with AMOS declaring the model inadmissible in further trial testing.

A similar case was present for “mission, meaningful, purpose, goals” (Ori.F1) and “general roles responsibilities, and autonomy” (Ori.F4). These multicollinearity issues were solved by merging the two highly correlated factors [35], resulting in an admissible model. The merging of two sets of highly correlated factors was also in accordance with the conceptual framework used as a reference [24]. The factors of “team leadership” (Ori.F3) and “communication and information exchange” (Ori.F5) were recognized and validated as antecedent factors for “interprofessional collaborative practice” and the factors of “mission, meaningful, purpose, goals” (Ori.F1) and “general roles responsibilities, and autonomy” (Ori.F4) were contributors to “interprofessional collaborative practice”. During this process, the EFA was run repeatedly to confirm each subscale's unidimensionality and each subscale's internal consistency was verified with Cronbach's α scores. The final EFA factorial structure (i.e., the initial CFA model) is presented in Figure 4A.

Initial CFA modeling results demonstrated a need for model refinement in both datasets with CFI or comparable measure <0.95, RMSEA > 0.06, and SRMR barely within the acceptable range of <0.08. Since the model fit indices will inevitably deteriorate when tested for invariance measurements, refinement was performed to improve the fit indices. Smaller factor loadings were indicated for items 14, 34, and 53 in the practitioner dataset (the estimates were less than 0.30), whereas moderate loadings were observed in the student dataset (standardized estimates ranged from 0.47 to 0.55). Removing item 14 increased the average variance extracted (AVE) of subscale “barriers to team collaboration” in both sets to over 0.50. Removing items 34 and 53 significantly increased the CR and/or the AVE of subscale “conflict management and decision-making” in both datasets (practitioner: CR = 0.64 to CR = 0.71; student: AVE = 0.39 increased to AVE = 0.53).

After removing items 14, 34, and 53, CFA was rerun to assess the acceptability of the 7-factor 50-item model to both datasets, with the practitioner dataset tested first. The CFA results indicated that the model had one negative variance. In addition, 12 covariances had MI > 20 (ranging from 20.25–91.07). There were four covariances between error terms in different constructs, eight in a similar construct, three of which correlated with item 28 (with potential par changes of 0.10, 0.12, and 0.14).

Based on these results, trial tests were conducted to determine whether applying all or some of the five covariances between error terms within the same construct, combined with (or without) deleting item 28, would significantly increase the model fit indices. The most significant model fit improvement was obtained by removing item 28 and generating covariances between five correlated error terms in a similar construct. As a result, RMSEA decreased by 0.1 points to 0.070, and SRMR decreased by 0.141 points to 0.066. Despite CFI remaining below 0.950 (CFI = 0.824), these improvements in RMSEA and SRMR were deemed significant for preparing the practitioner dataset for measurement invariance tests.

Assuming an adequate model fit has been generated for the practitioner dataset, the same 7-factor 49-item model (items 14, 34, 28, and 53 removed) was applied to the student dataset. CFA resulted in fit indices of CFI = 0.742, RMSEA = 0.080, and SRMR = 0.072. The modification indices showed ten covariances had MI greater than 20 (ranging from 20.77 to 40.85). One of the covariances involved error terms in different constructs, and the remaining nine involved error terms in the same construct — three of which involved item 39 (with potential par changes of 0.15, 0.19, and 0.23). Repeating the same procedure previously performed on the practitioner dataset, the model fit was corrected significantly by removing item 28 and generating covariances between six correlated error terms within a similar construct. As a result, RMSEA decreased by 0.1 points to 0.069, and SRMR decreased by 0.04 points to 0.068. CFI in the student dataset was inadequate (CFI = 0.787); therefore, this improvement in the RMSEA index was deemed essential to increase the suitability of the student dataset for measurement invariance tests. CFA was run on the final 7-factor 48-item model (items 13, 28, 34, 39, and 53 removed; Figure 4B) in the student dataset provided indices of SRMR = 0.068, RMSEA = 0.069, CFI = 0.787, χ2(1048) = 2196.09, and Chi-square minimum discrepancy function (CMIN)/degree of freedom (df) = 2.05, fulfilling the COSMIN criteria for a good model fit [16].

To assess the acceptability of the practitioner data to the 7-factor 48-item model, CFA was rerun on the practitioner dataset and generated a good model fit with SRMR = 0.065, RMSEA = 0.070, CFI = 0.829, χ2(1048) = 2397.02, CMIN/df = 2.29, fulfilling the COSMIN criteria for a good model fit [16]. These good fit indices provided statistical support for performing an MG-CFA. Using the final CFA model (Figure 4B), an MG-CFA was performed to confirm the model's fit across the two groups. The model fit was good with SRMR = 0.065, RMSEA = 0.049, CFI = 0.812, χ2(2096) = 4593.14, CMIN/df = 2.19, fulfilling the COSMIN criteria for a good model fit [16].

Internal consistency reliability

Comprehensive results of item estimates and each subscale CR and AVE are presented in Table 3. The MG-CFA showed good internal consistency reliability for all subscales, with composite reliability scores all ≥0.70.

Table 3. Item estimates, internal consistency reliabilities, and factorial validity.

Subscales	Items#	Practitioners			Students
		Estimates	CR	AVE	Estimates	CR	AVE
Members relationships (9 items)	CPAT1	0.65	0.86	0.42	0.60	0.86	0.41
	CPAT2	0.71			0.74
	CPAT3	0.54			0.60
	CPAT4	0.82			0.73
	CPAT5	0.75			0.66
	CPAT6	0.58			0.61
	CPAT7	0.49			0.62
	CPAT8	0.58			0.59
	CPAT9	0.63			0.57
Barriers to team collaboration (4 items)	CPAT10	0.65	0.86	0.61	0.72	0.86	0.61
	CPAT11	0.90			0.80
	CPAT12	0.82			0.87
	CPAT13	0.73			0.72
	CPAT14	Deleted			Deleted
Community empowerment (4 items)	CPAT15	0.84	0.88	0.66	0.71	0.82	0.54
	CPAT16	0.90			0.89
	CPAT17	0.79			0.59
	CPAT18	0.70			0.73
Antecedent factors: leadership & communication (13 items)	CPAT19	0.56	0.92	0.48	0.50	0.86	0.33
	CPAT20	0.64			0.57
	CPAT21	0.71			0.51
	CPAT22	0.72			0.52
	CPAT23	0.70			0.63
	CPAT26	0.68			0.51
	CPAT27	0.68			0.63
	CPAT30	0.79			0.61
	CPAT32	0.66			0.37
	CPAT35	0.68			0.70
	CPAT36	0.79			0.62
	CPAT37	0.76			0.72
	CPAT38	0.56			0.51
	CPAT39	Deleted			Deleted
Conflict management and decision-making (2 items)	CPAT33	0.49	0.71	0.58	0.63	0.70	0.54
	CPAT34	Deleted			Deleted
	CPAT52	0.96			0.83
	CPAT53	Deleted			Deleted
Interprofessional collaborative practice: shared goals & roles understanding (13 items)	CPAT24	0.68	0.93	0.50	0.63	0.89	0.38
	CPAT25	0.72			0.62
	CPAT28	Deleted			Deleted
	CPAT29	0.73			0.62
	CPAT31	0.71			0.61
	CPAT40	0.80			0.59
	CPAT41	0.80			0.63
	CPAT42	0.59			0.56
	CPAT43	0.73			0.67
	CPAT44	0.56			0.46
	CPAT45	0.76			0.62
	CPAT46	0.71			0.65
	CPAT47	0.58			0.58
	CPAT48	0.74			0.71
Patient Involvement (3 items)	CPAT49	0.64	0.71	0.45	0.58	0.72	0.47
	CPAT50	0.69			0.66
	CPAT51	0.69			0.80

AVE: average variance extracted; CPAT: collaborative practice assessment tool; CR: composite reliability

Measurement invariance

Using the same model for MG-CFA (Figure 4B), measurement invariances were tested for the two groups. Because the data were analyzed simultaneously, the resulting fit indices referred to the group data rather than individual datasets and were thus presented accordingly. The configural invariance testing showed a good fit with RMSEA = 0.049, SRMR = 0.065, CFI = 0.812, χ2(2096) = 4593.14 (CMIN/df = 2.19), and PClose = 0.80, fulfilling the COSMIN criteria for a good model fit. These results indicated that the unconstrained model achieved a good fit for the factor structure for each group. Configural invariance was achieved, thus passing the requirement for using metric invariance testing.

The metric invariance demonstrated a good fit, with RMSEA = 0.049, SRMR = 0.071, CFI = 0.807, and χ2(2137) = 4699.87 (CMIN/df = 2.20), PClose = 0.76, fulfilling the COSMIN criteria for a good model fit. The CFI difference between the configural and metric models was less than 0.01 (ΔCFI = 0.005). This ΔCFI confirmed that metric invariance was achieved, indicating that the items tested do not differ across the tested groups regarding the structural modeling of 7 factors with 48 items. The results supported the use of a scalar invariance test.

The scalar invariance test showed a good fit with RMSEA = 0.049, SRMR = 0.083, CFI = 0.803, χ2(2165) = 4775.92 (CMIN/df = 2.21), and PClose = 0.072, thus fulfilling the COSMIN criteria for a good model fit. The CFI difference was less than 0.01 (ΔCFI = 0.004), indicating no significant differences in the item factor loadings and intercepts between the practitioner and student datasets, resulting in scalar invariance. As predicted, imposing constraints on factor loading and intercepts can cause a decrease in the fit indices, as shown by SRMR decreasing to 0.083 (cut off <0.8). The models’ fit indices for both tested groups for model comparison are presented in Table 4.

Table 4.

Full model comparison

Models	CMIN (df)	CMIN/df	CFI	ΔCFI	SRMR	RMSEA	PClose	Invariance
Unconstrained	4593.14 (2096)	2.19	0.812	-	0.065	0.049	0.80	Yes
Metric invariance (measurement weights)	4699.86 (2137)	2.20	0.807	0.005	0.071	0.049	0.76	Yes
Scalar invariance (measurement intercepts)	4775.92 (2165)	2.21	0.803	0.004	0.083	0.049	0.72	Yes

CFI: comparative fit index; CMIN: chi-square minimum discrepancy function; df: degree of freedom; RMSEA: root mean square error of approximation; SRMR: standard root mean square of the residual

Hypotheses testing for construct validity

A path of causal effect model was generated based on the selected interprofessional conceptual framework to test the predefined assumptions [24]. The practitioners’ dataset (Figure 5A) showed a good model fit with SRMR = 0.048, CFI = 0.930, GFI = 0.981 (the χ2(15) = 76.05 (CMIN/df = 5.07), fulfilling the COSMIN criteria for a good model fit [16]. The model obtained an inadequate fit for the student dataset (Figure 5B) with SRMR = 0.089, CFI = 0.818, GFI = 0.861, χ2(15) = 130.34 (CMIN/df = 8.69).

Figure 5.

Path analysis of assumptions model for practitioners (5A) and students (5B).

The model had 13 covariances with two admissible correlations of error terms within a similar construct of the consequences factors. The inclusion of these covariances resulted in a significant improvement of the model with SRMR = 0.076, GFI = 0.900, CFI = 0.859, RMSEA = 0.172, χ2(13) = 102.34 (CMIN/df = 7.873), fulfilling the COSMIN criteria for a good model fit [16]. These results confirmed the model fit for hypotheses testing in both datasets.

The study assessed the mediating role of “IPCP: shared goals and roles understanding” between “antecedent factors: leadership and communication” on the “consequences factors”. Bootstrap properties were set to 5000 number of samples with a bias-corrected confidence interval of 95%. In both datasets, the results revealed positive and significant direct effects of impact on antecedent factors: “ leadership and communication” on “shared goals - roles understanding”; and “shared goals - roles understanding” on “members relationships, barriers to team collaboration, conflict management and decision-making”, “patient Involvement”, and “community empowerment”, thus supporting HDir.1, HDir.2, HDir.3, HDir.4, HDir.5, and HDir.6 in both cohorts. The mediation analysis indicated that the “leadership and communication” relationship to “members relationships” and “patient involvement” was only partially mediated by “shared goals - roles understanding” in both cohorts. Full mediation of “shared goals - roles understanding” was provided in the relationship of “leadership and communication” on “barriers to team collaboration”. In both cohorts, no mediation effect was involved in the relationships of “leadership and communication on conflict management”, “decision-making”, and “community empowerment”. Summaries of hypotheses testing are presented in Table 5 and Table 6. With the tested hypotheses showing 77.3% positive and significant values, the COSMIN requirement was met, with at least 75% of the tested hypotheses being accepted.

Table 5. Interprofessional collaborative practice assumptions for practitioners.

Table 6. Interprofessional collaborative practice assumptions for students.

Discussion

This study aimed to conduct cross-cultural validation of the CPAT in Indonesian by examining the previously translated and validated instrument's internal structure and performing hypotheses testing. In line with the aim of the study–to provide a quality measure for assessing interprofessional education and collaborative practice among health practitioners and students in the Indonesian context–respondents were selected from these two groups. Multilevel analyses performed on the newly validated CPAT, consisting of evaluations of the internal structure and construct validity, indicated that the original CPAT's [22] factorial structure was not the best model for this current study population. Instead, the newly validated CPAT confirmed a 7-factor 48-item conformation with a factorial solution, closely resembling the previous Indonesian CPAT.

One novelty offered by this study, which the previous Indonesian CPAT did not provide, is that it validates the use for potential end users of the instrument (i.e., healthcare practitioners and students). The CPAT was previously validated only for practitioners, making its use in students not recommended [18,19]. Simultaneously validating the instrument for both practitioners and students is critical to bridging interprofessional education and interprofessional collaborative practice [12]. These two approaches are inseparable and mutually dependent [2,3,7,41,42]. IPE provides a training environment for better IPCP in actual practice, while IPCP reflects improvement and direction necessary for IPE training.

However, to ensure that the instrument is invariant across the tested groups, extreme perceptions that cannot be generalized across both groups must be moderated [37,38]. Consequently, items that are strongly endorsed by one cohort but not by another should be discarded. In this study, five CPAT items demonstrated these consequences. The CFA performed separately for each cohort indicated the need for conformational change to a 7-factor 48-item model. As a result, five items were removed from the previously validated Indonesian CPAT [13]. Items 13, 34, and 53 were removed to improve the dataset's composite reliability (CR) and average variance extracted (AVE). Furthermore, refinements were made to increase the fit indices of the factor 48-item model for the subsequent measurement invariance tests, resulting in the removal of items 28 and 39. After removing these five items, the CFA indicated good acceptability of both datasets to the 7-factor 48-item model. The MG-CFA, which simultaneously analyzed the two cohorts, further corroborated that the factorial structure of the 7-factor 48-item model was suitable for both datasets (the fit indices (i.e., CFI/TLI, or RMSEA, or SRMR) required by COSMIN were all met), with good internal consistency reliability for total scores and each subscale for both datasets. Furthermore, measurement invariance analysis indicated that the newly validated Indonesian CPAT for practitioners and students met configural, metric, and scalar invariances.

With the configural and metric invariances met, indicating that concerning the measure tested, practitioners and students agreed on the structural modeling of 7 factors, the positioning of the 48 items to the relevant constructs, and their understanding of the items comprising the domains in CPAT was the same. Thus, regardless of the stage of development of their interprofessional collaboration, practitioners and students perceived the constructs underlying the CPAT domains to have the same meaning.

The achievement of scalar invariance indicated that the mean scores for the two cohorts were  comparable when assessed using the newly validated Indonesian CPAT conformational structure. Comparable scores between the two cohorts have an important advantage. The newly validated CPAT Indonesia can measure students’ interprofessional development at the training stage and compare it with practitioners’ achievements in the actual practice settings. Collectively, these findings confirmed the equivalency of the newly validated Indonesian CPAT (7-factor 48-item model) for both cohorts.

Another highlight of this study is related to the latent factor of “conflict management and decision-making” (the current study Cronbach's α=0.71 for practitioner, and Cronbach's α=0.70 for student), which was also a relatively weak domain in both the original CPAT (Cronbach's α=0.67) and the previous Indonesian CPAT (Cronbach's α=0.70). Trial testing was conducted to explore the possibility of getting a significantly better fitting model if this subscale was to be omitted or retained. The trial resulted in similar trends for both datasets in the current study (i.e., no change in the SRMR and the chi-square), worsening of RMSEA (by 0.001–0.002 points), and a slight increase in CFI (by 0.002–0.003 points). Statistically, removing the subscale did not improve the model fit. In addition, “conflict management and decision-making” are essential theoretical constructs in the interprofessional collaborative practice conceptual framework [24,43-45]. Therefore, removing this subscale would have significantly reduced the comprehensiveness of the CPAT as an outcome measure for interprofessional collaborative practice, and it was hence not removed. More rigorous analysis, such as that offered by item response theory, e.g., Rasch analysis [46,47] could provide more nuanced statistical information when deciding whether to retain or drop an item.

Evaluation of construct validity using the model proposed in Figure 4 and Figure 5 confirms several assumptions significant to the theoretical framework of the constructs underlying interprofessional collaboration. First, team leadership and communication can influence team values regarding shared goals, clarification and division of tasks, and relationships between team members. Leadership and communication can also directly and positively influence how a team resolves conflict and makes decisions, as well as the team's position regarding openness to the idea of engaging patients, their families, and communities in care. Both tested groups supported all of these assumptions and re-confirmed the conceptual framework used by the suggested model [24] and other studies [41,45,48-50].

This research also confirms that the role of leadership and communication within a team can be strengthened to optimize member relationships and patient engagement in their care if the team conforms to their shared goals and clarifies its roles (as suggested by the accepted path analysis for hypotheses tests on HMed. 1 and HMed. 4 in both datasets). However, characteristics related to leadership and communication may not directly influence the team's ability to handle conflict and community empowerment in patient care, even though the team firmly maintained its shared goals and had clear roles within it (as suggested by the rejected path analysis for hypotheses tests on HMed. 3 and HMed. 5 in both datasets). Interestingly, practitioners felt leadership and communication can directly influence how people view issues that hinder their interprofessional teamwork (without necessarily sharing common goals and clear roles; no mediation function was identified). In comparison, students believed that leadership and communication functions would only persevere if the team adopted the principles of shared goals and clear roles (full mediation function). Previous research has broadly confirmed assumptions regarding the direct impact of one or more interprofessional constructs on other constructs [41, 45, 48-56]. However, the mediating role of one construct in optimizing or reducing the functional roles of other constructs is still limited, so further comparisons were not possible.

This study has strengths and limitations. Representing the voices of relevant stakeholders is critical in developing outcome measures [16,18,19]. These stakeholders, who will be the instrument's end users, namely healthcare practitioners and students, were well represented in this study with adequate sample sizes, according to COSMIN guidelines (practitioners, n = 266; students, n = 232). With this satisfactory number of participants, more diverse and robust data analyses such as the multi-group confirmatory factor analysis, measurement invariance tests, and hypotheses testing can be performed, requiring both cohorts to be analyzed simultaneously with adequate respective samples. However, the diversity of health professions was restricted as participants were dominated by physicians/medical students and nurses/nursing students in both cohorts.

COSMIN highlights that content validity requirements (assessing the items’ relevance, comprehensibility, and comprehensiveness) is an essential aspect of an instrument. However, because this study used a previously translated version for validation, the 53 Indonesian CPAT items were not piloted and thus not tested for content validity requirements. In addition, one subscale, “conflict management and decision-making”, was weak in the previous two versions of the CPAT, a finding that was corroborated in this current study. Future studies should carefully examine the items associated with this subscale and consider developing new items using item response theory (Rasch analysis) to verify the reliability and validity of these new items.

Conclusion

Based on COSMIN standards of psychometric parameters, the findings suggest that the newly validated Indonesian CPAT (7-factors 48-item model) has good psychometric properties in terms of internal structure (i.e., structural validity, internal consistency, and measurement invariance) and hypotheses testing. Therefore, it serves as a quality measure for assessing interprofessional education and collaborative practice for health students and practitioners in Indonesia.

Ethics approval

The Ethical approval for this study was obtained from the Curtin University Australia Human Research Ethics (Approval number: HREC2021–0274) and Faculty Medicine of Hasanuddin University Ethics Board (170/UN4.6.4.5.31/PP36/2023). All participants consented to participate in the study before completing the survey. Participants were presented with the previous Indonesian CPAT and requested to rate each item using the original instrument's 7- point Likert scale to validate the instrument.

Competing interests

All the authors declare that there are no conflicts of interest.

Funding

The first author is a PhD candidate at Curtin University, Australia. She received the Australia Awards Scholarship and the Curtin University Higher Research Degree Scholarship for her studies. The other authors did not receive external funding for this work.

Underlying data

All data underlying the results are available as part of the article, and no additional source data are required. The Underlying data is available at https://doi.org/10.5281/zenodo.13690762. The datasets used for this article are available under DOI: 10.5281/zenodo.7797090

How to cite

Ardyansyah BD, Cordier R, Brewer M, Parsons D. A psychometric evaluation of the Indonesian version of the collaborative practice assessment tool (CPAT) for assessing interprofessional education and collaborative practice among health practitioners and students. Narra J 2024; 4 (3): e1106 - http://doi.org/10.52225/narra.v4i3.1106.