Evaluating the test–retest reliability of the chiropractic professional identity embodiment scale

Abstract

Objective

The study evaluated the test–retest reliability of the scores obtained from the Chiropractic Professional Identity Embodiment Scale (CPIES), a brief self-report tool assessing professional identity in chiropractors and students.

Methods

Students were invited across all 4 year levels at a chiropractic college to complete an online survey featuring 15 CPIES statements rated on a 5-point Likert scale. The survey was administered twice, 4 weeks apart, with participants matched across time points using three sociodemographic identifiers. Test–retest reliability was assessed using the intraclass correlation coefficient (ICC) with a 2-way mixed-effects model for absolute agreement.

Results

Due to anonymous participation, the exact response rate could not be calculated; the estimated response rate was 24%. Of these respondents, 23 participants (35%) were matched across 2 time points. The CPIES sum score demonstrated good to excellent test–retest reliability (ICC = 0.92; 95% CI, 0.81–0.97). Most items demonstrated moderate reliability across assessments.

Conclusion

Data collected using the CPIES demonstrated good to excellent test–retest reliability, supporting its temporal stability. This provides further evidence to support use of the CPIES in research, education, and policy, when professional identity is increasingly recognized as a key competency.

INTRODUCTION

Professional identity has long been studied in sociology and psychology.^1,2 As a multifaceted and evolving construct, professional identity extends beyond mere affiliation with a profession, encompassing a dynamic interplay of individual beliefs and perceptions and the shared attitudes, values, knowledge, and skills that define a professional community.^3,4 Its development is shaped by the continuous acquisition of knowledge and skills; an understanding of professional realities and requirements; and adherence to ethical, moral, personal, and professional values.^5,6

Although professional identity evolves over time, Ibarra⁷ conceptualizes it as a stable and enduring construct shaped by attributes, beliefs, motives, and experiences within one’s professional role. Earlier career professionals, however, tend to exhibit greater malleability.⁸ This tension between stability and change highlights the relevance of latent state-trait.

Latent state-trait theory provides a framework for examining the stability and variability of psychological constructs over time.⁹ Within this model, traits, such as identity, are stable and enduring characteristics that remain consistent across contexts, whereas states, such as mood, are temporary and influenced by situational factors. Latent state-trait theory suggests that professional identity predominantly exhibits trait-like stability with some dynamic fluctuations in response to life events or external influences.¹⁰ For example, research indicates that measurement items can reflect both stable and dynamic elements of a construct,^11,12 such as temporary changes in chiropractic professional identity (CPI) during seasonal variations in practice volume.

Professional identity aligns with the trait end of the state-trait continuum, demonstrating stability and endurance rather than rapid or transient changes.^13,14 By extension, CPI is expected to share these characteristics. Identity formation is especially relevant during formal education and career transitions. In chiropractic students, influences include preprofessional education, institutional philosophy, and professional affiliation.^15–17 A clear professional identity strengthens confidence, advocacy, and clinical effectiveness.¹⁸

When professional identity is unclear, practitioners may lack confidence in advocating for professional opinions, contributing to feelings of being undervalued.¹⁹ While research in chiropractic is limited, evidence suggests that some chiropractors struggle to articulate their professional concepts clearly, often due to internal inconsistencies and a disconnect between professional identity and public perception.^20,21 In contrast, chiropractors who communicate their professional concepts effectively report greater success in applying interventions and achieving improved patient outcomes.²²

A poorly defined professional identity can have wide-ranging consequences for practitioners, patients, and the profession. Internal inconsistencies and external misperceptions contribute to role ambiguity and occupational stress,²³ which are associated with emotional exhaustion, depersonalization, and reduced personal accomplishment, key elements of professional burnout in chiropractors.²⁴ Conversely, a well-defined professional identity fosters professional responsibility,²⁵ enhances competence,²⁶ strengthens decision-making confidence,²⁷ and ultimately improves clinical practice and patient outcomes.^28,29

Given its significance, several instruments have been developed to assess professional identity. To be meaningful, these tools must demonstrate strong psychometric properties, particularly validity and reliability.³ Reliability is a prerequisite for validity as an instrument must first measure a construct consistently before its accuracy and interpretation can be trusted.³⁰ A highly reliable measure produces precise, reproducible, and stable scores when administered to the same group under similar conditions, ensuring minimal variation.³¹

Test–retest reliability, typically assessed using the intraclass correlation coefficient (ICC), examines whether scores remain stable when the underlying construct has not changed between assessments.³² The ICC is widely considered the preferred statistic for this purpose as it captures both the degree of correlation and the agreement between repeated measurements based on mean squares derived from analysis of variance.³³ A high ICC indicates that scores are consistent and repeatable, reinforcing the credibility and reliability of the instrument.

A range of tools has been applied to professional groups. Some are broad, such as the Professional Self Identity Questionnaire for medical health and social care students,²⁵ whereas others are discipline-specific, such as the Nurses Self-Concept Questionnaire,³⁴ the Development Scale for physicians,³⁵ and the Professional Identity Scale in Counseling.³⁶

In chiropractic, Glucina et al¹⁸ define professional identity as “a chiropractor’s self-perception and ownership of their practice philosophies, roles and functions, and their pride, engagement, and knowledge of their profession” (p. 80). However, until recently, no profession-specific measurement tool existed to assess professional identity among chiropractors. To address this gap, a mixed-methods sequential exploratory study developed the Chiropractic Professional Identity Embodiment Scale (CPIES).³⁷ Psychometric analyses demonstrated that the CPIES scores can be reliably measured either by 6 subscales or as a unidimensional 15-item scale, and it showed strong internal consistency and validity (α = 0.87, ω = 0.86) although its temporal stability had not yet been examined.

Building on the favorable internal consistency of the unidimensional CPIES, the primary objective of this study was to establish its test–retest reliability. Demonstrating temporal stability will determine whether the CPIES can serve as a brief, reliable, and profession-specific instrument for chiropractic research. A secondary objective was to examine whether CPI functions as a stable, trait-like construct or fluctuates in response to contextual factors.

METHODS

Participants

Because attitude statements examining aspects of CPI have previously been explored in a student population,¹⁵ this study recruited students from the New Zealand College of Chiropractic across all 4 year levels.

Study Design

Participants were recruited via convenience sampling in March–April 2023 through in-class and Microsoft Teams announcements. Surveys were completed online via Qualtrics at 2 time points, 4 weeks apart. Participation was voluntary and anonymous and required informed consent. A minimum sample size of 20 was set based on prior work.³⁸ The exact number of students present during in-class announcements or who accessed the Teams chat is unknown. Responses were included if sociodemographic variables (age, ethnicity, gender) were identical for the same individual and sufficiently unique to prevent duplicate matches. Participants completed the 15-item CPIES using a 5-point Likert scale, along with demographic questions. Age was categorized into 5-year brackets, and ethnic classifications followed Statistics New Zealand classifications.³⁹

Ethics

Ethical approval for this study was obtained from the Health and Disability Ethics Committee (2023 OOS 15421).

Statistical Analysis

Descriptive statistics and analyses were calculated using IBM SPSS v.29 (IBM Corp). Test–retest reliability was assessed using ICCs based on a 2-way mixed-effects model with absolute agreement⁴⁰ as appropriate for repeated measurements from the same individuals, where generalization beyond the sample was not intended. The model assumes no systematic differences between time points, homogeneity of variance across assessments, and approximately normally distributed measurement errors. The absence of a significant mean change supported the assumption of no systematic time effects, and visual inspection of score distributions and T1–T2 scatterplots indicated no obvious departures from normality or homogeneity of variance. Identical CPIES items, response options, and scoring procedures were used at both time points, supporting a consistent measurement metric and variance structure across administrations. Interpretation followed Koo and Li,³³ where ICC <0.5 = poor, 0.5–0.75 = moderate, 0.75–0.90 = good, and >0.90 = excellent reliability.

RESULTS

Due to anonymous participation and uncertainty about lecture attendance or Teams engagement, the exact response rate was unknown. Based on the sampling frame, the estimated response rate was 24% with 23 participants matched across T1 and T2, representing 35% of respondents. Year cohorts ranged from 57 in year 3 to 74 in year 1 (Table 1). Median completion time was 5 minutes at T1 and 4 minutes at T2. No significant mean differences were observed between assessments.

Table 1.

Sample sizes by year level and time point

Group	Cohort Size, n	Time 1, n	Time 2, n
Year 1	74	14	6
Year 2	69	20	12
Year 3	57	16	2
Year 4	78	16	3
Total	278	66	23

The typical respondent was a 20- to 24-year-old female, year 2 student of European descent, holding a diploma in applied science (Table 2).

Table 2.

Demographic characteristics of the participants

Demographic Characteristic	Category	Percentage
Age range	Less than 20	13
	20–24	52.2
	25–29	13
	30–34	4.3
	35–39	13
	40–44	4.3
Gender	Male	34.8
	Female	65.2
Ethnicity	European	30.4
	Māori	13
	Asian	13
	MELAA	13
	Pasifika	8.7
	Other-American	8.7
	Other-Canadian	8.7
	Other	4.3
Highest qualification	High school	17.4
	Diploma	47.8
	Undergraduate degree	21.7
	Post-graduate Certificate/diploma	13

MELAA, Middle Eastern, Latin American, and African.

For the agreement between T1 and T2 CPIES scores for a single measure (Table 3), test–retest reliability was classified as good to excellent (ICC = 0.92; 95% CI, 0.81–0.97). This suggests that 92% of the variance in scores was due to true score differences, whereas the remaining 8% was attributable to error variance, likely stemming from testing conditions or changes in respondents’ states between T1 and T2. At the item level, most demonstrated moderate stability. Item 8 had excellent reliability (ICC = 0.92; 95% CI, 0.82–0.97), whereas items 5 (ICC = 0.38; 95% CI, −0.06 to 0.69), 10 (ICC = 0.49; 95% CI, 0.09–0.75), and 12 (ICC = 0.48; 95% CI, 0.10–0.74) demonstrated poor stability.

Table 3.

CPIES item intraclass coefficient between time 1 and time 2

Item	Statement	ICC (interpretation)
1	Chiropractors need to understand the traditional chiropractic theory that adjusting the spine corrects “dis-ease”	0.59 (moderate)
2	It is important to maintain a mechanistic medical model perspective when conceptualizing a patient’s presenting concern	0.51 (moderate)
3	I recommend my profession to those who are searching for a new career	0.76 (good)
4	Patients should set their care plan based on their symptoms	0.51 (moderate)
5	The inclusion of clinical chiropractic training internships in integrative medical settings is essential to progress the profession	0.38 (poor)
6	Chiropractic history and the philosophy of chiropractic is what sets it apart from other professions	0.74 (moderate)
7	Chiropractic care is beneficial for the patient irrespective of symptom change/improvement	0.64 (moderate)
8	Chiropractic is predominately a wellness-based approach to health and well-being	0.92 (excellent)
9	The vertebral subluxation is the underlying cause of most health problems and clinical presentations	0.62 (moderate)
10	Multidisciplinary collaborative practice is the way forward for the chiropractic profession	0.49 (poor)
11	Contemporary evolving scientific evidence is more important than traditional chiropractic principles	0.55 (moderate)
12	Generally, I believe the body can heal itself without medicines (eg, use of vaccinations, antibiotics for a respiratory infection)	0.48 (poor)
13	It is vital to make a differential diagnosis(es) or clinical impression(s) when seeing a patient	0.72 (moderate)
14	I can clearly explain DD Palmer’s core chiropractic principles (eg, universal and innate intelligence etc)	0.75 (good)
15	I term people that receive chiropractic as “practice members” or “clients” (or similar) instead of “patients”	0.75 (moderate)
CPIES sum		0.92 (excellent)

DISCUSSION

The CPIES is a brief self-report instrument assessing professional identity in chiropractors and students. Previous evaluations showed that CPIES data exhibited strong internal consistency and validity;³⁷ however, test–retest reliability of CPIES scores had not been established, which this study addressed.

Test–retest studies are essential for evaluating the stability of measurement instruments across time in both research and clinical contexts.⁴¹ Stable test–retest reliability indicates that a construct is trait-like with minimal measurement error, whereas instability may reflect transient state effects or random error. In the absence of external influencing factors, such as semester breaks or exam periods, state effects are assumed to average out to 0 across participants.^41,42 Exploring test–retest reliability strengthens the psychometric credibility of CPIES scores and provides a deeper understanding of the CPI construct itself. Consistent with this rationale, CPIES scores remained stable over the 4-week interval with most variance reflecting true differences between individuals rather than measurement error. Stable reliability supports the interpretation of CPI as a predominantly trait-like construct over short intervals.

This study addressed 2 key conditions for test–retest reliability. The testing effect was mitigated by selecting CPI, a theoretically stable construct over short intervals, consistent with the assumption that the underlying characteristic remains unchanged between assessments.⁴³ Memory effects were minimized through an appropriate retest interval, reducing the likelihood that participants recalled previous responses, ensuring no substantial shifts in professional identity occurred.⁴⁴ These measures support the validity of the test–retest findings.

Future Studies

Future studies with larger samples could explicitly examine these assumptions using variance comparisons and residual diagnostics to strengthen confidence in reliability estimates. Future research should evaluate the CPIES in larger and more diverse populations, including educators and practitioners, and assess its discriminant validity relative to identity measures from other health professions.⁴⁵ Incorporating additional measurement occasions and advanced analytic approaches, such as generalizability theory,⁴² would allow clearer separation of state-like and trait-like variance. Further work should also examine whether stronger CPI predicts outcomes such as interprofessional collaboration, therapeutic alliance, patient satisfaction, health outcomes, and career success.^37,46,47

Strengths

This study used an appropriate 4-week test–retest design and an established ICC approach to show that CPIES scores are highly stable over time. Standardized measurement conditions and strong prior evidence for internal consistency and validity support its overall psychometric robustness.

Limitations

The primary limitation of this study was the sample size. While comparable to other reliability studies,⁴⁸ it falls below commonly recommended guidelines of 30 to 100 participants^31,49 for test–retest designs. Smaller samples can increase standard error and widen confidence intervals, reducing the precision of item-level stability estimates;³⁸ therefore, findings should be interpreted with caution. The modest sample size also constrained the ability to formally test key assumptions underlying ICC estimation, including homogeneity of variance across time points and normality of measurement error. The sociodemographic matching procedure further reduced the effective sample size and may have introduced bias; the use of participant generated identifiers could improve future studies. Sensitization effects are also possible as repeated exposure to items may reduce novelty.⁵⁰ Finally, the sample consisted solely of students in the formative stages of professional identity,⁵ limiting generalizability to practicing chiropractors.

CONCLUSION

The CPIES scores demonstrated good to excellent test–retest reliability with moderate item-level agreement, supporting the temporal stability of the measured construct and contributing to evidence of score validity. These findings support the scale’s broader use in research, education, and policy, where professional identity is increasingly recognized as a core competency.

DECLARATIONS

Funding

Funding was provided by the Centre for Chiropractic Research Supporters, New Zealand College of Chiropractic, and Haavik Research Ltd. The funders played no role in the design, conduct, or reporting of this study.

Competing Interests

The authors declare no competing interests.

Data Availability

Data are available upon request.

Artificial Intelligence

Artificial intelligence was not used to prepare this paper.

Contributorship

Concept development: TTG. Design: TG, CUK, KS, KH. Supervision: CUK, KH, KS. Data collection/processing: TTG. Analysis/interpretation: TTG, CUK, KS, KH. Literature search: TTG. Writing: TTG. Critical review: CUK, KH, KS.