An Initial Validation Study of the Self-Rating Scale of Self-Directed Learning for Pharmacy Education

Abstract

Objective. The purpose of this study was to explore and validate the factor structure of the original SRSSDL scale with pharmacy students enrolled in a four-year Doctor of Pharmacy program at a southeastern university, and to assess the differences in the self-directed learning behaviors across different class years of students.

Methods. Factor analysis was used to identify the factor structure of a self-rating scale of self-directed learning (SRSSDL) among pharmacy students (n=872) and to examine students’ self-directed learning (SDL) behaviors by year in the pharmacy education curriculum.

Results. Five factors – intrinsic motivation, awareness, collaboration, reflection and application – showed acceptable levels of reliability. P4 students scored significantly higher than P2 students on the total scale. P4 students scored significantly higher on awareness than P1 and P2 students, while P2 students had a significantly higher collaboration score compared to P1 students.

Conclusion. The revised 55-item SRSSDL is a valid and homogenous scale of pharmacy students’ self-directed learning within one pharmacy program. However, due to differences in factor structure compared to earlier studies, further research is needed before this survey tool can be broadly implemented in pharmacy education.

INTRODUCTION

Due to the rapid advances in health care, the knowledge and skills of a Doctor of Pharmacy graduate can quickly become obsolete. This reality has led pharmacy educators to develop curricular approaches that will prepare graduates to be self-directed learners. To assess achievement of this educational outcome, evaluation methods that measure self-directed learning (SDL) are needed.

The literature characterizes SDL as a concept with two related components. The first component encompasses the personal attributes necessary for SDL such as having a goal orientation, personal autonomy, self-management, and motivation. The second component views SDL as an educational process that the learner must be able to enact. For example, Malcolm Knowles described the process as requiring individuals to take initiative and diagnose their learning needs, formulate learning goals, identify learning resources, select and use learning strategies, and self-evaluate achievement of the learning goals.¹ Consistent with these two components, Benedict and colleagues have defined SDL in pharmacy education as “the development of the skills and attitudes necessary to become an independent, confident, and life-long adult learner.”² SDL and lifelong learning are sometimes used interchangeably. However, SDL is seen as a pre-requisite for lifelong learning.³ Lifelong learning refers to individuals moving in and out of educational programs throughout their lifetime.⁴ The literature also asserts that the motivation and ability to be self-directed varies with the context for learning.^3,5 The ability to be self-directed in learning is influenced by relevant study skills, social, cultural and educational setting, past experience and self-concept. Eva suggested that individuals are limited in their SDL ability because context specificity limits the capacity to self-assess.^3,6

The validity of SRSSDL for use in pharmacy education has not been explored. There is a need to evaluate the change of pharmacy students’ SDL as they progress across the curriculum. The purpose of this study was to explore and validate the factor structure of the original SRSSDL scale with pharmacy students enrolled in a four-year Doctor of Pharmacy program at a southeastern university, and to assess the differences in the self-directed learning behaviors across different class years of students.

Although several authors have published scales for measuring SDL, there have been questions about their validity. Guglielmino’s SDL readiness scale (SDLR) is a practical instrument for measuring an individual’s attitudes, abilities, and skills, necessary for SDL readiness.⁷ However, criticism was leveled against the original SDLR regarding its reliability, validity and failure to confirm its eight-factor structure when applied to various racial and class populations.^8,9 Recognizing these problems, Fisher and colleagues developed an SDLR and evaluated it using factor analysis.¹⁰ Hendry and Ginns investigated the validity of the Fisher SDLR with medical students.^10,11 The factors identified in their study with medical students did not correspond well with those initially reported by Fisher that involved nursing students. Hendry and Ginns concluded that the Fisher SDLR was not stable across different types of students and recommended further research.^10,11

In 2007, Williamson developed the Self-Rating Scale of Self-Directed Learning (SRSSDL).¹² Instead of measuring readiness like the Guglielmino SDLR and the Fisher SDLR, the SRSSDL was developed to measure levels of self-directed learning behaviors. Following a review of Guglielmino’s, Knowles’, Candy’s, Hiemstra’s and Brookfield’s works, Williamson developed a list of 75 items related to self-directed learners’ attributes, skills and competencies.^5,7,13-15 Using the Delphi method, the researchers developed an instrument with the 60 items categorized into five broad areas, each consisting of 12 items. The categories included: awareness (understanding the factors that contribute to being self-directed learners); learning strategies (strategies recommended for being self-directed learners); learning activities (activities often used in self-directed learning); evaluation (attributes that help learners monitor their learning activities) and interpersonal skills (skills considered pre-requisite to becoming self-directed learners). Thirty nursing students then took the survey and rated the items using a 5-point Likert scale, where 5=always, 4=often, 3=sometimes, 2=seldom, and 1=never. This study did not validate the SRSSDL using factor analysis. The subscales of each area had acceptable internal consistency with Cronbach’s alpha coefficients ranging between 0.71 and 0.79. The researchers established construct validity of the scale by comparing the total scale scores of first- and fourth-year students.

Cadorin and colleagues determined the factor structure of the Self-Rating Scale of Self-Directed Learning (SRSSDL) by consecutively surveying Italian nursing and radiology students and practitioners who attended hospital-sponsored educational seminars, initiatives and workshops between 2009 and 2010.^16,17 This study sample consisted of 847 participants, including 453 nurses, 141 radiology technicians, 182 nursing students and 68 radiology technician students. Using factor analysis, Cadorin and colleagues developed a revised SRSSDL that consisted of 40 items instead of the original 60 items proposed by Williamson.¹² This Italian version was also found to have eight factors instead of the five factors established in the initial version.

METHODS

The 2007 version of the SRSSDL was administered to pharmacy students. As described by Williamson, the scale for responding to the items was 5=always, 4=often, 3=sometimes, 2=seldom, and 1=never. This survey consisted of 60 items and therefore, a minimum sample size of at least 300 (>5:1 ratio of items: students) was considered necessary to perform factor analysis on the data set.¹⁸ The SRSSDL was administered to all first-, second-, and third-year pharmacy students during the fall semester and to fourth-year pharmacy students at graduation. The students completed a paper-based version of the SRSSDL during a regularly scheduled class session; participation was voluntary. They were allotted 20 minutes to complete the survey. These students were located on four campuses in the state. The paper-based responses from all four class cohorts were coded and entered into Microsoft Excel (Redmond, WA) with accuracy of data entry verified by two individuals. Factor analysis of the SRSSDL instrument and other statistical analyses as described below were conducted using SPSS 23.0 (IBM Corp., Armonk, NY).

A preliminary analysis was performed to determine whether exploratory factor analysis (EFA) was appropriate for the data set. EFA is a procedure that determines how instrument items are related and whether the items can be categorized into a smaller number of unobserved variables called factors. This preliminary analysis consisted of two tests. The Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy was computed to determine whether there was intercorrelation among the variables within the dataset. A KMO value of >0.30 was considered to infer sampling adequacy. Bartlett's Test of Sphericity was used to determine whether there was redundancy of variables and value of <0.05 inferred sampling adequacy. Intercorrelations of the items were examined to determine if any items were highly correlated. Items with correlation values of >0.9 were removed from the scale.

An exploratory factor analysis (EFA) was then conducted to identify the underlying factor structure that best fit the data. Principle component analysis (PCA) was used to identify the small number of uncorrelated components. Factors were only included if the eigenvalues were >1. A Scree Plot was done to visually examine the “elbow” curve of eigenvalues. The factor communality was examined to ensure adequacy. Items were considered “weak” if their factor loading was less than 0.3 and were deleted from the scale.¹⁹After the initial solution was determined, factor loading was rotated by using Varimax rotation to allow factors to be independent and achieve simple factor structure. Cronbach’s alpha (α) was calculated to measure the construct internal consistency of the resulting factors and an α of >0.60 was considered acceptable.²⁰ Finally, the factor name was labelled based on the theoretical framework.

After the SRSSDL scale was developed and validated, the data set was analyzed to identify any significant differences in SRSSDL scores among the four class years of students. ANOVA and multiple comparison t-tests were performed to examine the mean total and factor scores among the four student groups. Fisher’s least squares difference (LSD) correction was used to control for Type I error. The institutional review board (IRB) designated exempt status for this study.

RESULTS

The final data set consisted of 872 participants with first-year (P1) students (29.1%), second-year (P2) students (28.1%), third-year (P3) students (26.7%) and fourth-year (P4) students (16.1%) (Table 1). There was a smaller percentage of P4 students because of the timing of student attendance on those campuses in which only one of the three campuses were able to participate in the study. Students on all four campuses had similar GPAs and had performed equally well and similar to descriptions of previous cohorts.²¹

Table 1.

Descriptive Statistics for Each Factor Across Student Year Groups

Before performing factor analysis, the data were examined, and no missing data were identified or recoded. The data set was considered suitable for factor analysis since the KMO sampling adequacy was .96, Bartlett’s Sphericity test was significant (p<.001) and the inter-correlation among 60 items were generally above 0.3 (56 of 60).

Principal component analysis was then performed to extract the factors. The initial eigenvalues and scree plot revealed that the first five factors explained 49.7% of the variance. The structure with four, five and six factors was then examined separately by using Varimax rotations. The five-factor solution, was determined to be most appropriate based on eigenvalues and scree plot (Table 2). The five factors and their variance are as follows: intrinsic motivation (31%), awareness (7.7%), collaboration (5%), reflection (3.4%) and application (2.5). Five items were deleted because four of the items loaded on more than one factor and one of the items exhibited weak loading. Specifically, the following items were deleted because they loaded on more than one factor: “I consider teachers as facilitators of learning rather than providing information only” (Item 1.3), “I feel that I am learning despite not being instructed by a lecturer” (Item 1.12), “I find modern educational interactive technology enhances my learning process” (Item 2.11), and “I review and reflect on my learning activities” (Item 4.10). Item 5.12 (“I find it challenging to pursue learning in a culturally diverse milieu”) was deleted given its weak factor loading.

Table 2.

Component, Item-loadings, Explained Variance and Internal Consistency

The Cronbach’s alpha for each factor ranged from 0.7 to 1.0. The internal reliability for the construct is 0.9 (Table 2). Table 3 outlines the final 55-item version of the SRSSDL with categorization into the five factors.

Table 3.

Final 55-Item Version of the SRSSDL After Factor Analysis with Notation of the Factor Number in the Original Williamson Instrument

Construct validity was documented by determining differences in mean total scores among the four student groups. Higher scores indicate higher self-directed learning behaviors. As reported in Table 4, the mean total score of P4 students (219.1) was significantly higher than that of P2 students (212.6, p=.029). For the awareness factor, P4 students had a significantly higher mean score than P1 and P2 students. (p=.47 and p=.009) (Table 4). Additionally, P2 students’ mean (22.9) had a significantly higher collaboration score than the P1 students’ (22.0 p=.02). No significant results were found for other factors.

Table 4.

Significant Mean Scores by Subscale and Student Year

DISCUSSION

This study focused on exploring and validating the factor structure of SRSSDL scale for pharmacy students. This factor analysis categorized the 55 items into five factors similar to the original SRSSDL. However, the items loaded onto the five factors were different in this study with pharmacy students as compared to the original Williamson study.¹² In a preliminary Italian study, Cadorin and colleagues administered the original 2007 survey tool and reported the same five factors as Williamson.¹⁴ However, in the final Italian study where the original 2007 survey was administered to both practitioners and students, there were eight factors.¹⁵ This factor instability suggests further research is needed to assess the stability of factors across different health professional groups and learning settings.

Others have pointed out that contextual factors such as social, cultural, educational setting, and past experiences impact the ability and motivation of learners to be self-directed.^3,5,22 These contextual factors may explain the issues of stability across different health professional groups and learning settings. Therefore, further research in this area is needed before broad implementation of the 2007 Williamson survey in pharmacy education can be done.

The total and mean score for subcategories were calculated and compared across four student groups using multiple-comparison t-tests. The results showed that the total mean scores were all above 210. These high scores indicate that pharmacy students had high levels of SDL behaviors across all four class years with an increase across the curriculum. Since P1 students completed the survey as they began their curriculum in the fall semester, these findings suggest that they have high levels of SDL behaviors as they entered the PharmD program.

Graduating P4 students showed a significantly higher total score compared to P2 students who completed the survey as they started the curriculum. P2, P3, and P4 students experienced the same curriculum and program expectations. Perhaps the curriculum was the reason that P4 students, who have more training and pharmacy-related experience, demonstrated higher SDL behaviors during the learning process. Additionally, P4 students demonstrated significantly higher scores in awareness compared to P1 and P2 students. Consistent with previous studies, higher self-management or awareness in learning, and identifying study needs result in higher individual career achievement.^16,23 These results could be a reflection of the students’ positive response to the pharmacy curriculum that includes student-centered learning approaches.

P2 students’ significantly higher level of collaboration compared to P1 students is likely because P1 students were just beginning the curriculum when they completed the survey and because P2 students had participated in teamwork and collaboration during their first year. Students generally rated their capacity of self-motivating, effectively reflecting and applying their learning into practice high.¹⁶

Worth noting is that P1 students’ scores were generally high on SRSSDL scale subcategories. As a self-reported assessment was conducted, perhaps these entering students simply provided responses in terms of what they believe should be important and that they over evaluated their ability of SDL skills or simply demonstrated ceiling effect and/or social desirability bias. Another limitation is that the study was conducted only among pharmacy students across four campuses in Florida and access to P4 students was limited. Also, the study was conducted only on the one-time point. Thus, this study lacked information on test retest reliability and generalization of SRSSDL scale for pharmacy education.

CONCLUSION

The 55-item version of the SRSSDL is a homogeneous and valid tool for measuring SDL behaviors for pharmacy students. However, since it does not identify the same constructs as the original Williamson SRSSDL or the Cadorin and colleagues study, there is a concern about stability. Further study is needed before broad adoption of the SRSSDL in pharmacy education can be done. In addition, P4 students had a significantly higher total SDL score compared to P2 students.