Abstract
Objective
To identify individual and social factors associated with pharmacy students' level of reflection in an advanced pharmacy practice experience (APPE).
Methods
A postal questionnaire, including a reflective assignment, was sent to all pharmacy interns (n=262) at Uppsala University, Sweden, for 4 semesters in 2005-2007.
Results
In a univariate analysis, 7 factors were found to be associated with students' level of reflection. After controlling for covariates, 3 social factors were found to be independently associated with reflection: having a formal preceptor (OR=5.3), being at a small pharmacy (OR=19.8), and students' perception of the importance of discussing critical thinking with the preceptor (OR=1.2). No correlation could be observed between level of reflection and critical thinking, nor learning style.
Conclusion
Social components seem to be of higher importance than individual components in students' reflective levels after pharmacy internship experience. Trained preceptors are important to foster reflection skills.
Keywords: advanced pharmacy practice experience, internship, reflection, critical thinking, learning styles, assessment
INTRODUCTION
Reflection is important in developing and fostering professionalism in pharmacy. It is commonly used as a teaching method in nursing, medicine, and sometimes in pharmacy.1-9 Reflective skills are often important during pharmacy practice experiences (known in Sweden as internships), where prior theoretical knowledge is applied in a practical setting. In American pharmacy education, reflective portfolios have been mandatory in several different courses, including advanced pharmacy practice experiences (APPEs), since 2007.5,6 Improving students' abilities to reflection is possible by introducing teaching methods.10 In order to develop such methods and implement them successfully, a greater understanding of the factors associated with reflection is useful.
In practice, much of the knowledge used is tacit and has to be elucidated and reframed in order to develop practice. Problems are seldom well defined and scientific, but rather include empirical knowledge and “gut feelings” and are closely connected to the activity performed.12,13 Schön suggests that reframing and change is essential in order to achieve a higher level of practice.13 From a learning perspective, the central function of reflection is to validate prior learning, or draw attention to the grounds or justification for our beliefs.14 If there is reason to doubt prior learning or beliefs because of changes in knowledge, social interactions, personal feelings, or intentions, one must resolve these issues before continuing to learn. Reflection can be seen as the central dynamic, essential to higher learning and changes in behavior when involved in problem-posing and transformation of meaning perspectives.14 In order to foster reflection skills, reflective essays, portfolios, and other educational techniques are often introduced in higher education.1-9 When introducing reflection, an ability to measure the outcome is necessary to assess students' progress and evaluate the education.8,15-17 Measuring reflection is challenging; however, a categorization scheme can be used to assess written essays and determine the level of reflection reached.8,17
Factors Associated With Reflection Skills
A number of factors can hypothetically be associated with students' level of reflection after an APPE. As in most learning, these might include individual as well as social components (Figure 1).18-21
Figure 1.
Simple model of components that are hypothesized to be associated with reflection
Individual factors potentially influencing students' reflective ability include student characteristics, critical thinking ability, and learning style.15,18,19,22,23 Students' interest in learning the profession is also of great importance.24
Critical thinking is frequently used to evaluate reflective interventions and reflective curricula, since critical thinking is supposed to be closely related to reflection.15,25 There are several well-validated instruments to measure critical thinking ability and there are several definitions of critical thinking.9,26 Critical thinking can be seen as a way to make assumptions from reasoning, especially scientific reasoning based on arguments.9,26 Reasoning is also an important element in reflection.9 However, reflection encompasses a broader range of knowledge and emotions.13,27 Subjective ideas are also included within reflection as applying them can bring out new ways of handling situations in practice.13 In that way, reflection can be seen as a way for an individual to apply critical thinking to one's own work and situation, expanding it to include feelings, hunches, and subjective feelings. It allows personal feelings and artistry to be a part of professionalism and knowledge.13 In this sense, reflective ability can be seen as an individual process.19
The term “learning style” can be defined as “the way in which individuals approach a certain learning situation.”28 Learning style tools are often used to promote self-reflection among students and practitioners.22 Learning styles are cognitive styles that reflect a person's individuality in a learning context.29 The way in which individuals approach learning has an impact on achievement of learning outcomes28 and this may include reflection. If students' learning styles do not align with an instructor's teaching style, this can interfere with students' motivation.22
Social components potentially influencing reflective ability include factors such as workplace environment and interaction with a preceptor.21 The importance of mentors (preceptors) is discussed by Schön, who finds them of primary importance for developing the reflective practitioner.12 The discussion between the preceptor and student stimulates reflection, and can be regarded as a social component. At the Uppsala University in Sweden, the preceptors assigned to pharmacy students are trained in facilitating reflective discussions.8
Environmental factors in a pharmacy such as stress, personal responsibility, and time are also social components that can effect the learning opportunities provided, and hence the students' ability to reflect.21,30-34 The organization's attitudes towards education is also important for personal professional development.21,32,35 Furthermore, the university can contribute to students' learning environment by giving the students different assignments, such as reflective diaries and reflective essays.8,7,36,37
The APPE (pharmacy internship course) in Sweden has undergone major changes to introduce a curriculum where the students' development of reflective abilities is a central component. The course is a 6-month APPE at the end of the 5-year pharmacy program, and students practice at community pharmacies supervised by a trained pharmacy preceptor (referred to in Sweden as a tutor).8,38 The formal preceptor can be either a pharmacist (5-year academic education, master of science in pharmacy) or a prescriptionist. The prescriptionist (3-year academic education, bachelor of pharmacy) has the same legal responsibilities and rights as pharmacists; both are allowed to dispense medicines, serve, and counsel patients at pharmacies. The students have had no prior APPE in the pharmacy program and are not introduced to the concept of reflection prior to the APPE.8
Several qualitative studies have been done in order to determine factors of importance for students' reflective skills, which could be of importance when developing new, reflective courses.1,21,39 However, no quantitative studies confirming these associations could be found in the literature.
The objective of this study was to quantitatively identify factors associated with pharmacy students' level of reflection by the end of the APPE, including individual components such as personal characteristics, critical thinking ability, and learning style, as well as social components, such as precepting and working environment.
METHODS
The study population consisted of all APPE students in the tenth semester of the master of science in pharmacy program during fall 2005 (n=83), fall 2006 (n=63), spring 2007 (n=53), and fall 2007 (n=65); in total, 262 students.
Two questionnaires were developed: 1 to be administered at the beginning of the semester and 1 to be administered at the end of the semester. The questions covered factors that could have an impact on students' development during the APPE, including a range of both individual and social factors (for the complete questionnaire, please contact the corresponding). There were questions about student characteristics, precepting, pharmacy placement, and learning style. Student characteristics included age, sex, year of APPE, native language, previous work experience, and perceived importance of the APPE. The questionnaires were piloted and revised before distribution. The questionnaires were distributed to students at the beginning and end of the course in the fall terms of 2005, 2006, and 2007. An additional merged questionnaire, that included questions from both questionnaires was sent out at the end of the spring term in 2007. This study was based on the end-of-course questionnaires from 2005, 2006, and the 2 questionnaires from 2007. The questionnaires were sent to the students on paper and returned anonymously by mail. It was clearly stated in the questionnaire that their answers would not affect their course grading. The participating students were not rewarded for their participation, but were allowed to fill out the questionnaires and take the CTA test during APPE working hours.
Level of Reflection
Students' reflective essays about their views on patient counseling at the pharmacy during the last 2 weeks of their APPEs were used to assess the students' levels of reflection.8 Writing the essays was mandatory but students were able to opt out from having their essays included in the study. The essays from fall 2005 and fall 2006 were assessed independently by 2 raters, one of the authors (A.W.) and a pharmacy master thesis student, and rated according to predetermined levels of reflection in a validated categorization scheme.8 Differences were resolved by discussions between the 2 raters, resulting in consensus versions of the categorization. The essays written in 2007 were assessed by 1 of the raters of the 2005-2006 rating (A.W.). The level of reflection ranged from 1 to 6, where levels 1-3 were defined as non-reflective and levels 4-6 were reflective. The definition of the different levels of reflection are further described elsewhere (Figure 2).8
Figure 2.
Categorization Scheme for Determining the Level of Reflection Based on Written Essays8
Critical Thinking
Of several instruments available for measuring critical thinking, the Watson-Glaser Critical Thinking Appraisal (CTA) is the only one available in Swedish.40-42 Each of the CTA's 5 subsets assesses a different aspect of critical thinking. The 80-item test was distributed to the students' preceptors at the same time as the end questionnaire, 2 weeks before the end of the APPE. The preceptors then administered the CTA to the students, allowing them 40 minutes to complete the test.
Learning Styles
The Pharmacists' Inventory of Learning Styles (PILS), a pharmacist-specific instrument for defining, describing, and measuring learning styles, is constructed around 2 primary axes: structured vs. unstructured and reflecting vs. doing (Figure 3).22 “Doing” individuals (enactors and directors) are those who prefer the opportunity to experiment and try out, while “reflecting” individuals (creators and producers) are those who prefer to observe and practice before trying. Unstructured environments, preferred by enactors and creators, are those in which expectations are defined individually or personally, while structured environments, preferred by directors and producers, are those where expectations are defined externally. The 2 axes create 4 quadrants with 4 corresponding learning styles: enactors, producers, directors, and creators (Figure 3). The 17-item inventory was translated into Swedish by the authors and piloted, and then the translation was revised. The final version of the PILS was included in the questionnaire administered to APPE students 2 weeks before the end of the course. The students were familiar with the test as they had filled it out at the beginning of the course as a part of the introduction to the APPE.
Figure 3.
Typology of the Pharmacists' Inventory of Learning Styles22
Statistical Analysis
The data analysis was carried out using SPSS for Windows, version 12.0.1 (SPSS, Inc, Chicago, 2003). When comparing reflective and non-reflective APPE students in the univariate analysis, the chi-square test was used to analyze categorical data and the independent t test was used to compare means.
Logistic regression analysis was used to identify factors associated with reflection, while adjusting for covariates. Significant variables in the univariate analysis with a p value of less than 0.05 were included in the logistic regression model.43
The confidentiality of the respondents was protected during the analysis by the use of code numbers.
RESULTS
The response rate varied between different subtests and different semesters. The overall response rate was 71% (186/262). The response rate for students completing all components was 50%, taking the internal attrition into account, but for each subtest the response rate varied between 30% and 92%. The mean age of the respondents was 26 years (range 23-42 years); 81% were women, 4% were parents, and 18% had a native language other than Swedish.
Three quarters of the students did their APPE at a community pharmacy in a city center, 15% at a community pharmacy located in a hospital area, and 7% at a community pharmacy located at a primary care center. The APPE pharmacies varied in size: 18.9% had fewer than 10 employees, 45.3% had 10-20 employees, 30.4% had 20-30 employees, and 5.4% more than 30 employees. Forty-five percent of the pharmacies were located in large cities (>200,000 citizens), while 31% were in cities with 50,000 - 200,000 citizens, 21% in towns with 10,000 - 50,000 inhabitants, and 3% in villages with less than 10,000.
The working environment was seen as acceptable, with a mean of 6.2 on a scale from 1 to10. The stress level at the pharmacies was perceived as moderate. Fifty-seven percent stated that, at the pharmacy, all personnel always had time to think, discuss, and educate the APPE students, or that this was the case except at times when the only focus was to serve customers. The rest (43%) stated that there was seldom or never time to think, discuss, and educate the APPE students and that most often, the only focus was on serving customers. Of the students, 92.4% thought that the APPE had met or exceeded their expectations.
Reflectivity, Critical Thinking, and Learning Style
The students' mean score on the level of reflection measured by the scheme was 4 ± 0.8 (Table 1). Of the 191 students, 73.8% were rated as reflective (defined as a score of 4 to 6).
Table 1.
Level of Reflection Among Students in Advanced Pharmacy Practice Experiences (N = 191)
The students' mean score on the CTA was 59.8 ± 9.3 out of a maximum possible score of 80. The median score was 61. The scores on subtest 2, “inference” were slightly lower than the scores obtained in other subtests. The CTA results were dichotomized into high critical thinkers (defined as above median) and low critical thinkers (below median).
According to the PILS test on learning styles, the distribution of learning styles among the APPE students was uneven. The majority of the respondents were categorized as directors and producers, while only a few students were categorized as creators or enactors (Table 2).
Table 2.
Learning Styles Among Students According to PILS
Abbreviations: PILS = The Pharmacists' Inventory of Learning Styles
aMultiple styles is defined as scoring equally high in 2 or more of the 4 categories.
Factors Associated with Reflection
No correlation could be found between the level of reflection and age, gender, students with or without children, or native language (Table 3). However, APPE students who already had secured an employment to begin after graduation were more reflective than students who had not (81.3% vs. 60.4%, respectively; p = 0.006).
No correlation was found between student cohorts completing the APPE in different years, but a strong linearity was found, indicating that the later year, the higher proportion students who were rated reflective (r2=0.92).
There was no correlation between students' level of reflection and critical thinking (p = 0.86). Nor was there any correlation between students with high levels of reflection and those who were high critical thinkers (p = 0.68). Analysis of students' scores on the CTA subtests revealed no correlations either.
A slightly higher proportion of students with producer and creator learning styles were reflective than those with other learning styles, but the difference was not significant.
Differences that were significant at the 90% level could be seen between students in the reflecting categories (B, D and BD) and students in the doing categories (A, C and AC); 80.9% were reflective as compared to 66.1% (p = 0.056).
APPE students with a prescriptionist as their preceptor were reflective to a higher degree than students with a pharmacist as their preceptor (p = 0.044). Students with female preceptors also were more reflective (p = 0.009). Further, students who had an informal preceptor responsible for their training were reflective to a lesser degree (p = 0.028) than those who received precepting from a formal preceptor. Students who found it important that the preceptor taught them critical thinking were more reflective (p = 0.01). On the other hand, no differences could be observed between students who thought it was important that the preceptor helped them to reflect and those who did not. Students who were satisfied with preceptor discussions about the pharmacist's role were more reflective than those who were not (p = 0.042). Neither students' satisfaction with the pharmacy and their preceptor, nor whether the APPE had met, exceeded, or failed their expectations were correlated with the student's ability to reflect.
The smaller the pharmacy, the higher the proportion of students who were rated reflective (p = 0.017). The working environment at the pharmacy, perceived level of stress, and location of the pharmacy, however, were not related to the students' level of reflection.
Multivariate Analysis
Data regarding the logistic regression analysis of factors associated with high reflection, while adjusting for covariates, is presented in Table 4. All significant variables with a p value <0.05 were included and semester was added due to its linearity (Table 3). The 8 included variables were: semester, number of employees at the pharmacy, the student already having a job arranged for after graduation, the preceptor's level of education, the preceptor's sex, precepting received from trained formal preceptor, perceived importance of discussing critical thinking with the preceptor, and satisfaction with preceptor discussions about the pharmacist role. Complete data were available for 120 students (46%) who were then included in the logistic regression analysis. According to the logistic regression analysis, receiving formal precepting (OR = 5.3), being at a small pharmacy (1-10 employees) (OR = 19.8), and a high perceived importance of discussing critical thinking with the preceptor remained (OR = 1.2) independently associated with reflection.
Table 4.
Logistic Regression Analysis of Factors Associated with Reflection
aIncluding all variables with p < 0.05, and semester from the univariate analysis
bp < 0.05
DISCUSSION
The aim of this study was to quantitatively identify both individual and social components associated with reflection. The only individual factor found to be associated in the univariate test was employment after graduation. The other factors found to be associated with reflective ability were all social components (Table 3).
Individual Components
No correlation between any personal characteristics and reflective ability could be found. The results indicate that individual components seem to be of less importance to the student's ability to reflection at the end of the course.
The ability to get a job after graduation possibly could be an outcome of good reflective skills, and not a contributing factor to the ability to reflect, but there is not sufficient data to reach this conclusion.
No correlation found between CTA score and reflection strengthens that reflection and critical thinking is not the same, and that critical thinking is not an optimal way to evaluate interventions aiming at improving students' reflective ability as discussed by Bourner.15 However, the limited number of respondents in this study may have underestimated potential correlations.
No relationship was found between learning styles and reflection. The argument for the assumption that learning style is a factor associated with reflection is that according to the axis used in PILS, creators and producers should score higher on level of reflection than enactors and directors. On the other hand, enactors and directors are more geared to learning by doing; hence, they might reach a higher level of reflection in the measurement used.
The association between reflection and perceived importance of discussing critical thinking with the preceptor could be a result of a mix-up between reflection and critical thinking. Students who perceive discussing critical thinking with the preceptor as important may actually mean that reflective discussions are important, but more easily identify this as critical thinking instead of reflection. However, discussion with the preceptor is a social interaction, and if students perceive it as an important part of their APPE, they are more reflective, which strengthens the hypothesis that interaction with the preceptor is important in developing an ability to reflect. Also, it stresses that the students' attitudes toward precepting are important to their ability to reflection.
Social Components
Preceptors' are important for training students in an actual workplace21,32,34,39 and seems to have a major impact on students' abilities to reflect as well. The informal preceptors are probably not as receptive to reflective discussions as formally trained preceptors. Another reason may be that the formal preceptors constitute a self-selected group with a special interest in mentoring. Preceptors who are willing to invest the time can foster students' learning.32,34,44 The preceptor's function as a role model is confirmed in other works.39,44 Preceptors can help students reflect by discussing and assessing reflection and professional learning.24,44 The differences in precepting and the reasons for these findings need further investigation, but these results point out the importance of training of the preceptors and that preceptor can increase students' reflective skills.
It was found, surprisingly, that prescriptionist preceptors, with the lower academic degree, contributed more to students' reflection than preceptors who were pharmacists according to univariate analysis. The explanation for this may be that the proportion of prescriptionist preceptors is higher at smaller pharmacies, where more reflective students were found.
The proportion of reflective students increased over time, from one cohort of APPE students to the next. This could be an effect of an increased focus on reflection in the APPE course curricula, but further studies are needed. Since the spring of 2006, preceptors have been trained in reflective precepting and reflection has been stimulated by several assignments during the APPE.8 However, the full effect of the changes may not have been reached yet, since the teaching is mediated by preceptors who need to adopt new ways of teaching.
The identified association between reflective ability and the number of employees at the APPE pharmacy could depend on differences in the working environment, which is another social component. At small pharmacies with few employees, individuals have to reflect more about their own roles and about how the work is structured in order to get it all together. Smaller groups that share the same values and face similar conditions are a good ground for learning.45 At a larger workplace, every individual has a narrower area of expertise and it is possible for a professional to maintain a more rigid position within the structure. Environmental factors seem to be associated with reflective capability; hence, reflection could be seen as a socially learned skill. The odds ratios for the smallest pharmacies were significant and high (OR=19.8), but this result is based on a few students who had an APPE at a small pharmacy (n= 28). The confidence interval (1.9 - 206.1) indicates that this is an uncertain result. Further, since students can influence their placement to some degree, the students who actively go for small pharmacies may differ somewhat from those who are attracted to larger pharmacies.
Reflection as an Individual Process or a Social Interaction
Taking part in the APPE gives experience that may lead to reflection. This reflection can be supported by different social and individual components as shown in this work, and may subsequently contribute to the development of professional skills.
During the APPE, students are exposed to different problems and situations in a professional workplace environment. They use prior theoretical knowledge and transform it into practical skills. This can be seen as an individual process where learning is performed by the individual student who develops his/her own skills. Schön and Mezirow's views of reflection are that it is a personal process that can be stimulated by external factors such as discussion with a preceptor. This is assumed in the model for learning presented in the introduction. On the other hand, the preceptor's influence also can be seen as an external, social component. Other social factors associated with reflection, and hence learning the ability to reflect, have to be put into their social contexts. The level of reflection achieved is connected to the size of the pharmacy and hence probably to social factors associated with smaller working groups, but this needs to be further investigated. The support from the social context seems to be of importance for novice practitioners' ability to reflect.21,32 The social setting is probably more important for students, since they are still learning the basics and lack the skills to identify and reflect on their own professional behavior. From a social context learning perspective, students learn by imitating behaviors of more experienced professionals at the workplace.46 Learning from more experienced professionals may support the students' reflection in developing a role as a professional practitioner.13,21
Limitations
In a cross-sectional study, the causality of the factors associated with reflection cannot be established. Furthermore the response rate was rather low, which is a concern since it may have led to selection bias. Students who were more interested in completing the questionnaires and the essay assignment might have been more reflective, and hence more willing to take part in the study. The small sample size, due to the low number of pharmacy students in Sweden, also means that the study may have been underpowered to detect all differences. It was not possible to include more students from the only other school of pharmacy in Sweden due to logistical reasons.
The method for measuring reflection is recently developed and only partly validated. However, the scheme for measuring reflection was reliable and was developed specifically for pharmacy students and professionals.8 The same critique could apply to the choice of using PILS to determine learning styles. PILS is relatively untested compared to Kolb's Learning Style Inventory (LSI)47 but it correlates well with LSI.22 PILS is also developed for pharmacy professionals, which motivated the choice of this particular test. The pattern for distribution of respondents on different learning styles has similarities to those in earlier studies of practicing pharmacists.48 In our study, students seemed to be more oriented towards structured learning styles than practitioners. This could be an outcome of students lacking the experience of making their own decisions in a professional environment and may stress their need for support. A larger sample size may have proven the link between reflection and learning styles, since this association had borderline significance.
Critical thinking was measured with the CTA. This instrument has been used in different research settings and for different categories of respondents.42 It was the most common measurement until the 1980s.49 The CTA had been validated for grading at a nursing course with good results.50 However, it might be too insensitive for measuring small changes in critical thinking among students, since it is developed for measuring critical thinking in a general population.51,52 Lately, the California Critical Thinking Skills Assessment (CCTSA) has been more frequently used in studies, especially in the United States.53 The reason for not choosing the CCTSA in this study was that it is not available in Swedish. However, CCTSA and CTA correlate well.53 The results reached by the students on the CTA included in this study are high, with a mean of 59.45 compared to other studies.41,54 The level of reflection is also high compared to other studies.17,55 In this combination, this could mask the supposed relationship between reflection and critical thinking due to ceiling effects.
CONCLUSIONS
In conclusion, this study shows that social components are of higher importance than individual components to students' level of reflective skills at the end of the APPE. The ability to reflect was independently associated with the student having a formal preceptor, the size of the APPE pharmacy, and the student's perceived importance of discussing critical thinking with the preceptor. Students who received most of their precepting from a formal preceptor had a 5-fold higher chance to be reflective at the end of the course. Students assigned to small pharmacies (1-10 employees) and students who perceived discussing critical thinking as being of high importance were more reflective. No correlation between reflection and individual characteristics, critical thinking, or learning styles could be observed.
The results emphasize the importance of training preceptors since a trained preceptor can foster reflective skills among students. The social interaction and how learning occurs at pharmacies, including the student-preceptor-pharmacy interaction, needs further investigation in order to optimize and develop new pedagogic methods for the pharmacy APPE program. Another area for research that needs to be explored further is how the ability to reflect can contribute to learning the pharmacy profession.
ACKNOWLEDGMENTS
This project was financed by grants from the educational board at the Faculty of Pharmacy, Uppsala University. The authors also acknowledge the contributions of Stina Hall, Karen Pieters, and Anders Hallzon, master's thesis students, to this study. Special thanks to the pharmacy students who completed study surveys and to the preceptors who facilitated data collection at the pharmacies.
REFERENCES
- 1.Lowe M, Rappolt S, Jaglal S, Macdonald G. The role of reflection in implementing learning from continuing education into practice. J Continuing Educ Health Prof. 2007;27(3):143–8. doi: 10.1002/chp.117. [DOI] [PubMed] [Google Scholar]
- 2.Mamede S, Schmidt HG. The structure of reflective practice in medicine. Med Educ. 2004;38(12):1302–8. doi: 10.1111/j.1365-2929.2004.01917.x. [DOI] [PubMed] [Google Scholar]
- 3.Maudsley G, Strivens J. Promoting professional knowledge, experimental learning and critical thinking for medical students. Med Educ. 2000;34(7):535–44. doi: 10.1046/j.1365-2923.2000.00632.x. [DOI] [PubMed] [Google Scholar]
- 4.Nicholl H, Higgins A. Reflection in preregistration nursing curricula. J Adv Nurs. 2004;46(6):578–85. doi: 10.1111/j.1365-2648.2004.03048.x. [DOI] [PubMed] [Google Scholar]
- 5.Plaza CM, Draugalis JR, Slack MK, Skrepnek GH, Sauer KA. Use of reflective portfolios in health sciences education. Am J Pharm Educ. 2007;71(2) doi: 10.5688/aj710234. Article 34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Accreditation Council for Pharmacy Education. Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree. The Accreditation Council for Pharmacy Education; 2006. http://www.acpe-accredit.org/pdf/ACPE_Revised_PharmD_Standards_Adopted_Jan152006.pdf Accessed May 12, 2009.
- 7. Pre-registration Training: Royal Pharmaceutical Society of Great Britain. http://www.rpsgb.org.uk/acareerinpharmacy/preregistrationtraining/ Accessed May 12, 2009.
- 8.Wallman A, Kettis-Lindblad Å Håll S, Lundmark A, Ring L. Evaluation of a Categorization Scheme for Assessing Pharmacy Internship Students' Level of Reflection. Am J Pharm Educ. 2008;72(1) doi: 10.5688/aj720105. Article 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Taylor BJ. Reflective Practice - A Guide for Nurses and Midwives. 2nd ed. Berkshire: McGraw-Hill Education; 2006. [Google Scholar]
- 10.Boud D, Knights S. Course design for reflective practice. In: Gould NG, Taylor I, editors. Reflective Learning for Social Work. Aldershot: Arena; 1996. [Google Scholar]
- 11.Droege M. The role of reflective practice in pharmacy. Educ Health (Abingdon). 2003;16(1):68–74. doi: 10.1080/1357628031000066778. [DOI] [PubMed] [Google Scholar]
- 12.Schön DA. Educating the Reflective Practitioner. San Francisco: Jossey-Bass; 1987. [Google Scholar]
- 13.Schön DA. The Reflective Practitioner: How Professionals Think in Action. Basic Books; 1983. [Google Scholar]
- 14.Mezirow J. Transformative Dimensions of Adult Learning. San Fransisco: Jossey-Bass; 1991. pp. 104–11. [Google Scholar]
- 15.Bourner T. Assessing reflective learning. Educ Training. 2003;45(5):267–72. [Google Scholar]
- 16. The role of the pharmacist in the health care system. Preparing the future pharmacist. Curricular development: report of a third WHO Consultative Group on the Role of the Pharmacist, Vancouver, Canada, August 27-29, 1997. World Health Organization, Vancouver; 1997. http://apps.who.int/medicinedocs/en/d/Js2214e/.
- 17.Kember D, Jones A, Loke A, et al. Determining the level of reflective thinking from students' written journals using a coding scheme based on the work of Mezirow. Int J Lifelong Educ. 1999;18(1):18–30. [Google Scholar]
- 18. Säljö R. Lärande i praktiken - ett Sociokulturellt Perspektiv. [Learning in practice – A socio-cultural perspective.] Stockholm: Prisma; 2000.
- 19. Ellström P-E. Kompetens, Utbildning och Lärande i Arbetslivet - Problem, Begrepp och Teoretiska Perspektiv. [Competence and learning in working life – Problems, concepts, and theoretical perspectives.] Stockholm: Norstedts Juridik AB, 2006.
- 20.Marton F. Towards a pedagogical theory of learning. In: Entwistle N, Tomlinsson P, editors. Student Learning and University Teaching. Leicester: BJEP Monograph Series II; 2007. pp. 19–30. [Google Scholar]
- 21.Eraut M. Early career learning at work and its implications for universities. In: Entwistle N, Tomlinsson P, editors. Student Learning and University Teaching. Leicester: BJEP Monograph Series II; 2007. pp. 113–133. [Google Scholar]
- 22.Austin Z. Development and Validation of the Pharmacists' Inventory of Learning Styles (PILS) Am J Pharm Educ. 2004;68(2) Article 37. [Google Scholar]
- 23.Bondestam F. Könsmedveten Pedagogik för Universitetslärare - en Introduktion och Bibliografi. [Gender-aware education for university teachers – An introduction and bibliography.] In: Jämställdhetsfunktionen [The gender equality function.] 2nd ed. Uppsala: UPI; 2003. [Google Scholar]
- 24.Paterson BL. Developing and maintaining reflection in clinical journals. Nurs Educ Today. 1995;(15):211–20. doi: 10.1016/s0260-6917(95)80108-1. [DOI] [PubMed] [Google Scholar]
- 25. Paton M. Reflective journals and critical thinking. Uniserve Science Assessment Symposium; 2006. Uniserve Science Assessment Symposium Proceedings. http://science.uniserve.edu.au/pubs/procs/2006/paton.pdf.
- 26. American Philosophical Association. Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. “The Delphi Report”: Committee on Pre-College Philosophy, 1990:ERIC Doc. No. ED 315 423.
- 27.Rolf B, Ekstedt E, Barnett R. Kvalitet och Kunskapsprocess i Högre Utbildning. [Quality and the knowledge process in higher education.) Nora: Nya Doxa, 1993. [Google Scholar]
- 28.Cassidy S. Learning styles: An overview of theories, models, and measures. Educ Psychol. 2004;24(4):419–44. [Google Scholar]
- 29.Cockerton T, Naz R, Sheppard S. Factorial validity and internal reliability of Honey and Mumford's learning styles questionnaire. Psychol Rep. 2002;91(2):503–19. doi: 10.2466/pr0.2002.91.2.503. [DOI] [PubMed] [Google Scholar]
- 30.Lizzio A, Stokes L, Wilson K. Approaches to learning in professional supervision: supervisee perceptions of processes and outcome. Stud Contin Educ. 2005;27(3):239–56. [Google Scholar]
- 31.Lauvås P, Handal G. Handledning och Praktisk Yrkesteori. [Guidance and practical professional theory.] Lund: Studentlitteratur;1993. [Google Scholar]
- 32. Eraut M. Learning during the first three years of postgraduate employment – the LINEA Project EARLI conference. Padua, Italy; 2003. http://www.sussex.ac.uk/education/documents/linea_publication_4.pdf.
- 33. Steadman S. The LiNEA Project. SRHE Conference; 2002; Glasgow, Scotland. http://www.sussex.ac.uk/education/documents/linea_publication_5.pdf.
- 34. Eraut M, Maillardet F, Miller C, Steadman S, Ali A, Blackman C, Furner J. Learning in the professional workplace: relationship between learning factors and contextual factors. AREA conference paper; 2004; San Diego. http://www.sussex.ac.uk/education/documents/linea_publication_8.pdf.
- 35.Ekman A. Lärande Organisationer i Teori och Praktik. [Learning organizations in theory and practice (thesis)]. Uppsala: Uppsala University, 2004. [Google Scholar]
- 36. Studieophold på apotek. [Pre-registration training at pharmacy.] Köpenhamn: Det Farmaceutiske Fakultet, Köpenhamns http://www.kursus.dfuni.dk/course/category.php?id=8 Accessed May 12, 2009.
- 37. Clinical placement reflective Learning Diary: Bath University. http://www.bath.ac.uk/pharmacy/ Accessed May 12, 2009.
- 38. Pre-registration training with pharmaceutical legislation Course description Uppsala: Department of Pharmacy, 2006. http://www.apoteksfarmaci.se Accessed September 7, 2009.
- 39.Katajavuori N, Lindblom-Ylänne S, Hirvonen J. The significance of practical training in linking theoretical studies with practice. Higher Educ. 2006;(51):439–64. [Google Scholar]
- 40. Ennis RH. An annoted list of critical thinking tests. University of Illinois; 2006. http://faculty.ed.uiuc.edu/rhennis/testlistrevised606.htm Accessed August 28, 2009.
- 41.Watson G, Glaser E. Critical Thinking Appraisal Manual. San Antonio: Psychological Corp; 1980. [Google Scholar]
- 42.Watson G, Glaser EM. Bedömning av Kritiskt Tänkande. [Evaluation of critical thinking.] Stockholm: Psykologiförlaget, 1999. [Google Scholar]
- 43.Petrie A, Sabin C. Medical Statistics at a Glance. 2nd ed. Oxford: Blackwell Publishing Ltd; 2005. [Google Scholar]
- 44.Eurat M. Developing Professional Knowledge and Competence. London: The Falmer Press; 1994. [Google Scholar]
- 45.Wenger E. Communities of Practice - Learning, Meaning, and Identity. Cambridge: Cambridge University Press; 1998. [Google Scholar]
- 46.Lave J, Wenger E. Situated Learning. Legitimate Peripheral Participation. Cambridge: University of Cambridge Press; 1991. [Google Scholar]
- 47.Kolb DA. The Learning Style Inventory: Technical Manual. Boston: McBeer & Co; 1976. [Google Scholar]
- 48.Austin Z. Learning styles of pharmacists: Impact on career decisions, practice patterns and teaching method preferences. Pharm Educ. 2004;4(1):13–22. [Google Scholar]
- 49.McMillan JH. Enhancing college students' critical thinking: A review of studies. Res Higher Educ. 1987;26(1):3–29. [Google Scholar]
- 50.Gadzella B. Psychometric properties of Watson-Glaser critical thinking appraisal for a sample of education majors. Psychol Rep. 2003;92(3 Pt 1):1249–54. doi: 10.2466/pr0.2003.92.3c.1249. [DOI] [PubMed] [Google Scholar]
- 51.Beling J. Impact of service learning on physical therapist students' knowledge of attitudes toward older adults and on their critical thinking ability. J Physical Therapy Educ. 2004;18(1):13–21. [Google Scholar]
- 52.Daly WM. The development of an alternative method in the assessment of critical thinking as an outcome of nursing education. J Adv Nurs. 2001;36(1):120–30. doi: 10.1046/j.1365-2648.2001.01949.x. [DOI] [PubMed] [Google Scholar]
- 53.Facione NC, Peter A. Critical Thinking Assessment in Nursing Education Programs: an Aggregate Data Analysis. Millbrae, CA: The California Academic Press; 1997. Facione. [Google Scholar]
- 54.Shin S, Ha J, Shin K, Davis MK. Critical thinking ability of associate, baccalaureate and RN-BSN senior students in Korea. Nurs Outlook. 2006;54(6):328–33. doi: 10.1016/j.outlook.2006.09.008. [DOI] [PubMed] [Google Scholar]
- 55.Kansanaho H, Cordina M, Puumalainen I, Airaksinen M. Practising pharmacists' patient counseling skills in the context of reflectivity. Pharm Educ. 2005;5(1):19–26. [Google Scholar]