A Categorization Scheme for Assessing Pharmacy Students' Levels of Reflection During Internships

Abstract

Objective

To test the reliability, feasibility, and responsiveness of a categorization scheme for assessing pharmacy students' levels of reflection during internships.

Methods

Pharmacy interns at Uppsala University were asked to write a reflective essay about patient counseling at the start and end of their internships. A modified version of Kember's categorization scheme for assessing the level of reflection was used to evaluate these essays.

Results

Based on their essay scores, the students' levels of reflection increased during the internship course (p < 0.001) The mean time for categorization was 3 minutes per essay. The interrater reliability of the 182 essays was κ = 0.63.

Conclusions

The evaluation of the categorization scheme showed that it has good interrater reliability, feasibility, and responsiveness. This scheme might be useful in pharmacy practice educational settings, but needs further validation.

INTRODUCTION

The pharmacist's professional role has changed and is still changing.¹ Today pharmacists have to function as healthcare professionals, as defined by Hepler and Strand in their definition of pharmaceutical care.² This role puts great demands on pharmacists as practitioners, and requires continuous reevaluation of their professional knowledge and behavior and an adaptation to each patient that they meet.^3,4

Pharmacists' confidence in their professional role is crucial because they cannot allow their decision making to be halted when faced with challenging or new situations and problems.¹ This confidence can be reached by continuously reflecting upon experiences, and by developing a greater understanding of the profession and its contribution to the healthcare system as a whole. As stated by Droege: “In order for pharmacists to partake in truly interdisciplinary healthcare teams and the profession of pharmacy to demonstrate its unique and indispensable contribution to quality healthcare, pharmacy curricula would teach towards reflective practice…”¹

This also requires a positive attitude towards new knowledge and a willingness to change. New procedures have to be developed within the profession to make this continuous learning process possible. Reflection is an essential skill in being able to reevaluate old knowledge, and develop and incorporate new knowledge into practice, in order to reach greater competence and confidence in practice.¹ Reflective skills should be introduced to pharmacy education since the goal of pharmacy education should be to give students both a good knowledge base for functioning as practitioners and also motivation and ability for continuous professional development (CPD) throughout their working lives. Introducing reflection into the education could be a way to achieve this.

The International Pharmaceutical Federation (FIP) and WHO state in the Good Pharmacy Practice (GPP) document⁵ that “Pharmacists in each practice setting should accept personal responsibility for maintaining and assessing their own competence throughout their professional working life.” In other statements by FIP and WHO, it is concluded that one of the roles of today's pharmacist is that of a lifelong learner.^6,7 They also state that personal assessment and having a personal plan for CPD needs is required.⁸ The revised US Accreditation Council for Pharmacy Education's (ACPE) accreditation standards and guidelines for the professional program leading to the doctor of pharmacy degree, Standards 2007, also reflect the increased emphasis on the student as a professional and lifelong learner.⁹ ACPE guideline 15.1 specifically demand the use of student reflective portfolios in pharmacy education, ie, a collection of evidence of personal and professional development through critical analysis and reflection.

There is an international movement towards lifelong learning in general and reflective portfolios specifically, that has to be effectively met by pharmacy schools and colleges. In their review, Plaza identifies factors that have to be considered to implement reflective portfolios.¹⁰ These include the importance of faculty member and student buy-in, well-functioning interaction between students and faculty members, and students' accurately synthesizing the portfolio content. Other challenges include the potential conflict between using portfolios both for learning and assessment, as well as reliability and validity concerns in assessing reflective portfolios.¹⁰

Reflective Theory – The Reflective Practitioner

The reflective practitioner, a term introduced by Donald Schön, embraces the idea that the world is constantly changing and that change is something positive.^11,12 Reflection can be a way to deal with changes and increase professional confidence and competence.¹¹ The professional practitioner has to continuously adapt to stay at the top of his/her area of expertise. Continuous learning is one of the defining characteristics of professional practice.¹¹ Practitioners usually know more than they can say, and some of this tacit knowledge can be brought to the surface by reflection and used to develop new knowledge. Tacit knowledge is part of the artistry of practice, which is also central to the theory of the reflective practitioner. A profession can be seen as artistry and science combined. As stated by Schön, “It is the entire process of reflection-in-action which is central to the “art” by which practitioners sometimes deal well with situations of uncertainty, instability, uniqueness and value conflict.” ^{11 (p50)} Mezirow goes a step further and combines reflection and other learning processes into an overall system for learning, stating that reflective action is essential to the learning process.¹³ He identifies different levels of reflection and sorts them into a taxonomy in which he explains learning behavior as a transformative process founded on reflection. A high level of reflection is essential for the perspective transformation that should be the outcome of learning.¹³

Measuring Reflection

Systematic reflection on experiences in practice has, during the past 10 years, come more and more into focus in many curricula in higher education,^7,9,10,14 including nursing education,^15-18 medical education,^19,20 and lately, pharmacy education as well.^1,9,10,21-24 Being able to assess reflective practice/learning is thus essential in order to provide feedback to students and to certify that students have reached learning goals with regard to reflective learning.¹⁰ However, reflective practice/learning is not easily measured and the complexity lies in that it is an intangible skill that has to be assessed indirectly. Furthermore, a reflective process is difficult to assess because ideally there is no planned learning outcome, but instead learning goals should evolve from the reflective process.¹⁴ Schön sees this as an ongoing process that the student and the coach need to reflect upon constantly.¹² He does not, however, suggest any systematic way to assess the reflective process.

Nonetheless, a range of measures that can be used to assess individual reflection processing has been developed. Reflective portfolios are commonly used and assessed as a whole.^9,10 Journal writing and essays have been used both to promote reflection among students and for assessing reflection.^{15-16,22,25-28} These journals and essays have then been assessed by different approaches. Examples of approaches used are taxonomies, critical incidents, portfolios, dialogues, and interviews.²⁹ Instruments using taxonomies to measure incremental levels of conceptual and reflective thought include the Levels of Reflectivity and the Framework for Reflective Pedagogical Thinking.²⁹ Sparks-Langer et al measured reflection within a taxonomy in their Framework for Reflective Pedagogical Thinking.¹⁵ The level of reflection was identified by the language and thinking used in interviews and written reports, and categorized into 7 predefined levels of reflection.

Two different works were identified as especially interesting for developing a pharmacy-specific measurement for assessing reflective skills. Kember developed a model for assessing students' levels of reflection using reflective journals.^25,26 Students were asked to write a short essay, which was graded according to the level of reflection achieved.²⁶ They used a 7-level scheme, based on Mezirow's 6 levels of reflection in which the level of reflection increases from top to bottom in the scheme, with more than one level at each step. Each level of reflection was defined by a brief descriptive text.²⁶

Kansanaho assessed the level of reflection among pharmacists attending a course in patient counseling.²² The analysis was based on free-form essays in which the pharmacists described their perceptions of themselves as patient counselors. The pharmacists' patient counseling skills were assessed and categorized according to the US Pharmacopoeia (USP) medication counseling stages. They then used a two-dimensional scheme where the pharmacists' levels of patient-counseling skills were linked to a corresponding level of reflection as defined by Mezirow. The linking was based on theoretical discussions; thus, reflectivity was measured in an indirect way. Practicing pharmacists had a very low level of reflection, and the authors concluded that there was a need for new methods to improve CPD within the pharmacy profession, as well as for new teaching methods to be able to introduce reflective practice in the undergraduate education.²²

Pharmacy Specific Measure for Level of Reflection

Reflective thinking is often stated as a goal in pharmacy curricula, but is seldom systematically assessed. In the United States, ACPE Standards 2007 demands assessment of reflective components in the education.⁹ Hence, there is a need for specific measures for assessing reflection in pharmacy settings. It is important that such measures are feasible, ie, that they are easy to use and interpret. Ideally, they should be easily applicable to regular course assignments and contribute to students' learning during their courses. Pharmacy-specific measures also need to use examples relating to pharmacy practice in defining different levels of reflection. Such measures could provide a possibility to measure students' development in the area of reflective thinking and be an assessment tool, among others, for assessing portfolios.

The objective of this study was to test the interrater reliability, feasibility, and responsiveness of a modified version of Kember's categorization scheme for assessing pharmacy students' reflective writing.

METHODS

The study population consisted of pharmacy internship students at the Faculty of Pharmacy, Uppsala University, Sweden, during the fall semester of 2005 (n = 83) and spring semester of 2006 (n = 63). The Swedish pharmacy education and the 6 month mandatory, pharmacy practice training internship course at community pharmacies (Table 1)

Table 1.

Outline of Swedish Pharmacy Education With Focus on the Internship

In order to develop a scheme for assessing the level of reflection among pharmacy interns, a pharmacy-specific assignment had to be developed first. These essays, resulting from the assignment, function as a basis for assessing reflective skills.

The outline of the assignment was inspired by the assignment used by Kansanaho.²² The students were asked to write short reflective essays (1 to 2 pages) about their personal views of patient counseling. They were also asked to fill out a brief questionnaire about how they perceived the assignment. The assignment was piloted on pharmacy internship students (n = 17) by the end of their internship in the spring of 2005. The assignment was revised, based on both the analysis of the essays and the comments given by students. The revision included providing some questions to the assignment, for the students to use as a starting point for writing the essay. The revised final assignment used in the study is shown in Table 2.

Table 2.

Instructions to Students on Writing Their Essays

The categorizing scheme developed by Kember²⁶ was the starting point for the categorizing scheme used in this study. Kember's scheme was translated into Swedish from English and tested by rating the essays from the pilot run (n = 17). After analyzing the results from the pilot, and after further theoretical discussions within the research group, a modified categorizing scheme was developed based on the 6 original Mezirow's levels of reflection,¹³ as opposed to the 7 categories used by Kember. In our categorizing scheme, each level builds upon the earlier one and every level of reflection is at one separate step in the scheme (Appendix 1). Of the 6 levels used, there are 3 non-reflective levels: habitual action, thoughtful action, and introspection; and 3 reflective levels: content reflection, process reflection, and premise reflection.

To better guide the raters in performing the categorization, each level of reflection in the categorizing scheme was exemplified by a pharmacy-specific example extracted from the essays from the pilot study. A majority of the examples chosen were of an emotional character, focusing on an angry customer, but other examples based on situations like pharmacotherapeutic dilemmas could have been used equally well.

The scheme analysis consists of 3 steps:

• (1) Identify parts of the text carrying meaning regarding reflection.

• (2) Categorize the quotes one by one into a specific level of reflection according to the categorizations scheme as described below.

• (3) Read through the entire essay in order to decide upon a final category that responds to the overall achieved level of reflection in the essay.

This overall categorization was given a score ranging from 1-6 and was guided by the highest level achieved. The 6 different levels of reflection are described in Appendix 1.

The essay assignment (Table 2) was sent out to all fall 2005 and spring 2006 pharmacy internship students at the beginning and end of the course (4 weeks before the examination). In fall 2005, the essay was not part of the curriculum; however, by spring 2006, it was included as a mandatory assignment within the internship course. All students were informed about the essays being a part of a research study about learning during the internship course and that it was voluntary to have their essays included in the study. The students were allowed to write the essay during working hours at the pharmacy and the estimated time given for completing it was 2 hours. All students were reassured that the outcome of the analysis of the essays would not influence the course grading. Furthermore, this was not possible since the rating was done after the students' graduation. The privacy of the respondents was protected during the analysis by the use of code numbers. Since this study was based on self-reported data from participants who had given informed consent, no ethical approval was necessary according to Swedish law.

The essays from fall semester 2005 and spring semester 2006 (n = 182) were used in the analysis of the students' levels of reflection. The developed categorization scheme was used to give a score based on the highest level of reflection found in each essay as described above.

The rating was carried out by 2 raters (2 of the authors): A.W. and S.H. Both raters had a good understanding of the definitions of reflection before starting the analysis. The 2 raters developed their categorization skills by practicing on the pilot material and further by discussing the categorization process and the outcome.

The essays were categorized and scored independently. These categorizations were then compared to calculate the interrater reliability. Furthermore, the final score for the level of reflection for each essay was determined by consensus.

The interrater reliability was calculated by Cohen's kappa.³⁰ Cohen's kappa is used to estimate the level of agreement between 2 raters, taking the agreement occurring by chance into account. It is calculated by inserting the raters' scores into a confusion matrix and calculating a kappa value (κ). The kappa value ranges from 0 to 1 and if there is perfect agreement κ would equal 1. A Cohen's kappa value between 0-0.4 is considered poor, 0.4-0.6 fair, 0.6-0.75 good, and >0.75 outstanding.³¹

A feasibility test of the overall scoring procedure was performed based on the spring 2006 essays (n = 126). The 2 raters measured the time necessary to categorize blocks of 5 essays. This assessment included the total time needed to assign an overall score for the 5 essays. No instructions were given to the raters on how to perform the categorization process, except to follow the developed categorization scheme as described above. To get an in-depth understanding of the feasibility of the categorization process, the raters were asked to describe their own categorization processes in detail and document any reflections they made regarding the feasibility of the method.

A responsiveness, or sensitivity test, is used to determine whether an instrument is able to detect a change regarding the parameter studied.³² The hypothesis for the test was that if students participate in a course with a reflective curriculum, their ability to reflect would increase. By comparing students' capacities at the beginning and the end of the course, the scores could be compared over time for each individual. If the score increased, the test had captured the expected increase in reflective ability. A paired t test was used to evaluate the change in reflective ability as measured by the categorizing scheme.

RESULTS

The data collection in the spring 2005 (n = 83 students) generated 56 essays: 30 essays written at the start (response rate = 36%) and 26 essays written at the end of the internship course (response rate = 32%). The data collection in spring 2006 (n = 63 students) generated 126 essays (63 essays at the start and 63 at the end of the semester) for a response rate of 100%. Eighty students wrote both essays (beginning and end of semester).

Of the respondents in fall 2005, 74% (n = 29) were female, and their mean age was 26.3 years (SD = 3.3 years). Corresponding demographics for spring 2006 were 79% (n = 50) female, with a mean age of 27.2 years (SD = 3.9 years).

The fall semester 2005 essays (n = 56) were analyzed regarding the level of reflection, and an interrater reliability of κ = 0.59 was reached. The disagreements in scoring showed no systematic differences between the raters. The essays from spring 2006 (n = 126) were analyzed next and the interrater reliability was slightly higher, κ = 0.65. In this second round of analysis, rater A.W. tended to categorize the essays as demonstrating a higher level of reflection than rater S.H. did. Both raters reported an improved confidence in categorizing the essays into the different levels in the second categorization round and by discussing the results and categorization, a greater understanding for the categorization scheme was developed. The total Cohen's kappa for all essays (n = 182) was κ = 0.63, a score that is regarded as good.³¹

All essays in spring 2006 were included (n = 126) in the feasibility test of the categorization scheme. The 2 raters used slightly different approaches to the categorization process. A.W. used a method of reading and categorizing sections and the whole essay in one step, and only controlling any insecure categorizations in a second reading. SH read all essays one time, categorized sections in a second reading, and the whole essay in a third reading, while controlling for possible errors. The mean time for categorizing one essay was 3 minutes. The times differed between the 2 raters. The mean time for A.W. was 1 min,11 sec and for S.H., 4 min, 50 sec. The time measured to categorize a set of 5 essays for judge A.W. was 5 min, 42 sec (range 4:21-8:06 minutes) and for rater S.H., 24 min, 06 sec (range 17:00-32:30 minutes). In their discussions, the 2 raters identified 2 main reasons for this difference in time used. They believed that it might be due to A.W.'s previous teaching experience of assessing written essays, which S.H. did not have, and A.W.'s faster reading speed.

All students who wrote both the start and the end essays in both semesters (n = 80, generating 160 essays) were included in the test for statistical difference in mean scores between the beginning and the end of the internship. None of the 160 essays were categorized either at the lowest or the highest level of reflection. An increase in the level of reflection scores was observed when essays written at the start of the semester were compared with essays written at the end of the semester by the same student (Figure 1).

Figure 1.

Change in level of reflection over time based on essays at the start and end of the internship (n = 160). (Level 1 is the lowest level of reflection and 6 the highest.)

The mean score at the start was 3.1 (SD = 0.9), and at the end 3.8 (SD = 0.9). The change was significant (p < 0.001). At the start of the internship, 25% of the students were categorized as reflective (level 4-6) and at the end 60 % were categorized as reflective (p < 0.114). Viewed proportionally, 58% increased their levels of reflection between the start and the end of the internship, 31% maintained the same level of reflection, and 11% decreased their level.

DISCUSSION

The evaluation of the categorization scheme showed that it has a good interrater reliability, feasibility, and responsiveness. The interrater reliability score indicates that the raters interpreted and applied the coding scheme consistently. According to the feasibility test, the time used for coding was reasonable. Finally, the responsiveness test indicated that changes in the level of reflection can be captured by this assessment method, and that students' levels of reflectivity increased over time during the internship course.

Measuring the level of reflection is difficult. Hence, the validation of the categorization scheme is a concern in this study. Did the scheme really measure reflection or was it the students' ability to express thoughts in written form that was assessed? Some students can probably be reflective in their work and have a reflective thinking process, but might not be able to formulate this in a short written essay. Some students may also have a resistance towards writing this kind of assignment regardless of their levels of reflection. This might be reflected in the poor response rate during the first semester when the assignment was not a part of the curricula. Only 20.5% (17 of 83 students) chose to write the essays at both the start and end of the course. This resistance could be a result of a long schooling period during which priority had not been given to personal reflection of knowledge, but rather to a repetition of facts.

Since the scheme was used as a research tool in this study, the feasibility test does not include the time spent giving feedback to students about their reflective skills and discussing their progress. When using essays as a part of the reflective curricula in routine education, the assessment should be accompanied by discussions with tutors and peers to make full use of the learning opportunity. This will add to the time taken by the grader on each paper.

Schon was among the first to describe and propose reflective thinking as the basis for learning in the professional setting. His theory is well-known and has been used in several different educational settings.^18,25,26 By choosing Schön's theory about the reflective practitioner as a theoretical framework, and combining it with Mezirow's way of describing different levels of reflection, the scheme used in this study was developed on a solid theoretical basis. The foundation for our coding scheme was a test already developed by Kember et al. They showed that their coding scheme, based on Mezirow's theory, is valuable when assessing students' levels of reflection.²⁶ Several studies have used reflective journals and showed that, by using predefined levels, different raters can reach the same result.^25,26,28 However, Wong et al experienced difficulties with reliability when grading using several different levels instead of only 2,²⁵ although these were resolved by Kember et al in later work.²⁶ Still, validity is an important concern. By using Kember et al's existing coding scheme and illustrating the different levels of reflection with examples from the written pilot essays, its validity for the pharmacy practice context has been strengthened. The examples we used were of an emotional character, but other examples could have been used equally well, for instance, by focusing on pharmacotherapeutic dilemmas. The main purpose of the examples was to better guide the raters.

Our study indicates that the modified scheme is a valid way to measure reflection, but further research is warranted to confirm this. This includes determining whether the rating generated by applying the scheme is associated with related skills such as critical thinking, and whether it is related to factors such as working experience and pharmacy working environment. Since there is a prevailing assumption about the relationship between reflective skills and professional outcomes in pharmacy practice (for instance, patient counseling skills), the scheme's predictive validity would also be important to determine. Further, external validity should be strengthened by repeated research at different schools of pharmacy in different counties.

The change in the interrater reliability score from 0.59 for the essays from fall 2005 to 0.65 for the spring 2006 essays indicates, but does not prove, that there is a training effect and that the accuracy of the rating increases with increased experience. However, both raters reported an improved confidence in the second categorization and we therefore strongly recommend pilot testing in order to have material to analyze and discuss before doing the actual rating. The interrater reliability was classified as good, and considering the difficulties in measuring a subjective outcome as the level of reflection, the result was very good.²⁶

The feasibility test showed that using this scheme would be a reasonably fast method of assessing the level of reflection in written essays, which is very important if this method is to be implemented in teaching settings. As mentioned earlier, the time measured was only for the grading procedure and is prone to increase if used in formative assessments.

There is also a difference in the time used by the 2 raters. The raters used slightly different processes' for grading, and according to the raters themselves, differences in teaching experience and reading speed contributed to the difference in time required. The previous experience of rating large amounts of essays and other examinations might have made the decision process faster for AW. However, these hypotheses have to be formally tested, and there might also be other factors that explain this difference.

In order to accurately determine reasons behind the differences in time used, more raters have to be used. Nevertheless, our study indicates that although the rating procedure and time used is prone to vary between raters, the interrater reliability is good.

The students' reflective skills increased during the internship. This is consistent with our hypothesis for the responsiveness test. Reflection would be expected to increase when students, for the first time in their education, apply theoretical knowledge in real situations, write reflective diaries, and have reflective discussions with their tutor.^17,33

The increase in level of reflection is probably not only due to the reflective curriculum during the internship course, but also due to several other factors. There might, for example, be a training effect among the students when completing their second essay, which possibly could contribute to the increased level of reflection. In this study the training effect was minimized by not providing any feedback to the students between the start and end essays. To conclusively discriminate between the instrument's responsiveness to true change in reflectivity and mere instrument sensitization, a controlled study would be needed.

The fact that none of the students reached the highest level of reflection and had a fairly low mean level of reflection is consistent with other studies.^18,22,25 We found that the highest level of reflection (level 6) is difficult to detect with written essays, since the respondents have to prove that the reflection has altered their way of approaching different problems and that this understanding has been internalized into their professional understanding. This is also discussed by Kember.²⁶ However, in the pilot study, one student was writing in such a manner that we could categorize it at the highest level.

Reflective essays and journals have also been used by earlier studies.^{14,16,22,25,27} We used a modified version of Kember's categorization scheme for assessing pharmacy interns' levels of reflection. This scheme is exemplified by situations from pharmacy practice and has 6 levels that build upon each other. The essay assignment used is also straightforward and easily integrated into the internship course. The topic of the assignment – communication and patient counseling – was intentionally selected because pharmacy students and pharmacists always have opinions about this subject. It is therefore usable both as a baseline measure, before any educational interventions, and as a final outcome. By using a scheme with well-defined levels of reflection, assessing an intangible skill such as reflection is possible. The progress of learning professionalism is often hard to assess and this scheme for categorization of written essays can be a valuable complement in helping both the student and the tutor in assessing and developing reflective skills. It can, however, not replace the professional eye of a trained pharmacy tutor and have to be carefully implemented. Although it can be used both as a formative and summative assessment, reflective writing may be affected by the assessment situation.¹⁰ In composing essays that are to be assessed, students may conform their writing to what they think is expected of them rather than giving free scope to their own, independent thinking.¹⁰ It is also important to ensure student buy-in.¹⁰ Efforts have to be made to explain the purpose of reflective writing and its function as a learning experience.

CONCLUSIONS

The scheme evaluated in this study seems to be a promising and feasible way to assess pharmacy students' reflective thinking during their internships. It might be useful as a routine assessment of reflective skills in pharmacy education settings. However, it needs further validation.

Appendix 1. The categorization scheme used for analysis of reflective essays. In the scheme, the categories build upon each other as the students reach higher levels. The level of reflective thinking increases from the bottom to the top. Levels 1 to 3 are non-reflective, while levels 4 to 6 are reflective

Non-reflection

1. Habitual action. Habitual action is a unconscious act that takes place without thought and can be performed at the same time as another act. A description of an act performed without thought or having to focus could be, for example, writing using a keyboard (for a skilled typist). A description of the course of events can be categorized as habitual action. For example: “At first, I received the prescription, then I registered it and, finally, I handed over the drug to the customer, while I gave him/her information.

2. Thoughtful action. Thoughtful action draws upon existing knowledge. The starting point lies in previously existing knowledge, and choices between different alternatives regarding how to perform the task are made either unconsciously or not at all. Why a certain choice is made is not questioned and no interpretation is made. No thought is given to the consequences of the act except according to the previously learned action. An example of this is a description of communication with a customer that corresponds totally to existing theoretical knowledge without evaluation of different options. “If a customer comes into the store angry, it is important not to get angry yourself”.

3. Introspection. Introspection refers to thoughts about oneself, one's own thoughts or feelings about performing a task. There is no comparison between the actual task and/or one's previous experiences, nor are there any thoughts as to why these feelings occur or what they might lead to. An example of this is a description of how it feels to learn something, or how the student feels in a counseling situation. “An angry customer came into the pharmacy, and that felt terrible.”

Reflection. The definition of reflection as it is used below, is that a situation is identified in relation to an actual experience. This problem must somehow be analyzed in order for the task to be executable. Previous knowledge is used in the specific situation and is questioned and criticized when necessary.

4. Content reflection. Content reflection pertains to what one perceives, thinks, or feels, or how one acts when doing a task. There should be a questioning or an interpretation of behavior in order to be categorized as reflection, otherwise it is most often categorized as “2. Thoughtful action”. Content reflection, on the other hand, is based on a person's previous knowledge or a previous experience and the person consciously thinks of what he/she does in order to solve the actual problem. They do not, however, reflect upon why the action taken works or how their own behavior developed. What effect the thought, feeling, or act may have should be discussed. For example, “When I meet an angry customer, I smile to get a positive reaction in return. It's usually easier that way”.

5. Process reflection. Process reflection refers to how one performs the functions of perceiving, thinking, feeling, or acting, and to an assessment of how effective the performance is. There should be a proposal for, or an interpretation of, behavior for a categorization as process reflection. For example, a person smiles to solve the problem, but also thinks further on how he or she thinks it might work out. They also consider how a kind reception from another person can reduce their own irritation. Reflection of process can also contain reflection of how they feel and act themselves when they meet the angry customer, and how this is considered as a problem, as well as how they handle their own feelings. One's thoughts and beliefs about how the thought, feeling, or act has an effect should be discussed in addition to how others apprehend the act. For example, “When an angry customer enters the pharmacy, I often feel that I easily become irritated myself. I know that this won't improve the situation, so I try to answer with a smile to calm the customer. Most often the customer is not annoyed with me, rather it is the waiting time or something completely different. To answer with a smile is often nicer and I don't gain anything from getting annoyed myself.”

6. Premise reflection (Theoretical reflection). Premise reflection relates to why one apprehends, thinks, feels, or acts the way one does and the consequences of that existing knowledge sets the framework for how one acts in different situations. This should include an analysis of the whole situation/problem; “what” and “how” should be put into context. Consequences should be considered so that they can be included in a deeper understanding or reinterpretation of the problem. Alternative methods should also be considered, often leading to questioning of prejudice based on a theoretical reasoning. This could lead to a reinterpretation of the situation so that the starting point is different the next time the same kind of problem occurs, and thus the action becomes different. This can be very hard to identify in written essays; the behavior must be controlled the next time it happens. For example, “When an angry customer enters the pharmacy, I often feel that I easily become irritated myself. I know that the situation won't be improved by this, so I try to answer with a smile to calm the customer. Most often it's not me he or she is annoyed with, rather it is the waiting time or something completely different. To answer with a smile is often nicer and I don't gain anything from getting annoyed myself. I have tried different alternatives and when I get annoyed it's better to get help from someone else. This takes time, though. However, I have to make sure not to take this personally and let it affect other things that I do and so on…”.