Abstract
Objectives
To determine change in cultural competency knowledge and perceived confidence of second-year pharmacy students to deliver culturally competent care after completing a required cultural competency curriculum.
Design
Cultural competence material was covered in the second-year PharmD curriculum through lectures, laboratories, and an experiential/out-of-class assignment.
Assessment
Eighty-five second-year (P2) pharmacy students completed a survey which assessed influence of classroom activities related to cultural competence. Mean values for knowledge and perceived confidence were significantly higher for posttest compared to pretest (p < 0.01), after cultural competency activities. Focus groups were used to solicit students' opinions on instructional effectiveness, relevance of activities, and areas for enhancement.
Conclusion
The cultural competency curriculum increased pharmacy students' awareness of and confidence in addressing cultural diversity issues that affect pharmaceutical care delivery.
Keywords: culture, cultural competence, diversity, assessment
INTRODUCTION
The United States population is becoming more diverse, with significant increases in numbers and proportions of people representing various ethnic groups.1 The US Census Bureau's national population projections predict that by the year 2050, less than 53% of the US population will be non-Hispanic white; 16% will be black; 23% of Hispanic origin; 10% Asian and Pacific Islander; and about 1% American Indian.1 Clearly, future pharmacists will have to interact with patients from diverse backgrounds who have different languages and cultures, as well as different opinions, beliefs, behaviors, and practices about healthcare and illness. These differences directly impact the patient-provider communication process2 and may increase healthcare disparities.3 Helping pharmacists become culturally competent is a cited approach to eliminate long-standing health disparities.4-6
Culture and Cultural Competence
Culture is the first and most important frame of reference from which one's identity evolves.7 Culture refers to integrated patterns of human behaviors that include the language, thoughts, communications, actions, customs, beliefs, values, and institutions of racial, ethnic, religious, or social groups.5 Culture can be defined as a mix of beliefs, values, and behavior that are shared within a group of people.8 Patients from diverse cultural background may bring to the healthcare setting various preformed perspectives, beliefs, and behaviors regarding health and well-being. These preconceived beliefs about health and the healthcare system may affect the patient-provider communication.
Cultural competence is an important tool for dealing with culturally diverse patients, and is a key measure of quality of care for patients from diverse cultural backgrounds.9 There are many accepted definitions of cultural competence in the literature.10,11 The US Department of Health and Human Services defines cultural competence as the level of knowledge-based skills required to provide effective clinical care to patients from a particular ethnic or racial group.10 All accepted definitions of cultural competence highlight the importance of having knowledge and understanding of cultures to interact with patients of different ethnic groups. Cultural competence education in pharmacy curricula can increase cultural competence knowledge of pharmacy students and build their confidence in serving the diverse patients.
Including information about culture and its impact on healthcare competence in the curricula not only increased the cultural competence of nursing and medical students but also improved their understanding of the role of culture and its impact on healthcare perceptions.12-16 Pharmacy schools have started incorporating cultural competence training in their curricula17; however, there are few studies to date assessing the cultural competence of pharmacy students.
Assemi et al found that an elective course in cultural competence that used didactic instruction raised pharmacy students' awareness of diversity and provided them with basic knowledge and skills pertaining to cultural competence.18 Mansi et al used a role-play exercise to determine the influence of an educational intervention on pharmacy students' cultural disposition and perceived communication skills.19 Evans20 and Westberg et al21 also evaluated an elective course on cultural competence for pharmacy students. Both studies20,21 used similar assessment techniques such as in-class assignments, small group discussions, and guest speakers. Students showed a greater understanding of the cultural issues facing pharmacists, they gained an appreciation for the importance of acknowledging cultural differences, and an understanding of how to overcome some of the cultural barriers.
These studies used quantitative methodologies or experimental methods rather than qualitative research techniques, such as focus group interviews, to determine students' opinions about and experiences with the cultural competence curricula. Focus groups allow participants to freely express their opinions and ideas about a topic.22 This direct student feedback is needed to assess the design and delivery of the cultural competence curriculum.
This paper describes a study designed to assess the change in cultural competence knowledge of second-year (P2) pharmacy students following completion of the cultural competence modules that are a part of the PharmD curricula at the University of Toledo. The study also determined whether there was any change in the students' perceived confidence in their ability to serve patients from diverse backgrounds. This quantitative approach was followed by focus group interviews, asking students their opinions about the information they received in the cultural competence module and how it could be improved for future pharmacy students.
DESIGN
Cultural competence material that met the guidelines of the Accreditation Council for Pharmacy Education (ACPE)17 was presented to students in the second-year PharmD curriculum through lectures, laboratories, and an experiential out-of-class assignment (Table 1). Three videos related to cultural competence issues faced by pharmacists were shown in a classroom setting to students during their second year (course: PHPR 4400), each followed by class discussion (Table 2). Students were given an out-of-class assignment, the Human Differences paper, which was designed to expand students' awareness of peoples' differences by participating in an intentional experience in diversity. For this assignment, diversity was defined broadly and included, but not limited to, racial, ethnic, religious, or ability-based differences. Visiting homeless shelters; conducting personal interviews of patients of diverse cultural backgrounds; and attending a program, function, or organizational meeting of a group unfamiliar to the student were some of the activities that students completed.
Table 1.
Table 2.
a The Cross Cultural Health Care Program (CCHCP). Communicating Effectively Through An Interpreter. Seattle, WA; 1998.
b Pfizer Inc. The Power of Words: What You Say Can Make A Difference. Fairfax Station, VA; 2001.
The human differences paper was prepared in 2 parts. The first segment involved completion of an “intention form” with the following information: (1) the group or individual selected by the student to visit, (2) major questions the student had about the individual/group, and (3) the process by which the student planned to gather the information. The second part of the project involved writing a paper that contained the following information: (1) background information on the selected person/group; (2) major issues faced by the person/group; (3) a description of the human differences activity completed; (4) information gathered/questions answered by participating in the activity; and, (5) a reflection on what a pharmacist should know about the person/group to provide more effective care. This paper was an individually developed and executed project and accounted for a significant portion of the class grade.
ASSESSMENT AND EVALUATION
Pretest/Posttest Study Design
The study population consisted of 105, second-year PharmD students enrolled at The University of Toledo College of Pharmacy in the fall and spring semesters. This study population was selected on the criteria that the cultural competence material is taught in several courses throughout the second-year curricula at this institution (Table 1). Students who attended class on the days the questionnaires were administered were the participants of the study.
A 32-item questionnaire was designed to measure changes in students' cultural competence knowledge and to assess their self-reported confidence in their ability to provide culturally competent care before and after exposure to the cultural competence classroom modules. Survey items were adapted from the literature available on cultural competence in pharmacy curricula.18-21,23 Content validity for the questionnaire was provided by pharmacy faculty members experienced in the field of cultural competence.
The 8 questions in section 1 of the questionnaire assessed the pharmacy student's cultural competence knowledge. The 6 questions in section 2 rated the student's ability to understand and effectively use patient counseling models/methods for eliciting patients' illness experiences. The 15 questions in section 3 rated the student's confidence in ability to serve patients of diverse populations. The final questions elicited demographic data from the respondent. The questionnaire items were designed to cover the objectives of the course syllabi and the information provided in the courses (Table 1).
Published studies assessing cultural competence constructs in pharmacy students have used traditional numeric scales in questionnaires/survey instruments. However, cultural competence is not an endpoint, it is a continuum. Thus, it is difficult to describe numerically an individual's level of cultural competence. Response options for this study included a visual analogue scale, which is a commonly accepted technique to measure pain intensity.24-27 The scale is based on the theory that pain intensity is continuous, without jumps or intervals, and therefore categorical scales cannot adequately reflect the variations in the intensity of pain.27 The researchers thought the visual analogue scale would be a better measure of the students' level of cultural competence. The anchors for the scale used in this study were, “no understanding” and “complete understanding” for items 1-14 and “no confidence” and “complete confidence” for items 15-29. Students were instructed to mark on the line the point that they felt best represented their response to that particular item. The student's rating or score for each item was determined by measuring from the left end of the line to the point of the student's mark.
The pretest and posttest questionnaires were administered in the Human Interaction in Healthcare course at the beginning of the fall semester and in the Pharmaceutical Marketing and Management at the end of the spring semester. The research was approved by the University of Toledo Social, Behavioral & Educational Institutional Review Board. Participation in the study was voluntary and students were informed that their decision to participate or their responses would not affect their grades in the course. Each survey instrument was assigned a unique identifier so students' pretest and posttest could be matched. This identifier was coded and known only to the researcher. The questions did not contain any personal or sensitive information unrelated to the material covered in the respective classes. The confidentiality and anonymity of the students were maintained.
Data Analysis
Responses to survey items were measured using a scale and a number was entered next to the item. Data from the surveys were analyzed using SPSS, version 16.0 (SPSS Corporation, Chicago, IL). The significance level for the analysis was set at p < 0.01.32 Cronbach's alpha values were calculated for the entire questionnaire and individually on the first 3 sections of the questionnaire to determine its reliability. A paired t test was used to assess the change from mean pretest and posttest scores for the items asked in the questionnaire.
One hundred five students were enrolled in the classes in which the questionnaires were administered. However, due to absences and the voluntary nature of participation, only 85 paired responses were received (81% response rate). Demographics of the participants are presented in Table 3. The Cronbach's α estimate of internal consistency was 0.90 for the pretest scale and 0.94 for the posttest scale. The mean inter-item correlations were 0.25 and 0.41 for the pretest and posttest questionnaire items, respectively, suggesting sufficient covariation to warrant formation of scales.33
Table 3.
Students' scores on all items significantly increased on the posttest (Tables 4 and 5). The mean values of posttest responses for the knowledge section of the questionnaire were significantly higher than those for the pretest on all 14 items (P <0.01) (Table 4). The mean scores for items in the knowledge section of the questionnaire ranged from 0.5 to 3.0 in the pretest and from 3.0 to 4.0 in the posttest. Students showed the most improvement in their perceived understanding and ability to effectively use the patient counseling models/methods for eliciting patients' experiences of illness. The highest change in mean scores in items in the knowledge section was in students' understanding and ability to effectively use the RESPECT Model (Table 4). There was no significant change in students' perceptions regarding their understanding of the terms culture, race, and ethnicity.
Table 4.
aThe scale used was a 5-inch Visual Analog Scale (VAS) ranging from “no understanding” to “complete understanding,” ie, a marking at 0 inches denoted “no understanding,” whereas a marking at 5 inches denoted “complete understanding.”
Table 5.
aThe scale used was a 5-inch Visual Analog Scale (VAS) ranging from “no confidence” to “complete confidence,” ie, a marking at 0 inches denoted “no confidence,” whereas a marking at 5 inches denoted “complete confidence.”
The information covered in the cultural competence modules had a positive influence on students' perceived confidence to serve their patients of diverse backgrounds. Similar to the findings from the knowledge section of the questionnaire, the mean values of posttest responses for the confidence section were significantly higher than those for the pretest on all 15 items (P < 0.01) (Table 5). The mean scores for items in the confidence section of the questionnaire ranged from 1.5 to 2.9 on the pretest and from 2.9 to 3.5 on the posttest. The highest change in mean scores in items in the confidence section was in students' perceived confidence in obtaining the resources of an interpreter when needed (Table 5). The lowest mean score change in the confidence section was for the item that measured students' perceived confidence in developing rapport with all patients, regardless of their cultural background (Table 5).
Focus Group Interviews
The researchers/faculty members referred to the literature on qualitative assessment technique to develop/establish the protocol for the focus group interviews.28-31 At the end of the same class period in which the posttest questionnaires were administered, the instructor asked for students to volunteer to participate in the focus group. Ten students volunteered and the format of the interviews was explained verbally to them.
Focus group interviews were conducted by a moderator and an observer who were not associated with the college of pharmacy and had experience in conducting focus group interviews. The significance of the study and the purpose of the student interviews were explained to the facilitators and they were given the interview questions and transcript. Thirteen questions were developed for the interviews (Table 6). During the interview students did not identify themselves. The roles of the moderator and observer, and the purpose of the interviews were explained to the students prior to starting the interviews. The participants signed a consent letter acknowledging their voluntary participation in the interview. The interviews were conducted solely by the moderator. The role of the observer was to observe and note any nonverbal communication that took place during the interview. For example, a student's body language or level of enthusiasm when answering a particular question. All interviews were audiotaped using a voice recorder. Refreshments were provided and a $10 gift card was given to each of the 10 students who participated in the interview.
Table 6.
Qualitative Findings
The purpose of the focus group interview was to identify students' opinions about the cultural competency information covered in their curricula and changes that may be needed to enhance instruction for future pharmacy students. The 45-minute focus group interview was conducted in a conference room towards the end of the spring semester. The focus group consisted of 4 males and 6 females and the majority of the participants (8 of 10) were Caucasian. Of those participating in the focus group, 3 students were fluent in a language other than English.
Students agreed that through the information covered in the curriculum they gained an understanding of how to approach patients from diverse cultures. The information covered in the curricula helped them learn about patients from different backgrounds. The information also helped them develop a respect for the health beliefs of patients from diverse cultures. Students also mentioned that the cultural competency modules made them aware of the impact of patients' cultural beliefs on healthcare decisions.
The students felt it would be useful to have greater instruction on herbal medications so they could better address questions from patients who use alternative or traditional medicines. The students also felt that interactive sessions, where people from different cultural groups could meet with the students to share their healthcare experiences, would be a good addition to the curriculum. Students recommended that at least 1 of their pharmacy practice experiences be at a culturally diverse location, example a pharmacy in Chinatown, New York, NY. More clinical exposure and guest speakers were some of the other suggestions students made for enhancing the cultural competency curriculum. As noted by the observer, students' nonverbal communication during the interview included:
- Initial hesitation to speak out in the discussion (several participants);
- Quizzical looks when asked of any past experiences with cultural competence (several participants);
- Reluctance to reveal that too much cultural competency information was included in the curriculum (one participant);
- Smiled but did not respond when asked whether prepared for assisting patients of diverse cultural backgrounds (one participant);
- Relaxed demeanor when responding to some questions, while serious demeanor when responding to other questions (several participants).
DISCUSSION
With a rapidly changing and ethnically diverse population in the United States, pharmacists should be aware of how culture and cultural beliefs can influence patients' perceptions of and decisions about healthcare. The objectives of this study were to assess the change in cultural competence knowledge and perceived confidence of pharmacy students to serve patients of diverse backgrounds, following didactic and experiential modules provided in the PharmD curricula.
It is difficult to compare specific sections of the results of this study to the previous literature since cultural competence modules in this study were covered through a series of required courses and multiple pedagogies were used. Nonetheless, the results of this study were consistent with the results reported from the cultural competency studies performed at pharmacy programs at the University of California San Francisco,18 the University of Mississippi,19 and the University of Minnesota, Duluth.21 Specifically, the results showed a significant increase in students' perceived cultural competence knowledge and perceived confidence from fall to spring semester.
Students showed the most growth in their understanding and ability to effectively use patient counseling models/methods for eliciting a patient's experiences of illness. Models in cultural competence not only help the healthcare provider identify the health beliefs of patients with different cultural backgrounds, but also provide a base for practitioners to identify problems and create solutions to improving the effectiveness of health care delivery. Developers of instructional modules covering information on cultural competency might find it useful to include these models in their classes. Some parts of the knowledge section of this study did not show much improvement from the pre- and posttesting. This muted improvement may be attributed to the fact that students had scored high initially for some of these items.
Students' focus group responses confirmed and supported the findings from the quantitative part of the study. Students enjoyed and appreciated the cultural competency information provided to them in their classes. They felt that the information covered made them aware of the importance of the topic of culture in their curricula. After exposure to the information, they felt more confident and prepared to talk to patients from different cultures. When asked for suggestions on enhancing the cultural competency information provided to them, participants in the focus group interview thought that an elective in history of different cultures would be beneficial. Some of the suggestions provided by students, for example, inviting guest speakers from diverse ethnic background, are being considered for inclusion in the curriculum. Because there is no standardized training and assessment of cultural competency information provided in pharmacy schools, some of the suggestions provided by students may prove useful in designing future cultural competency courses.
Results of this study should be interpreted with the following limitations in mind. The study was performed at a single institution with a certain set of cultural competence activities. Also, the study sample was not ethnically diverse (86% of the students who completed the survey were white and 8 out of 10 focus group participants were white). Given these limitations, generalizing the results to other pharmacy schools, student populations, or curriculums would be not be appropriate. The study was designed to assess only 1 component of cultural competence, ie, self-perception of knowledge. It did not cover skills and attitude components of cultural competence. Since there was a 6-month gap between the pretest and the posttest, other influences, including professional experiences, may have affected the students' responses. There was no possible way to control the events which may have occurred during this time period and influenced students' responses on the posttest. The pretest may have sensitized the students to the type of information that would be asked on the posttest. However, the 6-month period between the tests minimizes the likely impact. The use of a visual analogue scale for survey responses is not common in social behavioral research. Students are usually more comfortable providing their responses on a Likert-type scale. Drawing a straight line to represent the subjective information they received may have been difficult for the students. Especially considering the fact that to avoid bias in their responses they were not shown their pretest markings.
Cultural competency is not an endpoint, it is a continuum. With increasing diversity in the US population, pharmacy schools must address the need to provide cultural competency information to pharmacy students to ensure provision of quality care. To work effectively with patients from diverse backgrounds, a basic knowledge about the influence of different cultures in healthcare seems mandatory for pharmacy students. Future research can be done to assess the change in the different dimensions of cultural competence such as knowledge, confidence, skills, and attitudes by following a class of pharmacy students from their first year in pharmacy school until graduation.
The cultural competence information covered in the pharmacy curricula not only increased students' knowledge about the impact of culture in healthcare decisions but also, increased students' perceived confidence in their ability to serve patients from diverse backgrounds.
ACKNOWLEDGEMENTS
We would like to acknowledge the student participants in this study. A special thanks to Mr. Seth Cohen- Powless and Ms. Smitha Ramani for their help in conducting the focus group interview. We would like to thank Mr. Robert Bechtol, MS, and Ms. Zaina Qureshi, PhD, for their assistance in the data collection process. We also would like to acknowledge Ms. Chamika Hawkins-Taylor, MHA, and Jon Schommer, PhD.
REFERENCES
- 1. US Census Bureau. Population Profile of the United States. National Population Projections. http://www.census.gov/population/www/pop-profile/natproj.html Accessed on July 21, 2010.
- 2.Schouten BC, Meeuwesen L, Tromp F, Harmsen HA. Cultural diversity in patient participation: The influence of patients' characteristics and doctors' communicative behavior. Pat Educ Couns. 2007;67(2):214–223. doi: 10.1016/j.pec.2007.03.018. [DOI] [PubMed] [Google Scholar]
- 3.Pesquera M, Yoder L, Lynk M. Improving cross-cultural awareness and skills to reduce health disparities in cancer. Med Surg Nurs. 2008;17(2):114–120. [PubMed] [Google Scholar]
- 4.O'Connell MB, Rickles NM, Sias JJ, Korner EJ. Cultural competency in health care and its implications for pharmacy. Pharmacotherapy. 2007;27(7):1062–1079. doi: 10.1592/phco.27.7.1062. [DOI] [PubMed] [Google Scholar]
- 5.Bazaldua O, Sias J. Cultural competence: a pharmacy perspective. J Pharm Pract. 2004;17(3):160–166. [Google Scholar]
- 6.Horner R, Slazar W, Geiger J, et al. Changing healthcare professionals' behaviors to eliminate disparities in healthcare: What do we know? How might we proceed? Am J Manag Care. 2004;10:SP12–SP19. [PubMed] [Google Scholar]
- 7.Lipson J, Dibble S, Minarik P. Cultural & Nursing Care: A Pocket Guide. San Francisco, CA: UCSF Nursing Press; 2002. [Google Scholar]
- 8.Guidon M, Sobhany M. Toward cultural competency in diagnosis. Int J Adv Couns. 2001;23(2):269–282. [Google Scholar]
- 9.Brown C, Nicholas-English G. Dealing with patient diversity in pharmacy practice. Drug Topics. 1999;143(17):61–68. [Google Scholar]
- 10. Georgetown University Center for Child and Human Development. National Center for Cultural Competence Curricular Enhancement Module Series. Definitions of Cultural Competence. Available at URL http://www.nccccurricula.info/culturalcompetence.html Accessed on July 21, 2010.
- 11. The Henry J. Kaiser Family Foundation. Compendium of Cultural Competence Initiatives in Health Care. 2003. http://www.kff.org/uninsured/loader.cfm?url=/commonspot/security/getfile.cfm&PageID=14365 Accessed on July 21, 2010.
- 12.Sargent SE, Sedlak CA, Martsolf DS. Cultural competence among nursing students and faculty. Nurse Educ Today. 2005;25(3):214–221. doi: 10.1016/j.nedt.2004.12.005. [DOI] [PubMed] [Google Scholar]
- 13.Benkert R, Tanner C, Guthrie B, Oakley D, Pohl JM. Cultural competence of nurse practitioner students: a consortium's experience. J Nurs Educ. 2005;44(5):225–233. doi: 10.3928/01484834-20050501-05. [DOI] [PubMed] [Google Scholar]
- 14.Lim RF, Wegelin J, Lisa L, et al. Evaluating a lecture on cultural competence in the medical school preclinical curricula. Acad Psych. 2008;32(4):327–331. doi: 10.1176/appi.ap.32.4.327. [DOI] [PubMed] [Google Scholar]
- 15.Crandall S, George G, Marion G, Davis S. Applying theory to the design of cultural competency training for medical students: a case study. Acad Med. 2003;78(6):588–594. doi: 10.1097/00001888-200306000-00007. [DOI] [PubMed] [Google Scholar]
- 16.Bussey-Jones J, Genao I, George DM, Corbie-Smith G. Knowledge of cultural competence among third-year medical students. J Natl Med Assoc. 2005;97(9):1272–1276. [PMC free article] [PubMed] [Google Scholar]
- 17. Accreditation Council for Pharmacy Education (ACPE). ACPE Guidelines. http://www.acpe-accredit.org/pdf/ACPE_Revised_PharmD_Standards_Adopted_Jan152006.pdf Accessed on July 21, 2010.
- 18.Assemi M, Cullander C, Hudmon K. Implementation and evaluation of cultural competency training for pharmacy students. Ann Pharmacother. 2004;38(5):781–785. doi: 10.1345/aph.1D402. [DOI] [PubMed] [Google Scholar]
- 19.Mansi B. Shah, Sean K, Patel Amit. Intercultural disposition and communication competence of future pharmacists. Am J Pharm Educ. 2004;68(5) Article 111. [Google Scholar]
- 20.Evans E. An elective course in cultural competence for healthcare professionals. Am J Pharm Educ. 2006;70(3) doi: 10.5688/aj700355. Article 55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Westberg S, Melissa A, Lind P. Enhancing cultural competency in a college of pharmacy curricula. Am J Pharm Educ. 2005;69(5):1–9. [Google Scholar]
- 22.Neuman L. Social Research Methods: Qualitative and Quantitative Approaches. 6th ed. Boston, MA: Pearson Education Inc; 2006. 412. [Google Scholar]
- 23.Guo G. Assessing Pharmacists' Cultural Competence Across Cultural Barriers [masters thesis] Toledo, OH: The University of Toledo; 2005. [Google Scholar]
- 24.Wewers ME, Lowe NK. A critical review of visual analogue scales in the measurement of clinical phenomena. Res Nurs Health. 1990;13:227–236. doi: 10.1002/nur.4770130405. [DOI] [PubMed] [Google Scholar]
- 25.Gallagher EJ, Bijur PE, Latimer C, Silver W. Reliability and validity of a visual analog scale for acute abdominal pain in the ED. Am J Emerg Med. 2002;20(4):287–290. doi: 10.1053/ajem.2002.33778. [DOI] [PubMed] [Google Scholar]
- 26.McCarthy M, Chang C, Pickard S, et al. Visual analog scale for assessing surgical pain. J Am Coll Surg. 2005;201(2):245–252. doi: 10.1016/j.jamcollsurg.2005.03.034. [DOI] [PubMed] [Google Scholar]
- 27.Larroy C. Comparing visual-analog and numeric scales for assessing menstrual pain. Behav Med. 2002;27(4):179–181. doi: 10.1080/08964280209596043. [DOI] [PubMed] [Google Scholar]
- 28. Bruce B. Qualitative Research Methods for Social Sciences. 2nd ed. Needham Heights, MA: Allyn and Bacon.
- 29.Vaughn S, Schumm J, Sinagub J. Focus Group Interviews in Education and Psychology. Thousand Oaks, CA: Sage Publications, Inc; 1996. [Google Scholar]
- 30.Diamond W, Gagnon J. Obtaining pharmacy class feedback through the use of focus group interviews. Am J Pharm Educ. 1985;49(1):49–54. [Google Scholar]
- 31. Krueger R, Casey M. Focus Groups. 3rd ed. Thousand Oaks, CA: Sage Publications, Inc.
- 32.Olejnik SF. Planning educational research: determining the necessary sample size. J Exp Educ. 1984;53(1):40–48. [Google Scholar]
- 33.Factor Pett MA, Lackey NR, Sullivan JJ. 1st ed. Thousand Oaks, CA: Sage Publications, Inc; 2003. Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. [Google Scholar]