Abstract
Interest in nephrology among trainees is waning in the US. Early perceptions and attitudes to subject matter can be linked to the quality of pre-clinical curricula. We wanted to explore these attitudes in the setting of modern curriculum redesign. We utilized Q methodology to understand first-year medical student attitudes after an innovative kidney physiology curriculum redesign that focuses on blending multiple learning methods. First-year medical students were invited to take a Q sort survey at the conclusion of a kidney physiology course. Students prioritized statements related to their understanding of kidney physiology, learning preferences, preferred course characteristics, perceived clinical relevance of kidney physiology, and interest in nephrology as a career. Factor analysis was performed to identify different student viewpoints. At the conclusion of our modified course, all students (n = 108) were invited to take the survey and 44 (41%) Q sorts were returned. Two dominant viewpoints were defined according to interest in nephrology. The Potentials are students who understand kidney physiology, perceive kidney physiology as clinically relevant, attend class sessions, utilize videos, and are willing to shadow a nephrologist. The Uninterested are students who are less satisfied with their kidney physiology knowledge, prefer to study alone with a textbook, avoid lectures, and are not interested in learning about nephrology. In an updated renal physiology course, students that use multiple learning methods also have favorable attitudes towards learning kidney physiology. Thus, modern curriculum changes that accommodate a variety of learning styles may promote positive attitudes toward nephrology.
Keywords: renal physiology, medical student, nephrology, attitudes, medical education
Introduction
The field of nephrology is at a crossroads. Historically, it was a growing specialty with a steady recruitment pipeline of qualified internal medicine physicians and researchers. However, over the last decade, the number of US medical school graduates entering nephrology has been steadily decreasing to an all-time low.1
Likewise, the number of applicants to nephrology through the National Residency Match Program (NRMP) has decreased over time, and for the 2015 appointment year nearly 50% of nephrology training programs did not fill all of their spots.2
This crisis gives our specialty the opportunity to self-reflect on a trend that has been brewing for many years: negative perceptions of nephrology within US medical education. On Internet message boards and within the “hidden curriculum” passed down from senior residents to junior trainees, negative attitudes and opinions about nephrology persist.3 A survey of internal medicine subspecialty fellows found that those who did not choose nephrology for subspecialty training perceived the care of patients with kidney disease complicated or too difficult.4 Almost one-third of the respondents also reported kidney physiology was the most difficult course during medical school, with acid-base disorders and electrolyte disorders cited as the most difficult topics to grasp. A quarter of these fellows also reported they would have considered nephrology if it had been “taught well” at their medical school. Among fellows who chose nephrology, 76% noted that they chose their specialty because they found it to be an interesting subject during medical school or residency.5
The quality of pre-clinical medical school courses has a role in the early formation of positive or negative perceptions of kidney disease and nephrology. However, classroom instruction has changed drastically over the years. Traditionally, medical physiology courses were taught via live lectures and/or an assigned textbook. Today, the typical classrooms are incorporating more clinical cases, implementing medical simulation, and adopting adult learning theory to give students more control over the learning process.6–7 Although there is a variety of learning techniques to choose from (team-based learning, problem-based learning, blended learning, massive open online courses), it remains unclear which will be ideal for kidney education.8–10
In this context, we wanted to better understand how the delivery of content relates to student attitudes about learning and interest in nephrology. As part of a curriculum renewal effort, we re-designed our kidney physiology course to include a wider variety of learning resources: internet-based electronic blackboard videos, live lectures, video recorded lectures, and newly written lecture notes available in electronic form. The only mandatory component of the course was a series of interactive, case-based sessions focused on clinical correlations to kidney physiology. In order to better understand student viewpoints following the new course format, we conducted a study using Q methodology, which is a hybrid qualitative-quantitative research technique where participants provide meaning to statements through a sorting procedure.11
Materials and Methods
Participants in a Q study create Q sorts by placing a series of statements onto a score sheet that has been ranked with columns representing the spectrum from strongly disagree (−4) to strongly agree (+4). In our study, participants are forced to place statement cards into the required number of spaces assigned under each column (see Figure 1A–B). This design requires participants to closely consider each statement more carefully, which is especially true for statements placed at each tail of the distribution. A Q sort is complete when all statements have been ranked, and in our case they will reveal a student’s viewpoint toward kidney physiology and nephrology. All of the Q sorts in a Q study are then subjected to by-person factor analysis, where similar Q sorts are correlated together into a unique viewpoint, or factor. By looking at the defining statements in each factor, the viewpoint is easily defined. In addition to the Q sort procedure, participants are asked to provide text responses to the two statements ranked at each extreme. The qualitative data is then used to inform interpretation and provide reasoning behind statement ranks.
Definition of the Statement Set
The statements used in a Q study need to be carefully crafted to represent the concourse of opinions present on a given topic. In our case, we created a statement set that represented a variety of issues relevant to learning kidney physiology in the undergraduate medical curriculum. To inform our statements, we reviewed medical student course evaluations of kidney physiology courses from the past two years from our institution and a peer academic institution. These course evaluations included both Likert scale ratings on multiple items and unstructured student commentary. In addition, we examined peer-reviewed journal articles related to student and resident interest in nephrology. Among these, statements were informed by survey data and editorials that reflect medical student and resident opinions of nephrology, kidney physiology, and the pre-clinical curriculum.3–5,10,12–14 We also included research related to medical student learning preferences and behavior in undergraduate courses.15–17 After reviewing the range of opinions related to learning kidney physiology, we identified five major domains: the student’s comprehension of kidney physiology, learning preferences, course characteristics, perceived clinical relevance of kidney physiology, and perception of nephrology as a career. The primary researcher (J.K.R.) formulated an initial set of statements within each domain. Finally, the statements were further refined for clarity and length by both researchers (R.W.L and J.K.R.). The final statement set is comprised of 37 statements.
Study Participants
We conducted our study between November and December 2014 at the Duke University School of Medicine. Our target audience was comprised of first-year medical students enrolled in a course entitled Normal Body that includes organ-based modules on anatomy, histology, and physiology. We administered the survey to the entire first-year class at the conclusion of the kidney physiology module. The students had already completed cardiovascular, pulmonary, and gastrointestinal physiology modules. The class was comprised of 108 students, with roughly equal numbers of male and female students.
Study Curriculum
Our study was conducted during the kidney physiology module of the Normal Body course. The major topics in the kidney physiology module were glomerular filtration, tubular reabsorption, regulation of body fluid osmolality, regulation of extracellular fluid volume, acid-base physiology, and potassium homeostasis. For each major topic, students had access to a multitude of learning resources: live lectures delivered by clinicians, video recordings of live lectures (available the same day), lecture notes, and Internet-based video “pencasts” (blackboard-style, narrated videos that heavily use diagrams, drawings, and cartoons). Course notes and video pencasts for all topics were accessible simultaneously at the start of the course. Our curriculum also included five large group clinical application sessions. The first session of kidney physiology consisted of an overview of the kidney system and an introduction to physiologic concepts such as homeostasis and mass balance. This session also included an interview between a nephrologist and a young patient with end-stage kidney disease. The patient told her story of developing progressive chronic kidney disease, which led to end-stage disease and peritoneal dialysis. The interview highlighted her personal struggle with kidney disease, as well as how loss of normal kidney physiology leads to a symptomatic imbalance in body fluid volume and composition. Other clinical application sessions consisted of case-based discussions highlighting disorders of tubular function, definition and treatment of hypotonic hyponatremia, and cases comparing and contrasting lactic acidosis with acidosis from kidney failure. At the conclusion of our kidney physiology section, we administered a large group simulation experience utilizing SimMan® technology. In this session, volunteer students examined a simulated patient with acute kidney failure, pulmonary edema, metabolic acidosis, and symptomatic hyperkalemia. All clinical decisions (diagnostic and therapeutic orders) were decided through multiple-choice questions and an electronic audience-response system. The primary focus of the case was managing hyperkalemia, but it also integrated other course topics such as acid-base disorders and extracellular fluid volume overload.
Sorting Procedure and Student Commentary
We delivered our survey using FlashQ (Version 1.0, Hackert and Braehler, Germany), a free, web-based program designed to both administer a Q sort and electronically record the data. We invited students to take the survey on the final day of their kidney physiology module. Participation in the survey was voluntary and separate from the Duke School of Medicine mandated course evaluations administered at the conclusion of the Normal Body course. Following an introduction with instructions for the survey, FlashQ randomly presented each of the 37 statements as a card to be sorted into one of three piles: agree, neutral, or disagree. Next, the student was asked to pull cards from these piles and place them into the Q sort diagram columns (Figure 1A–B) rated from strongly disagree (−4) to strongly agree (+4). Participants were given one last chance to swap card positions across the diagram so they more closely match his or her subjective viewpoint. Finally, participants had the opportunity to provide written commentary on the most agreed and most disagreed statements selected from the statement set. We stored the data from FlashQ in a de-identified text file on a secure server within our institution’s information technology department. The Duke Institutional Review Board deemed this survey exempt from review.
Factor Analysis
We analyzed the data using PQMethod software (Version 2.33, Schmolck, Germany), which performs by-person factor analysis. We initially extracted factors using the centroid method and retained all factors with Eigenvalues > 1. A Scree plot showed that a majority of the variance could be explained by the first two factors (total cumulative variance 51%). These factor structures were examined for interpretability, and a two-factor solution was chosen. We performed varimax rotation initially to simplify factor structure. In order to better separate students with different attitudes towards nephrology, we manually rotated the data set so that the students with positive attitudes towards nephrology were aligned in the first factor. A student’s Q sort was called a “factor definer” when it significantly loaded (γ >0.41 for a P <0.01) on only one unique factor.
Results
A total of 108 students received the invitation to the Q sort by email and 44 unique surveys were returned (41% response rate). During the initial sorting into agree, neutral, and disagree categories, the mean number of statements in each pile was 16, 11, and 10, respectively. Thirty-eight students provided commentary on statements ranked at both extremes on the score sheet. This resulted in a total of 152 unique comments. Table 1 lists the statements that were ranked at each extreme along with examples of student commentary in response to each particular statement. Table 2 shows the utilization of each learning resource in the modified course format. Live lecture attendance ranged from 33–44% at each session, which is consistent with the average attendance rate for non-mandatory lectures at our institution. Course notes for each topic were downloaded 71–85 times, and each subgroup of pencast videos was viewed an average of 98 to 264 times.
Table 1.
Statements | Sample Comments |
---|---|
Strongly Agree | |
It is important for me to understand the kidney |
|
By graduation, I should know how to interpret lab tests correctly |
|
I am a visual learner |
|
I wish other courses were taught like this one |
|
I learn much of what is needed for class from online lectures and videos |
|
I learn better when I am around other students |
|
I learn in my own unique way |
|
Strongly Disagree | |
After this course, I can forget about the kidney |
|
Our instructors don’t seem to enjoy teaching |
|
Too much of what we learn is not clinically relevant |
|
I learn much of what is needed for class from attending the lectures |
|
Interactions during class help me understand concepts |
|
Physiology is easier than other courses |
|
It is worth my time to come to class |
|
Table 2.
Topic | Lecture Attendance (%) | Note Utilization (Total Downloads) | Video Utilization (Mean Views) |
---|---|---|---|
Glomerular Filtration | 44 | 85 | 264 (1 video) |
Tubular Function and Reabsorption | 44 | 84 | 154 (2 videos) |
Regulation of Osmolality | 35 | 71 | 103 (2 videos |
Regulation of ECFV | 32 | 73 | 107 (2 videos) |
Acid Base Physiology | 34 | 72 | 115 (4 videos) |
Potassium Homeostasis | 34 | 74 | 98 (2 videos) |
Factor analysis produced a two factor solution from the 44 student Q-sorts. Manual rotation of the data set to separate attitudes towards nephrology loaded 39 unique sorts onto factor one and 4 unique sorts onto factor two. One Q sort did not load significantly onto either factor (see Figure 2). Figure 3 shows both factors represented as idealized Q sorts (mean values for each statement score), identifying statements that significantly distinguish the factors (distinguishing statements) and those that are statistically similar (consensus statements).
The Potentials (Factor One)
Twenty-four of the thirty seven statements were found to be statistically distinguishing statements for the students in factor one compared with students in factor two (see Figure 3). In this viewpoint, students think kidney physiology will be clinically relevant in the future. They also value the need for knowing how the kidney functions (“It will be important for me to understand the kidney”: +4), how to interpret lab tests correctly (“By graduation, I should know how to interpret lab tests correctly”: +4), and they perceive a future need to take care of patients with chronic kidney disease (“I will likely have to care for patients with kidney disease”: +3). In terms of learning characteristics, these students were more likely to learn from online lectures and videos (“I learn much of what is needed for class from online lectures and videos”: +1), and find value in attending live lectures (“Going to class is a waste of my time”: −2). These students are confident (“Nothing is too difficult for me to learn”: +2) and finished the course feeling like they understand the kidney (“The Kidney is a black box”: −3) and acid-base disorders (“I do not understand acid-base disorders”: −3). Students think the course is clinically relevant (“Too much of what we learn is not clinically relevant”: −3), they hold favorable perceptions of the instructors (“Our instructors don’t seem to enjoy teaching”: −4). They are willing to shadow a nephrologist (“I would be willing to shadow a nephrologist”: +1) and they are also familiar with the nephrology procedures (“I am unfamiliar with the procedures a nephrologist performs”: −2). We have designated the first factor “The Potentials” because they complete the course with favorable attitudes towards kidney physiology and they are potentially more interested in nephrology.
The Uninterested (Factor Two)
We have designated this second viewpoint “The Uninterested” based on how they leave the course with less perceived comprehension of the kidney system and no interest in nephrology. These students prefer to learn while alone (“I learn better when I’m by myself”: +4) and using only a textbook (I learn much of what is needed for class from textbooks”: +3). Students in this viewpoint feel they do not learn well through attending live lectures (I learn much of what is needed for class from attending the lectures”: −4) and thus, feel like it is a waste of time to attend (“Going to class is a waste of my time”: +4 and “It is worth my time to come to class”: −4). Compared with the Potentials, this viewpoint also values lab interpretation, understanding the kidney system, and caring for patients with kidney disease, however, the strength of these attitudes were significantly lower in this viewpoint (see Figure 3). Likewise, students in this viewpoint felt that much of what is taught is not clinically relevant (“Too much of what we learn is not clinically relevant”: +3) and they ended up less satisfied with how well they understand kidney physiology (I am satisfied with how well I understand physiology”: −3 and “The kidney makes sense”: −1). Students in this factor also felt strongly about not having enough time to keep up in class (“I have enough time to keep up with everything”: −3). These students were less familiar with the procedures a nephrologist performs and they are not interested in learning more about nephrology, shadowing a nephrologist, or hearing more about kidney disease research (see Figure 3).
Discussion
To explore our students’ perceptions of kidney physiology and nephrology, we used Q methodology, which is a quantitative-qualitative approach to defining subjectivity in humans. Using a form of multivariate analysis, a Q study is designed to extract the number of viewpoints in a sample while identifying the distinctions and similarities between each. Introduced in 1935, Q methodology was borne out of social science research, but its application has expanded in a variety of disciplines including higher education18–19, health professions education20, and graduate medical education.21–23 In a prior study, we also successfully used Q methodology to define the dominant learning styles present in a cohort of first-year medical students.24
This study indicates that there are two primary viewpoints among medical students enrolled in a contemporary kidney physiology course. In one viewpoint, the students that had slightly more favorable attitudes towards nephrology were the students who valued both attending live lectures and using multiple learning resources like electronic pencast videos. The second viewpoint included students with less favorable attitudes towards nephrology. The students in this viewpoint prefer to learn from only textbooks, they prefer studying alone, and they strongly feel like it is a waste of time to attend class (see Figure 3). These Uninterested students found the kidney physiology course not clinically relevant, and were dissatisfied with their kidney physiology knowledge. Since the responses were de-identified, we are unable to correlate attitudes to actual attendance/participation or examination scores, but it is reasonable to assume that these students were not attending lectures or clinical sessions, and not utilizing the full breadth of our available learning resources.
The Potentials, however, developed a stronger familiarity with the field of nephrology while learning kidney physiology. These students, who felt like they better understand kidney physiology, also found the course to be highly clinically relevant compared to the students in the Uninterested viewpoint (see Figure 3). The Potentials are able to effectively learn from lectures, watching videos, and/or reading notes, while the Uninterested students learn best from a textbook alone. Therefore, it is possible that being flexible to multiple learning methods has some impact on knowledge, satisfaction, and curiosity towards a given discipline. The student commentary we collected supports this hypothesis as students noted how the availability of flexible learning methods and classroom interactions directly impacted course satisfaction (see Table 1). For example, the large group simulation experience allowed us to integrate multiple topics (potassium balance, acid-base disturbances, volume overload) while jumping back and forth from basic science concepts to the patient in real time. The Potentials also have high levels of self-confidence with regard to learning, while the Uninterested students do not, despite both groups finding kidney physiology equally difficult (see Figure 3). This suggests that a learner’s self-confidence, or other unmeasured variables, may interact with the learning process and/or student engagement, both of which appear to be critical for developing interest in intrinsically difficult topics such as nephrology. In conclusion, this study found associations between learner self-confidence, use of multiple learning resources, perceived relevance of kidney physiology, and interest in nephrology among first year medical students.
Because factor analysis of Q sort data is a correlational procedure, we cannot conclude causation between the various attitudes and interest in nephrology. Because of the qualitative study design, we do not have predetermined outcomes of interest and we were simply interested in understanding student attitudes as they completed our course. However, these results better define our learners, and now we can better understand their needs, preferences, and responses to future educational interventions. Among the various outcomes in education, attitudes are generally the most difficult to measure. Electronic questionnaires with Likert scales are a popular method for measuring attitudes, however Likert scale survey data are frequently analyzed inappropriately and when misused, invalid for measuring subjectivity.25 Research on attitudinal assessment with the Likert scale has shown that responses are influenced by survey structure, question characteristics, category characteristics, and response scale factors, all of which can potentially lead to systematic error and misguided interventions.26–29 Therefore, we chose to use Q methodology, which is a more valid measurement of participant subjectivity, to understand how attitudes correlate in medical students who completed a modern kidney physiology course.
There are several important limitations to this study. First, our final analysis included 44 Q sorts out of the 108 students who took our modified kidney physiology course, yielding a response rate of 41%. This could have produced a biased sample if satisfied students with favorable opinions were more likely to respond than unsatisfied students with less favorable opinions. This could have limited the breadth of definition within the second factor (the Uninterested) since they were defined by a fewer number of Q sorts. It should be noted that Q methodology is designed to define the quantity and quality of viewpoints in a sample, and it is less dependent on sample size. In this vein, we are more concerned with how the Uninterested feel, rather than how numerous they are in the sample. Second, imbalance in agreeability of the Q sort statements themselves is a potential source of bias that can affect the results of a Q study. During the first stage of the survey, initial card sorting into agree, neutral, and disagree piles produced a mean of 16, 11, and 10 cards, respectively. This indicates a slight bias towards agreement, but demonstrates a reasonable overall balance in the range of available statements. Third, our students’ attitudes may be specific for the student population at our institution and not necessarily generalizable to other student cohorts. Fourth, a weakness with subjective student ratings is that perception of learning or value with a given learning resource or strategy may not translate into actual learning or value. Because our students’ responses were de-identified, we were unable to correlate Q sort values to examination performance or other demographic information. The purpose of this study was to more accurately define student viewpoints after we implemented a planned course modification. Because we are not comparing these results to a control group, any correlation between course attributes and attitudes can be inferred directly from student commentary (see Table 1). In order to measure the effect of such a curriculum change on other learning outcomes such as medical knowledge or higher-order cognitive skills, a prospective, controlled, educational interventional trial is required.
Our study was designed to explore the opinions and beliefs held by first-year medical students who participated in our kidney physiology course. The modified kidney physiology curriculum was borne out of a rigorous curriculum improvement process and the desire to meet the call to educational reform while capturing the interest of a captive audience. Our study reinforces the fact that learners are heterogeneous and contain a complicated mix of attitudes, beliefs, and preferences. After passing through our kidney physiology course, we identified a group of “Potential” students who understand the kidney, appreciate the application of kidney physiology, and have favorable attitudes towards the specialty of nephrology.
This constellation of attitudes represents an intentional component of our physiology curriculum. For example, in our Introduction to kidney physiology lecture, we utilized an interview with a young patient with ESRD to highlight the prevalence of CKD/ESRD, the primary functions of the kidney, and the ramifications of losing kidney function through the patient’s words. In this session, we weaved in the longitudinal role of the nephrologist, the concept of peritoneal dialysis, and the current role of kidney transplantation for kidney replacement. Afterwards, the course was divided into “Foundational Material,” which was provided three ways (live lectures, notes, and pencasts), and “Clinical Applications,” which directly allowed students to see (through cases, problem-solving, and simulation) how the knowledge helps a nephrologist treat seriously ill patients. In this paradigm, we create the environment where individualized learning and clinically relevant instruction help promote career interest in real time (see Figure 4).
Kidney physiology is a complex and daunting topic, but early understanding and knowledge are potentially linked to developing an interest in the specialty of nephrology. Early exposure to nephrology appears to have a role in specialty interest: close to a quarter of nephrology fellows (23%) chose nephrology during medical school, and the majority (76%) chose nephrology because it was an interesting subject.5 However, in many classrooms, students are forced to learn the complexity of the kidney out of clinical context, where many will get turned off right at the start and develop a “love it or hate it” attitude that is often difficult to reverse.13–14
For first-year medical students, our goal is to educate them properly and show them how this knowledge will make them better physicians. The best-case scenario with a large captive audience like this is to simply not turn them away from considering nephrology. If we can help them see the relevance and importance of the kidney system, then they are primed to take the next step through clinical care of patients and perhaps more willing to shadow a nephrologist or seek out a mentor. Medical students, like any large group, are heterogeneous learners with a variety of attitudes. Based on the results of this pilot qualitative study, it is possible that a number of interventions can be effective in transforming the quality of kidney education and promoting early interest in nephrology (see Table 3). A longitudinal, prospective, controlled trial will be necessary to determine the impact of such interventions on learning and interest in our specialty. In our pilot study, students who completed our modified kidney physiology curriculum developed one of two primary viewpoints. We found that students who are more confident with regard to learning, learn through a wide variety of learning resources, and appreciate the clinical relevance of kidney physiology, are also the students who develop a healthier perception of nephrology.
Table 3.
|
Acknowledgments
Funding/Support: J.K.R. was supported by the Duke Training Grant in Nephrology (5T32DK007731). M.A.S. is funded by Career Development Award IK2BX002240 from the Department of Veterans Affairs, Office of Research and Development, Biomedical Laboratory Research and Development Service.
The authors wish to thank Dr. Charles W. Hargett III, MD, the Duke School of Medicine, and the Division of Nephrology at Duke University Medical Center.
Footnotes
Author contributions: JKR and RWL conception and design of research; JKR administered survey; JKR analyzed data; JKR and RWL interpreted results; JKR prepared figures and tables; JKR drafted manuscript; JKR, MAS, and RWL edited and revised manuscript; JKR, MAS, and RWL approved final version of manuscript.
Conflict of Interest/disclosures: The authors have no conflicts of interest to declare. The results presented in this paper have not been published previously in whole or part.
Ethical approval: The Duke Institutional Review Board deemed this survey exempt from review (Project 00049444)
Disclaimer: None
Previous presentations: None
References
- 1.Salsberg E, Masselink L, Wu X American Society of Nephrology. [Accessed April 1, 2015];The US Nephrology Workforce: Developments and Trends. https://www.asn-online.org/education/training/workforce/Nephrology_Workforce_Study_Report.pdf.
- 2.National Residency Matching Program. [Accessed April 30, 2015];Fellowship Match Data 2015 Appointment Year. http://www.nrmp.org/wp-content/uploads/2015/02/Results-and-Data-SMS-2015.pdf.
- 3.Parker MG, Ibrahim T, Shaffer R, Rosner MH, Molitoris BA. The future nephrology workforce: will there be one? Clin J Am Soc Nephrol. 2011;6:1501–6. doi: 10.2215/CJN.01290211. [DOI] [PubMed] [Google Scholar]
- 4.Jhaveri KD, Sparks MA, Shah HH, Khan S, Chawla A, Desai T, Iglesia E, Ferris M, Parker MG, Kohan DE. Why not nephrology? A survey of US internal medicine subspecialty fellows. Am J Kidney Dis. 2013;61:540–546. doi: 10.1053/j.ajkd.2012.10.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Shah HH, Jhaveri KD, Sparks MA, Mattana J. Career Choice and Satisfaction among US Adult Nephrology Fellows. Clin J Am Soc Nephrol. 2012;7:1513–1520. doi: 10.2215/CJN.01620212. [DOI] [PubMed] [Google Scholar]
- 6.Irby DM, Cooke M, O’Brien BC. Calls for reform of medical education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Academic Medicine. 2012;85:220–227. doi: 10.1097/ACM.0b013e3181c88449. [DOI] [PubMed] [Google Scholar]
- 7.Knowles MS. Andragogy in action: Applying modern principles of adult learning. San Francisco: Jossey-Bass; 1987. [Google Scholar]
- 8.Doherty I, Sharma N, Harbutt D. Contemporary and future eLearning trends in medical education. Med Teach. 2015;37(1):1–3. doi: 10.3109/0142159X.2014.947925. [DOI] [PubMed] [Google Scholar]
- 9.Onyura B, Baker L, Cameron B, Friesen F, Leslie K. Evidence for curricular and instructional design approaches in undergraduate medical education: An umbrella review. Med Teach. 2015;10:1–12. doi: 10.3109/0142159X.2015.1009019. [DOI] [PubMed] [Google Scholar]
- 10.Jhaveri KD, Sparks MA, Shah HH. Novel educational approaches to enhance learning and interest in nephrology. Adv Chronic Kidney Dis. 2013;20(4):336–346. doi: 10.1053/j.ackd.2013.03.006. [DOI] [PubMed] [Google Scholar]
- 11.Stephenson W. Technique of factor analysis. Nature. 1935;136:297. [Google Scholar]
- 12.Rosner M, Parker M, Kohan D. Nephrology as a career choice: A survey of medical students [ASN Abstract] J Am Soc Nephrol. 2009;20:SA-PO2867. [Google Scholar]
- 13.Patel AB, Balzer MS. On becoming a nephrologist: medical students’ ideas to enhance interest in a career in nephrology. Am J Kidney Dis. 2013;62:450–452. doi: 10.1053/j.ajkd.2013.01.020. [DOI] [PubMed] [Google Scholar]
- 14.Hoenig MP, Shapiro E, Hladik GA. Lessons learned from the ASN Renal Educator Listserv and survey. Clin J Am Soc Nephrol. 2013;8:1054–1060. doi: 10.2215/CJN.07410712. [DOI] [PubMed] [Google Scholar]
- 15.Cardall S, Krupat E, Ulrich M. Live lecture versus video-recorded lecture: are students voting with their feet? Acad Med. 2008;83:1174–1178. doi: 10.1097/ACM.0b013e31818c6902. [DOI] [PubMed] [Google Scholar]
- 16.Billings-Gagliardi S, Mazor KM. Student decisions about lecture attendance: do electronic course materials matter? Acad Med. 2007;82:S73–76. doi: 10.1097/ACM.0b013e31813e651e. [DOI] [PubMed] [Google Scholar]
- 17.Gupta A, Saks NS. Exploring medical student decisions regarding attending live lectures and using recorded lectures. Med Teach. 2013;35:767–771. doi: 10.3109/0142159X.2013.801940. [DOI] [PubMed] [Google Scholar]
- 18.Ramlo SE, Mconnell D, Zhong-Hui D, Moore FB. Evaluating an Inquiry-based Bioinformatics Course Using Q Methodology. Journal of Science Education and Technology. 2008;17:219–225. [Google Scholar]
- 19.Ramlo SE. Determining Faculty and Student Views: Applications of Q Methodology in Higher Education. Journal of Research in Education. 2012;22:86–107. [Google Scholar]
- 20.Petit dit Dariel O, Wharrad H, Windle R. Developing Q-methodology to explore staff views toward the use of technology in nurse education. Nurse Res. 2010;18:58–71. doi: 10.7748/nr2010.10.18.1.58.c8048. [DOI] [PubMed] [Google Scholar]
- 21.Gaebler-Uhing C. Q-methodology: a systematic approach to assessing learners in palliative care education. J Palliat Med. 2003;6:438–442. doi: 10.1089/109662103322144790. [DOI] [PubMed] [Google Scholar]
- 22.Thammasitboon S, Mariscalco MM, Yudkowsky R, Hetland MD, Noronha PA, Mrtek RG. Exploring individual opinions of potential evaluators in a 360-degree assessment: four distinct viewpoints of a competent resident. Teach Learn Med. 2008;20:314–322. doi: 10.1080/10401330802384680. [DOI] [PubMed] [Google Scholar]
- 23.Meade LB, Caverzagie KJ, Swing SR, Jones RR, O’Malley CW, Yamazaki K, Zaas AK. Playing with curricular milestones in the educational sandbox: Q-sort results from an internal medicine educational collaborative. Acad Med. 2013;88:1142–1148. doi: 10.1097/ACM.0b013e31829a3967. [DOI] [PubMed] [Google Scholar]
- 24.Roberts JK, Hargett CW, Nagler A, Jakoi E, Lehrich RW. Exploring student preferences with a Q-sort: the development of an individualized renal physiology curriculum. Adv Physiol Educ. 2015;39:149–57. doi: 10.1152/advan.00028.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Jamieson S. Likert scales: how to (ab)use them. Med Educ. 2004;38:1217–8. doi: 10.1111/j.1365-2929.2004.02012.x. [DOI] [PubMed] [Google Scholar]
- 26.Baumgartner H, Steenkamp JEM. Response styles in marketing research: A cross-national investigation. Journal of Marketing Research. 2001;38:143–156. [Google Scholar]
- 27.Ogden J, Lo J. How meaningful are data from Likert scales? An evaluation of how ratings are made and the role of the response shift in the socially disadvantaged. J Health Psychol. 2012;17:350–61. doi: 10.1177/1359105311417192. [DOI] [PubMed] [Google Scholar]
- 28.Kieruj N, Moors G. Variations in response style behaviour by response scale format in attitude research. Journal of Public Opinion Research. 2012;22:320–42. [Google Scholar]
- 29.Krosnick JA, Berent MK. Comparisons of party identification and policy preferences: The impact of survey question format. American Journal of Political Science. 1993;37:941–64. [Google Scholar]
- 30.Guo P, Kim J, Rubin R. [Accessed March 6, 2015];How Video Production Affects Student Engagement: An Empirical Study of MOOC Videos. 2014 https://groups.csail.mit.edu/uid/other-pubs/Las2014-pguo-engagement.pdf.