Abstract
Context:
Knowledge and experience may be important factors for understanding expertise based upon a clinician's ability to select and execute an appropriate response as a clinician during injury evaluation.
Objective:
To describe how collegiate male certified athletic trainers represent injury-evaluation domain knowledge during a situational interview using a think-aloud protocol.
Design:
Qualitative.
Setting:
National Collegiate Athletic Association Division I and II colleges in National Athletic Trainers' Association District 3.
Patients or Other Participants:
A total of 20 male certified athletic trainers (n = 10 with less than 2 years of experience in the college setting and n = 10 with at least 10 years of experience in the college setting) participated in the study.
Data Collection and Analysis:
We collected data using a situational interview and questionnaire. Data were transcribed, reduced to meaningful units, and analyzed using verbal analysis procedures. Member checks, triangulation of data, field journaling, and peer-debriefing techniques were used to ensure trustworthiness of the data. Knowledge concepts were enumerated to describe differences between experts and novices.
Results:
Compared with novices, experts had more knowledge concepts of patient history and predictions and fewer concepts of situation appraisal.
Conclusions:
Expertise in athletic training shares traits with other areas in health care. Athletic training education and professional development may benefit from our understanding which cognitive processes differentiate expert practice. Future investigators should attempt to describe other settings and study diagnostic problem solving in a natural environment.
Key Words: expertise, professional development, qualitative research
Key Points
Compared with novice collegiate male athletic trainers, expert collegiate male athletic trainers had more knowledge concepts that focused on the patient history and predicting the outcome.
A better understanding of the cognitive processes that differentiate expert clinicians from novices may help us to improve athletic training educational programs and professional development.
Diagnostic problem solving in health care is based upon assessment, problem identification, intervention, and continual evaluation. Underlying this sequence are complex cognitive processes involving critical thinking and reasoning that influence appropriate response selections and execution within a practice domain. Critical-thinking behavior in athletic training has been described as the process of “skillful application of knowledge and experience in making discriminating judgments and evaluations” toward a conclusion.1 Consequently, knowledge and experience may be important factors in understanding expertise based upon the clinician's ability to select and execute an appropriate response during injury evaluation.
Clinical expertise is defined as consistently superior response selection and execution on reproducible tasks within a practice domain.2–4 It is well established that 10 years or 10 000 hours of deliberate practice is required to achieve expert performance.2–6 Traditionally, expertise in most disciplines is studied using 2 different methodologic approaches: (1) an absolute approach, in which top performers in the discipline are evaluated on a specific task, and (2) relative expertise, in which individuals are measured in relationship to nonexperts in the field.7 In medicine, the common approach has been relative expertise, in which medical doctors are compared with medical students, residents, and other physicians outside their specialty area.
Authors of studies in medical expertise have primarily examined the cognitive aspects of proficiency and identified differences in knowledge representation (ie, the way an individual depicts what he or she has committed to memory) through concepts, decision making and reasoning, and diagnostic accuracy.8–10 These researchers have demonstrated that expertise is facilitated by highly organized and thoroughly interconnected knowledge structures within a practice domain.4 These approaches have sought to identify the cognitive differences in nurses, medical students, residents, and physicians with regard to their clinical reasoning and decision making.11
In athletic training, we have yet to document the behaviors and processes that discriminate experts in our practice domains from novices. The term domain refers to an area of activity, and our use of this term will refer to the athletic training practice domains of (1) prevention, (2) clinical evaluation and diagnosis, (3) immediate care, (4) treatment, rehabilitation, and reconditioning, (5) organization and administration, and (6) professional responsibility as defined by the fifth Role Delineation Study.12 Education in other health care professions (ie, medicine, nursing, physical therapy) has established a clear benefit of using the expertise literature and its subsequent effect on professional development and educational practices.2,13,14 Interconnecting knowledge structures in specific situations can be learned and practiced in laboratory and clinical education classes during athletic training education and may promote the development of expert-like behaviors.15,16
Authors in the athletic training literature have made attempts to bridge theory and practice in the development of expertise by beginning inquiry into how education can model professional practice. These investigators focus more on postprofessional education and career paths and are starting to shape what we know about improving one's knowledge base and skills over time.17,18 However, the knowledge base for athletic training has a limited breadth and scope of contributions that describe specific cognitive processes used by professionals to sort through problems. Few reports have described the traits and deliberate practice activities of experts in the athletic training practice domains.18 In other areas of health care, expert performers have been studied in order to improve educational practices, professional development, and ultimately clinical practice.2
Based upon the aforementioned research in health care, athletic trainers engaging in consistent clinical practice activities over time may develop highly interconnected knowledge structures that influence their critical thoughts and clinical reasoning. However, the differences between experts who are highly experienced and novices with limited experience in the athletic training field remain empirically unclear. With a better understanding of the differences between experts and novices in an athletic training practice domain, athletic training educators and clinicians can better understand how to teach critical-thinking tasks to model expert thinking and reasoning. In addition, initial investigations into these cognitive processes may provide a theoretical basis for additional studies on cognitive processes in the field of athletic training.
Quantitative approaches to studying expertise have involved laboratories to recreate and capture what separates highly skilled clinicians from those who are less skilled. These approaches typically measure dependent variables such as diagnostic accuracy, recall, speed of diagnosis, and cue relevance.8 Authors of such studies have provided evidence that highly skilled individuals in medicine are typically faster, are more accurate, can recall more, and monitor more relevant cues than the less skilled. Qualitative approaches to studying expertise have primarily relied on verbalizations through protocol analysis to describe the process that the more highly skilled use to reason in comparison with the less skilled.8,19,20 Verbalizations are commonly obtained through think-aloud or talk-aloud interviews that may or may not incorporate situational prompts.8,19,21–23 These approaches, although qualitative in nature, use an enumerative process to reduce the verbal data to numbers that can be analyzed. Researchers have used verbalizations to identify correct protocols, to show knowledge representation, and to depict reasoning strategies or direction of reasoning during the evaluation of illnesses.8 Knowledge representation is defined as the way an individual depicts his or her knowledge base.
The purpose of our study was to describe how certified athletic trainers represent clinical evaluation and diagnosis domain knowledge during a situational interview using a think-aloud protocol. Specifically, we sought to describe how expert and novice athletic trainers represent knowledge during an injury-evaluation task. Five injury scenarios were used to prompt participants during the interview. We believed that experts and novices in our study would engage in different types of cognitive processing and that knowledge would be represented in different ways based upon their level of experience. We hypothesized that expert knowledge concepts would appear in a way that was similar to their appearance in studies on medical experts, and that the experts in our study would demonstrate combinations of knowledge encapsulation and illness-scripting strategies, whereas novices would focus on information networking.
METHODS
We used a think-aloud procedure to elicit verbal data and record the cognitive protocols of participants for 5 injury scenarios. A think-aloud interview technique asks participants to verbalize their thoughts as they attempt to solve a problem.15,19 Each interview took place in person at the home institution of each participant. No time restraints were placed on each interview; however, the average completion time was 41 minutes. We obtained institutional review board approval before contacting any potential participants or collecting any data.
Participants
All participants interviewed reported having current positions in collegiate athletics within National Athletic Trainers' Association District 3. There were 10 relative expert participants and 10 relative novice participants in the study; a summary of participants' age and experience is found in Table 1.
Table 1.
Age and Experience (y) of Participants (Mean ± SD)a
| Athletic Trainers |
Age |
Experiencea |
| Expert | 41.9 ± 9.78 | 17.9 ± 10.24 |
| Novice | 24 ± 1.94 | 1.3 ± 0.48 |
n = 10 in each category. Experience refers to years as a Board of Certification–certified athletic trainer.
A criterion sampling strategy was used to recruit participants. Criterion sampling is the selection of cases or participants that meet some predetermined standard of importance.24 The inclusion criterion required each expert to have practiced as a Board of Certification–certified athletic trainer for a minimum of 10 years in the college or university setting. The novice inclusion criterion required each participant to have less than 2 years of professional experience in the college or university setting.
College athletic trainers provided a convenient sample for us to investigate. Furthermore, participants in this study represented Divisions I and II of the National Collegiate Athletic Association. Contact information for each participant was obtained through university athletics directories of institutions of higher education. We e-mailed potential volunteers to inform them of the study and ask for their participation. Those who replied were contacted via telephone to establish a mutually agreed-upon time and location for the interview.
Instrumentation
Five scenarios of injuries sustained to the upper limb, the thorax, and the lower limb were presented to each participant. Each scenario was obtained from an injury described in a published case report. The first 2 scenarios described athletes with injuries to the upper limb.25,26 The third scenario described an athlete with a thorax injury.27 The last 2 scenarios described athletes with injuries to the lower extremity.28,29 Scenarios were organized in a standard progression of history, inspection, palpation, and special tests consistent with the format typically used in athletic training education.30 We provided information in the scenario to each participant with this format, including a background to the case, a relevant history, remarkable findings from the inspection and palpation, and range-of-motion and special tests performed and the outcome of each.
Pilot Study
We conducted a pilot study with 2 aims. Our first aim was to determine if the procedures were clear and understandable to all participants. Ericsson and Simon19 stated that when using verbal reports as data to establish a protocol or set of rules for a participant's thought process in cognitive psychology, the procedures must explain exactly what one wants to occur and must be repeatable. The second aim was to determine if the tasks in the situation interview were representative enough of real injuries to discriminate decision making of experts and novices in athletic training. This is important because the verbalizations needed to represent what the individual would actually think about when evaluating a patient with an injury, and, because we could not study actual patient encounters, these scenario descriptions needed to be as close to real life as possible. We selected 4 participants based on availability and years of experience. Each expert participant met the inclusion criterion of being certified for at least 10 years, and novices met the criterion of less than 2 years of experience as a certified athletic trainer.
During the pilot study, each participant completed a demographic questionnaire and a think-aloud interview with the same 5 scenarios described previously. We used systematic, written procedures to establish consistency of instructions for each participant, which consisted of (1) reading the directions for the think-aloud procedure, (2) providing a practice scenario, (3) presenting each scenario one at a time for the participant to read, (4) taking the scenario description away, and (5) asking the participant to verbalize thoughts about the scenario as he reasoned toward a diagnosis.
The findings of the pilot study validated our chosen methodologic procedures for answering our research questions. First, the demographic questionnaire was a quick and efficient way to ensure that participants met the inclusion criteria and were grouped properly. Second, no participants appeared to have difficulty understanding the directions for the think-aloud interview and all were able to easily complete the task. Third, interviews took an average of 41 minutes, and each interview was conducted in less than 1 hour. Finally, when the scenario description was taken away, participants described having difficulty with the think-aloud process because they could not remember what they had read. As a result, the fourth participant was allowed to refer back to the scenario description to determine if the cognitive load on memory could be alleviated. This participant appeared to have much less difficulty concentrating on thinking aloud during each scenario because he did not have to recall the scenario text. As a result, we altered the study procedures to allow participants to keep and refer back to the scenario description. This also eliminated the amount of text recalled directly from the cases presented.
Procedures
We asked participants to think aloud as they reasoned toward a solution to each of the 5 scenarios. All interviews occurred in a one-on-one encounter between the primary investigator and each participant. The think-aloud technique carries the assumption that verbalizations represent an individual's underlying knowledge structures in memory and is documented as an effective technique to study decision making and reasoning in cognitive psychology,21 sport studies,31 and medicine.32 After instructions for the think-aloud process were provided, 1 practice scenario was supplied to acquaint the participant with the interview format. The same interviewer, who was trained in qualitative procedures, met with all participants. The order of case descriptions remained the same for each interview, and prompts to guide responses were also consistent. The participants were told that there was no time limit on how long they could spend working through each scenario and that they should respond to each as thoroughly as possible. The interviewer read the directions to ensure consistency in each interview. The participants were provided with a written copy of the complete scenario to read over, with no time limit for how long each reading could take. After reading the scenario, the participant was allowed to keep the text for reference and to reduce cognitive overload of memory. The participant was prompted to begin thinking aloud as he recalled the scenario. During the recall, the researcher used the phrase “Anything else?” as a prompt to ensure that each participant's response to the scenario was complete before moving to the next one. At the end of each scenario, participants were asked for a differential diagnosis and “Are there any alternatives?” before moving on to the next scenario. Asking for alternatives was done to ensure that the participant's thoughts were complete regarding each scenario. All dialogue during the situation interview was audio recorded and transcribed, and all nonverbal behaviors, including how often participants needed to refer back to the case description, were documented in a field journal. We used field notes in the journal to clarify verbalizations on the audio recordings that were difficult to understand because of background noise, mumbling by the participant, direct reading out loud, and other nonverbal behavior during the interview that later informed how the data were transcribed and coded.
Data Analysis
Data reduction for the interviews took place in several steps. Traditionally, verbal protocol analysis requires that the correct response be identified before the analysis. There was no correct protocol (ie, procedure or system of governing how a task is completed) created to depict the proper sequence of steps to take toward a solution. Therefore, we used verbal analysis rather than protocol analysis. Verbal analysis is a specific technique for analyzing verbal data when there is no template for correctness of a participant's response selection.33 Verbal analysis assumes that a verbal protocol is collected through qualitative means and then transcribed.
Verbal analysis may appear to use terminology that is uncommon to even those familiar with qualitative methods, and thus the following definitions should guide the reader regarding specific language used in the Methods: (1) Protocol refers to the rule system or set of procedures used by each participant to make a decision. (2) Segmenting is the process of reducing verbal data that have been transcribed into a smaller statement that allows the analyst to begin to examine only information that is not repeated from the scenario. (3) Coding scheme refers to the manner in which data are organized into groups. (4) Mapping is the process of examining how a participant connected knowledge. (5) Grain size and unit of analysis are the smallest pieces of data that, once segmented, can be identified as the functional unit of data to be analyzed. The method of coding and analyzing the data consists of 8 functional steps described by Chi33: (1) reducing or sampling the protocols, (2) segmenting the reduced or sampled protocols, (3) developing or choosing a coding scheme or formalism, (4) operationalizing the evidence in the coded protocols that constitutes a mapping to some chosen formalism, (5) depicting the mapped formalism, (6) seeking patterns in the mapped formalism, (7) interpreting the patterns, and (8) repeating the process, often at a different grain size.
The data were transcribed and member checks were completed with all interview participants to ensure accuracy and completeness of statements. Member checking is a qualitative research process in which transcripts are reviewed by participants to ensure that statements are accurate.24 We segmented the transcript from each interview into single statements and numbered the statements for referencing purposes. We established each segment by comparing the audio file of each participant with the actual transcript. The boundaries for each segment were established as a pause before verbalizing to the next pause after a verbalization, which was consistent with previous studies.8 The segments were then further reduced into meaningful units. These units were individual clauses (eg, phrases or ideas) that we considered the unit of analysis to produce a protocol to identify knowledge concepts.22
Coding
We performed qualitative analysis to determine if there were unique components to participants' protocols. Unique components were defined as not part of the scenario presented. The analysis did not focus on how well participants read and restated the contents of each scenario text. We generated taxonomies (ie, information grouping systems) to compare data across groups. This data-reduction procedure is a specific analytic approach described by Grbich34 in which knowledge categories or concepts are created through a series of steps to condense data. The process began as we identified descriptive terms from meaningful units and then identified common items through free listing that fit into a connected subset or domain. Next, we sorted items to identify the structure of the domain. The resultant taxonomy provided a description for knowledge concepts contained within participants' statements that were not repetitions from the scenarios.
We established operational definitions for each of the knowledge concepts before analyzing the data. A goal statement was defined as a personal aim based upon the information presented in the scenarios. We defined situation appraisal as statements made by the participant in an attempt to make sense of what was read in the scenarios, particularly when participants did not exactly know where to start with their decision and repeated information to give themselves a sense of what had taken place. Networking referred to unique statements made by the participant demonstrating that he was attempting to determine relevant history and the results of observation, palpation, special tests, and functional tests from the presented scenario. During their reasoning, some participants attempted to consider the diagnosis based upon either how they would treat the condition or whether they wanted to request that additional tests, such as diagnostic imaging, be conducted by a physician. When this occurred, we coded attempts to diagnose the injury based upon how it would be treated as therapeutic, and any request for additional testing was coded as diagnostic. Prior experience was used to code statements that referred to the participant's practical contact with a particular part of the scenario. Prediction was used to code data that suggested the participant was expressing thoughts that indicated that he had specific expectations for the conditions presented in the scenario, and, in some cases, that he attempted to provide a prognosis. Unknown was the code for data that clearly indicated the participant did not know or did not have any idea regarding the conditions presented in the scenario.
Trustworthiness
We took several steps to ensure the data collected were credible, rigorously obtained, and reliably analyzed. Member checks, triangulation of data, a field journal, and peer-debriefing strategies helped to ensure the trustworthiness of the data.34 Member checks were conducted by having each of the 20 participants review his own transcript to ensure that his statements matched the transcripts and knowledge concepts of the analysis. We established triangulation of data by comparing codes during the analysis to the actual words used in the statements of our participants, and we used the participant's statement as the code. A field journal was maintained to track parts of the dialogue that may have seemed unclear during the interview. The journal aided in transcription of the data and in retaining its clarity during the analysis. Peer debriefing was conducted by having a second researcher review the process of data analysis and make comments on the emergent knowledge concepts to determine that the process was systematic and valid and that the data appeared unbiased.
RESULTS
The knowledge concepts identified were (1) goal, (2) situation appraisal, (3) networking (ie, history, observation, palpation, special tests, and functional tests), (4) therapeutic, (5) diagnostic, (6) prior experiences, (7) predictions, and (8) unknown information. We enumerated the frequency of each concept and reported unique participant knowledge concepts in relation to the total number of meaningful units after data reduction. Examples of textual statements representative of each knowledge concept for experts and novices are provided to give context for how statements were reduced and grouped into codes. Enumerated results of knowledge concepts for each of the 5 case scenarios are found in Tables 2 through 6.
Table 2.
Knowledge Concepts Accessed in Scenario 1
| Concept |
Experts | Novices |
| No. (%)a |
No. (%)a |
|
| Goal | 5 (7.24) | 7 (7.87) |
| Situation appraisal | 13 (18.84) | 22 (24.72) |
| Networking | ||
| History | 13 (18.84) | 12 (13.48) |
| Observation | 4 (5.80) | 11 (12.36) |
| Palpation | 2 (2.90) | 1 (1.12) |
| Special tests | 10 (14.49) | 7 (7.87) |
| Functional tests | — | 1 (1.12) |
| Therapeutic | 8 (11.59) | 10 (11.23) |
| Diagnostic | 8 (11.59) | 5 (5.62) |
| Prior experiences | 1 (1.45) | 3 (3.37) |
| Prediction | 5 (7.24) | 3 (3.37) |
| Unknown | — | 4 (4.49) |
% is the percentage of total statements (expert n = 69, novice n = 89) in the situation that were unique (ie, not contained in the scenario and that came solely from the participant).
Table 6.
Knowledge Concepts Accessed in Scenario 5
| Experts | Novices | |
| Concept |
No. (%)a |
No. (%)a |
| Goal | 2 (4.54) | 3 (6.52) |
| Situation appraisal | 12 (27.27) | 13 (28.26) |
| Networking | ||
| History | 5 (11.36) | 3 (6.52) |
| Observation | — | 2 (4.34) |
| Palpation | 1 (2.27) | 1 (2.17) |
| Special tests | 5 (11.36) | 5 (10.87) |
| Functional tests | 1 (2.27) | — |
| Therapeutic | 2 (4.54) | 5 (10.87) |
| Diagnostic | 11 (25) | 13 (28.26) |
| Prior experiences | 2 (4.54) | 1 (2.17) |
| Prediction | 3 (6.82) | — |
| Unknown | — | — |
% is the percentage of total statements (expert n = 44, novice n = 46) in the situation that were unique (ie, not contained in the scenario and that came solely from the participant).
Table 3.
Knowledge Concepts Accessed in Scenario 2
| Concept |
Experts | Novices |
| No. (%)a |
No. (%)a |
|
| Goal | 4 (13.33) | 10 (17.24) |
| Situation appraisal | 4 (13.33) | 12 (20.68) |
| Networking | ||
| History | 11 (36.67) | 11 (18.97) |
| Observation | — | — |
| Palpation | — | 4 (6.90) |
| Special tests | 5 (16.67) | 9 (15.52) |
| Functional tests | 1 (3.33) | 1 (1.72) |
| Therapeutic | — | 5 (8.62) |
| Diagnostic | 2 (6.67) | 3 (5.17) |
| Prior experiences | — | 1 (1.72) |
| Prediction | 3 (10) | 1 (1.72) |
| Unknown | — | 1 (1.72) |
% is the a percentage of total statements (expert n = 30, novice n = 58) in the situation that were unique (ie, not contained in the scenario and that came solely from the participant).
Table 4.
Knowledge Concepts Accessed in Scenario 3
| Concept |
Experts | Novices |
| No. (%)a |
No. (%)a |
|
| Goal | 5 (16.13) | 5 (8.47) |
| Situation appraisal | — | 19 (32.20) |
| Networking | ||
| History | 10 (32.25) | 14 (23.73) |
| Observation | 3 (9.68) | 3 (5.08) |
| Palpation | 1 (3.23) | 3 (5.08) |
| Special tests | 1 (3.23) | 2 (3.39) |
| Functional tests | — | — |
| Therapeutic | 1 (3.23) | 2 (3.39) |
| Diagnostic | 5 (16.13) | 11 (18.64) |
| Prior experiences | — | — |
| Prediction | 5 (16.16) | — |
| Unknown | — | — |
% is the percentage of total statements (expert n = 31, novice n = 59) in the situation that were unique (ie, not contained in the scenario and that came solely from the participant).
Table 5.
Knowledge Concepts Accessed in Scenario 4
| Concept |
Experts | Novices |
| No. (%)a |
No. (%)a |
|
| Goal | 9 (20) | 6 (11.32) |
| Situation appraisal | 1 (2.22) | 8 (15.09) |
| Networking | ||
| History | 11 (24.44) | 7 (13.21) |
| Observation | 1 (2.22) | 2 (3.77) |
| Palpation | 1 (2.22) | 2 (3.77) |
| Special tests | 1 (2.22) | 5 (9.43) |
| Functional tests | — | — |
| Therapeutic | 2 (4.44) | 12 (22.64) |
| Diagnostic | 9 (20) | 11 (20.75) |
| Prior experiences | 3 (6.67) | — |
| Prediction | 7 (15.56) | — |
| Unknown | — | — |
% is the percentage of total statements (expert n = 45, novice n = 53) in the situation that were unique (ie, not contained in the scenario and that came solely from the participant).
Goal Concept
A goal statement was defined as a personal aim based upon what information was presented in the scenarios. Across all scenarios, statements from experts contained 25 goal concepts and novice statements contained 31 goal concepts. The following statements contain examples of this knowledge concept.
Expert:
Some of the things that jump out in my mind…looking at just reading through this were the pain that was there just existing for some time…you kind of want to look at a SLAP [superior labrum anterior-posterior] tear in terms of the dull achy or the dead arm…some other things that jump out…some type of impingement syndrome or some type of labral tear going on or you could look at the possibility of…I think a lot of this…no paresthesia, no neck pain that's not a surprise…or if he had some type of venous clot maybe DVT [deep vein thrombosis] or venous TOS [thoracic outlet syndrome] he doesn't have paresthesia but that wouldn't necessarily be associated with it.
Novice:
I want to check his AC [acromioclavicular joint] make sure he didn't have any AC issues but according to this he would not…observation was unremarkable…my only test that was positive was resistive internal rotation…biceps seems to be unremarkable…liftoff test…it doesn't state what kind of pain he was in so I'm going to need to know what kind of pain he was in…if his pain was minimal according to this…then he may have a slight strain of his subscapularis…this doesn't state the amount of pain he's in…so obviously I want to rule out that he didn't sublux when he hit the fence…so it didn't pop forward and go back in which I would expect he would be in a lot of pain and I would expect that he would have a lot more weakness if that was the case.
These examples of statements containing this knowledge concept may be reduced to the early stage of the clinical diagnosis. More specifically, the participant attempts to establish a goal or diagnostic hypothesis such as “SLAP tear,” “impingement syndrome,” “labral tear,” or “AC sprain,” and link specific conditions in the scenario to what he is now attempting to prove or disprove with evidence.
Situation Appraisal Concept
A situation appraisal concept included statements made by the participant in an attempt to make sense of what was read in the scenarios, particularly when he did not exactly know where to start with the decision and repeated information to give himself a sense of what had taken place. Across all scenarios, statements from experts contained 30 appraisal concepts and novice statements contained 74 appraisal concepts. The following statements contain examples of this knowledge concept.
Expert:
I'm thinking and what comes to my mind is throwing mechanics. Has he altered it previously due to previous bursitis and previous injury…has he changed anything and my other concern would be has he…he just hasn't done anything and he had time off and he came back and is 3 weeks into the season…what was his throwing progression like, did he actually go through a throwing interval training…did he go through a full throwing progression, you know, did he throw flat ground, did he throw off the mound, or was it something that he kind of jumped into…what was he doing previously to this…initially he's going to have some soreness and stuff depending on how hard he's getting into it.
Novice:
With this individual I'm noting that…the position first of all of the right elbow, observations unremarkable, range of motion, resisted range of motion, induced pain for internal rotation, and then a 4 out of 5 strength…
From these passages, statements such as “previous bursitis and previous injury” and “observations unremarkable” appeared to simply be repetitions of information that was contained within the scenario. As seen in the expert statement provided, some participants then processed the information, which gave some indication that they had an idea of which actions might be needed next. As seen in the novice statement, other participants continued to repeat additional information in their stream of thoughts, paused, or stopped altogether.
Networking Concept
A networking concept referred to unique statements made by the participant demonstrating that he was attempting to determine relevant history and the results of observation, palpation, special tests, and functional tests from the presented scenario. Statements from experts contained a total of 90 references to networking, whereas novice statements contained a total of 106 networking concepts. The following statements contain examples of this knowledge concept.
Expert:
I think somebody didn't do a complete evaluation on this person…even with my ankles in these kind of situations and you do your anterior drawer and your talar tilt and with the ankle especially I check both sides…I would have done my external rotation the first time…I would have done Kleiger's, palpated up anterior tibia, fibula, and up the interosseous membrane to see if…there was something…involved because just because he inverted, he could have caught something and had a high ankle sprain…and with that I teach the students about the Ottawa Ankle Rules and I check those spots every time I have an ankle injury. I go ahead and rule those out, I know that it says that he was able to weight bear within 24 hours, but what kind of thing did we find, but with the person coming in now after the fact one thing that I would look at is did he have the high ankle sprain initially and did he aggravate his syndesmosis at some point and that's…if he didn't have that stability there…where is his dull achy pain exactly located and that would give me a better idea structure wise and at this point I'd go ahead and do a full reevaluation and figure out where our deficits are coming from or he did a little too much too quickly.
Novice:
I would start if asking if they were able to continue with whatever event they were doing and obviously this athlete was able to compete 1 and 2 waited before she came to see me for an evaluation for 3 days. I would be curious if they would be able to tell me an exact point in time when the pain began or if they could tell me a movement or action that she was doing when the pain began, here she is just warming up at the time and nothing specific. With observation once again I'm looking for the 5 signs of inflammation and 3 of them are nixed so the chance that the other 2 are present are pretty good and obviously with one of those being that she reported pain. I would then palpate the area where she felt the pain. Another thing I like to do is have the athlete point with 1 finger where the pain is and in this particular situation it would not be possible due to the region where the pain is located with it being a large region.
Statements such as “I would have done a Kleiger's,” “I would have palpated,” and “I would ask,” indicated during the data reduction, coding, and analysis that participants were attempting to organize and process information about the case based upon the components of a clinical evaluation. Because an evaluation involves a history and a physical examination (ie, inspection, palpation, special and functional testing), we organized and enumerated data according to these components. The count of each individual networking component is reported by scenario and as a total in Table 7.
Table 7.
Summary of Knowledge Conceptsa
| Concept |
1 |
2 |
3 |
4 |
5 |
Total |
||||||
| E |
N |
E |
N |
E |
N |
E |
N |
E |
N |
E |
N |
|
| Goal | 5 | 7 | 4 | 10 | 5 | 5 | 9 | 6 | 2 | 3 | 25 | 31 |
| Situation appraisal | 13 | 22b | 4 | 12b | c— | 19b | 1 | 8b | 12 | 13 | 30 | 74b |
| History | 13 | 12 | 11 | 11 | 10 | 14 | 11 | 7 | 5 | 3 | 50 | 45 |
| Observation | 4 | 11b | — | — | 3 | 3 | 1 | 2 | — | 2 | 8 | 18b |
| Palpation | 2 | 1 | — | 4 | 1 | 3 | 1 | 2 | 1 | 1 | 5 | 11b |
| Special tests | 10 | 7 | 5 | 9b | 1 | 2 | 1 | 5b | 5 | 5 | 23 | 28 |
| Functional tests | — | 1 | 1 | 1 | — | — | — | — | 1 | — | 2 | 2 |
| Therapeutic | 8 | 10 | — | 5b | 1 | 2 | 2 | 12b | 2 | 5b | 13 | 34b |
| Diagnostic | 8 | 5 | 2 | 3 | 5 | 11b | 9 | 11 | 11 | 13 | 35 | 43 |
| Prior experiences | 1 | 3 | — | 1 | — | — | 3b | — | 2 | 1 | 7 | 5 |
| Prediction | 5 | 3 | 3b | 1 | 5b | — | 7b | — | 3b | — | 23b | 4 |
| Unknown | — | 4 | — | 1 | — | — | — | — | — | — | — | 5b |
Abbreviations: E, experts; N, novices.
Values reported are frequencies in each scenario.
Major difference but the data were not analyzed with statistical comparisons because of the qualitative nature and because the sample was not large enough for valid nonparametric comparisons.
No value to report.
Therapeutic Concept
A therapeutic concept was coded when a statement was made regarding the clinical evaluation and diagnosis through treatment. Because the instructions in the interview protocol asked participants to reach a clinical or differential diagnosis, their attempts to treat the condition were recorded and coded to display differences. Statements from experts contained only 13 references to these concepts, and novice statements contained 34 references. The following sample statements contain examples of this knowledge concept.
Expert:
There's really no history of any illnesses or anything like that with this…I think that…there is some tenderness along the lateral border of the scapulae and I think we'd focus in on that to see where we'd go with that. As far as treatments we'd calm his pain down and explore treatments to try and wrap him back up as far as his throwing. At this point I wouldn't categorize this as tendinitis or anything major…well I wouldn't say major, but I'd like to find out what his pain level is and find out what he'd been doing to make his pain progress in intensity and duration, especially after throwing, and if there was a specific pitch he was throwing or anything like that and neck pain…
Novice:
I guess it says he can't weight bear, but in certain things does he grimace more…in flexion in extension and any type of rotation or stress test to medial or lateral side…you may have to wait until the tenderness has gone down before you can even do anything…I guess before I would even do anything, I would wait for the tenderness to go down or a couple hours or maybe the next day.
In both excerpts, statements such as “calm his pain down,” “explore treatments,” and “wait for the tenderness to go down” fit the definition for this knowledge concept. When these statements were given during the interview process, no attempt was made to interfere with a participant's stream of thought; however, the interviewer did ask all participants to provide a clinical or differential diagnosis when none was given before ending each scenario.
Diagnostic Concept
A code of diagnostic concept was given if a statement indicated that the participant wanted the patient to be seen by a physician or if a specific statement was made regarding the need for any diagnostic medical test. Expert statements contained 35 of these references, and novice statements contained 43 references. The following statements contain examples of this knowledge concept.
Expert:
I would talk to this young lady and get her history and find out if she has had problems with the rib cage or costal cartilage before…any mechanisms for other injuries as far as being hit or doing some other activity that would cause some other injury in that area…it says the observation part here…probably to see about continuity we would just get an x-ray just to rule out anything along those lines, but based on the history and physical exam here, I would say it's probably some type of costal cartilage injury…
Novice:
So when I'm doing these tests and obviously those that are positive I'm going to lean more toward musculoskeletal…no other findings were remarkable, so I'm going to assume that if a proper evaluation was done, then I would have noted something with that. But I'm going to look at her back and I'm going to look for the spondylopathies and with that I'm going to refer for x-rays.
In these statements, the participants specifically mentioned “get an x-ray” and “refer for x-rays.” The interviewer encouraged participants to continue their stream of thoughts until they reached a clinical diagnosis or differential diagnosis. This practice was used to eliminate any assumptions during the analysis to document these statements or requests for a diagnostic test as a clinical diagnosis of a fracture.
Prior Experiences Concept
A prior experience was coded for statements that participants made regarding having experienced or seen this type of injury before. Statements from experts contained 7 references to prior experiences, whereas novice statements contained 5 of these references. The following statements contain examples of this knowledge concept.
Expert:
With this one it says he, observation revealed a deformity on the anterolateral aspect of the knee…it's still there, so this one didn't reduce and I've only seen one that didn't spontaneously reduce.
Novice:
First I would see what his…how he was pitching in regard to whether he had any change in mechanics recently. I would talk with the coach to see if he had any change control or velocity recently. That's one thing that I found to be very valuable in working with baseball. If they have a change in control, then it's usually an elbow issue, and if there is a change in velocity, then it's probably a labral issue and direct my questioning from there, and that would probably be some of the first questions that I'd ask.
In the above examples, mention of “only seen one that didn't spontaneously reduce” and “that's the one thing that I found to be very valuable in working with baseball” make specific reference to experiences that have shaped how the participant thought of the scenario.
Prediction Concept
A prediction was used to code data that suggested the participant was expressing thoughts that indicated he had specific expectations for the conditions presented in the scenario. In addition, this code was used for an attempt to provide some prognosis regarding the course of the injury or condition. Across all scenarios, statements from experts contained 23 prediction concepts, and novice statements contained only 4 prediction concepts. The following statements contain examples of this knowledge concept.
Expert:
He heard the pop, and with this particular one, the odds are that he's dislocated his fibular head, even though they say that it doesn't happen…it doesn't say what kind of stress he had…
Novice:
I guess the first thing that pops into my mind is if there is a fracture. Three weeks post…and send for an x-ray, and if that comes back negative, then continue to work on strengthening plantar flexion and eversion…depending on where you are in the season…it makes a difference if you're going to tape them up and (laughing) let them get through it.
In both examples, the participants very clearly state what they think will happen with the scenario presented.
Unknown Concept
Unknown was the code for data that clearly indicated that the participant did not know or did not have any idea regarding the conditions presented in the scenario. No unknown codes were used for experts; however, novice statements contained 5 unknown codes.
Novice:
I'm looking for…it's not supposed to get worse with rest over the 5 days that…is something scarring down is something is the condition getting worse…it's worse over 5 days, is that worse the same worse as when you are running…is it something truly getting worse…your pain scales…your symptom scores…let's see…alright. This is definitely an athlete that I would refer because watching this or reading this and going through stuff in my head…I'm not expecting to get a positive test on something and say oh here is what it is…so this is probably…definitely an athlete that I'm sending on…you hope for the best and plan for the worst.
In the statement above, mention of “it's not supposed to get worse with rest over the 5 days” indicated during the analysis that the participant had reached a point in his knowledge base where no explanation could be reached.
DISCUSSION
The purpose of our study was to describe how male certified athletic trainers in the collegiate setting represent clinical evaluation and diagnosis domain knowledge. We were particularly interested in the different knowledge concepts represented by relative experts and novices during a situational interview. The findings of our study suggest that male expert and novice collegiate athletic trainers represent their accrued knowledge differently. Across the 5 injury-evaluation tasks, experts differed from novices in how much of their knowledge focused on the history of the case and was used to make predictive statements and focused very little on appraising the situation for understanding. These results can be interpreted as differences in the organization and search-and-retrieval mechanisms of expert and novice memory structures. This interpretation suggests that experiences and deliberate practice activities may be organized to influence expert-like cognitive processes.
Previous authors35,36 of studies in medicine suggested that experts also organize their knowledge differently, thus making them more efficient and able to recognize patterns to identify illnesses and injuries faster. Our results indicate that participants with more experience were engaged in different search-and-retrieval patterns during their verbalizations, and we believe this to result from how their experience has shaped their injury-evaluation domain knowledge. This finding may be interpreted as evidence of higher-level clinical reasoning (ie, knowledge encapsulation and illness scripting) and is consistent with what has been discussed in medicine. One question that this raises is, if these higher-level decision-making and reasoning processes are a result of experience and deliberate practice activities, then can they be taught? Discussions on this topic in medical education2 suggest that multiple models of clinical evaluation should be taught to those learning in health care fields and naturalistic simulations should be used to aid in the process. However, additional studies on these methods are needed to determine the benefit they have in shaping declarative and procedural knowledge.
Findings from this study support previous ideas regarding the domain eminence, pattern recognition, depth of problem representation, and qualitative analysis of problems by experts in medicine.7 A deeper understanding of domain knowledge is also supported by the presence of more prediction concepts in the experts' protocols. These prediction concepts suggest that during their reasoning, expert athletic trainers were able to access indexed information and retrieve it in a way that allowed them to anticipate the direction of the injury state. As a result, these findings support those of previous researchers20 who claimed that novices lack these sophisticated search-and-retrieval systems. This interpretation may also assume some relationship of these findings with expert illness scripting and knowledge encapsulation.
Illness scripting assumes a focus on patient contextual risk factors to identify a problem.36 Knowledge encapsulation is the process linking patient findings to clinical concepts with less reliance on predetermined steps or networks. Experts demonstrated the use of illness scripts, which comprise (1) monitoring the conditions or constraints under which a disease occurs (ie, enabling conditions), (2) the pathophysiologic process taking place in a specific disease (ie, faults), and (3) the outcomes of the signs and symptoms of the specific disease (ie, consequences).35,36 Knowledge encapsulation is more process oriented, suggesting that a person understands the conditions presented in the history of an evaluation enough to move on to special testing. Both processes are more advanced than what is traditionally taught in athletic training education and health care education because the basic process for learning knowledge about injuries is to organize it with rule systems. These rule systems are process driven, and, in order to understand the underlying condition, all information in each step has to be present, as in the “history, inspection, palpation, special tests” model. As experience shapes what an athletic trainer knows, the athletic trainer moves beyond this and begins to cycle through illness scripting and encapsulating injury knowledge. Authors37 of literature in athletic training education have recommended that the profession examine ways to recreate this process in order to move toward the modeling of expert behavior, although more research is needed on the actual practice activities of experts and novices.
Future Research
It is still uncertain, however, whether athletic training expertise is setting specific or general. For example, can athletic trainers providing health care services for a specific patient population (with a unique set of injury or illness risks) perform superiorly outside of their settings, or are they limited to highly skilled performances only within the scope of their experiences?
Further research may need to be designed creatively to include the study of procedural knowledge in order to bridge the gap between theory, which is based on cognitive studies, and practice, which reflects procedural knowledge. Each time simulations are used to study expert and novice differences, the natural setting in which the experts excel becomes diluted. Restricting the environment may remove what makes experts excel. Future investigators may need to be creative to make simulations more real and naturalistic. The best way to accomplish this may be to have experts and novices work with real patients while protecting patients' rights.
We made no attempt to determine how these differences in expert and novice knowledge concepts were related to accuracy and a correct decision path. Future researchers should consider this when designing studies on experts and novices in athletic training. Lastly, many other documented traits differentiate experts and novices, such as speed, amount of recall from memory, perceptual ability, and focus. These traits must also be tested in the domains of athletic training so we can learn how much we can apply and benefit from the study of experts.
Limitations
Although our study may contribute to the understanding of differences between expert and novice male certified athletic trainers in the collegiate setting and raise many questions for future investigation, we must address some limitations. The goal of qualitative research is not to generalize but to identify and describe phenomena. Therefore, our findings should serve as a reference from which to raise questions for future investigations rather than a generalized description of all domain knowledge differences in athletic training.
The participants in this study were drawn from only 1 setting and only 1 National Athletic Trainers' Association district. As a result, it is possible that the responses of these 20 participants do not completely represent the college or university setting or the entire National Athletic Trainers' Association membership. In addition, the fact that all 20 participants were male may affect the interpretations of the findings. We chose not to include female athletic trainers for 2 reasons. First, there are hypothesized differences in male and female communication and ethical decision making.38,39 Second, we wanted a sample that was homogeneous to prevent unexplainable results, because previous authors18,40 have acknowledged that more evidence is needed to support a lack of gender differences in critical-thinking behaviors, so we followed the procedures cited in previous research. However, female certified athletic trainers should be considered in future investigations to provide a better understanding of all athletic trainers.
Also, we looked at relative expertise, yet the participants in the novice group were not true novices in the continuum of expert practice. Future authors should address the behaviors of athletic training students before certification. For example, it would be naïve to assume that all athletic trainers are the same and that all athletic training education programs offer the same instruction with the same progression of clinical experiences. Therefore, the behaviors of athletic training students should be investigated further and may inform how expertise is developed. Readers should determine the extent to which these findings may inform decisions for structuring education to promote expert reasoning behaviors and future research.
CONCLUSIONS
We attempted to identify the influence of experience on the diagnostic problem-solving abilities of male athletic trainers in collegiate settings. The content of their protocols verified trends showing differences in the represented knowledge of expert and novice athletic trainers (Table 8). These differences included knowledge concepts of situation appraisal, patient contextual information (ie, history), and predictive concepts. The study of athletic training expertise, and more specifically underlying knowledge constructs that are influenced by experience, may enhance what we know about how athletic trainers develop skills over time. Having a better understanding of athletic training skill acquisition may provide direction for educational standards and practices for entry-level and advanced-studies athletic training programs. Improving standards for how athletic trainers are prepared may ultimately lead to improvements in the quality of health care delivered.
Table 8.
Trends in Knowledge Representation in Athletic Traininga
| Concept |
Expert |
Novice |
| Valuation and goal setting | Few valuation concepts and similar goal-setting concepts | 20% of all concepts appraised situations and similar goal-setting concepts |
| Condition networks | More concepts on history (contextual patient factors) | Fewer history concepts but more observation, palpation, and special test concepts |
| Action plans | Focused on diagnostics to determine injury | Focused on diagnostics and therapeutics to determine injury |
| Inference capabilities | 10% of all concepts predicted some injury state | Almost none |
Summary of findings.
REFERENCES
- 1.Fuller D. Critical thinking in undergraduate athletic training education. J Athl Train. 1997;32(3):242–247. [PMC free article] [PubMed] [Google Scholar]
- 2.Ericsson KA. An expert-performance perspective of research on medical expertise: the study of clinical performance. Med Educ. 2007;41(12):1124–1130. doi: 10.1111/j.1365-2923.2007.02946.x. [DOI] [PubMed] [Google Scholar]
- 3.Ericsson KA, Charness N. Expert performance: its structure and acquisition. Am Psychol. 1994;49(8):725–747. [Google Scholar]
- 4.Ericsson KA, Smith J. Toward a General Theory of Expertise: Prospects and Limits. New York, NY: Cambridge University Press; 1991. eds. [Google Scholar]
- 5.Ericsson KA, Charness N, Feltovich PJ, Hoffman RR. The Cambridge Handbook of Expertise and Expert Performance. New York, NY: Cambridge University Press; 2006. eds. [Google Scholar]
- 6.Ericsson KA, Krampe RT, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100(3):363–406. [Google Scholar]
- 7.Chi MTH, Glaser R, Farr MJ. The Nature of Expertise. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers; 1988. eds. [Google Scholar]
- 8.Arocha JF, Wang D, Patel VL. Identifying reasoning strategies in medical decision making: a methodological guide. J Biomed Inform. 2005;38(2):154–171. doi: 10.1016/j.jbi.2005.02.001. [DOI] [PubMed] [Google Scholar]
- 9.Cimino JJ. Development of expertise in medical practice. In: Sternberg RJ, Horvath JA, editors. Tacit Knowledge in Professional Practice: Resarcher and Practitioner Perspectives. Mahwah, NJ: Lawrence Erlbaum Associates; 1999. pp. 101–120. In. eds. [Google Scholar]
- 10.Patel VL, Glaser R, Arocha JF. Cognition and expertise: acquisition of medical competence. Clin Invest Med. 2000;23(4):256–260. [PubMed] [Google Scholar]
- 11.Hobus PP, Schmidt HG, Boshuizen HP, Patel VL. Contextual factors in the activation of first diagnostic hypothesis: expert and novice differences. Med Educ. 1987;21(6):471–476. doi: 10.1111/j.1365-2923.1987.tb01405.x. [DOI] [PubMed] [Google Scholar]
- 12.Board of Certification. Role Delineation Study for the Entry-Level Certified Athletic Trainer. 5th ed. Omaha, NE: Board of Certification; 2004. [Google Scholar]
- 13.Eva KW, Hatala RM, LeBlanc VR, Brooks LR. Teaching from the clinical reasoning literature: combined case reasoning strategies help novice diagnosticians overcome misleading information. Med Educ. 2007;41(12):1152–1158. doi: 10.1111/j.1365-2923.2007.02923.x. [DOI] [PubMed] [Google Scholar]
- 14.Norman G, Young M, Brooks L. Non-analytical models of clinical reasoning: the role of experience. Med Educ. 2007;41(12):1140–1145. doi: 10.1111/j.1365-2923.2007.02914.x. [DOI] [PubMed] [Google Scholar]
- 15.Gardin FA. The “think-aloud” method to promote student modeling of expert thinking. Athl Ther Today. 2010;15(4):18–21. [Google Scholar]
- 16.Gardner G, Sexton P, Guyer MS, et al. Clinical instruction for professional practice. Athl Train Educ J. 2009;4(1):28–31. [Google Scholar]
- 17.Neibert PJ. Novice to expert practice via postprofessional athletic training education: a grounded theory. J Athl Train. 2009;44(4):378–390. doi: 10.4085/1062-6050-44.4.378. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Malasarn R, Bloom GA, Crumpton R. The development of expert male National Collegiate Athletic Association Division I certified athletic trainers. J Athl Train. 2002;37(1):55–62. [PMC free article] [PubMed] [Google Scholar]
- 19.Ericsson KA, Simon HA. Protocol Analysis: Verbal Reports as Data. Rev ed. Cambridge, MA: MIT Press; 1993. [Google Scholar]
- 20.Allen VG, Arocha JF, Patel VL. Evaluating against diagnostic hypotheses in clinical decision making by students, residents, and physicians. Int J Med Inform. 1998;51((2–3)):91–105. doi: 10.1016/s1386-5056(98)00107-5. [DOI] [PubMed] [Google Scholar]
- 21.Chi MTH, Feltovich PJ, Glaser R. Categorization and representation of physics problems by experts and novices. Cogn Sci. 1981;5(2):121–152. [Google Scholar]
- 22.Frederiksen CH. Representing logical and semantic structure of knowledge acquired from discourse. Cogn Psychol. 1975;7(3):371–458. [Google Scholar]
- 23.van Someren MW, Barnard YF, Sandberg JAC. The Think Aloud Method: A Practical Guide to Modeling Cognitive Processes. London, United Kingdom: Academic Press; 1994. [Google Scholar]
- 24.Creswell JW. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 2nd ed. Thousand Oaks, CA: Sage Publications; 2003. [Google Scholar]
- 25.Iwamoto J, Takeda T, Ogawa K, Matsumoto H. Muscle strain of the subscapularis muscle: a case report. Keio J Med. 2007;56(3):92–95. doi: 10.2302/kjm.56.92. [DOI] [PubMed] [Google Scholar]
- 26.Herickhoff PK, Keyurapan E, Fayad LM, Silberstein CE, McFarland EG. Scapular stress fracture in a professional baseball player: a case report and review of the literature. Am J Sports Med. 2007;35(7):1193–1196. doi: 10.1177/0363546506298278. [DOI] [PubMed] [Google Scholar]
- 27.Udermann BE, Cavanaugh DG, Gibson MH, Doberstein ST, Mayer JM, Murray SR. Slipping rib syndrome in a collegiate swimmer: a case report. J Athl Train. 2005;40(2):120–122. [PMC free article] [PubMed] [Google Scholar]
- 28.Kennedy MA, Sama AE, Sigman M. Tibiofibular syndesmosis and ossification. Case report: sequelae of ankle sprain in an adolescent football player. J Emerg Med. 2000;18(2):233–240. doi: 10.1016/s0736-4679(99)00201-2. [DOI] [PubMed] [Google Scholar]
- 29.Ahmad R, Case R. Dislocation of the fibular head in an unusual sports injury: a case report. J Med Case Rep. 2008;2:158. doi: 10.1186/1752-1947-2-158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Prentice WE. Arnheim's Principles of Athletic Training. 13th ed. New York, NY: McGraw-Hill; 2009. [Google Scholar]
- 31.French KE, McPherson SL. Development of expertise in sport. In: Weiss MR, editor. Developmental Sport and Exercise Psychology: A Lifespan Perspective. Morgantown, WV: Fitness Information Technology Inc; 2002. pp. 403–423. In. ed. [Google Scholar]
- 32.Patel VL, Groen GJ. Developmental accounts of the transition from medical student to doctor: some problems and suggestions. Med Educ. 1991;25(6):527–535. doi: 10.1111/j.1365-2923.1991.tb00106.x. [DOI] [PubMed] [Google Scholar]
- 33.Chi MTH. Quantifying qualitative analysis of verbal data: a practical guide. J Learn Sci. 1997;6(3):271–315. [Google Scholar]
- 34.Grbich C. Qualitative Data Analysis: An Introduction. Thousand Oaks, CA: Sage Publications; 2007. [Google Scholar]
- 35.Higgs J, Jones M. Clinical Reasoning in the Health Professions. 2nd ed. Boston, MA: Butterworth-Heinemann; 2000. eds. [Google Scholar]
- 36.Schmidt HG, Rikers RM. How expertise develops in medicine: knowledge encapsulation and illness script formation. Med Educ. 2007;41(12):1133–1139. doi: 10.1111/j.1365-2923.2007.02915.x. [DOI] [PubMed] [Google Scholar]
- 37.Geisler PR, Lazenby TW. Clinical reasoning in athletic training education: modeling expert thinking. Athl Train Educ J. 2009;4(2):52–65. [Google Scholar]
- 38.Glover SH, Bumpus MA, Sharp GF, Munchus GA. Gender differences in ethical decision making. Women Manag Rev. 2002;17(5):217–227. [Google Scholar]
- 39.Roter DL, Hall JA, Aoki Y. Physician gender effects in medical communication: a meta-analytic review. JAMA. 2002;288(6):756–764. doi: 10.1001/jama.288.6.756. [DOI] [PubMed] [Google Scholar]
- 40.Kennedy M, Fisher MB, Ennis RH. Critical thinking: literature review and needed research. In: Idol L, Jones BF, editors. Educational Values and Cognitive Instruction: Implications and Reform. Hillsdale, NJ: Lawrence Erlbaum Associates Inc; 1991. pp. 11–40. In. eds. [Google Scholar]