Assessing Physicians' Orientation Toward Lifelong Learning

Abstract

BACKGROUND

Despite the importance of lifelong learning as an element of professionalism, no psychometrically sound instrument is available for its assessment among physicians.

OBJECTIVE

To assess the validity and reliability of an instrument developed to measure physicians' orientation toward lifelong learning.

DESIGN

Mail survey.

PARTICIPANTS

Seven hundred and twenty-one physicians, of whom 444 (62%) responded.

MEASUREMENT

The Jefferson Scale of Physician Lifelong Learning (JSPLL), which includes 19 items answered on a 4-point Likert scale, was used with additional questions about respondents' professional activities related to continuous learning.

RESULTS

Factor analysis of the JSPLL yielded 4 subscales entitled: “professional learning beliefs and motivation,”“scholarly activities,”“attention to learning opportunities,” and “technical skills in seeking information,” which are consistent with widely recognized features of lifelong learning. The validity of the scale and its subscales was supported by significant correlations with a set of criterion measures that presumably require continuous learning. The internal consistency reliability (coefficient α) of the JSPLL was 0.89, and the test-retest reliability was 0.91.

CONCLUSIONS

Empirical evidence supports the validity and reliability of the JSPLL.

Medical education is a lifelong learning process that begins in medical school, extends into graduate medical education, and continues throughout a physician's professional life.^1,2 The importance of preparing students to become lifelong learners has received widespread attention by professional organizations such as the Association of American Medical Colleges,³ and the Liaison Committee on Medical Education,⁴ and developing lifelong learning habit has been a consistent recommendation in virtually all proposals for medical education reform.⁵ Lifelong learning was among 5 competencies considered as very important by more than 75% of physicians in a national survey,⁶ and has been described as an important element of professionalism.^7–12 One of the 9 Principles of Medical Ethics adopted by the House of Delegates of the American Medical Association on June 17, 2001 specified that: “A physician shall continue to study, apply, and advance scientific knowledge….”¹³

Despite the emphasis placed on physicians' lifelong learning, no universally accepted definition has been proposed.¹⁴ Lifelong learning is a complex and multidimensional concept,^15,16 as reflected in the definition suggested by the European Lifelong Learning Initiative: “Lifelong learning is the development of human potential through a continuously supportive process which stimulates and empowers individuals to acquire all the knowledge, values, skills, and understanding they will require throughout their lifetimes and to apply them with confidence, creativity and enjoyment in all roles, circumstances and environments”¹⁶ (p. 592). Facets of this broad definition, such as “human potential,”“supportive process,”“creativity,” and “enjoyment” impede empirical research because they defy measurement.

For the practical purpose of developing an operational measure of lifelong learning, based on a review of relevant literature and panel discussions in our pilot studies,¹⁷ we defined lifelong learning as “a concept that involves a set of self-initiated activities (behavioral aspect), and information seeking skills (capabilities) that are activated in individuals with a sustained motivation to learn and the ability to recognize their own learning needs (cognitition).” The 4 key concepts in this definition that have been frequently described in the literature^18–21 are in italics to underscore their importance.

Although a few instruments have been used to measure self-directed learning in the general population,^19,22,23 they are neither specific to physicians nor designed to address lifelong learning as conceptualized in this study. To the best of our knowledge, before the development of the Jefferson Scale of Physician Lifelong Learning, no psychometrically sound instrument was available to measure orientation toward lifelong learning and its empirically derived components among physicians. This study was designed to assess the psychometric properties of an instrument developed to measure physicians' lifelong learning.

METHODS

Participants

Eligible participants included 721 physicians in the Jefferson Health System, affiliated with Thomas Jefferson University Hospital and Jefferson Medical College in the Greater Delaware Valley Region around Philadelphia. The final sample included 444 physicians who responded to the survey, representing a 62% response rate including 124 (28%) women, 124 (28%) physicians practicing in primary care specialties, 320 (72%) in other specialties, 378 (88%) holding an MD degree, 35 (8%) a DO, and 18 (4%) a combined MD-PhD degree.

Instrument

The Jefferson Scale of Physician Lifelong Learning (JSPLL) was used. The JSPLL contains 19 items answered on a 4-point Likert scale (strongly disagree=1, disagree=2, agree=3, strongly agree=4). The higher the score on the JSPLL, the greater the orientation toward lifelong learning. Details about the development of its conceptual framework, the steps used to generate its items, and the results of preliminary psychometric analyses based on a small sample of 160 physicians were reported elsewhere.¹⁷

Criterion Measures for Validity Assessments

We used 26 additional items (1 global indicator of lifelong learning, 13 supplementary, and 12 checklist items) to assess validity. The global indicator of lifelong learning asked respondents to rate the extent of their orientation toward lifelong learning on a 10-point scale (1=not committed to lifelong learning, 10=a tireless advocate of lifelong learning).

The 13 supplementary items, answered on a 10-point scale (1=not true about me at all, 10=completely true about me), were used to assess the convergent validity of the JSPLL. These items addressed the following 4 areas of criterion measures determined by a factor analytic study: “intrinsic motivation” for lifelong learning (4 items, e.g., I can easily recognize my professional strengths and weaknesses), “extrinsic motivation” (2 items, e.g., learning cannot be initiated by itself, there should be an external factor to initiate learning), “research interest” (4 items, e.g., I consider myself a researcher as well as a clinician), and “information seeking skills” (3 items, e.g., I believe that the skills to surf websites to find out what's going on in medicine is important for all physicians in order to catch up with news and advances) (see Appendix A).

The 12 additional checklist items were used to assess the validity of the JSPLL by the method of contrasted groups²⁴ (p. 144). These items surveyed the respondents' professional activities that presumably require continuous learning (see Table 3). Items about the respondents' sex and specialty were also included.

Table 3.

Correlations Between Subscale Scores of the Jefferson Scale of Physician Lifelong Learning (JSPLL) and Criterion Measures

Criterion Measures	Subscales of the JSPLL
	Learning Beliefs & Motivation	Scholarly Activities	Attention to Learning Opportunities	Technical Skills in Information Seeking	Total Score
Global indicator of lifelong learning	0.50	0.42	0.37	0.29	0.53
Intrinsic motivation factor	0.52	0.29	0.39	0.23	0.48
Research activities factor	0.38	0.78	0.50	0.37	0.69
Computer skills factor	0.19	0.44	0.27	0.57	0.45
Extrinsic motivation factor	0.20	0.03	0.07	0.15	0.15

All of the correlations above .15 are statistically significant (P<.01)

Procedures

Upon the approval of the Institutional Review Board of Thomas Jefferson University, we mailed surveys to 721 physicians. We assigned a numeric code to each survey to compare respondents with nonrespondents and to match the forms in a test-retest reliability study. The cover letter explained that the code was assigned to enable follow-up while concealing respondents' personal identities. Two follow-up reminders were sent approximately 4 weeks apart. The final sample included 444 physicians (62% response rate), who returned useable surveys. Approximately 3 months later, we sent a second form to a random sample of 100 of the 444 physicians (71 responded). The cover letter informed the physicians that they had been chosen for a study of the scores' stability over time. Statistical methods included χ², Pearson correlation, principal factor analysis with varimax rotation, t-test, analysis of variance, and analysis of covariance.

RESULTS

Respondents

We compared respondents (n = 444) and nonrespondents (n = 277) to assure representativeness of the sample with respect to sex, degree (MD, MD-PhD, DO), and specialty. No significant differences were observed between respondents and nonrespondents on sex and type of degree. However, primary care physicians were less likely to respond (62% response rate) than others (72% response rate) (χ₍₁₎=7.8, P < .01).

Descriptive Statistics of the JSPLL

The frequency distribution, mean, median, standard deviation, range, quartiles, and reliability coefficients of the JSPLL are reported in Table 1. The internal consistency reliability (coefficient α) for the scale was 0.89 and the test-retest reliability was 0.91.

Table 1.

Frequency Distribution of Scores, Percentile Ranks, Descriptive Statistics, and Reliability Coefficients for the Jefferson Scale of Physician Lifelong Learning

Score	Frequency		Percentile Rank
<30	1		<1
31 to 40	1		<1
41 to 43	8		2
44 to 46	8		4
47 to 49	20		8
50 to 52	35		16
53 to 55	48		27
56 to 58	50		38
59 to 61	49		49
62 to 64	57		62
65 to 67	49		73
68 to 70	46		84
71 to 73	45		94
74 to 76	27		100
Descriptive Statistics
Median (50th percentile)		62.00
Standard deviation		8.60
25th percentile		55.00
75th percentile		68.00
Possible range		19 to 76
Actual range		19 to 76
Coefficient α reliability		0.89
Test-retest reliability*		0.91

^*Thetest-retest reliability coefficient was calculated for 71 physicians who completed the scale for a second time within approximately a 3-month interval between testing

Item Statistics and Item-Total Score Correlations

The mean item scores for the 19 items ranged from a high of 3.86 (SD=0.37) for “Lifelong learning is a professional responsibility of all physicians” to a low of 2.36 (SD=1.1) for “I frequently publish articles in peer-reviewed journals.”

Item-total score correlations were all positive and statistically significant (P < .01), ranging from a low of r = .37 for “Searching for the answer to a question is, in and by itself rewarding” to a high of r = .72 for “I actively conduct research as a principal investigator or co-investigator.” The median of the item-total score correlations was r = .60.

Construct Validity

The underlying components of the JSPLL were examined by factor analysis. Four factors with eigenvalues greater than 1 accounted for 60% of the total variance. Each factor can be considered as a subscale of the JSPLL. Rotated factor coefficients and summary results of factor analysis are reported in Table 2.

Table 2.

Rotated Factor Matrix of the Jefferson Scale of Physician Lifelong Learning

Items		Factors
		1	2	3	4
1.	Rapid changes in medical science require constant updating of knowledge and development of new professional skills	0.76	0.15	0.10	0.05
2.	I recognize my need to constantly acquire new professional knowledge	0.71	0.15	0.21	0.13
3.	Lifelong learning is a professional responsibility of all physicians	0.69	0.04	0.05	0.00
4.	I believe that I would fall behind if I stopped learning about new developments in my profession	0.69	0.18	0.11	0.12
5.	One important mission of undergraduate medical education is to develop the habit of lifelong learning	0.66	0.24	0.03	0.00
6.	I enjoy reading articles in which issues of my professional interest are discussed	0.66	0.02	0.23	0.10
7.	I always make time for self-directed learning, even when I have a busy practice schedule and other professional and family obligations	0.58	0.10	0.28	0.06
8.	Searching for an answer to a question is, in and by itself rewarding	0.53	−0.03	−0.08	0.32
9.	I review professional journals every week	0.46	0.17	0.31	0.15
10.	I actively conduct research as a principal investigator or a co-investigator	0.13	0.86	0.10	0.20
11.	I give on average at least one presentation at professional meetings in every given year	0.17	0.85	0.16	0.10
12.	I frequently publish articles in peer-reviewed journals	0.15	0.84	0.20	0.09
13.	I routinely exchange e-mail messages with my colleagues	0.10	0.60	0.31	0.39
14.	I routinely attend grand rounds offered in my field regardless of whether a certificate for attendance is offered	0.08	0.14	0.79	0.15
15.	I routinely attend annual meetings of professional medical organization	0.19	0.40	0.59	0.02
16.	I attend professional development programs regardless of whether CME credit is offered	0.24	0.46	0.56	−0.05
17.	I take any opportunity to gain new knowledge/skills that are important to my profession	0.44	0.08	0.54	0.28
18.	My preferred approach in finding an answer to a question is to search the appropriate computer databases	0.10	0.16	0.10	0.84
19.	I search computer databases (e.g., MEDLINE) to find out about new developments in my field	0.20	0.26	0.14	0.80
Eigenvalue		6.7	2.3	1.3	1.1
% Variance		36	12	7	5

Items are listed by the order of magnitude of the factor coefficients within each factor. Values greater than 0.45 are in bold. Items were answered on a 4-point Likert-type scale (1=strongly disagree, 4=strongly agree)

Factor 1: A construct involving “Professional learning beliefs and motivation.”

Factor 2: A construct involving “Scholarly activities.”

Factor 3: A construct involving “Attention to learning opportunities.”

Factor 4: A construct involving “Technical skills in information seeking.”

The first subscale (36%) was entitled “professional learning beliefs and motivation.” Nine items had coefficients greater than 0.45 on this subscale. Coefficient α for this subscale was 0.79. The second subscale (12%) is related to a construct involving “scholarly activities.” Five items had coefficients greater than 0.45 on this subscale (coefficient α=0.89). The third subscale (7%) was entitled “attention to learning opportunities.” Four items had coefficients greater than 0.45 on this subscale (coefficient α=0.74). The final subscale, accounting for 5% of the variance, was entitled “technical skills in seeking information.” Two items had a high loading on this subscale (coefficient α=0.82).

We calculated subscale scores by summing scores for items with the highest factor coefficients (reported in bold in Table 2). Correlations among the 4 subscale scores ranged from a low of 0.36 (between “professional learning beliefs and motivation” and “technical skills in information seeking” subscales) to a high of 0.64 (between “professional learning beliefs and motivation” and “attention to learning opportunities” subscales).

Convergent Validity

The convergent validity of an instrument is addressed by examining correlations between scores on the instrument and conceptually relevant criterion measures²⁴ (p. 156). We found a significant correlation between JSPLL total scores and the global indicator of lifelong learning (r = .53, P < .01). Correlations between JSPLL subscale scores and the global indicator of lifelong learning ranged from a low of 0.29 (for the “technical skills in information seeking” subscale) to a high of 0.50 (for the “learning beliefs and motivation” subscale). Correlations with the 4 criterion measures reported in Table 3 indicate that the total JSPLL scores correlated significantly with all 4 criterion measures. The highest correlation was obtained for “research activities” (r = .69) and the lowest for “extrinsic motivation” (r = .15). Correlations between JSPLL subscale scores and the 4 criterion measures ranged from a high of 0.78 between the “scholarly activities” subscale and the “research activities” criterion measure to lows of 0.03 and 0.04 between the “extrinsic motivation” criterion measure and the JSPLL of “scholarly activities” and “attention to learning opportunities,” respectively.

Validity by the Method of Contrasted Groups

One approach to studying the validity of an instrument is to determine whether the average scores obtained by contrasting groups are in the expected direction.²⁴ Two groups of physicians were compared on the total and the 4 subscale scores of the JSPLL. Physicians who reported that they had been involved during the past 5 years with a particular professional activity presumably requiring continuous learning were contrasted to those who were not involved in the activity. The 12 professional activities, the number of physicians (involved with the activity=yes, not involved=no), and the effect sizes of the differences between the 2 groups are presented in Table 4.

Table 4.

Effect Sizes^† of Group Differences on the Scores of the Jefferson Scale of Physician Lifelong Learning by Professional Activities, Comparing Physicians Who Were Involved with the Activity and Those Who Were Not Involved

Professional Activities	Involvement		Effect Sizes
	Yes	No	Professional Beliefs and Motivation	Scholarly Activities	Attention to Learning Opportunities	Technical Skills	Total Score
1. Published article(s) in professional journals	299	144	0.48**	1.4**	0.67**	0.56**	1.1**
2. Presented paper(s) before national meetings	243	200	0.34**	1.5**	0.79**	0.52**	1.1**
3. Collaborated in the conduct of research studies or clinical trials	306	137	0.42**	1.3**	0.71**	0.61**	1.0**
4. Received a grant for research or training	146	296	0.67**	1.4**	0.92**	0.69**	1.2**
5. Received professional awards or honors	203	238	0.71**	1.1**	0.79**	0.61**	1.0**
6. Held office in national professional organizations	103	339	0.55**	1.3**	0.71**	0.38**	0.97**
7. Served on professional committees (hospital, society)	327	115	0.51**	0.92**	0.67**	0.54**	0.82**
8. Served as an editor, or on the editorial board of a professional journal	97	346	0.67**	1.3**	0.92**	0.48**	1.1**
9. Served as a reviewer for a professional journal	220	223	0.70**	1.5**	0.96**	0.59**	1.2**
10. Shared in developing medical/surgical procedures, instruments, drugs or techniques described in the literature	123	320	0.61**	1.3**	0.97**	0.50**	1.1**
11. Presented patient education/research findings in public media, or community groups	216	221	0.45**	1.0**	0.71**	0.39*	0.77**
12. Been involved in teaching medical students or residents	413	26	0.54**	1.1**	0.89**	0.63**	1.0**

^*P<.05

^**P<.01

^†Effect sizes were calculated based on this formula: (Mean_yes−Mean_no)/pooled SD

Results of the t-tests indicated that the differences between the 2 groups of physicians were statistically significant (P < .01) on all 12 professional activities. Effect sizes around 0.25 or less can be considered small and clinically negligible, those around 0.50 are moderate, and those around or larger than 0.75 are clinically important.^25,26 Based on these guidelines, none of the effect sizes reported in Table 4 can be considered negligible, and most are clinically important.

Gender Differences

We compared the total JSPLL scores for men (M = 61.7, SD=8.8) and women (M = 59.3, SD=8.1). Although the difference was statistically significant (t₍₄₄₂₎=2.59, P < .01), the effect size estimate was small and negligible (d=0.28), indicating that the sex difference could not be considered as clinically important.

Specialty

Primary care physicians (n = 124) in family medicine, general internal medicine, and general pediatrics were compared with other specialists (n = 320) on the total scores of the JSPLL. Results indicate that other specialists scored higher (M = 63.0, SD=8.2) than the primary care physicians (M = 56.1, SD=7.7) (t₍₄₄₂₎=8.12, P < .01), effect size=.87. No change in patterns of findings was noticed after statistical control for sex by using analysis of covariance.

Type of Degree

We compared the JSPLL scores of 3 groups of physicians with MD (n = 378), MD-PhD (n = 18), and DO (n = 35) degrees by using 1-way analysis of variance. Results showed that physicians with MD-PhD degrees obtained the highest mean score (M = 66.9, SD=7.7), followed by those with an MD degree (M = 61.2, SD=8.5). The lowest mean score was obtained by those with a DO degree (M = 56.2, SD=8.6) (F_{(2, 437)}=10.1, P < .01). The pattern of findings did not change when we used analysis of covariance to control for the confounding effect of sex.

DISCUSSION

The findings of this study provide convincing psychometric support for the JSPLL. Its content validity was supported by 2 pilot studies during the initial stages of test development.¹⁷ The positive item-total score correlations found in this study confirm that each item was scored in the proper direction, and the magnitude of correlations indicates that each item contributed significantly to the total score.

The 4 subscales identified in this study are conceptually relevant to the components of lifelong learning described by others.^{11,19–21,27} The scale appears to measure orientation toward, and scholarly outcomes of lifelong learning. However, we realize that 1 important outcome of physician lifelong learning that does not lend itself to measurement by physicians' self-report is patient outcome. We believe that it is important to incorporate this element in the assessment of physician lifelong learning by obtaining relevant information from peers, patients or clinical charts for a more comprehensive assessment. Despite this limitation, the identification of the 4 subscales suggests that lifelong learning is a multidimensional concept.²⁸ They are also consistent with the competencies of self-directed learning such as skills for information retrieval, motivation and self-initiation, attention to learning opportunities and scholarly activities,²⁰ and identification of learning needs,²⁷ thus providing support for the construct validity of the scale. The notion that professionals must have a high degree of intrinsic motivation for learning to maintain competence^29,30 is reflected in the “professional beliefs and motivation” subscale, and a desire to learn and to initiate self-study, described as facets of lifelong learning,²⁹ is reflected in the “professional learning beliefs and motivation” and “attention to learning opportunity” subscales. The “technical skills in information seeking” subscale is consistent with the notion that information technology, advanced telecommunications, and the Internet are major vehicles for pursuing lifelong learning.^30,31 This finding is also in agreement with the guidelines proposed in the Medical School Objectives Project to ensure that medical school graduates demonstrate the ability to retrieve information from electronic databases and other resources for solving problems relevant to the patient³ (p. 7).

The 4 subscales are similar to the factors found in our preliminary study with 160 physicians.¹⁷ For example, the “professional beliefs and motivation” subscale in the present study corresponds to the 2 factors of “motivation” and “self-initiated learning” of the preliminary study. The “scholarly activities” subscale corresponds to the “research endeavor” factor, the “attention to learning opportunity” subscale corresponds to the “need recognition” factor, and the “technical skills in seeking information” subscale corresponds to the “technical skills” factor extracted in the preliminary study.

The pattern of correlations between the subscale scores and the 4 criterion measures suggests that each subscale of the JSPLL yields the highest correlation with the most conceptually relevant area of the criterion measure, providing support for the convergent validity of the subscales. The findings by the method of contrasted groups (Table 4) suggest that scores of the JSPLL yield statistical and practical significant link to behavioral manifestation of continuous learning (e.g., inventions, research activities, and appearance on public media), thus providing additional support for the validity of the JSPLL.

The difference between the mean scores for primary care physicians and for other specialists could be explained by our previous findings³² showing that specialists were more likely than generalists to be involved in research and that they produced more publications, activities that are linked to continuous learning. Comparisons of the 2 groups in the present study confirmed that specialists reported more involvement with the professional activities listed in Table 4. The difference between the mean scores in favor of those with MD-PhD degrees could be explained in part by considering the research training of this group that increased the likelihood of involvement with continuous learning activities. This speculation was confirmed by observing that those with MD-PhD degrees reported more involvement with the professional activities listed in Table 4 than the other physicians.

The support for the validity of the JSPLL and its subscales provided in this study, and the magnitude of the internal consistency and test-retest reliabilities of the scale and subscales confirm that the scale and its subscales are psychometrically sound. To the best of our knowledge, the JSPLL is the first instrument developed specifically to measure physicians' orientation toward lifelong learning. This brief, self-administered instrument has potential utility and numerous implications in medical education research.

FUTURE RESEARCH

Although the psychometric evidence reported in support of the JSPLL is convincing, further research is needed with a representative national sample of physicians in different specialties (e.g., from the American Medical Association's Physician Masterfile) for further psychometric analyses and for the construction of norm tables with score distribution and percentile ranks for subgroups of physicians. Also, it is important to determine whether the factor structure of the JSPLL can be maintained for physicians in different specialties and practice settings.

The social desirability response style may jeopardize the validity of the scores in self-report instruments. We attempted to address the issue by using a single item, “I am known by my patients as a physician who loves poetry,” as a proxy measure for social desirability response style. Although we found no relationship between responses to this item and scores of the JSPLL, the issue of the possible influence of social desirability bias on test scores deserves further research attention.

Additional research is also needed to examine the validity of the JSPLL scores as predictors of board certification, citation rate for publications, clinical outcomes, patients' satisfaction, and malpractice claims. Supported by an invitational grant from the NBME Stemmler Medical Educational Research Fund, we are continuing our research to generate 2 subtests of the JSPLL. One will be applicable to physicians who are full-time clinicians and are not involved in research and teaching activities. The other will be applicable to academician-clinician physicians involved in teaching or research as well as clinical activities. To enhance our understanding of physician lifelong learning, we also plan to examine the antecedents and consequences of lifelong learning for physicians who are full-time clinicians and their academic counterparts.

Supplementary Material

The following supplementary material is available for this article online at http://www.blackwell-synergy.com