Abstract
BACKGROUND:
The Journal Club is an excellent platform for participants to engage in the critical evaluation of articles and the extraction of evidence to support evidence-based nursing practices. The study aims to develop and validate a reliable instrument, the Nursing Journal club perception scale (NJCPS), for assessing the perceived educational value and the perception of virtual journal club experiences.
MATERIALS AND METHODS:
The current study is a scale development and validation study. The scale-level content validity indices (S-CVI) and item-level content validity indices (I-CVI) were assessed by eight experts. Eleven experts from various nursing fields evaluated the instrument regarding its relevance, clarity, meaningfulness, and completeness. Finally, the scale was introduced to 90 clinical nurses from two facilities to assess the internal consistency during the period of March- April 2022.
RESULTS:
The content validity of the scale S-CVI/average and S-CVI/UA was 0.97 and 0.86, respectively, which indicates adequate relevance of the questionnaire content. Principal component analysis indicated that the construct validity of the perceived educational value and the perception of the virtual journal club domain was 67.8% and 66.5%, respectively. In the two domains of the perception of educational value (supporting clinical practice and supporting research), Cronbach's Alpha was 0.93 and 0.91, respectively. Similarly, for the two domains of virtual JC (learning experience and benefits of virtual journal club)), Cronbach' Alpha was 0.95 and 0.74, respectively.
CONCLUSIONS:
The NJCPS tool is a valid and reliable scale to measure the educational value and virtual journal club experience of the participants from various fields of healthcare.
Keywords: Educational, evidence-based practice, journal, journal club, perception, virtual
Introduction
Journal Club (JC) is one of the important pedagogies used to discuss the major findings, strengths, and limitations of published research among various healthcare workers. The common framework of the JC provides the didactic discussion and knowledge base for the participants and is an excellent way to learn and practice evidence-based medicine.[1] Nursing JCs are excellent learning opportunities that can be applied by both students and clinical nurses to facilitate evidence-based care and improve patient outcomes. Moreover, JCs are an established method for increasing exposure to different types of research methods and critical appraisal skills among nurses. Participation of clinical nurses in JCs is vital to the successful implementation of evidence-based practice in nursing care. Nursing JCs offer excellent opportunities for students and clinical nurses to facilitate evidence-based care and improve their patients' outcomes. Nurses can also gain exposure to different types of research methods and critical appraisal skills through nursing JCs. It is also evident that clinical nurses participating in JCs can enhance their research skills, interest, critical appraisal skills, reading habits, and confidence.[2,3,4,5,6,7,8]
Nowadays, JCs are innovative to ensure that the tasks of searching, identifying, and appraising relevant literature provide a supportive environment where healthcare practitioners can improve their knowledge of research methodologies, increase confidence in critiquing the articles, share their clinical experiences, and evaluate current practices with colleagues while focusing on translating research evidence into their clinical area.[9] JC is a creative way for nurses to learn about EBP and research, and it enables them to synthesize the evidence. The recent publication about barriers to evidence-based practice reveals that the identification of sources of literature and deficit knowledge in the synthesis of evidence were the major hindering factors to implementing evidence-based practice.[10]
The nursing JC was launched in 2015 across all facilities under the corporation. The primary purpose of nursing JCs is to develop scholarly reading habits, critically appraise research studies, and apply research findings to clinical practice to improve patient outcomes. During the emergence of the pandemic, JC switched from using traditional face-to-face methods to a virtual platform using the Microsoft team. As an initiative to move forward with the pandemic, JC activities were conducted online to prepare novice nurses and update clinical nurses on evidence-based practice.
A systematic review was conducted to synthesize the evidence of the educational benefits of an online JC.[11] Many studies attempted to assess nurses' attitudes toward JCs using self-structured scales, and the evidence generated by these studies cannot be generalized due to a lack of standard scales. There is currently a lack of measurement scales constructed to evaluate the perceived educational values of the nursing JC. Such a scale would facilitate the evaluation of educational values and serve as a guide for clinical nurses to reflect on their JC activities. In this context, assessing the perception of the nurses regarding the educational values of the JC and the experience of virtual JC is crucial. Therefore, the authors aim to develop and validate an instrument, the Nursing Journal Club Perception Scale (NJCPS), for assessing the self-reported perception of the educational value of the nursing JC and the experience of a virtual JC.
Materials and Methods
Study design and setting
This is a tool development, and validation study utilized a cross-sectional research design. The present study was conducted on clinical nurses in two public hospitals in Qatar to develop and validate a scale to assess their JC activities.
Study participants and sampling
The data was collected between March and April 2022. For the validation of the questionnaire, convenience sampling was employed to select 90 respondents from the same population as the main study, and the sample size was sufficient according to Bryant and Fox[12,13]; for every one item, a minimum of three responses are required.[14] The study enrolled only those participants who had experienced face-to-face and virtual JC activities. The participants who did not attend any JC activities in the last year were excluded from the survey since virtual JC activities started during the pandemic period.
Data collection tool and technique
The study was conducted in two hospitals in the public health sector in Qatar. The first phase (generating the item pool), based on the items generated through the literature review, was discussed with educators and clinical nurses using the convenient sampling method between November and December 2021. The second phase of the study (evaluating the quality of the items: content validity) was assessed by eight experts in the fields of nursing and scale development. A content validation form was used to assess the quality of the items.
In the third phase (testing the scale's reliability and validity), the data was collected from the 90 clinical nurses who agreed to participate in the survey. The information sheet containing the details of the study along with the survey link was disseminated to the nurses in the selected hospitals through the official email. To examine the test–retest reliability of the scale, we collected the same data from 90 clinical nurses who participated in the initial survey after two weeks. The average time required to complete the survey was 15–20 min for each participant.
Phase I: Instrument development
This study followed the ten steps and three-phase model of scale development and validation by Alenka.[15] The first phase, the theoretical importance and existence of the construct, includes three steps: content domain specification, item pool generation, and content validity generation. The second phase deals with the representativeness and appropriateness of data collection, consisting of four steps: questionnaire development and evaluation; translation and back translation; pilot study; and finally, sampling and data collection. The third phase regards the statistical analysis and statistical evidence of the construct with three steps, including dimensionality assessment, reliability assessment, and construct validity assessment (convergent validity and divergent validity).
The first step in scale development regarded the content domain specification since developing a new construct begins with defining its domain.[15] In the second step, a pool of potential sample items for the new scale was extracted by an in-depth literature review of specific research articles and interviews with experts in nursing education and research.[16] The literature search was mainly conducted in PubMed, EMBASE, and OVID databases to collect information covering the list of statements related to NJCPS. Mesh terms and free-text terms such as “journal club” AND “nurses”, “journal club” AND “perception”, “journal club” AND “virtual”, “journal club” AND “education” were used for this purpose. The articles published in the English language and studies used to assess the JC activities of multidisciplinary healthcare professionals were included. This study excluded the gray literature, editorials, expert reviews, and opinion articles.
The initial phase of the scale consists of 36 items, which were reduced to 29 after discussion with the experts in the field, considering the possibility of duplication of items. The items covering similar ideas were merged into a single question. All the questions on the scale are fixed with five-level responses on the Likert scale. Each item indicates “strongly disagree”, “disagree”, “neutral”, “agree”, and “strongly agree”, respectively. All the items on the scale consisted of positive phrasing statements. The total score is calculated by the sum of all item scores, and higher scores in the perception domain indicate that a JC possesses high educational value and a positive perception of the virtual JC. The readability analysis was conducted by using the Flesch Reading Ease score and Kincaid Grade Level to measure the understandability of the questionnaire and for spelling and grammar checking by using Grammarly software.
Coding
The questionnaire consists of two domains about the JC activities, namely, “perception of educational value” and “perception of virtual journal club,” and each item in the questionnaire is related to both domains. A five-level bipolar Likert scale was used; “Strongly disagree,” “disagree,” “neither agree nor disagree,” “agree,” and “strongly agree” were coded as “−2,” “−1,” “0,” “+1,” and “+2,” respectively.[17] All items in the questionnaire were positively keyed statements. The domain score was calculated by summing up all items, and it was named a specific domain index [Supplementary Table 1].
Face validity
The face validity of the questionnaire was assessed by eleven experts from the clinical area, including five clinical nurses, three senior educators, two professors of nursing college, and one clinical pharmacist. The participants were requested to evaluate each item in the questionnaire with respect to relevancy, suitable words, common mistakes like confusing and paired questions, ambiguity, and grammar using 5-point Likert scale ranging from 5 (highly important) to 1 (not important at all). The impact score was assessed by multiplying the frequency (%) with importance.[18,19]
Content validity
Expert teams use various approaches to determine content validity, such as content validity ratios and content validation forms. The current study used content validation forms to establish the validity, which included two methods: qualitative and quantitative. The qualitative assessment of the scale was assessed by eight experts from the clinical area and scale development, including senior educators, professors of nursing college, nurse researchers, and clinical pharmacist. The questionnaire was assessed for its accuracy, clarity of language, understandability, completeness, comprehensiveness, and communication of a clear and nonleading message to the reader.
In the quantitative evaluation of the scale, content validity indices were calculated based on eight experts' ratings of each item for relevance (1) not relevant or not clear, (2) somewhat relevant/item needs some revision, (3) quite relevant/clear but needs minor revision, and (4) highly relevant/very clear. Four experts are working in the nursing education field, and the other four are experts in scale development and validation. Due to COVID restrictions, we had to use the non-face-to-face method, an online content validation form sent to the expert's email address with clear instructions regarding the scoring process and advising them to mark according to their expert opinion.
There are two ways to calculate the forms of content validity index (CVI), which include the CVI for individual-level CVI (I-CVI) and the CVI for scale (S-CVI). The S-CVI can be calculated by two methods; one is the average of the I-CVI scores for all items on the scale (S-CVI/Ave) and the second method is the proportion of items on the scale that passed the relevance test by all experts by three or four scores (S-CVI/UA). We used the individual level content validity (I-CVI), scale level content validity index (S-CVI) with the average method, and S-CVI with the universal average method (UA).[20] The experts were requested to provide a written comment to improve the relevance of items in each domain. All comments and suggestions from the experts were taken into consideration, like grammar and clarity. The items in each domain were refined accordingly, which could be used for the study.
Construct validity
Principal component analysis (PCA) was used to cluster items into common factors that interpret each component according to the items loaded on it and summarise the items into a small number of factors (latent variables) for assessing factor loading. The number of extracted factors, whose sum should equal the number of items exposed to component analysis, was reflected in the initial total eigenvalue. The greater the component contribution, the higher the eigenvalue of loading. Each factor explained a percentage of variance, and the cumulative percentage represented the component's total variance when added to the previous component. To maximize the total variance, a Varimax rotation was used. The total rotation sum of squared loading (squared correlation between variables and components) describes the variance attributed to each factor following rotation. Another tool for determining how many elements to keep and when the curve begins to flatten was the scree plot graph. The eigenvalues were plotted on the y-axis, while the number of components was plotted on the x-axis. For each component analysis, the Kaiser–Meyer–Olkin (KMO) values were used to determine sample adequacy [Supplementary Figures 1 (218.6KB, tif) and 2 (218.6KB, tif) ].
Reliability
The test-retest reliability of the item was evaluated by presenting the same questionnaire to the same respondents twice in a 15–18 day interval.[21,22] We also calculated the intra-class coefficient (ICC) for the test-retest measures.
Ethical consideration
The Ethical approval was obtained from the IRB of the Medical Research Center (MRC), protocol # MRC-01-21-498, from August 11, 2021 to 2022 period. Participation in the study was voluntary, and the identity & confidentiality of the subjects was maintained throughout the study period.
Consent
Oral consent was obtained from all the participants, and an information sheet covering all relevant information of the study was shared with the participants. The confidentiality of the participants including their identity and data was maintained throughout the study.
Statistical analysis
The readability analysis was assessed using the Flesch Reading Ease score and Kincaid Grade Level of the questionnaire using Microsoft Office. The frequency and percentages of demographic factors were calculated. The questionnaire's relevance was tested using content validity. PCA was employed to retail the meaningful variables from the factor structure, and confirmatory factor analysis was performed to verify model-data fit, convergent validity, and discriminant validity. To assess the domain structure, construct validity was measured using PCA with the component approach and the Varimax rotation method.[23] To ensure sampling adequacy, a KMO value of 0.8 or above was used. The Kaiser criterion was used to identify factors with an eigenvalue of 1, and a scree plot was used to show descending variances for factor extraction in the form of a graph. In component analysis, eigenvalues of 1 and above are regarded to explain at least the same amount of variance as a single variable. For internal consistency, Cronbach's coefficient was utilized to determine the homogeneity of question items in each domain index. Internal consistency for the questionnaire is defined as a coefficient of 0.7 or above.[24] Intraclass correlation was used to construct each domain score as an index variable at the pre-and post-level.[25] A significant level was defined as a P value of 0.05 (two-tailed). The analysis was carried out using the STATA 17.0 statistical tool.
Results
Ninety clinical nurses were enrolled in the survey. The participants were above 25 years of age, and the majority belonged to the 35-44-year age group (53.33%). The male-to-female ratio was 1:8, 63.33% of participants in the survey had a bachelor's degree in nursing, and 23.3% held a master's degree in nursing. Most of them, 33 (36.67%), have been working for around 5–10 years, and 21 (23.3%) of them have less than 5 years of clinical experience. Registered nurses (55.56%) and charge nurses (14.44%) accounted for the majority of participants [Table 1]. The mean and SD of the perceived educational value and virtual journal experience of the participants were 18.73 ± 7.68 and 12.17 ± 9.33, respectively.
Table 1.
Sociodemographic and professional characteristics of study participants
| Characteristics | Frequency (%) |
|---|---|
| Sex | |
| Female | 80 (88.89) |
| Male | 10 (11.11) |
| Age in years | |
| 25–34 | 24 (26.67) |
| 35–44 | 48 (53.33) |
| 45–54 | 17 (18.89) |
| >55 | 1 (1.11) |
| Highest Qualification | |
| Diploma | 10 (11.11) |
| BSN | 57 (63.33) |
| MSN | 21 (23.33) |
| PhD | 2 (2.22) |
| Years of Experience in HMC | |
| <5 | 21 (23.33) |
| 5–10 | 33 (36.67) |
| 11–15 | 11 (12.22) |
| >15 | 25 (27.78) |
| Current enrolment in academic activities | |
| Yes | 22 (24.44) |
| No | 68 (75.56) |
| Current role in HMC | |
| Registered nurse 50 (55.56) | |
| Registered Midwife | 5 (5.56) |
| Charge nurse | 13 (14.44) |
| Head nurse | 6 (6.67) |
| Clinical midwife specialist | 2 (2.22) |
| Educator | 9 (10.00) |
| Others | 8 (5.56) |
The readability statistics were calculated for the questionnaire to determine the understandability of the questions to the readers. The Flesch Reading Ease Score was 38, which indicated that it is best understood by university graduates. The Flesh–Kincaid Grade Level of the questionnaire was 11.7; this also indicates that the English language used in the questionnaire could be easily understood by 12-grade students. Since the users of this questionnaire will have a higher education than the 12th grade, the words used in the questionnaire will be readable.[26] During face validity, eleven participants evaluated the items using a 5-point Likert scale. The scores of all questionnaires were collected and analyzed. The impact score was greater than 1.5, which is considered acceptable.
The comprehensiveness and representativeness of the questionnaire were used to measure the content validity. A 4-point CVI was used to evaluate the content in the form of S-CVI/Average and S-CVI/UA indices on behalf of the judgment of eight experts in the field. The current study showed a value of 0.97 for S-CVI/Average and a value of 0.86 for S-CVI/UA, whereas the total agreement was 25 for 29 items, suggesting adequate relevance to the current questionnaire.
Construct validity was calculated separately for two different domains. The first domain contained the perceived educational value of the JC, and the second domain contained the perception of the virtual journal club. The perceived educational value contained information regarding the influence of JC in clinical practice, research involvement, implementation of evidence-based practice, and enhancing patient outcomes with both platforms (face to face and virtual). The virtual latent variable contained information regarding the experience of virtual JC, time utilization, social interaction, and motivation related to research activities on the virtual platform.
PCA was used for determining the construct validity for each domain (perceived education value of JC and perception of virtual JC) separately. KMO values were 0.91 and 0.90, respectively. The total variance explained for the perception of educational value and virtual JC domains was 67.8% and 66.5%, respectively [Tables 2 and 3]. We found two components after fitting the PCA for perceived educational value. The first component contained the question related to supporting clinical practice, and the second one related to supporting research. For the virtual journal club, the first component represents the learning experience, and the second component contains the items related to the benefits of the virtual journal club [Tables 2 and 3].
Table 2.
Principal component analysis of perception: Construct validity
| Component | Initial eigenvalues | Variance | Difference | Proportion | Cumulative |
|---|---|---|---|---|---|
| Supporting Clinical Practice | 9.08658 | 6.53666 | 2.90064 | 0.4358 | 0.4358 |
| Supporting Research | 1.0861 | 3.63602 | 0.2424 | 0.6782 |
Table 3.
Principal component analysis of virtual: Construct validity
| Component | Initial eigenvalues | Variance | Difference | Proportion | Cumulative |
|---|---|---|---|---|---|
| Learning experience | 8.22835 | 7.99808 | 6.67332 | 0.5713 | 0.5713 |
| Benefits of virtual journal club | 1.09449 | 1.32476 | 0.0946 | 0.6659 |
We did confirmatory analysis separately for both domains (perception of educational value and virtual JC). The confirmatory analysis was performed on 15 questions for the perception of the educational value of JC, with two components named as supporting clinical practice and supporting research.
The Chi-square test of goodness-of-fit yielded a significant value; χ2= 265.16 (df = 89), ratio of Chi-square to degrees-of-freedom (χ2/df = 3) (acceptable if χ2/df <5), Root mean square error of approximation (RMSEA) = 0.149 and Standardized Root Mean Squared Residual (SRMR) = 0.068. Additionally, the value of Comparative Fit Index (CFI) = 0.84, Nonnormed Fit Index (NNFI) = 0.7840, Parsimony Normed Fit Index (PNFI) 0.843, and Incremental Fit Index (IFI) = 0.845 showed moderate fit of the model. All the factor loadings yielded a value of >0.70, indicating no items needed to be removed. The highest factor loading was for item Q14 = 1.23, whereas the lowest was for Q11 = 0.85. The results suggest that the educational value of JC construct validity was good enough to use the questionnaires for the survey.
The separate confirmatory analysis was performed on 14 questions for the virtual JC. The Chi-square test of goodness-of-fit yielded a significant value: χ2= 204.10 (df = 77), ratio of Chi-square to degrees-of-freedom (χ2/df = 2.7) (acceptable if χ2/df <5), RMSEA = 0.136, and SRMR = 0.060. Additionally, the value of CFI = 0.865, NNFI = 0.802, PNFI 0.865, and IFI = 0.867 showed moderate fit of the model. All the factor loadings yielded a value of >0.50, except one question vir12 had a factor loading 0.40, indicating low factor loading. The highest factor loading was for Item Vir5 = 1.20, whereas the lowest was for Vir12 = 0.40. The results suggested that the virtual JC construct validity was good enough for validity.
Convergent and divergent (discriminant) validities was examined through the correlation matrix [Table 4]. Elements on the diagonal (correlations between an item and the rest score of its dimension) are displayed in Table 4. Twenty-eight out of twenty-nine items (96.6%) have a correlation coefficient with the score of their own dimension greater than 0.4 and 25/29 items (86.2%) have a correlation coefficient with the score of their own dimension greater than those computed with other scores.
Table 4.
Correlation matrix Internal consistency and test–retest validity of questionnaire domain indices
| Item | Perceived Educational value of Journal club | Perception regarding virtual journal club | Cronbach’s Alpha | Intraclass correlation [95%CI] | ||
|---|---|---|---|---|---|---|
|
| ||||||
| Supporting Clinical Practice | Supporting Research | Learning Experience | Benefits of virtual JC | |||
| Q1 | 0.707 | 0.622 | 0.414 | 0.38 | 0.93 | 0.66 [0.52;0.76] |
| Q2 | 0.748 | 0.729 | 0.37 | 0.357 | ||
| Q5 | 0.731 | 0.668 | 0.353 | 0.316 | ||
| Q9 | 0.758 | 0.784 | 0.493 | 0.534 | ||
| Q12 | 0.692 | 0.6 | 0.292 | 0.316 | ||
| Q13 | 0.844 | 0.635 | 0.52 | 0.498 | ||
| Q14 | 0.846 | 0.676 | 0.453 | 0.412 | ||
| Q15 | 0.776 | 0.655 | 0.464 | 0.42 | ||
| Q3 | 0.647 | 0.766 | 0.407 | 0.382 | 0.91 | 0.77 [0.67;0.84] |
| Q4 | 0.486 | 0.605 | 0.408 | 0.401 | ||
| Q6 | 0.769 | 0.762 | 0.447 | 0.48 | ||
| Q7 | 0.776 | 0.782 | 0.425 | 0.412 | ||
| Q8 | 0.616 | 0.724 | 0.439 | 0.414 | ||
| Q10 | 0.723 | 0.735 | 0.528 | 0.53 | ||
| Q11 | 0.6 | 0.631 | 0.382 | 0.391 | ||
| vir1 | 0.349 | 0.287 | 0.701 | 0.737 | 0.95 | 0.75 [0.64;0.82] |
| vir2 | 0.704 | 0.605 | 0.683 | 0.686 | ||
| vir3 | 0.496 | 0.551 | 0.775 | 0.846 | ||
| vir4 | 0.41 | 0.464 | 0.852 | 0.822 | ||
| vir5 | 0.392 | 0.5 | 0.797 | 0.704 | ||
| vir6 | 0.379 | 0.391 | 0.76 | 0.752 | ||
| vir7 | 0.522 | 0.535 | 0.722 | 0.701 | ||
| vir8 | 0.414 | 0.441 | 0.738 | 0.693 | ||
| vir9 | 0.461 | 0.478 | 0.809 | 0.74 | ||
| vir10 | 0.217 | 0.238 | 0.652 | 0.7 | ||
| vir11 | 0.335 | 0.422 | 0.814 | 0.768 | ||
| vir14 | 0.322 | 0.402 | 0.789 | 0.748 | ||
| vir1 | 0.349 | 0.287 | 0.757 | 0.537 | 0.74 | 0.65 [0.51;0.76] |
| vir3 | 0.496 | 0.551 | 0.813 | 0.738 | ||
| vir10 | 0.217 | 0.238 | 0.719 | 0.471 | ||
| vir12 | 0.351 | 0.462 | 0.329 | 0.262 | ||
| vir13 | 0.327 | 0.397 | 0.59 | 0.537 | ||
The Cronbach's alpha is used to assess the reliability in terms of set of items. The strength and consistency of the items were measured by Cronbach's alpha. The scores were obtained for all the items within the two domains in the perception of educational value; Cronbach's Alpha was 0.93 and 0.91, respectively. Similarly, in the two domains of virtual JC, Cronbach's Alpha were 0.95 and 0.74, respectively, and it fell under the acceptable limit of 0.60–1.00, which indicates that the questionnaire has a good level of internal consistency. The ICC for the two domains of the educational value of JC was 0.66 (95% CI; 0.52, 0.76) and 0.77 (95% CI; 0.67, 0.84), whereas the ICC for the virtual JC perception was 0.75 (95% CI; 0.64, 0.82) and 0.65 (95% CI; 0.51, 0.76), respectively, which indicated adequate pre- and post-validity of the questionnaire [Table 4].
Discussion
This study aimed to develop and evaluate the psychometric properties of a new scale for assessing the educational value and virtual journal club experience of nurses. The literature review revealed a lack of validated scales to assess the nursing JC activities. This scale could be considered a useful tool to evaluate the virtual and educational value of the nursing JC.
The face validity was assessed by administering the scale to eleven people from the target population. The face validity of the scale was >1.5, which was considered an acceptable limit, which is consistent with the findings of a similar study.[18] Content validity denotes the representation of currently available knowledge in the construct of interest (JC). In the current study, the content validity was assessed using a panel of eight experts from various fields, including senior nurse educators, nurse researchers, clinical pharmacists, and professors from nursing universities. The number of experts in content validity was consistent with the recommendation by Polit and Lynn.[27,28] According to Polit, the minimum CVI score for a panel of six to eight experts is 0.83. The CVI/Ave score of the current items is 0.97. The total variance explained for both domains was 67.8% and 66.5%, respectively. Results suggested mediocre construct accuracy of the questionnaire.[29]
In terms of construct validity, from PCA, we found two domains, the educational value of the JC (supporting clinical practice and supporting research) and the perception of the virtual journal club (learning experience and benefits from the virtual journal club). The supporting clinical practice domain contained the questions related to updating clinical practice, possessing high educational standards, critical appraisal skills, improving evidence-based practice, updating recent patient care, inspiring continuing nursing education, and new and positive learning of clinical practices. However, the supporting research domain contained the questions, i.e., enhancing research knowledge, promoting education and presentation skills, encouraging to read the research articles, preparing research proposals, identifying research gaps, and fostering professional collaborations.
In terms of the learning, experience domain contained the question related to providing uniform educational standards as in traditional classroom sections, effective for understanding the research article, social networking, motivation, more interactive, effective utilization of time, exposure to different digital platforms, informative discussions, easier for critical appraisal, convenience, more beneficial, and a better allocation of time for participants. The benefit domain of a virtual journal club contained the question, i.e., same education standard, social networking, convenient, more effective, and no distraction from other participants [Supplementary Figures 3 (506.5KB, tif) and 4 (320.7KB, tif) ].
The construct validity was tested from the aspects of model-data fit, convergent validity, and discriminant validity. The indices demonstrated an acceptable model-data fit implicating the confirmation of the internal factor structure of the questionnaire. The final version of the questionnaire NJCPS (29 items) proved to be a reliable and valid measuring tool to assess the nurses' JC activities. Convergent validity examines whether elements that should be linked are really connected. Discriminant validity tests ensure that factors and items that should not be related actually do not have any connections.[30] The correlation coefficient score was greater than 0.4 out of 28/29 items (96.6%).
The internal consistency of the scale was estimated to determine the reliability of the scale. In the present study, we finalized the scale with 29 items: 15 items with 2 domains for the perception of the educational value of JC s, i.e., supporting clinical practice and research, and 14 items with 2 domains for learning experiences and benefits regarding virtual journal clubs. The internal consistency of the items was measured by Cronbach's α. The Cronbach's α for the NJCPS scale for both domains of the educational value of JC (supporting clinical practice and supporting research) ranged from 0.93, 0.91 and for virtual journal club it was 0.95 and 0.74, respectively, which indicates good internal consistency of the new scale[31] NJCPS.
In the final scale, NJCPS, the perceived educational value of JC activities can be measured by summing the 15 items, and the maximum possible score is 30. The higher score suggests that the participants perceived high value in the JC activities, while low scores indicate the low educational value of JC. Another subscale, the perception regarding the virtual journal club, includes 14 items. The minimum possible score is − 28, and the maximum possible score is 28.[14] The higher score suggests that the participants have more preference for virtual journal clubs, while low scores indicate a lower preference for virtual journal clubs. The current study shows that the participants perceived moderate educational value in the JC and virtual journal experiences.
The current study successfully verified the rigorous validity analysis, including content, face, construct, and reliability of the scale (internal consistency) of the NJCPS. Overall, the analysis reveals that the scale is a valid measure of the educational value and virtual journal club experience. Therefore, the NJCPS scale can be used to assess the perception of educational value and the virtual journal club experience of participants from various fields.
Limitations and recommendation
The present study develops and validates a questionnaire to assess the perception of the JC regarding traditional and virtual methods. For instance, this study selected two facility nurses to have an understanding of how JC skills are developed in nurses. This study used quantitative methods to validate the questionnaire with advanced statistical methods. The study has been strengthened by eight experts who participated in content validation. Although the convenience sampling method has been adopted for data collection, data were collected from two hospitals proportionately after selecting randomly from 14 facilities.
Conclusion
The NJCPS is a valid tool that can be used to measure the educational value and virtual journal club experience of participants from different fields of healthcare. Our study tool includes the most important indicators of JC activities that are required to be achieved by each participant, including educational values and virtual journal club experience. Therefore, the present tool can play an important role in increasing the nurses' awareness by accurately evaluating their JC performance.
In the current study, in the initial validation for the developed tool, it is highly recommended that a large-scale study should be conducted to reconfirm the construct validity of the tool and complete the validation process.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
Supplementary File
Table 1.
Description of questionnaire items/constructs
| Perceived Educational value of Journal club | Number of items | Descriptions | |
|---|---|---|---|
| 1 | The journal club activities help to update my clinical practice. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 2 | The journal club activities in my facility possess high educational standards. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 3 | The journal club activities in my unit or facility enhance my research knowledge. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 4 | Journal club activities inspire me to pursue further education. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 5 | The journal club activities help my critical appraisal skills. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 6 | The journal club activities enhance my presentation skills. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 7 | Journal club activities encourage me to read more research articles. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 8 | The journal club activities help to prepare a research protocol. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 9 | Journal clubs facilitate the dissemination and reinforcement of evidence-based practice. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 10 | Journal clubs provide a valuable platform to foster and maintain professional collaborations. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 11 | Journal clubs enable me to identify gaps in professional practice. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 12 | Journal clubs enable me to communicate with colleagues about the latest developments in patient care. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 13 | Participating in a journal club is a part of my continuing nursing education. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 14 | Journal clubs are a productive way to learn new clinical practices. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 15 | Journal club activities provide a positive learning experience. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
|
| |||
| Perception regarding virtual journal club | Number of items | Descriptions | |
|
| |||
| 1 | Virtual journal clubs can provide the same educational standards as traditional classroom sections. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 2 | The teaching method utilize in the virtual journal club is effective for understanding the research article. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 3 | I enjoy the features of the new social networking at Virtual Journal Club. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 4 | The virtual journal club is motivating me to involve in research related activities. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 5 | The virtual journal club is more interactive than the traditional journal club. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 6 | The virtual journal club provides the opportunity for the best use of my time. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 7 | Introducing a new technology like Microsoft team, Zoom, WebEx meet, or Google meet in the virtual journal club offers a great opportunity to learn about new digital platforms | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 8 | The virtual journal club has more informative discussions. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 9 | The virtual journal club makes critical appraisal easier. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 10 | A virtual journal club is more convenient to attend than a traditional journal club. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 11 | The virtual journal club is more beneficial than the face-to-face method. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 12 | A “hybrid” (face-to-face and virtual) model of the journal club would be more effective. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 13 | I am not distracted by other participants during the virtual journal club. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
| 14 | A virtual journal club is allocating enough time to its participants. | 1 | Agreement with (coded: strongly agree = +2; agree = +1; Neither agree nor disagree=0; disagree = − 1; strongly disagree = −2). |
Scree plot of Perception domain of the NJCPS scale
Scree plot of Virtual domain of the NJCPS scale
Standardized factor loadings for the best fit model of Perceived educational Value of Journal club
Standardized factor loadings for the best fit model perception of Virtual Journal club
Acknowledgement
The author would like to extend appreciation to the experts who participated in content validity and the nurses who participated in the pilot study from the Women's Wellness Center Research Center and Emergency department of Al Khor Hospital.
References
- 1.Ranganath R, Elizabeth S, Mahadevwala D. Students’ perception of implementing journal clubs in an undergraduate medical curriculum. Res Dev Med Educ. 2021;10:5. [Google Scholar]
- 2.Sandefer CI, Gordy C. Undergraduate journal club as an intervention to improve student development in applying the scientific process. J Coll Sci Teach. 2016;45:52–8. [PMC free article] [PubMed] [Google Scholar]
- 3.Gurney MK, Buckley K, Karr S. Evaluation of a journal club preparatory session on student confidence for a graded journal club. Curr Pharm Teach Learn. 2019;11:1221–30. doi: 10.1016/j.cptl.2019.09.011. [DOI] [PubMed] [Google Scholar]
- 4.Landi M, Springer S, Estus E, Ward K. The impact of a student-run journal club on pharmacy students' self-assessment of critical appraisal skills. Consult Pharm. 2015;30:356–60. doi: 10.4140/TCP.n.2015.356. [DOI] [PubMed] [Google Scholar]
- 5.Lavin KA. Use of a journal club during level II fieldwork to facilitate confidence and skills for evidence-based practice. Open J Occup Ther. 2018;6:11. [Google Scholar]
- 6.Kim SC, Sabel S, McHargue S, Bloom RD. Impact of an extracurricular, student-led journal club on evidence-based practice among baccalaureate nursing students. Int J Nurs Educ Scholarsh. 2020:17. doi: 10.1515/ijnes-2020-0004. doi: 10.1515/ijnes-2020-0004. [DOI] [PubMed] [Google Scholar]
- 7.Mayer DDM. Nursing journal club to teach EBP critical appraisal skills. Worldviews Evid Based Nurs. 2019;16:329–30. doi: 10.1111/wvn.12361. [DOI] [PubMed] [Google Scholar]
- 8.Kjerholt M, Hølge-Hazelton B. Cultivating a culture of research in nursing through a journal club for leaders: A pilot study. J Nurs Manag. 2018;26:42–9. doi: 10.1111/jonm.12518. [DOI] [PubMed] [Google Scholar]
- 9.Lizarondo LM, Grimmer-Somers K, Kumar S. Exploring the perspectives of allied health practitioners toward the use of journal clubs as a medium for promoting evidence-based practice: A qualitative study. BMC Med Educ. 2011;11:66. doi: 10.1186/1472-6920-11-66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Al-Lenjawi B, Kunjavara J, Hassan N, Mannethodi K, Martinez E, Joy GV, Singh K. Evidence-Based Practice among Critical Care Nurse's/Midwives in Qatar. Open Journal of Nursing. 2022 Jan 26;12(1):42–59. [Google Scholar]
- 11.Aweid B, Haider Z, Wehbe M, Hunter A. Educational benefits of the online journal club: A systematic review. Med Teach. 2022;44:57–62. doi: 10.1080/0142159X.2021.1963424. [DOI] [PubMed] [Google Scholar]
- 12.Bryant FB, Yarnold PR. Principal-components analysis and exploratory and confirmatory factor analysis. 1995 [Google Scholar]
- 13.Fox N, Hunn A, Mathers N. Sampling and sample size calculation. East Midlands/Yorkshire: The National Institutes for Health Research. Research Design Service for the East Midlands/Yorkshire & the Humber. 2009 [Google Scholar]
- 14.MacCallum RC, Widaman KF, Zhang S, Hong S. Sample size in factor analysis. Psychol Methods. 1999;4:84. [Google Scholar]
- 15.Slavec A, Drnovšek M. A perspective on scale development in entrepreneurship research. Econ Bus Rev. 2012;14:3. [Google Scholar]
- 16.Netemeyer RG, Bearden WO, Sharma S. Scaling procedures: Issues and applications. London, United Kingdom: Sage publications; 2003 Mar 12. [Google Scholar]
- 17.Singh R, Agarwal TM, Al-Thani H, Al Maslamani Y, El-Menyar A. Validation of a survey questionnaire on organ donation: An Arabic world scenario. J Transplant. 2018 doi: 10.1155/2018/9309486. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Sheikhbardsiri H, Nekoei-Moghadam M, Yarmohammadian MH, Khankeh H, Aminizadeh M, Ebadi A. Developing and psychometric testing of the evaluation tool for disaster exercises design stage: A mixed method study. PLoS One. 2022;17:e0260581. doi: 10.1371/journal.pone.0260581. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mousazadeh S, Rakhshan M, Mohammadi F. Investigation of content and face validity and reliability of sociocultural attitude towards appearance questionnaire-3 (SATAQ-3) among female adolescents. Iran J Psychiatry. 2017;12:15. [PMC free article] [PubMed] [Google Scholar]
- 20.Bere E, Bjørkelund LA. Test-retest reliability of a new self reported comprehensive questionnaire measuring frequencies of different modes of adolescents commuting to school and their parents commuting to work-the ATN questionnaire. Int J Behav Nutr Phys Act. 2009;6:1–5. doi: 10.1186/1479-5868-6-68. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Zamanzadeh V, Ghahramanian A, Rassouli M, Abbaszadeh A, Alavi-Majd H, Nikanfar AR. Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. J Caring Sci. 2015;4:165. doi: 10.15171/jcs.2015.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.McHugh ML. Interrater reliability: The kappa statistic. Biochem Med. 2012;22:276–82. [PMC free article] [PubMed] [Google Scholar]
- 23.Kaiser HF. The varimax criterion for analytic rotation in factor analysis. Psychometrika. 1958;23:187–200. [Google Scholar]
- 24.DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. 2007;39:155–64. doi: 10.1111/j.1547-5069.2007.00161.x. [DOI] [PubMed] [Google Scholar]
- 25.Wei H, Xu G. Research on objectifying pulse examination from overall, dynamic and balanced view of traditional Chinese medicine. J Tradit Chin Med. 2014;55:25–7. [Google Scholar]
- 26.Gavgani VZ, Mirzadeh-Qasabeh S, Hanaee J, Hamishehkar H. Calculating reading ease score of patient package inserts in Iran. Drug Healthc Patient Saf. 2018;10:9. doi: 10.2147/DHPS.S150428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30:459–67. doi: 10.1002/nur.20199. [DOI] [PubMed] [Google Scholar]
- 28.Lynn MR. Determination and quantification of content validity. Nursing research. 1986 Nov 1;35(6):382–6. [PubMed] [Google Scholar]
- 29.Sofroniou N, Hutcheson GD. The multivariate social scientist. The Multivariate Social Scientist. 1999:1–288. [Google Scholar]
- 30.Carter SR. Using confirmatory factor analysis to manage discriminant validity issues in social pharmacy research. Int J Clin Pharm. 2016;38:731–7. doi: 10.1007/s11096-016-0302-9. [DOI] [PubMed] [Google Scholar]
- 31.Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011;2:53. doi: 10.5116/ijme.4dfb.8dfd. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Scree plot of Perception domain of the NJCPS scale
Scree plot of Virtual domain of the NJCPS scale
Standardized factor loadings for the best fit model of Perceived educational Value of Journal club
Standardized factor loadings for the best fit model perception of Virtual Journal club