Development and Validation of the Soft Skills Questionnaire for Nurses

Abstract

Introduction

Skills challenge has been given great attention for its impact on employees’ performance. Various approaches have been put forward to design professional development programs that help in preparing nurses to work in the field and ongoing training programs to keep them up to each new method and technique at the interpersonal improvement level.

Objective

To develop and validate a questionnaire that tests the communication and management skills, emotional intelligence, and confidentiality of nurses in Lebanon.

Methods

The questionnaire, consisting of 25 statements, was generated and developed by experts in the field of nursing, soft skills, and questionnaire development. The items of the questionnaire were assessed using face, content, and construct validity and data validation was examined through psychometric properties at the final stage of the validation. Internal consistency and reliability were assessed through Cronbach alpha ($\alpha$). Further analyses were carried out to decide on the number of factors to be extracted using Oblimin Rotation method. All statistical tests were performed using Statistical Package for the Social Sciences (SPSS 20.0).

Results

Nineteen items of the 25-item scale had an I-CVI of 1.00 and the remaining six items had an I-CVI of 0.87. The S-CVI/UA was 0.76 and the S-CVI/Ave was 0.97, which meant that the items were good for the underlying construct. For the psychometric measures, the results were quite accepted and satisfactory. The Kaiser–Meyer–Oklin measure of sampling adequacy and the significance for the Bartlett's test for the whole questionnaire were quite acceptable with 0.680 and 0.000, respectively. Furthermore, the Cronbach alpha (α) value was 0.824 indicating a high internal consistency between the items of the questionnaire. While obtaining exploratory factor analysis for each section, results showed that Oblimin Rotation method should be applied to the last section in which three items were eliminated to maintain a simple factor structure.

Conclusion

This study shows that the 25-item Soft Skills Questionnaire is a valid and reliable tool for evaluating the level of communication of nurses, their emotional intelligence, confidentiality, and management.

Introduction

Skills challenge has been given great attention for its impact on employees’ performance. People having acquaintances and abilities in their jobs are irreplaceable and represent a valuable quality who can stand for great challenges (Barney & Wright, 1998; Saa-Perez & García-Falcón, 2002). Regardless of the type of work, training has been exclusively used as a tool to deliver the required knowledge and techniques to both old and new employees to achieve their job responsibilities to their best performance (Black & Mendenhall, 1990). In the field of healthcare, skills, one of the main competence attribute measures among nurses, are still unidentified (Belita et al., 2020). Various approaches have been put forward to design professional development programs that help in preparing nurses to work in the field and ongoing training programs to keep them up to each new method and technique at the interpersonal improvement level (El-soudany, 2018).

Literature Review

Recently, considerable literature has grown up around the relevance of training programs on the employees’ performance (Goldstein, 1980; Tai, 2006). An article published in Forbes Magazine reported that training contributes to the employees’ performance in a good way pointing that US spending on corporate training grew by 15% in 2013 (the highest growth rate in 7 years) to over $70 billion in the US and over $130 worldwide (Bersin, 2022). In the same study, it was stated that organizations today are suffering from a “skill supply chain” challenge, in which more than 70% of them mentioned “capability gap” as one of their top five challenges (Bersin, 2022). Previous studies have noted the significance of the teamwork processes at the level of caregivers such as communication, coordination, and cooperation on the effect of patient's safety (Parker & Coiera, 2000; Urbina & Monks, 2020; Vermeir et al., 2015). Additional studies focused on the importance of the nurses’ soft skills on the healthcare services (Ng, 2020). Adding to that, the training obtained by nursing students during their college experience is far from the fact of their professional life (Solvik & Struksnes, 2018). Although students are aware of all the theoretical policies and procedures, they are not usually familiar with engaging their theoretical background information in the practical field (Murphy, n.d.; Ng, 2020). Hence, the lack of experience in working under stress rises a critical concern in healthcare as patients’ lives depend on the nurses’ level of performance.

According to the American Association of Critical Care Nurses, soft skills are a crucial part of nursing that contribute to safe patient care and a healthy work atmosphere in which personal qualities, traits, and behaviors are displayed (C Health, 2021). Consequently, a broad comprehensive training and development program contributes in improving the soft skills required by nurses to acquire in fulfilling their day-to-day operational activities (Peteraf, 1993).

Quality in healthcare services cannot be maintained without regular follow-ups on each practice, in which patients’ rating of their overall experience is influenced by the level of soft skills acquired by the nurses (Murphy, n.d.). Assessment is one of the most important techniques which serves as a diagnostic “pulse-taking” tool to monitor, evaluate, and track the deficiencies in clinical practices. It follows that, having a well-outlined evaluation criteria based on the communication skills’ measures and characteristics in a field that secures health and well-being is a must, especially that those skills have a significant effect on the burnout of healthcare professionals (Semaan et al., 2021). This suggests that bringing about a validated instrument to identify the strengths and the weaknesses of the nurses’ communication skills is being considered now more than ever. With such measurable tool, the issues of miscommunication problems and the deficiencies at the level of the internal interaction can be determined (Norcini et al., 2018). Besides, interestingly, evaluation of communication skills could be created by projecting the communication skills’ aspects, characteristics, and criteria into set of items in which each item is tackling an aspect of those skills. Undoubtedly, a good evaluation form is the form that addresses all the needed criteria with less items. Specifically, when it comes to nurses, most of them complain about the evaluation process as time is a critical factor in their jobs.

Realizing the lack of tools measuring nurses’ application of their communication skills in a practical field in Lebanon, it is pivotal to develop a tool for this purpose. This study is one of its kind in Lebanon that aims to create and develop a tool to evaluate the level of communication and management skills, emotional intelligence, and confidentiality of nurses in Lebanon. This paper aims to validate the obtained instrument qualitatively and quantitatively using psychometric measures. Through this tool, a better understanding of the nurses’ strengths in both theoretical and practical knowledge will be provided.

Methods

Study Design

A multi-assessment model including qualitative and quantitative techniques was applied. The questionnaire was generated and developed over two stages: Stage I for item generation and development and stage II for instrument testing and validation. Figure 1 illustrates the validation process followed in this project from its early steps starting with the research question moving to the three developed consecutive drafts passing by face validity then content validity till reaching the final step at construct validity.

Figure 1.

Overview of the Validation Process.

The study received ethical approval from the Institutional Review Board at MUBS (MU-20190601-13). Participation in the qualitative and the quantitative assessment was voluntary, anonymous, and confidential and participants were introduced to the aim and the procedure of the study through a clear written consent form stating the importance of their contribution and their right for withdrawal from the questionnaire at any time. This study was conducted on a sample size of 56 nurses employed in two hospitals in Lebanon. Data were kept on a laptop with a password and only the researchers have access to them.

Stage I—Qualitative Phase/Development of the Soft Skills Questionnaire

This stage included two steps: (1) Generation of the items; (2) Item reduction and development of the Soft Skills Questionnaire (SSQ).

Generation of the Items

The idea was generated from language used between the nurses and the patients that was addressed and reported to the top management of the healthcare institutions in Lebanon. It was stated and described by either the in-patients at the hospitals, where they shared their experience during their stay in the “Suggestions and Complaints Box” or the head nurses’ observations, where they raised their commentary to the top management. The observations showed the areas that should be focused on to execute a measurable instrument.

A thorough literature search was carried out on the communication between healthcare professionals and their patients. This aimed to explore the strengths and the weaknesses of nurses at the level of communication. The findings of the research helped in targeting the key factors that should be focused on to outline an effective evaluation tool.

This literature review highlighted the area of study which focused on three categories: general communication, confidentiality, and management and emotional intelligence, in which the questionnaire was based on. The three categories that emerged from the qualitative evaluation of the critical reading and assessment of the previous research formed the skeleton of the prospect questionnaire. The first draft of the questionnaire was developed with 30-scale items. A great effort was put in stating the items in a clear, readable, and an understandable way using a simple language. To avoid the copy-paste way of answering, the items were stated in both positive and negative directions.

The 30 items were extracted from the qualitative analysis and were distributed over three main sections: general communication, confidentiality, and management and emotional intelligence, tackling every single detail related to the target of the study.

Item Reduction and Development of the SSQ

In the first draft, the 30 items were approved by a five-member expert panel coming from a nursing management, soft skills, and questionnaire development background. The terms and conditions that should be satisfied for an item to be accepted to the next step were as follows: appropriateness for the factor being tested, relevance to nursing soft skills, and simplicity and clarity of statement (language). Through this step, items were removed, kept, or modified by the experts following their evaluation criteria. Five items failed to match the criteria of assessment and they were nominated by all the experts to be removed from the prospect questionnaire. However, modifications were applied to 11 items that can be adjusted to serve the purpose of the questionnaire. The modified and kept items formed the second draft of the questionnaire. Before moving to the second step in this phase of validation, the questionnaire was translated from English to Arabic language to make sure that the nurses will not face any misunderstanding of the text or language barrier. For a better understanding for each item, both English and Arabic languages were available in one copy to provide a clear, comprehensible, and understandable version of each item.

With reference to the qualitative assessment of the five experts, some items were subjected to minor modifications to make them more explicit and clearer. In the process of face and content validation and based on the evaluation of the five experts, there were no further modifications.

A questionnaire of two sections was developed based on the 25 items. In the first section, the nurse was asked to fill in a set of personal information questions such as the age, gender, educational level, position at work, years of experience, number of working hours per week, and so on (Table 1, Appendix A). Each inquiry in this section reveals an input that may contribute to the performance of each nurse at the level of communication. The reason of choosing each inquiry in section one and its impact are presented clearly in Table 1.

Table 1.

Factors Influencing Nurses’ Soft Skills.

Factor	Importance of each factor
Age	This is an open answer which can be later classified into intervals during the analysis phase if needed. It shows the maturity of the nurse in dealing with any situation, which enhances the cognitive ability (Chung et al., 2015).
Gender	Studies showed that females have a higher level of organizational commitment than males (Messner, 2017). There is a relationship between gender and emotional commitment (Goodarzvand & Amin Rodposhti, 2012). Many factors contribute to the organizational commitment between men and women such as family ties, job attributes, and flexibility of working hours (Marsden et al., 1993; Scandura & Lankau, 1997).
Nursing degree attained	Three choices BSN, TS/LT, or BT This reveals the academic background and qualifications. Lebanon’s Education system has two types of Higher Education institutions: University Institute, which covers the Bachelor degree in Nursing Sciences (BSN), and the Vocational Institute, which covers three levels of nursing degrees (BT, TS, and LT): –Technical Baccalaureate (BT): It requires three academic years of study, after obtaining the official brevet certificate, and passing the official examination conducted by the Professional and Technical Education institution in Lebanon. Obtaining this degree allows the graduate student to work, as a Nurse Assistant in a healthcare setting. –Superior Technique (TS): The technical nurse is the one who has completed a vocational education program in the field of nursing with not less than three academic years after either obtaining the BT degree, or the official Baccalaureate exams. The TS degree is issued by the Professional and Technical Education institution in Lebanon. To be able to practice nursing in a healthcare setting, the student must obtain a practice permission from the Ministry of Public Health in Lebanon. –Technical License (LT): After passing the official Baccalaureate exams conducted by the Ministry of Education and Higher Education in Lebanon, the duration of three academic years in the nursing major shall be adopted, and in order to practice nursing in a healthcare setting the student must pass the official exams, in addition to a national exam (Professional colloquium) for students of private institutes (done by the Ministry of Education and Higher Education and the Lebanese Governmental University), and a practice permission from the Ministry of Public Health in Lebanon.
Hospital name	This is an open answer. Public and private hospitals can be compared in future studies.
Unit	Level of stress is related to type of work and its responsibilities. This has been explored in previous studies which reported that stressful situations are related to a particular type of hospital units (Moustaka & Constantinidis, 2010; Najimi et al., 2012).
Position	There are two choices: Head Nurse (HN) or Registered Nurse (RN).
Years of experience	Previous research has shown that years of experience influence the nurses’ emotional competency and their responsiveness to the work stress (Humpel & Caputi, 2001), where the work experience through years has a positive and significant effect on the work performance (Putri, 2020; Ratnawati et al., 2020); however, the newly employed professional nurses experience stress and frustration (Abrahams et al., 2021).
Shift	This includes four choices: Day, Evening, Night, or Mixed. Knowing that nurses may perform 8-, 10-, or 12-hours shifts depending on the employer and the hospital’s policy (Berg, 2022; Rivas, 2022). However, the hospitals where the study was held follows 3 shifts per day, 8-hours each with a possibility to mix shifts if necessary. Timing of the shift affects nurses’ behavior and attitude. The shift working schedule has a great impact on the nurse’s mindfulness and eventually on the sustainability of the profession (Savic et al., 2019).
Number of duty hours/week	The number of duty hours/week has received critical attention as this affects the quality of responsiveness of each nurse (Hoedl et al., 2021).
Second language	There are three choices suggested: English, French or both. Language plays an important role in analyzing and reporting everything around appropriately; however, more research is needed to explore patients' perspective (Ali & Johnson, 2017; Francis, O’Connor et al., 2007).

In the second section, 25 items were listed randomly using a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree). Those items were distributed over three sections, each section dealt with a topic called “dimension”: first dimension: level of communication skills addressed by 13 items; second dimension: level of confidentiality described by 6 items; and third dimension: level of management and emotional intelligence tackled by 6 items.

Stage II—Quantitative Phase/Psychometrics Properties of the SSQ

This stage included three steps: (1) Face validity; (2) Content validity; and (3) Construct validity.

Face Validity

To test whether an indicator seems to be a reasonable measure of its underlying construct “on its face,” face validity was obtained using qualitative and quantitative tools. The qualitative face validity technique was performed using the qualitative face validity questionnaire (Appendix B) asking 10 nurses about the difficulty and the obscurity of each item over a 4-point Likert scale. Among the ten nurses, five of them were heads of nursing department in different hospitals, two of them were experts in the field of soft skills development and the others were registered nurses. However, the quantitative face validity was completed by the same nurses using the quantitative face validity questionnaire (Appendix B) by calculating the impact score indicating the importance of each of the questionnaire items over a 4-point Likert scale.

The face validity results of difficulty, obscurity, and level of importance of each item, rated by 10 experts over a 4-Likert scale (with 1 = very difficult and 4 = not difficult for the difficulty of items, 1 = very obscure and 4 = not obscure for the obscurity of items; and 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant for the importance of each item), were calculated by dividing the sum of the ratings of the 10 experts by 40 (the sum of the maximum rates of the 10 experts) (Salkind, 2010).

Content Validity

Content validity refers to the extent to which an instrument measures a given construct from all facets and aspects, which can be performed by evaluating the validity of inferences derived from test score (Sireci, 1998). Qualitative and quantitative methods were used to determine the content validity. The qualitative assessment required asking eight experts in the field of tool development about the simplicity, clarity, and appropriateness of items based on their location in the questionnaire.

The reviews of the experts in the field of tool development were recorded for each item. Qualitatively, each expert rated the simplicity, clarity, and the appropriateness of each item over a scale from 1 to 4 (with 1 as minimum and 4 as maximum). To find the ratio of each item, the sum of the votes of the eight experts were divided by 32 (the sum of the maximum rates of the eight experts).

The quantitative assessment was achieved by examining the Content Validity Ratio (CVR) and the content validity index (CVI), where a group of experts were contacted by telephone, in which the questionnaire was presented to them to rate the necessity and the relevance of each item.

1. Content Validity Ratio

At this level, Lawshe's model was applied to the questionnaire to find out the CVR (Lawshe, 1975). The experts reflected on the necessity of each item, in which their feedback was classified as essential, useful, not essential, and not useful. The experts’ judgments were quantified through the CVR:

$$CV{R}_i={\displaystyle \frac{N_e-{\displaystyle \frac{N}{2}}}{{\displaystyle \frac{N}{2}}}}$$

Where $CV{R}_i$ is the CVR value for the ith measurement item; $N_e$  is the number of panelists indicating a measurement item as “essential”; N is the total number of panelists.

According to the content validity equation, the ratios can vary between −1 (perfect disagreement) and +1 (perfect agreement) (Lawshe, 1975). According to Lawshe's model, the acceptable CVR depends on the number of experts participating in the validation process (Lawshe, 1975). In this study, the questionnaire was evaluated by eight experts which requires a critical cut-off value of 0.75. Items scoring below this critical value should be removed.

• (b) Content Validity Index

CVI is the most used approach to ensure the overall validation of an instrument on a 4-point Likert scale (1 = not relevant, 2 = somewhat relevant, 3 = relevant, 4 = very relevant). Validation at the item-level (I-CVI) and at the scale level (S-CVI) can be calculated using this index, which refers to the ratio of agreement on the relevance of each item (Rungtusanatham, 1998).

The I-CVI is measured by the number of experts providing a rating of “very relevant” for each item divided by the total number of experts. CVIs should range between 0.00 and 1.00 where items with CVI > 0.79 are considered relevant, those between 0.70 and 0.79 need some revision, whereas those with CVI < 0.70 should be eliminated (Zamanzadeh et al., 2015).

The S-CVI is measured by two methods: The Universal Agreement among experts (S-CVI/UA) and the Average CVI (S-CVI/Ave). The S-CVI/UA is calculated by dividing the number of items with CVI = 1.00 by the total number of items under study. Instruments with UA above 0.80 are labelled as high content validity. The S-CV0I/Ave is determined by dividing the sum of the I-CVIs by the total number of items being studied (Yusoff, 2019). Questionnaires with average value above 0.90 have excellent content validity.

Construct Validity

Following face and content validation, the construct validation process was carried out using Exploratory Factor Analysis (EFA), in which no expectation can be made about the nature and the number of factors. To test the reliability and the internal consistency of the questionnaire as a whole and the items in particular, Cronbach alpha ($\alpha$) was measured and analyzed. Nurses with different positions, shifts, and years of experiences were eligible to complete the questionnaire. The estimated time needed for a nurse to finish the questionnaire in a continuous manner was around 15 min, however, it took some of the respondents an average of 30 min to fill in all the required items due to their job urgent calls and duties. The nurses were told to complete the questionnaire without missing any of the items.

Data Collection

The pilot study was applied on two different private hospitals in Mount Lebanon from June 2019 till June 2020. The questionnaire was held on during the shifts of the nurses and upon the availability of each nurse, which was completely voluntary and anonymous. The rate of responsiveness across the targeted hospitals was 56% (corresponding to 56 nurses in total from the Emergency Unit [ER], Surgery Unit, Intensive Care Unit [ICU], and Pediatrics) which is considered quite acceptable, considering the critical duties of the nurses in charge.

Data Analysis

The validity and the reliability of the questionnaire was assessed as follows: construct validity was determined by EFA. Values for Kaiser–Meyer–Oklin (KMO) measure of sampling adequacy and Bartlett's test were used to assess the suitability of data for factorization. For the KMO, the accepted index needs to be over 0.60, whereas the Bartlett's test of sphericity must be less than 0.05. EFA was used to explore the common factors in the latent variable using Statistical Package for the Social Sciences (SPSS 20.0).

Similar analysis based on factor loading criterion was carried out to determine the number of factors to extract and Oblimin Rotation method was used to identify the meaningful components of the questionnaire. This technique of factor analysis is considered as conclusive psychometric analysis that delivers evidence about the reliability, validity, and quality of items being tested.

Results

This is, to the best of our knowledge, the first questionnaire of its kind in Lebanon to evaluate the soft skills for nurses, therefore providing current insights on the practices and techniques that are critical in simplifying communication and are adaptable and applicable in Lebanon. The survey was conducted in two private hospitals in Mount Lebanon on nurses during their shifts.

Face Validity

Looking at the results of the two qualitative criteria of face validity, the difficulty, and the obscurity of each item respectively, the ratings of the 10 experts ranged between a minimum of 0.875 and a maximum of 1.000 (Table 2).

Table 2.

Results of the Face Validation for the Lowest Scores Items.

Face validation for the lowest scores items
Qualitative results			Quantitative results
Difficulty	Item 10	0.925	Level of importance	Item 12	0.875
	Item 12	0.925
	Item 23	0.925
Obscurity	Item 13	0.875
	Item 23	0.875

Starting with the rates of difficulty of each item, the minimum agreement received was 92.5% on items 10, 12, and 23, which is quite satisfying. As for the rates of obscurity, a minimum agreement of 87.5% was noticed for items 13 and 23, which indicates an excellent agreement for the other 23 items (above 90%).

The quantitative results of the face validity were determined by the level of importance of each item, which ranged between a minimum of 0.875 (item 12) and a maximum of 1.000, which was reached by 13 items.

Content Validity

Based on testing the three qualitative criteria of content validity (simplicity, clarity, and appropriateness of each item, respectively), the ratings of the eight experts ranged between a minimum of 0.8125 and a maximum of 1.000 (Table 3).

Table 3.

Results of the Content Validation for the Lowest Items.

Content validation for the lowest scores items
Qualitative results			Quantitative results
Simplicity	Item 12	0.8125	Content validity ratio (CVR)	None of the items
	Item 13	0.8100
	Item 23	0.8438
Clarity	Item 12	0.8125	Content validity index (CVI)	Item 12	0.750
	Item 23	0.8438
Appropriateness	Item 12	0.8438		Item 23	0.750
	Item 23	0.8125

For the simplicity criterion, three items received an agreement of less than 85% with item 12 having 81.25%, item 13 having 81%, and item 23 having 84.38%. The other 22 items of the questionnaire obtained an agreement above 85%, which is considered very acceptable.

For the clarity criterion, only two items received an agreement of less than 85% with item 12 having 81.25% and item 23 having 84.38%. The other 23 items of the questionnaire obtained an agreement above 85%, which is considered very acceptable.

For the appropriateness criterion, two items received an agreement of less than 85% with item 12 having 84.38% and item 23 having 81.25%. The other 23 items of the questionnaire obtained an agreement above 85%, which is considered very acceptable.

Content Validity Ratio

The CVR was generated for each item. According to Lawshe's model, items rated by eight experts with CVR < 0.75 were marked as non-essential and should be removed. None of the 25 items scored below the critical value 0.75. Consequently, none of the items should be removed. In addition, the average CVR for the whole questionnaire was 0.92 indicating a very acceptable CVR (Appendix C).

Content Validity Index

The I-CVIs ranged between 0.875 and 1.000 except for two items; item 12 and item 23 where each of them scored 0.750 indicating the need of some revision.

The UA was calculated by dividing the number of items with CVI = 1.00 (22 items) by the total number of items under study (25 items): S-CVI/UA = 0.88 and the average was determined by dividing the sum of CVIs by 25: S-CVI/Ave = 0.96. Both the UA and the average approach showed a high content validity with S-CVI/UA > 0.80 and S-CVI/Ave > 0.90 (Appendix C).

Construct Validity

Demographic Data

All demographic data are summarized in Table 4. Among the 56 nurses who participated in the pilot study, there were 20 males (35.7%) and 36 females (64.3%). As for the level of education, 42 nurses (75%) had a Bachelor of Science in Nursing (BSN) degree, nine nurses (16.1%) had a License Technique (LT) degree, three nurses (5.4%) had a Technique Superior (TS) degree, and two nurses (3.6%) had a Baccalaureate Technique (BT) degree. As indicated in the table, most of the nurses in the study worked on the ER unit (32.1%), 19.6% of the nurses worked at each of the ICU and the medicine department and the rest worked at other units (Pediatric, surgery, and chemotherapy). Most of the nurses in the study are registered nurses (87.5%) whereas 12.5% are head nurses. Forty-four of the nurses (78.6%) have an experience up to 10 years. Half of the nurses in the sample worked in mixed shifts (day, evening, and night), close to them the day-shift nurses (44.6%). Forty-nine nurses (87.5%) worked between 21 and 40 h per week. As shown in Table 4, 62.5% of nurses speak English as their second language, 28.6% speak French, and 8.9% speak both.

Table 4.

Descriptive Measures for the Pilot Study.

		Frequency	Percentage
Gender	Males	20	35.7%
	Females	36	64.3%
Degree	BSN	42	75%
	LT	9	16.1%
	TS	3	5.4%
	BT	2	3.6%
Unit	ER	18	32.1%
	Medicine	11	19.6%
	ICU	11	19.6%
	Surgery	8	14.3%
	Chemotherapy	5	8.9%
	Pediatric	3	5.4%
Position	Head Nurse	7	12.5%
	Registered Nurse	49	87.5%
Years of experience	0–4	24	42.9%
	5–9	20	35.7%
	10–14	3	5.4%
	15–19	2	3.6%
	20–24	7	12.5%
Shifts	Day	25	44.6%
	Evening	1	1.8%
	Night	2	3.6%
	Mixed	28	50%
Number of working hours per week	0–20	6	10.7%
	21–40	16	28.6%
	41–60	33	58.9%
	More than 60 h	1	1.8%
Second language	English	35	62.5%
	French	16	28.6%
	Both	5	8.9%

The questionnaire was distributed on 56 nurses from two Lebanese private hospitals. The nurses had the chance to report any unclear item or ambiguous expressions. However, nurses’ feedback emphasized the clarity in the language of the questionnaire. The items were clear, readable, and explicit in assessing the level of communication between a nurse and patients.

Internal Consistency and Reliability

To measure the extent to which subjects’ responses are stable over time, data were examined for internal consistency among all items of the questionnaire using the Cronbach alpha test ($\alpha =$ 0.824), indicating high internal consistency between the items under study for the questionnaire.

In testing the internal consistency for each section of the questionnaire, the results have shown sufficient reliability for the first two sections; “Communication Skills” and “Confidentiality” with $\alpha =$ 0.867 and $\alpha =$ 0.730, respectively. Unlike the results in last section “Management and Emotional Intelligence” that have shown an unacceptable internal consistency with $\alpha =$ 0.272, which means that some of the items in the last section are confusing and they should be considered for re-testing.

Exploratory Factor Analysis

To track the misleading items in the questionnaire, the 25 items were analyzed using EFA to examine the factorial validity of the scale for each section separately. The dimensions that can justify the observed variation among the variables were identified using Eigen value greater than one criterion. The factorability of the data was verified for each of the three sections of the questionnaire using two paired tests: the KMO and the Bartlett's test of sphericity (Taherdoost et al., 2014).

Estimation of communalities illustrated how various issues can affect the design of the research variable. Statistically speaking, it is the amount of variance and variable shared with all the other variables being considered. A value of variance which is below 0.30 indicated that the item does not belong to the other items measuring the variable (Taherdoost et al., 2014).

The criterion for loading and cross-loading was set at 0.40 as a cut-off value for the extraction method of the factor analysis. Consequently, items with loading below 0.40 and cross loading over 0.40 were removed. This procedure was performed over and over until a simple structure was achieved where loadings were increased on accepted factors and reduced on the others.

To start up with, factor analysis was examined first for the questionnaire as a whole and then for each section separately.

The Questionnaire as a Whole

KMO measure of sampling adequacy was 0.680 and significance for Bartlett's test of sphericity reached statistical significance (${\chi }^2=$812.68, p = .000), which shows that these items are factorable. The communalities of all the items are above 0.30, none of the items should be removed. The Cronbach's alpha (α) value was 0.824 indicating that the internal consistency between items is very high (Table 5).

Table 5.

Psychometric Measures for the Whole Questionnaire.

Section 1: Communication Skills

In this section, the communication-related skills were represented by 13 items (1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 17, 21, and 22).

KMO measure of sampling adequacy was 0.727 and the p-value for Bartlett's test of sphericity reached statistical significance (p = .000), which shows that these items are factorable, and the data are very suitable for factor analysis. The communalities of all the items are above 0.30 thus none of the items should be removed. The Cronbach's alpha (α) value was 0.867 indicating that the internal consistency between items is very high (Table 6).

Table 6.

Psychometric Measures for Nursing Communications Skills.

Section 2: Confidentiality

In this section, the confidentiality-related skills were represented by six items (15, 18, 19, 20, 24, and 25).

KMO measure of sampling adequacy was 0.693 and the p-value for Bartlett's test of sphericity reached statistical significance (p = .000), which shows that these items are factorable, and the data are very suitable for factor analysis. The communalities of all the items are above 0.30 thus none of the items should be removed. The Cronbach's alpha (α) value was 0.730 thus the internal consistency between items is very high (Table 7).

Table 7.

Psychometric Measures for Confidentiality.

Section 3: Management and Emotional Intelligence

In this section, the management and emotional intelligence-related skills were represented by six items (8, 12, 13, 14, 16, and 23).

KMO measure of sampling adequacy was 0.626 and the p-value for Bartlett's test of sphericity reached statistical significance (p = .006), which shows that these items are factorable, and the data are very suitable for factor analysis. The communalities of all the items were above 0.30 thus none of the items should be removed. The Cronbach's alpha (α) value was 0.272 indicating that the internal consistency between items is very weak (Table 8).

Table 8.

Psychometric Measures for Management and Emotional Intelligence.

Oblimin Rotation Method

Among the six items of this section, some of them should be removed to identify meaningful components of the questionnaire. To determine the components that should be removed, the Oblimin Rotation method was carried out. The three items 12, 13, and 23 were eliminated to maintain a simple factor structure as they failed to meet some minimum criteria of having a primary factor loading of 0.40 or above (Corner, 2009).

The loading factors for each item are shown in the Pattern Matrix, where the partial standardized regression coefficients of each item are represented with a particular factor: the structure matrix, where the simple zero-order correlations of the items are represented with each item and the component matrix, where the components can be interpreted as the correlation of each item with the component (Table 9).

Table 9.

Loading Factors for the Management and Emotional Intelligence Section.

The extraction method showed that items 8, 14, and 16 are loading meaningfully on the first component and hence the dimensions of the factors are better accounted for by the items. However, items 12, 13, and 23 are not contributing meaningfully to this component as their cross-loading factor did not exceed 0.40.

Reaching for better validated results, items 12, 13, and 23 were removed from the questionnaire and factor analysis tests were applied again to items 8, 14, and 16. KMO measure of sampling adequacy was 0.623 and the p-value for Bartlett's test of sphericity reached statistical significance (p= .006), which shows that these items are factorable, and the data are very suitable for factor analysis. The communalities of the three items were above 0.30 and none of the items should be removed. However, Cronbach's alpha (α) value was 0.621 showing that the internal consistency between items is quite acceptable. The factor-loading matrix for this final solution is presented in the component matrix (Table 10). All the items are loading to one component with loading factors above 0.70.

Table 10.

Loading Factors for Items 8, 14, and 16.

Discussion

In the generation of the questionnaire, the sections extracted from the literature review address three aspects of communication: the general communication, the confidentiality, and management and emotional intelligence. In the development of the item pool, the 25 items from the qualitative findings were considered in the questionnaire to build on the related constructs and direct the development of the item pool. The draft on the item pool was based on a critical review on existing related literature and assessment tools as well as the interview data in the qualitative phase. During the development of item pool phase, the language of the generated items was revised and edited over and over to fit a readable and understandable structure (Navabi et al., 2016).

In the development of the questionnaire, the items were tested qualitatively and quantitatively from the perspective of nurses and head nurses in the face validity stage, and then from the perspective of soft skills’ experts and professionals specialized in the field of questionnaire development in the content validity stage (Aravamudhan & Krishnaveni, 2015).

In both face and content validations, items 12, 13, and 23 appeared repeatedly as weak items compared to the other item of the study (Figure 2).

Figure 2.

Items with Lowest Scores as Shown Using the Face and Content Validity.

In the evaluation of the psychometric properties of the questionnaire, the proof of the construct validity was represented by exploratory and confirmatory factor analysis. In the study, EFA of the SSQ suggested the elimination of items 12, 13, and 23 as they do not contribute to the Model Fit of the component. This was suggested as well by the face and content validity, where each of the suggested items received the lowest rate of votes showing a weak agreement from the expertise. The test where each of the suggested items received the lowest agreement is represented in the chart of Figure 2. The minimum scores reached by items 12, 13, and 23 in the face and the content validity tests alone were low enough to be eliminated from the questionnaire. The obtained scores indicated that some changes are recommended to be applied to those items. However, the scores recorded by the psychometric evaluation ensured that items 12, 13, and 23 should be either modified to fit the structure or eliminated from the questionnaire.

Item 12 received the lowest agreement in the qualitative and quantitative assessment of the face validity (test of difficulty and level of importance, respectively). Moreover, item 12 received the lowest score in the qualitative and quantitative assessment of the content validity (tests of simplicity, clarity, appropriateness, and in the CVI, respectively).

Item 13 received the lowest agreement in the qualitative assessment of the face and content validity (test of obscurity and simplicity, respectively).

Item 23 received the lowest agreement in the qualitative assessment of the face validity (test of difficulty and obscurity). Moreover, item 23 received the lowest score in the qualitative and quantitative assessment of the content validity (tests of simplicity, clarity, appropriateness, and in the CVI, respectively).

Decision-Making Step

Approved by face, content, and construct validation, the SSQ is a 25-item model that addresses the communication skills of the healthcare providers. EFA was inspected to validate the derived factor structures of the SSQ. Outcomes have shown a relevant useful instrument, providing confirmatory evidence for the factor structure.

Further studies can be done on the three eliminated items (item 12, 13, and 23) and some amendments should be applied so that to meet the research question and fit the research measures.

Building on the findings of this study, the SSQ is considered a valid and reliable tool for evaluating the communication skills factors among nurses in Lebanon. The SSQ can be utilized to assess the level of communication of nurses, their emotional intelligence, confidentiality, and management. The SSQ can be used as an identification tool to assess the skills that a nurse should acquire in her/his career.

Limitations

This study serves as a confirmation card for the SSQ to be used in the upcoming studies. However, several limitations should be taken into consideration, such as the inappropriateness of the number of items as compared to the factors under study. The length of the questionnaire is considered long for a nurse, an unconditionally working person, so a time-consuming questionnaire is inconvenient and tiresome for a nurse to fill. Consequently, most of the nurses could not complete the questionnaire on a continuous basis as they had to leave it several times to respond the duty calls, eventually, this added an uncontrolled constrain factor to the study. Another limitation of the study is the size of the sample considered in the pilot study which can be larger.

Implications for Practice

To maintain a good, sensible, and fine nursing care, an accurate, reasonable, and practical assessment tool is required. Having a validated assessment tool that tests the communication and management skills, emotional intelligence, and confidentiality of nurses in Lebanon is crucial and plays an important role in evaluating the nurses’ behavior and in tracking the unnecessary practices. This questionnaire serves as an appraisal instrument that can be used for regular evaluation at hospitals and medical care centers.

Conclusion

The SSQ is a well-validated, reliable, and a reasonably comprehensive instrument for measuring the communication skills factors among nurses in Lebanon. Taken together, the SSQ can be utilized to assess the level of communication of nurses, their emotional intelligence, confidentiality, and management. Nonetheless, more research is always needed to strengthen the future development of SSQ for a timely appropriate questionnaire.

Appendix A

A Modern Approach in Continual Education in Lebanon: Outcomes of Conducting Communication-Skilled Trainings for Nurses in the Workplace Nursing Research Questionnaire

This questionnaire aims to assess employees’ transferable skills in the formal communication channel both verbally and non-verbally in hospitals in order to develop a series of trainings that focuses on enriching the employees’ repertoire of transferable skills that are the main reason behind promotions and continual achievements. This study is conducted by Ms. Manal Zeineddine, English Coordinator; Ms. Samiha Charifeh, Acting Chair of the Nursing Department; Ms. Mona Aridi, Instructor at the International School of Business; Dr. Wissam Ghach, Assistant Professor of Chemistry at the School of Health Sciences; and Dr. Nisreen Alwan, Dean of School of Health Sciences at the Modern University for Business and Science (MUBS). Data will be used to help assess the effectiveness of designing and implementing a “Sustainable Employability and Effective Communication Skills” trainings for nurses. This project has been reviewed by and received ethics approval from the MUBS Institutional Review Board.

All responses to this questionnaire will be recorded anonymously and information will not be personally identifiable. Completion of the questionnaire is voluntary and return of the questionnaire implies consent to participate in the study. All answers to the questions will be treated in the strictest of confidence and will be stored securely in MUBS until the end of the study and then destroyed. If you have any further questions about participation in this study please contact the primary researchers:

Ms. Manal Zeineddine: mzeineddine@mubs.edu.lb

Ms. Samiha Charifeh: scharife@mubs.edu.lb

The questionnaire contains 25 questions. Please answer all of them if possible. The questionnaire should take a maximum of 10 minutes to be completed. If you require any assistance, please contact the researchers. Please return the fully completed questionnaire to one of the MUBS representatives present.

Participant no: ………………….

Age: ………………..

Gender: M □ F □

Nursing degree attained: BSN □ TS/LT □ BT □

Hospital name: …………………………..

Unit: ………………………………..

Position: Head Nurse □ Registered Nurse □

Years of experience: …………….

Shift: Day □ Evening □ Night □ Mixed □

Number of duty hours/week: …………………..

2^nd Language: English □ French □

Circle the correct numeric response to each statement:

#	Statements	Scale: 1 = Strongly Disagree 2 = Disagree 3 = Neutral 4 = Agree 5 = Strongly Agree
1	I usually greet my patients when checking on them.	1	2	3	4	5
2	I always introduce myself to my new patients.	1	2	3	4	5
3	Once forgetting my patient's name, I either use Mr. or Mrs. to identify the patient.	1	2	3	4	5
4	I rarely forget to introduce my role as a nurse to the patient.	1	2	3	4	5
5	Written technique can’t be used to answer patient's inquiries.	1	2	3	4	5
6	I never interrupt my peers during their lunch breaks.	1	2	3	4	5
7	I demonstrate steps and actions to help my patient understand any new procedure.	1	2	3	4	5
8	I only show up in the patient room, upon my patient's call.	1	2	3	4	5
9	I usually prefer to discuss the patient's critical case with his/her family, rather than with the patient him/herself.	1	2	3	4	5
10	It is important to have the patient express his/her case before formulating a care plan.	1	2	3	4	5
11	I always establish active listening while in the patient's room.	1	2	3	4	5
12	I do not prefer to interfere when a peer is facing a problem (worries, needs, personal concerns).	1	2	3	4	5
13	I am sometimes aware of my basic needs (hunger, thirst…) to ensure excellent patient service.	1	2	3	4	5
14	I am not attentive to a patient's non justifying behavior (uncontrolled emotions), when am suffering from time shortage.	1	2	3	4	5
15	It is preferable to initiate one-to-one conversation with the patient about his/her personal life.	1	2	3	4	5
16	Presenting the necessary guidance to the patient is not the nurse's responsibility.	1	2	3	4	5
17	I always smile when delivering any type of care or even news to my patient.	1	2	3	4	5
18	The patient's privacy is not necessary in all situations.	1	2	3	4	5
19	I always separate men and women in patients’ room just for medical purposes.	1	2	3	4	5
20	It is not necessary to ask for the patient's permission, when starting any new treatment.	1	2	3	4	5
21	I try not to raise my voice when calling my peers.	1	2	3	4	5
22	I try not to ignore the question when not updated with the patient's case.	1	2	3	4	5
23	I always ask the patient if the physician visited and checked on him/her.	1	2	3	4	5
24	I don’t address the first degree family members with the patient's medical status.	1	2	3	4	5
25	I don’t believe that mutual trust should be built with the patient.	1	2	3	4	5

• Rate your communication skills:

Acceptable □ Good: □ Very good: □ Excellent: □

• Additional comments:

…………………………………………………………

…………………………………………………………

Thank you for your time.

Appendix B

The face validity survey invited head nurses to evaluate whether items in the SSQ are difficult or obscure through rating them on a scale from 1 to 4. The nurses were asked as well to rate the level of importance of each item on the same scale as follows:

Difficulty: How difficult is this item?	1 = Very difficult
	2 = Difficult
	3 = Somewhat difficult
	4 = Not difficult
Obscurity: Is this item obscure?	1 = Very obscure
	2 = Obscure
	3 = Somewhat obscure
	4 = Not obscure
Level of Importance: How important is this item?	1 = Not important
	2 = somewhat important
	3 = quite important
	4 = Very important

The table below shows the content validity assessment done by 10 head nurses on each item of the questionnaire.

	Face validity assessment by 10 nurses
	Qualitative		Quantitative
Item	FVI of difficulty	FVI of obscurity	FVI of importance
1	1	1	1
2	1	1	1
3	1	1	1
4	1	1	0.925
5	1	1	0.925
6	1	1	1
7	0.975	0.975	1
8	1	1	0.975
9	0.95	0.95	0.95
10	0.925	0.9	0.975
11	1	1	1
12	0.925	0.95	0.875
13	0.95	0.875	0.925
14	0.975	0.95	0.95
15	1	1	0.975
16	0.975	1	1
17	1	1	1
18	1	1	0.925
19	1	1	1
20	1	1	1
21	1	1	1
22	0.975	1	0.975
23	0.925	0.875	0.925
24	1	1	0.975
25	1	1	1
Average	0.983	0.979	0.971

Appendix C

Part A:

The content validity survey invited the experts to evaluate whether items in the SSQ are simple, clear, and appropriate based on their location through rating them as follows:

Simplicity: How simple is this item?	1 = Not simple
	2 = Somewhat simple
	3 = Quite simple
	4 = Very simple
Clarity: Is this item clear?	1 = Not clear 2 = item needs some revisio 3 = clear 4 = Very clear
Appropriateness: How essential is this item?	1 = Not Appropriate
	2 = somewhat appropriate
	3 = ???
	4 = Not appropriate
Relevance: How relevant is this item?	1 = Not relevant
	2 = somewhat relevant
	3 = quite relevant
	4 = Very relevant

The table below shows the content validity assessment done by eight experts on each item of the questionnaire.

	Content validity assessment by eight experts
	Qualitative			Quantitative
Item	CVI of simplicity	CVI of clarity	CVI of appropriateness	I-CVI	UA
1	1	1	1	1	1
2	1	1	1	1	1
3	1	1	1	1	1
4	1	1	1	0.875	0
5	0.875	0.875	0.875	0.875	0
6	1	1	0.90625	1	1
7	1	0.96875	1	1	1
8	1	1	1	1	1
9	1	1	1	1	1
10	1	1	0.9375	1	1
11	1	1	1	1	1
12	0.8125	0.8125	0.84375	0.875	0
13	0.8125	0.875	0.875	0.875	0
14	1	1	1	0.875	0
15	1	1	1	1	1
16	1	0.96875	1	1	1
17	1	1	1	1	1
18	1	1	1	1	1
19	1	0.96875	1	1	1
20	1	1	1	1	1
21	1	1	1	1	1
22	1	0.96875	1	0.875	0
23	0.7813	0.84375	0.8125	0.875	0
24	1	1	1	1	1
25	1	1	0.90625	1	1
Average	0.971	0.97125	0.96625	0.97	S-CVI/Ave 0.76 S-CVI/UA 0.97

Part B:

The content validity survey invited the experts to evaluate whether items in the SSQ are essential through rating them as follows:

Essential: How essential is this item?						1 = Not essential
						2 = Useful but not essential
						3 = Essential
	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	Expert 8	CVR
Item 1	x	x	x	x	x	x	x	x	1
Item 2	x	x	x	x	x	x	x	x	1
Item 3	x	x	x	x	x	x	x	x	1
Item 4	x	x	x	x	x	x	x	x	1
Item 5	x	x	x	x	x	x		x	0.75
Item 6	x	x		x	x	x	x	x	0.75
Item 7	x	x	x	x	x	x	x	x	1
Item 8	x	x	x	x	x	x	x	x	1
Item 9	x	x	x	x	x	x	x	x	1
Item 10	x	x	x	x	x	x	x	x	1
Item 11	x	x	x	x	x	x	x	x	1
Item 12	x	x	x		x	x	x	x	0.75
Item 13	x	x	x	x	x	x	x		0.75
Item 14	x	x	x	x	x	x	x	x	1
Item 15	x	x		x	x	x	x	x	0.75
Item 16	x	x	x	x	x	x	x	x	1
Item 17	x	x		x	x	x	x	x	0.75
Item 18	x	x	x	x	x	x	x	x	1
Item 19	x	x	x	x	x	x	x	x	1
Item 20	x	x	x	x	x	x	x	x	1
Item 21	x	x	x	x	x	x	x	x	1
Item 22	x	x	x	x	x	x	x	x	1
Item 23	x	x	x	x	x	x		x	0.75
Item 24	x	x		x	x	x	x	x	0.75
Item 25	x	x	x	x	x	x	x	x	1
CVR(Critical) for a panel size (N) of 8 is 0.75.									0.92