Brazilian questionnaire of competencies of oncology nurses: Construction and test of content validation

Abstract

Objective

This study aimed to describe construction and content validation of the Brazilian Questionnaire of Competencies of Oncology Nurses.

Methods

The methodological research was constructed based on the literature and observation in Brazilian hospitals searching to identify local evidence in the nursing practice. After, the construction of the 30 items distributed in eight sub-dimensions of competencies, the instrument was tested the content validation by 7 experts and 61 oncology nurses. Item responses were analyzed through content validity index and Cronbach's α were used in this test pilot.

Results

The content validity index of the scale was 0.90, and the content validity index of each item was 0.80–1.00. The mean of items varied between 4.98 (standard deviation = 0.13) and 4.70 (standard deviation = 0.53). The Cronbach's α of the instrument was 0.77. The Cronbach's α if each item was excluded ranged from 0.78 to 0.74, indicating consistency between items.

Conclusion

The instrument is concise and clear, resulting in a valid content on test pilot in conducting a self-assessment the oncology nurses. It can be used to facilitate decision-making by identifying sub-dimensions that require attention in nursing education to improve patient care.

What is known?

• •Concepts about competences in the nursing field.
• •Standards and guidelines regarding the essential practices of nursing in oncology units.
• •Instruments are fundamental tools to evaluate nurse's performance and identify skills and knowledge gaps in clinical care in oncology.

What is new?

• •Creation of a new instrument to evaluate oncology nurse's competencies in Brazil based in literature and observation data.
• •A valid, reliable and concise instrument was developed to evaluate oncology nurse's competencies in a Brazilian context.
• •This instrument contributes to identifying essential knowledge, skills, and attitudes for oncology nursing practice, guiding practice improvement in organizations, and conducting future research.

1. Introduction

Cancer is an enormous global health burden, touching every region and socioeconomic group. Today, cancer accounts for about 1 in every 7 deaths worldwide – more than HIV/AIDS, tuberculosis and malaria combined. In 2018, there were about 1.7 million new cancer cases diagnosed. By 2030, the global burden is expected to reach 21.6 million new cancer cases and 13.0 million cancer deaths solely due to the growth and aging of the population [1].

The Brazilian National Cancer Institute estimates about 600,000 new cases of cancer occurred in 2016–2017 and more than a 14.6% increase of total deaths in Brazil [2]. These numbers have been increasing progressively and require larger investments in prevention and control, early diagnosis and evidence-based treatment of patients. Among the possible actions toward better health care, we can mention essential investments in new cancer diagnosis and care technology, material resources to accommodate the demand for services and the availability of a sufficient supply of human resources, specifically nurses prepared to care for this specialty population [2].

Previous studies have identified the central role of nurses in the provision of care to cancer patients, including clinical interventions, clinical or practical advice, emotional support to patients [3], and spiritual [4]. Nurses cultivate a safe environment in oncology care, identify knowledge deficits and provide effective interventions for continuing competence [5]. However, there is often a lack of clarity regarding the assessment of the nurse's cancer care. Organizations seeking to assess staff competencies should include identifying knowledge, duties, and skills of professional practice [6].

Oncology Nursing Society (ONS) published expectations regarding the essential practice of nursing through Oncology Clinical Nurse Specialist Competencies (OCNS). They discuss the multi-organizational work of nurses and the core competencies important to advance cancer nursing education and practice as a way to respond to the needs of cancer patients [7]. ONS describes the competencies for the optimal organization of nursing practices in oncology. The four major fields of activities include evaluation; diagnosis, identification and planning; intervention; and evolution [7].

Researchers at the international level have proposed the creation of scales to identify the level of hospital nurses competencies [8,9]. An example is the Nurse Professional Competence (NPC) Scale, the instrument that helped to expand nursing work in the hospital because it evaluates the quality of educational programs in nursing and also identifies skills gaps in clinical care [8]. The study is an important guide that allows addressing general competencies of nurses in the hospital environment. However, that scale did not measure practices performed specifically in oncology services.

In 2010, Brixey and Mahon [5] developed a scale to identify the level of competencies for oncology nurses. The scale entitled Self-Assessment Tool for Oncology Nurses integrates 139 items organized under 14 categories. Despite the importance of this content validation study as a framework for the continuing education of the oncology nurses; the scale was not considered applicable for the oncology nursing practice observed in Brazilian hospitals.

An extensive literature search yielded a lack of instruments or tools that quantitatively measure the frequency of use of specific oncology nurse competencies for use in assessment practice performance. The current study is original because it uses ONS competencies and Nurse Professional Competence (NPC) Scale frameworks and clinical observation in Brazilian hospitals to identify and describe the competencies of nurses in a Brazilian context. This scale identifies the frequency of competencies utilized by nurses in their work including oncology unit management, actions to eliminate or reduce the incidence of chemotherapy extravasation, knowledge, and appropriate skills to perform care to cancer patients at end of life, among others issues. This study describes the process of construction and content validation of an instrument designed to evaluate the use of competencies of oncology nurses in Brazil.

2. Methods

2.1. Study design and setting

This methodological research involved theoretical and empirical phases [10]. The theoretical phase involves the planning, selection of the competencies’ components based in the ONS competencies and NPC Scale; clinical observations in Brazilian hospitals; construction of the instrument; content validation and semantic validation. The empirical phase included a pilot test.

The study was undertaken in six large Academic Medical Centers with oncology care units in Brazil. They were chosen because of a large number of nurses with clinical experience in oncology in the different geographical regions.

2.2. Research tool

2.2.1. Socio-demographic data

The socio-demographic data includes age, gender, years after graduation from nursing education program, years in hospital, years in the oncology unit, and post-graduate.

2.2.2. Brazilian Questionnaire of Competencies of Oncology Nurses

The 30-item self-administered questionnaire includes eight sub-dimension of competencies: Area 1-Nursing care management (4 items); Area 2-Patient safety and injuries prevention (4 items); Area 3- Theoretical and practical knowledge and skills to work in the oncology unit (4 items); Area 4-Support for oncology education and research (3 items); Area 5-Communication and interpersonal skills (4 items); Area 6- Professional practice and nursing documentation (4 items); Area 7-Leadership and teamwork (4 items) and Area 8-Oncology unit management (3 items). This instrument indicates how often nurses practice the professional competencies through a 5-point Likert Scale. The scale ranges were: 1 = did not perform; 2 = poorly perform; 3 = perform regularly; 4 = often I perform; and 5 = very often. The intent of this survey was to identify the frequency of actions performed. Once the most frequently provided actions are identified, future research will consider the quality of the actions.

2.3. Research process

The construction of the instrument was influenced by the ONS competencies [7] and NPC Scale [8] recommendations and clinical observations in Brazilian hospitals. ONS influences the development of evidence-based policies, best practice guidelines, and quality interventions by improving outcomes for cancer patients. Therefore, this study analyzed the document and used its concepts to construct the research instrument. Nilsson et al. [8] described nurses’ competencies related to nursing theory and practice, research, development and education/supervision, and leadership. These authors reported the results of the developing and validating of the Nurse Professional Competence (NPC) Scale. The survey, which consists of the compilation of single-item questions regarding these issues, was identified as a potential instrument to identify general competencies of nurses.

In November 2015, one researcher and experienced nurse who did not work at the oncology units observed and recorded the nurse's roles. The clinical observation period was included in order to identify items that assess oncology nursing competencies inherent in education and skills in a Brazilian context. It occurred over a ten-day period and across the three professional work shifts (morning, afternoon and evening) for a totalizing of 60 h of observation of nursing care performed by oncology nurses. As a result of the clinical observation, we identified specific behavioral, environmental, social and clinical factors of nurses to be included in the construction of the instrument. This allowed us to develop a more tailored scale for measuring the competencies in the Brazilian context.

After the design of the items, the first version of the instrument and the instruction were sent by email to seven experts. The expert committee was composed by five nursing faculty members who have a Ph.D. degree, work in oncology, and have conducted validation studies; and two manager's nurses who have Ph.D. degree, experience in health management oncology.

The researchers explained the purpose of the study and requested a review of all items on three criteria: clarity (to be intelligible even to the lowest strata of the population), relevance (to be consistent with the attribute to be measured), and appearance (to express desirability, preference, and simplicity). In addition, there was a space for comments of the appropriateness of the items and how well the items reflected the construct of expectations for the oncology nurse role.

Version 1 of the instrument was evaluated and the experts assigned values from 1 (not relevant) to 5 (very relevant). Based on this feedback, items were modified as recommended by the experts and the results were used to generate version 2 of the instrument.

After the adjustments suggested by the experts, the instrument was subjected to semantic analysis by ten clinical nurses, consisting of five nurses with the lowest stratum (inexperienced nurses) and five other nurses with highest stratum (oncology specialists with professional experience of at least five years). For the semantic analysis it was assigned three criteria of analysis: First, the level of understanding of each item. Second, whether the participant was able to reproduce each item of the instrument with their own words, proving the true understanding of the item, and third, the general assessment of the instrument such as the type of scale and the construction of the items for the essential professional practice in oncology. The nurses assigned scores from 1 (not relevant) to 5 (very relevant) in their analysis.

To ensure that the questionnaire instructions and items were clear, a pilot test of the instrument was completed. A convenient sampling method was used to select oncology nurses. The instrument was tested with a sample of the 61 oncology nurses who did not participate in the previous steps. Previous studies have shown that a sample of at least 50 individuals is considered adequate for the evaluation of pilot studies [11,12]. Criteria for inclusion were: clinical nurses of both genders, regardless of age, who have been working for more than one year in the oncology area during the time of data collection. Nurses who partially completed the instrument were excluded from the sample.

2.4. Data collection

The study was approved by the Research Ethics Committee of the Ribeirao Preto College of the Nursing University of Sao Paulo, approval protocol 170/2015. Further approvals were granted by each of the six Brazilian hospitals and the participants.

Data collection used the form generator through the Google Docs Online Tool. Participants were invited to complete the questionnaire using the following steps: 1) to click on the explicit link in the contact email previously sent, 2) the participant was then directed to an electronic platform with the immediate opening of informed consent, 3) based on acceptance and verified his/her consent, and 4) the instrument and its instructions opened automatically. If the nurse did not agree, the process was terminated automatically. The deadline for completing the instrument was 15 days after the date of sending the access link. Data collection occurred between February 2016 and May 2016.

2.5. Data analysis

Data management and analyses were conducted by using the STATA 12 as well as Microsoft Excel version 2010. Socio-demographic characteristics were analyzed by using descriptive statistical analyses. A 30-items self-administered questionnaire was used for the pilot and examined for content validity and internal consistency. The internal consistency of the instrument was estimated with the Cronbach's alpha. Values larger than or equal .70 for the Cronbach's α were considered acceptable [13].

Descriptive analyses were used to summarize the data obtained each item through the Content Validity Index (CVI), Mean, and Standard Deviation (SD). The CVI measures the proportion or percentage of experts who are in agreement on certain aspects of the instrument and its items. The level of concordance considered acceptable was a CVI for the items greater than or equal to 0.80 [14].

3. Results

3.1. Socio-demographic characteristics

A total of 120 oncology nurses were invited to participate in the research and a total of 61 questionnaires were returned (respond rate 51%). The age of the participants was between 26 and 35 years (69%), and almost all the participants were women (95%), and most those (54%) have above 5 years’ experience in oncology nursing. Most nurses (94%) have a Post-Graduation in Oncology as reported in Table 1.

Table 1.

Socio-demographic characteristics (n = 61).

Variables	n	%
Gender
Woman	58	95.1
Man	3	4.9
Age (years)
26–35	42	68.9
36–45	11	18.0
46–55	7	11.5
56–60	1	1.6
Graduation
1977–1988	2	3.3
1989–2000	14	23.0
2001–2012	45	73.8
Experience in the hospital
4 months–	9	14.8
1 year–	14	23.0
≥ 5 years	38	1.64
Experience in the oncology unit
4 months–	8	13.1
1 year –	20	32.8
5–10 years	33	54.1
Post-graduate education in oncology
Clinical expertise	51	83.6
Master degree	3	4.9
Doctorate degree	1	1.6
Graduate in management	2	3.3

3.2. Content validity

In the review of the instrument, the expert's committee judged 12 items to be confusing as a result of wording or an unclear relationship with oncology nursing practice so they pointed out that they needed to be modified. Three items in the sub-dimension 1 (2, 3, 4), one in the sub-dimension 2 (6), two in the sub-dimension 3 (9, 10), one item in the sub-dimension 4 (15), two in the sub-dimension 5 (17, 18), two in the sub-dimension 6 (21, 22), and one in the sub-dimension 8 (30). Fig. 1 shows the changes in language made based on this review.

Fig. 1.

Main changes made to the items of the Questionnaire Assessment of Competencies for the Oncology Nurses after the evaluation of the expert's committee.

The CVI of the instrument was 0.90. After then, a semantic analysis was conducted with a sample of the target population. The participants reported that they did not have difficulties with comprehension of the instrument. Data showed means equal or greater 4.78 proving the unanimity of responses among the participants an understanding of the items.

3.3. Pilot test

In this pilot phase, the scales' mean, SD, median, interval, Cronbach's α if the item is excluded, and Cronbach's α for each sub-dimension were determined. The mean of items varied between 4.98 ± 0.13 and 4.70 ± 0.53. Total Cronbach's α was 0.77, which represents a satisfactory level of internal consistency of items. The Cronbach's α values in the case when each item was excluded ranged between 0.78 and 0.74 showing little variation across the variables (Table 2). These values are similar to those of another study [8] where the items with the highest and lowest value coefficients found in the instrument presented Cronbach's α ranging from 0.94 to 0.75 for the individual factors and 0.97 for the entire instrument. Another study aimed to test and validate an instrument titled Core Competence Nurses in Palliative Care, the Cronbach's α of items was obtained ranging from 0.40 to 0.96 [15].

Table 2.

Cronbach's α values and descriptive statistics of the items.

Items	Mean	Standard Deviation	Median	Interval	Cronbach's α if the item is excluded	Cronbach's α for each sub- dimension
Nursing care management						0.04
1- You apply the nursing process (data collection, nursing diagnosis, planning, implementation, and evolution).	4.97	0.18	5	1–5	0.77
2-You measure pain in cancer patients, using the rating scale and offering a therapeutic approach with a multidisciplinary team.	4.92	0.33	5	1–5	0.76
3- You identify the physical, psychosocial, cultural, and spiritual needs of cancer patients.	4.93	0.25	5	1–5	0.75
4- You guide on the appropriate treatment and care, patients, families, and caregivers on medical admission and discharge.	4.98	0.13	5	1–5	0.77
Patient safety and injuries prevention						0.21
5-You manage risk for infection control and prevention of transmission of microorganisms in immunosuppressed oncology patients.	4.89	0.37	5	1–5	0.76
6-You use adverse events prevention behaviors in cancer treatment.	4.80	0.51	5	1–5	0.77
7-You develop behaviors for elimination or reduction of chemotherapeutic extravasation incidences.	4.79	0.55	5	1–5	0.75
8-You analyze pre-chemotherapy and hormone therapy laboratory results.	4.84	0.49	5	1–5	0.75
Theoretical and practical knowledge and skills to work in the oncology unit						0.37
9-You have the knowledge and appropriate skills to perform care to cancer patients at the end of life.	4.79	0.45	5	1–5	0.76
10-You know the cancer pathophysiology, clinical diagnosis, and pharmacology used in the treatment.	4.80	0.60	5	1–5	0.76
11-You participate and encourage the team in the oncological treatment and rehabilitation strategies.	4.85	0.44	5	1–5	0.76
12-You have the technical ability while performing invasive procedures in patients with cancer.	4.85	0.40	5	1–5	0.76
Support for oncology education and research						0.06
13-You engage in communication with the patient, family, and caregivers in a clear and objective way during orientation activities.	4.84	0.42	5	1–5	0.75
14-You use materials and technological resources for the education of patients, families, and caregivers.	4.87	0.34	5	1–5	0.78
15-You participate and develop continuing education strategies for nursing staff.	4.87	0.34	5	1–5	0.76
Communication and interpersonal skills						0.39
16-You identify and mediate conflicts in the nursing team and the multidisciplinary team.	4.90	0.30	5	1–5	0.76
17-You facilitate communication among members of the nursing team and multidisciplinary team to prevent fragmentation of care.	4.70	0.53	5	1–5	0.77
18-You establish emotional bonds that allow you to become a reference as a nurse who assists the demands of patient care, family or caregivers.	4.92	0.33	5	1–5	0.75
19-You insert the patient, family, and caregivers in care planning, guiding the benefits and risks of cancer treatment.	4.92	0.28	5	1–5	0.77
Professional practice and nursing documentation						0.69
20-You document all interventions to cancer patients.	4.92	0.28	5	1–5	0.75
21-You comply with institutional protocols and report adverse events, anomalies, and nonconformities	4.98	0.13	5	1–5	0.76
22-You evaluate their professional attitude and ethics in nursing care.	4.95	0.28	5	1–5	0.75
23-You perform dimensioning of nursing staff to care for the patient according to their complexity and team needs.	4.95	0.22	5	1–5	0.74
Leadership and teamwork						0.18
24-You make decisions and allow the solution to problems in the oncology unit.	4.97	0.18	5	1–5	0.78
25-You supervise care developed and implemented by the nursing team.	4.87	0.34	5	1–5	0.77
26-You recognize the professional skills of each member of the nursing team and the values.	4.97	0.18	5	1–5	0.77
27-You delegate tasks to the nursing staff according to professional practice.	4.93	0.25	5	1–5	0.76
Oncology unit management						0.47
28-You provide and predict physical and material resources of the oncology unit.	4.90	0.40	5	1–5	0.76
29- You promote and encourage the team to waste management and practice sustainable actions related to nursing activities.	4.93	0.25	5	1–5	0.77
30-You promote actions for proper location, labeling, and storage of high-vigilance medications for cancer treatment.	4.93	0.25	5	1–5	0.76

4. Discussion

The purpose of this study was to present a new competency assessment tool specifically designed for Brazilian oncology units. The ONS provided international nursing practice competencies to assist nurses in helping patients in coping with the diagnosis of cancer and to implement safe and efficient nursing care in all aspects of their jobs. The NPC scale offered standard competencies for clinical nurses related to nursing care, value-based nursing care, medical technical care, teaching, documentation, legislation in nursing, leadership, and education. However, the literature needed to be accompanied by local observation of the context of the study.

The clinical observation in Brazilian hospitals assisted in understanding how the clinical nurse planned and coordinated the implementation of oncology nursing care. We observe specific training, protocol, and guidelines for nursing work. As a result, we constructed a feasible instrument to assess the competencies of nurses through their oncology practice experienced in Brazil.

The questionnaire was construed and followed all stages of the content validation. It focuses on competency validation while recognizing the development of nursing care and also the relationships among nurses, staff, patients and their caregivers.

We can infer from this preliminary study four essential factors about the unanimity of the scores of the instrument tested by the target population. First, the high level of agreement and unanimity may be because nurses recognize that in the oncology there are essential competencies common to cancer care across geographic location such as a national and international standard approach to developing and implementing competencies.

Secondly, the sample included primarily nurses (94%) who had some professional expertise in the oncology. Therefore, were more likely to report higher of the competencies for the oncology practice.

Next, the sample of nurses was collected from six Academic Medical Center. In general, these institutions have strengthened Continuing Education Centers which are responsible for providing operational protocols, training through evidence-based practice, and updated knowledge for nurses enrolled in institutions. It is also likely that nurses that work in the same unit have reported similar answers to the questionnaire due to this institutional influence.

Ultimately, this pilot study allowed preliminary tests on a small sample of six hospitals; thus justify the little variability in answers as well as in nursing practices observed in Brazilian hospitals, consequently may affect the unanimity of the score.

Although the response rate was adequate for the pilot test, there were 59 non-respondents. The non-respondents might have been more likely to report lower competencies. Therefore, the mean of the instrument may be overestimated. In addition, the sample size might also influence reliability estimates. Helms et al. [16] argue that maximizing the number of participants responding to a scale can increase the value of alpha by increasing the amount of covariance among item responses. Moreover, Helms et al. identified that a more heterogeneous sample should yield higher reliability estimates in comparison to a more homogeneous group.

Despite the Cronbach's α of the instrument was adequate and presented a high correlation among the items. The Cronbach's α for each sub-dimension presented low scores. These findings do not invalidate the scale consistency because of the small number of the items (4 or 3 items) in each sub-dimension of the scale. According to Graham [17] if the number of test items is small it may violate the assumption of equivalence and will underestimate reliability. Future studies with larger samples are recommended to test its psychometric properties to improve the reliability tests.

5. Conclusion

The Questionnaire of Competencies of Oncology Nurses was designed for identifying competencies in oncology nurses in Brazil. It exhibited satisfactory content validity. This study makes a significant contribution by allowing the measurement of the competencies through a self-administered instrument. It can be used as the first step in the identification of strengths as well as gaps in knowledge, skills, and attitudes of the nursing care, helping the future strategic planning to care quality.

Conflicts of interest

The authors have disclosed no potential conflicts of interest, financial or otherwise.

Funding

This study was financed by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brazil (CAPES) - Finance Code 001.

Appendix A. Supplementary data

The following is the Supplementary data to this article: