Development of Agile Scrum Perception Tool to Evaluate Students’ Opinions on Agile Methodology in Nursing Education

Abstract

Background and Aim:

Agile methodology (AM) is an innovative, active, team-based learning method that enhances higher-order thinking (metacognitive ability), collaboration, hands-on training, and student engagement in the classroom and clinical environment. Exploring the student’s opinions of AM in teaching helps understand how agile can be utilized in higher education to positively improve learning ability, leadership, and teamwork. Hence, the study aimed to develop the agile scrum perception tool (ASPT) to evaluate the opinions on AM among nursing students.

Materials and Methods:

A cross-sectional design was carried out to develop ASPT in different phases. The tool was initially derived from focus group discussion and literature review. The preliminary draft was revised and modified based on expert review and suggestions by content and construct validity. Reliability was calculated in terms of internal consistency. The developed tool was evaluated using data from 200 4^th-year nursing students.

Results:

A total of 23 items of ASPT have adequate psychometric properties in terms of reliability and validity. Internal consistency was checked by coefficient alpha, which was 0.87. Construct validity was analyzed by factor analysis; all items were loaded in 6 factors, accounting for 54% of the variance in the total ASPT score.

Conclusion:

ASPT has good validity, reliability, and broader scope in medical and allied education related to project management, clinical teaching, and classroom teaching.

Introduction

Agile methodology (AM) is an active, innovative teaching and learning method. Adopting AM improves students’ academic results and self-satisfaction.[1] AM was coined by Agile Manifesto (a group of 17 software developers).[2] In AM, scrum is a popular framework in which students work together to achieve a common goal. Scrum is derived from a rugby game in which team members pack together, placing the face down to gain and capture the ball. Scrum consists of three roles: product owner (PO)/outcome evaluator (OE)/teacher/faculty responsible for course construction, process, and evaluation; scrum master (SM)/team leader (TL) responsible for removing impediments in the learning process, continuously motivating the team to achieve a goal and conducting daily review and retrospective meetings; and development team/student team (ST) consisting of 7–8 members for achieving set objectives on time. Scrum events/activities are sprint (achievement of goal within the set time); sprint planning (choosing a task or work from the prioritized task, setting the time for each work, and deciding the type of assignment for each week); evaluation method (quiz and demonstration); daily standup meeting (every day 15 min dedicated to discussing the progress in the learning pattern); review (students can submit the assignment/demonstrate the given procedure, following feedback offered by the PO/researcher, head of the department [HOD], or dean); and retrospective meeting (ST alone discusses the problems faced and solved during the completed sprint and steps to be taken to improve the learning in the consecutive sprints). The scrum artifacts/protocol are product backlog (list of topics/objectives for the overall sprint), sprint backlog (choosing the topics/tasks/objectives from the prioritized list for a particular sprint), and increment (completed task within the set duration of each sprint).[3]

Scrum is adopted in the educational system called eduScrum.[4] Studies have documented that agile scrum promotes the student’s higher engagement, teamwork, collaborative learning, leadership ability, reflective ability, and self-regulated learning by adjusting and controlling their learning strategies through metacognitive skills (thinking about thinking) and clinical performance.[3,5,6,7,8,9,10]

AM fosters knowledge in the skills and enhances technical and nontechnical skills.[5,11] According to the 14^th annual state Agile Global Survey report, 58% of the respondents practiced the scrum framework, and 85% reported that daily meetings help in higher productivity, followed by a retrospective meeting, sprint planning, and review meetings. Only 4% of respondents used AM in health care, pharmaceutical companies, and educational institutions. While incorporating agile, some challenges documented are more training and education, fewer skills, lack of leadership, and sharing abilities.[11,12]

Incorporating agile into the nursing curriculum is challenging for the faculty. Agile further fostered the students’ abilities, such as leadership and problem-solving ability, and promoted them as self-directed lifelong learners.[13,14,15] Quantitative and qualitative studies are available to measure the project manager’s and engineering students’ opinions on AM, but its psychometric properties have not been mentioned.[7,11,15,16,17,18,19,20,21,22,23] There is a limited source for developing the opinionnaire in education setup in the Indian context; hence, there is a need to develop an agile scrum perception tool (ASPT) to evaluate the opinion on AM in nursing education.

Materials and Methods

In the present study, a quantitative research approach and cross-sectional design have been used. The study was conducted in five nursing colleges after obtaining the required permissions. The sample included 200 4^th-year B.Sc. nursing students selected using a convenient sampling technique. The tool was developed in different phases. Ethical clearance was obtained from the institutional ethics committee. Written informed consent was obtained from the nursing students involved in the study after providing a complete orientation on AM, including the scrum framework, roles and responsibilities of the scrum team, scrum events/activities, and scrum artifacts/protocol. Confidentiality and anonymity of the subjects were maintained.

Results

Phase I: Item selection and preliminary draft preparation

Items were generated on the basis of a literature search of existing tools,[24,25,26,27,28,29] and information were obtained from focus group discussions (FGD) with seven software architects practicing agile and scrum techniques in their projects. In the first FGD, the following open-ended questions[30,31,32,33,34] with probing were discussed:

• Is the implementation of AM in the academic context feasible?

• How do scrum values, principles, events, and protocols help in achieving learning outcomes?

• How are scrum roles and responsibilities important in the teaching and learning process?

• How does the scrum framework impact the individual student and teams’ performance in learning?

• What are the strengths and weaknesses of a scrum?

After reviewing the content from the first FGD, major themes and subthemes were derived, followed by a conclusion drawn, and suitable points were used to formulate items of the tool. In the 2^nd FGD, a prepared list of items was discussed regarding understandability, relevance, and content coverage with domains. Preliminary draft of ASPT with selected items under seven domains, i.e. opinion on agile training, scrum team responsibilities, PO/OE responsibilities, SM/TL responsibilities, ST responsibilities, scrum activities, and scrum approach, was deduced. Each item was rated on a 5-point Likert scale (5 = strongly agree, 4 = agree, 3 = neutral, 2 = disagree, and 1 = strongly disagree). Higher scores on the ASPT indicate higher agreement and acceptance of AM.

Phase II: Validation and reliability of the tool

The preliminary draft was given to the panel of 13 expert pediatricians (1), nursing professors (2), pediatric nursing professors (5), and software engineering professionals (5) who had knowledge and practice on agile scrum methodology. Experts were given a suggestion to convert some of the items that are applicable to nursing education, such as the role of scrum team-Product Owner (PO) as Outcome Evaluator (OE), teacher/facilitator, stakeholder (HOD), scrum leader as TL, product backlog (list of tasks/topics) and sprint backlog (choosing the topics for priority), burndown chart, retrospective meeting, and sprint for more clarity.

Based on the expert’s opinion the items were revised and clarified three times. A final approved draft of ASPT was devised with seven domains: collaborative learning in the scrum, quality of team leader, scrum learning outcome, opinion on agile training, scrum framework, and scrum team responsibilities. The relevancy of the tool was assessed using a 4-point scale ranging from 1 (not relevant) to 4 (highly relevant) as reported by Davis in 1992. On the basis of 13 expert evaluations, the content validity index (CVI) was calculated for the items (I-CVI) and for the scale (S-CVI). The mean I-CVI of items was 0.97, S-CVI/UA was 0.82, and S-CVI/Ave was 0.97.

A pilot study was conducted to assess the applicability of the revised version of the tool and was administered to 20 nursing students who took an average of 30 min to respond. No further suggestions for revision were received. Therefore, the content validity of the tool was satisfactory, and the ASPT was ready for use in the main phase of the study with 200 4^th-year B.Sc. nursing students. The participants were 21–23 years old.

Construct validity

Construct validity of ASPT was assessed using exploratory factor analysis (EFA), in which the principal component analysis method was applied using SPSS (IBM version 28). To check the suitability of the collected data for factor analysis, the adequacy of the sampling and the eligibility of tool items were assessed by calculating Kaiser–Meyer–Olkin (KMO) value and Bartlett’s test of sphericity. The KMO value of data in this study was 0.85, and a P value of Bartlett’s test of sphericity was <0.001, which was found statistically significant. It indicates that the data were suitable for factor analysis.

The principal component analysis extraction method was used to calculate the extraction communality value of each item. Initial communality was assumed as 1 (100%) for each item. Extraction commonality of items was in the range of 0.42–0.71, which indicated that data are suitable for proceeding for factor analysis. To formulate clusters of highly interrelated items from a correlation matrix, factor extraction was done. Factor loadings >0.30 were considered for the model. Since few factors had either 1 or 2 items loaded on them, these were not considered factors (a minimum of three items are required for the factor). By deleting 15 such items, 23 items were retained in the tool. The principal component analysis with varimax rotation yielded a six-factor solution with eigenvalues >1. The eigenvalues of the six factors range between 1.7 and 6.21. Factor structure converged on six iterations. Factor I, Factor II, Factor III, Factor IV, Factor V, and Factor VI have a rotational variance of 26.99, 6.31, 5.58, 5.3, 5.1, and 4.8, respectively. The six factors are TL/SM responsibilities, scrum team, ST, opinion on agile training, scrum activities, and benefits of scrum framework [Tables 1 and 2].

Table 1.

Factor structure of agile scrum perception tool

Items	Factor loadings
	TL responsibilities	Scrum team	ST	Opinion on agile training	Scrum activities	Benefits of scrum framework
Q11. Time management was good	0.7
Q21. TL was skilled in maintaining the team	0.67
Q19. Time management by TL was good	0.57
Q23. Team effort was good in achieving the outcome	0.53
Q12. The course objective was well explained by OE		0.44
Q14. PO/OE (OE and stakeholder HOD) offered continuous formative feedback to the individual and the team		0.68
Q16. All the learning activities were facilitated by the TL/SM		0.8
Q22. Members allotted to the team were adequate		0.44
Q28. Team members were able to help the underperformed students		0.39
Q36. Product backlog (list of topics for learning) was clear and adequate		0.44
Q37. Sprint backlog (prioritized topics selected from the list of topics) was well carried out		0.47
Q24. Active communication and interaction within and between DT/ST were improved			0.71
Q29. Burndown chart (graphical representation of work left to do versus time) making was made easy			0.51
Q35. Retrospective meeting (ST discussion about what went well during just completed work/sprint) was helped for further improvement in learning			0.68
Q1. Agile was useful				0.56
Q3. Learning was facilitated by agile methodology				0.79
Q6. Agile framework was easy to follow				0.62
Q26. Accountability of ST was improved in each sprint					0.53
Q30. Sprint (event within a fixed duration of time to complete the task/objectives/work) duration was adequate to achieve a goal (within 3 weeks)					0.74
Q31. Sprint planning (work of planning for learning) was effective					0.36
Q9. Promoted hands-on experience						0.80
Q8. It helped to adopt the changes						0.64
Q20. Collaboration between PO/OE and TL was good						0.36
Eigenvalue	6.207	1.481	1.284	1.218	1.165	1.107
Variance accounted for (%)	26.99	6.31	5.58	5.3	5.1	4.8
Coefficient α	0.70	0.76	0.54	0.51	0.602	0.55
Coefficient α (total tool with 23 items)	0.87

TL/SM: Team leader/scrum master; PO/OE: Product owner/outcome evaluator; ST/DT: Student team/development team; HOD: Head of the department

Table 2.

Agile scrum perception tool

Items	SA (5)	Agree (4)	Neutral (3)	DA (2)	SDA (1)
Factor 1. TL responsibilities
Time management was good
TL was skilled in maintaining the team
Time management by TL was good
Team effort was good in achieving the outcome
Factor 2. Scrum team
The course objective was well explained by OE
PO/OE (OE and stakeholder HOD) offered continuous formative feedback to the individual and the team
All the learning activities were facilitated by the TL/SM
Members allotted to the team were adequate
Team members were able to help the underperformed students
Product backlog (list of topics for learning) was clear and adequate
Sprint backlog (prioritized topics selected from the list of topics) was well carried out
Factor 3. ST
Active communication and interaction within and between DT/ST were improved
Burndown chart (graphical representation of work left to do versus time) making was made easy
Retrospective meeting (ST discussion about what went well during just completed work/sprint) was helped for further improvement in learning
Factor 4. Opinion on agile training
Agile was useful
Learning was facilitated by agile methodology
Agile framework was easy to follow
Factor 5. Scrum activities
Accountability of ST was improved in each sprint
Sprint (event within a fixed duration of time to complete the task/objectives/work) duration was adequate to achieve a goal (within 3 weeks)
Sprint planning (work of planning for learning) was effective
Factor 6. Scrum benefits
Promoted hands-on experience
It helped to adopt the changes
Collaboration between PO/OE and TL was good

PO: Product owner; OE: Outcome evaluator; TL: Team leader; ST: Student team; SA: Strongly agree; DA: Disagree; SDA: Strongly disagree; ST/DT: Student team/development team; HOD: Head of the department; SM: Scrum master

Scree plot

It is a graphical representation that depicts the eigenvalues against all the factors. The studies suggested using the scree plot in conjunction with the eigenvalues to determine the number of factors to retain. The points of interest are where the curve starts to flatten. This study showed that the initial six factors had major contributions toward total variance (point of first inflection). Subsequent to the first six factors, the scree plot curve is almost smoother without any more inflection. Hence, the initial 6 factors and all 23 items were retained on the basis of a scree plot [Figure 1].

Figure 1.

Scree plot

Reliability of agile scrum perception tool

The internal consistency of the tool was calculated by computing coefficient alpha. The coefficient alpha for the final tool was 0.87, which indicates that the instrument has good internal consistency. The coefficient alpha for the factors was: factor 1 (0.70), factor 2 (0.76), factor 3 (0.54), factor 4 (0.51), factor 5 (0.58), and factor 6 (0.55) [Table 1].

To find the individual contribution of each item, item-wise reliability was calculated. Each item was deleted one by one, and any differences in the coefficient alpha values were measured, which found that coefficient alpha values remained the same if the item was deleted. Hence, all the 23 items were retained in the tool.

Phase III: Evaluation

After completing the phases of tool development, ASPT was finalized with 23 items. The ASPT has good reliability and validity. The tool was used after the agile training session to evaluate nursing students’ opinions on AM. Feedback regarding the instrument was obtained from 32 nursing students, and the results are shown in Figure 2.

Figure 2.

Students’ feedback on agile scrum perception tool

Discussion

The present study was conducted to develop ASPT for evaluating the opinion on AM among nursing students. The coefficient alpha for the tool was found to be 0.87. Coefficient alpha is dependent on the magnitude of correlations among items and the number of items in the scale. The range should be between 0.70 and 0.90.[35] Various studies showed that the smaller alphas are acceptable with smaller subscales.[36,37,38] A study by Ibrahim et al. developed and validated a Zigzag Depression Scale that had 46 items with 16 domains. The internal consistency of the tool and each factor/domain was performed with coefficent alpha i.e., .897 and .370 to .758 respectively.[39]

Content validity was done by the expert’s opinion, and it indicates a good validity. Finally, EFA yielded a meaningful factor structure supporting the construct validity of the tool. The result showed that the six-factor solution explained 54 of the cumulative variances. The first factor accounts for 26.99% of the variance and represents TL responsibilities, which include time management, skills in maintaining the team, and teamwork in achieving the outcome. Overall, this first factor focuses on the TL’s responsibility. The second factor accounts for 6.31% of the variance, and describes scrum team responsibilities include the course objectives was well explained by PO/OE, PO and stakeholders offered continuous feedback, all learning activities facilitated by TL, team members were helped the underperformed students, adequate members were allotted to the team to complete the task, product backlog and sprint backlog was clear and well carried out. Overall, this factor enhances the scrum team’s responsibilities.

The third factor accounts for 5.58% of the variance. It has items representing the domain of the ST, which include effective communication between the ST, transparency in the learning through the burndown chart by the ST, and further improvement in learning through retrospective meetings by the ST. The fourth factor accounts for 5.3% of the variance, represents opinion on agile training include usefulness, easy to follow, and agile facilitates learning. The fifth factor accounts for 5.1% of the variance. It has items representing the domain of scrum activities which include adequacy of sprint duration, effective sprint planning, and improvement in accountability of the ST in each sprint. The sixth factor accounts for 4.8% of the variance. It represents the benefits of the scrum framework, including ease of adopting the changes, promoting hands-on experience, and improving team collaboration. Overall, this factor emphasizes the advantages of the scrum framework.

Existing tools are available to measure the opinion on agile/scrum adaptation among agile practitioners and engineering students. Hanslo et al. developed a tool to assess the opinion among agile practitioners on scrum adaptation, which yielded four-factor solutions with 67.8% cumulative variance. Three factors were highly correlated with scrum adaptation: sprint management, complexity, and relative advantage. The tool has the higher reliability and validity.[40] Another study by Chow and Cao assessed the critical success factors in agile software projects. The tool focused on agile adaptation in the software projects and identified six-factor solutions focusing on delivery strategy, software engineering techniques, team capability, project management process, team environment, and customer involvement. The psychometric properties of the tool were good.[41]

The tool developed under this research project is not required for any linguistic changes for administration. Moreover, the tool can be freely administered after the agile training, when the students are well versed with the agile process and scrum roles and responsibilities. It requires less time (if the students know the methodology) to complete the questionnaire. However, the stability of the factor structure needs to be replicated on a larger sample.

Conclusion

The 23-item, 6-factor ASPT tool is a brief yet comprehensive tool, which can be used after the agile training and can be used routinely for the students undergoing AM with project management, classroom teaching, and clinical teaching.

Ethical statement

No. NIMHANS/26^th IEC (BEC.SC.DIV.) 2020-21.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.