Development of a framework to measure health profession regulation strengthening

Abstract

This paper describes the development of a framework to evaluate the progress and impact of a multi-year US government initiative to strengthen nursing and midwifery professional regulation in sub-Saharan Africa. The framework was designed as a capability maturity model, which is a stepwise series of performance levels that describe the sophistication of processes necessary to achieve an organization’s objectives. A model from the field of software design was adapted to comprise the key functions of a nursing and midwifery regulatory body and describe five stages of advancing each function. The framework was used to measure the progress of five countries that received direct assistance to strengthen regulations and to benchmark the status of regulations in the 17 countries participating in the initiative. The framework captured meaningful advancements in regulatory strengthening in the five supported countries and the level of regulatory capacity in participating countries. The project uses the framework to assess yearly progress of supported countries, track the overall impact of the project on national and regional nursing regulation, and to identify national and regional priorities for regulatory strengthening. It is the first of its kind to document and measure progress toward sustainably strengthening nursing and midwifery regulation in Africa.

1. Introduction

Achieving universal access to anti-retroviral therapy (ART) and other AIDS-Free Generation targets for 2015 targets will require an even greater scale-up of HIV¹ services in sub-Saharan Africa (UNAIDS, 2010; WHO, UNAIDS, & UNICEF, 2011). Across much of this region, nurses and midwives play an increasingly important role in delivering HIV care, including initiating and managing ART, which is being integrated in the prevention of mother-to-child transmission (PMTCT) of HIV (De Cock, El-Sadr, & Ghebreyesus, 2011; McPake & Mensah, 2008; Van Damme, Kober, & Kegels, 2008; WHO, 2012, 2013). Recent recommendations by the World Health Organization (WHO) reinforce the importance of nurses and midwives in initiating and maintaining HIV-infected patients on first-line antiretroviral therapy and recognize their role as essential to the rational distribution of HIV care and treatment tasks among health workforce teams. Furthermore, global guidance for safe and sustainable task sharing within a health system include ensuring that health policies permit task sharing for HIV care and health professional regulation reflect those policies (IOM, 2010; WHO/PEPFAR/UNAIDS, 2008). Health professional regulation is intended to protect the public by ensuring the safety and quality of health professional practice and education (ICM, 2011; ICN, 2009; Walshe, 2003). Nursing and midwifery councils are typically responsible for issuing and updating various practice and education regulations (ICM, 2011; ICN, 2009). These activities may include efforts to expand the scope of practice, such as, authorizing nursing initiated and managed ART (NIMART); requiring standard in-service trainings or updates, referred to as continuing professional development (CPD), for routine re-licensure; and accrediting HIV curricula taught in pre-service education programs (ICM, 2011; Miles, Clutterbuck, Seitio, Sebego, & Riley, 2007; Morris et al., 2009). However, not all nursing and midwifery councils in east, central, and southern Africa have adequate resources (financial, human, or technical) or capacity to undertake necessary changes associated with practice regulation (McCarthy et al., 2013; Munjanja, Kibuka, & Dovlo, 2005; Nabudere, Asiimwe, & Mijumbi, 2011). The objectives of this paper are twofold: first, to describe a nurse and midwifery initiative funded by the President’s Emergency Plan for AIDS Relief (PEPFAR) that strengthens health professional regulation and supports NIMART task-sharing, and secondly, present an evaluation framework that assesses the impact of this initiative.

The African Health Profession Regulatory Collaborative (ARC) for nurses and midwives is a 17-country initiative created to bolster the capacity of nursing and midwifery regulatory bodies and strengthen regulation in east, central, and southern African (ECSA) (McCarthy & Riley, 2012). The initiative is a partnership between the U.S. Centers for Disease Control and Prevention, Emory University, the Commonwealth Secretariat, Commonwealth Nurses Federation, and ECSA College of Nursing. ARC is a regional south-to-south (i.e., peer led) collaborative in which national nursing and midwifery leadership teams convene annually with global experts to discuss and identify priorities for modifying or implementing regulation which will facilitate task sharing and movement toward reaching HIV targets (PEPFAR, 2012). Through ARC’s annual competitive grant process, country leadership teams, led by the national nursing and midwifery council, propose projects that address a priority for nursing and midwifery regulation in the context of their national HIV scale-up strategy. Selected proposals receive up to $10,000 for a 12-month project period. During the year, country teams receive technical assistance (TA) visits and attend two meetings to share successes and challenges in implementing their projects and receive feedback from other countries and technical experts (Gross, McCarthy, & Kelley, 2011). The ARC approach is modeled after the Institute for Healthcare Improvement “clinical collaborative” model (IHI, 2003).

As of July 2013, ARC supported 11 regulation strengthening grants in ten countries (Table 1). Seven of 11 ARC grants have focused on developing or strengthening CPD regulation. CPD–which refers to education following completion of formal training–is a necessary component for maintaining professional practice standards and is in some instances a requirement for professional re-licensure or re-certification (Iliffe, 2011). Four grant proposal submissions targeted other aspects for strengthening professional councils, such as updating standards for midwifery educators and strengthening the role of the national nursing council. Over a 12-month period ARC convened two learning sessions designed to assist country team implementation of their respective project. In addition, ARC provided in-country TA by request. Each funded country team submitted quarterly accounts of progress toward stated objectives and an end-of-project report.

Table 1.

ARC grants by country, year, and regulatory strengthening priority.

Country	Regulatory strengthening priority
ARC Year 1 (July 2011–June 2012)
Lesotho	Develop a continuing professional development framework for nurses and midwives
Malawi	Strengthen the existing continuing professional development framework for nurses and midwives
Mauritius*	Establish standards for midwifery tutors providing pre-service education
Seychelles*	Review and revise the national nurses and midwives act
Swaziland	Develop a continuing professional development framework for nurses and midwives
ARC Year 2 (July 2012–June 2013)
Botswana	Develop a national continuing professional development framework for nurses and midwives
Kenya	Decentralize key regulation services (registration and re-licensure) to zonal offices
Swaziland	Strengthen the existing continuing professional development framework for nurses and midwives developed in ARC year 1
Tanzania	Finalize and launch the national continuing professional development program
Uganda	Establish scopes of practice for all nurse and midwife cadres
Zimbabwe	Strengthen the existing continuing professional development framework for nurses and midwives

^*Grant supported by the Commonwealth Secretariat.

The ARC organizers were also interested in evaluating the initiative’s impact and identifying a standardized method of assessing the progress or change resulting from this novel approach to regulatory strengthening. However, to the best of their knowledge, an appropriate instrument for this purpose did not exist. Accordingly, the ARC partners collaborated with participating country teams to design a tool capable of measuring the effectiveness of the ARC approach in strengthening professional regulation. The tool needed to represent varying levels of regulation present in participating ARC countries; capture the types of improvements supported by ARC grants; and reflect global nursing and midwifery standards to which all countries aspire. The purpose of this paper is to describe the framework developed by ARC partners to evaluate the impact of this initiative on the capacity of nursing and midwifery regulatory councils and present examples of how it can be used to benchmark current capacity, measure progress, and target areas needing regional and national regulatory strengthening.

2. Tool design

The ARC tool was designed as a capability maturity model (CMM) (Humphrey, 1987). A CMM is an approach to assessing, in a structured, sequential manner, an organization’s ability to perform necessary functions (Paulk, Weber, Curtis, & Chrissis, 1994). A CMM is created by identifying an organization’s essential functions and describing the maturation of each function according to a linear scale of increasing capability (Paulk et al., 1994). The scale for each essential function comprises five discrete and successive stages, beginning with a stage in which capability is low and ending with a stage in which it is high (Gillies & Howard, 2007). Each stage is characterized by key competencies instrumental to advancing to the next stage. Together the stages create an “evolutionary improvement path” upon which organizations can advance (Fig. 1) (Paulk et al., 1994). Progression through the stages is intended to be sequential with advancement to a stage representing a meaningful improvement in functioning (Humphrey, 1987). The generic nature of the CMM makes it adaptable to use by a variety of groups or disciplines interested in specific organizational improvements (Gillies & Howard, 2007; Paulk et al., 1994).

Fig. 1.

Stepwise progression through five stages of a capability maturity model.

Design of ARC’s CMM began with a literature search to identify functions highly relevant to the nursing and midwifery councils involved in ARC within the context of HIV programming and national scale-up of services. To ensure alignment with normative guidance, functions for the ARC CMM were selected by reviewing the regulatory elements included in nursing and midwifery global and regional standards recognized by the International Council of Nurses (ICN), the International Confederation of Midwives (ICM), the World Health Organization (WHO), the United Nations, and the East, Central, and Southern Africa College of Nursing (ECSACON) (ECSACON, 2001; ICM, 2011; ICN, 2007; UNFPA, 2011; WHO-EMRO, 2002) (Table 2). Regulatory elements included by three or more normative bodies were chosen for the ARC CMM.

Table 2.

Essential nursing and midwifery regulatory elements per normative bodies.

Nursing and midwifery regulatory element	ICN Model nursing act (ICN, 2007)	ICM Global standards for midwifery regulation (ICM, 2011)	WHO Nursing and midwifery a guide to professional regulation (WHO-EMRO, 2002)	WHO/UNAIDS/PEPFAR Task shifting global recommendations and guidelines (WHO/PEPFAR/UNAIDS, 2008)	ECSACON Professional regulatory framework (ECSACON, 2001)
“Standards”	✓
“Education”	✓	✓
Competencies and content					✓
Registration	✓	✓	✓	✓
Licensure		✓	✓	✓
Scope of practice		✓		✓	✓
Continuing professional development		✓	✓	✓
Accreditation	✓	✓		✓
Professional conduct & discipline	✓	✓	✓	✓
Revision of nursing and midwifery legislation	✓	✓	✓	✓
Education standards			✓		✓
Practice standards			✓		✓

In order to characterize the five different levels of capability in regulatory functioning in the region, a CMM from Carnegie Mellon University was adapted to reflect advancement of the regulatory processes, the use of technology, and the use of measurement to improve the quality of regulatory processes (Table 3) (Humphrey, 1987). For example, in the “planning” stage, the regulatory processes are defined as basic; data collection is typically “ad hoc” and done without the use of technology. At the opposite end, the highly functional “optimizing” stage, the regulatory processes are sophisticated and efficient and include the use of technology and data to continuously improve regulatory processes. The ARC CMM delineates stages of functional maturity with real world examples using national regulations, recommendations, and stated obstacles to strengthening nursing and midwifery practices that are referenced in the peer-reviewed and grey literature (ICN, 2007; Miles, Seitio, & McGilvray, 2006; PEPFAR, 2012; Uebel, Fairall, Van Rensburg, Mollentze, & Bachman, 2011; USAID, 2010; Zachariah et al., 2009). Literature which included specific challenges and recommendations for nursing and midwifery regulatory reform in the context of HIV scale-up in the ECSA region was considered highly relevant.

Table 3.

Application of capability maturity model stages to nursing and midwifery regulation.

Stage	Description
1	Regulations not in place or not uniformly applied throughout the country. Paper-based systems are used instead of electronic technology. Data collection is ad hoc.
2	Regulations exist in basic forms across the country or new regulations are being piloted in certain settings. Minimal technology used. Data collection on basic indicators.
3	Regulations are well-established across the country. Systems are primarily electronic. Data collection is systematic and can reflect compliance with regulations.
4	Regulations are comprehensive and compliance with them is high. Only electronic systems are used. Data is automatically generated and used for advanced queries and performance analysis.
5	All regulations reflect best practices and align with regional standards or global guidelines. Technology is sought out to improve performance. Data is used to understand and continually improve the effectiveness of regulations.

This process resulted in a draft CMM with seven distinct and essential functions associated with regulatory oversight of nursing and midwifery practice in the ECSA region and five stages of each function. The draft CMM was presented to ARC country teams at regional meetings in June and October 2011. Participating teams provided individual feedback on the tool and participated in a group discussion of each specific function and how best to describe stages of advancing capability. Interested country teams pilot-tested the CMM by selecting the stage (1–5) which best characterized the current state of each of the seven regulatory functions in their country. If a stage did not adequately reflect a country’s status with respect to a regulation, the team was instructed not to select a stage for that function. Feedback from the participating countries suggested the ARC CMM adequately reflected the current level of capability in each regulation and captured incremental changes in regulations.

2.1. The regulatory function framework

The resulting CMM for ARC is called the regulatory function framework (RFF) (Fig. 2). The RFF comprises seven regulatory functions and five progressive stages for each function. The seven functions are (1) developing or revising nursing and midwifery legislation, (2) registration and use of registration data, (3) licensure, (4) scope of practice, (5) continuing professional development, (6) accreditation of pre-service programs, and (7) professional conduct and discipline. These seven functions are not exhaustive but encompass recommendations by the normative bodies and are relevant to the adaptation of regulation to support task sharing. Each function has five stages which together describe a stepwise progression in advancing the processes, technology, and measurement involved in carrying out the function; the fifth stage of each function represents the attainment of regional standards or global guidelines. Each stage comprises up to three characteristics or criteria of that stage, all of which must be met in order to be in considered in that stage. To use the ARC CMM, one would select a function and begin by reading the description of Stage 2. If a council’s activities in this function did not meet the three criteria in Stage 2, they would be considered in Stage 1 of this function. If a council met all the criteria in stage 2 they would then look at the characteristics of Stage 3. If the only two of three criteria of Stage 3 are met, the function would be considered in Stage 2 until the third criteria is met, at which point they would advance to Stage 3.

Fig. 2.

The ARC regulatory function framework (RFF).

3. Tool application and results

Country teams funded in year 1 used the RFF to report progress by indicating the stage they were in when they received their ARC grant (solid shaded square) and the stage they were in at the end of the one-year project period (circled square) (Fig. 3). Lesotho and Swaziland began their project at Stage 1 of CPD regulation – the ministry of health had issued a policy that all health professionals should be required to undergo CPD to ensure competency, but the council had not yet designed the CPD regulation for nurses and midwives. Over the course of the ARC initiative, both countries developed a draft of the rules and requirements for CPD and developed a plan for implementing the regulation in pilot districts. This progress moved both countries from Stage 1 to Stage 2. Malawi also had a CPD project but began at Stage 3 and wanted to strengthen compliance with the regulation and increase the use of technology. Through ARC, Malawi improved the process of delivering CPD to nurses and midwives; however, at the time of reporting, they had not yet fully developed the system to electronically track which nurses met the CPD requirement. For that reason, the Malawi team reported that their activity has not yet advanced to Stage 4. When Seychelles received their one-year ARC grant, they were at Stage 1 of revising the national nursing and midwifery legislation. Over the course of the ARC initiative, Seychelles moved from gathering consensus with stakeholders, to commitment by the Ministry of Health to advance the draft legislation (Stage 3). Mauritius advanced to Stage 2 with draft legislation that had stakeholder and MOH support, but decided to revise it for wider stakeholder buy-in. Without that accomplishment, Mauritius’ regulatory function did not advance to Stage 2 by the end of the one-year ARC project period.

Fig. 3.

Progress in ARC countries in year 1. (For interpretation of the references to color in text near the reference citation, the reader is referred to the web version of this article.)

At the ARC Summative Congress in June 2012, all countries used the RFF to indicate their current stage on each of the regulatory functions (Fig. 4). This information allowed for cross-country comparisons of nursing and midwifery regulation and the capacity of councils to carry out key regulatory functions in the ECSA region. For example, the function of providing CPD for nurses and midwives is at the lowest overall stage – 14 of all 17 countries are either in Stage 1 or 2 and no countries in the region are yet above stage 3. Six ARC countries have no formal nursing or midwifery scope of practice. Accreditation of pre-service education is also an area where ARC countries are in the earlier stages of functioning – almost all ARC countries are in Stages 1–3. The broadest continuum of stages is in the conduct and discipline function; nine ARC countries are in Stage 3 or above for the licensure function. Notably, 12 of 17 countries are in Stages 1 or 2 of the registration and use of registration data function.

Fig. 4.

Stages (1–5) of maturity of seven regulatory functions in 17 countries.

4. Discussion

A CMM for assessing the impact of ARC-supported efforts to improve key nursing and midwifery regulations was developed and vetted with stakeholders. Feedback and pilot testing indicated that the RFF adequately reflects actual stages of regulations of countries in the ECSA region and could be used to assess progress with national and regional regulations. The RFF helps set a common pathway for improvement and documents progress not only if countries reach Stage 5, but also by identifying meaningful incremental achievements. When used to assess country progress during ARC year 1, the RFF successfully captured each country’s respective baseline status of capability in the prioritized regulation and measured where meaningful advancements were made. Our experience suggests the RFF can be a valuable instrument for measuring capacity building.

When used by country teams and the ARC organizers, this tool facilitates objective assessment of regulatory functions and targets areas needing technical assistance with greater precision. For example, in response to the low capacity in CPD (Fig. 4), ARC supported an increasing number of countries (eight) to develop or strengthen their CPD programs; created a step-by-step “CPD Toolkit” for developing a national CPD system; provided in-country TA on CPD to seven countries; and facilitated south-to-south sharing of CPD resources and tools, such as CPD needs assessments and tracking systems during ARC meetings. In year 3, ARC will assist countries with incorporating NIMART-specific in-service trainings into CPD system requirements. The similarly low level of capability found in accreditation of pre-service education revealed this topic as also needing ARC’s technical support. In response, ARC included an expert panel on nursing and midwifery accreditation at a regional meeting and devoted an entire day of the ARC Summative Congress (July 2013 in Nairobi, Kenya) to the WHO’s health professional education recommendations for pre-service accreditation systems. Because the RFF identified several countries lacking updated scopes of practice consistent with WHO’s 2013 HIV Guidelines, this topic was subsequently prioritized in ARC’s year 3 request for proposal guidance.

By documenting actual capabilities, the RFF allows for consideration of regulation in terms of stages of organizational aptitude, as opposed to just the presence or absence of certain regulations. Regardless of what stage a given regulation is in (excluding Stage 5), the RFF provides the criteria for reaching the next stage. This feature facilitates setting improvement goals and helps identify appropriate actions to strengthen regulations. ARC encourages countries applying for grants to use the RFF in assessing their country’s current stage in their prioritized regulation and what stage or criteria they would like to achieve over the course of the year. When used this way, the RFF is a helpful tool for country-level planning and regulatory priority-setting with respect to advancing task sharing. By identifying areas needing regulatory strengthening, the RFF facilitates more targeted and efficient use of limited resources while generating evidence for sustaining (or revising) investments over time. The RFF could be easily adapted for use with other health care cadres, such as laboratory professionals, clinical officers, or other mid-level or senior level clinicians in this or other regions of the world. Global health initiatives might apply the RFF principles to the inherently difficult task of measuring the impact of capacity building and health system strengthening efforts.

The regulatory elements in the RFF are consistently recommended in global and regional guidance from ICN, ICM, WHO, and ECSACON. Further alignment with normative bodies was ensured by incorporating the global or regional standards into stage 5 of each element. Development of the RFF complements calls in the peer-reviewed literature for updating regulatory frameworks by providing practical examples of activities that countries and donors alike can undertake in support of national regulatory reform.

There are a number of limitations to the RFF. While the RFF has been vetted by stakeholders, it has not yet been formally validated. Furthermore, the RFF was developed in conjunction with those it intended to measure, thus potentially reducing the objectivity of the tool. In some countries, attributing advancements in regulation solely to ARC would overlook the contributions of others groups providing health systems support. Attribution of effect is further complicated with countries leveraging ARC grants to secure additional support for regulatory strengthening from other development partners (something ARC encourages). The RFF does not incorporate all the functions of a regulatory council nor reflect all the issues involved in advancing regulation. Instead, it focuses only on a limited set of regulatory functions and deliberate actions to improve them. To remain relevant, the RFF must continuously evolve to reflect changing practice and education guidelines, technological advancements, and to incorporate user feedback. More formal validity testing of the RFF is needed to strengthen its scientific application and assess its contribution to the evidence base on improving national human resources for health infra-structure.

The RFF provides the ARC initiative a regionally relevant tool with which to benchmark the capacity of national nursing and midwifery councils for carrying out key regulatory functions and for measuring the impact of efforts designed to strengthen regulation. ARC will use the RFF to better understand the capacity building needs especially pertinent for advancing task sharing, which is seen as critical for realizing an AIDS-free generation. Lastly, the RFF is important for generating a health systems evidence base, documenting improvements in the field of regulation and for fostering dialog regarding standards in health workforce regulation. With continued use, the RFF can contribute to wider discourse on planning and performance measurement in the field of health workforce or regulation.

Biographies

Carey F. McCarthy, PhD, MPH, RN, is a Health Systems Scientist in the Division of Global HIV/AIDS at the U.S. Centers for Disease Control and Prevention. She is a Health Systems Scientist in CDC’s Division of Global HIV/AIDS. She trained as a registered nurse in emergency care before pursuing a Masters in Public Health at Johns Hopkins University. After a number of years working in global health, she completed a PhD program in Nursing Science at the University of Washington. She did her doctoral research on nursing and midwifery regulation with the Division of Global HIV/AIDS and then joined the Health Systems and Human Resources Team.

Maureen A. Kelley, PhD, CNM is the Chair of the Family and Community Nursing Department in the Nell Hodgson Woodruff School of Nursing at Emory University. She is also faculty in the Lillian Carter Center for Global Health and Social Responsibility. She is globally recognized expert in midwifery and midwifery education. She was involved in launching the first university-based midwifery program in the Southeast United States.

Andre R. Verani, JD, MPH, is a Health Policy Advisor in the area of Health Systems and Human Resources for Health in the Division of Global HIV/AIDS at the U.S. Centers for Disease Control and Prevention (CDC). Andre earned his law degree at New York University and his public health degree from Emory University where he is adjunct professor of global heath law.

Michael E. St. Louis, MD is the Regional Country Manager for Southern Africa in the Division of Global HIV/AIDS at the Centers for Disease Control & Prevention (CDC) where he oversees PEPFAR activities in sub-Saharan Africa. He began his career as a Epidemic Intelligence Service and has been in the Public Health Service for over 30 years.

Patricia L. Riley, MPH, CNM, FACNM, is the Senior Technical Advisor for the Maternal and Child Health Branch of the Division of Global HIV/AIDS at the Centers for Disease Control and Prevention (CDC). Pat has over 30 years of experience working in and evaluating public health programs at the state, regional and federal level.