Abstract
Aim
This study aims to improve the efficiency of the application for registration process for internationally educated nurses seeking licensure to practice.
Background
The licensure and employment of internationally educated nurses has been one strategy to address the global nursing shortage. However, little is known about which application characteristics relate to success in obtaining licensure.
Design
This project uses evidence from a retrospective statistical analysis of four years of internationally educated nurse application data to inform the development and implementation of changes to policies and practices at the College and Association of Registered Nurses of Alberta. Analysis of application data will also be conducted to evaluate the impact of the changes on outcomes and timelines.
Methods
The project encompasses four phases, with funding from Health Canada's Internationally Educated Health Professionals Initiative approved in March 2011. Phase One focuses on a statistical analysis of application data to identify characteristics associated with success in the application process and quantify timelines for the phases of the process. The resulting knowledge will inform Phase Two, the development and implementation of policy and practice changes. Further analysis will be completed in Phase Three, comparing outcomes and timelines between pre‐ and postimplementation data using statistical analyses and exemplar‐based statistics. Phase Four will focus on dissemination and knowledge transfer, potentially leading to further changes.
Discussion
Findings, policy and practice adaptations and implementation evaluation will provide evidence about the internationally educated nurse application for registration process to Registered Nurse regulators, educational institutions, internationally educated nurses, employers and governments.
Keywords: decision‐making, nurses, professional regulation, quantitative approaches, research in practice, research methods, statistics
Why is this research needed?
The Learning from Experience research project will provide evidence to support assessment decisions made during the internationally educated nurse application for registration process.
Evidence will be used to develop policy and practice changes designed to improve the efficiency of the internationally educated nurse application for registration process and changes will be evaluated statistically.
Data analysis and policy and practice changes will build capacity and leadership in the area of nursing regulation.
Introduction
Global shortages of Registered Nurses (RNs) have been well documented by various organizations including the World Health Organization (2002) and Canadian Nurses Association (2009). One strategy to address this shortage is the employment of internationally educated nurses (IENs) (Canadian Nurses Association 2009). Recognizing the scope of this strategy, the International Council of Nurses (2007) developed a position statement on ethical nurse recruitment in regard to international migration.
However, while there is considerable literature published on the international migration of nurses as a whole, most of it focuses on acclimatization and acculturation of the nurse in the receiving country, especially related to employment (Sherman & Eggenberger 2008, Blythe et al. 2009, Xu et al. 2010, Walani 2013, Spetz et al. 2014). Little evidence is available surrounding licensure and the registration process (Jeans et al. 2005, Ogilvie et al. 2007, Benton 2011), or about application characteristics that support success in health professional licensure. Yet the successful transition of IENs is also a regulatory issue (Xu 2010) as nurse registration is critical to the livelihood of IENs, the health workforce, the health system and public safety.
Background
In Alberta, a Western Canadian prairie province, healthcare employers responded to the global nursing shortage by launching international recruitment initiatives in 2007–2008. Consequently, the College and Association of Registered Nurses of Alberta (CARNA) experienced an approximately ten‐fold increase in the number of applications from internationally educated nurses over a short period of time, challenging the administrative and assessment capacity of the college. With the benefit of financial support from the provincial government and employers, CARNA was able to respond to the increased volume of applications and develop processes and expertise to increase operational capacity.
In 2009, a province‐wide restructuring of the healthcare system during a global economic recession led to a dramatic change in the employment market for RNs. Locally, this included a temporary halt to the hiring of IENs and subsequently the number of applications CARNA received dropped sharply. With the resulting respite from the heavy demand, CARNA registration staff members recognized an opportunity to analyse the IEN application data and develop evidence to inform their IEN application for registration policies and practices.
The study
Aim
The Learning from Experience: Improving the Process of Internationally Educated Nurses' Application for Registration (LFE) research project seeks to improve the efficiency of the IEN application for registration process while upholding CARNA's commitment to public safety and to the principles of fairness and transparency. The project strives to enhance the process by making it more evidence‐based, clear and concise while building capacity and leadership in nursing regulation. It is funded by Alberta Health through Health Canada's Internationally Educated Health Professionals Initiative (funding approved in March 2011).
This study protocol reflects the methodologies that will be used to analyse retrospective application file data from a peak application time to inform the development of policy and practice changes for implementation. Further analysis will investigate pre‐ and postimplementation data to evaluate the changes.
Design/methodology
The LFE Project study design is a pre–post quasi‐experimental design with an intervention, without a control group. IEN application for registration processes vary across jurisdictions therefore it is not possible to include a control group. The project occurs in four phases. In Phase One, data from 4 years of IEN applications received by CARNA will be analysed to provide evidence to inform the IEN application for registration process. The resulting knowledge will inform Phase Two, the development and implementation of policy and practice changes at CARNA and recommendations about the delivery of remedial education. During Phase Two, assessment decisions will be based on evidence from the data analysis. In Phase Three, further statistical analysis will be completed and outcomes will be compared between pre‐ and postimplementation data. The findings and policy and practice adaptations will provide information to other RN regulators, educational institutions, IENs, employers, and provincial and federal governments in Phase Four which focuses on dissemination and knowledge transfer.
Phase one: retrospective review
Sample
During this phase, the LFE Project will review application files received from IENs by CARNA between 1 January 2008–31 December 2011. Applications will be included or excluded in this phase of the study based on the scope of the study, timeframe and practice considerations.
Data from applicants meeting the following criteria will be included in Phase One of the LFE Project:
Internationally Educated Nurse (entry‐level nursing education received outside of Canada)
Completed application (all application documents received) between 1 January 2008–31 December 2011
Assessed under the Health Professions Act (Government of Alberta 1999, Schedule 24)
Applications that were ‘active’ or ‘lapsed’ (24 months of inactivity) after the application was considered completed
The following consideration will also be applied:
Re‐applications: The applicant's most recent data will be used in place of the original application data.
Data collection
Information is gathered as part of routine data collection during the application for registration process and reported in CARNA's electronic database and/or paper application files. Existing data fields in the database will be reviewed for inclusion in the project data and modifications will be made to include additional data fields that are identified in a review of paper files, literature review, environmental scan and stakeholder consultations. Data available only from paper files will be entered manually. Each file will be individually reviewed by a data entry clerk to ensure consistent data entry within the entry process, including audits to ensure accuracy. Procedural changes will be made in the Registration Services department to facilitate these additional fields as a permanent administrative change to allow for ongoing electronic capture of this data.
Variables representing the applicants' demographics, education, licensure history, nursing practice information and process outcomes; and variables representing process timeline data will be analysed. An exploratory analysis will be conducted to identify variables for inclusion in confirmatory analysis modelling. Process date variables will be used to understand application for registration process timelines and will be analysed separately from the other variables. Some timeline data may also be used in modelling.
Outcome variables provide information on key points in the IEN application for registration process as follows:
Initial Assessment Outcome: The assessment decision made by CARNA following a review of the applicant's education, licensure history and nursing experience.
Substantially Equivalent Competency (SEC) Assessment Outcome: The assessment decision made by CARNA following the review of the report from an assessment of the applicant's nursing competencies. The SEC Assessment is an evaluation designed to determine whether the applicant possesses the required entry‐to‐practice competencies to fulfil the roles and responsibilities expected of RNs in Alberta, Canada. It uses a variety of strategies to assess professional knowledge, skills, attributes, values and judgements including: multiple choice and short answer exams, interview‐based case management situations, clinical lab role play situations and self‐assessment.
Course Completion Outcome: Whether the applicant has successfully completed the assigned bridging education.
Temporary Permit (TP) Eligibility: Whether an applicant has been deemed eligible for a Temporary Permit (thereby able to work as a Graduate Nurse in an Alberta practice setting) and able to write the national RN entry‐to‐practice exam (i.e. considered to have competencies that are substantially equivalent to an Alberta nursing school graduate).
National Registered Nurse entry‐to‐practice Exam Outcome: Whether an applicant has passed or failed the national RN entry‐to‐practice exam.
Initial RN Registration Outcome: Whether an applicant obtains RN registration in Alberta.
IEN application data are collected at CARNA for administrative purposes. While highly relevant for decision‐making, there are several limitations including quality control (Mack et al. 2005, Statistics Canada 2014). Yet information from administrative databases can be very accurate therefore, given the nature of the data in this study, data cleaning is appropriate in lieu of validity procedures (Herbert et al. 2004). Data entry and cleaning will follow a standardized protocol comparing information from paper applications with the database. An auditing process will be implemented during data entry to increase the accuracy of the application data for all data sets. Further data cleaning will also be undertaken after each data extraction to increase the reliability and accuracy of the data.
Data analysis
Statistical analysis of characteristic data will be conducted in two stages: exploratory and confirmatory data analysis, to examine the association of application characteristic variables with outcomes. A timeline analysis based on date information for each step in the IEN application for registration process will also be conducted.
Exploratory data analyses will be conducted using the SPSS (Version 20) and SAS 9·2 statistical programs. Descriptive statistics (frequency, cross tabulation, chi‐square test, variance, mean and histograms) will be generated for each variable. Continuous variables will be evaluated both as continuous and categorical variables during the exploratory data analyses. They will be categorized and described by frequency and in cross tabulations. The categories for continuous variables in the final analyses will be based on advice from the Registration Services department at CARNA. For example, age will be analysed as a continuous variable by providing mean, standard deviation and range. When age is considered in the final model, it will be categorized to allow the nonlinear relationship to be modelled while controlling for other confounding factors in the model.
Confirmatory data analysis will be carried out using the STATA 12 statistical program. If there is less than two per cent of the data missing from the variable, it will still be used for analysis. Pairwise missing data will be used for descriptive analysis in valid variables. Listwise missing data will be used for confirmatory analysis in regression models.
Logistic regression analysis with stepwise selection will be conducted to summarize the independent associations of multiple exploratory variables simultaneously. Stepwise logistic regression will provide a list of candidate variables for the final model. Additional variables may be identified for inclusion in the final model based on relevance, experience and logical rationale.
Nursing is a culturally embedded practice and it is often assumed that application outcomes would be similar for those applicants educated in the same country (Benton 2011). Therefore, a multi‐level logistic regression model will also be used to analyse the association between applicants' education country and their outcomes with consideration for individual differences. Multi‐level modelling with two levels, individual and country, will be developed. All information collected from the applicants will be analysed at the individual level.
Country level specific information will not be collected for this study. For example, information about the country's education system, health system and economics will not be compared with Alberta. As well, some countries may only have a few applicants included in the study (<5) and therefore will be aggregated into groups to have more balanced data to fit a regression model. Random‐intercept models will be used to explain the variations of outcomes and education country group effects will be modelled as random intercepts. Intraclass correlation coefficients (ICC) are a way of calculating whether specific data points are more alike in groups than across groups (Woltman et al. 2012). Thus, ICC will be calculated for each model to measure the correlation of observations in a country group.
Logistic regression analysis with stepwise selection will also be conducted to narrow the number of candidate variables. Model diagnosis, goodness of fit tests and type III error tests of fixed effects will also be performed to ensure models are valid and inferences are statistically sound according to best practice.
For the timeline analysis, process date variables will be used to understand application process timelines and will be analysed separately from the other variables. Timeline analysis will be based on date information for each phase in the IEN application for registration process and grouped according to major process areas (e.g. Time to TP refers to the time from the date the complete application is received until the date the applicant applies for their first Temporary Permit). Descriptive statistics will be generated for each timeframe.
Phase two: development and implementation of policy and practice changes
Based on the outcomes of the retrospective data analysis, members of the research team (representatives from CARNA, the University of Alberta and Mount Royal University), CARNA's Registration Services department and CARNA's Registration Committee will develop policy and practice changes for the IEN application for registration process at CARNA during facilitated sessions. Major findings from the statistical analysis will be combined with existing practice and the knowledge and expertise of the registration assessors to identify viable policy and practice changes that could be implemented within the study timeframe. Collaboration with the educational institution that offers bridging education in Alberta may enable the policy and practice changes to include alterations to the approach to bridging education.
Phase three: pre‐ and postimplementation analysis
Sample and data analysis
Analysis of the pre‐ and postimplementation data will inform the evaluation of the policy and practice changes. The statistical analysis that will be employed for the retrospective review (Phase One) will likewise be conducted on both the pre‐implementation (completed applications received between 1 January 2012–26 August 2013) and postimplementation (completed applications received between 27 August 2013–31 May 2015) data sets. However, as the anticipated sample sizes will be significantly smaller than the retrospective review, an exemplar‐based analysis will also be employed for comparison between the pre‐ and postimplementation data. The research team will choose and describe exemplars based on frequent characteristics to represent the modal case or best example in the study populations. Exemplars will be derived from an integration of the statistical and theoretical descriptions of the most common scenarios representing groups of applications.
Univariate analyses will be performed on both pre‐implementation and postimplementation data. Demographic statistics will be provided for both data sets. To measure the intervention effects and minimize bias due to applicants' characteristics and other potential risk factors, the comparison analysis will be performed on both exemplar samples and overall samples by controlling confounding factors such as education country group, education credential and practice currency. As IEN application for registration processes vary across jurisdictions it is not possible to include a control group in the exemplar analysis.
Statistical tests (for example, independent t‐tests or non‐parametric Mann–Whitney U‐tests) will be used to compare the means of each continuous outcome for exemplar groups from pre‐ and postintervention data. Frequency and Chi‐square tests will be performed for categorical outcomes for both comparison groups. The Chi‐squared tests will be used to determine whether there is a significant difference between the expected frequencies and the observed frequencies in one or more categorical outcomes. If there is any significant association between the differences in the two data sets and application outcomes through descriptive analysis, confirmatory analysis will be performed. A hierarchical linear model will be built on outcomes using demographics and intervention group indicators as candidate variables.
Survival analyses will be used to compare the timelines among different exemplar groups and to overall data. Kaplan–Meier survival function estimation will be used to describe the median time interval for the process for both pre‐ and postimplementation data sets, while Log‐Rank tests will be used to compare the timeline intervals between pre‐ and postintervention groups for exemplars, analysing covariates (such as demographic, intervention effects) that may be associated with the quantity of outcome timelines. A Cox regression model will be built to estimate the relationship of multiple candidate variables to process time intervals.
Phase four: knowledge transfer
Phase Four will focus on dissemination and knowledge transfer, potentially leading to further changes at CARNA or for other IEN application for registration stakeholders.
Ethical considerations
Following the completion of an ethics screening tool developed by the Alberta Research Ethics Community Consensus Initiative Network (www.aihealthsolutions.ca/arecci/screening) and discussions with Network staff, it was determined that this project did not require approval from an ethics committee as there is minimal risk and data will be analysed and reported at an aggregate‐level.
Discussion
Findings from the data analysis in Phase One were used to inform the development and implementation of policy and practice changes in Phase Two, which in turn will be evaluated in Phase Three. Interventions that were developed in Phase Two focused on the development of a framework for initial assessments, changes to the application process policies and practices focused on increasing efficiency and the revision and development of communication tools to increase transparency.
The exemplars for the Phase Three data sets will also be developed based on the Phase One data analyses results and consultation with CARNA registration assessors. Three criteria will be used to stratify data to extract exemplars:
Education country group/education in a country with a similar nursing scope of practice;
Education credential; and
Practice currency (defined as 1125 hours of practice or graduation within 5 years prior to application).
If the exemplar sample size is large enough after applying these preliminary criteria, additional selection criteria, such as number of years since last practiced or years of employment experience, will be applied to further refine the exemplar sample groups.
Limitations
IEN application information is collected as part of routine data gathering during the application for registration process. The project will draw data from an active database not designed for research purposes therefore the project data will not be entered and cleaned in isolation. The research team, data entry clerks and data analyst will thoroughly review and clean the data used for this analysis; however, variations for future data analysis may exist.
The study sample in Phase One includes four years of applications submitted to CARNA during a period of very high application volumes. The characteristics of the applications may be influenced by recruitment efforts undertaken by Alberta employers.
Characteristics analysed in this study will be identified based on a review of the hard copy files, literature review, environmental scan and stakeholder consultations. However, this study is unable to measure some of the primary variables affecting IEN integration into the workplace as described in the literature: culture and language (Alberta Network of Immigrant Women 2002, Benton 2011, Higginbottom 2011). While information on English language test results is collected, there is no requirement for IEN applicants to submit failed English test results therefore attempts at analysis of the influence of this factor would be inconclusive. As well, demonstration of English proficiency is mandatory on application and throughout the process. Likewise, information contained in the data does not allow for assessment of the impact of variables such as culture, family support, immigration challenges and conflicting obligations which experience and literature indicate are significant factors in the IEN application for registration process and transition to the workforce (Tregunno et al. 2007, Philippine Nurses Association of Manitoba 2012).
Many components of the IEN application for registration process are outside of CARNA's direct control and it is recognized that the implementation of the policy and practice changes will not occur in isolation of external factors. Outside influences may impact the validity of the pre‐ and postimplementation comparison. External factors will be noted in the analysis of the pre‐ and postimplementation data and evaluation of the policy and practice changes.
Conclusion
There is little research on application characteristics that support success in health professional licensure therefore the Learning from Experience: Improving the Process of Internationally Educated Nurses' Application for Registration research project will provide substantial evidence to inform IEN application for registration processes for regulatory bodies across Canada and the world. The project is also designed to support future collaboration with other jurisdictions for data collection, analysis and policy development.
The policy and practice changes adopted during the study may impact the SEC Assessment process and bridging education programs in regard to volumes and delivery models and there may be further implications based on the pre‐ and postimplementation findings. The project process and outcomes will support evidence‐based policy that may lead to improved healthcare human resource system planning and the resulting health system performance. Findings from this project also have the potential to increase collaboration on and coordination of responses to several major healthcare system priorities, particularly around the effective use of health human resources. This is evident from the underlying intention to include and consider the practices of other jurisdictions, stakeholder consultations and symposiums throughout the project. Ultimately, this project strives to inform an improved assessment process that will contribute to a higher quality and safer healthcare system.
Funding
The Learning from Experience: Improving the Process of Internationally Educated Nurses' Applications for Registration project is funded by Alberta Health through Health Canada's Internationally Educated Health Professionals Initiative. The views expressed herein do not necessarily represent the views of Health Canada or Alberta Health.
Conflict of interest
No conflict of interest has been declared by the author(s).
Author contributions
All authors have agreed on the final version and meet at least one of the following criteria [recommended by the ICMJE (http://www.icmje.org/recommendations/)]:
substantial contributions to conception and design, acquisition of data or analysis and interpretation of data;
drafting the article or revising it critically for important intellectual content.
Acknowledgements
The authors acknowledge the dedicated work of the members of the LFE Research Team and Data Entry Clerks, and Shelby Corley's contribution to the development of this publication. They also acknowledge the active participation and tremendous support of the College and Association of Registered Nurses of Alberta Registration Services department and Leadership Team.
Giblin C., Lemermeyer G., Cummings G., Wang M. & Kwan J.A. (2016) Learning from experience: improving the process of internationally educated nurses' application for registration – a study protocol. Journal of Advanced Nursing 72(3), 650–657. doi: 10.1111/jan.12859
[The copyright line for this article was changed on 25 April 2016 after original online publication.]
References
- Alberta Network of Immigrant Women (2002) Access to Licensure for Foreign Qualified Nurses. Alberta Network of Immigrant Women, Calgary, AB. [Google Scholar]
- Benton D.C. (2011) International issues and trends in nursing regulation. Journal of Nursing Regulation 1(4), 4–8. doi:10.1016/S2155‐8256(15)30316‐1. [Google Scholar]
- Blythe J., Baumann A., Rhéaume A. & McIntosh K. (2009) Nurse migration to Canada: pathways and pitfalls of workforce integration. Journal of Transcultural Nursing 20(2), 202–210. doi:10.1177/1043659608330349. [DOI] [PubMed] [Google Scholar]
- Canadian Nurses Association (2009) Tested Solutions for Eliminating Canada's Registered Nurse Shortage. Canadian Nurses Association, Ottawa, ON. [Google Scholar]
- Government of Alberta (1999) Schedule 24. Health Professions Act. Government of Alberta, Edmonton, AB: Retrieved from http://www.qp.alberta.ca/documents/Acts/H07.pdf on 01 October 2010. [Google Scholar]
- Herbert M., Prince S. & Williams J. (2004) Are unaudited records from an outcomes registry database accurate? Annals of Thoracic Surgery 77, 1960–1965. doi:10.1016/j.athoracsur.2003.12.018. [DOI] [PubMed] [Google Scholar]
- Higginbottom G. (2011) The transitioning experiences of internationally educated nurses into a Canadian health care system: a focused ethnography. BMC Nursing 10(1), 14–26. doi:10.1186/1472‐6955‐10‐14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- International Council of Nurses (2007) Ethical Nurse Recruitment. Position Statement. International Council of Nurses, Geneva, CH: Retrieved from http://www.icn.ch/images/stories/documents/publications/position_statements/C03_Ethical_Nurse_Recruitment.pdf on 07 November 2011. [Google Scholar]
- Jeans M.E., Hadley F., Green J. & Da Prat C. (2005) Navigating to Become a Nurse in Canada: Assessment of International Nurse Applicants. Canadian Nurses Association, Ottawa, ON: Retrieved from https://www.cna-aiic.ca/~/media/cna/page-content/pdf-fr/ien_technical_report_e.pdf?la=en on 07 November 2011. [Google Scholar]
- Mack M., Herbert M., Prince S., Dewey T.M. & Magee M.J. (2005) Does reporting of coronary artery bypass grafting from administrative databases accurately reflect actual clinical outcomes? Journal of Thoracic and Cardiovascular Surgery 129, 1309–1317. doi:10.1016/j.jtcvs.2004.10.036. [DOI] [PubMed] [Google Scholar]
- Ogilvie L., Leung B., Gushuliak T., McGuire M. & Burgess‐Pinto E. (2007) Licensure of internationally educated nurses seeking professional careers in the province of Alberta in Canada. Journal of International Migration and Integration 8(2), 223–241. doi:10.1007/s12134‐007‐0015‐y. [Google Scholar]
- Philippine Nurses Association of Manitoba (2012) CRNE Preparation for Internationally Educated Nurses (IENs) – The Manitoba Way. 6th National Conference for Educators, Provincial Regulators and Employers of IENs, Winnipeg, MB. [Google Scholar]
- Sherman R.O. & Eggenberger T. (2008) Transitioning internationally recruited nurses into clinical settings. Journal of Continuing Education in Nursing 39(12), 535. doi:10.3928/00220124‐20081201‐03. [DOI] [PubMed] [Google Scholar]
- Spetz J., Gates M. & Jones C.B. (2014) Internationally educated nurses in the United States: their origins and roles. Nursing Outlook 62(1), 8–15. doi:10.1016/j.outlook.2013.05.001. [DOI] [PubMed] [Google Scholar]
- Statistics Canada (2014) Use of Administrative Data. Retrieved from http://www.statcan.gc.ca/pub/12-539-x/2009001/administrative-administratives-eng.htm on 01 June 2015. [Google Scholar]
- Tregunno D., Campbell H., Allen D., Sousa D. & College of Nurses of Ontario (2007) Internationally Educated Nurses (IEN) Knowledge Translation Project Report: Learn from My Experience. College of Nurses of Ontario, Toronto, ON: Retrieved from http://www.cno.org/Global/docs/policy/IENexternalsummaryjan2308.pdf on 31 October 2012. [Google Scholar]
- Walani S.R. (2013) Earnings of the internationally educated nurses in the U.S. labor market. Nursing Research 62(3), 169–177. doi:10.1097/NNR.0b013e318286b7ac. [DOI] [PubMed] [Google Scholar]
- Woltman H., Feldstain A., MacKay J.C. & Rocchi M. (2012) An introduction to hierarchical linear modeling. Tutorials in Quantitative Methods for Psychology 8(1), 52–69. Retrieved from http://www.tqmp.org/RegularArticles/vol08-1/p052/p052.pdf on 19 February 2012. [Google Scholar]
- World Health Organization (2002) Nursing and Midwifery Services Strategic Directions 2002–2008. World Health Organization, Geneva, CH: Retrieved from http://www.who.int/hrh/nursing_midwifery/strategic_directions_en.pdf on 21 February 2013. [Google Scholar]
- Xu Y. (2010) Is transition of internationally educated nurses a regulatory issue? Policy, Politics & Nursing Practice 11(1), 62–70. doi:10.1177/1527154410371382. [DOI] [PubMed] [Google Scholar]
- Xu Y., Zaikina‐Montgomery H. & Shen J.J. (2010) Characteristics of internationally educated nurses in the United States: an update from the 2004 National Sample Survey of Registered Nurses. Nursing Economics 28(1), 19–25. [PubMed] [Google Scholar]
