A Mixed-Methods Study to Explore the Impact of Hospital Accreditation

Abstract

There are several hospital accreditors globally but there is limited understanding of how accreditation impacts on hospital performance and the health system objectives. The objective of the study were to explore the impact of hospital accreditation and inform policy decision-making. We adopted a mixed-methods approach to include an online survey and 3 focus groups. We report 27 of 36 private hospitals who responded to the survey. Key reasons for accreditation were to improve quality (n = 23), implement evidence-based practice (n = 17), continuity of accreditation (n = 15), and popularity (n = 11). Reported improvements include quality of care (27), patient care (26), organizational processes (21), and patient satisfaction (19) among others. Average stakeholder satisfaction rate was 74%. Participants from the 3 focus group discussions felt that staff hours and stress levels were high during the accreditation process, and some standards were useful while others were deemed non-essential. There was support for a local accreditation body with an emphasis on best practice. The findings from the study suggest accreditation to have an impact on structure and process measures, but the gains in key areas were short-lived. There is a need to strengthen governance and develop performance measures to evidence outcome improvement, assure alignment with regulation and the health system objectives.

What do we already know about this topic?

• Accreditation is generally considered voluntary (self-regulatory) and often initiated within the organization however, there are some examples where it has been mandated for reimbursement or quality assurance purposes.

• Accreditation is perceived to facilitate a learning culture and improve performance.

• There are many hospital accreditors globally, yet there is limited understanding of how accreditation impacts on hospital performance and the health system objectives.

How does your research contribute to the field?

• Having a prominent accreditor in the market presents several challenges. It prevents comparative analysis on performance and cost-effectiveness for both hospital and the regulator.

• Approximately 75% of survey respondents reported spending over 100 days on accreditation which could be considered a hidden cost.

• Participants reported cost, visibility, and familiarity of the accreditation requirements as key influencing factors.

• Sustainability of accreditation was raised by participants as lacking.

• The link between accreditation and healthcare quality could be strengthened if there was greater emphasis on measuring quality by accreditors and regulators alike.

What are your research’s implications towards theory, practice, or policy?

• The scope of accreditation should align with regulatory and health system objectives.

• There is a need to performance manage accreditors to ensure added value is achieved among hospital providers and within the health system.

• There is a need for accreditors to be more transparent on their pricing structure, standards and performance measures.

Introduction

Hospitals are under continuous pressure to improve access, healthcare quality and control cost. In response to this challenge, many have adopted accreditation to drive structural, process and outcome improvements. Accreditation is perceived to facilitate a learning culture and improve performance.^1,2 Although the term certification and accreditation are often used interchangeably, they are not the same. Certification provides product, process or service conformity assurance for specified requirements within an organization, however, unlike accreditation, it does not involve external survey, peer review assessment of performance against current healthcare practice or improvement across the entire organisation.³ Accreditation is generally considered voluntary (self-regulatory) and often initiated from within an organization but there are some instances where it has been mandated for reimbursement or for quality assurance purposes.^4,5 In 2014, hospital accreditation was mandated by the regulator in an effort to improve patient safety, healthcare quality, and expedite the transfer best practices in tandem with a growing health sector and medical tourism. Three accreditation bodies were approved by the regulator, Joint Commission International (JCI), Accreditation Canada International, and The Australian Council on Healthcare Standards (ACHS).^6-8 Accreditation was viewed as necessary to standardize professional practices and workflows given that many physicians were from a wide range of international medical backgrounds.⁹

The process for accreditation is generally initiated by the organizations choice for an accreditor, accreditor discussions with the healthcare organization to assist in understanding the basic requirements, agreement on organizational resources (working group) and on the education process for organizational staff followed by submission of preliminary information and an onsite survey by the accreditor. The working group for the organization may include clinical and non-clinical staff working across a variety of departments. The working group is generally responsible to undertake a risk and gap analysis (self-assessment), prepare for the accreditors onsite visit (survey), develop and implement necessary internal policies and action plans, improve compliance and performance, and submit evidence to the accreditors for review. Working group members may entail quality leads, heads of department, estates, managers, directors, clinicians, and operational staff. Accreditors are in most cases independent contractors from different clinical and non-clinical backgrounds and come from different parts of the world to provide independent and expert advice on the requirements and evidence needed to achieve accreditation. Accreditor cost and time spent within an organization varies and is dependent on several factors such as the accreditation type, organizations experience with accreditation, size and scope of organization, geographical location, time and number of accreditors required for the preliminary review, onsite visit and final review. While there are many hospital accreditors globally, there is limited understanding of how accreditation impacts on hospital performance and the health system objectives.^10,11 Given the timeframe since the issuance of the accreditation policy on the health sector and the continued expansion of medical tourism and healthcare market, there was a need to explore the impact of hospital accreditation and inform future policy decision-making.

Methods

A retrospective summative study design with a mixed-methods approach was adopted to include a Survey, and semi-structured Focus Group Discussions (FDGs).^12-14

The study adopted the Donabedian framework to explore the impact of hospital accreditation across private sector hospitals only (Figure 1) as the public sector was not under the scope of the regulator at the time. Contact details for the Clinical Director were extracted from the regulators facility licensing system. Clinical Directors were informed of the study and asked to share the survey link with the person who leads on healthcare accreditation. Survey questions included type of accreditation, reasons for accreditation, number of patients per annum, number of staff employed, whether resources to achieve accreditation were internal or through external support, reported areas of improvement, satisfaction and comments on accreditation. Reminders for the survey were sent out by email at the end of the first week, and a final phone call was undertaken 3 days before the end of the survey period. Survey data capturing and reporting was undertaken through Survey Monkey. Cross tab analysis was used to highlight significant differences (95% confidence level) were performed using Survey Monkey tool (cost of accreditation by number of patients and staff). Data was also extrapolated to Microsoft Excel Spreadsheet for analysis and ranking on the reasons for accreditation and satisfaction responses. Participants from the survey were invited to take part in Focus Group Discussions (FGDs). To capture different perspectives, hospital providers were asked to bring along senior colleagues and members that had a role in healthcare quality. Participants from the survey and focus groups were assured anonymity and consented for their responses to be used to inform best practice and research. Three FDGs were held at the Mohammed Bin Rashid School of Government (MBRSG) to allow for recording and transcribing. FGDs were run by a policy representative from the regulator and 2 MBRSG research staff. Participants were presented with the Donabedian Framework and asked to comment on 5 themes including how structure, process, and outcome were impacted by accreditation within their hospital. FDG data were transcribed and coded for better data management and themed for content analysis by MBRSG. The study approach supports complex policy decision-making for healthcare quality.^15-19 Furthermore, collection of data at a single point aids the efficient use of finite resources within a specified timeframe while taking into consideration the time needed for policy maturation.

Figure 1.

Donabedian framework for healthcare quality.

Ethical Approval

Ethical approval was not required as the study formed part of an evaluation to inform policy decision-making and did not include confidential or patient identifiable information.

Results

Stakeholder Survey

We report 27 of the 36 private hospitals who responded to the survey. Eighteen respondents were from general hospitals, and 9 were from specialized hospitals. Survey respondents included Quality leads (18), Medical Director (3), Chief Nursing Officer (1), Managers (4), and Admin Staff (1). General hospitals reported to accommodate more than 50,000 patients per annum while specialized hospitals reported 10,000 to 50,000 patients per annum. Twenty-three hospitals were Joint Commission International (JCI) accredited, 3 accredited by Accreditation Canada International (ACI), 1 accredited by The Australian Council on Healthcare Standards (ACHS), and 1 was identified as a new hospital and in the process of accreditation. Reasons for accreditation were varied (Figure 2). Reasons include quality (N = 23), evidence base (N = 17), continuation of accreditation (N = 15), popularity (N = 11), relation to service/specialty (N = 9), validity (N = 8), cost (N = 6), and simplicity (N = 1). Ten hospitals reported to spend more than 100,000 AED (27,225 USD) on accreditation per year followed by 3 that spent 81,000 to 100,000 AED (22,000-27,225 USD), 5 spent 61,000 to 80,000 AED (6600-21,780 USD), and 9 spent 0 to 60,000 AED (0-16,335 USD). Twenty-two hospitals reported to spend more than 100 days preparing for accreditation, 3 reported to spend 81 to 100 days, and 2 reported to spend 61 to 80 days on accreditation. We report no association between number of staff and number of patients seen with choice of accreditor. Nineteen hospitals reported to have been through at least 1 accreditation cycle. Ten hospitals reported to have renewed their accreditation status with the same accreditor. Accreditation was reported to have improved in several areas. Improvement was reported on quality of care (27), patient care (26), organizational processes (21), patient satisfaction (19), reduction in adverse or sentinel events (15), leadership (13), staff satisfaction (13), and organizational structure and governance (9). Satisfaction was measured on a scale of 1 to 10 (1 = very unsatisfied to 10 = very satisfied). We report 16 of 27 responses on satisfaction were above the average satisfaction score (74%), 4 reported a 60% satisfaction rate, and 3 responses were below the 40% (Figure 3). There was no significance in the cross tab analysis for cost and staff or patient numbers.

Figure 2.

Reasons for accreditation.

Figure 3.

Hospital satisfaction rate.

Note. Satisfaction scale (1 = very unsatisfied to 10 = very satisfied).

Focus Group Discussions (FDGs)

Participants for the 3 focus groups were twelve (12), eleven (11), and twelve (12), respectively and covered 5 themes. Participants included Quality Leads (15), Nursing Directors (7), Director other (4), Manger (3), Officers (3), Infection Control Lead (1), Chief Executive (1), and Consultant (1).

Theme 1: Views on accreditation

General viewpoints on accreditation were focused on the need to improve healthcare quality. Participants felt accreditation was more holistic and longer-term and included the entire workforce with onsite inspection and review. Accreditation was seen as a platform or framework for the hospital to maintain a level of quality and governance. Advantages include problem finding, better communication, more departmental collaboration, uniformity of standards, and improved branding. The disadvantages included subjectivity of surveyors, too much focus on documentation (tick box exercise), too much emphasis on processes, limited clinical engagement, divisive in terms of who takes on the workload, high levels of stress among staff and questionable benefit in regards to improving patient care and quality.

“Even though there are no published studies that show a strong positive impact, it gives brand visibility. It puts you in the competitive market, gives more room to negotiate, and you are acknowledged more.”

“It stresses the staff because they know a piece of paper will judge them.”

“Management says they don’t want a single non-compliance. It puts a lot of added pressure.”

Theme 2: Views on structure

The majority of participants felt that the accreditor covered design, human resources and scope of work. Governance was a critical feature that was being addressed by the accreditor and governance brought all levels of staff together, which was not always being addressed internally. Participants felt there was room for accreditation to align with the regulator health facility guidelines (structure) and focus more on practice.

“All these accreditation guidelines are new, and some of our facilities are old, so the surveyors need to consider this.”

“They give feedback and what could be improved but spend most of the time looking at documents rather than being on the ground.”

Theme 3: Views on process

Process was defined into 2 categories (clinical and non-clinical). Clinical processes included patient confidentiality, communication, care pathways, and all areas of clinical practice. It was conveyed and perceived that accreditors go through the patient journey through a critical viewpoint to improve efficiencies and question the evidence base but never set out what the process ought to be within the organization. Participants felt the internal time invested in processes was generally to ensure that accreditation is achieved. Non-clinical areas related to process include contract involvement, outsourcing staff recruitment and qualifications, feedback, up-to-date licensing, entry to exit process, supply chain management, and disaster management. Processes that were deemed to have had an impact include organ transplant, disease outbreaks, and emergency preparedness. There was an eagerness to move away from processes and focus more on outcomes.

“Sometimes, it makes us do something that doesn’t necessarily make us better, but we’re just doing it to get the accreditation.”

“We should start with the outcomes rather than focusing on administrative things. We’re not achieving our outcomes because we’re focusing on documentation.”

Theme 4: Views on outcome

Participants reported to use the regulators Key Performance Indicators (KPIs) for clinical practice; however; accreditors do not offer guidance on performance improvement but rather question the reasoning behind their use. Only 1 of the 3 accreditors offered to benchmark hospital performance with an international peer. Non-clinical KPIs included patient visits, satisfaction rate, cancellations, the discharge process, employee satisfaction, and complaints. It was perceived that there was a lot of celebration following accreditation; however the novelty faded within 3 to 6 months primarily due to lack of physician engagement and commitment, and because higher management were reluctant to invest in improvements if it had no bearing toward the achievement of accreditation.

“They don’t look at details.”

“When it comes to quality, it should be done with the organization, not to it.”

“In our group, we have a quality officer who puts the benchmarks. If you don’t have a robust quality team that is impartial and separate from management and staff. . . some things hospitals need to stick to.”

Theme 5: Views on accreditor selection, satisfaction, and the alternative

The regulator mandates accreditors to be approved by the International Society for Quality in Health Care (ISQUA) under ISQUA External Evaluation Association (IEEA), which to some respect restricts the opportunity to look at others. Participants felt that cost was a key factor as it included site visits, accommodation, and in some cases, costly pre-assessments followed by a full assessment. Furthermore, the return on investment was not easily evidenced toward tangible improvements or patient outcomes. For example, digitizing medical records and having a paperless policy would undoubtly improve the access of medical records demonstrable through reduced staffing hours chasing medical records and would lead to cost savings on printing. Decision-making on choice of accreditation was mainly left to the quality team and the organization’s leadership. The majority of participants were satisfied with their accreditor due to the standardization of goals, streamlined processes, continuous improvement plans, and the opportunity to bring teams together. Some participants were neutral because they felt accreditation provides a well-structured framework with a general assessment but without direction. A few participants were dissatisfied because the duration of 3 years was too far apart. Also, the return on investment was questionable, and finding an alternative solution was not easy at present.

“When recruiting staff from different backgrounds when you ask them about their knowledge of accreditation standards. When they know about it then you immediately know they are already on a certain level.”

“Smaller organizations go for accreditation just because its mandatory. Otherwise, a framework could be put in place with the leadership, and they could focus more on healthcare quality and less on accreditation.”

Majority of participants felt that the offering of accreditation by the regulator (or nominated entity) would be a practical alternative. Potential benefits include cost reduction, increased tailoring to the local organization structure and the market, and the ability to benchmark against local and regional peers. Participants expressed their view on the need to maintain international standards and experiences through a diverse group of surveyors. Confidentiality and surveyor subjectivity was also raised as an area that needed to be considered.

Limitations

We report 27 of 36 eligible private hospitals responded to the survey. None of the responses from Survey or FDG participants were from the public sector. Public sector hospitals were not under the scope of the regulator at the time which would entail 4 more hospitals. There may have been some responder bias with responses to the survey and FDGs due to historical relationships between providers and accreditors. Furthermore, time allocated for FDGs may have restricted the time spent on each theme which was approximately 15 to 20 mins. There was no historic data for accreditation and comparative and statistical analysis of survey data against control was not possible due to the nature of policy wide implementation of hospital accreditation. The extent of variables that would need to be accounted for within each hospital setting and health system to measure outcome remain a challenge and therefore a limitation. Finally, the choice of framework for the study may be perceived to provide a narrow viewpoint on the complexity and intricacy of activities and efforts that take place within a hospital setting and indeed, the healthcare system.

Discussion

Survey Data

The findings on cost suggest a variation in pricing. In part, this may be due to the size of hospitals and the scope of services. Mumford et al²⁰ conducted a study on accreditation costs across six hospitals and highlighted the burden cost has on smaller healthcare providers with incremental cost variation by the accreditor survey process. Thus, cost may affect choice and may lead to inappropriate selection of accreditation. Approximately 75% of survey respondents reported spending over 100 days on accreditation which could be considered a hidden cost. This raises questions on the exact cost and potential alternatives for accreditation which may include appointing a full-time quality lead. Key reasons for choosing the accreditor were to improve quality, implement evidence-based practice, and maintain continuity of standards and popularity. These factors may intrinsically be linked with the need to improve healthcare quality, minimize disruption of practices, improve cost savings, attract patients, and influence payers.^21-23 Respondents reported on several areas that had improved due to accreditation, but the findings were varied suggesting wide coverage and lack of focus on priority areas thus, a one size fits all approach may limit opportunities for improvement.^24,25 Having a prominent accreditor in the market presents several challenges. It prevents comparative analysis on performance and cost-effectiveness for both the hospital and the regulator.²⁶ It inhibits the creation of contextual knowledge on which surveys and standards are most effective and may skew opportunities for other accreditors to enter the market due provider perceptions on branding alone. In part, the presence of a dominant accreditor in the market was reflected through the satisfaction scores.

Focus Group Discussions (FGDs)

Participants report to have benefitted on structure by having in place a framework and development tool within their healthcare setting in addition to improved branding. In this regard, accreditation was viewed to have a positive effect on structure and seen as essential for hospitals, especially in circumstances where the culture for continuous improvement within the organization is not prioritized.²⁷ Governance and staffing levels were seen as an area that had improved through accreditation, but participants suggested there were some areas of misalignment in facility design between the accreditor and the requirements set out by the regulator. Facility design is known to influence the provision of care and should, therefore, be aligned to meet local need.^28,29

Viewpoints on process suggest categorization of processes into clinical and non-clinical setting, but not all the components within these 2 domains were considered. Those that were critiqued were done in the context of justification of need rather than benchmarking or assuring core process elements are in place, for example, accreditors lacked oversight on processes that should be in place from the point of patient admission to discharge. The flexibility for healthcare providers to opt-in and out of best practice processes, although cost-saving may lead to variation in practices. It may also affect patient outcomes. For example, the absence of a surgical checklist can lead to adverse or sentinel events and concerns about patient safety.³⁰ While processes may lead to outcome it is plausible that outcome as a starting point will lead back to either good or bad processes.³¹

Participants felt that focus on outcome was primarily being directed by the regulator outcome measures. While accreditors take on an inquiry approach to clinical performance measures, participants reported that accreditors only question why specific outcomes are being measured. In this regard, there is an opportunity for accreditors to direct interventions that will enhance performance in tangible areas such as Patient Reported Outcome Measures (PROMS).³¹ The establishment of the International Society for Quality in Health Care External Evaluation Association (IEEA), by ISQUA in 2018 to deliver external evaluation services on organization, standards and training is likely to benefit accreditors on quality over the coming years.³²

Sustainability across accreditors was raised by participants as lacking. The prospect of waiting 3 years for another survey was too far away with a tendency for critical improvements to have diminished within 3 to 6 months. Lack of clinical engagement was reported to be an important factor among others such as lack of tools for sustainability, data infrastructure, staff turnover, seasonal priorities, staff morale, absence of nominated leads within departments, and lack of alignment with core objectives. Assuring sustainability and continuous learning is essential for transformational change throughout the organization therefore, greater emphasis should be placed on sustainability.^33-35

Accreditation selection was influenced by several factors and differed from the survey findings. Participants reported cost, visibility, and familiarity of the accreditation requirements as key influencing factors. It could be perceived that these issues relate to cost and brand value as well as the expectations for improvement. In addition, ten hospitals reported to have gone with the same accreditor upon renewal due to simplicity and continuation of existing requirements and familiarity of staff. This includes but is not limited to documentation, healthcare standards and workflow processes. Some participants reported strength of brand in the market to also influence their decision but this was determined by senior management. Participants were generally satisfied with their accreditor but felt more could be done to evidence outcome improvement. In this regard, some participants had decided to either explore or try other accreditors who were more focused on improving outcomes. While participants were eager for accreditors to benchmark their performance against local and international peers, they also expressed the need to account for several variables to assure fairness, for example, number of beds, specialization, patient groups, and methodology for data collection and reporting.

Viewpoints for the regulator to develop a local accreditation scheme for hospitals was supported with interest. Participants perceived the presence of a local accreditor to be better priced, more transparent with their pricing structure and offer of greater value. They also perceived the local accrediting entity to have greater flexibility to tailor to their needs with local knowledge of what works. While this may hold true, there are challenges in setting up a fit for purpose entity over a short timeframe and simultaneously assure continuous improvement. Also, it would be prudent for the regulator to maintain its function to oversee the performance of both accreditors and hospital providers.

Conclusion

The findings from the study suggest accreditation to have an impact on structure and process measures, but the gains in key areas were short-lived. There is a need to strengthen governance and develop performance measures to evidence outcome improvement, assure alignment with regulation and the health system objectives.

Implications

There are several implications from the study. First, the scope of accreditation should align with regulatory and health system objectives. Second, performance metrics are needed by local regulators to determine the extent hospital accreditation improves healthcare outcomes. Third, efforts by IEEA should be directed toward transparency of pricing structure, standards and the extent by which accreditors influence improvement in clinical outcomes. Finally, further research is needed to determine what impact hospital accreditation may have on Patient Reported Experience Measures (PREMS) and the wider community.