Design of an instrument to measure the quality of care in Physical Therapy

ABSTRACT

Objective:

To design an instrument composed of domains that would demonstrate physical therapy activities and generate a consistent index to represent the quality of care in physical therapy.

Methods:

The methodology Lean Six Sigma was used to design the tool. The discussion involved seven different management groups staff. By means of brainstorming and Cause & Effect Matrix, we set up the process map.

Results:

Five requirements composed the quality of care index in physical therapy, after application of the tool called Cause & Effect Matrix. The following requirements were assessed: physical therapist performance, care outcome indicator, adherence to physical therapy protocols, measure whether the prognosis and treatment outcome was achieved and Infrastructure.

Conclusion:

The proposed design allowed evaluating several items related to physical therapy service, enabling customization, reproducibility and benchmarking with other organizations. For management, this index provides the opportunity to identify areas for improvement and the strengths of the team and process of physical therapy care.

INTRODUCTION

Knowledge of the work applied to a certain population helps in critically assessing results and guides the decision-making process. The current indicators in the field of physical therapy are still poorly developed and disseminated. These indicators are not a package of information that can guide the results in respect to quality of care in a physical therapy service. In cases where they already exist and integrate a history of data collection, they are usually restricted to the organizations. The future will be permeated by this work methodology, which, some time from now, will allow organizations to disclose their results. The next step after determining the indicators to be used in the project is sharing this information, announcing results and creating possibilities to improve healthcare services in the country and the quality of care in physical therapy.

In a brief retrospective analysis of the use of evaluations in healthcare, the oldest historical accounts can be found in ancient Greece. The classical philosophers already spoke about the use of standards of knowledge in the medical art. Plato, in some of his speeches, pointed to concepts of excellence and highlighted the importance of comparing these products with each other and most importantly with the respective standard.⁽¹⁾

The development of tracking (monitoring) and evaluation instruments is a great challenge, because evaluating is quite different from meeting schedules, keeping track of the actions performed, and improvising. Therefore, we often need to create or improve evaluation indicators.

Indicators, as the name suggests, have the purpose of measuring specific situations. They are used as per conventional standards or can be created based on specific methodologies. Nowadays, there is an increasing number of qualitative assessments, which are mostly used in process evaluations. From this perspective, the use of evaluations with the development of indicators relevant to local realities can contribute not only to managerial work, but to the entire planning process.⁽²⁾

There is a solid concern with customer satisfaction and the quality of services provided. In evaluation, emphasis is given to the effectiveness of actions, i.e. to the impact exerted by the results of those actions/ services. Effectiveness means resulting in changes in a certain reality or transforming a certain situation. This concept is widely used to evaluate results.

Thus, some healthcare settings seem to encourage this type of approach. As of 1990, initiatives related to the process of hospital accreditation started in Brazil. The Ministry of Health created the Brazilian Program of Hospital Accreditation, aiming to implement a process to improve patient care. This accreditation program promoted, through financial incentives for retrofitting and renovating hospitals, improvements in patient care and hospital management.⁽³⁾

Many different tools can be used to monitor quality indicators. Surveys, charts and metrics can be used to measure and assess the quality of a service. This should provide the necessary information to improve the service at issue.

The tools we selected for this proposal are applicable to quality management. The first is brainstorming, a broad instrument capable of providing a more systemic view of the object at issue. To improve the focus, the most important aspects of the brainstorming were discussed point-by-point using a tool known as Cause & Effect Matrix. This allowed us to propose the instrument considering the variability between the service volume and the number of professionals. To this end, we developed a logical table, including the items to be evaluated with their proper weights, according to the volume and the type of physical therapy service.

The overall consensus showed that a complete and accurate set of quality indicators would enable organizations or companies to focus on their strengths and find the appropriate solutions for their weakenesses. The correct review and interpretation of quality indicators should be the goal of the efforts in order to allow for appropriate decision-making.⁽⁴⁾

Quality is maintained through the constant assessment and reassessment of performance and efficiency. The quality of the product or service may even increase with the improvements or innovations implemented to push the bar. A service may become a reference after it has exceeded all others in terms of quality.⁽⁵⁾

Choosing the appropriate indicator for quality is essential for an accurate and detailed evaluation of a product or service. It is important that managers know which answers, in respect to indicators, they are looking for to develop their product or service. Being able to provide a superior product or service to customers represents a competitive advantage for any company or organization. Maintaining and improving quality is key for the success and sustainability of organizations.^(6–8)

Managers are always seeking to prospect new customers, have quality and safety indicators, design new products, shorten production cycles and ensure employee satisfaction. The measure expected for this end is adding performance to the quality assurance process.

Considering the lack of consensus in the literature on which components should integrate the construct, as well as the lack of consistency in the studies previously performed, we defined the main results indicators recognized by the management to form opinions on the quality of care in physical therapy. In physical therapy, there are some initiatives to assess the quality of care but they are very fragmented and focused on isolated procedures.

These initiatives advocate assessing the quality of care, for example, based on high volume processes such as the discontinuation of mechanical ventilation. The result is interpreted as quality of care in physical therapy, but it is clearly just a partial picture of the quality of care. They do not contemplate aspects related with professional education and experience of the staff with protocols, or even the reach of this quality of care.

That is why we proposed the development of a methodology to allow for a consistent and representative evaluation of physical therapy services. Isolated actions to measure the quality of care do not represent a set of standards against which physical therapy staff can be evaluated. Having a set of indicators to assess the quality of physical therapy care allows for better control and establishment of the level of service to be provided by physical therapists.

OBJECTIVE

To design an instrument composed of different domains to demonstrate the activities of the physical therapy service and generate a consistent index to express the quality of care in physical therapy.

METHODS

To design the instrument, we used the process review and outline methodology known as Lean Six Sigma. This methodology is structured for problem-solving and continuous improvement of processes through the use of statistical tools aimed to identify opportunities for process enhancements. The most important factors were related with increased customer satisfaction, creating competitive differentiators against the competition, productivity gains, cost savings, increased profitability, focus on eliminating defects from the customer's perspective, eliminating process variability and decision-making requirements based on facts and data.⁽⁹⁾

This instrument was specifically developed as a variation of the method known as “development of new products and processes” (within the Lean Six Sigma methodology).

The discussion involved seven different management groups. The mapping of the process relied on the brainstorming tool and the Cause & Effect Matrix. The brainstorming involved a description of the items for evaluation of the quality of care in physical therapy. We discussed the items currently assessed and those that were not yet assessed but were relevant in determining the quality of care in physical therapy. The brainstorming - the items of the survey - were subject to ranking by weight and importance divided into the following pillars: knowledge of the case (weight 2), training and skills of the professional (weight 3), clinical decision (weight 5) and treatment outcome (weight 5). These four pillars of patient care were graded in the following order of correlation: strong correlation if graded 10 – 9 – 8; medium correlation if graded 4 – 5 – 6 – 7; low correlation if 3 – 2 – 1; no correlation, if zero. After reviewing the scores and weights, we applied the Cause & Effect Matrix.

After this step, we finished developing the instrument composed of some requirements from which we set up a logical table including the items to be evaluated with their corresponding weights according to the volume characteristics and type of physical therapy service. All requirements were composed of indicators previously discussed in the working group, representing the results of quality physical therapy care.

The endpoints were represented by an index calculated against the volume of each management unit and one calculated against the requirements. In the end, the results were compiled in two ways: (1) observing the global behavior of the requirements for all management units; (2) observing the behavior of each management unit separately, for all requirements.

The indicators making up these two indices allowed us to find the service outcomes and create mechanisms to appropriately monitor them, ensuring quality of care in physical therapy. This study was submitted to the Research Ethics Committee of Hospital Israelita Albert Einstein and approved under number 270/2011 - CAAE: 0125.0.028.000-11.

RESULTS

The results were demonstrated in two ways: first, globally, allowing the indices derived from the development tools to be viewed, and second, with detailed indicators demonstrating the consistency of the requirements.

The brainstorming found 28 items distributed throughout the four pillars of patient care (knowledge of the case, training/skills, clinical decision and treatment outcome) as described in the Cause & Effect Matrix.

Each item under the four pillars was graded 0 to 10. After completing the matrix, we defined the five items with the highest score, which were interpreted as the source for the five requirements composing the index of quality of care in physical therapy (Figure 1).

Figure 1. Description of the 28 items covered in the brainstorming and the Cause & Effect Matrix.

The requirements of the evaluation instrument resulting from this phase were the following: measure whether the treatment prognosis and outcomes were achieved (150 points); measure the care outcome (144 points); evaluate adherence to physical therapy protocols (140 points); grade the minimum infrastructure criterion for physical therapy care (144 points); and individually assess the physical therapist performance (131 points). From these five requirements, we determined which processes consistently represented the service outcome.

We built a logical table (Figure 2) to define the calculations represented in each indicator and a process map (Figure 3) involving the different management units.

Figure 2. Logic to define the weights of the requirements.

Figure 3. Description of the process map demonstrating the method for calculation of the indicators related with requirements.

The following are the detailed requirements respecting the specificity of each service profile.

Requirement 1: individual assessment of the physical therapist's performance

This requirement included the checklist, a technical test and adherence to the training sessions proposed to the physical therapy team (Figure 4).

Figure 4. Description of the service quality checklist applied to the requirement.

This item of the domain included four factors shared by all management units, which could be customized by management unit within the subject. Each of the items should be graded zero (no), 5 (partial) or 10 (yes).

The items evaluated in the interview were: item 1 - Knowledge of the case (25%); item 2 - therapeutic reasoning (prognosis/approach) (25%); item 3 - systemic view of the therapy (involves multidisciplinary view and differentiated solutions) (25%); and item 4 – evolution routine (attesting items 1, 2 and 3) (25%).

This score was translated into percentual compliance. A compliance range was then established to define the quality target of the physical therapist's performance at 85%:

Concept of the technical test

In this factor, the evaluation was formally conducted by means of a theoretical test applied to the staff, with minimum pass score over 6.1 (performance target compliant with expectations (over 6.1):

Adherence to training sessions

In this factor, the information was the percentage of professionals attending the training sessions proposed, i.e. the percentual adherence of the staff to training sessions.

Requirement 2: package of service outcome indicators

For each indicator, an expected range of results was established. Outcome targets were established and included into a package of indicators to be rated as compliant/non-compliant with the targets for these indicators. The results of the indicators were described as percentages. The reference values set as parameters for each indicator were based on evidence from the Literature⁽⁸⁾ or the organization history (values are described for the respective.

Indicator 1

Indicator 1 focused on the evolution of mobility/ functionality. The physical therapy deliverable was to track the evolution of mobility through measurement instruments based on the characteristics of the treated population. The therapist must apply and interpret the results according to the prognostic loss or gain of mobility of patients. This result demonstrates the correctness of the approach used by the team in maintaining the mobility/functionality of patients.

The reference value was established based on the scores chosen by each of the participating management units. We used the Functional Independence Measure (FIM) score for adults, and the Alberta Infant Motor Scale for children. The FIM consists of items that assess postural changes, independence in performing daily activities (such as getting dressed), and cognitive assessment items. The score is determined by the total sum of these items:

Indicator 2

Indicator 2 was the perception of dyspnea based on the Borg scale. The most common symptom in patients on respiratory physical therapy was the presence of dyspnea (Figure 5).

Figure 5. Description of the Borg score for assessment of dyspnea.

The physical therapy deliverable was the use of procedures to improve this symptom. We must be able to measure whether the physical therapy approach decreased dyspnea symptoms for treated patients, demonstrating the correctness of the therapy applied by the team.

The reference value is determined after applying the score at the beginning and end of the therapy. The goal was to decrease the score by 2 points. This value was based on the historical results obtained by the physical therapy service at issue:

Indicator 3

The spontaneous breathing trial (SBT) time was indicator number 3. The physical therapy deliverable was subjecting the patient to the SBT for 2 hours. This result demonstrated the correctness of the use of this approach by the team:

The reference value was as follows: the test had to last for 2 hours to enable validation of the therapeutic decision to extubate the patient - the reference value was 12 to 14% of the total patients extubated according to the plan.

Indicator 4

Extubation failure was indicator number 4. The physical therapy deliverable was patient extubation according to the weaning protocol criteria, demonstrating the correctness of the approach as performed by the team. The indicator must also measure the rate of failure in mechanical ventilation weaning.

Indicator 5

The fifth indicator was the change in physical activity levels. The physical therapy deliverable was to track the level of physical activity performed by patients over the target period. This measure pointed to changes in the behavior related with inclusion of physical activity in patients’ lives, and the reach of the target within this population.

The reference value was an increase of at least 1 point in the physical activity evaluation.

Requirement 3: assess the adherence to physical therapy protocols

Adherence within this domain meant that the quality of the service depended on whether the patient/individual had been enrolled in the protocol established as the best practice.

Requirement 4: measure whether the prognostic treatment outcome was achieved

For this domain, we measured whether the prognostic outcome of the treatment was achieved by counting the objectives fulfilled as proposed by the team during the physical therapy training. This item was assessed by reviewing the therapy goals page in the patient chart. The sample was calculated for a 15% error rate.

We use a menu of objectives with metric values and/or service level for the print copy, such as: “Walk with _________ aid within _______ days (use FIM)”:

Requirement 5: infrastructure

In this domain, we considered three aspects that contributed for the best practice, namely the service level, as defined by the relationship between patient demand and the number of physical therapists; equipment and materials used for quality therapy performance, and the appropriate physical area according to the characteristics of the service. For this domain, we defined the minimum structure items for good patient care. Each item corresponds to 1 point and is considered over the total items included in each of the three aspects in inpatient and outpatient care (Figure 6).

Figure 6. Table of points to be measured for Requirement 5 - Infrastructure.

DISCUSSION

This index translated the results of each management unit, i.e. it was able to identify the most important activity for each management unit. This means that, according to the proposed requirements and the structure of the logical matrix used to develop the index, it may be used for any type of physical therapy service due to its reproducibility and customizability. This also means that the service outcome indicators (or any other indicators used to establish the requirements) may be chosen according to the strategic planning of the area or replaced whenever the previous indicators cease to be of interest or become unstable, without the need for further monitoring and control.

Some authors identify two phases in the process of defining quality. The first phase is the correction of errors and the second is the fine-tuning of the appropriate standard. This fine-tuning includes not only the service deliverable or outcome, but also the knowledge inputs that can guide service improvements. The outcomes also depend on the patients’ interaction and reaction and as well as on the behavior of the professional, and therefore are difficult to measure. These same authors point out that feedback in the private sector is easily detectable by its variation.⁽¹⁰⁾

The Institute of Medicine (IOM) defines quality in healthcare as “the degree to which health services provided to individuals and populations improve the likelihood of desired results, consistent with current professional knowledge.”⁽¹¹⁾

We emphasize that the effort to measure outcomes is directly related to patient satisfaction.⁽¹²⁾

Currently, we still face difficulties in finding publications about physical therapy. Despite the ubiquitous integration of physical therapy professionals into hospital staff and the discussions about inpatient care indicators, whether specific or not, physical therapy is not covered in the assessment of indicators.⁽¹³⁾

Therefore, the proposal to build a composite index that makes sense to determine the quality of a physical therapy service is more robust when compared to isolated indicators representing parts of the service process.

CONCLUSION

The instrument proposed enabled the assessment of several items related with the physical therapy service, allowing for customization, reproducibility and benchmarking among institutions. For management purposes, this five-requirement index demonstrates the quality and training of professionals, the service outcomes, the results of therapeutic planning and the structure of the physical therapy service. It creates an opportunity to identify points for potential improvement, as well as the strengths of the staff and the physical therapy service process.