Studying the Experience of Care Through Latent Class Analysis: An Application to Italian Neonatal Intensive Care Units

Abstract

Though many data on the experience of care of patients and caregivers are collected, they are rarely used to improve the quality of health care delivery. One of the main causes is the widespread struggle in interpreting and enhancing these data, requiring the introduction of new techniques to extract intelligible, meaningful, and actionable information. This research explores the potentiality of the latent class analysis (LCA) statistical model in studying experience data. A cross-sectional survey was administered to 482 parents of infants hospitalized in several Italian neonatal intensive care units. Through a 3-step LCA, four subgroups of parents with specific experience profiles, sociodemographic characteristics, and levels of satisfaction were identified. These were composed of parents who reported (1) a positive experience (36%), (2) problematic communication with unit staff (30%), (3) limited access to the unit and poor participation in their baby's care (26%), and (4) a negative experience (8%). Through its explorative segmentation, LCA can provide valuable information to design quality improvement interventions tailored to the specific needs and concerns of each subgroup.

Introduction

Patient experience is increasingly recognized as a key indicator of health care quality (1,2), consistently and positively associated with other “traditional” quality measures, such as clinical effectiveness and patient safety (3,4). Patient experience data allow identifying critical touchpoints and shortcomings in the areas where poor experiences are reported and can inform and stimulate quality improvement interventions. However, their value is currently under debate, as their actual use to drive quality improvement in different health care settings is often limited (5–7). Possible causes of such result include a lack of the necessary time, resources, and staff training to collect, analyze, and act on patient experience data as well as general skepticism among health staff about their validity (6,8,9). Others may relate to the absence of appropriate organizational and leadership structures to engage with these data, which leads to misalignment among various health professional groups, such as management and frontline staff (10,11). Furthermore, their implementation might depend on the organizational recognition, or rejection, of the specific forms in which patient experience data can be conveyed (9).

While most of the commentators have focused on contextual elements of organizational, attitudinal, and cultural inertia, less attention has been paid to the technical approaches used to analyze patient experience data. Methodological innovation should be one of the key instruments to employ to underpin the science behind patient experience (7), easing the overall confusion about its value and interpretation. Patient experiences are usually collected through surveys (12): new techniques capable of extracting intelligible, meaningful, and actionable information from them should be explored.

In this paper, we propose the use of a novel statistical model: latent class analysis (LCA). As opposed to traditional “variable-centric” analyses that investigate how variables relate to each other (e.g., in regressions), LCA is “person-centric,” determining how variables combine across individuals (13). In so doing, the methodology allows for the investigation of heterogeneity across individuals by letting latent classes emerge from data. Therefore, populations are clustered in distinctive latent classes, each of which presents similar behaviors. The methodology has been previously and successfully applied to several fields of social sciences (13–15), and is here innovatively applied to study experience data.

The setting of our study is the neonatal intensive care unit (NICU), where infants with the most severe medical conditions—usually related to extreme prematurity—can be hospitalized for up to several weeks. For obvious reasons, in the NICU, the experience of care cannot be collected directly from the little patients. Therefore, parents are the ones reporting their experience, accounting for the care they and their children receive (16–18). For parents, the period in the NICU is often traumatic with varying forms of psychological distress that can be developed such as depression, emotional pain, anxiety, and acute stress (19,20). No parent can be fully prepared for the extreme range of emotions associated with caring for a critically ill newborn and the unknown, unfamiliar, and sometimes hostile environment of an intensive care unit. Furthermore, after their baby's discharge, these parents are at a high risk of developing post-traumatic stress disorders and encountering difficulties in the development of their parenting role (21,22). For these reasons, in the NICU, it is of vital importance to take appropriate care of parents too. In this sense, parents should be allowed to be present in the NICU 24/7 with few or no limitations, participate in the daily care of their baby, be involved in medical decision making, be engaged in frequent, clear, honest, and empathic communication with the health staff, and receive different forms of support (23,24). These latter represent the main principles of the family-centered care (FCC) model, which is the gold standard of assistance in the NICU and has been systematically proved to be beneficial for both infants and their parents (25). Evaluating the parent experience can provide evidence of whether and how these practices are applied in the NICU (26).

Given these premises, this paper aims to discover the potentiality of LCA in analyzing experience data by applying the methodology to the study of the parent experience of NICU. Specifically, the following research questions were formulated:

• What is the capability of LCA in interpreting and deriving meaning from experience data?

• What are the advantages of using LCA to study the parent experience of NICU?

Methods

Design

A cross-sectional design was employed.

Setting and Data

This research was carried out in collaboration with the Italian Association of Neonatology (“Società Italiana di Neonatologia”, SIN). Parent experiences were collected using a shorter version of the Neonatal Survey designed by the Picker Institute Europe. This questionnaire aims to determine the extent to which the provided care is “family-centered” (27). The survey was translated and adapted to the Italian context. Survey validity was provided by content revision, performed by the neonatologists of the SIN, and a pilot test carried out in one NICU located in northern Italy. The questionnaire (Appendix A) is composed of 14 items investigating the following dimensions of the experience of parents: confidence and support received, communication and information sharing, involvement in child care and medical discussion, and empowerment in decision making. Furthermore, the survey collects parents’ and infants’ descriptive characteristics with eight nominal questions. Finally, an item on the overall satisfaction with care in the NICU (evaluated with a 5-point Likert scale) is posed at the end of the questionnaire. The survey was addressed to the parents of infants who were hospitalized in any Italian NICU during the six months before survey completion. Survey distribution was implemented in Fall 2019 by contacting several parents of preemies associations. The associations that decided to participate in the study forwarded to their members the invitation to complete the survey online on Qualtrics^®. The choice of a specialized web-based survey platform guaranteed an automatic de-identification and anonymization of the collected responses. All participants gave informed consent to join the study.

Data Analysis

Frequency distributions were used to report parents’ and infants’ characteristics and the overall level of satisfaction. Moreover, similar to the Neonatal Survey study (27), a score (from 0 to 100) was associated with each experience question. The higher the score, the better the parents evaluated their experience for that specific item. According to their scores, the experience items were divided into three ranges:

• Score > 75: positive experience.

• 75 ≥ Score ≥ 50: poor experience.

• Score < 50: negative experience.

After the preliminary descriptive analysis, we employed a 3-step LCA. LCA is a mixture modeling approach that allows grouping observations that are similar on the basis of a number of indicators, employed to identify latent classes (14). Thus, latent class membership of each observation is defined based on dichotomous indicators that, in our study, were represented by the experiences of NICU parents.

Step 1 of the analysis consisted of defining the number of subgroups (i.e., latent classes) that better described our data. Response options for the 14 experience indicators were dichotomized. This means that those response options indicating poor or negative experiences were coded as 0, while options denoting positive experiences were coded as 1 (i.e., we created binary indicators with only two possible values). To decide on the most suitable number of latent groups, classes were added stepwise (i.e., incrementally) and statistical fit indexes were compared across models (28). These are indexes that allow selecting the best number of classes on the basis of objective statistical metrics, thus reinforcing the robustness of the model. Specifically, the Bayesian information criterion (BIC) indicates the K-class model (where K denotes the number of classes under consideration) when the index reaches a minimum value, while the Lo-Mendell-Rubin (LMR) test indicates that K classes should be retained when the K + 1 P-value is no longer significant (29). Moreover, we used entropy as an indicator of good separation between classes. Indeed, entropy is an estimate of the probability that each individual is in each of the classes. The closer the entropy is to 1, the better the model's statistical fit (30).

In Step 2, multinomial regressions (i.e., regressions in which the output variable is represented by the latent classes) were run to characterize the groups through parents’ and infants’ descriptive characteristics. In addition to the variables retrieved from the questionnaire, the size of the NICUs involved in the study was considered according to the number of intensive care cots available (31). Using the suggestion of the neonatologists of the SIN, wards were divided into large (> 12 intensive care cots), medium-size (between 8 and 12), and small (< 8) NICUs.

Finally, Step 3 made it possible to verify whether an outcome varied across groups. In our study, we tested whether parents, who had different experiences of NICU, also differed in their level of perceived satisfaction. In doing so, we computed a Pearson chi-square test to detect significant differences across the latent classes obtained from Step 1.

Results

Sample Characteristics and Descriptive Statistics

In total, 482 surveys were collected. The descriptive characteristics of the sample are shown in Table 1.

Table 1.

Sample Descriptive Characteristics (n = 482).

Parents’ and infants’ characteristics	n	%
Parents who filled the survey while their child was still hospitalized	106	22%
Parents who had another child hospitalized in the NICU before	74	15%
Length of hospitalization
Less than 1 week	22	5%
1 week	14	3%
More than 1 week but less than 4 weeks	101	21%
More than 4 weeks but less than 8 weeks	130	27%
More than 8 weeks but less than 12 weeks	97	20%
More than 12 weeks	118	24%
Time from discharge
Less than 1 week	4	1%
1 week	3	1%
More than 1 week but less than 4 weeks	20	5%
More than 4 weeks but less than 8 weeks	23	6%
More than 8 weeks but less than 12 weeks	26	7%
More than 12 weeks	300	80%
Baby's gestational age at birth
Less than 25 weeks	28	6%
More than 25 but less than 29 weeks	193	40%
More than 29 but less than 32 weeks	122	25%
More than 32 but less than 38 weeks	116	24%
More than 38 weeks	23	5%
Baby's weight at birth
Less than 1000 g (1 kg / 2 lb 3 oz)	155	32%
1000 g to 1500 g (1 kg to 1.5 kg / 2 lb 3 oz to 3 lb 5 oz)	165	34%
1500 g to 2500 g (1.5 kg to 2.5 kg / 3 lb 5 oz to 5 lb 8 oz)	131	27%
More than 2500 g (2.5 kg / 5 lb 8 oz)	31	7%
Age group (mother):
16-27	53	11%
28–31	106	22%
32–35	205	42%
36 or more	118	25%
NICU size
Large	155	32%
Medium	193	40%
Small	134	28%

Abbreviation: NICU = neonatal intensive care unit.

Figure 1 reports the scores assigned to the experience items. The availability, empathy, and support provided by the NICUs staff were given high scores, and 92% of the parents reported a high or very high level of overall satisfaction. However, several issues emerged concerning many aspects of the experience of parents. Considerable obstacles to the parental presence in the unit, participation in their infant's daily care, and involvement in medical discussion and decision-making were observed. Furthermore, the evaluation of parent–staff communication produced controversial results at different stages of the care path.

Figure 1.

Experience items scores.

3-Step Latent Class Analysis (LCA)

The fit indexes used to select the number of subgroups in our model are presented in Appendix B. The BIC reaches a minimum when the number of latent classes is equal to four. In addition, the LMR test struggles in identifying the first two classes, but then clearly points to a four-class specification model, given that the P-value is not significant when the number of classes reaches five. Given that the level of entropy of the four-class model is sufficiently high (30), as it is equal to 0.816 up to 1, we selected the four-class model as the best fit for our data.

The profile of the classes is represented in Figure 2, in which the horizontal axis presents the 14 indicators used to create the subgroups, and the vertical axis reports the proportion of observations in each class that revealed a positive experience in each item domain.

Figure 2.

Latent classes’ profile (n = 482).

The largest class comprised parents who had a positive experience (36% of the observations) during their child's hospitalization in the NICU. Specifically, they perceived high involvement in communicational aspects, high accessibility of the NICU during hospitalization, and a helpful approach by the staff. Moreover, they perceived substantial attention by the staff to aspects related to the child's care.

The second-largest class comprised parents who experienced problematic parent–staff communication (30% of the observations). These parents had a particularly poor experience concerning aspects related to communication at different touchpoints and low involvement in the medical discussion. However, these parents reported a particularly high degree of accessibility to the NICU and perceived a good level of availability and support provided by the staff.

On the contrary, the third class, making up 26% of the total observations, perceived limited access and participation. This cluster of parents struggled in accessing the NICU and perceived that they could not visit their baby as much as they wanted to. Moreover, they also reported limited participation in their child's care, while, also for this group, the staff's attitude was evaluated positively.

The final class comprised parents who perceived a negative experience during their child's hospitalization, consisting of 8% of the observations. Overall, this cluster reported very poor experiences and the lowest perceptions on almost all items.

Having identified and described the latent classes, Step 2 allowed us to characterize the classes by using parents’ and infants’ descriptive characteristics, as reported in Table 2. The results are reported in comparison to a reference group, here represented by the positive experience class. The only significant personal information across classes was related to the mother's age: the younger the mother, the higher the probability she belonged to the limited access and participation class rather than to the positive experience class. The second relevant aspect was related to how much time passed from hospital discharge. The longer the amount of time that passed, the higher the probability that the parents would belong to either the limited access and participation class or the negative experience class (respectively, 1.19 and 1.18 times more probable) rather than to the positive experience class. Finally, the size of the NICU mattered in terms of the quality of communication. Indeed, parents in larger hospitals had a slightly greater probability of belonging to the problematic parent–staff communication class (1.09 times) or the negative experience class (1.05 times).

Table 2.

Steps 2 and 3 of the LCA.

	Problematic parents-staff communication			Limited access and participation			Negative experience			Positive experience (Reference group)
	Mean	Coef.	Odds ratio	Mean	Coef.	Odds ratio	Mean	Coef.	Odds ratio	Mean
Q3. Length of hospitalization	4.111	−0.023		4.610	0.283		3.875	−0.255		4.310
Q4. Time from discharge	3.618	−0.091		5.169	0.173**	1.19	4.850	0.168*	1.18	4.230
Q5. Other child hospitalized before	0.125	−0.062		0.202	0.269		0.125	−0.319		0.149
Q6. Gestational age at birth	2.931	0.071		2.685	0.092		3.125	0.227		2.753
Q7. Baby's weight	2.188	0.116		1.968	0.159		2.300	−0.017		2.017
Q8. Mom's age	2.806	−0.25		2.653	−0.393**	0.68	2.675	−0.356		2.937
NICU size (small, medium, large NICU)	14.124	0.084***	1.09	10.823	0.015		12.150	0.047*	1.05	12.124
LCA: Step 3
	Mean	Standard error		Mean	Standard error		Mean	Standard error		Mean	Standard error
Q29. Satisfaction level	4.301	0.063		4.413	0.053		2.865	0.184		4.868	0.041

Abbreviations: LCA = latent class analysis; NICU = neonatal intensive care unit.

***P-value<.01; **P-value<.05; *P-value<.1. In Step 3, the pairwise comparison of the average level of satisfaction is statistically significant across all classes (P-value<.01) except between Limited access and participation and problematic parents-staff communication (P-value = 0.18).

Finally, Step 3 allowed us to test whether the four classes differed in their overall level of perceived satisfaction with care in the NICU. The bottom panel of Table 2 depicts the results, highlighting that the positive experience class reported the highest level of satisfaction (4.9 on a 5-point scale), which was significantly higher than that of all of the other classes. In contrast, the negative experience class reported the poorest level of satisfaction (2.9), significantly lower than that of all of the other classes. Finally, the problematic parent–staff communication and the limited access and participation classes did not differ from one another. Both classes reported a level of satisfaction that was significantly higher than that reported by the negative experience class and significantly —but only slightly— lower than that reported by the positive experience class (respectively, 4.3 for the problematic parent–staff communication class and 4.4 for the limited access and participation class).

Discussion

We explored the potentiality of the LCA statistical model in analyzing experience data identifying patterns in parents’ evaluation of the care they and their infant received in the NICU. We implemented this novel technique as opposed to other traditional statistical methods as we aimed to explore a potential methodological innovation, one which could enhance the interpretation and use of experience data (7).

From our study, we inferred a set of novel considerations. First, LCA permits the development of experience profiles that identify different patterns of responses for the variables of interest, thereby explaining subgroup behavior. Typically, most studies on experience data have been based on aggregated analyses, falling short of indicating what different groups need and value (8). Therefore, relevant information needed to fuel quality improvement efforts is at risk of remaining latent or unexplored. LCA instead provides much more granularity with regard to the description of the sample, allowing for the design of tailored and meaningful interventions to address the specific concerns that emerge for each latent subgroup. In our study, a typology composed of four latent classes was identified from experience data, clustering parents into a positive experience class (36% of the observations), a class that experienced problematic communication with the staff (30%), a limited access and participation class (26%), and a negative experience class (8%). Each class presented a defined set of indicators related to different experience dimensions (see Figure 2) that should be targeted (or not) with specific interventions to enhance each class's experience of NICU.

Second, these tailored interventions can be further reinforced by the results of the multinomial regressions performed during Step 2 of the LCA methodology, which provides additional information, in terms of sociodemographic characteristics, about the latent classes identified. In our research, we determined that young mothers are more likely to experience limited access and participation in the care of their baby. This finding is consistent with prior studies on parents’ presence and participation in the NICU (32,33). Furthermore, poor parent–staff communication is more likely to occur in larger NICUs. We could not find other studies reporting a similar or contrasting association. However, organizational complexity, which could reasonably be a common trait of larger units, has been associated with worse parent–staff communication in the NICU (34). Finally, parents were more likely to report worse experiences the more time had passed from their baby's discharge. This result is consistent with the fact that when experience surveys are conducted further in time from the clinical encounter, poorer experiences are typically measured (35,36).

Third, the final step of the LCA methodology verifies whether an outcome varies across latent classes and allows the linking of the experience measure to output indicators. Linking different health care quality measures is key to prompting data triangulation and implementing holistic quality improvement plans, avoiding singular quality reports to remain isolated and unexploited (1,7). In our study, the parents’ overall level of satisfaction with care in the NICU was used as the outcome in the third step of the LCA. Parents reported an averagely high level of satisfaction (92%). However, poor experiences emerged on several occasions: only 36% of the parents belonged to the positive experience class. This result is in line with previous research where the discrepancy between the experience and satisfaction measures was reported (37–39). While experience considers objectively events and interactions, satisfaction is a more volatile and subjective measure influenced by individual preferences and expectations as well as by clinical outcomes (40,41). In this sense, satisfaction may provide a limited and optimistic picture (37), and depend more on factors external to the health system than on the experience of care (42). Therefore, positive satisfaction ratings may be reported despite negative experiences, especially when users believe that poor care is not under the direct control of the medical staff (43). Consistently with this assumption, in our study, satisfaction dropped only when the attitude and support provided by the NICU staff were negatively evaluated by parents (i.e., parents belonging to the negative experience class). Despite the difficulties in communicating with the staff or accessing the NICU and participating in the care of their baby (i.e., parents belonging to the problematic communication with the staff or to the limited access and participation classes), instead, a fairly high level of satisfaction was still reported. These findings provide relevant insights on which may be the most salient drivers of the parent satisfaction with NICU.

To the best of our knowledge, this was the first research to provide a nationwide picture of the parent experience of NICU in Italy. In this respect, some practitioner-oriented implications may be derived. Poor experiences were often reported for several key dimensions of the FCC model. This result is consistent with a recent study that analyzed the application of FCC in Italian NICUs from the perspective of health care professionals (44). Furthermore, our analysis confirmed the chronic issue of limited access, which, as shown in previous studies (45,46), interests several Italian NICUs. Our findings may stimulate reflection and further research on how the FCC standards are implemented in Italian NICUs.

As a further extension to our research, a more thorough depiction of the subgroups identified could be produced by including other sociodemographic variables in our LCA model, as well as other indicators of health care quality as outcomes in the final step of the methodology. This would allow the creation of a more comprehensive picture of performance. Furthermore, since the collection phase of our survey ended just a few weeks before the outbreak of the COVID-19 pandemic in Italy, our study could be replicated with a comparable sample to discover the impact of the pandemic and subsequent restrictions on the experiences of NICU parents. By replicating our study among a diverse sample of NICU parents instead, it would be possible to better ground the arguments about our findings. Finally, since our work provides fresh insights into the application of LCA in the study of experience data, this technique should be further tested also in different health care settings.

Limitations

Our study was subject to some limitations. First, we surveyed a national sample of NICU parents to provide an overall, nationwide picture. Thus, our findings may not be completely generalizable to the experiences of individual NICUs, each of which may present unique characteristics. Second, since several parents evaluated their experience some weeks after their baby's discharge, recall bias is a potential concern. Third, the anonymity requirements of our research meant that it was not possible to link the experience of care to other quality indicators, like patient safety and clinical effectiveness. This also meant that it was not possible to use these indicators as final outcomes in the LCA and to create complementary models alongside the one presented in this study.

Conclusion

LCA proved to be a useful tool, capable of effectively differentiating parent subgroups’ needs, concerns, sociodemographic characteristics, and outcomes. Studying experience data through the LCA methodology could aid researchers, practitioners, health service managers, and policymakers in better targeting different groups improving the performance of health systems by effectively responding to specific people's expectations.