External validation of Villalta score in high-middle income country patients with deep vein thrombosis

Abstract

Post-thrombotic syndrome (PTS) is a late complication that does not have a cure yet, with a prevalence estimated between 20 to 75%, associated with previous deep vein thrombosis event. Although the Villalta score (VS) is the gold-standard clinical tool for diagnostic and prognostic evaluation of PTS, there are currently no VS intra-rater agreement established and no validation studies for VS’ application into Brazilian Portuguese. We sought to translate and validate VS reliability systematically; and, secondarily, to compare the ultrasound findings with the severity of PTS.

We systematically translated the original VS into Brazilian Portuguese (BP). Fifty participants who underwent two outpatient visits were evaluated using the translated VS. We assessed its intra-rater and inter-rater agreement and compared BP VS versus CEAP clinical component (CEAP C), and the clinical PTS severity versus the duplex ultrasound (DUS) findings. The study and its report followed the Guidelines for Reporting Reliability and Agreement Studies.

The intra-rater evaluation of VS grades had a simple Kappa coefficient of 0.73, and the simple Kappa coefficient inter-rater for VS grades was 0.67. When VS was compared to CEAP C, it established a remarkably high correlation over 0.9. There was difference among VS values compared to DUS initial deep vein thrombosis territory, with femoropopliteal showing higher values than distal veins. Higher VS values were correlated to DUS venous recanalization and reflux.

There was a substantial inter-rater and intra-rater agreement when the BP VS was applied; and when compared to CEAP C, VS showed a high correlation. When VS grading was compared to DUS characteristics, there were significant statistical and clinical correlation, with presence of reflux and recanalization showing higher VS values. This external VS validation also changes the clinical practice allowing the VS use in a different population and establishes the VS intra-rater agreement.

1. Introduction

Post-thrombotic syndrome (PTS) is a late complication frequently associated with patients who previously developed deep vein thrombosis (DVT), with a prevalence estimated between 20 to 75% of those patients with previous DVT.^[1–4] PTS has severe implications for people quality of life (QoL), interfering directly in their social life and work capacity.^[3,5]

The primary pathophysiological mechanism is venous hypertension, which is secondary to venous wall remodelling, valvular lesion, plus flow obstruction.^[3,6] The main predictor to develop PTS after an episode of DVT is if a proximal leg vein (common femoral or iliac) is compromised, with a relative risk of two to three times higher than in distal veins.^[7]

The diagnostic of this condition involves the analysis of the patient at least six months after the development of DVT, and there are several clinical scales made throughout the years to help in this task. The PTS severity and prognosis are also evaluated by these scales and scores such as Villalta score (VS), Ginsberg, Brandjes, Widmer, Clinical-Etiological-Anatomical-Pathophysiological (CEAP) classification, and Venous Clinical Severity Score.^[1,8–10]

Prandoni et al developed the VS in 1992, and it was published in an abstract by Villalta in 1994;^[11] it is used to evaluate the severity and prognosis of PTS, based on the following criteria: five symptoms (pain, cramps, heaviness, paresthesia, pruritus); six signs (pretibial oedema, skin induration, hyperpigmentation, pain during calf compression, venous ectasia, and redness); and venous ulcer.

VS assigns points for each criterion, varying from none (0), mild (1), moderate (2), and severe (3), which are easily applicable by trained professionals. The presence of more or equal than five (5) points is the cutline to diagnose PTS and graduates its severity between mild (5 to 9 points), moderate (10 to 14 points), or severe (14 to 33 points or venous ulcer).^[3,8,11] Even after almost 20 years from VS’ first publication, a number of studies worldwide are still discussing its validity and correlations with other tools and through different populations.^[12–14]

Previous studies set the VS, associated with a QoL specific for venous disease, as the gold-standard for diagnostic and prognostic evaluation of PTS.^[8] However, there are currently no VS intra-rater agreement established, and no external validation studies for VS’ application into Brazilian Portuguese.

This study aimed to systematically translate VS from the English language, as published in the original article in 1994, into Brazilian Portuguese (BP), and to validate its reliability and agreement in a population with at least six months of a previous DVT episode, confirmed by an objective method and showing symptoms compatible with PTS. Moreover, the secondary objective was to compare the ultrasound findings (reflux, recanalisation and initial DVT vein territory) with the severity of PTS in the same population.

2. Methods

The Local Research Ethics Commission prospectively approved this external validation cohort under the number 2.250.698. The study was conducted in accordance with the Brazilian Ethical Review System for research involving human beings and also conformed to the World Medical Association's Declaration of Helsinki (June 1964) and subsequent amendments. All participants or legal representatives provided written informed consent after the procedures had been fully explained to them, and prior to their inclusion in the study; anonymity was assured. The study was conducted and reported according to the Guidelines for Reporting Reliability and Agreement Studies.^[15]

2.1. Translation

Two independent translators, both fluent and certified in English and BP, translated the VS from English into BP. Idiomatic, semantic, conceptual and cultural equivalences were considered during the translation phase. The leading researcher compared both versions, merged, and decided the first BP version of VS. This first BP version was translated back into English to evaluate if there were any significant difference or language lost in translation, and none were found. The back-translation English version was compared with the original one by the leading researcher, in order to correct possible errors or discrepancies made during back-translation.

Considering the equivalence between both English and BP versions, the researchers discussed the applicability of the score, reaching consensus and the final BP version, after minor corrections. The consensus version of the VS in BP was appropriately adapted to the linguistic and cultural context of the target population, while maintaining all the essential characteristics of the original instrument in English. This final BP version of VS was used for the purpose of this study. The English version used was adapted from a previous CC BY 4.0 licensed publication (Table 1).^[16] The final BP version of VS (Table 2) is also available online as a clinical practice application.^[17,18]

Table 1.

English Villalta score.

Symptoms	None	Mild	Moderate	Severe
Pain	0	1	2	3
Cramps	0	1	2	3
Heaviness	0	1	2	3
Paresthesia	0	1	2	3
Pruritus	0	1	2	3

Signs	None	Mild	Moderate	Severe
Pretibial oedema	0	1	2	3
Skin induration	0	1	2	3
Hyperpigmentation	0	1	2	3
Pain during calf compression	0	1	2	3
Venous ectasia	0	1	2	3
Redness	0	1	2	3
Venous ulcer	Absent	-	-	Present

0 to 4: No disease; 5 to 9: Mild disease; 10 to 14: Moderate disease; 15 or more, or venous ulcer present: Severe disease; Adapted from Greeff W, Dehghan-Dehnavi AR, Marle J van. Venous function after pharmacomechanical thrombolysis for extensive iliofemoral deep vein thrombosis. South Afr J Radiol. 2017;21: 5. doi:10.4102/sajr.v21i1.1214.

Table 2.

Brazilian Portuguese version Villalta score.

Sintomas	Ausência	Leve	Moderado	Grave
Dor	0	1	2	3
Cãibras	0	1	2	3
Sensação de peso	0	1	2	3
Parestesia	0	1	2	3
Prurido	0	1	2	3

Sinais	Ausência	Leve	Moderado	Grave
Edema pré-tibial	0	1	2	3
Endurecimento de pele	0	1	2	3
Hiperpigmentação	0	1	2	3
Dor à compressão da panturrilha	0	1	2	3
Ectasia venosa	0	1	2	3
Eritema	0	1	2	3
Úlcera venosa	Ausência	-	-	Presença

0 a 4: Sem doença; 5 a 9: Doença leve; 10 a 14: Doença moderada; 15 ou mais, ou presença de úlcera venosa: Doença grave.

2.2. Population

People of both sexes aged 18 years or older, able to understand the interview questions and with full acceptance of the study, with DVT since at least six months ago and with a confirmed diagnosis by an objective method (e.g. duplex ultrasound (DUS), or angiography by computed tomography, magnetic resonance or digital subtraction), and who used anticoagulation at least for three months after diagnostic of DVT were recruited at the vascular surgery outpatient clinic of a public university hospital in Brazil between August 2017 and August of 2019. A vascular surgeon with expertise in DVT management, who is also the leading researcher, performed the clinical assessment and management on all patients, in accordance with the best available evidence.^{[2,3,19–23]}

People younger than 18 years, who were unable to understand the interview questions and the study purpose, those who did not accept study participation, those who have not adequately used anticoagulant therapy, pregnant, who was in the use of vena cava filter, with active neoplasm, or those who already had chronic venous insufficiency before the DVT episode were not included in the study.

For this study, based on the most applied rules for sampling size in scale-validation studies of a ratio of the number of subjects (N) to the number of items evaluated (p) between three and ten, we established a sample size (N) of at least 36 patients (worst limb stricken in the case of bilateral disease), based on the criteria evaluated on VS (11 items plus venous ulcer), the prevalence of PTS, and how hard was to apply this scale.^[24–26]

2.3. Outpatient visits and measurements

Two independent, experienced, and certified doctors with training in vascular surgery were designated to apply the VS in all participants. Both researchers already had previous knowledge of the original VS in English and have employed it in clinical practice. In the first outpatient visit, both researchers applied the VS to evaluate the agreement between observers. A researcher explained the purpose of the study and obtained the signed informed consent form. Following inclusion, the first clinical evaluation was performed in an outpatient setting, with natural light, without any contact or information exchange between participants. Each participant was submitted to two independent initial clinical evaluations by the two assigned independent researchers (inter-observer analysis). All written, collected information was de-identified and stored in separate sealed and opaque envelopes, and the staffs were blinded for outcomes assessment until the end of the data collection.

After two to three weeks, only one researcher applied the VS tool to evaluate the intra-observer reliability at a second outpatient visit. None of the independent researchers had prior access to the patient's medical records on both visits. The interval time between test and retest was established after observing other validation studies and following the GRRAS guideline.^[15,27,28] PTS is a late complication of DVT, occurring at least six months after DVT and more frequently after two years.^[1–4] Therefore, six months after the initial DVT episode (inclusion criterion), an additional interval of 21 days did not significantly change the disease and was essential to avoid additional bias by participant and staff memorisation of previous results. After all data collection, the opaque envelopes were opened, and the data were transcribed to digital spreadsheets for statistical analysis as to agreement and reliability. The DUS related to include participants were documented and included in the analysis to evaluate the relationship between severity and ultrasonographic findings.

Demographic, clinical and DUS data were collected from each participant at the first visit. We used the classical clinical component of the CEAP classification, depicted in Table 3, as the standard of clinical evaluation and compared it with the PTS severity by VS.^[29] The researchers considered the vessel that was not compressible on B-mode ultrasound as venous occlusion and the cut-off of 1.0 second or more for iliac, femoral and popliteal vein reflux and 0.5 seconds for calf-deep veins reflux diagnosis.^[30] Veins with previous occlusion and that were partially compressed on DUS were considered as partially recanalised. The researchers used the vein terminology described by Eklof et al (2009) for definitions not specified in this report.^[31] The participants were graded into no PTS (<5 points), mild PTS (5 to 9 points), moderate PTS (10 to 14 points), or severe PTS (14 to 33 points or venous ulcer) according to the total mean VS.^[3,11,14,16]

Table 3.

Clinical component of the CEAP classification.

Grade	Symptoms and Signs
C0	Absence of visible or palpable signs of venous disease
C1	Telangiectasias or reticular veins
C2	Varicose veins
C3	Edema
C4	Skin changes secondary to chronic venous diseaseA: Pigmentation or eczemaB: LipodermatosclerosisC: Corona phlebectatica
C5	Healed Ulceration
C6	Active Ulceration

Adapted from Lurie F, Passman M, Meisner M, Dalsing M, Masuda E, Welch H, et al. The 2020 update of the CEAP classification system and reporting standards. J Vasc Surg Venous Lymphat Disord. 2020 May;8(3):342–52.

2.4. Statistical analysis

The researchers conducted the descriptive statistical analysis, with base demographic findings of the population and the DUS findings included. For the evaluation of agreement and reliability of VS, the simple Kappa Coefficient was used, which analyses perfect agreement between data and was applied to the measures between different researchers (inter-observer) and the same researcher (intra-observer) with the following cut-offs: 0 to 0.20: no agreement; 0.21 to 0.40: minimal agreement; 0.41 to 0.60: moderate agreement; 0.61 to 0.80: substantial agreement; 0.81 to 1: perfect agreement.^[32]

The researchers performed an ANOVA test to establish a possible relationship between the DUS findings and the severity of PTS, for numerical variables, considering an abnormal distribution by the Shapiro–Wilk test. If there was statistical significance between data, a Tukey test was performed. For categorical variables, the chi-squared test of independence was employed.

2.5. Patient and public involvement

Participants were not involved in the design, conduction, reporting or dissemination plans of our research.

3. Results

3.1. Participants

Seventy-two participants were recruited, based on the established criteria. After excluding patients who had not adequately used anticoagulant therapy, were pregnant, who were using a vena cava filter, had an active neoplasm, or those who already had primary chronic venous disease before the DVT, 54 patients were eligible for inclusion, however, four refused consent to participate. See Figure 1 for a flowchart of participants. The main demographic, clinical, and DUS characteristics of the 50 included participants are presented in Table 4.

Figure 1.

Participants flowchart. DVT: deep vein thrombosis.

Table 4.

Demographic, clinical, and duplex ultrasound characteristics.

		Male (n)	(%)	Female (n)	(%)	Total	(%)	P value∗
	Sex	20	40	30	60	50	100	–
	Mean age (yr)	56.8	–	51.7	–	53.7	–	–
	Mean DVT diagnosis time (yr)	11.7	–	12.8	–	12.3	–	–
Limb	Right	4	20	8	27	12	24	.58
	Left	12	60	15	50	27	54	.48
	Bilateral	4	20	7	23	11	22	.78
CEAP (Clinical)	0	0	0	0	0	0	0	.92
	1	0	0	5	17	5	10
	2	4	20	5	17	9	18
	3	8	40	9	30	17	34
	4	5	25	6	20	11	22
	5	1	5	3	10	4	8
	6	2	10	2	7	4	8
	DUS Reflux	14	70	15	50	29	58	.16
	DUS Recanalisation	19	95	26	87	45	90	.33
DVT Territory (DUS)	Iliac	1	5	8	27	9	18	.053
	Femoral	12	60	16	53	28	56	.77
	Popliteal	12	60	13	43	25	50	.37
	Distal	6	30	8	27	14	28	.86

CEAP = clinical-etiological-anatomical-pathophysiological classification, DVT = deep vein thrombosis, DUS = duplex ultrasound, n: number of participants.

^∗Chi-squared test comparing male and female participants.

The participants were invited to their first outpatient visit immediately after inclusion. From 50 included participants, 30 (60%) were women, the mean age was 53.7 years, and the mean previous DVT diagnosis was made 12.3 years before the visit date. Only 11 participants had bilateral lower limb DVT, and the most affected limb was the left (54%, n = 27), followed by the right (24%, n = 12), and bilateral (22%, n = 11), with no statistical difference between right and left limbs DVT involvement (p = 0.97). Overall, the most participants (72%) presented with chronic venous insufficiency, i.e., clinical CEAP 3 to 6.^[31] Regarding DUS characteristics, 58% of participants had venous reflux (n = 29), 90% had some degree (partial or total) of venous recanalization (n = 45), and the most affected territory by initial DVT was, by order, femoropopliteal (66%, n = 33), iliofemoral (16%, n = 8), and distal veins (18%, n = 9). Some participants had more than one affected territory in the same ultrasound examination.

3.2. Villalta Score evaluation

Each included participant underwent two outpatient visits. In the first one (V1), they were evaluated by two independent observers (O1 and O2), and in the second one (V2), just by one of them (O1). See Table 5 for details of the VS. Comparing the two different examiners evaluating all 50 participants, the mean total VS was 9.2 at the first visit with the first observer (O1V1), and 9.68 at the first visit with the second observer (O2V1). Regarding signs component of VS, the mean score was 4.2 at O1V1, and 4.48 at O2V1. Regarding symtoms component of VS, the mean score was 5.09 at O1V1, and 5.2 at O2V1. According to total mean VS, the most participants had mild PTS (O1V1 46%, second visit with the first observer (O1V2) 48%, O2V1 48%), followed by moderate PTS (O1V1 20%, O1V2 18%, O2V1 22%), severe PTS (O1V1 16%, O1V2 22%, O2V1 20%), and no disease (O1V1 18%, O1V2 12%, O2V1 10%). There was no statistical difference among the VS grading groups (P = .91), and the prevalence of PTS in this sample varied from 82% to 90% (O1V1 = 82%, O1V2 = 88%, and O2V1 = 90%).

Table 5.

Villalta score by visit and gradings.

	O1V1 (mean)	SD	O1V2 (mean)	SD	O2V1 (mean)	SD
Total VS	9.2	5.51	9.46	5.07	9.68	4.93
VS Signs	4.12	2.39	4.5	2.35	4.48	2.24
VS Symptoms	5.09	3.37	4.96	3.06	5.2	3.09

	N	(%)	n	(%)	n	(%)
VS Grading - no PTS (<5 points)∗	9	18	6	12	5	10
VS Grading - Mild PTS (5–9 points)∗	23	46	24	48	24	48
VS Grading - Moderate PTS (10–14 points)∗	10	20	9	18	11	22
VS Grading - Severe PTS (14–33 points or venous ulcer)∗	8	16	11	22	10	20

n: number of participants, O1V1 = first visit with the first observer, O1V2 = second visit with the first observer, O2V1 = first visit with the second observer, PTS = post-thrombotic syndrome, VS = Villalta score.

^∗P = .91 (chi-squared test).

The comparison of the different evaluations from the same observer provided the intra-rater evaluation of VS and the second visit was made in approximately two to three weeks from the first one (O1V1 versus O1V2 comparison). The simple Kappa coefficient, that is, the absolute agreement between values, used to establish the VS agreement between O1V1 and O1V2 (no disease, mild, moderate and severe) was 0.73, showing substantial agreement, as depicted in Figure 2.

Figure 2.

Intra-observer VS comparison. O1: observer 1, VS: Villalta score. ^∗ simple Kappa coefficient = 0.73.

The inter-rater comparison (O1V1 versus O2V1) was made comparing the values obtained in the same visit with two independent researchers. The simple Kappa coefficient between O1V1 and O2V1 VS grades was 0.67, also showing substantial agreement. See Figure 3 for details.

Figure 3.

Inter-observer VS comparison. O1: observer 1, O2: observer 2, VS: Villalta score. ^∗ simple Kappa coefficient = 0.67.

VS evaluation was also compared with a clinical component of CEAP classification, a well-established clinical scale, to external validate VS. There was a high Pearson correlation of 0.886 for O1V1, 0.890 for O1V2, and 0.886 for O2V1.

3.3. Ultrasonographic findings

The DUS characteristics such as initial DVT territory, deep venous reflux presence and venous recanalisation (partial or total) were compared to VS grading (no disease, mild, moderate and severe), to establish a possible relationship among ultrasonographic findings and VS assessment. The ANOVA test was used to determine statistical significance among VS and DUS reflux, recanalisation, and initial DVT territory.

There was significant difference between territory of initial DVT and VS category (P = .02), and when the Tukey test was applied, higher VS categories were correlated to femoropopliteal territory when compared to distal veins (P = .03) and there was no statistical difference when iliofemoral territory was compared to both groups.

There was difference among VS absolute score compared to DUS vein reflux (P = .01), with higher VS values associated with presence of any degree of deep venous reflux, and higher VS values were correlated to DUS venous recanalisation (P = .002; Figs. 4 and 5).

Figure 4.

Initial DVT territory by DUS versus VS comparison. DUS: duplex ultrasound, DVT: deep vein thrombosis, O1V1: first visit assessment by observer 1, O1V2: second visit assessment by observer 1, O2V1: first visit assessment by observer 2, VS: Villalta score. ^∗ Chi-squared test correlation between VS and DUS: P = .514 for vein territory.

Figure 5.

DUS reflux and recanalization versus VS comparison. DUS: duplex ultrasound, O1V1: first visit assessment by researcher 1, O1V2: second visit assessment by researcher 1, O2V1: first visit assessment by researcher 2, VS: Villalta score. ∗ Chi-squared test correlation between VS and DUS: P = .468 for vein reflux, P = .002 for vein recanalization.

4. Discussion

VS is based on typical signs and symptoms associated with chronic venous disease related to a previous DVT episode and has proven to be reliable and of easy to reproduce. Since Villalta et al proposed the systematic score to diagnose and follow-up patients with PTS in 1994 it was consistently validated and used worldwide, and after several reviews, it was established as the gold standard for the diagnosis of PTS.^{[11,12,14,16]} Kahn et al described the reliability, through two previous studies, including a prospective multicentre cohort with a high number of participants (n = 646) with a high weighted Kappa coefficient, and showed a good to an excellent inter-rater agreement. However, the intra-rater agreement was not previously evaluated.^[7,14]

To external validate VS for Brazilian patients with DVT, the researchers translated the original VS into Brazilian Portuguese, assessed its intra-rater and inter-rater agreement, and compared VS to another commonly used scale, the CEAP clinical component. The importance of the CEAP classification for the clinical routine was reinforced after a recent updating, but without any great modification based on the clinical component used in this study.^[10,29,33] The results of this study showed significant agreement, both for intra-rater and inter-rater, confirming the BP VS version reliability; and a high agreement with a high Pearson correlation coefficient (from 0.886 to 0.890) when comparing VS with clinical evaluation of CEAP. This high correlation shown in the present study was probably due to the high prevalence of moderate and severe cases, with chronic venous insufficiency.

Although VS is considered the gold standard for PTS diagnosis, a disease with no established cure yet, there is no other study that external validated VS into Brazilian Portuguese. The high intra-rater VS agreement is also another cornerstone added with this study. The high agreement scores can be favoured by the higher prevalence of PTS in this sample (82%–90%) compared to general population (20%–75%).^[2,3] This high PTS prevalence was attributed to the selection of participants in a university reference centre of an urban metropolis. However, the high agreement VS scores were also found in the sample without PTS (VS < 5).

The main ultrasonographic characteristics, as venous reflux, obstruction, recanalisation, and affected territory, were previously resumed through a simplified scale, named the venous segmental disease score, which evaluates 11 pre-defined segments through DUS and established scores depending on the DUS alteration.^[34] Previous studies have not found any relationship between venous segmental disease score and VS, only showing the disease's anatomical cause, but without establishing severity.^[12] This study tried to establish a possible relationship between limb DUS characteristics and PTS severity by VS. There was statistical difference related to initial DVT territory, with femoropopliteal territory showing higher VS values than distal veins and no statistical difference between iliofemoral and the other groups – this was probably due to the low prevalence of iliac disease in the present study. There was significant statistical difference when we compared deep vein reflux and VS, with higher VS values when there was presence of reflux. Participants with recanalised veins were more related to severe VS, and it can be related to vein reflux, a typical component for PTS severity.^[3]

VS showed a significant correlation with commonly used QoL questionnaires, directly related to venous diseases, such as the disease-specific quality of life instrument for use in venous diseases of the leg. However, additional study's limitation is that herein, we have not used any instrument to evaluate the patient's perception of the burden caused by PTS on everyday life.^[35]

Strengths of this study overcome its limitations. This is a pioneer study, in which we external validated a worldwide relevant tool – VS – into Brazilian Portuguese, a high-middle income country, and pioneering established its reliability intra-rater. This study supports the current scientific literature of using VS for diagnosis and follow-up of PTS, because of its high inter-rater and intra-rater agreement and correlation of other used clinical scales for chronic venous disease. This external validation also changes the clinical practice allowing the VS use in a different population.

Future research should aim to confirm this validation in larger sample sizes of prospective cohorts from the general population, and not only in reference centres, to establish possible relationships between ultrasound characteristics and PTS severity. Another issue to be solved is to evaluate external validity and agreement of chronic venous disease related QoL questionnaires with VS in different cultural populations with DVT.

5. Conclusions

Through this study, we were able to systematically translate VS from the English language into Brazilian Portuguese; validate its agreement, with a high inter-rater and intra-rater agreement; and validate its reliability when compared to another scale, the clinical component of the CEAP, showing significant correlation.

Considering the secondary objective, there was significant statistical and clinical correlation when VS grading was compared to DUS characteristics, with the presence of reflux and recanalization showing higher VS values, and femoropopliteal initial DVT showing higher VS values than distal initial DVT. To our knowledge, this study is a pioneer in external validation of the VS in BP and establishing the VS intra-rater agreement.

Author contributions

RBA and RLGF designed the analysis. SVMD performed the statistical analysis. RBA and RLGF drafted the paper. GBM, SVMD, BPS, AR, JEA, HJGN, LCUN, and RLGF were involved in critical review of the report. All authors helped to revise the paper and approved the final version.

Conceptualization: Rafael Ávila, Ronald Flumignan.

Data curation: Giulianna Marcondes, Luis Nakano, Rafael Ávila, Ronald Flumignan.

Formal analysis: Beatriz Silveira, Jorge Amorim, Rafael Ávilainve, Silfayner Dias.

Funding acquisition: Henrique Guedes Neto, Ronald Flumignan.

Investigation: Beatriz Silveira, Giulianna Marcondes, Rafael Ávila, Silfayner Dias.

Methodology: Ronald Flumignan.

Project administration: Luis Nakano, Ronald Flumignan.

Resources: Giulianna Marcondes, Henrique Guedes Neto, Luis Nakano, Rafael Ávila.

Software: Beatriz Silveira, Silfayner Dias.

Supervision: Henrique Guedes Neto, Jorge Amorim.

Writing – original draft: Rafael Ávila.

Writing – review & editing: Beatriz Silveira, Giulianna Marcondes, Henrique Guedes Neto, Jorge Amorim, Luis Nakano, Ronald Flumignan, Silfayner Dias.