Comparison of Novel, Bach Mai Boston Tool (BBT) and the Patient-Generated Subjective Global Assessment (PG-SGA) for Oncology Inpatients

Abstract

Oncology inpatients are at high risk of malnutrition. Identification of at risk patients by nutrition screening requires a practical and easy to use tool. The aim of this study was to determine the validity of the Bach Mai Boston Tool (BBT) compared to a ‘gold standard’ full nutrition assessment using the Patient-Generated Subjective Global Assessment (PG-SGA). A cross-sectional study was conducted on 270 oncology inpatients from January to December 2016. Cohen’s Kappa, sensitivity, specificity and ROC analyses were performed. 270 inpatients were included in this study with a mean age of 56.3 ± 12.1 years old. Of these patients, 51.8% were male, and 74.1% had gastrointestinal cancer. The mean body mass index of patients was 20.6 ± 3.0 kg/m². The PG-SGA tool identified 146 (54.1%) malnourished patients, while the BBT identified 105 (39.9%) malnourished patients. The BBT had a medium consistency, with a Kappa value of 0.6. Using a cut-off point of ≥ 4, the BBT had a sensitivity of 87.7% and a specificity of 72.6%. On the other hand, a BBT with a cut-off point ≥ 5 resulted in a sensitivity of 67.1%, a specificity of 94.4%, and an AUC of 0.81. The BBT is a practical, informative and valid tool for detecting malnutrition in hospitalized oncology patients. We recommend using a cut-off point of 4 for screening the risk of malnutrition for oncology inpatients.

Introduction

Cancer is a systemic disease that directly affects the region of onset and is also able to metastasize to other organs, causing a variety of complications including loss of progressive organ function. The pace of cancer development can be slow at the start, but it rapidly evolves and unavoidably affects the nutritional status of patients.¹

Malnutrition is a possible complication in patients with cancer and can be the first symptom signifying the presence of the disease.^2–5 The prevalence of malnutrition among patients with cancer has been estimated to range from 15% to 80%⁶ with main symptoms including weight loss and asthenia of varying degrees. Malnutrition is also common in Vietnamese patients with cancer. A study in 120 patients with cancer from Hanoi Medical University Hospital showed that the percentage of malnutrition was 20% and 29.1% according to the body mass index and albumin serum level, respectively.⁷ Malnutrition was also reported to be common in Vietnamese patients with lung, gastrointestinal, and esophageal cancer.^8–10 Malnutrition can pose adverse impacts on curative treatment of the cancer, which significantly limits the treatment options and success rate for patients. Furthermore, malnutrition increases the likelihood of postoperative complications, such as delayed wound healing, dehiscence of anastomosis, morbidity, and mortality. It is also reported to be associated with an increase in the length of a patient’s hospital stay,^11–13 which markedly escalates the cost of treatment.¹³ Considering these possible complications, malnutrition is a poor prognostic factor and should thus be prevented or detected as early as possible.¹⁴

The scored patient-generated subjective global assessment (PG-SGA) is the preeminent interdisciplinary patient assessment (weight, intake, symptoms, functional status, disease state, metabolic stress, and nutritional physical examination) in oncology research. The PG-SGA is an easy to use nutrition assessment tool that allows quick identification and prioritization of malnutrition in hospitalized patients with cancer.¹⁵ However, in Vietnam, with the overload of patients admitted to hospitals and the shortage of human resources in nutrition, a quick and simple questionnaire with high sensitivity and specificity is needed for the purpose of screening patients. The PG-SGA tool is time-consuming to complete, as it consists of many questions. In addition, this tool is mainly used in research, and there are not any hospitals in Vietnam using this questionnaire to regularly screen patients with cancer. Vietnam Bach Mai Hospital, in collaboration with Boston University in the United States has developed a new assessment tool, the Bach Mai Boston Tool (BBT), to shorten the time taken by health professionals for nutritional screening.

The aim of this study was to determine the validity of the BBT and compare it to a “gold standard” full nutrition assessment, the PG-SGA.

Methods

Study Participants

Inpatients were included if they were aged 18 years or older and received chemotherapy treatment at the department of oncology and palliative care in Hanoi Medical University Hospital between January and December 2016. The patients were diagnosed with cancer by an oncology specialist at Hanoi Medical University Hospital (with sufficient clinical and subclinical evidence available to diagnose the cancer). Patients with a history of or the presence of other diseases that might affect their nutritional status, such as gastrointestinal diseases, chronic liver diseases, kidney diseases, heart failure, total or partial paralysis before the diagnosis of cancer, and a systemic inflammatory response, such as sepsis symptoms, lung diseases, and trauma were excluded from our study.

Study Design

Our study is a cross-sectional study.

Ethical Approval

Our study was approved by the Scientific Council of Hanoi Medical University, Vietnam (approval no. 6075/QD-DHYHN).

Sample Size

Two hundred seventy patients were eligible for the study.

Data Collection

The questionnaire developed by the research team was used to collect patients’ information. Descriptive information included age, gender, and cancer diagnosis. University students were trained in and practiced question delivery to patients in addition to note taking. Face-to-face interviews were conducted to assess every participant based on both the PG-SGA and BBT questionnaires. Participants’ weight and height was also measured by students during the study period. The PG-SGA score consisted of 4 patient-generated historical components (weight history, food intake, symptoms and activities, and function) and the professional part (diagnosis, age, metabolic stress, and physical examination). All of the relevant sections of the PG-SGA were completed and summarized to classify the patients into 3 main categories: well-nourished (PG-SGA-A), moderately or suspected of being malnourished (PG-SGA-B), or severely malnourished (PG-SGA-C). Meanwhile, the BBT had 3 questions about oral intake, body mass index (BMI), and weight loss in the last 3 months. There are 3 levels of the BBT score: level A (no risk), level B (low/mild risk), or level C (high risk).

The PG-SGA-A (good nutrition) implied stable weight or recent gain of weight and there was neither a reduction in the diet nor any abnormality in activities in the past month. Patients were classified as PG-SGA-B (mild or moderate malnutrition or risk of malnutrition) if they lost 5% of their weight during a month or 10% of their weight in 6 months, had symptoms affecting nutrition and lost a moderate amount of subcutaneous fat or muscle mass. The PG-SGA-C (severe malnutrition) implied more than 5% of a patient’s total weight loss in a month or 10% during 6 months, serious lack of dietary intake, presence of symptoms affecting food consumption, severely impaired function, and clear signs of malnutrition (loss of subcutaneous fat, muscle atrophy, etc).

Data analysis and statistical method

For PG-SGA data, the patients who fell into classifications “B” and “C” were deemed to be at risk of malnutrition. For the BBT, scores of 5+ indicated the risk of malnutrition. STATA version 12.0 software was used to plot a receiver operating characteristic (ROC) graph to explore the best cutoff points of the BBT and calculate the area under the curve (AUC). Area under the curve values were interpreted as outstanding if >0.90, excellent if between 0.80 and <0.90, and acceptable if between 0.70 and <0.80.¹⁶ The validity of the new tool was assessed using a combination of following methods: sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

In addition, the Cohen κ was used as a measure of inter-raters reliability. Cohen suggested the κ result be interpreted as follows: values ≤0 as no agreement and 0.01 to 0.20 as none to slight, 0.21 to 0.40 as fair, 0.41 to 0.60 as moderate, 0.61 to 0.80 as substantial, and 0.81 to 1.00 as almost perfect agreement.¹⁷

Results

Table 1 illustrates the general characteristics of the research participants. A total of 270 patients were included in the study, and the response rate was 100%. A total of 48.2% of patients were male (n = 130). The most common type of cancer diagnosed in the study was gastrointestinal (74.1%), followed by breast/cervical/ovarian cancer (9.6%), and lung/liver cancer (7.4%). A total of 211 (78.1%) patients were undergoing chemotherapy, and 6.3% were on palliative care. A total of 22.2% participants had a BMI <18.5 kg/m², and the mean participant BMI was 20.6 ± 3.0 kg/m². The mean patient age was 56.3 ± 12.1 years.

Table 1.

Patient Characteristics.

Summary of Variables (n = 270)		Count, n (%)
Gender	Male	130 (48.2)
	Female	140 (51.8)
Diagnosis	Gastrointestinal	200 (74.1)
	Breast/cervical/ovarian	26 (9.6)
	Lung/liver	20 (7.4)
	Others	24 (8.9)
Body mass index (kg/m2)	<18.5	60 (22.2)
	18.5-22.9	149 (55.2)
	≥ 23.0	61 (22.6)
		Mean (SD)
Height (m)		1.594 (0.08)
Weight (kg)		52.4 (9.3)
Body mass index (kg/m2)		20.6 (3.0)
Age (years)		56.3 (12.1)

Table 2 showed that the κ was 0.6, implying medium consistency. P < .05 suggested the κ was statistically significant and that the consistency existed.

Table 2.

Categorization of Patients According to BBT in Comparison With PG-SGA and Calculation of Kappa.

BBT	PG-SGA (Gold Standard)			κ	P
	At Risk of Malnutrition (B/C)	Well Nourished (A)	Total
At risk of malnutrition (BBT B or C)	98 (67.1)	7 (5.7)	105 (38.9)	0.6	.000
Well nourished (BBT A)	48 (32.9)	117 (94.4)	165 (61.1)
Total	146 (100.0)	124 (100.0)	270 (100.0)

Abbreviations: BBT, Bach Mai Boston Tool; PG-SGA, patient-generated subjective global assessment.

The results in Table 3 illustrate that the sensitivity, specificity, PPV, and NPV of the BBT were 67.1%, 94.4%, 93.3%, and 70.9%, respectively, while the AUC was 0.81. Weight loss in the previous 3 months >5% was also predictive of PG-SGA score B or C, with sensitivity and specificity and AUCs of 70.5%, 84.7%, and 0.78, respectively. Other combinations of parameters were studied but AUCs were well below 0.75.

Table 3.

Sensitivity, Specificity, Predictive Values, and Accuracy in Predicting Malnutrition Assessed by the BBT¹ Rating (BBT B/C).

Screening BBT	Sensitivity (95% CI)	Specificity (95% CI)	Positive Predictive Value	Negative Predictive Value	Area Curve (95% CI)
At risk of malnutrition (B or C)	67.1 (58.9-74.7)	94.4 (88.7-97.7)	93.3 (86.7-97.3)	70.9 (63.3-77.7)	0.81 (0.76-0.85)
>5% weight loss in 3 months	70.5 (62.4-77.8)	84.7 (77.1-90.5)	84.4 (76.8-90.4)	70.9 (62.9-78.1)	0.78 (0.73-0.83)
Body mass index < 18.5 kg/m2	35.6 (27.9-44)	93.5 (87.7-97.2)	86.7 (75.4-94.1)	55.2 (48.2-62.1)	0.65 (0.6-0.7)
≤ 50% food intake vs normal	52.1 (43.6-60.4)	92.7 (86.7-96.6)	89.4 (80.8-95)	62.2 (54.8-69.2)	0.72 (0.67-0.77)

Abbreviation: BBT, Bach Mai Boston Tool.

Table 4 shows that the sensitivity and specificity of the cutoff point of 4 were 87.7% and 72.6%, respectively, while the figures for the cutoff point of 5 were 67.1% and 94.4%, respectively. The area under ROC curve for the BBT score was 0.88 (Figure 1).

Table 4.

Detailed Report of Sensitivity and Specificity of Cutoff Points.

Cutoff Point	Sensitivity (%)	Specificity (%)	Correctly Classified (%)	LR+	LR−
≥ 3	100	0	54.0	1.0
≥ 4	87.7	72.6	80.7	3.2	0.17
≥ 5	67.1	94.4	79.6	11.9	0.35
≥ 6	33.6	100	64.1		0.67
≥ 7	15.1	100	54.1		0.85
≥ 8	4.1	100	48.2		0.96
>8	0.0	100	45.9		1.00

Abbreviation: LR, Likelihood ratio.

We compared cutoff point 4 and cutoff point 5 to choose cutoff point with high sensitivity and speccificity.

Figure 1.

Area under the curve of Bach Mai Boston Tool score.

Discussion

Oncology patients are at risk of malnutrition, which results from both the disease itself and as a consequence of treatment (surgery, chemotherapy, radiation, etc). Malnutrition is common in patients with cancer and the degree of malnutrition is influenced by the severity, type, location, and stage of tumor as well as the treatment pathway.^18,19 There are several tools available for evaluating the nutritional status of patients with cancer. The Subjective Global Assessment (SGA) has generally been regarded as a gold standard in nutritional assessment, from which the patient-generated score was adapted for specific use in oncology.¹⁵ In 2002, Bauer showed that the PG-SGA score had a sensitivity of 98% and a specificity of 82% in predicting SGA classification.¹⁵ However, despite being developed by the Australian Research Institutes, the PG-SGA tool has not been widely used in this country because it is time-consuming for patients to complete. The PG-SGA Short Form has been recommended instead because, as its name suggests, it is much shorter than the original version. An ideal nutrition screening tool should be quick, easy to use, sensitive, and reliable.

The BBT was designed by Clinical Nutrition Center of Bach Mai Hospital and Boston Medical Center of Boston Massachusetts in the United States²⁰ to provide a tool that was simple, quick, valid, and reliable to identify patients at risk of malnutrition. According to the BBT, the prevalence of malnutrition in our study was 38.9%, compared to the global result of 54.1% by the PG-SGA. These findings were to be expected as hospitalized oncology patients were much more likely to have malnutrition.^21–23 The results were in accordance with a study by Huong et al with a risk of malnutrition and severe malnutrition (PG-SGA B or C) of 51.7%, in which the severe malnutrition rate was 8.2%.²⁴ It was higher than the results of a study by Thuy et al in patients with head and neck cancer, as the malnutrition rate was reported at 8.9% and 40% of patients needed nutritional intervention according to the PG-SGA.²⁵ Another study by Roop et al showed that 60% of patients were assessed at risk of malnutrition or severe malnutrition.²⁶ In 2010, another study in patients with stage IIIB/IV lung cancer concluded that 83.3% of patients were at risk of malnutrition or severe malnutrition (42.7% and 40.6% classified as PG-SGA-B and PG-SGA-C, respectively).²⁷ The differences in these studies highlighted the evidence that late-stage patients with cancer faced a higher risk of malnutrition than patients with cancer at an earlier stage. Furthermore, the BBT was shown to have acceptable inter-rater reliability (κ = 0.6, P < .001).

The ideal nutrition assessment tool needs to achieve 100% specificity and sensitivity. However, as there is always a trade-off between sensitivity and specificity, the need to correctly classify all patients who are malnourished (sensitivity) takes precedence over the classification of well-nourished patients (specificity). The BBT was able to identify malnourished patients with 67.1% sensitivity and 94.4% specificity at a cutoff point ≥5. Meanwhile, the results from a study in 2013 indicated that the sensitivity and specificity of the novel screening tool was 83.3% and 86.7%, respectively.²⁰

Concurrent validity of the BBT was measured against the PG-SGA using 3 available parameters of nutritional status (weight loss in the previous 3 months, BMI, and oral intake compared with normal). The study illustrated that weight loss of > 5% in the previous 3 months was also predictive of PG-SGA type B or C with a sensitivity and specificity of 84.7% and 70.5%, respectively. In this study, the BBT appeared to be accurate in discriminating between well-nourished and malnourished patients according to the PG-SGA with an AUC of 0.81. This was in accordance with a study by Lien et al at Bach Mai Hospital, which found the SGA had an AUC of 0.85 in correctly identifying malnutrition.²⁰

Regarding the determination of an optimal cutoff score for the BBT, it is important to consider the purpose of nutritional screening, which is to identify patients at risk of malnutrition and to implement interventions before the onset of or further progression of malnutrition. The BBT cutoff point of ≥4 had a high sensitivity of 87.7% and specificity of 72.6%. Increasing the cutoff point to ≥5 improved the specificity to 94.4% but markedly reduced the sensitivity to 67.1%. Thus, the cutoff point ≥5 had lower sensitivity but higher specificity than the cutoff point ≥4. However, the reduction of the sensitivity meant more malnourished patients went undetected. As there must be a trade-off between 2 indicators, and it is better to correctly classify malnourished people as positive rather than precisely identifying well-nourished people as “healthy,” the optimal cutoff point for the BBT would be ≥4.

This study has several limitations. It might have potential inaccurate results as recall bias could have emerged when the patients answered the questionnaires. Moreover, the small sample size was also likely to reduce the statistical power of our research.

Conclusion

In summary, the BBT has been validated against the PG-SGA for use among oncology patients and with the cutoff point ≥ 4, and it has good sensitivity and specificity in this setting. These results demonstrate that the BBT is a quick, valid and reliable nutrition assessment tool that enables malnourished patients with cancer to be identified and triaged for nutritional support.