Relative validity of a Diet Risk Score (DRS) for Chinese American adults

Abstract

Objective

The objective of this study was to evaluate the relative validity of the nine-item Diet Risk Score (DRS) among Chinese American adults using Healthy Eating Index (HEI)-2015 scores. We provide insights into the application of the Automated Self-Administered 24-Hour Dietary Assessment Tool (ASA24) for this population, and report on lessons learned from carrying out participant recruitment during the COVID-19 pandemic.

Methods

Thirty-three Chinese American adults (mean age=40; 36% male) were recruited from the community and through ResearchMatch. Participants completed the DRS and two 24-hour food records, which were entered into the ASA 24-Hour Dietary Assessment Tool (ASA24) by community health workers (CHWs). HEI-2015 scores were calculated from each food record and an average score was obtained for each participant. One-way analysis of variance and Spearman correlations were used to compare total and component scores between the DRS and HEI-2015.

Results

Mean HEI-2015 score was 56.7/100 (SD 10.6) and mean DRS score was 11.8/27 (SD 4.7), with higher scores reflecting better and worse diets, respectively. HEI-2015 and DRS scores were inversely correlated (r=−0.43, p<0.05). The strongest correlations were between HEI-2015 Total Vegetables and DRS Vegetables (r=−0.5, p<0.01), HEI-2015 Total Vegetables and Green Vegetables (r=−0.43, p=0.01) and HEI-2015 Seafood/Plant Protein and DRS Fish (r=−0.47, p<0.01). The inability to advertise and recruit for the study in person at community centres due to pandemic restrictions impeded the recruitment of less-acculturated individuals. A lack of cultural food items in the ASA24 database made it difficult to record dietary intake as reported by participants.

Conclusion

The DRS can be a valuable tool for physicians to identify and reach Chinese Americans at risk of cardiometabolic disease.

WHAT IS ALREADY KNOWN ON THIS TOPIC.

• Different racial/ethnic groups in the US display a wide variation in the prevalence rates of diet-associated cardiometabolic disease. There is a need for short assessment instruments that can be used in the clinical setting to capture the unique intake patterns of the Asian American population.

WHAT THIS STUDY ADDS

• This study establishes the relative validity of a nine-item Diet Risk Score (DRS) among Chinese American adults and provides insights for conducting a nutrition study in this population.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• The DRS is a brief, reliable measure that could be a useful tool for developing targeted efforts to reduce cardiometabolic risk in a historically under-represented population in public health research.

Introduction

Poor dietary habits are associated with many risk factors for cardiometabolic disease (CMD), such as heart disease, stroke and type 2 diabetes,¹ and data on dietary choices can provide important information to guide discussions of disease risk.² Brief dietary assessment tools can help identify individuals that may benefit from behavioural interventions to promote healthy lifestyle changes, which in turn may prevent diseases associated with poor diets. Such instruments may be useful in clinical and research settings where more detailed measures are not necessary or appropriate.

Cardiovascular, cerebrovascular and metabolic disease disproportionately affect certain population groups. Asian Americans are among the fastest growing segments of the US population and while rates of disease vary by subgroup, trends in disease rates have been stagnant or increasing over the past twenty years.³ Modifiable risk factors, such as diet, may help reduce disease prevalence and mortality rates, but nutrition counselling is not routinely included in prevention or management of these conditions. While several short dietary assessment instruments have been developed and validated for the general US population,⁴ few, if any, adequately capture the wide variation in dietary practices among various multiethnic populations. Different racial/ethnic groups in the US display wide variation in the prevalence rates of CMDs, which may be more closely associated with dietary practices than genetic differences.⁵ Developing high-quality tools that account for the diversity of the US population is critical to implementing nutrition interventions that do not inadvertently widen health disparities.

Only two brief dietary assessment instruments exist that specifically assess the dietary intake patterns of a multiethnic Asian population. The first and only validated instrument was developed in Singapore and consists of 37 questionnaire items.² The second is currently undergoing validation and is a 27-item culturally adapted version of the Dietary Screener Questionnaire used in the National Health and Nutrition Examination Survey (NHANES).⁶ While both instruments make it possible to assess dietary intake in a shorter amount of time than a total dietary assessment, they may still be too time-intensive for time-constrained settings, such as a doctor’s office visit. It is estimated that a 35-item questionnaire takes about 15 min to complete, which may exceed the time allotted for an entire clinical visit in some settings.⁷

In response to the need for brief and actionable dietary assessment tools that could be used in clinical practice, Johnston and colleagues developed the nine-item Diet Risk Score (DRS) to quickly and reliably assess suboptimal intake of foods or food groups.⁸ The DRS was validated in US adults recruited through ResearchMatch against a food frequency questionnaire scored using the Healthy Eating Index (HEI)-2015, a comprehensive measure of diet quality based on the 2015–2020 Dietary Guidelines for Americans. The DRS took individuals less than 3 min to complete on average in the validation study. The aim of this study is to evaluate the relative validity of the DRS among Chinese Americans using HEI-2015 scores calculated from food record data. We hypothesised that the two measures would be inversely correlated. We used the Automated Self-Administered 24-Hour (ASA24) Dietary Assessment Tool to obtain and automatically code food records. Here, we provide insights into its application as a self-administered tool for this population. We also report on lessons learned from carrying out participant recruitment procedures while under COVID-19 pandemic restrictions.

Materials and methods

Translation of the DRS questionnaire into simplified Chinese

The development of the DRS has been described elsewhere.⁸ Briefly, the DRS is a nine-item questionnaire that estimates dietary risk for CMD based on a comparative risk assessment model that was developed using NHANES data and data from meta-analyses of previously published cohort studies to estimate diet–disease relationships ¹ (questionnaire available in online supplemental appendix 1). The DRS assesses the dietary components that contribute most to cardiometabolic risk: excess intake of sodium, processed meats and sugar-sweetened beverages, and inadequate intake of fruits, vegetables, nuts and marine omega-3 fatty acids. The DRS assesses intake with the following question: ‘For the following foods, please select the frequency that best describes how often you eat each food or group of foods in a normal week’. Participants have the option to choose ‘daily’, ‘2–3 times per week’, ‘1 time per week’ and ‘never’. Each answer is assigned a score of 0 (lowest) to 3 (highest), for a maximum risk score of 27. The final score is divided into tertiles of risk: 0–8 for low risk; 9–18 for moderate risk; 19–27 for high risk.

The translation of the DRS into simplified Chinese involved several rounds of review. The initial translation was completed by a bilingual CHW at the NYU Centre for the Study of Asian American Health (CSAAH). The translated document was reviewed by another bilingual CHW and sent back to the first CHW with comments. The first CHW reviewed and integrated the comments into the original translated document and sent the updated document to two bilingual study team members for further review. On receiving the two reviewers’ comments, the first CHW finalised the document for use in the study.

Recruitment

The eligibility criteria were: adults 18 years of age or older; self-identifying as Chinese American or Chinese immigrant; being able to speak and read in English or Mandarin (Simplified Chinese); and having access to a computer or tablet with internet and email.

From January to August 2021, participants were recruited from the community through the NYU CSAAH network and ResearchMatch, an online database for research volunteers (researchmatch.org). Potential participants were sent emails with study information and a weblink to the study’s screening questionnaire inquiring about basic demographics such as age on REDCap, a secure survey platform.^{9 10} Eligible and interested participants provided electronic informed consent and were subsequently emailed a link to the DRS questionnaire on REDCap. Participants received a US$30 electronic Amazon gift card to acknowledge their contributions to the project.

ASA24 Dietary Assessment

Dietary intake data were obtained using the ASA24 Dietary Assessment Tool, a web-based tool developed by the National Cancer Institute that enables automatically coded 24-hour dietary recalls and food diaries.⁴ While the ASA24 is intended to be a self-administered tool, the study team’s prior experience with the web-based platform indicated that non-English speakers and those with low health or technology literacy may find it difficult to navigate.^{11 12} Rather than having participants complete food records on the online ASA24 platform directly, participants were provided paper-based material (ie, a food diary template document with instructions and a food measurements guide document) to complete written food diaries.

Participants completed 2 days of food diaries, one on a weekday and one a weekend day. No questions were asked regarding whether the data represented usual intake. Completed food diaries were sent to the study team via email. A research assistant inputted data from participants’ written food diaries into the ASA24 platform and took note of any additional information that was required by the system. Any missing information was then collected during phone interviews with participants. As per study design, all participants completed phone interviews.

Tests of validity

In order to determine whether the DRS could accurately identify individuals with poor diet quality, the DRS was compared with participants’ HEI-2015 scores calculated using data from participants’ ASA24 food records. The HEI-2015 is a measure of diet quality used to evaluate the extent to which Americans are following key recommendations from the 2015–2020 Dietary Guidelines for Americans.¹³ Evaluations of the HEI-2015 have demonstrated construct validity, reliability and criterion validity.¹⁴

Statistical Analysis

ASA24 data were reviewed and cleaned per recommended procedures.⁴ Average HEI-2015 scores were derived for each participant. Descriptive statistics (means and SDs) were computed for demographic and HEI-2015 components. One-way analysis of variance was used to determine statistical difference between DRS and HEI-2015 total scores. Spearman correlations were used to measure level of agreement between DRS and HEI-2015 component scores. Sensitivity analyses were conducted to compare the DRS and HEI-2015 scores using a single day of the ASA24. A power calculation indicated that a sample of 30 individuals would provide 80% power at an alpha level of 0.05 to detect a minimum acceptable correlation coefficient of r=0.50 between the two tests.¹⁵ Data analysis was conducted using SAS V.9.4 (SAS Institute).¹⁶

Results

Individuals that completed both the DRS and ASA24 study components were included in the analyses. Twenty-two participants (67%) were recruited from the community by emailing study flyers to contacts of study team members and using snowball sampling. ResearchMatch recruitment emails were sent to 500 individuals that had ‘Asian American’ as their race/ethnicity. Of those, 24 individuals signed informed consent forms, 13 individuals partially completed the surveys and food diaries, and 11 participants (33%) completed all assessments and were included in the study. Energy and nutrient ranges fell within range for all participants, so no participants that completed both study components were excluded from the analysis.¹⁷

Mean age of participants was 40 (range 21–62 years), and 36% were male. The average DRS score of respondents was 11.8 (SD 4.7) out of a maximum score of 27 (lower score represents lower risk), and the average HEI-2015 score was 56.7 (SD 10.6) out of 100 (higher score represents higher diet quality) (table 1). For moderation components, the average HEI component score for sodium was 2.2 out of 10, saturated fat 6.1 out of 10 and added sugars 8.8 out of 10.

Table 1.

Descriptive characteristics (n=33)

Characteristic	Mean (SD) or frequency (%)
Age	40 (12)
Male	12 (36%)
Language
English only	16 (48)
English and Mandarin	13 (39)
Mandarin only	4 (12)
Diet Risk Score (range: 0–27)	11.8 (4.7)
HEI-2015 Total Score (100)*	56.7 (10.6)
Total vegetables (5)	4.2 (1.0)
Greens and beans (5)	2.9 (1.7)
Total fruits (5)	2.6 (1.7)
Whole fruits (5)	3.2 (2.0)
Whole grains (10)	2.3 (2.8)
Total dairy (10)	4.1 (2.6)
Total protein (5)	4.8 (0.5)
Seafood and plant protein (5)	3.0 (1.8)
Fatty acids (10)	6.0 (2.8)
Sodium (10)	2.2 (2.7)
Refined grain (10)	6.6 (2.4)
Saturated fat (10)	6.1 (3.1)
Added sugar (10)	8.8 (1.9)

*Maximum score indicated in parentheses.

HEI, Healthy Eating Index.

Each of the DRS components had variation in responses, with the exception that no participant reported consuming vegetables less than 2–3 times per week (table 2). Mean HEI-2015 score did not differ significantly by tertile of DRS (table 3).

Table 2.

Diet Risk Score component responses

		Daily	2–3 times per week	1 time per week	Never
Fast food	Sit-down or take-out meals, frozen dinners or other fast food type meals, including pizza*	2	9	14	8
Breads	Bread, rolls, sandwiches*	8	12	9	4
Snacks	Chips, popcorn, pretzels, snack mixes, crackers*	7	8	14	4
Processed meats	Sausage, cured or deli meat, hot dogs†	1	4	21	7
Sugar-sweetened beverages	Regular soda, sweetened iced tea, juice, flavoured milk or flavoured coffee drinks‡	3	5	12	13
Nuts	Peanuts, tree nuts, seeds, peanut butter or other nut butter§	5	12	12	4
Fish	Fish or shellfish¶	2	13	13	5
Vegetables	Vegetables (not including potatoes, peas, corn or beans)**	26	7	0	0
Fruit	Fruit††	17	12	3	1

*Serving information and rationale for score of 3 (high risk): sodium >2300 mg per day.

†Serving information and rationale for score of 3 (high risk): Processed meat >2 ounces per day.

‡Serving information and rationale for score of 3 (high risk): sugar-sweetened beverages >8 ounces per day.

§Serving information and rationale for score of 3 (high risk): low nuts/seeds <1 ounce per week.

¶Serving information and rationale for score of 3 (high risk): seafood <100 mg omega-3 fats per day.

**Serving Information and rationale for score of 3 (high risk): low vegetables <100 g or <1 serving per day.

††Serving information and rationale for score of 3 (high risk): low fruit <100 g or <1 serving per day.

Table 3.

Mean HEI-2015 score by Diet Risk Score (DRS) category

DRS score	Mean HEI-2015 score (SD)*	n
1–8 (low risk)	61.1 (11.8)	8
9–17 (moderate risk)	55.7 (9.8)	22
18–27 (high risk)	52.1 (12.7)	3

*Data presented as means (SD) from one-way ANOVA, p=0.35.

ANOVA, analysis of variance; HEI, Healthy Eating Index.

The DRS Vegetables component correlated moderately with HEI-2015 Total Vegetables and Greens and Beans components (r=−0.50, –0.43 respectively, both p≤0.01) (table 4). The DRS Fish component correlated moderately with the HEI-2015 Seafood/Plant Protein component (r=−0.47, p=0.006). Analyses were repeated using a single day of ASA-24, and results were similar (data not shown).

Table 4.

Alignment between DRS and HEI-2015 component scores

DRS component	HEI-2015 component	Correlation*	P value
Fast food	Sodium	0.24	0.17
Breads		0.17	0.34
Snacks		0.34	0.05
Processed meats		−0.08	0.66
	Saturated fat	−0.16	0.36
Sugar-sweetened beverages	Added sugars	−0.07	0.69
Nuts	Seafood/plant protein	−0.32	0.07
Fish		−0.47	0.006
Vegetables	Total vegetables	−0.5	0.003
	Green vegetables, beans	−0.43	0.01
Fruit	Total fruits	−0.16	0.38
	Whole fruits	−0.05	0.79
Total		−0.43	0.01

*Spearman correlations, p<0.05 defines statistical significance; values in bold are statistically signficant.

DRS, Diet Risk Score; HEI, Healthy Eating Index.

Discussion

Total DRS scores were moderately, inversely correlated with total HEI-2015 scores derived from the ASA24 (r=−0.43, p<0.05) for a sample of Chinese American adults recruited from the community and ResearchMatch. DRS Vegetables correlated moderately with the HEI-2015 Total Vegetables (r=−0.5, p<0.01) and Green Vegetables (r=−0.43, p=0.01) components. DRS Fish correlated moderately with the HEI-2015 Seafood/Plant Protein component (r=−0.47, p<0.01). DRS Nuts also correlated moderately with the HEI-2015 Seafood/Plant Protein component but the results were not significant.

There were no correlations between DRS Fruit and the HEI-2015 Total Fruit (includes juice) or Whole Fruit (excludes juice) components (r=−0.16, p=0.38 and r=−0.05, p=0.79, respectively). In the original validation study by Johnston et al,⁸ these were among the strongest correlations between the DRS and HEI-2015, with r=−0.67 for DRS Total Fruit and r=−0.68 for DRS Whole Fruit at p<0.001. Poor agreement may have been due to discrepancies in the responses provided by participants on the two self-report measures and differences in the way the ASA24 and DRS capture fruit intake. The DRS assesses fruit intake in a normal week: ‘fruit (fresh, canned or dried; not including juice)’ quantified only by ‘daily’, ‘2–3 times per week’, ‘1 time per week’ and ‘never’. The ASA24 collects data on all foods and drinks eaten by participants over a 24-hour period and tabulates intake of foods that are coded as fruits in the database.¹⁸

Consistent with findings from the original validation study, correlations with sodium intake were non-significant. Sodium intake is difficult to accurately capture without the use of a 24-hour urinary assessment.¹⁹ The DRS does not attempt to quantify sodium intake, and instead provides a score based on intake of four categories (fast food, breads, snacks and processed meats) related to intake of the major sources of sodium in the American diet, processed and restaurant food; breads, rolls and sandwiches; salty snacks.²⁰ The lowest HEI component score was for sodium, just 2.2 out of 10 possible points, indicating that sodium intake was contributing to the lower HEI scores. Future iterations of the DRS may benefit from consideration of the variance in sources of sodium intake by race/ethnicity. For example, salt added at home (in cooking and at the table) and soy sauce are the largest dietary sources of sodium in East Asian populations.²¹ A 2017 study by Firestone et al reported that the top 10 sources of sodium for Asian Americans as an aggregate were: (1) soups, (2) rice, (3) yeast breads, (4) stir-fry and soy-based sauces, (5) fish, (6) chicken (whole pieces), (7) fried rice and lo/chow mein, (8) soy-based condiments, (9) pizza and (10) dips, gravies and other sauces.²²

Nevertheless, the DRS can be used to initiate a discussion of the impact of dietary choices on CMD. The inclusion of discussion of adverse influence of high sodium intake on blood pressure may be particularly salient for Asian Americans, who have the highest intake of sodium among racial/ethnic groups. Analysis of data from NHANES 2011–2012 by Bailey et al showed that only 8% of non-Hispanic Asian Americans consumed the recommended amount of sodium (≤2300 mg/day), compared with 13%, 16% and 12% of non-Hispanic whites, non-Hispanic blacks and Hispanics, respectively.²³ In NHANES 2015–2016, non-Hispanic Asians had the lowest mean score on the HEI-2015 sodium component (0.6 out of 10) compared with Hispanics (4.0), non-Hispanic whites (3.9) and non-Hispanic blacks (4.1).²⁴ Excessive sodium intake is associated with hypertension and mortality due to CMD.¹ Such data strongly support the need to create and refine dietary screeners for the Asian American population, including those that can inform and engage healthcare providers in a clinical setting.

Less than 25% of patients receive any dietary assessment or nutrition counselling from a physician,²⁵ but physicians can play a key role in helping patients improve diet quality,²⁶ particularly in the Chinese American community. In a US study assessing trust in physicians among 3159 community-dwelling Chinese older adults, participants displayed high levels of trust in physicians’ knowledge and skills.²⁷ In traditional Chinese culture, the benevolent intent of physicians is emphasised, and physicians are socially respected, considered to possess special knowledge and expertise, and deemed highly capable.^{28 29} Given that current dietary public health messaging falls short of reaching Asian Americans at risk of CMD, physicians can occupy a highly impactful role in mitigating disease risk among Chinese Americans through lifestyle change. Short and actionable screening tools, such as the DRS, can help physicians start the conversation by addressing previously cited knowledge and time barriers to providing nutritional counselling.^{30 31}

An analysis of diet quality based on NHANES data showed that Asian Americans perceived their diet quality more accurately than other ethnic groups when asked to report how healthy their dietary intake is.³² Those with the highest self-rated diet quality had higher HEI Total Fruits, Whole Fruits, Added Sugars and Saturated Fats component scores than those with lower self-rated diet quality. Additionally, higher intake of fruit has been associated with lower BMI and waist circumference in Asian Americans, but these associations are not significant among total HEI score, just component scores.³³

Limitations

There are several limitations to this study. We used a comparison between two dietary self-report methods. Such methods are prone to inaccuracies due to imperfect memory and social desirability bias, which leads to under-reporting of energy intake.^{34 35} We did not collect data on subject characteristics apart from age and sex, and we report them here only to describe the sample; this precludes subgroup analyses.

Another limitation is the small, mostly female, sample (n=33) and lack of generalisability due to eligibility criteria requiring access to internet and email. Due to challenges brought on by the COVID-19 pandemic, recruitment was slow and original plans to do in-person community-based recruitment were not feasible. While the study is sufficiently powered with the current sample size, we had difficulty recruiting a majority of participants to be less acculturated individuals from the community as we had initially planned. Despite offering the study in Mandarin Chinese, using CHWs with strong ties to the community, and recruiting through the NYU CSAAH, a National Institutes of Health National Institute on Minority Health and Health Disparities‒funded centre with extensive experience in conducting research in the Asian American community, we were limited by our inability to use in-person recruitment tactics through community organisations. Therefore, the participants included in the study were more likely to speak English and had a higher level of digital literacy than our intended sample. Subgroup analyses and identification of meaningful cut points for categorisation were also precluded due to the small sample size.

Beans and legumes were not included in the DRS vegetable question (see online supplemental appendix 1), but were included in the underlying data used to create the DRS. We suspect that the consequence of this oversight would be listing someone’s risk as higher than it truly is by not counting certain foods that are protective. The extent to which this impacted the current study is unclear, but likely small.

Finally, the lack of culturally appropriate food items on the ASA24 made it difficult to report dietary intake as provided by participants. Because we anticipated recruiting participants from the community who might experience language, technology and health literacy barriers, food records were initially collected on paper by participants and entered into the system by trained CHWs. In a postrecruitment interview, one CHW stated that she often had to resort to recipe-building by entering in individual ingredients one-by-one, a process that was time-consuming, frequently required substitutions and missed the ‘flavour’ of traditional Chinese dishes (Chinese cooking incorporates many sauces that were missing on the ASA24). She felt that having prior knowledge of the limitations of the platform through practice runs was necessary to ensure that the most accurate record was inputted for participants eating a traditional Chinese diet. This would not be possible if participants were completing their food records directly on the ASA24. We searched for all of the items that the CHWs highlighted as absent from ASA24 and 78% of them were also not available in another validated resource, Nutrition Data System for Research 2022 (Minneapolis, Minnesota). In the absence of more culturally relevant platforms, future studies using the ASA24 (or other dietary recall systems) for a Chinese American population should consider using a similar approach of having a trained individual enter data. It is recommended to include a CHW or other individual with knowledge and experience pertinent to the specific population in the planning of future studies, which may help to reduce some of these barriers.

Conclusion

The DRS is a brief, reliable measure that could be a useful tool in clinical practice. This tool is an important first step to developing targeted efforts to reduce cardiometabolic risk in a population that has historically been under-represented in public health research. Studies inclusive of older populations and more recent immigrants would aid in the development of a more valid measure. The original DRS has been tested in clinical practice (manuscript under review); planned studies include translation of the DRS into Spanish and validation in Spanish-speaking populations. In low-resource settings, medical nutrition therapy and other lifestyle interventions may be difficult to access. Creation of DRS cut points for clinical action, such as referral to a registered dietitian nutritionist or to a lifestyle change programme such as a Diabetes Prevention Programme would also contribute to the utility of this tool to identify at risk individuals for meaningful use of sparse resources to contribute to CMD risk reduction.

Data availability statement

Data are available on reasonable request.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

The study was approved by the New York University Grossman School of Medicine Institutional Review Board (i20-01433). Participants gave informed consent to participate in the study before taking part.