Validity and reliability of the Malaysian Healthy Diet Online Survey (MHDOS) for assessing diet quality in Malaysian adults

Abstract

Background and Objectives

Malaysian Healthy Diet Online Survey (MHDOS) is an online survey designed to measure diet quality of Malaysian adults. This study aimed to assess the relative validity and test-retest reliability of the MHDOS.

Methods and Study Design

This nationwide cross-sectional study was conducted from May to November 2022 among 218 Malaysian adults. Participants completed the MHDOS, underwent an interview-administered 24-hour diet recall (24DR), and repeated the MHDOS within two weeks. Relative validity was assessed by correlating food group servings from the MHDOS and 24DR using Spearman's correlation coefficients. Construct validity was evaluated by comparing Diet Score tertiles with food group servings, energy, and nutrient intakes from the 24DR. Linear trend analysis was used to compare food group and nutrient intakes across the Diet Score tertiles. Reliability was measured using the Intra-class correlation coefficient (ICC) between the initial and repeated MHDOS administrations.

Results

The MHDOS demonstrated moderate-to-good reliability, with ICC ranging from 0.70-0.86 for different components and 0.90 for the total Diet Score. Spearman correlation coefficients for mean food group intakes estimated from the MHDOS and 24DR ranged from 0.21-0.44 (p <0.001). Higher Diet Scores were associated with greater intake of total fibre, vitamin C, thiamine, niacin, potassium, calcium, phosphorus, and iron, as well as increased consumption of vegetables, fruits, and water (p-trend <0.01).

Conclusions

MHDOS has good test-retest reliability and its derived Diet Score is associated with better nutrient and food group intake as estimated from 24DR. The MHDOS is a valid and reliable tool for assessing diet quality among Malaysian adults.

Introduction

National dietary guidelines offer evidence-based recommendations to promote overall population health and prevent diet-related non-communicable diseases.¹ Policymakers and health professionals utilise these guidelines to design and implement nutrition promotion programs, aiming to enhance the nutritional status of the population. Regular and timely large-scale national surveys are therefore crucial for monitoring program implementation and evaluating population adherence to dietary guidelines.² Traditional dietary assessment methods, such as 24-hour dietary recalls, food records, and food frequency questionnaires, are often labour- and resource-intensive.³ Although these methods provide a deep understanding of nutrient intakes and valuable insights into population dietary habits, the time and resource constraints associated with them can make it unfeasible to administer them regularly to provide up-to-date intake data.⁴ The limitations of traditional methods have prompted a quest to explore alternative, potentially more efficient and scalable approaches to understanding population dietary patterns and diet quality.³ Incorporating technology into traditional methods is one approach researchers have explored to alleviate the burden of measuring dietary intake.⁵

Diet quality is a multidimensional concept that encompasses diet adequacy, variety, moderation, and overall balance.⁶ Assessing diet quality is crucial for identifying dietary pattern linked to health outcomes and guiding public health interventions. Diet quality is often measured by diet indices which consider various aspects of diet, often integrating a range of food groups and nutrients, and condense dietary patterns into an overall score based on predefined criteria.⁷ When aligned with established dietary guidelines, diet indices facilitate a direct comparison of individuals’ overall dietary intake relative to these guidelines, allowing for evaluations of adherence to an optimal recommended diet.^{7, 8}

Several dietary quality assessment tools have been developed globally to evaluate adherence to dietary recommendations. Notable examples include the Healthy Eating Index (HEI),⁹ the Diet Quality Index (DQI),¹⁰ and the Alternative Healthy Eating Index (AHEI),¹¹ which assess diet quality based on various components derived from national dietary guidelines. While these tools have been widely used in research and public health settings, they often require detailed dietary intake data obtained through 24-hour dietary recalls and food frequency questionnaires, which can be burdensome for large-scale implementation. As technology advances, web-based and mobile dietary assessment tools have emerged as promising alternatives that offer improved accessibility, ease of administration, and real-time data collection.¹²

The Commonwealth Scientific and Industrial Research Organisation (CSIRO) Healthy Diet Score Survey is an online dietary assessment tool developed to address the need for a valid, reliable, and user-friendly method for estimating overall diet quality among Australian adults.^{13, 14} The survey provides an overall diet quality score which assess individuals’ compliance with the Australian Dietary Guidelines. Recognising the adaptable nature of the CSIRO Healthy Diet Score Survey in collecting dietary data, and the similarities in the structure of food-based dietary guidelines between Australia and Malaysia,^{15, 16} an online survey namely Malaysian Healthy Diet Online Survey (MHDOS) was developed by adapting the CSIRO Healthy Diet Score Survey.^{9, 10}

The MHDOS is a practical dietary assessment tool designed for the Malaysian population, offering a scalable, efficient, and user-friendly approach to evaluating diet quality. By leveraging the advantages of an online platform, it enables the collection of dietary data in alignment with the latest national dietary guidelines while minimising participant burden and allowing for real-time data capture. This study aimed to determine the validity and reliability of the newly developed MHDOS for use among Malaysian adults.

Methods

Study participants

In this cross-sectional study, participants were recruited from 13 states and 3 federal territories in Malaysia using convenience sampling. Inclusion criteria comprised Malaysian adults aged between 18-59 years and currently residing in the country. Exclusion criteria included pregnant and lactating women, as well as individuals unable to respond to the survey in Malay, Mandarin, Tamil, or English. The protocol of the study and sample size calculation were previously described.¹⁷ Initially, 316 Malaysian adults were recruited to the study (see Supplementary Materials), and 218 were included in the final analysis after excluding outliers and individuals with misreported energy intake (Figure 1). Willett and Lenhart¹⁸ suggested a sample size of 100-200 participants for validation studies involving dietary questionnaires. In accordance with these recommendations, the final sample size in the present study was deemed more than adequate for assessing validity and reliability. The study received approval from the Medical Research and Ethics Committee (MREC), Ministry of Health Malaysia (Ref: NMRR ID-22-00158-75G). Data were collected between May and November 2022. Implied consent was obtained from the participants prior to data collection. Reporting is according to the Strengthening the Reporting of Observational studies in Epidemiology - Nutritional Epidemiology (STROBE-nut) guidelines.¹⁹

Figure 1.

Flow chart of study participants

Development of the Malaysian Healthy Diet Online Survey (MHDOS)

The MHDOS was developed by adapting the CSIRO Healthy Diet Score survey^{13, 14} to local diets, with the scoring system modified to align with the recommendations of the Malaysian Dietary Guidelines 2020.15 Content validation and adaptation were conducted by a panel of nutrition experts who reviewed the adapted survey and provided feedback on the appropriateness of serving sizes and examples of food items relevant to the local context. Standard serving sizes were aligned with national recommendations¹⁵ while the names of foods were modified to be culturally relevant, using local terminology where necessary. Food items not commonly consumed in Malaysia were either replaced or removed to improve relevance and accuracy in reporting.

The adapted MHDOS included 38 questions about participants’ usual intake in terms of frequency and quantity across five core food groups, namely (i) vegetables; (ii) fruits; (iii) rice, other cereals, whole grain cereal-based products and tubers; (iv) fish, poultry, eggs, meat and legumes; and (v) milk and milk products. Each question featured coloured images of standard servings to aid in the estimation of intake portions. The survey also consisted of questions assessing habitual intake of beverages and discretionary foods (foods that are higher in saturated fats, added salt, added sugar, and calories), food choices, and variety within each core food group. Individuals reported the frequency and servings of foods consumed, which allowed for the estimation of daily consumption in servings per day.

The survey was translated into Malay, Mandarin, and Tamil using a forward-backward translation process to ensure linguistic and conceptual accuracy. A pretest (n = 60) was conducted to assess clarity and usability across language versions, with refinements made to food nomenclature, frequency categories, and serving size descriptions based on participant feedback. The finalised survey was then converted into a consumer-grade version using the online platform Alchemer, hosted on a secure cloud server.²⁰

To summarise the diet quality, daily consumption of core and discretionary foods was compared to the recommended servings in the Malaysian Dietary Guidelines 2020, considering sex and physical activity levels. The scoring criteria for the MHDOS are described in Table 1. A nine-component Diet Score, calculated as sub-scores out of 10 (except for discretionary foods, scored out of 20) was derived from the MHDOS. Summing all components provided a total Diet Score out of 100, where a higher score indicated greater adherence to the national dietary guidelines. For ease of comparison in the presentation of results, the overall Diet Score and component scores were presented as a score out of 100.

Table 1.

Components and scoring of the Diet Score derived from the MHDOS

Component	Description	Criteria for minimum score	Criteria for maximum score	Score allocated
Vegetables†	Comparison of total daily servings of vegetables with sex and physical activity levels specific recommendations.	0 servings	≥3 servings	10
Fruits‡	Comparison of total daily servings of fruits with sex and physical activity levels specific recommendations.	0 servings	≥2 servings	10
Rice, other cereals, whole grain cereal-based products & tubers§	Comparison of total daily servings of rice, other cereals, whole grain cereal-based products, and tubers with sex and physical activity levels specific recommendations plus the frequency of wholegrains consumption.	0 servings and rarely / never consumed wholegrains	≥4-6 servings and always consumed wholegrains	10
Fish, poultry, eggs, meat & legumes¶	Comparison of total daily servings of fish, poultry, eggs, meat, and legumes with sex and physical activity levels specific recommendations.	0 servings	≥1 serving of fish, ≥1 serving of legumes, ≥1-2 servings of poultry/meat/ eggs	10
Milk & milk products††	Comparison of total daily servings of milk and milk products with sex- and physical activity-specific recommendations plus reduced-fat dairy consumption.	0 servings and consumed whole or full cream milk	≥2 servings and consumed skimmed milk	10
Fluids‡‡	Comparison of the total glasses of plain water consumed per day with the total fluid intake per day.	No water consumed	100% water	10
Healthy fats§§	Types of mainly consumed fats and oils.	Coconut oil, butter, ghee, margarine	Blended oil, mono- / poly-unsaturated oils	5
	Trimming of fat from meat before cooking.	Never trimmed meat	Always trimmed meat	5
Discretionary / snack foods¶¶	Comparison of total daily servings of discretionary / snack foods with recommendations based on energy intake from fat and sugar.	>2 servings	≤2 servings	20
Variety	Total variety of foods consumed from the 5 main food groups	<0.5 servings over 2 days	2 points for each of the 5 core food groups	10
			Total Diet Score	100

^†Include raw salad vegetables and cooked vegetables.

^‡Include fresh fruits, dried fruits and canned fruits.

^§Include cooked rice, noodles, pasta or other cooked cereals/grains, bread, breakfast cereal, and tubers.

^¶Include red meat, poultry, eggs, fish/seafoods, legumes or meat alternatives.

^††Include milk, cheese, and yoghurt.

^‡‡Include water, fruit juice, sugar-sweetened beverages, and premixed drinks.

^§§The component score for healthy fats refers to the sum of scores for types of fats and oils mainly consumed, as well as the trimming of fat from meat before cooking.

^¶¶Include fast foods, fried potato products, savoury snacks, sweet baked items, pastries, snack-type bars, chocolate and sweets, ice confections, processed meat products, sugar-sweetened beverages, and premixed drinks. Servings size calculations adopted from Australian Dietary Guidelines¹² (1 serving of discretionary food = 600kJ = 143kcal).

Validation of the MHDOS

Validation of the MHDOS included 316 participants who completed the MHDOS, followed by a multiple-pass, single 24-hour diet recall interview (as a reference method). In the 24-hour diet recall (24DR), detailed descriptions of the foods and beverages consumed, including brand names and preparation methods were obtained from the participants. The amounts of foods and beverages consumed were estimated using household measurements. Dietary data obtained from the 24DR was analysed using Nutritionist ProTM software version 5.3.0 (Axxya System, Washington, USA) for conversion into energy and nutrient intakes.

Reliability of the MHDOS

The reliability of the MHDOS was assessed in 315 participants (one participant did not complete the survey on two occasions). The same MHDOS, referred to as MHDOS1 for the first administration and MHDOS2 for the second administration, was administered one-week apart.

Usability testing of the MHDOS

Usability testing of the MHDOS involved 60 participants, distinct from those who participated in the validation and reliability study. The survey's usability in each language version was evaluated using the System Usability Scale.²¹ Participants used a five-point Likert scale to rate their agreement with 10 statements assessing the perceived ease-of-use of MHDOS. A total score ranging from 0 (very poor) to 100 (excellent perceived usability) was generated, with scores above 68 considered as above average.²²

Statistical analysis

Data were analysed using IBM SPSS (version 29.0; SPSS Inc., Chicago, IL, USA). Prior to analysis, extreme outliers for the number of food servings consumed (on the MHDOS and 24-hour recall) and energy intake (on 24-hour recall) were identified and excluded. Extreme outliers for servings consumed were defined as any values beyond the upper outer fence calculated as Q3 + (3 * IQR), where Q3 represents the 75th percentile and the IQR is the interquartile range. Under- and over-reporting of energy intake were identified using the revised Goldberg cut-off.²³ Subjects were categorised as under-reporters if their energy intake (EI) to basal metabolic rate (BMR) ratio was below 0.87, plausible reporters if the EI:BMR ratio ranged from 0.87 to 2.75, and over-reporters if the EI:BMR ratio exceeded 2.75. After removing all outliers in the two dietary assessment methods, a final sample of 218 subjects was included in the analysis.

To determine the relative validity, correlations between the servings of food groups consumed derived from the MHDOS1 and those derived from the 24DR were examined using Spearman's correlation coefficients. Wilcoxon signed-rank test was used to determine the mean differences in the reported food group intakes. Additionally, the tertiles of the Diet Score derived from MHDOS1 was compared with food group servings, energy, and nutrient intakes obtained from the 24-hour diet recall assessing the construct validity of the MHDOS. Non-parametric linear trend analysis was used to compare food group and nutrient intakes across the tertiles of the Diet Score. Bland-Altman plots were used to analyse the mean bias and 95% limits of agreement between the two methods in deriving the Diet Score. The reliability of the MHDOS was assessed using the Intra-class correlation coefficient (ICC) between the initial and subsequent administrations (MHDOS1 and MHDOS2) to determine the agreement between repeated surveys over a one-week interval period. The statistical significance was set at p <0.05.

Figure 2.

Food group intake servings estimated from MHDOS1 and 24DR

Results

Among 60 participants who tested the usability of the MHDOS, 40% of participants rated its ease of use as above average, with a mean score of 64.5 ± 16.0 out of 100 (Supplementary 1). On average, it took participants 14 min to complete the MHDOS, with 68.3% responding that the survey length was appropriate.

Table 2 shows the characteristics of the 218 participants included in the analysis. Among them, 41.7% fell within the 20-29 years age group, with the majority being females (70.2%), Malays (56.9%), and half having attained a tertiary education (50.0%). Additionally, a significant proportion were employed in professional occupations (43.6%), reported moderate physical activity levels (44.0%), and half of the sample had a normal body weight (50.5%). A significantly higher total Diet Score was found among individuals aged 50-59 years (55.0 ± 19.5 years) and those with tertiary education (p <0.05). However, no significant associations were observed for other socio-demographic variables.

Table 2.

Characteristics of the participants and total Diet Score by demographic characteristics

Characteristics	Count (n)	Percentage (%)	Total Diet Score
			Mean±SD	p-value
Age group (years)
20-29	91	41.7	45.7±12.2	0.028*
30-39	87	39.9	48.5±11.5
40-49	33	15.1	52.3±14.5
50-59	7	3.2	55.0±19.5
Sex
Male	65	29.8	46.9±12.8	0.375
Female	153	70.2	48.6±12.7
Ethnicity
Malay	124	56.9	47.7±12.8	0.127
Chinese	56	25.7	51.3±14.4
Indian	11	5.0	46.2±6.1
Bumiputera of Sabah	22	10.1	44.6±8.9
Bumiputera of Sarawak	5	2.3	41.2±9.1
Highest educational level
Secondary	109	50.0	46.3±12.6	0.040*
Tertiary	109	50.0	49.8±12.7
Occupation group
Managers	9	4.1	45.4±10.4	0.618
Professionals	95	43.6	49.4±13.7
Technicians & associate professionals	16	7.3	41.7±8.3
Clerical support workers	27	12.4	47.7±13.7
Student	33	15.1	48.5±12.8
Elementary occupations	4	1.8	47.9±1.7
Service & sales workers	16	7.3	45.1±12.4
Homemaker	3	1.4	52.6±9.0
Unemployed	11	5.0	49.5±14.0
Others	4	1.8	52.3±3.1
Physical activity level
Low active	95	43.6	47.4±11.7	0.797
Moderately active	96	44.0	48.7±12.8
Active	27	12.4	48.4±16.1
BMI status
Underweight (<18.5 kg/m2)	9	4.1	40.9±8.3	0.269
Normal weight (18.5-24.9 kg/m2)	110	50.5	49.0±14.1
Overweight (25.0-29.9 kg/m2)	64	29.4	47.1±11.7
Obesity (≥30 kg/m2)	35	16.1	48.7±10.6

BMI, body mass index; SD, standard deviation.

^*p<0.05.

Table 3 presents the mean total Diet Score and component scores for the two administrations of MHDOS. The overall mean total Diet Score was 48.1 ± 12.7 for MHDOS1 and 48.6 ± 13.4 for MHDOS2, with component scores ranging from 19.2 for milk and milk products to 89.4 for fluids on the MHDOS1 and 19.7 for milk and milk products to 90.4 for fluids on the MHDOS2. The MHDOS demonstrated moderate to good test-retest reliability, with intraclass correlation coefficients ranging from 0.70 to 0.86 for different components and 0.90 for the total Diet Score. The rank order of the mean component scores was the same for MHDOS1 and MHDOS2. The ICCs were highest for milk and milk products, healthy fats, discretionary foods and fruits. Significant associations were found between the two administrations of the MHDOS with Spearman's correlation coefficient ranging from 0.55 for vegetables to 0.82 for fruits and 0.79 for the total Diet Score (all p <0.001).

Table 3.

Total Diet Score and components scores, test-retest reliability, and Spearman correlation coefficient between MHDOS1 and MHDOS2

	MHDOS1†		MHDOS2‡		Test-retest reliability§		Correlation¶
	Mean	SD	Mean	SD	ICC	95% CI	r	p-value
Total Diet Score (out of 100)	48.1	12.7	48.6	13.4	0.90	0.87-0.93	0.79	<0.001*
Component Scores (out of 100)
Vegetables	58.6	32.8	56.2	34.3	0.71	0.62-0.78	0.55	<0.001*
Fruits	34.0	30.3	35.7	33.8	0.82	0.79-0.88	0.82	<0.001*
Rice, other cereals, whole grain cereal-based products & tubers	50.9	22.3	51.2	22.3	0.75	0.67-0.81	0.61	<0.001*
Fish, poultry, eggs, meat & legumes	43.2	19.4	43.5	19.8	0.79	0.72-0.84	0.66	<0.001*
Milk & milk products	19.2	20.8	19.7	21.6	0.86	0.82-0.89	0.74	<0.001*
Fluids	89.4	14.1	90.4	11.6	0.70	0.61-0.77	0.75	<0.001*
Healthy fats	60.7	24.4	58.9	25.0	0.85	0.80-0.89	0.70	<0.001*
Discretionary/snack foods	37.9	35.3	41.6	35.9	0.85	0.80-0.89	0.74	<0.001*
Variety	49.2	12.1	47.6	12.8	0.72	0.63-0.78	0.56	<0.001*

CI, confidence interval; ICC, intraclass correlation coefficient; MHDOS, Malaysian Healthy Diet Online Survey; SD, standard deviation.

^†MHDOS1 = First administration of MHDOS.

^‡MHDOS2 = Second administration of MHDOS.

^§ICC calculated using a two-way mixed model, type: absolute agreement.

^¶p values are derived from the Spearman correlation analysis.

^*p<0.001.

Table 4 presents the mean servings per day for each component assessed using the MHDOS1 and a single 24-hour diet recall (24DR). Spearman's correlation analysis indicates that the association between the MHDOS1 and 24DR was significant for all components, ranging from 0.21 for discretionary foods to 0.44 for milk and milk products (all p <0.001) (Table 4). Results from the Wilcoxon signed-rank test show that the estimated servings of fruits, rice and other cereals, as well as fish, poultry, eggs, meat, and legumes were not different between the MHDOS1 and 24DR (p >0.05). However, for vegetables, milk and milk products, and fluids the estimated servings from the MHDOS1 were significantly higher than the 24DR (p <0.001). In contrast, servings of discretionary foods from MHDOS1 were significantly lower than those from 24DR (p <0.001).

Table 4.

Comparison of food group intakes estimated from MHDOS1 and 24DR

MHDOS components	Intake (servings/d)				Difference (servings/d)†		Correlation‡
	MHDOS1		24DR		MHDOS1 v. 24DR
	Mean	SD	Mean	SD	Mean	p-value	r	p-value
Vegetables	2.1	1.7	1.6	1.6	0.5	<0.001	0.37	<0.001*
Fruits	0.7	0.7	0.7	1.1	0	0.113	0.38	<0.001*
Rice, other cereals, whole grain cereal-based products & tubers	3.0	1.6	2.9	1.4	0.1	0.576	0.28	<0.001*
Fish, poultry, eggs, meat & legumes	2.5	1.8	2.3	1.1	0.2	0.655	0.25	<0.001*
Milk & milk products	0.5	0.6	0.2	0.4	0.3	<0.001	0.44	<0.001*
Fluids	7.9	2.9	6.8	3.2	1.1	<0.001	0.30	<0.001*
Discretionary/snack foods	1.8	1.6	2.8	2.6	-1.0	<0.001	0.21	<0.001*

24DR, single 24-hour diet recall; MHDOS, Malaysian Healthy Diet Online Survey; MHDOS1, first administration of MHDOS; SD, standard deviation.

^†p values are derived from the Wilcoxon signed-rank test.

^‡p values are derived from the Spearman correlation analysis.

^*p<0.001.

To assess the construct validity of the survey, food groups, as well as energy and nutrients intake were compared across the tertiles of the Diet Score calculated using data from the MHDOS1 (Table 5). A higher Diet Score estimated using the MHDOS was associated with significantly higher intake servings of vegetables, fruits, and fluids (all p-trend <0.01). Conversely, higher Diet Scores were associated with lower intake of discretionary foods (p-trend <0.001). The highest tertiles of Diet Score was significantly associated with higher intakes of energy-adjusted total fiber, vitamin C, thiamine, niacin, potassium, calcium, phosphorus, and iron (all p-trend <0.01).

Table 5.

Food groups and nutrient intakes according to Diet Score tertiles calculated using the MHDOS

	Tertiles of total Diet Score†
	Low (16.9-41.5)	Medium (41.6-52.3)	High (52.4-88.4)	p for trend‡
Food groups (servings/day)
Vegetables	1.1	1.3	2.3	<0.001**
Fruits	0.4	0.6	1.0	<0.001**
Rice & cereals	3.1	2.9	2.8	0.151
Meat & alternatives	2.1	2.2	2.5	0.059
Milk & milk products	0.2	0.2	0.2	0.241
Fluids	6.1	6.6	7.7	0.009*
Discretionary foods	3.2	3.0	2.2	<0.001**
Energy and nutrients§
Energy (kcal)	1936	1972	1884	0.551
Protein (g)	58.6	58.0	62.2	0.098
Carbohydrate (g)	214.4	209.9	207.9	0.326
Fat (g)	70.5	72.8	71.8	0.649
Total fibre (g)	6.9	7.1	9.6	0.002*
Vitamin A (ug)	690.9	653.7	638.9	0.542
Vitamin C (mg)	84.6	88.8	146.3	<0.001**
Thiamin (mg)	0.7	0.7	0.8	0.002*
Riboflavin (mg)	1.2	1.0	1.1	0.685
Niacin (mg)	9.3	8.0	10.2	0.002*
Sodium (mg)	2963.2	2949.4	3242.2	0.104
Potassium (mg)	966.9	1089.1	1359.6	<0.001**
Calcium (mg)	468.4	405.0	481.6	0.017*
Phosphorus (mg)	595.2	584.4	617.5	0.029*
Iron (mg)	12.1	12.0	14.2	0.012*

MHDOS, Malaysian Healthy Diet Online Survey.

^†Values are presented as mean.

^‡p values are derived from the Jonckheere-Terpstra Test.

^§Energy-adjusted nutrients.

^*p <0.05

^**p <0.001.

The results of the Bland-Altman analyses of each food group intake are shown in Figure 3. Differences in the estimated servings between the MHDOS1 and the reference method (24DR) are plotted on the Y-axis, while the mean intake derived from the two tools is presented on the X-axis. The mean difference and 95% lower and upper limits for each food group intake were: vegetables 0.52 servings (-3.28, 4.32); fruits -0.002 servings (-2.03, 2.03); rice, other cereals products, and tubers 0.05 servings (-3.59, 3.69); fish, poultry, eggs, meat and legumes 0.23 servings (-3.56, 4.03); milk and milk products 0.26 servings (-0.88, 1.40); fluids 1.11 glasses (-6.21, 8.44); and discretionary foods -0.97 servings (-6.58, 4.64).

Figure 3.

Bland-Altman plot showing the mean bias and 95% limits of agreement between the MHDOS and 24DR. (A) Vegetables. (B) Fruits. (C) Rice & cereal. (D) Meat & alternatives intakes. (E) Milk & milk products. (F) Fluids. (G) Discretionary foods

Discussion

The present study evaluated the reliability of the MHDOS and its relative and construct validity compared to a single 24DR, which served as the reference method. The MHDOS is adapted from the CSIRO Healthy Diet Score Survey, an online tool designed to assess overall diet quality in adults by measuring adherence to national dietary guidelines from self-reported short questions. Our findings suggest that the MHDOS was able to estimate food group intake within half a serving or less of the reference method for most food groups, except discretionary foods and fluids, making it a valuable tool with moderate validity and good reliability for evaluating diet quality among healthy adults in Malaysia.

The MHDOS allows for the calculation of a Diet Score (out of 100), reflecting adherence to all recommendations outlined in the Malaysian Dietary Guidelines 2020.¹⁵ A higher score indicates greater compliance with these guidelines. Based on the first administration of MHDOS, the total Diet Score within this sample ranged between 17 and 88 with a mean score of 48.1 out of 100. This wide range indicates that the MHDOS can detect varying levels of adherence to dietary guidelines, capturing both low and high compliance among participants. Notably, participants scored highest in fluids intake, with a mean score of 89.4 out of 100. The average scores were lowest for milk and milk products (19.2 out of 100), fruits (34.0 out of 100), and discretionary foods (37.9 out of 100), highlighting low adherence to guidelines across a number of different dietary components.

In terms of test-retest reliability, the present study reported an ICC of 0.90 for the total Diet Score and ICCs ranging from 0.70 to 0.86 for the component scores, indicating moderate-to-good reliability of the survey in evaluating diet quality. The interval for test-retest was one week, and similar findings were reported for the foundational study among Australian adults, with an ICC of 0.71 for the administrations of the Australian survey in 61 adults, also one week apart.¹³ Another study to report comparable results for reliability is a study of 751 Dutch adults which reported an ICC of 0.91 for a Health Diet Index assessed using a food frequency questionnaire administered between two administrations, with an average interval of 3.8 months.²⁴ These findings suggest the reliability of the MHDOS was strong for the Diet Score overall, and moderate to good across all nine component scores.

The relative validity of the MHDOS was assessed by determining the association between food group daily intake servings derived from MHDOS1 and 24DR. Weak-to-moderate correlations (range between 0.21-0.44) were observed between the estimated intake for all food groups between the two methods. Among these, only milk and milk products showed moderate correlations of 0.44, while all other food groups reported correlations of less than 0.40. The observed correlations in this study were comparable to other studies, including those reported by Hendrie et al.¹³ for the Australian study, who found a moderate correlation of 0.43 between Diet Index scores derived from a short food survey and three 24-hour dietary recalls. Cleghorn et al.²⁵ evaluated the agreement between a short-form FFQ and an extensive FFQ in UK adults, observing a correlation of 0.38 for an overall Diet Quality Score. Correlations reported by Whitton et al.²⁶ for a study in Singapore residents were slightly higher, with a Spearman's correlation coefficient of 0.51 reported when comparing Diet Screeners and FFQs. The low correlations between the two methods reported in this study may be due to the reference method being a single 24-hour diet recall which captures a shorter duration and may not reflect individuals’ habitual intake, which is the intent of the MHDOS. In MHDOS, participants respond to questions about how often they consume specific foods or food groups, potentially capturing day-to-day variation in food choices and intakes, although it relies on individuals’ ability to recall and reflect this variation in their estimate of intake. A single 24-hour diet recall also relies on individuals to retrospectively recall their intake, but only over a 24-hour period. While this may help with recall, the 24-hour recall period may not reflect usual intake and could result in under- or overestimation of dietary intake.^{27, 28}

In the present study, differences in daily servings for food groups between MHDOS1 and 24DR were examined using Wilcoxon signed-rank tests. The MHDOS was able to estimate food group intake within about a half of one serving for fruits, vegetables, rice and other cereals, tubers, meat, and milk products; and within about one serving for fluids and discretionary foods. The results indicated that the mean intakes for fruits, rice and other cereals, fish, poultry, eggs, meat, and legumes were not significantly different between the two methods. This finding may be attributed to the frequent consumption of these food groups, particularly rice, which serves as the staple food in Malaysia, as well as meat and meat alternatives.^{29, 30} Foods, like these, that are regularly consumed may make it easier for participants to estimate usual consumption. And how foods are consumed, such as fruit being consumed whole portion of about one serving, may also make it easier to estimate intake.¹⁴ Results also showed that intake reported through the MHDOS was higher than 24DR for vegetables, milk and milk products, and fluids; but lower for discretionary foods. The significant differences found between the two methods may be attributed to the day-to-day variation in intakes for these food groups, but it is known that under- or over-estimation of dietary intake is common in all forms of dietary assessment, but in particular short questions where social bias and participants’ inability to accurately estimate portion sizes are more likely to influence results.^{23, 24}

To evaluate the construct validity of the MHDOS, food groups and nutrient intakes derived from the 24DR were compared across tertiles of diet quality assessed by MHDOS. Results showed those with higher diet quality reported higher consumption of vegetables, fruits, and fluids, along with lower intake of discretionary foods compared to those with lower diet quality. These findings are consistent with those reported by Drake et al.,³¹ who observed similar findings among the Swedish population. This is consistent with what would be expected from a valid diet quality score, as fruits, vegetables, and water are typically core to a healthy diet. Conversely, discretionary foods, which often include items high in added sugars, unhealthy fats, and calories but low in essential nutrients, are generally associated with poorer dietary quality when consumed in excess. Consistent with previous studies,^{32, 33} our study also found that higher tertiles of the Diet Score were associated with higher intakes of total fiber, vitamin C, thiamine, niacin, potassium, calcium, phosphorus, and iron – all nutrients that are associated with higher consumption of healthy foods and a healthier dietary pattern. It is important to note that the Diet Score did not significantly differ for energy and the macronutrients (protein, carbohydrate, and fat) across the tertiles. Our findings suggest that the observed improvements in diet quality associated with higher Diet Scores were not driven by changes in overall energy intake or macronutrient composition. Instead, they appear to be primarily driven by shifts in food choices towards nutrient-dense “healthier” options such as fruits and vegetables, and lower consumption of discretionary foods. In other words, the Diet Score effectively captures variations in dietary patterns related to nutrient-rich foods, independent of overall energy intake or macronutrient distribution.

Although an increasing number of studies have employed diet indices to describe diet quality in adults across various countries,^13,26,32,33 research in this area remains relatively scarce in Malaysia. Several attempts have been made to develop diet quality measures based on the MDG, such as the Malaysian Diet Quality Index (MDQI)³⁴ and the New Standardized Malaysian Healthy Eating Index (S-MHEI).³⁵ The MDQI was derived from a FFQ administered among local university students and validated using exploratory factor analysis.³⁴ The components of the S-MHEI were derived by referencing the MDG 2010 and MDG for Children and Adolescents 2013, but no validation of the index has been conducted to date.³⁵ It is important to note that both of these diet quality indices were developed based on older version of MDG, potentially limiting their relevance to current dietary patterns and recommendations.

A key strength of this study was the inclusion of a large, diverse sample from 13 states and 3 federal territories in Malaysia. This geographical diversity enhances the generalisability of the findings, as it ensures that a wide range of dietary habits and response capabilities are captured, and therefore making the results applicable to the broader population. This study has demonstrated the ability of the MHDOS to comprehensively assess overall diet quality while capturing compliance with all recommendations in the latest MDG through a relatively short questionnaire. This online survey not only reduces respondent burden but also offers high convenience for large-scale population studies, which may be useful in monitoring dietary compliance across diverse demographic groups. Additionally, MHDOS incorporates coloured visual aids that illustrate examples of portion sizes for various food items in a single serving, aiding participants in estimating their dietary intake more accurately.

Several limitations of the study should be taken into consideration. Firstly, the MHDOS was designed to capture the diet quality of the general adult population in Malaysia. While the distribution of body weight status aligned with the broader Malaysian population as reported in national surveys,36 it is important to note that the majority of participants were below 40 years old and from professional occupations. This overrepresentation of younger generations might be expected in online surveys, as younger and highly educated individuals are generally more IT-literate and therefore responsive to an online survey like MHDOS. Consequently, this sampling bias may limit the generalisability of the findings to older or less educated populations, particularly in assessing age-specific dietary patterns. Additionally, there is a potential overestimation of overall diet quality, as younger professionals may have greater nutrition awareness. Future multicentre studies should implement targeted recruitment strategies across diverse age groups and educational backgrounds to enhance the validity of these findings, while considering hybrid administration methods to improve population coverage. Secondly, the 24DR method was used as a reference method to evaluate the relative and construct validity of the MHDOS. To minimize intra-observer error, dietary data were collected by trained state nutritionists through multiple-pass, single 24-hour diet recall interviews. However, relying on a single 24DR may not fully represent the participants’ usual intake and is susceptible to measurement and social desirability bias. This limitation is particularly relevant for episodically consumed items, such as discretionary foods, potentially weakening validity correlations with the MHDOS. Social desirability bias may also be more pronounced in face-to-face recalls compared to the self-reported MHDOS, inflating agreement between methods. Additionally, the absence of an updated food composition database and incomplete nutrient data, meant that saturated fats and total sugar in the local nutrient database could not be assessed, and may have impacted the estimated nutrient contribution of certain food items reported in the 24DR, particularly discretionary foods. These limitations highlight the need for future validation of the MHDOS against multiple dietary recalls or biological markers.

As the first of its kind in Malaysia, the MHDOS showed good reliability and moderate validity as a tool to evaluate diet quality among healthy adults in Malaysia. The MHDOS exhibited good test-retest reliability, with strong intraclass correlation coefficients for total and component Diet Scores. Notably, the attainment of higher Diet Scores derived from the MHDOS correlated with higher consumption of healthier food intakes (vegetables, fruits, fluids) and beneficial nutrients (fiber, vitamins, minerals), while lower scores correlated with higher discretionary food intake. Although correlations with the reference method (24DR) were moderate, the MHDOS provided comparable estimates for several food groups. Despite slight overestimation of vegetable and fluid servings compared to 24DR, the MHDOS effectively discriminates dietary patterns aligned with national guidelines. Its applicability is further supported by consistent performance across most of the key demographic groups. The online format was well-received, with most participants finding it easy to use and appropriately timed for completion. These findings highlight the MHDOS as a practical, cost-effective, and scalable tool for dietary assessment, reducing participant burden compared to traditional methods. Its integration into large-scale dietary surveillance and public health initiatives could enhance dietary monitoring, identify at-risk populations, and inform targeted nutrition interventions. Policymakers and healthcare professionals may leverage MHDOS data to design evidence-based dietary guidelines and intervention programs that address population dietary gaps. To enhance its validity and applicability, future studies should address current methodological limitations through expanded validation approaches and assess its ability in predicting health outcomes.

Supplementary Materials

All supplementary materials are available upon request to the editorial office.

Acknowledgements

The authors would like to thank the Director General of Health Malaysia for the permission to publish this paper. The authors also thank the state nutritionists from the Nutrition Division, Ministry of Health, Malaysia for their assistance in data collection. CSIRO provided in-kind contributions, including access to the survey and expertise.

Conflict of Interest and Funding Disclosures

The authors report no conflicts of interest.

This research was funded by the Nutrition Society of Malaysia (NSM). The article publication fee was supported by Universiti Kebangsaan Malaysia.