Consumer testing away from a sensory facility: Application of home-use test and no-contact home-use test

Abstract

COVID-19 pandemic-related restrictions for approximately three years have heavily influenced sensory evaluations. People have become accustomed to working remotely and communicating online. This has led to opportunities in sensory testing paired with logistics systems and information technologies, resulting in a wide application of the home-use test (HUT), wherein panelists evaluate samples from their homes or other off-site locations. This study aimed to compare three sensory evaluation conditions: a central location test (CLT, n = 104), a HUT (n = 120), and a no-contact HUT (N-HUT, n = 111). We recruited participants via the local community website, delivered samples using a delivery service, and conducted sensory testing using a smartphone for the N-HUT. Participants were requested to report the acceptance ratings, sensory profiles, and emotion responses to four coffee samples. Some differences in the acceptance ratings might be due to the different attitudes participating in the evaluation. In the sensory profiling of the samples, multi-factor analysis (MFA) revealed highly similar sensory characteristics across the three types of tests. All RV coefficients (RVs) among the test conditions were above 0.93. The emotion responses to coffee samples were similar among test conditions based on the MFA with RV values greater than 0.84. In conclusion, we found that N-HUT produced similar results regarding the descriptions of sensory profiles and emotions, indicating that N-HUT is a suitable test method for collecting sensory data and overcoming CLT and HUT's regional limitations. Modern logistics systems and information technologies make it possible to conduct nationwide sensory evaluations without in-person contact or participant attendance at sensory testing facilities.

1. Introduction

The Covid-19 pandemic brought about a situation that people have never experienced. The era before corona (BC) was distinctly different from that after corona (AC), as the resulting regulations irreversibly altered the lives and habits of people (Lee & Ham, 2021). The Covid-19 pandemic significantly changed the daily lives of individuals by increasing the activities associated with risk aversion, which may have also affected their food-related behavior and preferences (Im et al., 2021). During the pandemic period, no-contact methods have explosively increased in popularity in the food service industry, with people preferring indirect interaction (e.g., person-machine/person-machine-person) instead of direct person-to-person interactions (Dinnella et al., 2022, Kim et al., 2017, Lee and Lee, 2021, Pokhrel and Chhetri, 2021, Varela, 2022). Covid-19 is almost over, but the lifestyle habits that have changed due to this remain continuous. This will provide new challenges in sensory testing together with logistics systems and IT technology.

Consumer tests are typically conducted at the sensory facility, which is conducted in a set of standardized sensory booths and a controlled environment. After the pandemic, many food industry establishments and universities have not conducted sensory evaluations of food or beverage samples at their indoor facilities to minimize the potential risk of contracting and spreading Covid-19. Many educational and research institutions have also not allowed face-to-face human research, including sensory and consumer studies (Seo et al., 2021). The Covid-19 pandemic has resulted in a global pivot to home-use tests (HUT) or no-contact tests, where panelists can evaluate samples in their homes or isolated areas, respectively (Shi et al., 2021). Romeo-Arroyo et al. (2022) studied the effect of the central location test (CLT) and HUT on the sweetness perception and liking of four butter cookies. Although the CLT context showed more discriminating power among the samples, no significant differences were found between CLT and HUT evaluations in hedonic and just-about-right scale evaluations. Sinesio et al. (2021) focused on the influence of wine traits and context (CLT, CLT-evoked, and HUT) on liking, intention to consume, wine-evoked emotions, and perceived sensory sensations. Their results showed that liking and intention to consume did not vary significantly among the test conditions. However, the context influenced emotion and sensory terms by lowering positive and negative emotions and sensory terms in the HUT. Seo et al., (2021) studied whether a drive-in booth (DIB) environment could be an effective alternative to a laboratory sensory booth (LSB) environment by comparing the two conditions using four beverage samples with respect to sensory and emotion responses. The sensory and emotion responses were similar in the two test conditions. However, participants felt safer during the sensory evaluation in the DIB compared to the LSB.

Although the Covid-19 pandemic is close to ending at this time, the long-term effects still heavily influence sensory testing. People are more open to using information technology, such as smartphones and online meeting software, and have become accustomed to online shopping. This atmosphere makes conducting HUT easier through specific data acquisition software and online tests, which increase the speed of carrying out HUT (Niimi et al., 2022). With the modern IT and logistic system, HUT can be widely used in sensory testing. CLT is generally conducted with consumers from certain cities or areas, whereas HUT can be used to conduct domestic and international testing. Although HUT was frequently used in the food industry, there is a lack of studies evaluating the effect of test conditions compared to CLT. There are many aspects to consider regarding HUT: sample size (size, minimum consumption, package effect, product type), procedure (transportation to the lab, personal visit, delivery service), evaluation method (questionnaire on paper, online questionnaire, order of samples, presence of other people), and time (evaluation time, forced interval time) (Lee & Lee, 2021). Recently, Lee and Lee (2021) conducted HUT using a parcel delivery service in Korea. This study investigated and compared the results obtained using a home-use test by three groups differing in ‘time and order’ control using six commercial snacks with different textures. Overall, the results were similar between the evaluated groups, regardless of the degree of control. Smartphone usage is over 95% in the Republic of Korea (Gallup Korea, 2021), and a modernized logistic system can deliver parcels within two days to most areas of Korea. These two advantages can make HUT an efficient approach in Korea, thereby eliminating the need for participants to be in direct contact at the sensory testing booths. Additionally, this test condition is similar to the online ordering of food products and their delivery to the purchaser’s location. This different test condition context compared to CLT and HUT could generate a discrepancy in the result. Therefore, this study aimed to compare three test conditions - CLT, HUT, and no-contact HUT (N-HUT) - to investigate the effect of sensory evaluation testing outside of a sensory facility and the sample receipt method on the results. Samples were evaluated in sensory booths (CLT), at home with face-to-face receipt of samples (HUT), and at home with no-contact delivery (N-HUT).

2. Materials and methods

2.1. Sample selection and preparation

The coffee samples used in this study were carefully selected, with consideration given to their practicality for use in sensory evaluation. One important factor was the ability to individually package and store the samples at room temperature, enabling easy transportation by courier. Additionally, it was essential to select samples that were easy to prepare, yet not too easy as to impede their ability to distinguish differences between the test conditions.

Four types of coffee were used in this study. Two different coffee varieties, Coffea arabica (Colombia Supremo Narino 17/18 washed, GSC International, Co. Ltd., Seoul, Korea) and Coffea canephora (Vietnam Robusta G1 polished, GSC International, Co. Ltd., Seoul, Korea) were used. Green coffee beans (20 kg) were roasted at two different roasting degrees using a Loring Smart Roaster (S35 Kestrel, Loring, Santa Rosa, CA, USA) by a professional coffee roaster. For medium roasting of two coffee varieties, green coffee beans were put into a roaster at 170 °C, the roasting temperature was then dropped to 66 °C and finished at 225 °C. Roasting was conducted for 11 min and 47 s. Dark roasting was carried out similarly to medium roasting, but the final temperature was 232 °C, and the roasting duration was 12 min 26 s. The roasted coffee beans (approximately 1 kg) were packaged and degassed for three days, then ground, and finally, 10 g of the ground coffee was placed in a brewing coffee bag. The bags with coffee were sealed using a three-layer (polyethylene terephthalate/aluminum/oriented polypropylene) package with an oxygen absorber (Everfresh, Lipmen, Co. Ltd., Incheon, Korea) to minimize lipid acidification during storage and labeled as AM for Arabica medium roasting, AD for Arabica dark roasting, RM for Robusta medium roasting, and RD for Robusta dark roasting. The samples were kept in the refrigerator at approximately 4 °C before the testing.

2.2. Participants

All participants were born and grown in Korea. Only people who drank at least one cup of coffee (400 mL or more) per week and could complete the questionnaire using a smartphone or computer were eligible to participate in the study. The study was approved by the Korea Food Research Institute (KFRI) Institutional Review Board.

2.3. Central location test (CLT)

A total of 108 participants were recruited through an online bulletin board at the KFRI (Wanju-gun, Korea). Among them, 104 out of 108 (39 males and 65 females, aged 20–60 years) participated in the CLT test. The CLT was conducted in individual booths at the sensory facilities of the KFRI under a fluorescent lighting system. Four samples with a 3-digit randomized code were served monadically with filtered water (100 mL) at 95 °C in a heat-resisting plastic beaker (300 mL, MEP International, Goyang-si, Korea). The approximate drinking temperature of coffee based on the pre-experiment was 68–70 °C. Participants performed self-brewing in the booth. They pour given hot water into a coffee bag (ground coffee in a coffee filter), hung in in a white paper cup (384 mL) for 30 s, waited for 3 min, removed the brewed coffee bag, and evaluated for 3 min, following the instructions and timer function in the online questionnaire. Each sample was evaluated for approximately 10 min, and the total evaluation was conducted for approximately 45 min. Crackers (Original Carr’s table water, United Biscuits Ltd., Carlisle, UK) and filtered water served as palate cleaners to minimize the carry-over effect. The online questionnaire (Compusense Cloud software, Compusense Inc., Guelph, Canada) was used to collect data on acceptance, sensory profiles, and emotions from the participants.

2.4. Home-use test (HUT)

A total of 120 local residents (47 males and 73 females, aged 20–60 years) were recruited from Jeonju-si and Wanju-gun, Korea. The participants were asked to visit and pick up the evaluation packages containing four samples with a 3-digit randomized code on a package, crackers (Original Carr’s table water), four white paper cups (384 mL), a heat-resisting plastic beaker (300 mL, MEP International), and an instruction sheet describing how to access the ballot and conduct a sample evaluation (Fig. S1). All participants were instructed to evaluate within four days of the sample pick-up at KFRI. Similar to the CLT, participants were asked to evaluate four samples, one after the other. The tasting protocol was the same as that of CLT; however, the participants had to prepare hot water and measure 100 mL of water by themselves. The order of tasting of the samples was designated on the electronic ballot. Compusense Cloud software was used for data collection with the same evaluation questions used in the CLT.

2.5. No-contact home-use test (N-HUT)

A total of 120 participants were recruited through an online used goods trading application (Daangeun Market Inc., Seoul, Korea). The recruiting information was restricted to users in the Jeollabuk-do area in Korea, which covers 8067 km² and has 1.87 million residents. Their intentions to participate in coffee evaluation at home and addresses were collected using an online survey tool (Naver, Co., Seongnam-si, Korea). Respondents who agreed to provide their personal information and bank account number for sample delivery and compensation were selected as participants. The participants were asked to complete an online consent form using a legally effective electronic contract service (Modusign Inc., Seoul, Korea). Among them, 113 of 120 participants who completed the online consent form were eligible to receive sample packages for sensory evaluation. They received the same evaluation materials as the HUT participants through a national parcel delivery service (ePOST, Naju-si, Korea). All parcels arrived within two days based on the tracking service of ePOST. All participants were instructed to evaluate within four days of receiving the sample at home. Like the CLT, participants were asked to evaluate four samples back-to-back. The tasting protocol was the same as that of the CLT; however, the participants had to prepare hot water, measure 100 mL of water, and brew coffee samples themselves. Among them, 111 out of 113 (51 males and 60 females, aged 20–60 years) completed their evaluation in five days using the Compusense Cloud software for data collection with the same evaluation questions used in the CLT.

2.6. Questionnaire

Participants filled out the electronic ballot using electronic devices such as smartphones, tablets, or laptops. The questionnaire consisted of queries regarding product acceptability, sensory profiling, and emotion responses for each coffee sample. Acceptability for overall liking and liking of color, aroma/odor, taste/flavor, and mouthfeel was asked using a 9-point hedonic scale (1 = ‘extremely dislike’, 5 = ‘neither like nor dislike’, and 9 = ‘extremely like’). Purchase intent was measured using a 5-point scale (1 = ‘I would definitely not buy’ and 5 = ‘I would definitely buy’). Consumers were then provided with a rate-all-that-apply (RATA) list of 16 sensory attributes (aroma: sour, citrus fruit, burnt; taste/flavor: bitter, sour, sweet, dark chocolate, burnt, citrus fruit, nutty, grain; mouthfeel: thickness, longevity, astringent, and mouth drying). These sensory attributes were selected based on previous studies (Cotter et al., 2021, Seo et al., 2009, World Coffee Research, 2017) and discussion by six sensory scientists. For emotion measurement, the RATA was used for 17 emotion terms: active, bored, disgusted, energetic, free, merry, peaceful, joyful, pleased, satisfied, wild, worried, good, pleasant, guilty, warm, and understanding. These emotion terms existed in both EsSense25® and Coffee Drinking Experience (CDE) profiles (Bhumiratana et al., 2014, Kanjanakorn and Lee, 2017, Pinsuwan et al., 2022). In the RATA method for sensory attributes and emotions, consumers were asked to check all the terms that they considered appropriate for describing samples (check-all-that-apply; CATA) and to rate the intensity of the applicable terms using a 3-point structured scale (1 = ‘weak’, 2 = ‘medium’, and 3 = ‘strong’) only for the terms they checked. Following the sensory evaluation, participants completed a questionnaire about their demographics and coffee consumption (Table S1).

2.7. Data analyses

For the liking and purchase intention data, analysis of variance (ANOVA) was conducted on the entire data set using a generalized linear mixed model (GLMM) to analyze the effect of the sample and test conditions. The statistical significance threshold was set at P < 0.05. Consumers were considered as a random factor. The following GLM model was applied: liking or familiarity = sample + test condition + sample × test condition + consumer × test condition. Liking and purchase intent scores were analyzed using one-way ANOVA to determine if there were any significant differences (P < 0.05) among samples within each test condition to present sample discriminations or testing conditions to investigate statistical significance across the conditions. The effects of roasting degrees and coffee varieties were included in the error terms since they were not the key parameters in the comparison of test conditions. Tukey's honest significance test (HSD) was conducted when a significant difference was found. Data were preprocessed by replacing unrated attributes with 0 in the data matrices of RATA (Meyners et al., 2016). The means of the sensory and emotion attributes for each product were calculated for each test condition. Principal component analyses (PCA) were performed on the average ratings of the terms to identify the relationships between samples and terms. In addition, multiple factor analysis (MFA) was performed to compare the configuration between the three test conditions using sensory or emotion terms as active variables for the computation of the factors of the MFA. RV coefficients were calculated to determine the degree of similarity between the emotion and sensory spaces resulting from the three testing conditions (Robert & Escoufier, 1976). The first two dimensions were considered in PCA and MFA. All data analyses were conducted using the XLSTAT version 2021 (Addinsoft, Paris, France).

3. Results

3.1. Comparison of test conditions and demographics

Resources used in the three sensory tests are listed in Table 1 . The final participant ratios for the CLT, HUT, and N-HUT groups were 96.3%, 100%, and 92.5%, respectively. The total labor hours for CLT, HUT, and N-HUT were 44 h, 13 h, and 17 h, respectively. For CLT, a large portion of labor hours was attributed to sample serving, whereas serving was unnecessary for HUT and N-HUT. Six people conducted participant greeting, sample preparation, serving samples and hot water, and clean-up. The total costs for CLT, HUT, and N-HUT, excluding labor costs, were USD 737.4, USD 1,742.8, and USD 2,326.2, respectively, and the cost per capita excluding labor fee for CLT, HUT, and N-HUT was USD 7.1, USD 14.5 and USD 20.9, respectively.

Table 1.

Participant, labor hours, and costs of the three test conditions.

Category	CLT1)	HUT	N-HUT
Participant
Recruiting	108	120	120
Finish	104	120	111
Attendance rate	96.3%	100.0%	92.5%
Labor hour (h)
Recruiting	1	1	3
Sample preparation and/or delivery	6	6	8
Sample serving	36	–	–
Panel monitoring		4	4
Paper work	1	2	2
Total	44	13	17
Cost (USD)2)
Materials and package3)	4.2	51.0	65.5
Delivery cost	–	–	304.64)
Panel compensation	733.1	1691.8	1956.2
Total	737.4	1742.8	2326.2
Per capita	7.1	14.5	20.9

¹CLT, HUT and N-HUT mean central location test, home-use test, and no-contact home-use test, respectively.

²USD: Korean Won is 1: 1,262, average exchange ratio from 14 April to 16 June 2022.

³Commonly used material costs were excluded.

⁴Twenty-five percent discount was offered due to bulk shipment (above 100).

Demographic information for the three test methodologies is shown in Table S1. The majority of the participants were aged 19–39. Approximately 60% of the participants in CLT and HUT were female; however, the female ratio was lower in N-HUT at 54.0%. The average monthly expenditure for coffee was similar across the test methodologies. The main consumption place was similar in CLT and HUT; however, more participants drank coffee at home and restaurants in N-HUT. Regarding the primary time for coffee consumption, participants in CLT and HUT drank more in the morning than in the N-HUT.

3.2. Effect of test conditions on acceptance for liking and purchase intent

Table 2 shows the results of the ANOVA using GLMM for the three test conditions regarding liking and purchase intent. There were significant differences among the samples for overall color and taste/flavor-liking (P < 0.05). Test condition, sample × test condition, and test condition × consumer had a significant effect (P < 0.05), whereas no significant differences were found for color liking by test condition and aroma/odor liking by sample × test condition (P > 0.05). Significant differences (P < 0.01) in liking and purchase intent were observed across the test conditions (Fig. 1 ). The highest ratings were obtained using N-HUT, whereas CLT and HUT produced similar ratings for all evaluated characteristics.

Table 2.

Effect of the sample, test location, and their interactions on the rating of liking and purchase intent for the coffee samples.

	Factors and their interaction
	S1)	T	S × T	T × C
Overall liking	*2)	***	*	***
Color liking	*	n.s.	*	***
Aroma liking	n.s.	***	n.s.	***
Taste and flavor liking	*	***	*	***
Mouthfeel liking	n.s.	**	*	***
Purchase intent	n.s.	**	*	**

¹S, T, and C denote sample, test condition, and consumer, respectively.

²Significance levels are as follows: (***) P < 0.001; (**) P < 0.01; (*) P < 0.05; n.s. meant non-significant.

Fig. 1.

Acceptability and purchase intent ratings obtained under the three test conditions. CLT, HUT and N-HUT mean central location test, home-use test, and no-contact home-use test, respectively. Significance levels are as follows: (***) P < 0.001; (**) P < 0.01.

In each test condition, the results of the overall liking rating between samples showed that RM had the highest score in the CLT. There were no significant differences between samples in HUT, and RD had the highest rating in N-HUT (Table 3 ). The number of items that showed a significant difference (P < 0.05) in ratings among the samples in each test condition was five for CLT and two for N-HUT (Table 2). In CLT, there was a significant difference (P < 0.05) between samples in all characteristics except for aroma/odor liking. In N-HUT, significant differences (P < 0.05) were observed in overall liking and taste/flavor liking (Table 3). The liking of each sample was not discriminated in HUT.

Table 3.

Acceptance and purchase intent ratings of the coffee samples in each test condition.

Test condition and sample	Overall liking1)	Color liking	Aroma liking	Taste/flavor liking	Mouthfeel liking	Purchase intent
CLT2)
AM3)	5.20ab4)	6.42ab	5.60	4.79ab	5.53ab	2.66a
AD	4.98b	6.33ab	5.45	4.73ab	5.57ab	2.59a
RM	5.71a	6.58a	5.79	5.40a	5.93a	3.01a
RD	4.94b	5.96b	5.27	4.66b	5.14b	2.59a
P-value	0.02	0.007	0.23	0.034	0.007	0.036
HUT
AM	4.93	6.17	5.47	4.81	5.68	2.65
AD	5.22	6.39	5.88	5.04	5.76	2.78
RM	5.20	6.30	5.61	4.98	5.60	2.65
RD	5.00	6.02	5.34	4.82	5.31	2.52
P-value	0.57	0.24	0.118	0.734	0.179	0.393
N-HUT
AM	5.17b	6.41	5.89	5.10b	5.82	2.79
AD	5.64ab	6.45	6.27	5.26ab	5.96	2.87
RM	5.78ab	6.67	6.16	5.61ab	6.03	3.00
RD	6.03a	6.63	6.34	5.91a	6.18	3.20
P-value	0.02	0.576	0.294	0.026	0.57	0.069

¹Liking ratings were measured using a 9-point hedonic scale, and purchase intent was measured by using a 5-point category scale.

²CLT, HUT and N-HUT mean central location test, home-use test, and no-contact home-use test, respectively.

³AM, AD, RM and RD represent Arabica medium roasting, Arabica dark roasting, Robusta medium roasting, and Robusta dark roasting, respectively.

⁴Different superscripts within a column meant significant difference at P < 0.05 by Tukey's honest significance test.

3.3. Sensory profiling of samples using RATA

The mean ratings of the sensory attributes from the RATA for the four coffee samples are shown in Table 4 . The number of sensory attributes that were significantly different across the samples was 11 for CLT and HUT and 10 for N-HUT. Additionally, the attributes that were significantly different across samples were similar across test conditions.

Table 4.

The intensity rating of sensory attributes of coffee samples in each test condition as determined using the RATA method.

Sensory attributes1)	CLT3)					HUT					N-HUT
	AM4)	AD	RM	RD	P-value	AM	AD	RM	RD	P-value	AM	AD	RM	RD	P-value
O_sour2)	1.77a5)	1.45a	0.57b	0.50b	***6)	1.88a	1.47b	0.73c	0.65c	***	1.57a	1.22a	0.74b	0.60b	***
O_citrus fuit	0.76a	0.61a	0.26b	0.23b	***	0.83a	0.56b	0.36bc	0.28c	***	0.75	0.61	0.55	0.44	n.s.
O_burnt	1.27b	1.54b	1.43b	2.15a	***	1.15b	1.33b	1.49b	1.98a	***	1.05b	1.27ab	1.00b	1.60a	***
T_bitter	1.40b	1.60ab	1.33b	1.84a	**	1.34b	1.53ab	1.38b	1.92a	***	1.19b	1.33b	0.99b	1.74a	***
T_sour	2.13a	1.65b	0.79c	0.64c	***	1.99a	1.75a	0.87b	0.69b	***	1.97a	1.55b	0.84c	0.72c	***
T_sweet	0.35	0.33	0.43	0.32	n.s.	0.48	0.44	0.45	0.32	n.s.	0.51	0.41	0.60	0.60	n.s.
F_roasted	1.02b	1.06b	1.87a	1.70a	***	1.08b	1.11b	1.84a	1.83a	***	0.91b	1.04ab	1.40a	1.40a	***
F_dark chocolate	0.51	0.69	0.60	0.73	n.s.	0.59	0.74	0.62	0.80	n.s.	0.62	0.71	0.65	0.89	n.s.
F_burnt	1.35b	1.58ab	1.28b	1.94a	***	1.11b	1.33b	1.38b	2.13a	***	1.15b	1.14b	1.14b	1.69a	***
F_citrus fruit	0.74a	0.56a	0.23b	0.17b	***	0.88a	0.58b	0.32c	0.25c	***	0.86a	0.65ab	0.44b	0.45b	***
F_nutty	0.61c	0.59c	1.41a	1.05b	***	0.81b	0.83b	1.45a	1.12ab	***	0.72c	0.90bc	1.47a	1.28ab	***
F_grain	0.33c	0.44c	1.43a	1.04b	***	0.49b	0.53b	1.44a	1.21a	***	0.57b	0.63b	1.20a	1.08a	***
M_thickness	1.82bc	2.01ab	1.49c	2.24a	***	1.58b	1.79ab	1.47b	2.04a	***	1.36b	1.46b	1.13b	1.85a	***
M_longevity	1.40	1.59	1.17	1.42	n.s.	1.28	1.28	1.13	1.42	n.s.	1.30	1.41	1.25	1.40	n.s.
M_astringent	0.94ab	1.08ab	0.76b	1.21a	*	0.83	1.02	0.93	1.08	n.s.	0.82	0.92	0.66	0.91	n.s.
M_mouth drying	0.77	0.69	0.55	0.80	n.s.	0.61	0.78	0.64	0.75	n.s.	0.63	0.59	0.67	0.68	n.s.

¹RATA method was asked using 3-point structured scale (1 = weak, 2 = medium, and 3 = strong).

²O: odor, T: taste, F: flavor, M: mouthfeel.

³CLT, HUT and N-HUT mean central location test, home-use test, and no-contact home-use test, respectively.

⁴AM, AD, RM and RD represent Arabica medium roasting, Arabica dark roasting, Robusta medium roasting, and Robusta dark roasting, respectively.

⁵Different superscripts within a row meant significant difference at P < 0.05 by Tukey's honest significance test.

⁶Significance levels are as follows: (***) P < 0.001; (**) P < 0.01; (*) P < 0.05; n.s. meant non-significant.

The sensory attributes from RATA for the four coffee samples are characterized through PCA (Fig. 2 ). The total variance of PCA was 99.73%, 98.46%, and 98.48% for CLT, HUT, and N-HUT, respectively. In CLT (Fig. 2a), Dimension (Dim) 1 (58.65%) was positively associated with sour and citrus fruit odors, sourness, and citrus fruit flavor in the AM and AD samples, and negatively correlated with roasted, grain, and nutty flavors in the RM sample. The attributes contributing to the positive side of Dim 2 (41.08%) were burnt odor and flavor, bitterness, and astringent and thick mouthfeel with the RD sample. The PCA plot of HUT (Fig. 2b) showed that Dim 1 (60.16%) was composed of roasted, grain, and nutty flavors with RM samples, which were positively loaded, and citrus fruit odor and flavor, and sour odor and taste with AM and AD samples, which were negatively loaded. Dim 2 (38.30%) was positively associated with burnt odor, flavor, bitterness, astringent and thick mouthfeel in the RD sample. In N-HUT (Fig. 2c), Dim 1 (58.99%) was positively related to roasted, grain, and nutty flavors for the RM sample and negatively associated with citrus fruit odors, sour odor, and taste in AM and AD samples. The attributes contributing to the positive side of Dim 2 (39.49%) were the burnt odor and flavor, bitterness, and mouthfeel thickness of the RD sample.

Fig. 2.

PCA plot of the sensory attribute terms for the coffee samples under the three test conditions. CLT, HUT and N-HUT mean central location test, home-use test, and no-contact home-use test, respectively. AM, AD, RM and RD represent Arabica medium roasting, Arabica dark roasting, Robusta medium roasting, and Robusta dark roasting, respectively. O, T, F, and M stand for odor, taste, flavor, and mouthfeel, respectively.

The MFA was computed to explore the configuration similarities and differences between the three test conditions for the four samples using the sensory attribute results (Fig. 3 ). Factor 1 (F1: 57.09%) and 2 (F2; 37.50%) explained 94.59% of the total variance. The same attributes were loaded closely regardless of the three test conditions (Fig. 3b). The largest difference in sensory profiling was observed in RM. The CLT showed the largest differences from HUT and N-HUT compared to the other samples. The RV coefficient uses values between 0 and 1 to present similarity of the proximity between two sets of matrices. The values between CLT and HUT, CLT and N-HUT, and HUT and N-HUT were 0.93, 0.96, and 0.98, respectively.

Fig. 3.

Multifactor analysis of (a) individual factor map and (b) correlation circle of sensory attributes for the coffee samples under the three test conditions. CLT, HUT and N-HUT mean central location test, home-use test, and no-contact home-use test, respectively. AM, AD, RM and RD represent Arabica medium roasting, Arabica dark roasting, Robusta medium roasting, and Robusta dark roasting, respectively. O, T, F, and M stand for odor, taste, flavor, and mouthfeel, respectively.

3.4. Emotion responses of samples

The number of emotion terms that were significantly different across the samples was seven for CLT, six for HUT, and four for N-HUT. The PCA biplots for these emotion terms associated with the four coffee samples under the three test conditions are shown in Fig. 4 . The total variances of the PCA were 98.65, 97.26, and 94.07% for CLT, HUT, and N-HUT, respectively. In CLT (Fig. 4a), Dim 1 (56.70%) was positively associated with ‘peaceful’, ‘warm’, ‘satisfied’, and ‘bored’ for the RM sample and negatively associated with ‘wild’ for the RD sample. The Dim 2 (41.94%) was positively associated with ‘worried’, ‘pleasant’, and ‘merry’ for the AD and AM samples. In HUT (Fig. 4b), Dim 1 (75.48%) was positively associated with ‘energetic’, ‘active’, and ‘merry’ in the case of the AD and AM samples and negatively associated with ‘warm’ and ‘peaceful’ in case of the RM sample. The positive aspect of Dim 2 (21.78%) was associated with ‘wild’ for the RD sample. The PCA plot of N-HUT (Fig. 4c) showed that Dim 1 (65.68%) was positively associated with ‘energetic’ and ‘merry’ for the AD and AM samples and with ‘warm’ for the RM sample. For the RD sample, Dim 2 (38.30%) was positively associated with ‘wild’. The emotion terms used to distinguish the samples in the three conditions were similar; RD – ‘wild’, RM – ‘warm’, and AM and AD – ‘merry’.

Fig. 4.

Principal component analysis of the emotional terms for the coffee samples under the three test conditions. CLT, HUT and N-HUT mean central location test, home-use test, and no-contact home-use test, respectively. AM, AD, RM and RD represent Arabica medium roasting, Arabica dark roasting, Robusta medium roasting, and Robusta dark roasting, respectively.

The MFA comparison of the similarity matrices from the CLT and C- and N-HUT groups for the samples evaluated using emotion terms showed high similarity among the three test conditions (Fig. 5 ). F1 (58.36%) and F2 (26.99%) accounted for 85.35% of the total variance. The three test conditions produced similar results for the AM and AD samples but not for the RD and RM samples. Especially the largest difference in sensory profiling was observed in RM. The CLT of RM showed the largest differences between HUT and N-HUT compared to the other samples. The RV coefficients between CLT and HUT, CLT and N-HUT, and HUT and N-HUT were 0.91, 0.84, and 0.86, respectively.

Fig. 5.

Multifactor analysis of (a) individual factor map and (b) correlation map of emotional terms for the coffee samples under the three test condition. CLT, HUT and N-HUT mean central location test, home-use test, and no-contact home-use test, respectively. AM, AD, RM and RD represent Arabica medium roasting, Arabica dark roasting, Robusta medium roasting, and Robusta dark roasting, respectively.

4. Discussion

4.1. Comparison of resources for conducting CLT, HUT, and N-HUT

The evaluation no-show rate was 3.7% in CLT. The participants reserved their visiting time at least two days prior to the testing date; however, there was a chance of an unexpected personal reason leading to the inability to participate in the sensory testing at the facility. All participants in the HUT finished the evaluation because they visited a certain location to pick up the sample. The participants were strongly motivated to finish the test and get compensation since they had already invested their time and effort to pick up the sample package before the evaluation. The ratio to complete the evaluation in the N-HUT condition was 92.5%, which was the lowest among the three test conditions. This is a drawback of the N-HUT because participants did not require any effort prior to the sensory evaluation. The participants could simply drop out even after obtaining the samples. Using N-HUT, the researchers may therefore bear the loss of participants and samples but can potentially obtain testing results from participants who live far from the testing site. Notably, we conducted CLT with in-house participants. When the random participants who were not institute employees were recruited and asked to visit the testing site, the ratio of participants who finished the test might be similar to that of N-HUT. Generally, researchers recruited 10–20% more participants when conducting CLT, considering the potential of dropping out. Similar additional recruiting needs to be considered when conducting N-HUT. One advantage of the N-HUT is that the male-to-female ratio was more balanced than the other test conditions. Since the samples were delivered through a parcel delivery service, male participants working during the daytime could participate more easily in the N-HUT.

Regarding labor hours and cost, the CLT required more manpower than the two HUTs because researchers needed to prepare hot water and serve the sample monadically. Four researchers were served samples on the test day with one supervisor. On the other hand, samples were packaged in a bag for the HUT and a paper box for the N-HUT. The total labor hours for HUT sample preparation were similar to the CLT. In the case of N-HUT, additional costs were incurred due to packaging sample boxes and shipping. Only two more hours were used in sample preparation and/or delivery for N-HUT (Table 1) since we dropped the parcels after packaging them at the designated place at the KFRI for picked up. The postal workers drove a truck to our institute and picked them up for delivery. As the participants at home conducted sample evaluations, researchers did not have to serve the samples; however, the researchers were still required to monitor the response rate and handle a few calls for an inquiry. Overall, conducting HUT or N-HUT can be more economical than conducting CLT when sample preparation is necessary. However, accurate sample preparation during HUT and N-HUT is doubtful, although instructions are supplied, because it is impossible to confirm how the participant prepares the samples for testing. Between HUT and N-HUT, N-HUT incurred slightly higher costs owing to the sample loss resulting from participants that dropped out and delivery cost; however, testing a wider range of participants was possible.

The compensations for the participants in the three test conditions differed in this study. Compensation for the CLT of in-house participants was the standard monetary compensation at the KFRI (approximately USD 7). The participants in the HUT and N-HUT tests received 100% and 150% higher monetary compensation, respectively, compared to the CLT. Although the compensation paid for the N-HUT was higher, the drop rate was also higher than that of the CLT and HUT. Despite the higher compensation, some panelists seemed to have either lost interest in participating or simply forgot to conduct the test due to the lower motivation, since there was no effort required to participate. Therefore, the lack of motivation to participate after obtaining a sample package was revealed as a factor to consider when conducting N-HUT.

4.2. Sample, test condition, and consumer effect on acceptance

The acceptability and purchase intent ratings obtained under the different test conditions were significant (P < 0.05) (Table 2), and the overall ratings in N-HUT were the highest compared to that of the other conditions (Fig. 1). In the case of N-HUT, the entire test procedure was similar to online shopping. The participants might have felt as if they had received food items ordered from the online shopping mall to enjoy rather than having felt a burden to conduct a testing task. Additionally, the participants likely felt less obligated to conduct the test because they had no face-to-face connections with the researchers, which resulted in a reduced response rate. These results are consistent with slightly higher response ratings under natural test conditions than at a sensory facility (Bangcuyo et al., 2015, Delarue et al., 2019). In contrast, in the case of the CLT or HUT, participants had to visit the booth or pick up the samples in person, respectively. The participants in the CLT and HUT groups might have an analytical attitude when they enter the booth or pick up the samples (Pound et al., 2000). Participants in the N-HUT group may be more likely to enjoy the samples like regular coffee drinking, whereas participants in the CLT and HUT groups might have a more analytical attitude toward coffee evaluation in general.

Regarding sample discrimination (Table 3), the overall liking showed different patterns depending on the test methods. The CLT group preferred mildly roasted coffee, whereas no significant differences were observed among the samples in the HUT group. In contrast to the CLT, Robusta or dark-roasted coffee was preferred in the N-HUT. Additionally, no sample discrimination was observed between HUT and N-HUT regarding color, aroma, mouthfeel liking, and purchase intent. Although the samples had different sensory characteristics (Table 3), the liking ratings seemed dependent on the participants. Previous studies have shown that acceptance ratings were product-dependent between CLT and HUT. High ratings using HUT have been reported in various food items, such as salted crackers, sparkling waters, chicken, juice, chocolate bars, and farmed cod fish (Boutrolle et al., 2007, De Wijk et al., 2019, Karin et al., 2015, Schouteten et al., 2021, Sveinsdottir et al., 2010). Similar ratings between CLT and HUT have also been reported for frozen pizza, chocolate, nuts, and wines (Ratanatriwong et al., 2006, Schouteten et al., 2021, Sinesio et al., 2021). Different consumer clusters for chocolate chip cookies were also observed even though the same consumers conducted three different remote test conditions (Tapia & Lee, 2022).

The optimal choice among the CLT, HUT, and N-HUT approaches that produce the most accurate acceptance ratings cannot yet be determined. The selection of a testing approach may depend on various factors such as self-sample preparation carried out for HUT, variation in sample, the product being tested, consumer perception variation, differences in testing conditions that could impact consumer perceptions, the cost associated with each testing approach, and the use of between-subject test designs. Researchers have considered several factors when choosing test methods to evaluate acceptance ratings successfully. Based on our study, an N-HUT approach using online recruiting and delivery service can expand the pool size of participants, removing a limitation of CLTs where subject recruitment is bound to the geographic area of the sensory laboratory.

4.3. Sensory profiling

The results of the sensory profile using RATA across three test conditions were similar (Table 4). The PCA results of the three groups showed very similar patterns in the attributes and sample loading in the plot (Fig. 2). We found that Dim 1 was associated with the fruity and grain attributes, which are typical characteristics of coffee, and Dim 2 was related to the degree of coffee roasting, such as the roasting temperature. Previous studies observed similar sample identifications through RATA profiling in several food products, including cold cuts, yogurts, potato chips, and juice (Niimi et al., 2022, Schouteten et al., 2019) between the CLT and HUT. The products used in these studies required little or no sample preparation before evaluation. In our study, several factors might lead to differences in sample preparation among the participants, potentially resulting in variations in the water temperature, amount of water, brewing speed, and waiting time, although the participants were asked to follow the instruction in two HUTs. Although all instructions regarding the brewing procedures depended on the participants, significant changes in the description of the samples were not observed. Some participants might not have followed the instructions; however, overall, the sensory profiling results obtained using RATA in HUT and N-HUT were similar to those obtained in CLT. An RV coefficient of 0.70 or higher indicates valid similarity between biplots (Hopfer & Heymann, 2013). The RV values between conditions that were determined based on MFA were 0.93 or higher in our study. This supports the finding that there was no significant effect on identifying sensory attributes among the CLT, HUT, and N-HUT methods. The effect of the testing conditions seemed to be limited to sensory profiles determined using RATA. Although the RV coefficient was not calculated, Niimi et al. (2022) showed similar sensory profiling results in cold cuts between CLT and HUT.

4.4. Emotion response

As a result of the PCA (Fig. 4), the RATA terms were similar among the three test conditions, but the number of emotion terms identified by the participants was higher in the CLT. It has been suggested that the two HUT conditions are more natural atmospheres than the CLT in coffee drinking. The CLT may induce a more analytical mindset than the HUT conditions, as the participants may feel more engaged in a scientific experiment at the sensory booths (Boutrolle et al., 2005, Schouteten et al., 2019). Therefore, participants in the CLT may feel motivated to identify a higher number of emotion terms by focusing on the samples.

Another difference between the conditions was that the negative emotions of 'worried' and 'bored' were used to discriminate among the samples in CLT, whereas the coffee-evoked positive emotion of 'energetic' differentiated the samples in HUT and N-HUT. Participants in the three test condition groups brewed coffee by themselves. However, brewing coffee at a sensory booth and home-generated different feelings. In the sensory booth, the act of brewing coffee by participants was only a mandatory action for evaluation. The booth differed from the regular coffee drinking environment; participants may have felt that the three-minute waiting time for coffee brewing was boring. This may have generated negative emotions toward the samples. On the other hand, the home is a comfortable place for drinking coffee, even if the participants were forced to brew coffee for the evaluation. Additionally, at home, the participants could choose when to drink coffee. Danner et al. (2016) examined the influence of three different environments (laboratory, home, and restaurant) on the emotion profile of wines. This study showed that when participants evaluated wine samples, they associated positive emotions (e.g., enthusiastic, happy, and adventurous) more intensively in the order of restaurant, home, and laboratory. Therefore, at home, the participants may be more likely to associate positive emotions such as ‘energetic’ with the evaluated samples. The higher mean intensity ratings for ‘energetic’ in the HUTs than in the CLT (CLT = 0.42, HUT = 0.47, N-HUT = 0.75) supports this hypothesis. Time of consumption may also play a key role in the emotions associated with the samples. Consumers of HUT can be expected to complete the sensory test at a more appropriate moment, whereas participants in CLT have to evaluate samples at fixed times (Schouteten et al., 2019). This pressure may have caused negative emotions.

Differences were also found regarding the terms ‘peaceful’ and ‘active’ between HUT and N-HUT. However, in N-HUT, the P-value for 'peaceful' (P = 0.056) marginally exceeded the significance limit of 0.05, and the order of scores (RM, RD, AD, and AM) of the samples was the same as in HUT. 'Active' (HUT, P < 0.001; N-HUT, P = 0.207) seemed to differ slightly in association with the attitude toward evaluating the sample depending on whether the participants personally received the sample or received it through a delivery service.

There were also some differences in the number of emotion terms associated with the samples and their intensities. However, all RV values between conditions were 0.83 or higher based on the results of the MFA, which indicates that the emotion responses were similar across the testing conditions. Additionally, based on sample loading on each PCA plot for each test condition, the participants had similar emotion responses to each sample. Therefore, the differences in emotion responses associated with testing conditions seemed to be limited, although slight differences were observed.

4.5. Limitations

The availability of rapid and reliable delivery services is a prerequisite for conducting an N-HUT nationwide. In Korea, delivery service systems are reliable and affordable. Two-day shipping is standard in most regions of the country. However, this delivery system is uncommon in many countries, especially underdeveloped countries. Additionally, several smartphone applications for local residences were required to recruit participants without contact in a short time. All procedures, including recruiting participants, obtaining a consent form, receiving compensation, and collecting results, were conducted using a smartphone. Therefore, conducting N-HUT may be impossible for many underdeveloped countries considering the two key factors which are prerequisites to the success of the N-HUT.

The types of samples used in N-HUT tests can also be limited to N-HUT. Delivering refrigerated or frozen samples is more complicated than room-temperature delivery when sending samples to participants using a parcel delivery service. In many countries, refrigerated and frozen foods cannot be purchased from online shopping malls or delivered through parcel delivery services. Only a few countries can deliver refrigerated or frozen samples, making it possible to obtain refrigerated or frozen samples for the N-HUT method. However, for evaluating room-temperature foods and consumer supplies, N-HUT can be an effective way of conducting testing in realistic consumption conditions.

5. Conclusions

In this study, three test conditions (CLT, HUT, and N-HUT) were compared for evaluating coffee samples. The RV coefficients in acceptability, sensory profiling, and emotion responses were high enough to conclude that the three test conditions produced similar results. In sensory profiling, the participants prepared hot coffee brewed using coffee bag, and the sensory profiling results across CLT, HUT, and N-HUT were comparable, with strong RV coefficients. Emotion response comparison also produced a strong RV coefficient. In comparison with CLT, HUT, N-HUT showed strong potential for collecting test results from a wider range of consumers without third-party survey firms located in regional areas. However, there are prerequisites for conducting N-HUT without face-to-face contact, such as the existence of parcel delivery systems, the use of smartphones, and internet access. Our study shows that modern logistics systems and information technology make it possible to conduct sensory evaluations nationwide without face-to-face contact and sensory testing facility.

Funding

This research was supported by the Main Research Program (Grant number: E0211300) of the Korea Food Research Institute (KFRI) funded by the Ministry of Science and ICT, Republic of Korea.

CRediT authorship contribution statement

Seyeong Park: Conceptualization, Methodology, Formal analysis, Investigation, Visualization, Writing – original draft. JeongAe Heo: Conceptualization, Methodology, Investigation. Jungmin Oh: Investigation, Resources, Project administration. Seo-Jin Chung: Writing – review & editing, Supervision. Han Sub Kwak: Conceptualization, Writing – original draft, Writing – review & editing, Supervision, Project administration, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

Data will be made available on request.