Abstract
Affect plays an important role in the consumer decision-making process and there is growing interest in the development of new technologies and computational approaches that can interpret and recognize the affects of consumers, with benefits for marketing described in relation to both academia and industry. From an interdisciplinary perspective, this paper aims to review past studies focused on electroencephalography (EEG)-based affective computing (AC) in marketing, which provides a promising avenue for studying the mechanisms underlying affective states and developing recognition computational models to predict the psychological responses of customers. This review offers an introduction to EEG technology and an overview of EEG-based AC; provides a snapshot of the current state of the literature. It briefly presents the themes, challenges, and trends in studies of affect evaluation, recognition, and classification; and further proposes potential guidelines for researchers and marketers.
Keywords: electroencephalography, affective computing, marketing, neural affective mechanisms, classification and recognition
Introduction
Affective computing (AC) is a continuously growing interdisciplinary research field spanning psychology, computer science, cognitive science, neuroscience, and more (Tao and Tan, 2005). In Picard’s landmark book, AC is defined as “computing that relates to, arises from, or deliberately influences emotion or other affective phenomena” (Picard, 1997). It mainly focuses on how technology can inform and deepen understanding of human affect and how systems can be designed to estimate the affective state using computational models from behavioral and physiological signals (Calvo and D’Mello, 2010).
Affect plays an important role in human cognition, specifically in perception processes, rational decision-making, human communication, and human intelligence (Ammar et al., 2010; Singh et al., 2013). In marketing, understanding and recognizing the affective states of users (or customers) has become a vital theme, which can reveal users’ true preferences and improve and assist in the purchasing process (Malär et al., 2011; Garrido-Morgado et al., 2015).
Conventional assessment methods of AC in marketing depend mostly on personal evaluations, such as surveys, focus groups, and interviews (Ariely and Berns, 2010; Yadava et al., 2017; Lin et al., 2018). However, customers may not express their true opinions because of social desirability bias (Paulhus, 2002; Vecchiato et al., 2011a). They may not say exactly how they are feeling but rather, how they feel others would reply (Calvert and Brammer, 2012). These post hoc analysis tools are also influenced by individuals’ mental states or environments at the time of self-reporting (Nilashi et al., 2020). Due to the limitations of traditional AC techniques, marketers and researchers try to supplement these shortcomings and seek alternative or complementary tools. Neuroscientific tools based on electrophysiological and neuroimaging techniques provide one such alternative, as well as a way to dig deeper into understanding the complex evaluation process and the dynamics of the affective state by directly accessing the physiological signals and fundamental cerebral structure from which an affective state occurs.
Considering its low cost and high temporal resolution (milliseconds), electroencephalography (EEG) has become common and is extensively used in the marketing industry. Furthermore, variations in EEG signals cannot be voluntarily controlled and therefore are a better objective indicator of affect (Singh et al., 2013). EEG-based AC provides a promising avenue for studying the mechanisms underlying affective states and developing recognition computational models to predict the psychological responses of customers. It can, therefore, be widely used to boost sales, advertising, pricing, package design, marketing campaigns, and so forth (Calvo and D’Mello, 2010).
In this review, we focus on EEG-based AC in marketing. First, we stress the need to incorporate the various features of neural signals that contribute to consumers’ affective states and evaluation processes beyond what traditional marketing measures already provide. Second, we state that marketing studies should adopt the methodologies and algorithms used in data processing and prediction modeling that are mature in other fields such as computer science and engineering. We examine AC literature in marketing on the general features extracted from EEG recordings and conclude with a general discussion of the challenges faced by this field, providing several recommended guidelines for the road ahead.
Consumers’ Neural Affective Mechanisms
It is expected that a deeper understanding of how the human brain works and responds will make marketing more effective (Kuan et al., 2014; Nagyová et al., 2014; Caratù et al., 2018). Researchers and marketers have attempted to reveal the neural correlates and neural affective mechanisms that affect decision making in customers and consumer behavior in economic processes.
Event-Related Potentials
As an extremely useful tool for investigating higher-level brain functions, event-related potentials (ERPs) have been widely used to explore the neural mechanisms underlying affect processing in marketing. For example, Ma et al. (2007) found that brand extension preferences were modulated by negative emotions. These underlying mechanisms were indicated by the amplitudes of N270. This research presents a method, which is automatic, objective, and non-verbal, for the use of ERP components as markers for emotional preferences. Jones et al. (2012) shifted their attention toward pricing and discount-related consumer behavior. The enhanced P3 in high math anxiety individuals indicated greater reliance on the emotional and motivational factors involved in the buying process. Social interaction in marketing is another important theme. In a study by Pozharliev et al. (2015), the late positive potential (LPP) was enlarged during shopping for luxury products. It was especially prominent in the presence of another person during buying decisions, suggesting that social interaction magnifies the emotional effect of the brand category. Furthermore, Chen et al. (2010) found that N500 was evoked by emotional conflicts when making counter conformity choices in buying books online.
By taking full advantage of the high temporal resolution of ERPs, a group of scientists conducted multistage experiments and multicomponent detections to decompose the time course of neural cognitive and affective activities. Handy et al. (2010) focused on the emotional appraisal of unfamiliar commercial logos. Using an ERP examination, they found that the judgment of logos can be divided into at least two stages: an initial formation of ones’ impression at the sensory-perceptual level and evaluative analysis at the cognitive-processing level. Jin et al. (2018) demonstrated that eco-labeling induced positive emotions in consumers and reduced cognitive conflict, as reflected by decreased P2 and N2 amplitudes, respectively. In another study, Shang et al. (2020) studied online shopping and focused on the influence of webpage layouts on consumer experience. They demonstrated that a low-order online shopping webpage facilitated consumers’ instant purchase decisions as indexed by increased P2 (attention engagement) and LPP (emotional self-control) amplitudes. Separating affective activities from cognitive activities and capturing independent processes within each stage is helpful to provide a deeper and more nuanced understanding of consumers’ neural affective mechanisms.
EEG Time-Frequency Components
Because EEG signals comprise abundant time-frequency information, multiple characteristic parameters, such as hemispheric asymmetry, event-related desynchronization/synchronization (ERD/ERS), and power spectral density (PSD), are closely related to affective states.
Concerned with the frontal EEG asymmetry, Vecchiato et al. (2010) revealed that the cerebral activity of the theta band on the left frontal hemisphere increased when television commercials were judged to be pleasant. As a follow-up to this, Vecchiato et al. (2014) further highlighted that the index of cortical hemispheric asymmetry, which was also a valid predictor of preferences, was significantly correlated with participants’ pleasantness ratings. Furthermore, the frontal alpha band, as reflected in cerebral hemispheric alpha asymmetry, has been widely used to represent affective responses to advertisements (Ohme et al., 2010; Venkatraman et al., 2015). When two subjects were viewing two versions of the almost same skincare product advertisement (differing only by the inclusion of a very short clip concerning a gesture by a female model), Ohme et al. (2009) found out that each version elicited significantly different emotional impacts, indexed by differences in the frontal alpha activity between the left and the right hemispheres. Similarly, Reeves et al. (1989) reported a significant interaction between hemisphere activity and the emotional content (positive and negative) of television advertisements for frontal alpha power but no interaction for occipital alpha power. It would be more beneficial for future research to specify the role of mediation and moderation that frontal alpha asymmetry plays in relation to advertisements and affective states (Palmiero and Piccardi, 2017; Allen et al., 2018).
With regard to the ERD/ERS indices, Vecchiato et al. (2011b) demonstrated that desynchronization of the left alpha band is positively related to judgments of high pleasantness. Other findings also supported the evidence that synchronization and desynchronization changes were the result of affective arousal and valence (Aftanas et al., 2001; Costa et al., 2006). Moreover, ERD/ERS is a useful tool for understanding variation in oscillatory activity occurring during the decision-making process related to subjective preferences. Khushaba et al. (2013) explored brain activity while subjects were choosing crackers using EEG and eye tracking data. They found clear phase synchronization between the left and right frontal and occipital regions during preference decision making for the different cracker characteristics, which can be used to predict consumers’ future choices and develop effective marketing strategies.
By performing power spectra and PSD analyses, Horska et al. (2016) and Berčík et al. (2016a) studied the emotions of consumers tasting different kinds of wine and used their findings to improve the practical selling strategies. Subsequently, Berčík et al. (2016b) conducted interdisciplinary research on the impact of illumination on the emotional state (valence) of customers in a food store by calculating and comparing alpha and beta spectral power. Lee (2016) explored the emotional mechanism of empathy indexed by the theta-band power spectra of the anterior cingulate cortex (ACC) for customer equity and willingness to pay. Moreover, Vecchiato et al. (2011b) showed that an increase in PSD in the left frontal lobe is negatively related to the degree of perceived pleasantness. These studies indicate that power spectra and PSD are reliable indicators in identifying emotional responses to marketing.
Taken together, the studies discussed above provide evidence that it is helpful to use EEG features and properties to understand neural affective mechanisms, which enables us to objectively reveal hidden consumer reactions and identify critical success factors in the process of buying and consuming certain products (Kuan et al., 2014; Telpaz et al., 2015). We believe that brain wave-based methodologies would further enrich marketing research and help marketers to go beyond traditional marketing paradigms.
Classification and Recognition of Affective States
Feature extraction is a critical step for obtaining an accurate classification result. It is beneficial for simplifying the models and the amount of resources, reducing the cost of data processing, improving data visualization, and avoiding overfitting (Übeyli, 2008; Al-Fahoum et al., 2014). Various methods have been used to extract the neural features from EEG signals. Among these methods are time frequency distributions (appropriate for great continuous segments), fast Fourier transform (suitable for narrowband signal and stationary signal), wavelet transform (good tool for sudden and transient signal changes), auto regressive methods (which are advantageous for short data segments), and so on (Tao and Tan, 2005; Singh et al., 2013; Al-Fahoum et al., 2014; Yadava et al., 2017). Researchers should make clear the signal types and apply the optimum method.
Once the features are extracted, the features will be fed into an affect classification algorithm. Many types of classifiers are commonly used including: support vector machines (SVMs), relevance vector machines (RVMs), logistic model trees (LMTs), Fisher linear discriminant analysis (FLDA), the k-nearest neighbors (KNN), hidden Markov models (HMMs), artificial neural networks (ANNs), artificial bee colonies (ABCs), random forests (RFs), and deep neural networks (DNNs). In the following paragraphs, we present the various types of affective classifiers used in marketing scenarios.
Many studies have used machine learning algorithms to assess the impact of advertising. Friedman et al. (2015) proposed an EEG data-driven approach to measure customers’ emotional valence when processing commercials. Their results indicated that hemispheric asymmetry was a good marker and the LMT algorithm (81.2%) provided better classification rates than the SVM algorithm (77.3%). Wei et al. (2018) explored a new method using low-cost EEG headbands to assess the influence of advertisements on purchasing. The EEG features that were closely related to emotions were gathered into different groups and the SVM method was applied to assess the ability of features to predict possible purchases, achieving an accuracy of 77.3%. Yang et al. (2015) developed an approach to evaluate the temporal patterns of EEG data (PSD) and extract affective indices such as happiness and surprise for TV commercial evaluation. FLDA was used to predict which parts of an advertisement could elicit positive emotional responses in customers. Similarly, Guixeres et al. (2017) attempted to forecast the effectiveness of advertisements during the Super Bowl sports event in the United States, based on EEG signals, including biometric responses such as the z-score of the global field power (GFP), hemispheric asymmetry (pleasantness index), and the relative number of peaks in the beta and theta bands (interest index). The results showed that the ANN was able to precisely classify and estimate the effect of each advertisement on the Internet via biometrics (82.9% of average accuracy). Marketers could consider the proposed approach at the technical design stages of advertising content. From a multimodal perspective, Gauba et al. (2017) developed a notable rating forecast framework for advertisement clips by using both EEG signals and sentiment analysis of online users’ comments from YouTube. The prediction was carried out using the RF regression and was later fused with the sentiment score to improve the overall prediction.
Some scholars pay special attention to the music or jingles of advertisements. Lin et al. (2014) focused on the emotion classification problem when listening to music and used the SVM classifier, generating accuracies of 82.5 and 79.1% for valence and arousal classification, respectively. Gupta and Falk (2016) used EEG graph-theoretic features to classify emotional states while watching music clips with the aid of two classifiers, the SVM and RVM, and the approach achieved a significant improvement in the classification accuracy (The percentage increase in classification performance ranged from 3 to 9%). More recently, Avinash et al. (2018) developed a very accurate tool for understanding consumers’ emotional responses to advertisement jingles. The KNN algorithm was used to classify positive and negative emotions based on theta power signals, achieving an accuracy of 100%.
Other studies have investigated consumers’ purchasing preferences. Chew et al. (2016) studied the aesthetic emotional responses in industrial design and buying decisions using EEG signals for virtual 3D shapes with motion. A classification accuracy of up to 80% was attained using the KNN with the alpha, theta, and delta rhythms as the features taken from frontal electrodes to classify two classes, like and dislike. Lobato and Garza (2017) developed a classification algorithm using neural networks and EEG signals to measure the affective states of “do like” and “don’t like” during buying processes. Yadava et al. (2017) proposed a predictive model using the HMM classifier to analyze EEG signals to understand consumers’ affective states and purchase preferences toward e-commerce goods, and the model achieved an accuracy of 70.33%. They also tested numerous other models, such as the SVM, RF, and KNN. Interestingly, Kumar et al. (2019) creatively put forward a multimodal rating prediction method by fusing the affective ratings from e-commerce websites and EEG data. The ABC and RF models were applied to optimize and compute the ratings from varied data sources, and the results showed that the combined method could achieve a lower Root Mean Square Error (RMSE) in rating prediction compared to a unimodal method. More recently, Aldayel et al. (2020) adopted a deep learning approach to assess consumer preferences (pleasant or unpleasant) by extracting the PSD and valence features. They built four different classifiers, namely, the DNN, RF, SVM, and KNN, which attained accuracies of 94, 92, 62, and 88%, respectively.
In summary, due to significant differences between the experimental design and the paradigm used in these studies, it is difficult to make direct comparisons of the classification accuracies achieved. Furthermore, it seems that there is no particular feature extractor or affective classifier that appears to be the single best choice for all marketing scenarios. In most situations, one should consider as many algorithms as possible from the studies mentioned above and then compare the results with a range of features and algorithms before choosing the one with the best performance for the given marketing application.
Discussion
In this review, we summarize previous studies analyzing EEG signals as biological markers in affective mechanism and recognition in the marketing area (as shown in Table 1). The majority of the studies, especially those using machine learning techniques and algorithms, have been published in the last 10 years. This review provides new directions regarding neuromarketing data analyses and fosters cooperation among scholars from miscellaneous disciplines, such as information science, neuroscience, marketing, and psychology. Although there has been a recent increase in the number of EEG-based AC studies in marketing with no signs of slowing down, theoretical and operational challenges must be settled before moving forward.
TABLE 1.
Reference | Journal | Marketing substance | Affective states | EEG features | Method (classification accuracy) |
Reeves et al., 1989 | Human Communication Research | TV commercials | Valence | Hemispheric differences (alpha) | ANOVA |
Ma et al., 2007 | Neuroreport | Brand | Conflict | ERPs (N270) | ANOVA |
Ohme et al., 2009 | Journal of Neuroscience, Psychology, and Economics | TV commercials | Valence | Hemispheric differences (alpha) | t tests and Pearson’s linear correlation |
Chen et al., 2010 | Biological Psychology | E-commerce products | Valence | ERPs (N500) | ANOVA |
Handy et al., 2010 | Journal of Cognitive Neuroscience | Commercial logos | Liking | ERPs (P1, N2) | ANOVA |
Ohme et al., 2010 | Journal of Economic Psychology | TV Commercials | Valence | Hemispheric differences (alpha) | ANOVA and post hoc tests |
Vecchiato et al., 2010 | Brain Topography | TV commercials | Pleasantness | GFP (theta, beta) | ANOVA |
Vecchiato et al., 2011b | Medical,Biological Engineering and Computing | TV commercials | Pleasantness | PSD, ERD (alpha, theta) | t-test |
Jones et al., 2012 | Biological Psychology | Pricing | Anxiety | ERPs (FN400, P3, LPC) | ANOVA |
Guo and Elgendi, 2013 | Journal of Advanced Management Science | Recommender system for e-commerce | Valence | Spectral power (alpha, beta) | Pearson’s linear correlation |
Khushaba et al., 2013 | Expert Systems with Applications | Food property | Liking | PSD, ERS (delta, theta, alpha, beta, gamma) | Phase locking value |
Lin et al., 2014 | Frontiers in Neuroscience | Music | Valence, arousal | PSD, DLAT, DCAU, MESH (delta, theta, alpha, beta, gamma) | SVM (valence: 82.5%; arousal: 79.1%) |
Kuan et al., 2014 | Journal of Management Information Systems | Group-buying information | Valence, liking | Hemispheric differences (alpha) | ANOVA |
Vecchiato et al., 2014 | Cognitive Computation | TV commercials | Valence, arousal | PSD (alpha), IAF (alpha) | t test |
Friedman et al., 2015 | International Conference on Affective Computing and Intelligent Interaction | TV commercials | Valence | Spectral power hemispheric differences (delta, theta, alpha, low beta, high beta) | MANOVA, SVM (77.3%), LMT (81.2%) |
Pozharliev et al., 2015 | Journal of Marketing Research | Luxury goods | Arousal | ERPs (P2, P3, LPP) | ANOVA |
Telpaz et al., 2015 | Journal of Marketing Research | Consumer goods | Liking | ERPs (N200), spectral power (theta) | t tests and spearman correlation |
Venkatraman et al., 2015 | Journal of Marketing Research | TV commercials | Valence, arousal | Occipital activity and frontal asymmetry (alpha) | SUR regression |
Yang et al., 2015 | Journal of Physiological Anthropology | TV commercials | Happiness, surprise | PSD (delta, theta, alpha, low beta, high beta, gamma) | ANOVA, FLDA (happiness: 88.6%; surprise: 87.5%) |
Berčík et al., 2016a | Periodica Polytechnica Social and Management Sciences | Music preferences | Pleasantness | Spectral power (alpha, beta) | Descriptive statistics |
Berčík et al., 2016b | Appetite | Store illumination | Valence, arousal, dominance | Spectral power (alpha, beta) | Non-parametric Wilcoxon signed rank test |
Chew et al., 2016 | Cognitive Neurodynamics | Industrial design | Liking | ERS/ERD (alpha, theta, delta) | SVM (79%), KNN (80%) |
Gupta and Falk, 2016 | Neurocomputing | Music videos | Valence, arousal, dominance, liking | EEG graph-theoretic features | SVM (valence: 64%; arousal: 64%; dominance: 59%; liking: 64%), RVM (valence: 65%; arousal: 68%; dominance: 63%; liking: 67%) |
Horska et al., 2016 | Agricultural Economics | Consumer preferences | Valence | Wave fluctuating tendency | Kruskal–Wallis test |
Lee, 2016 | Journal of Business Research | Willingness to pay | Valence | Spectral power (theta) | sLORETA |
Gauba et al., 2017 | Neural Networks | TV commercials | Valence | Statistical mean of band oscillations of each electrode | RF (68%) |
Guixeres et al., 2017 | Frontiers in Psychology | Online commercials | Liking | GFP (delta, theta, alpha, beta, Gamma) | ANN (82.9%) |
Lobato and Garza, 2017 | IEEE Latin America Transactions | Purchasing behaviors | Liking | Hemispheric differences (alpha) | ANN (76%) |
Yadava et al., 2017 | Multimedia Tools and Applications | E-commerce products | Liking | Band oscillations (delta, theta, alpha, beta, Gamma) | HMM (70.3%) |
Avinash et al., 2018 | Procedia Computer Science | Advertisement jingles | Valence | Frontal asymmetry (theta) | KNN (100%), FLDA (90%) |
Jin et al., 2018 | Frontiers in Human Neuroscience | Eco-labeled products | Valence | ERPs (P2, N2) | ANOVA |
Wei et al., 2018 | Frontiers in Neuroscience | Commercials | Valence | Wavelength, signal quality (delta, theta, low alpha, high alpha, low beta, high beta, low gamma, high gamma) | SVM (77.3%) |
Kumar et al., 2019 | Information Fusion | E-commerce products | Valence | Spectral power (delta, theta, alpha, beta, Gamma) | RF (48%), ABC + RF (72%) |
Aldayel et al., 2020 | Applied Sciences | Purchasing behaviors | Pleasantness | PSD (theta, alpha, beta, gamma) | DNN (94%), RF (92%), SVM (62%), KNN (88%) |
Shang et al., 2020 | Psychology Research and Behavior Management | Webpage layout | Valence | ERPs (P2, LPP) | ANOVA |
ANOVA, analysis of variance; ERPs, event-related potentials; GFP, global field power; PSD, power spectral density; SVM, support vector machine; IAF, individual alpha frequency; MANOVA, a multivariate analysis of variance; LMT, logistic model tree; FLDA, Fisher linear discriminant analysis; ERS/ERD, event-related synchronization/desynchronization; KNN, K-nearest neighbors; RVM, relevance vector machine; sLORETA, standardized low-resolution electromagnetic tomography; RF, random forest; ANN, artificial neural network; HMM, hidden Markov model; ABC, artificial bee colony; DNN, deep neural network.
First, it would be helpful to pay more attention to multiclass affective classification. As this review shows, most of the previous studies are based on dimensional emotion theory, typically concerning the dimensions of arousal, valence, liking, and dominance. The state of the art usually relies on the affective polarity of its components (e.g., positive or negative) and proposes approaches that mostly focus on binary affective classification. However, to study the affective states of consumers, it would be more interesting to go deeper into the classification and detect subtle affective changes in marketing. Furthermore, marketing scenarios may induce multiple emotions in customers. The phenomena of coexistence should be considered in affective tagging. We recommend that future studies should focus on two issues to develop a more accurate affective definition and conduct better forecasting: (1) they should aim for a deeper understanding of consumer ambivalence, characterized by the co-occurrence of positive and negative emotions (Kreibig and Gross, 2017; Hu et al., 2019); and (2) consider emotion dyads, namely, a mix of primary emotions, raised by Plutchik (1980).
The multidimensional and multimodal feature fusion can obtain better recognition performance. When studying EEG-affect relationships, EEG-based AC studies assume that EEG signals can sufficiently depict and predict human affective states. However, this hypothesis cannot always be assumed to be true because the relationship between physiological responses and psychological states could be very complex (Cacioppo and Tassinary, 1990; Hu et al., 2019). To achieve precise prediction and improved generalization, first, we suggest decreasing the abundant number of features from EEG signals and further perform feature selection and fusion. The most widely used features include differential symmetry, GFP, PSD, and ERPs. It might be the case that a fusion of features derived from different EEG signal types will lead to better recognition performance (Hakim and Levy, 2019). It is worth noting that future studies should be more cautious regarding the reliability and validity of “one-to-one” relationships (one affective state is associated with one and only one EEG feature) (Bridwell et al., 2018; Hu et al., 2019). Second, recent studies have revealed that multimodal frameworks can effectively increase emotion recognition accuracy and robustness compared to unimodal frameworks (Guixeres et al., 2017; Avinash et al., 2018; Kumar et al., 2019). The advantage of multiple modalities (for example, vision, sound, or smell) helps to increase the validity and usability since the weaknesses of one modality are offset by the strengths of another. Future studies may derive features from modalities other than EEG while collecting and analyzing data by using machine learning, natural language processing, and automatic speech recognition technology, evolving from unimodal analyses to multimodal fusion.
The use of portable wireless EEG devices and virtual reality (VR) technology can alleviate the lack of ecological validity in marketing studies. For EEG hardware devices, the whole-brain coverage, the time-consuming preparation procedure, and the prohibitive cost of a professional headset with wet electrodes make it impractical and difficult to transfer the laboratory to real-world applications in marketing. A group of recent studies has confirmed the feasibility of using consumer-level EEG headsets for AC with promising results. For example, the widely used wireless EPOC headset (e.g., Kuan et al., 2014; Lin et al., 2014; Friedman et al., 2015; Yang et al., 2015; Gauba et al., 2017; Yadava et al., 2017; Kumar et al., 2019), due to its light weight, low price, and ease of use, shows promise. Studies on the EPOC headset seem to agree that it can be applied to acquire reliable EEG signals in marketing, but researchers should pay attention to its relatively low signal-to-noise ratio and poor signal stability (Friedman et al., 2015). We suggest that researchers evaluate the performance of consumer-level devices using the standard testing procedures proposed by Hu et al. (2019). In addition, to bridge the gap between the laboratory environment and real market scenarios, the use of VR is an important trend that can effectively enhance the experience of immersive sensation. It enables consumers to get a direct, intuitive, and concrete understanding of the appearance, quality, and performance of products (Guo and Elgendi, 2013). Furthermore, VR makes it possible to simulate and assess retail and consumption environments under controlled laboratory conditions (Marín-Morales et al., 2017), allowing the isolation and modification of variables in a cost-effective manner.
Studying interactions among multiple customers is critical for understanding the marketing ecosystem, which consists of interrelated trends that shape consumer behaviors. Most AC studies in marketing have concentrated mainly on a single consumer’s EEG activity and may ignore the socio-affective interaction and processes related to consumer behavior (Hasson et al., 2012). The EEG-based hyperscanning technique [for a recent review, see Liu et al. (2018)] provides a way to explore dynamic brain activities between two or more interactive customers and their underlying neural affective mechanisms. In previous hyperscanning studies, interpersonal neural synchronization (INS) has been verified to be a crucial neural marker for different kinds of social interactions, such as communication (Stephens et al., 2010), collaborative decision making (Montague et al., 2002; Hu et al., 2018), and imitation (Pan et al., 2017). As consumer behavior is inherently social and interactive in nature, EEG-based INS could be used to study the biological mechanism for shared intentionality of consumption, panic buying, collective emotion, and group purchase.
Author Contributions
Both authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The handling editor declared a shared affiliation with TL and GP at the time of review.
Footnotes
Funding. This work was supported by Grant No. 2020M671819 from the China Postdoctoral Science Foundation. This work was also supported by Grant No. 111002-AC2001 from the Major Scientific Project of Zhejiang Lab.
References
- Aftanas L., Varlamov A., Pavlov S., Makhnev V., Reva N. (2001). Event-related synchronization and desynchronization during affective processing: emergence of valence-related time-dependent hemispheric asymmetries in theta and upper alpha band. Int. J. Neurosci. 110 197–219. 10.3109/00207450108986547 [DOI] [PubMed] [Google Scholar]
- Aldayel M., Ykhlef M., Al-Nafjan A. (2020). Deep learning for EEG-based preference classification in neuromarketing. Appl. Sci. 10:1525. 10.3390/app10041525 [DOI] [Google Scholar]
- Al-Fahoum A. S., Al-Fraihat A. A., Oliveira M. S., Grant A., Hinojosa J. A. (2014). Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains. ISRN Neurosci. 2014:730218. 10.1155/2014/730218 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Allen J. J., Keune P. M., Schönenberg M., Nusslock R. (2018). Frontal EEG alpha asymmetry and emotion: from neural underpinnings and methodological considerations to psychopathology and social cognition. Psychophysiology 55:e13028. 10.1111/psyp.13028 [DOI] [PubMed] [Google Scholar]
- Ammar M. B., Neji M., Alimi A. M., Gouardères G. (2010). The affective tutoring system. Expert Syst. Appl. 37 3013–3023. 10.1016/j.eswa.2009.09.031 [DOI] [Google Scholar]
- Ariely D., Berns G. S. (2010). Neuromarketing: the hope and hype of neuroimaging in business. Nat. Rev. Neurosci. 11 284–292. 10.1038/nrn2795 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Avinash T., Dikshant L., Seema S. (2018). Methods of neuromarketing and implication of the frontal theta asymmetry induced due to musical stimulus as choice modeling. Procedia Comput. Sci. 132 55–67. 10.1016/j.procs.2018.05.059 [DOI] [Google Scholar]
- Berčík J., Horská E., Gálová J., Margianti E. S. (2016a). Consumer neuroscience in practice: the impact of store atmosphere on consumer behavior. Periodica Polytechnica Soc.Manag. Sci. 24 96–101. 10.3311/PPso.8715 [DOI] [Google Scholar]
- Berčík J., Horská E., Wang R. W., Chen Y. (2016b). The impact of parameters of store illumination on food shopper response. Appetite 106 101–109. 10.1016/j.appet.2016.04.010 [DOI] [PubMed] [Google Scholar]
- Bridwell D. A., Cavanagh J. F., Collins A. G., Nunez M. D., Srinivasan R., Stober S., et al. (2018). Moving beyond ERP components: a selective review of approaches to integrate EEG and behavior. Front. Hum. Neurosci. 12:106. 10.3389/fnhum.2018.00106 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cacioppo J. T., Tassinary L. G. (1990). Inferring psychological significance from physiological signals. Am. Psychol. 45 16–28. 10.1037/0003-066X.45.1.16 [DOI] [PubMed] [Google Scholar]
- Calvert G. A., Brammer M. J. (2012). Predicting consumer behavior: using novel mind-reading approaches. IEEE Pulse 3 38–41. 10.1109/MPUL.2012.2189167 [DOI] [PubMed] [Google Scholar]
- Calvo R. A., D’Mello S. (2010). Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Trans. Affect. Comput. 1 18–37. 10.1109/T-AFFC.2010.1 [DOI] [Google Scholar]
- Caratù M., Cherubino P., Mattiacci A. (2018). “Application of neuro-marketing techniques to the wine tasting experience,” in Proceedings of the 11th Annual Conference of the EuroMed Academy of Business Valleta, Malta. [Google Scholar]
- Chen M., Ma Q., Li M., Lai H., Wang X., Shu L. (2010). Cognitive and emotional conflicts of counter-conformity choice in purchasing books online: an event-related potentials study. Biol. Psychol. 85 437–445. 10.1016/j.biopsycho.2010.09.006 [DOI] [PubMed] [Google Scholar]
- Chew L. H., Teo J., Mountstephens J. (2016). Aesthetic preference recognition of 3D shapes using EEG. Cogn. Neurodyn. 10 165–173. 10.1007/s11571-015-9363-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Costa T., Rognoni E., Galati D. (2006). EEG phase synchronization during emotional response to positive and negative film stimuli. Neurosci. Lett. 406 159–164. 10.1016/j.neulet.2006.06.039 [DOI] [PubMed] [Google Scholar]
- Friedman D., Shapira S., Jacobson L., Gruberger M. (2015). “A data-driven validation of frontal EEG asymmetry using a consumer device,” in Proceedings of the 2015 International Conference on Affective Computing and Intelligent Interaction, Xi’an, China (New York, NY: IEEE; ) [Google Scholar]
- Garrido-Morgado A., González-Benito O., Campo K., Martos-Partal M. (2015). “Improving sales of private labels in store,” in Proceedings of the Advances in National Brand and Private Label Marketing. Springer Proceedings in Business and Economics, eds Martínez-López F., Gázquez-Abad J., Sethuraman R. (Cham: Springer; ), 3–7. 10.1007/978-3-319-20182-5_1 [DOI] [Google Scholar]
- Gauba H., Kumar P., Roy P. P., Singh P., Dogra D. P., Raman B. (2017). Prediction of advertisement preference by fusing EEG response and sentiment analysis. Neural Netw. 92 77–88. 10.1016/j.neunet.2017.01.013 [DOI] [PubMed] [Google Scholar]
- Guixeres J., Bigné E., AusínAzofra J. M., Alcañiz Raya M., ColomerGranero A., Fuentes Hurtado F., et al. (2017). Consumer neuroscience-based metrics predict recall, liking and viewing rates in online advertising. Front. Psychol. 8:1808. 10.3389/fpsyg.2017.01808 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guo G., Elgendi M. (2013). A new recommender system for 3D e-commerce: an EEG based approach. J. Adv. Manag. Sci. 1 61–65. 10.12720/joams.1.1.61-65 [DOI] [Google Scholar]
- Gupta R., Falk T. H. (2016). Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization. Neurocomputing 174 875–884. 10.1016/j.neucom.2015.09.085 [DOI] [Google Scholar]
- Hakim A., Levy D. J. (2019). A gateway to consumers’ minds: achievements, caveats, and prospects of electroencephalography-based prediction in neuromarketing. Wiley Interdiscip. Rev. Cogn. Sci. 10:e1485. 10.1002/wcs.1485 [DOI] [PubMed] [Google Scholar]
- Handy T. C., Smilek D., Geiger L., Liu C., Schooler J. W. (2010). ERP evidence for rapid hedonic evaluation of logos. J. Cogn. Neurosci. 22 124–138. 10.1162/jocn.2008.21180 [DOI] [PubMed] [Google Scholar]
- Hasson U., Ghazanfar A. A., Galantucci B., Garrod S., Keysers C. (2012). Brain-to-brain coupling: a mechanism for creating and sharing a social world. Trends Cogn. Sci. 16 114–121. 10.1016/j.tics.2011.12.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Horska E., Bercik J., Krasnodebski A., Matysik-Pejas R., Bakayova H. (2016). Innovative approaches to examining consumer preferences when choosing wines. Agric. Econ. Blackwell 62 124–133. 10.17221/290/2015-AGRICECON [DOI] [Google Scholar]
- Hu X., Chen J., Wang F., Zhang D. (2019). Ten challenges for EEG-based affective computing. Brain Sci. Adv. 5 1–20. 10.1177/2096595819896200 [DOI] [Google Scholar]
- Hu Y., Pan Y., Shi X., Cai Q., Li X., Cheng X. (2018). Inter-brain synchrony and cooperation context in interactive decision making. Biol. Psychol. 133 54–62. 10.1016/j.biopsycho.2017.12.005 [DOI] [PubMed] [Google Scholar]
- Jin J., Dou X., Meng L., Yu H. (2018). Environmental-friendly eco-labeling matters: evidences from an ERPs Study. Front. Hum. Neurosci. 12:417. 10.3389/fnhum.2018.00417 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jones W. J., Childers T. L., Jiang Y. (2012). The shopping brain: Math anxiety modulates brain responses to buying decisions. Biol. Psychol. 89 201–213. 10.1016/j.biopsycho.2011.10.011 [DOI] [PubMed] [Google Scholar]
- Khushaba R. N., Wise C., Kodagoda S., Louviere J., Kahn B. E., Townsend C. (2013). Consumer neuroscience: assessing the brain response to marketing stimuli using electroencephalogram (EEG) and eye tracking. Expert Syst. Appl. 40 3803–3812. 10.1016/j.eswa.2012.12.095 [DOI] [Google Scholar]
- Kreibig S. D., Gross J. J. (2017). Understanding mixed emotions: paradigms and measures. Curr. Opin. Behav. Sci. 15 62–71. 10.1016/j.cobeha.2017.05.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kuan K. K., Zhong Y., Chau P. Y. (2014). Informational and normative social influence in group-buying: evidence from self-reported and EEG data. J. Manage. Inform. Syst. 30 151–178. 10.2753/mis0742-1222300406 [DOI] [Google Scholar]
- Kumar S., Yadava M., Roy P. P. (2019). Fusion of EEG response and sentiment analysis of products review to predict customer satisfaction. Inform. Fusion 52 41–52. 10.1016/j.inffus.2018.11.001 [DOI] [Google Scholar]
- Lee E. (2016). Empathy can increase customer equity related to pro-social brands. J. Bus. Res. 69 3748–3754. 10.1016/J.JBUSRES.2015.05.018 [DOI] [Google Scholar]
- Lin M. J., Cross S. N., Jones W. J., Childers T. L. (2018). Applying EEG in consumer neuroscience. Eur. J. Marketing 52 66–91. 10.1108/EJM-12-2016-0805 [DOI] [Google Scholar]
- Lin Y., Yang Y., Jung T. (2014). Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening. Front. Neurosci. 8 94–94. 10.3389/FNINS.2014.00094 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liu D., Liu S., Liu X., Zhang C., Li A., Jin C., et al. (2018). Interactive brain activity: review and progress on EEG-based hyperscanning in social interactions. Front. Psychol. 9:1862. 10.3389/fpsyg.2018.01862 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lobato B. Y. M., Garza A. A. (2017). Classification algorithm for measuring human emotion: “I like it” and “I do not like” in neuromarketing. IEEE Lat. Am. Trans. 15 2177–2184. 10.1109/TLA.2017.8070424 [DOI] [Google Scholar]
- Ma Q., Wang X., Dai S., Shu L. (2007). Event-related potential N270 correlates of brand extension. Neuroreport 18 1031–1034. 10.1097/WNR.0b013e3281667d59 [DOI] [PubMed] [Google Scholar]
- Malär L., Krohmer H., Hoyer W. D., Nyffenegger B. (2011). Emotional brand attachment and brand personality: the relative importance of the actual and the ideal self. J. Marketing 75 35–52. 10.1509/jmkg.75.4.35 11670861 [DOI] [Google Scholar]
- Marín-Morales J., Torrecilla-Moreno C., Guixeres Provinciale J., Llinares Millán M. D. C. (2017). Methodological bases for a new platform for the measurement of human behaviour in virtual environments. DYNA Ingeniería e Industria 92 34–38. 10.6036/7963 [DOI] [Google Scholar]
- Montague P. R., Berns G. S., Cohen J. D., McClure S. M., Pagnoni G., Dhamala M., et al. (2002). Hyperscanning: simultaneous fmri during linked social interactions. Neuroimage 16 1159–1164. 10.1006/nimg.2002.1150 [DOI] [PubMed] [Google Scholar]
- Nagyová L., Berčík J., Horská E. (2014). The efficiency, energy intensity and visual impact of the accent lighting in the retail grocery stores. Potravinarstvo Slovak J. Food Sci. 8 296–305. 10.5219/398 [DOI] [Google Scholar]
- Nilashi M., Samad S., Ahmadi N., Ahani A., Abumalloh R. A., Asadi S., et al. (2020). Neuromarketing: a review of research and implications for marketing. J. Soft Comput. Deci. Support Syst. 7 23–31. [Google Scholar]
- Ohme R., Reykowska D., Wiener D., Choromanska A. (2009). Analysis of neurophysiological reactions to advertising stimuli by means of EEG and galvanic skin response measures. J. Neurosci. Psychol. Econ. 2 21. 10.1037/a0015462 [DOI] [Google Scholar]
- Ohme R., Reykowska D., Wiener D., Choromanska A. (2010). Application of frontal EEG asymmetry to advertising research. J. Econ. Psychol. 31 785–793. 10.1016/j.joep.2010.03.008 [DOI] [Google Scholar]
- Palmiero M., Piccardi L. (2017). Frontal EEG asymmetry of mood: a mini-review. Front. Behav. Neurosci. 11:224. 10.3389/fnbeh.2017.00224 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pan Y., Cheng X., Zhang Z., Li X., Hu Y. (2017). Cooperation in lovers: an fnirs-based hyperscanning study. Hum. Brain Mapp. 38 831–841. 10.1002/hbm.23421 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Paulhus D. L. (2002). “Socially desirable responding: the evolution of a construct,” in The Role of Constructs in Psychological and Educational Measurement, eds Braun H. I., Jackson D. N., Wiley D. E. (New Jersey, NJ: Lawrence Erlbaum Associates Publishers; ). [Google Scholar]
- Picard R. W. (1997). Affective Computing. Cambridge, MA: MIT press. [Google Scholar]
- Plutchik R. (1980). “A general psychoevolutionary theory of emotion,” in Emotion: Theory, Research and Experience, Theories of Emotion Theories of Emotion, eds Plutchik R., Kellerman H. (New York, NY: Academic press; ), 3–33. 10.1016/b978-0-12-558701-3.50007-7 [DOI] [Google Scholar]
- Pozharliev R., Verbeke W. J., Van Strien J. W., Bagozzi R. P. (2015). Merely being with you increases my attention to luxury products: using EEG to understand consumers’ emotional experience with luxury branded products. J. Marketing Res. 52 546–558. 10.1509/jmr.13.0560 11670861 [DOI] [Google Scholar]
- Reeves B., Lang A., Thorson E., Rothschild M. (1989). Emotional television scenes and hemispheric specialization. Hum. Commun. Res. 15 493–508. 10.1111/j.1468-2958.1989.tb00196.x [DOI] [Google Scholar]
- Shang Q., Jin J., Pei G., Wang C., Wang X., Qiu J. (2020). Low-order webpage layout in online shopping facilitates purchase decisions: evidence from event-related potentials. Psychol. Res. Behav. Manag. 13:29. 10.2147/PRBM.S238581 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Singh M., Singh M., Gangwar S. (2013). Emotion recognition using electroencephalography (EEG): a review. Int. J. Inform. Technol.Knowl. Manag. 7 1–5. 10.1016/b978-0-12-804490-2.00001-4 [DOI] [Google Scholar]
- Stephens G. J., Silbert L. J., Hasson U. (2010). Speaker-listener neural coupling underlies successful communication. Proc. Natl. Acad. Sci. U.S.A. 107 14425–14430. 10.1073/pnas.1008662107 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tao J., Tan T. (2005). “Affective computing: a review,” in Affective Computing and Intelligent Interaction. ACII 2005. Lecture Notes in Computer Science, eds Tao J., Tan T., Picard R. W. Vol 3784 (Berlin: Springer; ), 981–995. 10.1007/11573548_125 [DOI] [Google Scholar]
- Telpaz A., Webb R., Levy D. J. (2015). Using EEG to predict consumers’ future choices. J. Marketing Res. 52 511–529. 10.1509/jmr.13.0564 11670861 [DOI] [Google Scholar]
- Übeyli E. D. (2008). Wavelet/mixture of experts network structure for EEG signals classification. Expert Syst. Appl. 34 1954–1962. 10.1016/j.eswa.2007.02.006 [DOI] [Google Scholar]
- Vecchiato G., Astolfi L., De VicoFallani F., Toppi J., Aloise F., Bez F., et al. (2011a). On the use of EEG or MEG brain imaging tools in neuromarketing research. Comput. Intell. Neurosci. 2011:643489. 10.1155/2011/643489 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vecchiato G., Astolfi L., Fallani F. D. V., Cincotti F., Mattia D., Salinari S., et al. (2010). Changes in brain activity during the observation of TV commercials by using EEG, GSR and HR measurements. Brain Topogr. 23 165–179. 10.1007/s10548-009-0127-0 [DOI] [PubMed] [Google Scholar]
- Vecchiato G., Cherubino P., Maglione A. G., Ezquierro M. T. H., Marinozzi F., Bini F., et al. (2014). How to measure cerebral correlates of emotions in marketing relevant tasks. Cogn. Comput. 6 856–871. 10.1007/s12559-014-9304-x [DOI] [Google Scholar]
- Vecchiato G., Toppi J., Astolfi L., Fallani F. D. V., Cincotti F., Mattia D., et al. (2011b). Spectral EEG frontal asymmetries correlate with the experienced pleasantness of TV commercial advertisements. Med. Biol. Eng. Comput. 49 579–583. 10.1007/s11517-011-0747-x [DOI] [PubMed] [Google Scholar]
- Venkatraman V., Dimoka A., Pavlou P. A., Vo K., Hampton W., Bollinger B., et al. (2015). Predicting advertising success beyond traditional measures: new insights from neurophysiological methods and market response modeling. J. Marketing Res. 52 436–452. 10.1509/jmr.13.0593 11670861 [DOI] [Google Scholar]
- Wei Z., Wu C., Wang X., Supratak A., Wang P., Guo Y. (2018). Using support vector machine on EEG for advertisement impact assessment. Front. Neurosci. 12:76. 10.3389/fnins.2018.00076 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yadava M., Kumar P., Saini R., Roy P. P., Dogra D. P. (2017). Analysis of EEG signals and its application to neuromarketing. Multimed. Tools Appl. 76 19087–19111. 10.1007/s11042-017-4580-6 [DOI] [Google Scholar]
- Yang T., Lee D., Kwak Y., Choi J., Kim C., Kim S. (2015). Evaluation of TV commercials using neurophysiological responses. J. Physiol. Anthropol. 34:19. 10.1186/s40101-015-0056-4 [DOI] [PMC free article] [PubMed] [Google Scholar]