Table 1.

Comparison between the previous methods and the proposed method of emotion recognition.

Category		Method	Advantages	Disadvantage
Using a single modality	Visible-light camera-based methods [3,4,5,6].	User’s emotion is recognized based on facial expression in an image.	- Providing comfort to the user without the attachment of sensors. - Less expensive method than bio-signal or thermal camera-based methods.	- Analyzing emotion is difficult if the person has no expression. - Extraction of the facial feature points can be affected by non-uniform illumination.
	Thermal camera-based methods [7,12].	Measuring the change of facial temperature according to emotion.	- Providing comfort to the user without the attachment of sensors. - Analyzing emotion is easy, even if the person has no expression. - Extraction of the facial feature points is not affected by illumination condition.	- More expensive method than visible-light camera-based method. - Difficult to detect regions of facial features because the texture of facial features is not distinct in the thermal image.
	Voice-based methods [2,8,9].	Measuring the change of voice features according to emotion.	Less expensive method than bio-signal or thermal camera-based method.	- The performance can be affected by surrounding noises.
	Physiological signal-based methods [10,11,16,17,18].	ECG [10] and EEG [11,16,17,18] are analyzed for emotion detection.	High accuracy of emotion detection and fast speed of data acquisition.	- More discomfort to the user because sensors are attached to the body. - More influenced by the motion of the head, body and muscles than camera- or voice-based methods.
Using multiple modalities	Multiple physiological signal-based methods [13,14,15].	- Using EEG, heart rate, SC, respiration, ST and psychometrical ratings [13]. - Using PPG, EMG, ECG, GSR and ST [14]. - Using SC, blood volume pressure, ST, EMG and respiration [15].	Higher accuracy of emotion detection compared to single modality-based methods.	- More discomfort to the user because many sensors are attached to the body. - More influenced by the motion of the head, body and muscles than single modality-based methods.
	Hybrid method using both physiological signals and non-intrusive camera-based methods (proposed method).	Using facial temperature, EEG, blinking rate and subjective evaluation for evaluating fear.	- Higher accuracy of emotion evaluation compared to single modality-based methods. - More comfort to the user and higher freedom of movement without the attachment of sensors, except for a wireless EEG device. - Facial feature positions on the thermal image can be easily detected using dual (visible-light and thermal) cameras.	Larger amount of data to be processed after acquiring the image sequences by dual cameras.