Eating Event Detection by Magnetic Proximity Sensing

Abstract

Eating event detection is an important problem in automatic dietary study using a wearable computer, such as the eButton. In this work, we approach this detection problem based on the use of a small magnet marker attached to a finger and a miniature magnetometer installed within the eButton. Our experimental results indicate that our magnetic approach is effective when the distance between the marker and the wearable computer is within 12cm, and the range of detection is approximately 15cm. We also found that the proximity signal patterns corresponding to eating and other daily activities are different, which can be used to reduce the false detection rate. In addition, our approach is convenient, low-cost and energy efficient, suitable for practical applications.

I. Introduction

Obesity has become an important public health problem in the United States [1]. For obesity prevention and treatment, it is important to evaluate individuals' food intake objectively. We have designed a wearable computer called eButton to accomplish this task [2]. This button size device is worn in front of the chest to record an individual's daily food intake. At the time of each meal, the eButton is turned on to record a sequence of images documenting the entire eating event using a control switch. Although this procedure is easy to perform, the wearer of the device often forgets to turn on the device, resulting in a loss of data. It is highly desirable that eating can be detected and the data recording by eButton can be controlled automatically.

Eating event detection has been approached by applying image processing techniques. In such approaches, images are acquired and analyzed continuously. Since at least one hand is utilized during eating, the current method identifies eating by detecting a person's hand in images. There are several methods reported in the literatures about image-based hand detection [3]. However, hand recognition is a difficult task, and numerous images must be processed, which is a time-consuming and energy-demanding process. In addition, hand could be obscured by food, utensils and other objects from the view of the camera. These issues hamper the feasibility of the imaging based approach.

We propose a magnetic-flied-based approach to detect the hand. In this approach, a miniature magnetic maker is worn on one finger of the hand. A magnetometer is embedded inside of the eButton. When the hand is close to the eButton, the output of the magnetometer will increase. If the hand is engaged in eating, a sequence of signals is produced (Fig. 1 (a)). By analyzing the patterns of the signals, the event of eating can be detected which can be used to switch on the camera within the eButton. Because the eating detection scheme involves a certain delay, in cases where the integrity of eating process is important, images can be recorded continuously, placed within a buffer, and saved with a predefined overhead segment when the output of the eating detector becomes positive.

Figure 1. (a) eButton embedded with magnetic sensor worn in front of the chest for diet assesment; (b) Several components of eButon.

II. Methods

As shown in Fig. 1(b), eButton is a button-size wearable computer for recording food intake. It is also capable of recording physical activity, social interaction, build environment and other lifestyle related events which are beyond the topic of this paper. A digital camera equipped by the device takes pictures automatically every one to four seconds (set by the user) to capture the events in front of the device. When the wearer ingests food, the pictures of the whole eating process including food are recorded and stored in a miniature SD card along with other forms of data, such as the GPS signal, accelerometer and gyroscope signals, and the clock time. The recorded pictures are analyzed off-line to identify the food being eaten and its volume. A food database is accessed to obtain calories and nutrients for the ingested food.

In order to enable the eButton to control the camera or mark the continuously recorded images, we investigated a low-cost, miniature magnetometer (Honeywell HMC5883L). It is a 3-axis, digital, and 3×3×0.9mm multi-chip designed for low-field magnetic sensing [4]. The schematic for connection between HMC5883L sensor and eButton is shown in Fig.2 where the I2C bus was utilized for data communication. Through I2C bus, HMC5883L sends the recorded magnetic flux density data to the eButton which stores these data in a mini SD card together with images and data from other sensors synchronously.

Figure 2. Connection between HMC5883L and eButton.

In our prototype system design, a permanent magnetic marker is affixed to a single finger of the hand. This marker can either be a small magnetic disk or a magnet ring containing a rare earth material. While the magnetic disk is more cost-effective, the magnet ring appears to be more desirable because the ring, which resembles a regular finger ring, does not interferes with functions of the hand. We have studied both cases. For a magnet disc, the D82-N52 magnet from K&J Magnetics Inc. was chosen as the magnetic marker as shown in Fig.3 (a). As for ring type marker, a 0.7 inch diameter magician magnet ring was chosen as shown in Fig. 3 (b).

Figure 3. (a) 0.3-inch diameter N52 magnet disc (b) 0.7-inch diameter magnet ring.

Besides a stable background magnetic field produced by the earth, the magnetic flux densities detected by the HMC5883L sensor mainly come from the magnetic marker. The magnetic flux density B due to this marker can be modeled as a magnetic dipole, expressed as:

$$\mathbf{B}=\frac{\eta \mu }{4\pi }(\frac{3(\mathbf{L}\cdot \mathbf{R})\mathbf{R}}{R^5}-\frac{\mathbf{L}}{R^3})$$ (1)

where R indicates the distance from magnet to the magnetometer location, R = ‖R‖ is the Euclidean norm of R, L refers to the dipole orientation, μ is the medium permeability, and ν is the magnetic moment of the magnet. From Eq. (1), it is obvious that, when the hand is close to the eButton, the magnet flux density increases proportionally to the cubic power of the distance between magnetic marker and eButton.

For hand to be detected when it is close to the eButton, a magnetic flux density threshold needs to be determined. This threshold should be set such that, when an eating event occurs, the magnet maker on the finger is strong enough to trigger a significant change in the magnetic flux density detectable by the magnetometer within the eButton. From our preliminary experiments, we found that, with an eating event takes place, the shortest distance between magnet marker on the finger and the magnetometer in the eButton is within the range from 5cm to 12cm, which is subject to: 1) the magnet marker type (magnet disc or ring), 2) the location of the eButon on the chest, and 3) the eating habit of the wearer. Our study indicates that the threshold chosen as the magnet flux density corresponding to the 12cm distance produces the best performance.

It is clear that the magnetometer also produces a signal in response to certain non-eating events such as scratching the head or resting the chin in the hand. Fortunately, in these cases, the detected signal patterns are generally different from those of eating events. In order to reduce the rate of false detection, signal patterns with respect to different behaviors must be studied. We have investigated this important issue experimentally.

III. Experiments

In our first experiment, a magnet marker was moved towards the magnet sensor starting from a distance of 30cm. The data acquired by the magnetic senosr were recorded until the magnetic flux density surpassed the sensor' work range (output integer values between -2048 and 2047). From the experimental result (Fig. 4), it can be observed that there was no significant change in the output between 15cm and 30 cm. Below 15cm, the output started to increase dramatically until the value exceeded sensor's work range (2047) at 8.5cm. Hence, the proximity of the hand can be detected within 12cm.

Figure 4. Experimental result of Magnetometer.

In our second experiment, we evaluated the proximity patterns of the hand for different daily events, including eating events. In this case, we utilized a commercial optical proximity sensor for higher accuracy. We monitored a male human subject during the day while he performed daily activity normally. Fig. 5 shows a 1.8-hour segment of this test. It can be seen that, although the subject performed a number of non-eating related events, the signal pattern during eating is quite different from those of other events. Therefore, the differences in signal patterns can be used to reduce the rate of false detection.

Figure 5. Proximity signals during a day-time experiment conclusion.

IV. Conclusion

In conclusion, we have found that the magnetic sensor can detect hand proximity when it is in the range of within 15cm of a chest worn magnetic sensor during an eating event. When a proper threshold (approximately 12 cm) and a signal classification algorithm are applied to the recorded proximity signal, the new approach can detect eating events accurately and efficiently.