Real-Time Learning from an Expert in Deep Recommendation Systems with Application to mHealth for Physical Exercises

Abstract

Recommendation systems play an important role in today’s digital world. They have found applications in various areas such as music platforms, e.g., Spotify, and movie streaming services, e.g., Netflix. Less research effort has been devoted to physical exercise recommendation systems. Sedentary lifestyles have become the major driver of several diseases as well as healthcare costs. In this paper, we develop a recommendation system to recommend daily exercise activities to users based on their history, profiles and similar users. The developed recommendation system uses a deep recurrent neural network with user-profile attention and temporal attention mechanisms. Moreover, exercise recommendation systems are significantly different from streaming recommendation systems in that we are not able to collect click feedback from the participants in exercise recommendation systems. Thus, we propose a real-time, expert-in-the-loop active learning procedure. The active learner calculates the uncertainty of the recommendation system at each time step for each user and asks an expert for recommendation when the certainty is low. In this paper, we derive the probability distribution function of marginal distance, and use it to determine when to ask experts for feedback. Our experimental results on a mHealth and MovieLens datasets show improved accuracy after incorporating the real-time active learner with the recommendation system.

I. Introduction

A MAJOR driver of healthcare costs in different countries are unhealthy behaviors such as physical inactivity, increased food intake, and unhealthy food choice [1], [2]. Behavioral and environmental health factors account for more deaths than genetics [1]. Pervasive computational, sensing, and communication technology can be leveraged to support individuals in their everyday lives to develop healthier lifestyles. For instance, the pervasive use of smartphones is a potential platform for the delivery of behavior-change methods at great economies of scale. Commercial systems such as noom [3] aim to provide psychological support via mobile health (mHealth) systems. Research platforms, such as the Fittle+ system [4], have demonstrated the efficacy of translating known behavior-change techniques [5] into personal mHealth applications. However, most work in the mHealth domain are limited to the development of smartphone applications and connecting subjects and expert knowledge.

On the other hand, recommendation systems are becoming popular in various applications. E-commerce websites have been using recommendation systems to suggest new products and items to existing and new users to entice them into purchasing new items [6], [7]. With the advent of streaming movies and music services, e.g., Netflix, Spotify, service providers need to keep their users interested or lose their customers and profits. Thus, these streaming service providers deploy recommendation systems to suggest new movies and musics to their existing and new users based on their history of watching movies and listening to musics [8], [9]. However, not many research studies have been devoted to developing recommendation systems for exercise activities [10].

In this paper, we develop an attention-based recommendation system for exercise activities to new users using a mHealth application. The proposed recommendation system is based on a deep recurrent neural network that takes advantage of users’ profiles and exercise characteristics as features and temporal attention mechanisms. However, one major difference between exercise activities and other domains is that the recommendation system is not able to collect users’ feedback. In movie, e-commerce, and music applications, recommendation systems constantly receive feedback from users’ clicks data. When a recommendation system suggests a new music (or a movie) to a user, the user may click on the suggested music (or movie) and listen to (or watch) it completely. The click and duration of playback is used as a feedback to fine-tune the recommendation system. Although users are able to provide ratings in these systems, they may choose not to do so. As a result, the systems will use the click data, duration of playback, etc. to assess whether the recommended music (or movie) was a good recommendation or not.

However, when a recommendation system suggests an exercise activity to the user, the application is not able to collect the click data because the application cannot observe the user, and the only way to collect the feedback is if the user chooses to rate the exercise. In other words, the recommendation system doesn’t know whether the user completed the exercise, i.e. if it was a good recommendation. In some mHealth apps [11], users were asked to provide that information manually, but the missing data is tremendous as most users were ignorant of that feedback. The problem is more challenging when the recommendation system faces a new user without any history. To address this issue, in this paper, we take advantage of a real time expert-in-the-loop mechanism. Individuals trust expert personal trainers to provide them with exercises plans based on the experts’ knowledge and experience. Our proposed system will leverage this trust to use experts’ knowledge when the system is uncertain. As a result, more individuals will have access to exercises plans with expert personal trainers in the loop. Our proposed network will calculate the certainty of a new recommendation using the probability distribution of the marginal distance, the difference between the highest probability and the second highest probability of exercise classes in the output of the recommendation system. To quantify the certainty, we derive the probability distribution of the marginal distance from the probability distribution of the last layer of the recommender. Even though the marginal distance has been used in active learning, this is the first work to provide its probability distribution for statistical hypothesis testing.

The other challenge with most recommendation systems is the initialization of the recommendation system for new users. Much work has been devoted to address this issue, including metal-learning approaches [12]. In this paper, we leverage the questionnaire filled by users and their demographic information to find the existing users with similar interests and demographic information. Then, the global recommendation model is fined tuned with the history of the similar users. Comparing to other existing methods, this approach has less complexity. Fig. 1 shows the overall architecture of the recommendation system.

Fig. 1.

The overall architecture of the proposed recommendation system with expert-in-the-loop. The exercise activities are recommended through a smartphone app, and their completion is collected from smartphones. The deep recommendation system is trained on the collected history data and its augmentation. The new recommender system is initialized for each new participant from the global trained model, and fine-tuned with similar users based on their profiles. At each time step, a new exercise is recommended to users. If the recommendation system is uncertain about the new recommendation, i.e. whether the user will complete the exercises or not, the recommendation system will ask the expert for correction.

The rest of this paper is organized as follows: Section II describes the mHealth data used in this study. The architecture of the proposed recommendation system is explained in Section III-A. Section III-B explains how we take advantage of the user profiles to initialize a recommendation system for new users. Section IV describes the proposed active learning procedure based on the distribution of the marginal distance. Section V is devoted to the experimental results and discussion.

A. Related Work

More recently, mHealth systems have found applications in different healthcare domains since the advent of smart phones. Dunsmuir et al. [13] developed mHealth for diagnosis and management of pregnant women with pre-eclampsia. In [14], the inter-pulse-interval security keys was used to authenticate entities for various mHealth applications. Schiza et al. [15] proposed a unified framework for an eHealth national healthcare system for European Union. In another work [16], WE-CARE, a mobile 7-lead ECG device, was developed to provide 24/7 cardiovascular monitoring system. In [17], an mHealth system was designed and developed to provide exercise advice to participants based on their Body Mass Index, Basal Metabolic Rate, and the energy used in each activity or sport, e.g. aerobic dancing, cycling, jogging working and swimming. However, this work does not use machine learning algorithms.

On the other hand, recommendation systems have been used in e-commerce and online shopping for several years [18]–[21]. The goal of recommendation systems is to recommend products that suit the consumers’ tastes. Traditional recommendation systems have used collaborative filters to suggest products similar to those the consumers have purchased. With the advancements in deep learning algorithms [22], several studies have proposed deep learning-based recommendation systems [23]. In [23], a multi-stack recurrent neural network (RNN) architecture is used to develop a recommendation system to suggest businesses in Yelp based on their reviews. Wu et al. [24] used an RNN with long-short term memories (LSTM) to predict future behavioral trajectories. A few studies have proposed exercise recommendation systems [25]. Sami et al. [25] used several independent variables to recommend various sports such as swimming using collaborative approaches. Ni et al. [26] developed an LSTM-based model called FitRec for estimating a user’s heart rate profile over candidate activities and then predicting activities on that basis. The model was tested against 250 thousand workout records with associated sensor measurements including heart rate.

In the health care domain, Yoon et al. [27] developed a recommendation system for a personalized clinical decision making. The system used electronic health records of different patients and their clinical decisions to recommend clinical decisions for new patients. In a recent work [28], support vector machines (SVMs), random forest, and logistic regression were used to recommend skin-health products based on genetic phenotypes of consumers. In a similar study [29], user contextual features and daily trajectories of steps over time were used to develop a recommendation system for planning an hour-by-hour activity. The model classifies users to subgroups and recommend activities based on the history of similar users. The proposed method doesn’t use any time series modeling, e.g., recurrent neural networks, to learn patterns, thus lacking the ability to generalize to more users.

In a related field, several works have been devoted to the cold-start problem in recommendation systems [30], [31]. The cold-start problem refers to the new users whose historical data is not available to the recommendation system. Thus, the recommendation system is not able to accurately recommend items, e.g., music, movies, etc., to the new users. Meta-learning approaches try to learn a global model from all users to initialize the recommendation systems for new users [30]. The shortcoming of these approaches is that they are not personalized in the beginning. The second approach is zero-shot, one-shot, or transfer learning methods [31], [32]. These approaches transfer a global model learned from other users’ historic data to new users. These approaches have great performance in classifiers, but they still have shortcomings as recommendation systems dealing with sequences of data. The proposed approach leverage experts (human personal trainers) to actively learn personalized exercise programs for new users. While many people use trainers to get recommendation for their daily exercise activities, the proposed approach leverages the expert knowledge to reach a broader group of users at a lower cost because the expert does not need to be continuously involved in recommending new exercises, just in the beginning when the system is uncertain about new users.

II. Mhealth Data

The data we use comes from the Konrad et al. [11] mHealth experiment with DStress. It was developed to provide coaching on exercise and meditation goals for adults seeking to reduce stress. The purpose of the experiment was to test the efficacy of an adaptive daily exercise recommender (DStress-adaptive) against two alternative exercises programs in which the daily exercises changed according to fixed schedules (Easy-fixed and Difficult-fixed). The DStress-adpative recommender was a hand engineered finite state machine. The transition rules are described in more detail in Konrad et al. [11], but they implement a policy whereby, if a person successfully completes all three exercises assigned for a day, they advance to the next higher level of exercise difficulty. If they do not succeed at exercises or meditation activities, then they are regressed to exercises or meditation activities at an easier level of difficulty. The 44 exercises used in DStree and their difficulty ratings were obtained from three certified personal trainers (e.g., Wall Pushups, Standing Knee Lifts, Squats, and Burpees, etc.). The experiment took place over a 28-day period. In a given week, users encountered three kinds of days: Exercise Days (occurring on Mondays, Wednesdays, Fridays), Meditation Days (Tuesdays, Thursdays, Saturdays), and Rest Days (Sundays).

72 adult participants (19–59 yr) were randomly assigned to three conditions with different 28-day goal progressions: (1) a DStress-adaptive condition using the adaptive coaching system in which goal difficulties adjusted to the user based on past performance, (2) an Easy-fixed condition in which the difficulty of daily goals increased at the same slow rate for all participants assigned to that condition, and (3) a Difficult-fixed condition in which the goal difficulties increased at a greater rate. Konrad et al. [11] found that the adaptive DStress-adaptive condition produced significant reductions in self-reported stress levels compared to the Easy-fixed and Difficult-fixed goal schedules. The DStress-adaptive condition also produced superior rates of performing assigned daily exercise goals.

User Profile:

A variety of pretest survey data were collected in the Konrad et al. [11] study that provides our user data. These included the (1) Perceived Stress Scale (PSS), which is a 10-item psychometric scale assessing perceived stress over the past month, (2) Depression, Anxiety, Stress Scale (DASS), a 21 item assessment of depression, anxiety, and stress, (3) the Cohen-Hoberman Inventory of Physical Symptoms (CHIPS): a 33-item scale measuring concerning physical symptoms over the past 2 week, (4) BMI, the Body Mass Index, and (5) Goldin Leisure-Time Exercise Questionnaire (GLTEQ), a 4-item scale measuring frequency of physical activity during leisure time, and (6) the Exercise Self-Efficacy Scale (EXSE), and 8-item assessment of self-efficacy about exercising in next 1–8 weeks.

The Perceived Stress Scale (PSS) is the most widely used instrument for measuring the perception of street [33], [34]. The Depression Anxiety Stress Scale [35] is a valid, reliable instrument measuring depression, anxiety, and stress [36]. The Cophen-Hoberman Inventory of Phsyical Symptoms (CHIPS) measures concerns about physical symptoms over the past 2 weeks was included because stress is often manifested in such symptoms [37]. The Godin Leisure-Time Exercise Questionnaire (GLTEQ; 4 item test) was included to assess pre-experimental activity levels [38]. The Exercise Self-efficacy Scale (EXSE) is an 8-item test assessing individuals’ beliefs in the ability to exercise [39].

Exercise Profile:

The original Konrad et al. study [11] obtained difficulty ratings of the exercises from three subject matter experts (SMEs; personal trainers) that were predictive of the probability of performing the exercises [40]. We augmented these data with exercise classification, attributes, and relations obtained from a 60 minute structured interview with an SME (fitness coach) that was followed up with specific clarification questions. The structured interview consisted of a card-sorting task and an exercise program planning task.

For the card-sorting task, exercise names and descriptions were placed on 3×5 cards. The cards were shuffled and the SME was asked to go through the cards to familiarize herself with the exercises. The SME was asked to sort the exercises into piles by whatever criteria “seemed natural.” The SME was asked to label those piles. These original piles were grouped into super-categories and labeled. Then the original piles were sorted into subcategories and labeled recursively until no further subgroups made sense to the SME. The SME was then asked if there was a possible “alternative grouping” of the exercises. The SME was also asked to rate the difficulty of each exercise. This card sorting produced an initial hierarchical classification of the exercises into a top level of resistance exercises and metabolic conditioning exercises. Within those supercategories there were subcategories for push, pull, squats, lunges, single leg stance, and core exercises. Further subcategories consisted of back, chest, legs. legs/glutes, core/abs, and abs/glute exercises. The alternative grouping consisted of full body, compound, and power categories.

III. Proposed Recommendation System

The goal of this paper is to recommend the next exercise for a given participants based on the history of exercises the participant completed. Let Xⁱ(0), Xⁱ(1), . . ., Xⁱ(T) represent the exercise history for the ith participant, where $X^i(.)\in {ℝ}^{N\times 1}$ is the one-hot encoding of the exercise and N is the total number of exercises (N = 44 in mHealth dataset). Let $U^i\in {ℝ}^{N_U\times 1}$ and $E^j\in {ℝ}^{N_I\times 1}$ represent the ith participant’s profile and the jth exercise’s profile, respectively. The recommender gets Xⁱ(t − ω + 1), . . ., Xⁱ(t − 1) as the input in addition to the ith user’s profile and the exercises’ profiles and predicts Xⁱ(t) to recommend to the user. The length of the window, ω, is determined by the developer. In this paper, we use autocorrelation function (ACF) of X to determine the appropriate window length, similar to the statistical time series analysis methods [41].

A. Network Architecture

The proposed network consists of five modules: encoder, decoder, recurrent neural network (RNN), user attention, exercise temporal attention. The hyper-parameters of this architecture are selected empirically. Fig. 2 shows the architecture and modules of the proposed deep recurrent neural network.

Fig. 2.

The architecture and modules of the proposed deep recurrent neural network. The recommendation system uses users and exercise profiles as attention mechanism. The attention mechanism will highlight the most relevant characters tics of the exercises to each user.

Exercise input:

The encoder is a fully connected (FC) linear layer that embeds the N-dimensional exercise names onto a K_X-dimensional vector space:

$$H_X^i(t)=ReLU\left(W_X\times X^i(t)\right),$$ (1)

where $H_X^i(t)\in {ℝ}^{K_X\times 1}$ is the exercise name’s embedding at time t for the ith participant, $W_X\in {ℝ}^{K_X\times N}$ is a trainable weight matrix learned from the training data, and ReLU is the activation function.

User Profile input:

The profile of the ith user $U^i\in {ℝ}^{N_u\times 1}$ is provided as a N_u-dimensional vector. The user profile is embedded onto a K_u-dimensional vector space:

$$H_u^i=ReLU\left(W_u\times U^i\right),$$ (2)

where $H_u^i\in {ℝ}^{K_u\times 1}$ is the embedding of the ith user’s profile, and $W_u\in {ℝ}^{K_u\times N_u}$.

Users’ profiles, e.g., demographic information, provide valuable information about paying attention to specific aspects and features of exercises. For example, age and gender are two important variables that can significantly affect the types of exercises users will likely to perform. The attention mechanism will highlights the features of the exercise (extracted by neural network) that are more relevant to the current user based on their demographic information. Thus, we combine the exercise name embedding and the user profile embedding using:

$$p_u(t)=softmax\left(W_1\times H_x^i(t)\oplus W_2\times H_u^i\right),$$ (3)

where $W_1\in {ℝ}^{K\times K_x}$ and $W_2\in {ℝ}^{K\times K_u}$ are trainable mapping matrices, $p_u(t)\in {ℝ}^{K\times 1}$ is the combination vector of the exercise and the user profiles which provides the attention probability for each entity of the input exercise. The final vector is the element-wise multiplication of the exercise name embedding and the attention probability: ${\psi }^i(t)=p_u(t)\odot (W_1\times H_x^i(t))$. The exercise name embedding $H_x^i$ is a vector in a K_x-dimensional space, and multiplying it with the attention probability p_u(t) rotates this vector in the K_x-dimensional space.

Exercise Profile input and temporal attention mechanism:

Similarly, the profile of the jth exercises is provided with a N_e-dimensional vector $E^j\in {ℝ}^{N_e\times 1}$ and embedded using a linear layer:

$$H_e^j(t)=ReLU\left(W_e\times E^j(t)\right),$$ (4)

where $H_e^j(t)\in {ℝ}^{K_e\times 1}$ is the embedding of the jth exercise, and $W_e\in {ℝ}^{K_e\times N_e}$ is the trainable matrix. Note that we use (t) in front of the exercise profile similar to the exercise names X(t) to represent the time information.

At each time step, the user performs an exercise that has a huge impact on the future exercises the user has desire to complete. For example, an user who is doing exercises focused on upper body may want to continue upper-body exercises for a day and then focus on the lower-body exercises on another day. In another example, the difficulty of the current exercise affects the difficulty level of the short-term future exercises. To incorporate this information into the recommendation system, we use the exercise profiles as an attention mechanism to give weights to different exercises at different time steps. The temporal attention mechanism assigns probability values to different time steps within the ω-length window. The exercise embeddings are combined with ψⁱ(t):

$$p_e(t)=softmax(W_3\times {\psi }^i(t)\oplus W_4\times H_e^j(t)),$$ (5)

where $W_3\in {ℝ}^{\omega \times K}$ and $W_4\in {ℝ}^{\omega \times K_e}$ are trainable mapping matrices, $p_e(t)\in {ℝ}^{\omega \times 1}$ is the combination vector of the user-attention exercise and exercise profiles giving the attention probability for each time step. The final time series of vectors with length ω used as the input to the RNN module is: ϕⁱ(t − k) = p_e(k) × ψⁱ(t − k) for k = {1, 2, . . ., ω}.

RNN:

The RNN module is responsible for learning the sequential pattern of the exercise history. Although there are different variants of RNN modules with long short-term memory (LSTM) and gated recurrent units (GRU), we will use regular RNN modules. The reason is that LSTMs and GRUs have built-in units to learn the dependencies of time series over time and accentuate or de-emphasize relevant information at different time steps. In this paper, we will use temporal attention mechanism that will take into account the importance of different time steps. Moreover, our auto-correlation function (ACF) analysis of this dataset shows a short-term dependencies for exercise recommendation. However, in developing recommendation systems for datasets with long-term dependencies, we will replace regular RNNs with RNNs with LSTM or GRU units. The RNN module consists of one hidden layer with ReLU activation function:

$$h(t)=ReLU\left(W_{\varphi }\times {\varphi }^i(t)+W_h\times h(t-1)\right),$$ (6)

where W_ϕ and W_h are trainable weights and h(t) is the hiddent state at time t.

Exercise name prediction:

The decoder converts the predicted hidden state at time t into exercise names:

$$Y^i(t)=softmax\left(W_y\times h(t)\right),$$ (7)

where W_y is a trainable matrix and $Y^i(t)\in {ℝ}^{N\times 1}$ is the multinomial distribution over N exercises.

Training:

The whole network is trained end-to-end with the cross entropy loss function:

$$\min -{\mathbb{E}}_{X(t)~p_{data}}[log(P(Y(t)\mid X(t-1),\dots ,X(t-\omega ),U,E)]$$ (8)

All modules are analytically differential. Thus, the gradient can back-propagate through decoder, recurrent layers through time, and user and temporal attention modules. Adam optimizer is used for training the network [42].

B. New User Initialization

The recommendation system is trained with a set of training data collected from different users. The system is general and not personalized for new users. To personalized the recommender, we fine tune our network with the training dataset of the users whose profiles are very similar to the new user. To achieve this goal, we calculate the similarity between Uⁱ and the existing users by Euclidean distance, $\sqrt{{\Vert U^i-U^j\Vert }^2}$ for i ≠ j, to find the most similar users. Then, users are sorted from the most similar to the least similar, and the k most similar users are selected. Afterwards, the recommender system is fine-tuned with the training data of the selected users, small learning rate, and one epoch.

IV. Active Learning

Active learning is used to ask experts for providing annotation for unlabeled samples [43], [44]. However, it is impossible to ask experts for annotating a large set of unlabeled samples. The active learner is usually presented with a limited budget to ask experts for annotation. The active learner selects the most informative samples based on the uncertainty that a trained classifier has about these samples. Two most common methods of measuring the uncertainty are entropy and marginal distance [45]–[47].

In this paper, we use active learning to personalize the recommender for each user. The recommender is trained on a set of training data from different people. Thus, the recommender is not tailored for new users. The profile of the users will provide the attention mechanism in the feature space, but not enough to personalize the recommender. When we get a new user, the recommender will be used. However, when the recommender is uncertain about the next recommended exercise, the recommender will ask the expert to intervene and provide the next recommendation. We will use the marginal distance and entropy of the series of recommendation as criteria to decide when to ask the expert. However, the existing work has set an arbitrary threshold on the marginal distance. In this paper, we derive the probability distribution function of the marginal distance.

A. Marginal Distance Random Variable

The output of the classifier (the last layer of the recommender) is a vector of N random variables, Y = [y₁, y₂, . . ., y_N]^T, with a Multinomial distribution. Ideally, one of the y_is is one and the rest are zeros. However, N random variables represent the probability of the input sample belonging to each of these classes. Let p = [p₁, p₂, . . ., _N ]^T represent the probability values. p_i has a Beta distribution and the vector p is drawn from a Dirichlet distribution [48]. Thus, these N random variables are sorted in ascending order y₍₁₎ ≤ y₍₂₎ ≤ … ≤ y_(N) and the marginal distance is defined as Z = y_(N) − y_(N−1). The distribution of the marginal distance is (see Appendix A for proof):

$$\begin{gathered} f(z)=\frac{\mathrm{\Gamma }\left({\alpha }_1+{\alpha }_2+{\alpha }_3\right)}{\mathrm{\Gamma }\left({\alpha }_1\right)\mathrm{\Gamma }\left({\alpha }_2\right)\mathrm{\Gamma }\left({\alpha }_3\right)} \\ \times {\int }_{y_{(N-1)}=0}^{\frac{1-z}{2}}{\left(y_{(N-1)}+z\right)}^{\left({\alpha }_1-1\right)}{y}_{(N-1)}^{\left({\alpha }_2-1\right)} \\ \times {\left(1-2y_{(N-1)}-z\right)}^{\left({\alpha }_3-1\right)}d{y}_{(N-1)} \end{gathered}$$ (9)

The Monte Carlo method will be used to approximate this distribution.

B. Active Learning Procedure

The key component in active learning is to determine when to ask the expert for feedback. The recommender system needs to ask for the expert opinion when the input sample is out of the distribution of the training dataset. Let ${\mathcal{D}}_s$ represents the distribution of the training dataset. We use ${\mathcal{D}}_s$ to derive the probability distribution function as the marginal distance as defined in § IV-A. Let ${\mathcal{D}}_{\mathcal{M}}$ represent this distribution, and zⁱ(t) represent the marginal distance for the ith user at time t. Then, our hypothesis is:

$$\{\begin{array}{ll}{H}_0:\hfill & z^i(t)\in {\mathcal{D}}_{\mathcal{M}}\hfill \\ H_1:\hfill & \text{otherwise}\hfill \end{array}$$

The α-level hypothesis testing determines whether the recommended exercise should be given to the user or ask the expert for the right exercise recommendation (feedback or label).Then, the trained recommender is fine-tuned with the feedback from the expert for personalization.

V. Experimental Results

In this section, the proposed approach with different components are evaluated on the offline mHealth dataset. The proposed architecture is also evaluated on the MovieLens100 K dataset [49].

A. mHealth

This section is devoted to the evaluation of the proposed exercises recommendation system on an exercises activity dataset. The dataset was described comprehensively in Section II.

1). Data Augmentation:

The challenge with recommendation systems is not having enough training data. One way to increase the amount of training data is to use data augmentation [50], [51]. While data augmentation in computer vision is straightforward and can be achieved by adding noise or cropping images randomly, it requires careful attention for sequential and symbolic data, e.g., language [50]. The random creation of sequential data has negative effect as it introduces noise to the data that does not follow the sequential pattern in the real data.

For exercise activities, there are two ways to augment the sequential data: asking the human expert or using association mining on the training data. In the first approach, an exercise expert was asked to categorize different exercises available for participants. Then, the augmentation algorithm goes over the sequence of exercises for each participant in the training data, chooses 10% of exercises and replace them with similar exercises in the same category.

In the second approach, we propose to use association rule mining [52] to extract rules from frequent itemsets. Then, the augmentation algorithms go over the sequence of exercises of each participant, choose 10% of exercises and replace them with their similar exercises based on these rules.

2). Experiment Setup and Discussion:

In order to find the appropriate length of window for the RNN model, we looked at the autocorrelation function of the exercise sequence. The autocorrelation function shows the degree of the dependencies of time series data and is often used to select the order of the time series analysis methods. Our ACF analysis shows that the length of the sequence for the RNN model should be w = 3. Processing the data with w = 3 results in 2343 sequence of training samples. The architecture of the proposed recommendation system is selected empirically as follows: K_X = 20, K_u = 15, K_e = 3, and K = 10.

Baseline:

We use the proposed RNN model with the user profile attention mechanism and the exercise profile temporal attention mechanism (described in § III-A) as our baseline method. The model is trained with Cross-entropy loss function and Adam optimizer [42] for 30 epochs. We use k-fold approach for evaluating the performance of the recommender. In our k-fold setup, we kept one participant out of the training dataset and used the remaining 71 participants data for the training. Then, the left-out participant is treated as a new participant and the trained recommender is used to recommend exercises to the new participant. The actual data from the left-out participant is used as the groundtruth to calculate the accuracy of the recommendation system. We calculate the top-1, top-5, and top-10 accuracy for evaluating the recommendation system. The experiment is repeated 5 times and the average of the top-k accuracy are reported in Table I. In the first experiment, we only use the demographic information of users in the attention mechanism of the proposed recommendation system. Table I row 1 shows the accuracy. In the second, experiment, we used all information extracted from questionnaires in addition to their demographic information for the attention mechanism, and observed a slight decline in the accuracy (Table I row 2). We hypothesize that most people don’t have an accurate evaluation of their own ability. Thus, the answers to the questionnaires may not accurately represent their profiles, leading to a conflict to what exercises they performed and what they answered. For example, two persons may give exact similar answers to the questions (possibly not accurate answers), but have different ability and interest to perform exercises. In the rest of this paper, we only use demographic information to represent the user’s profile.

TABLE I.

Accuracy of the Recommendation System

Method	top-1 Accuracy	top-5 Accuracy	top-10 Accuracy
1. Baseline (Demographic)	63.78%	92.19%	97.33%
2. Baseline (Full Profile)	61.16%	90.76%	96.98%
3. Baseline + Data Augmentation (Expert)	72.53%	95.53%	98.56%
4. Baseline + Data Augmentation (Rule based)	69.74%	95.28%	98.68%
5. Baseline (Demographic) + Active Learning	74.45%	95.29%	98.48%
6. Baseline (Demographic) + New user Init	65.91%	93.14%	97.65%
7. Baseline + Data Augmentation (Expert) + Active Learning	80.12%	97.23%	99.26%
8. Baseline + Data Augmentation (Expert) + New user Init	71.90%	95.27%	98.42%
9. Baseline + Data Augmentation (Expert) + New user Init + Active Learning	80.08%	97.00%	99.11%

We compared the baseline model with user demographic information with a slightly different network architecture to study the affect of the architecture. In the new architecture, we reduced the dimension of the RNN module to half, and also we reduce the dimension of the output module. The top-1, top-5, and top-10 accuracy are 60.87%, 90.55%, and 94.71%. Compared to the row 1 of Table I, the alternative architecture has a worse performance. The original structure proposed in Section III-A has the best performance for the mHealth data and is selected empirically.

Baseline with Data Augmentation:

The training dataset was augmented with two approaches described in § V-A1. The training and augmented data were used to train the baseline model and evaluated as described in the baseline section. Table I row 3 shows the accuracy results of the baseline model trained with the training data and the expert augmented data. The data augmentation generalizes the model and improves the accuracy compared to the baseline (Table I row 1).

On the other hand, we see a decline in the accuracy of the baseline model when it is trained on the training dataset and the augmented data by association rule mining algorithms (Table I row 4). This observation points to the importance of having the expert in the loop for exercise recommendation systems. Because in this experiment the augmented dataset with expert knowledge gives higher accuracy, we use this method for other experiments in this paper.

Baseline with Active Learning:

The training data was used to estimate the parameters of the Dirichlet distributions: ${\left[y_{(N)},y_{(N-1)},y\right]}^T~\mathcal{D}(1.59,0.42,0.31)$. The marginal distribution is calculated numerically, and the α = 0.01-level hypothesis testing results in:

$$\{\begin{array}{ll}{H}_0:\hfill & z^i(t)\ge 0.18\hfill \\ H_1:\hfill & z^i(t)<0.18\hfill \end{array}$$

where P(zⁱ(t) ≥ 0.18) > 1 − α. During the test, if zⁱ(t) falls bellow 0.18, then the recommender asks the expert for the feedback and fine-tunes the network with the feedback. Because we are evaluating the proposed model on a dataset collected in the past, we cannot ask for the expert for feedback in our evaluation. Therefore, whenever zⁱ(t) falls bellow 0.18, we take the actual exercise the test participant performed at that time step and provide it as the feedback by an expert to our active learning algorithm (Table I row 5). Comparing the results with our baseline model, the top-1 accuracy is increased by 10%. The increased in accuracy is the results of fine-tuning the recommender system with the feedback received from the expert, which makes the recommendation system personalized.

Baseline with New User Initialization:

We calculated the pairwise similarity across all participants. For each new participant, we fine-tuned the recommendation system with the training data of the three most similar participants based on their profiles. The recommendation system is initialized for the new user by the fine-tuned network. We didn’t use data augmentation and active learning in this experiment just to examine the effect of new user initialization. We see a minor improvement in top-1 accuracy (Table I row 6). However, we believe that when it comes to more diverse participants, e.g., different age groups, race, ethnicity, the proposed initialization strategy improve the accuracy significantly.

Baseline with Data Augmentation and Active Learning:

In this experiment, we combined the data augmentation and active learning as we hypothesise that the accuracy will improve. Table I row 7 shows the results. As we expected, the accuracy has improved with respect to only active learning (Table I row 5) and only augmentation (Table I row 3).

Baseline with Data Augmentation, New User Initialization, and Active Learning:

In the last experiment, we combined all modules. Table I row 9 represents the results indicating a very minor decline in accuracy by adding the new user initialization procedure (compared to Table I row 7). More study with different questionnaires, medical records, etc. may lead to increase in the accuracy of the new user initialization method.

Comparison with the state-of-the-art:

The proposed model is evaluated against some of the well-known sequential recommendation systems [53]–[56] ¹. Because the sequential models, including the proposed method, use the history of items for each users to propose the next item, they cannot be compared directly with matrix factorization approaches, e.g. SVD. The factorization approaches estimate the rating of different items without explicitly recommending the next item. However, the sequential models take into account the order of items in the history and recommend the next item without estimating the user’s rating.

To have a fair comparison, the baseline model with user demographic is used in this experiment. Table II shows the Top-k accuracy for different models. The proposed baseline model outperforms the existing state-of-the-art in top-1, top-5, top-10 accuracy. The best performance after the proposed baseline method in Table II belongs to the CNN method [55]. However, the performance of the CNN method is not as good as the proposed method. The superior performance of the proposed baseline model extends to the other variations of the proposed model, e.g. the baseline with active learning.

TABLE II.

Comparison of the Accuracy Across Different Recommendation Systems for the mHealth Data

Method	top-1 Accuracy	top-5 Accuracy	top-10 Accuracy
GRU4REC	7.89%	22.74%	40.04%
Pooling	8.83%	41.16%	50.38%
CNN	12.97%	33.83%	53.00%
Mixture	5.45%	20.48%	40.41%
Baseline (Demographic)	63.78%	92.19%	97.33%

B. Movielens

In the second experiment, the proposed method is applied to the MovieLens100 K dataset [49] and compared with the state of the art sequential recommendation systems [53]–[56]. The dataset contains 100,000 user IDs, movie IDs, the user ratings of movies, and timestamps. We group the dataset by the user IDs and sorted them with their timestamps.

1). Experiment Setup and Discussion:

Our proposed recommendation system was developed specifically for recommending exercises, when the number of exercises is limited (unlike the number of movies). However, the proposed method is evaluated on the MovieLens dataset to show its strength in scaling to other applications. To this end, we pre-processed the dataset and limited the number of movies in this dataset to the 100 most watched movies in Movielens100 K. Thus, the new subset of MovieLens100 K is more comparable to the exercises activity dataset when the number of items are limited.

Similar to the mHealth dataset, we selected the window length of w = 3 resulted in 29931 sequence samples, and are split to 80% training and 20% test samples. The architecture of the proposed recommendation system is selected empirically as follows: K_X = K_u = 1000, K_e = 3, and K = 300. Table III shows top-1, top-5, and top-10 accuracy of different recommendation systems. In recommending new movies in the subset of MovieLens100 K, the proposed method surpasses the accuracy of the state of the art methods. After the proposed method, the pooling method [54] comes second with accuracy less than the proposed baseline method.

TABLE III.

Comparison of the Accuracy Across Different Recommendation Systems for a Subset of MovieLens100 K

Method	top-1 Accuracy	top-5 Accuracy	top-10 Accuracy
GRU4REC	2.5%	12.27%	21.55%
Pooling	3.93%	18.91%	39.66%
CNN	2.26%	10.91%	20.23%
Mixture	2.47%	11.37%	21.34%
Baseline (Demographic)	5.90%	26.38%	46.32%

VI. Conclusion

In this paper, a physical exercise recommendation system was developed. The main challenge in developing recommendation systems is to make them personalized, especially for new users when the training dataset does not exist. Analyzing the outcomes of the experimental results indicates the importance of user and exercise profiles as attention mechanisms.Thus, the developed system took advantage of the health and demographic questionnaires filled out by users before joining the program to use as attention mechanism.

In spite of all fine tuning and the attention mechanism, the experimental results show that the perfect personalization cannot be achieved. While constant feedback from the user interaction can be collected in e-commerce, music and movie recommendation systems, having an expert in the loop is inevitable in exercise recommendation systems because we don’t know whether the user performed the exercise or not (while in e-commerce the user either buys the product or listen completely to a music on a music platform). The expert knowledge provides information when the recommender is uncertain and fails to make accurate predictions, which is why a real time active learning mechanism in conjunction with our developed recommendation systems showed a significant increase in the accuracy of the system.

Appendix A. Probability Distribution of The Marginal Distance

In marginal distance, the output variables are sorted in ascending order as y₍₁₎ ≤ y₍₂₎ ≤ … ≤ y_(N). The marginal distance is defined as M = y_(N) − y_(N−1). When the value of the marginal distance falls below a given threshold θ_m, the recommender asks the human expert for labeling. While in prior works the threshold was determined by users, we define the probability distribution of the marginal distance. The output variable is drawn from a Dirichlet probability distribution function, $\mathcal{D}\left({\alpha }_1,{\alpha }_2,\dots ,{\alpha }_N\right)$. In the Dirichlet distribution, the amount of the variables sum up to one: ${\sum }_{k=1}^N{y}_{(k)}=1$. In the marginal distance, we are only interested in y_(N) and y_(N−1). To simplify the calculation, we define a new random variable $y={\sum }_{k=1}^{N-2}{y}_{(k)}$. Note that the marginal distribution of y_(ks are Beta distribution and the summation of several Beta distribution, y, is a Beta distribution. The joint distribution of Y = [y_(N), y_(N−1), y]^T is a Dirichlet distribution, $\mathcal{D}\left({\alpha }_1,{\alpha }_2,{\alpha }_3\right)$:

$$f\left(y_{(N)},y_{(N-1)},y\right)=\frac{\mathrm{\Gamma }\left({\sum }_k{\alpha }_k\right)}{{\prod }_k\mathrm{\Gamma }\left({\alpha }_k\right)}{y}_{(N)}^{\left({\alpha }_1-1\right)}{y}_{(N-1)}^{\left({\alpha }_2-1\right)}{y}^{\left({\alpha }_3-1\right)}$$ (10)

where 0 ≤ y_(N),y_(N−1),y < 1, and y_(N) + y_(N−1) + y = 1. The parameters α₁, α₂ and α₃ are estimated from the training dataset using Maximum Likelihood approach [58], [59].

The probability distribution of the marginal distance is derived from transforming y_(N) and y_(N−1) based on Z = y_(N) − y_(N−1). Because the support of y_(N),y_(N−1),y is restricted to the hyper-plane defined by y_(N) + y_(N−1) + y = 1, the third argument, y, is known given y_(N) and y_(N−1). Fig. 3(a) shows the y_(N) + y_(N−1) + y = 1 hyper-plane. Because the support of y_(N), y_(N−1), y is restricted and the value of y depends on y_(N) and y_(N−1), we can just visualize the projection of the hyperplane on the y_(N) − y_(N−1) plane. This makes defining the boundaries of the integrals easier. Fig. 3(b) shows this projection with y_(N) > y_(N−1) and Z = y_(N) − y_(N−1) lines. In the first step, we derive the cumulative probability distribution F(z) = P(Z ≤ z), as in (11), shown at the top of next page. Then, to get the probability distribution function, we take the derivatives from both sides with respect to z to obtain (12), shown at the top of next page. Thus, we get the probability distribution function the marginal distance as in (13), shown at the top of next page. We calculate f(z) numerically by approximating the integral with a summation for different value of z, even though calculation of the closed form is not impossible.

$$\begin{gathered} P\left(y_{(N)}-y_{(N-1)}\le z\right)={\int }_{y_{(N-1)}=0}^{\frac{1-z}{2}}{\int }_{y_{(N)}=y_{(N-1)}}^{y_{(N-1)}+z}f\left(y_{(N)},y_{(N-1)},1-y_{(N)}-y_{(N-1)}\right)d{y}_{(N)}d{y}_{(N-1)} \\ +{\int }_{y_{(N-1)}=\frac{1-z}{2}}^{0.5}{\int }_{y_{(N)}=y_{(N-1)}}^{y_{(N-1)}+z}f\left(y_{(N)},y_{(N-1)},1-y_{(N)}-y_{(N-1)}\right)d{y}_{(N)}d{y}_{(N-1)} \end{gathered}$$ (11)

$$\begin{gathered} f(z)=\frac{-1}{2}{\int }_{y_{(N)}=\frac{1-z}{2}}^{\frac{1+z}{2}}f\left(y_{(N)},\frac{1-z}{2},1-y_{(N)}-\frac{1-z}{2}\right)d{y}_{(N)}+{\int }_{y_{(N-1)}=0}^{\frac{1-z}{2}}f\left(y_{(N-1)}+z,y_{(N-1)},1-2y_{(N-1)}-z\right)d{y}_{(N-1)} \\ +\frac{1}{2}{\int }_{y_{(N)}=\frac{1-z}{2}}^{\frac{1+z}{2}}f\left(y_{(N)},\frac{1-z}{2},1-y_{(N)}-\frac{1-z}{2}\right)d{y}_{(N)} \\ ={\int }_{y_{(N-1)}=0}^{\frac{1-z}{2}}f\left(y_{(N-1)}+z,y_{(N-1)},1-2y_{(N-1)}-z\right)d{y}_{(N-1)} \end{gathered}$$ (12)

$$f(z)=\frac{\mathrm{\Gamma }\left({\alpha }_1+{\alpha }_2+{\alpha }_3\right)}{\mathrm{\Gamma }\left({\alpha }_1\right)\mathrm{\Gamma }\left({\alpha }_2\right)\mathrm{\Gamma }\left({\alpha }_3\right)}{\int }_{y_{(N-1)}=0}^{\frac{1-z}{2}}{\left(y_{(N-1)}+z\right)}^{\left({\alpha }_1-1\right)}{y}_{(N-1)}^{\left({\alpha }_2-1\right)}{\left(1-2y_{(N-1)}-z\right)}^{\left({\alpha }_3-1\right)}d{y}_{(N-1)}$$ (13)

Fig. 3.

(a) The valid surface of y_(N),y_(N−1), y variables with the Dirichlet distribution. (b) The integral area projected on y_(N) − y_{(N−1) plane.}

Appendix B. Computational Cost

In this section, the computational cost of different modules are calculated. The computational cost of the exercise input module is $\mathcal{O}(N)$ because N > K_x. The computational cost of the User Profile input module is $\mathcal{O}\left(N_U\right)$, and that of the Exercise Profile input is $\mathcal{O}\left(N_e\right)$. The computational cost of the RNN module is $\mathcal{O}\left(K^{\omega }\right)$. K is smaller than N but not a lot smaller. So the overall computational cost is dominated by the RNN module. Thus, the overal computational cost is $\mathcal{O}\left(K^{\omega }\right)$.