A channel-independent transformer approach for ride-hailing demand prediction with internet sentiment and transport capacity

Graphical abstract

Abstract

Ride-hailing demand prediction poses significant challenges due to its inherent dynamism. Although deep learning models, notably Transformer, have shown impressive predictive capabilities, their accuracy needs further improvement, especially when confronted with multiple realistic factors in industrial practice, such as societal and consumer sentiment trends, and dynamic fluctuations in transport capacity. To tackle this challenge, this paper presents an enhanced Channel-independent Transformer (CiFormer) approach for ride-hailing demand prediction that incorporates Internet sentiment. Specifically, we propose an integrated model that combines a BERT-based architecture with BiLSTM and an attention mechanism to effectively extract Internet sentiment from textual data. This enhanced sentiment analysis is subsequently integrated into a customized CiFormer model, which concurrently incorporates multiple exogenous features, specifically transport capacity, to improve prediction accuracy. We validate our approach on a real-world dataset from the Beijing West Railway Station. Experimental results demonstrate that our approach outperforms benchmark models across multiple prediction metrics. Furthermore, the crucial impact of Internet sentiment and transport capacity on ride-hailing demand prediction is highlighted.

1. Introduction

With the rapid development of intelligent transportation systems, ride-hailing services, which connect passengers with private-car drivers via online platforms, have experienced significant growth since their inception [1], [2], [3]. According to a report by Statista, the number of ride-hailing users worldwide is expected to reach 1.97 billion by 2028.1 Predicting the demand for ride-hailing services is crucial for ride-hailing platforms. By accurately predicting demand, these platforms can more effectively utilize and allocate vehicle resources to meet passenger needs. This becomes especially critical for Large-scale Passenger Transport Hubs (LPTHs), where insufficient service vehicles can lead to a large number of passengers being stranded, posing significant safety hazards. Despite advancements in mobile Internet improving demand-supply matching, residual passenger demand persists due to inefficient demand prediction [4], [5], [6]. Therefore, ride-hailing demand prediction remains an issue of paramount importance, deserving significant attention and research efforts.

In existing research, the methods for ride-hailing demand prediction generally focus on classical statistical models, machine learning (ML) models, and deep learning (DL) models [1], [7], [8]. In the realm of traffic big data, ride-hailing demand is influenced by multiple realistic factors, exhibiting high dimensionality and nonlinearity. These characteristics present significant challenges for traditional statistical models and ML models in terms of feature extraction and model generalization. In recent years, propelled by advancements in DL, a plethora of demand prediction algorithms have emerged, notably Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) [9], [10], [11]. Despite advancements, these models still exhibit deficiencies in capturing long-range dependencies and encountering training issues like vanishing or exploding gradients. The attention mechanism addresses these shortcomings by dynamically capturing long-range dependencies and significant information within sequences, thereby enhancing model expressiveness and predictive performance. The Transformer model, a DL approach based on the multi-head attention mechanism that utilizes the encoder and decoder structure for sequence modeling [12], [13], enables effective modeling of temporal dependencies, providing a more efficient and accurate solution for sequence modeling [14], [15]. However, previous Transformer-based time series prediction models, such as Informer [16] and DLinear [17], often adopt a channel-mixing approach, where all dimensions of the multivariate time series are combined into a single vector. This approach can be problematic as it may lead to the loss of individual feature information, making it harder to capture the unique patterns of each variate and reducing the model’s ability to effectively model dependencies across different features. In contrast, the channel-independent approach [18], [19] addresses these issues by treating each variate as an independent process, preserving the individual characteristics of each feature and allowing for more effective modeling of dependencies without sacrificing feature-specific information. Therefore, we introduce a novel channel-independent approach based on Transformer architecture for ride-hailing demand prediction.

Ride-hailing demand poses significant challenges, primarily stemming from temporal and exogenous dependencies. Temporal dependencies indicate that ride-hailing demand exhibits periodic fluctuations, such as heightened demand during evening rush hours and reduced demand at midnight, and is further influenced by trends observed in recent historical ride-hailing demand. Regarding exogenous dependencies, factors such as weather [1], calendar [20], congestion degree and competitive services like subways are commonly considered, as they can significantly influence ride-hailing demand. However, transport capacity, defined as the available supply of ride-hailing services within a specified time frame, is often overlooked in prediction tasks, despite its significant impact on ride-hailing demand. This oversight stems from the reality that passengers’ decision-making regarding ride-hailing services is heavily influenced by real-time queue information and vehicle availability, highlighting the crucial role that ride-hailing transport capacity plays in shaping passenger choices.

While the aforementioned factors, characterized by numerical and categorical features, are crucial in ride-hailing demand prediction and form the foundation of current research [20], a substantial portion of textual information, including transportation policies, major emergencies, and public evaluations of ride-hailing services, is yet to be fully harnessed in the field of ride-hailing demand prediction [21]. For instance, during unforeseen large-scale public events, ride-hailing demand can undergo significant fluctuations as a result of alterations in traffic management measures and policies [22], particularly at LPTHs. Furthermore, Internet news articles and social media comments related to traffic conditions and ride-hailing services serve as pivotal elements in shaping public perception, which in turn influences individual travel behavior. In this context, it becomes evident that incorporating such textual data sources into ride-hailing demand prediction could provide valuable insights and enhance predictive accuracy. Textual information often carries sentiment polarity, which assesses the positivity or negativity embedded within the textual data. With continuous advancement in natural language processing methods, the utilization of pre-trained models such as Bidirectional Encoder Representations from Transformers (BERT) [23] enables the extraction of sentiment representation from Internet textual data. Converting textual data into numerical format further facilitates its integration as an Internet sentiment indicator into ride-hailing demand prediction models.

In summary, with the continuous expansion and evolution of the ride-hailing industry, accurately predicting ride-hailing demand emerges as a critical challenge for ensuring operational efficiency. Traditional prediction methods prove inadequate in capturing the complex dynamics of this growing demand. Furthermore, the full impact of Internet sentiment and transport capacity on ride-hailing demand prediction remains unrecognized, necessitating a comprehensive exploration. To address the shortcomings of existing research, we introduce a multi-source heterogeneous data driven ride-hailing demand prediction approach based on a Channel-independent Transformer (CiFormer) model that incorporates Internet sentiment and transport capacity. Utilizing a BERT model enhanced by Bidirectional LSTM (BiLSTM) and attention mechanism [24], [25], we perform sentiment analysis on textual data obtained via web crawling techniques, and incorporate the sentiment values into ride-hailing demand prediction. Additionally, we consider a novel feature, transport capacity, to investigate its impact on ride-hailing demand prediction. For prediction model, we employ the CiFormer model tailored for this multivariate ride-hailing demand prediction task. The main contributions of this paper are threefold:

• (1)We present an integrated model that combines BERT, BiLSTM, and attention mechanisms to extract Internet sentiment, which is then incorporated into a customized CiFormer model, utilizing both Internet sentiment and transport capacity to enhance ride-hailing demand prediction. Experimental results on the real-world dataset demonstrate that our approach achieves state-of-the-art performance, with MAE of 6.31, RMSE of 10.57, and MAPE of 19.74, respectively.
• (2)As confirmed by our experiments, transport capacity significantly enhances the accuracy of ride-hailing demand prediction. Upon incorporating transport capacity into the model, we observed notable improvements, with MAE, RMSE, and MAPE decreasing by 36.35%, 33.05%, and 13.49%, respectively.
• (3)By leveraging the advanced BERT-BiLSTM-Attention method, we efficiently extract Internet sentiment from textual data and seamlessly integrate it into ride-hailing demand prediction. Experimental findings reveal that this integration leads to further reductions in MAE, RMSE, and MAPE by 3.37%, 4.77%, and 3.80%, respectively, building upon the improvements already gained from considering transport capacity.

The rest of this paper is organized as follows. In Section 2, we present a literature review examining prior research exploring ride-hailing demand prediction methods and external factors influencing ride-hailing demand. In Section 3, we present the problem statement and our methodology, including the BERT-BiLSTM-Attention method and the CiFormer model. Section 4 presents the dataset description, prediction results, and performance comparisons on a real-world dataset. Ultimately, in the final section, we present the conclusions and future research directions.

2. Related literature

We have undertaken a thorough review of the existing literature, focusing on two main streams of research related to ride-hailing demand prediction: (i) methods for predicting ride-hailing demand; (ii) exogenous factors that influence ride-hailing demand.

2.1. Methods for predicting ride-hailing demand

Ride-hailing demand data typically exhibit time series characteristics, and prediction approaches can be broadly categorized into three types: those based on statistical analysis, those rooted in machine learning principles, and deep learning models that leverage deep neural networks [26], [27], [28], [29], [30].

Statistical methods encompass classical techniques such as Historical Average (HA) and parametric approaches like Autoregressive Integrated Moving Average (ARIMA) [20], [31], which are commonly used for traffic demand prediction. However, these models have inherent limitations, including their inability to effectively capture complex nonlinear dependencies between dependent and independent variables, as well as their sensitivity to data quality due to inherent architectural characteristics. While parametric approaches are suitable for addressing regular variations, their performance tends to be suboptimal when dealing with traffic data exhibiting significant stochastic and nonlinear characteristics.

In comparison to statistical models, ML models excel at handling complex, high-dimensional data and capturing intricate nonlinear relationships. ML algorithms gain significant traction in traffic demand prediction due to their ability to improve prediction accuracy while overcoming the limitations associated with statistical methods. Multiple linear regression [20] is a fundamental ML method used to establish and analyze the relationship between multiple independent variables and a dependent variable. In contrast, decision tree models [32] utilize a hierarchical, tree-like structure to define decision rules for prediction tasks. To further improve predictive performance, ensemble methods like Random Forest, which combines multiple decision trees, as well as gradient boosting frameworks such as Light Gradient Boosting Machine (LightGBM) and eXtreme Gradient Boosting (XGBoost), are applied in traffic demand prediction research [33], [34]. However, ML models face challenges when handling large-scale, high-dimensional datasets and suffer from limited generalization ability, especially when dealing with complex nonlinear relationships in traffic data.

DL models demonstrate exceptional performance in handling large-scale and high-dimensional data [35], [36], thanks to their potent parameterization capability and distributed representation capacity [37]. This enables DL models to effectively manage substantial datasets and intricate features, showcasing robust generalization prowess. Models such as RNN, LSTM, and BiLSTM have been shown to achieve promising predictive performance, as validated by several scholars [20], [38], [39], [40]. However, these models still exhibit drawbacks such as gradient explosion and vanishing, difficulties in capturing long-range dependencies, and challenges in parallel computation. The introduction of attention mechanisms [41], [42], [43] has mitigated these drawbacks and improved sequence modeling and information processing capabilities for DL models. Several studies show that incorporating attention mechanisms enhances predictive models for ride-hailing demand prediction [22], [44], [45]. The emergence of the Transformer model [12], which pioneered the self-attention mechanism [46], marked a significant milestone. This breakthrough design liberated models from reliance on traditional recursive structures, empowering them to effectively process longer and more intricate data. However, Transformer-based forecasters typically embed multiple variables from the same timestamp into indistinguishable channels, treating them as a single token, and apply attention to these temporal tokens to model temporal dependencies. This is a channel-mixing approach, where the distinctions between variables are blurred, potentially weakening the model’s ability to capture essential series representations and multivariate correlations. Thus, we innovatively applies the channel-independent approach based on Transformer architecture to the ride-hailing demand prediction.

2.2. Exogenous factors that influence ride-hailing demand

Ride-hailing demand prediction poses significant challenges primarily due to temporal dependencies and exogenous influences [47]. Given the inherent time series nature of ride-hailing demand data, incorporating temporal features is essential. For instance, holidays and weekends have already been introduced in existing research studies. Moreover, accounting for passenger travel demand in relation to natural factors such as weather, rainfall, or temperature is crucial. Previous studies, such as the one conducted by Kim et al. [20], revealed that ride-hailing demand exhibits distinct characteristics under various weather conditions, providing insights into how different weather scenarios impact usage patterns. Additionally, competitive services, especially buses and subways, are crucial components of urban transportation. When buses and subways are unavailable, passengers may turn to ride-hailing services, creating a partial substitution effect. Hence, it should be taken into account in ride-hailing demand prediction. Moreover, traffic congestion significantly impacts the demand for ride-hailing services, as congestion in ground transportation may lead passengers to choose alternative modes of transport, thereby changing the demand for ride-hailing services.

Beyond the factors mentioned earlier, there exists an overlooked category of data known as ride-hailing transport capacity, which has yet to receive significant attention. Generally, when ride-hailing capacity is sufficient, passengers are inclined to choose ride-hailing service. However, when insufficient capacity leads to prolonged unmet demand, passengers tend to choose alternative modes of transportation. Practically, ride-hailing platforms maintain consistent short-term service capacity and dispatching capabilities. This suggests a correlation between ride-hailing demand and transport capacity fluctuations. Integrating transport capacity as a predictive variable allows us to assess its impact on ride-hailing demand prediction, highlighting its role in operational stability and efficient management of these platforms.

Indeed, while numerical and categorical features are crucial for ride-hailing demand prediction, the significance of textual data with sentiment polarity tends to be overlooked [48]. The sentiment expressed across Internet news, transportation policies, and social media comments significantly influences passenger attitudes toward ride-hailing services. The practical application of textual information in transportation research has already demonstrated tangible utility and is a notable aspect within scholarly discourse [49], [50], [51]. However, previous research primarily focuses on exploring the textual content itself. For instance, Febriana et al. [52] investigate traffic conditions by analyzing text data collected from Twitter, illustrating a prevalent approach of leveraging text-based information to study transportation dynamics. Advancements in text mining techniques reach a level of sophistication that enables the accurate extraction of Internet sentiment polarity from textual data, making them suitable for incorporation as quantitative indicators within prediction modeling. However, it is worth noting that scholars in ride-hailing prediction make limited use of Internet sentiment in prediction models, highlighting a gap in integrating textual data with sentiment polarity in existing methodologies.

In summary, the impact of Internet sentiment and transport capacity on ride-hailing demand prediction has yet to be fully recognized. Internet sentiment serves as a barometer of emotional inclinations towards ride-hailing services, reflecting societal and consumer sentiment trends. Meanwhile, transport capacity provides a quantitative measure of the availability of ride-hailing services, yet its comprehensive influence on demand prediction remains largely unexplored.

3. Methodology

In this study, we focus on the ride-hailing demand prediction at an LPTH. Suppose that the historical dataset consists of $H$ days of ride-hailing demand data, transport capacity data and exogenous features data, and each day is partitioned into $I$ time periods. The ride-hailing demand data records the number of ride-hailing service orders, denoted by $D=\{d_i^h,h=1,2,\dots ,H,i=1,2,\dots ,I\}$, the transport capacity data denoted by $C=\{c_i^h,h=1,2,\dots ,H,i=1,2,\dots ,I\}$, and the exogenous features data includes calendar, weather, temperature, competitive service, congestion index, and Internet sentiment, denoted by $E=\{{\mathit{\text{e}}}_i^h,h=1,2,\dots ,H,i=1,2,\dots ,I\}$. Under the rolling prediction setting with a fixed size window, we have the sequence of historical observations spanning $T$ time steps with $N$ variates, denoted as ${\mathit{X}}_{1:T}=\{{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_T\}\in {\mathbb{R}}^{T\times N}$. ${\mathit{x}}_t=(d_t,c_t,{\mathit{\text{e}}}_t),t=1,2,\dots ,T$ is a concatenation of three vectors, where $d_t$, $c_t$ and ${\mathit{\text{e}}}_t$ represent the ride-hailing demand, transport capacity, and exogenous features vector at $t$th time period, respectively. For convenience, we denote ${\mathit{X}}_{t,:}$ as the set of simultaneously recorded time points at step $t$, and ${\mathit{X}}_{:,n}$ as the entire time series for each variate indexed by $n$ [18]. Our goal is to predict the future ride-hailing demand ${\mathit{Y}}_{T+1:T+S}=\{{\widehat{d}}_{T+1},\dots ,{\widehat{d}}_{T+S}\}\in {\mathbb{R}}^S$, where $S$ is the desirable horizon. The ride-hailing demand prediction task takes the following form

$${\mathit{Y}}_{T+1:T+S}=\mathcal{F}({\mathit{X}}_{1:T};\Phi ),$$ (1)

where $\mathcal{F}$ is the prediction model and $\Phi$ is the learnable parameters of the model.

For the problem described above, we propose a multi-source heterogeneous data driven approach to ride-hailing demand prediction. Our approach consists of two modules: (1) feature engineering and (2) prediction with multiple features. In the module of feature engineering, we apply different methodologies to three distinct types of features. For categorical features like “Calendar”, we utilize one-hot encoding. This transformation converts each possible value of a categorical feature into a binary vector consisting of 0 and 1. Each value corresponds to an element, where 1 indicates the presence of that value in the sample and 0 indicates its absence. This encoding method preserves the mutual exclusivity between categorical features and does not introduce any ordinal relationship. For numerical features like “Transport capacity”, in order to eliminate scale differences in input data and enhance the stability and convergence speed of model training, we normalize the data using the min-max method [53] as follows

$$x_{\text{norm}}=\frac{x-x_{\text{min}}}{x_{\text{max}}-x_{\text{min}}},$$ (2)

where $x$ represents the original data, $x_{\text{norm}}$ represents the normalized data, and $x_{\text{min}}$ and $x_{\text{max}}$ are the minimum and maximum values of the original data, respectively.

For the textual feature “Internet sentiment”, we design a BERT-BiLSTM-Attention hybrid method to extract sentiment polarity from textual data as numerical representations. In the module for predicting ride-hailing demand with multiple features, we employ a CiFormer model, which can process each channel (variate) independently when dealing with multi-channel inputs, and then fuse information across channels through the attention mechanism of the Transformer [18], [19]. This design enhances the model’s flexibility in handling variates, facilitating the capture of intricate relationships between different features. Subsequently, we provide detailed explanations of the BERT-BiLSTM-Attention method and the CiFormer model. Table 1 summarizes the model parameters and their descriptions.

Table 1.

Model parameters and description.

Parameters	Description
${\mathit{X}}_E\left(n\right)$	The embedded representation of the $n$th variate sequence
${\mathit{X}}_E$	The embedded representations of all input variates
${\mathit{X}}_I$	The output of multi-channel attention Transformer module
$\mathit{Q}$	The query matrix
$\mathit{K}$	The key matrix
$\mathit{V}$	The value matrix
$W_q$	The learnable parameter matrix of $\mathit{Q}$
$W_k$	The learnable parameter matrix of $\mathit{K}$
$W_v$	The learnable parameter matrix of $\mathit{V}$
$d_k$	The dimension of projection
$A_{i,j}$	Attention score between variates $i$ and $j$
${\mathit{q}}_i$	Query vector of the $i$th variate
${\mathit{k}}_j$	Key vector of the $j$th variate
$z_1,\dots ,z_n$	Pre-trained word and character vectors from BERT layer
$l_1,\dots ,l_n$	The output sequence of BiLSTM layer
$\overrightarrow{h_1},\cdots ,\overrightarrow{h_n}$	Forward hidden state sequence of BiLSTM layer
$\overleftarrow{h_n},\dots ,\overleftarrow{h_1}$	Backward hidden state sequence of BiLSTM layer

3.1. Extracting internet sentiment: BERT-BiLSTM-attention method

A substantial amount of textual data pertinent to ride-hailing demand can be found on the Internet. Prominent examples of this textual data encompass discussions about transportation policies, reports detailing major public health emergencies like influenza, user comments on social media platforms such as Weibo, and news articles focused on ride-hailing services. These textual sources embed sentiment that can be utilized for predictive analysis. To integrate Internet sentiment into ride-hailing demand prediction, it is necessary to convert textual data into a numerical format. The task involves three main processes: textual data collection, text preprocessing, and sentiment polarity computation. The overall framework is presented in Fig. 1.

Fig. 1.

BERT-BiLSTM-Attention method for Internet sentiment analysis.

The initial step is to collect textual data using web crawling techniques from Internet. Once collected, the data undergoes formatting for analysis, which includes text cleaning, tokenization, and embedding. Text cleaning primarily involves handling abnormal data, eliminating duplicate information, and removing irrelevant data. Using the WordPiece tokenizer for text tokenization involves constructing a vocabulary based on the most frequent character sequences found in the text, and segmenting the text into words or subwords. The input embeddings are the sum of three components: token embeddings, segment embeddings, and position embeddings. In this context, token embeddings map each token in the input text to a high-dimensional vector, segment embeddings encode the assignment of tokens to individual sentences, allowing the model to distinguish between different sentences in the input text, and position embeddings represent the positional encodings of each word within its sentence, helping the model understand the contextual relationships between words.

Constructing textual feature relies on computing sentiment polarity. In this study, we quantify the sentiment polarity of textual data via the BERT-BiLSTM-Attention method. Firstly, the input text undergoes processing through a BERT layer to extract profound textual representations, which are then fed into a BiLSTM layer to effectively capture relationships within the text. Finally, an Attention layer is employed to focus on crucial aspects of the text, enhancing the model’s ability to discern important information. After obtaining sentiment scores for text classified by date, the daily sentiment value is determined by calculating the average value from all the collected text data on that specific day. This approach results in textual feature with temporal characteristics, enabling a comprehensive analysis of sentiment trends over time.

BERT Layer: As a pre-trained Transformer model with the self-attention mechanism, BERT [23] excels at acquiring deep representations of the text from extensive corpora. The core of the BERT model lies in its bidirectional Transformer encoders, which leverage self-attention mechanism and feed-forward neural network layers for feature extraction. In our study, we utilize Google’s pre-trained BERT-Base-Chinese model, which comprises a 12-layer Transformer with a hidden layer dimension of 768 and 12 heads in the multi-head attention mechanism, resulting in a total of 110 million parameters. Once the model is loaded, it can directly output the pre-trained word and character vectors $z_1,\dots ,z_n$, which serves as inputs for the subsequent BiLSTM layer.

BiLSTM Layer: BiLSTM consists of a forward LSTM and a backward LSTM, enabling independent learning of the left and right context information of each word in the sequence. The BiLSTM layer takes the word vectors $z_1,\dots ,z_n$ from the BERT layer as input. The forward LSTM sequentially reads vectors from $z_1,\dots ,z_n$ and computes forward hidden state sequence $\overrightarrow{h_1},\cdots ,\overrightarrow{h_n}$. Simultaneously, the backward LSTM obtains a backward hidden state sequence $\overleftarrow{h_n},\dots ,\overleftarrow{h_1}$. By integrating forward and backward information, the BiLSTM layer captures both left and right context relationships, providing a more comprehensive input representation $l_1,\dots ,l_n$ for the subsequent Attention layer.

Attention Layer: The attention mechanism is pivotal in facilitating the learning process by enabling it to selectively focus on specific segments of the input text. The Attention layer receives the output $l_1,\dots ,l_n$ from the BiLSTM layer as its input, enhancing its importance in the final representation.

Finally, we utilize the sigmoid function to obtain the sentiment polarity, as follows

$$\sigma \left(x\right)=\frac{1}{1+e^{-x}},$$ (3)

where $x$ is the input from the Attention layer’s output and $\sigma \left(x\right)$ is the resulting sentiment polarity score. The closer $\sigma \left(x\right)$ is to 0, the more negative the sentiment, and the closer $\sigma \left(x\right)$ is to 1, the more positive the sentiment.

3.2. Predicting ride-hailing demand: channel-independent transformer model

Previously, Transformer-based models for multivariate forecasting tasks commonly employ a channel-mixing approach, where the vector formed by all dimensions of the multivariate time series at a single time point (analogous to a multi-channel signal) is directly projected into an embedding space to mix and fuse information from multiple channels. In contrast, the channel-independent approach implies that each input token contains information from only a single channel [18], [19]. Fig. 2 compares the differences between the channel-mixing and channel-independent approaches.

Fig. 2.

Comparison between the channel-mixing and channel-independent approaches[18].

In this subsection, we adopt a CiFormer model to predict ride-hailing demand based on multiple features. The overall architecture of our model, as shown in Fig. 3, consists of three components: the feature token embeddings layer, the multi-channel attention Transformer module, and the linear mapping layer. The feature token embeddings layer transforms the input data ${\mathit{X}}_{:,n}$ into a $d_{model}$-dimensional vector, denoted as ${\mathit{X}}_E\left(n\right)=\text{E}\left({\mathit{X}}_{:,n}\right)$. The resulting vectors after embedding all variates are collectively referred to as ${\mathit{X}}_E$. Subsequently, the vectors after the feature token embeddings are processed through the multi-channel attention Transformer module for training, with the output denoted as ${\mathit{X}}_I=\text{Trm}\left({\mathit{X}}_E\right)$. Finally, the prediction results are output through a linear mapping layer, denoted as ${\mathit{Y}}_{T+1:T+S}=\text{Linear}\left({\mathit{X}}_I\right)$.

Fig. 3.

The overall architecture of CiFormer model for ride-hailing demand prediction.

The multi-channel attention Transformer module is composed of identical Transformer blocks. Each block consists of two layers: a multi-channel attention mechanism layer and a fully connected feed-forward layer. The residual connection [54] is incorporated around each of these two layers, followed by layer normalization [18], [55].

• •
  Multi-channel attention mechanism: The self-attention mechanism, central to the original Transformer model, is commonly used to model temporal dependencies. In contrast, we adopt a multi-channel attention mechanism, which treats each variate’s entire series as an independent process, enabling better handling of individual variates. As shown in Fig. 4, the multi-channel attention mechanism operates on the tuple input (query, key, value) and performs a scaled dot-product. An attention function is used to transform a query vector and a set of key-value pair vectors into an output, as follows

  $$\text{Attention}(\mathit{Q},\mathit{K},\mathit{V})=\text{softmax}\left(\frac{\mathit{Q}{\mathit{K}}^T}{\sqrt{d_k}}\right)\mathit{V}$$ (4)

  where $\mathit{Q}\in {\mathbb{R}}^{N\times d_k},\mathit{K}\in {\mathbb{R}}^{N\times d_k},\mathit{V}\in {\mathbb{R}}^{N\times d_k}$, and $d_k$ is the dimension of projection [12]. The matrices $\mathit{Q}$, $\mathit{K}$, and $\mathit{V}$ are derived from the same input features by projecting them into distinct latent subspaces through learnable feedforward neural networks [13]. This process can be formulated as follows

  $$\{\begin{array}{c}\mathit{Q}={\mathit{X}}_E{W}_q\hfill \\ \mathit{K}={\mathit{X}}_E{W}_k\hfill \\ \mathit{V}={\mathit{X}}_E{W}_v\hfill \end{array}$$ (5)

  where ${\mathit{X}}_E$ is the input vector, and $W_q$, $W_k$, and $W_v$ are the learnable parameters of distinct feedforward networks. Specifically, for each time token of history observations, attention weights are computed to indicate the relevance of other tokens, enabling the aggregation of information based on these weights.
  In the multi-channel attention mechanism, ${\mathit{q}}_i,{\mathit{k}}_j\in {\mathbb{R}}^{d_k}$ represent the specific query and key vectors corresponding to a feature token. The attention score between variate $i$ and $j$ is calculated using the formula

  $$A_{i,j}=\frac{{\mathit{q}}_i^{\top }{\mathit{k}}_j}{\sqrt{d_k}}\propto {\mathit{q}}_i^{\top }{\mathit{k}}_j$$ (6)

  where the dot product ${\mathit{q}}_i^{\top }{\mathit{k}}_j$ measures the similarity between the two variates, and the scaling factor $\frac{1}{\sqrt{d_k}}$ is used to prevent instability caused by excessively large values from the dot product. The resulting score matrix $A\in {\mathbb{R}}^{N\times N}$ captures the pairwise correlations between all $N$ variates. Each token (i.e., a feature in the time series) is normalized on its embedding dimensions, and as a result, the dot product ${\mathit{q}}_i^{\top }{\mathit{k}}_j$ can be used to reflect the correlation between variates.
• •Feed-forward network: The original Transformer employs a feed-forward network as a core component to encode token representations, consistently applying it to each token. However, as previously noted, various channels associated with the same timestamp, which constitute the token, can become misaligned or excessively localized, thereby constraining their capacity to furnish adequate information for precise predictions. Therefore, we utilize the feed-forward network to extract the time series representation of each channel. According to the universal approximation theorem [56], they are capable of capturing complex representations to characterize a time series. This approach has been widely proven to be effective [18], [19], [57].
• •
  Layer normalization: Layer Normalization [55] is designed to accelerate network convergence and improve the training stability of deep networks. In Transformer-based forecasters, layer normalization is used to fuse multi-channel representations at the same timestamp, but mismatched time points can lead to interaction noise from noncausal or delayed processes. Therefore, we adopt the layer normalization to the series representation of each individual channel. The calculation process is shown as follows

  $$\text{LayerNorm}\left({\mathbf{X}}_L\right)=\frac{{\mathbf{X}}_L^n-\mu \left({\mathbf{X}}_L^n\right)}{\sigma \left({{\mathbf{X}}_L}^n\right)},n=1,\cdots ,N$$ (7)

  where ${\mathbf{X}}_L^n$ represents the intermediate embedding in the $L$th Transformer block.

Fig. 4.

Multi-channel attention mechanisim[12].

3.3. Experimental setup

Data: To generate a labeled dataset, we employ a fixed-length sliding time window approach to construct $X,Y$ pairs for model training and evaluation [58], as illustrated in Fig. 5. Prior to implementing the sliding window to extract features and labels, we adopt min-max scaling to normalize the dataset and divide it into training, validation, and test sets. Subsequently, we apply a sliding window to the standardized training set to obtain training samples consisting of variates and corresponding labels. In this context, the variates comprise the preceding $T$ observations, while the labels consist of the demand over the subsequent $S$ time steps.

Fig. 5.

Using sliding window to construct samples from ride-hailing demand dataset.

Optimizer: We use the Adam optimizer [59] with ${\beta }_1=0.9$, ${\beta }_2=0.99$, and $ϵ={10}^{-8}$. The learning rate is varied over the course of training according to the formula

$$l_{rate}=d_{model}^{-0.5}·\min ({\text{step}\_\text{num}}^{-0.5}\text{step}\_\text{num}·w\_ste{p}^{-1.5})$$ (8)

where $w\_steps=4000$. This dynamic learning rate adjustment strategy allows the model to adapt its learning rate based on the training progress, ensuring effective convergence and optimal performance.

Regularization: We apply a dropout rate of 0.1 to each sub-layer for regularization. Dropout regularization helps prevent overfitting by randomly setting a fraction of input units to zero during training, effectively introducing noise into the model and reducing its reliance on specific features.

Hyperparameters: We implement a computationally efficient channel-independent Transformer architecture consisting of 3 stacked Transformer blocks [18]. The embedding dimension is set to 512 [12]. The training protocol utilizes a batch size of 32 for 50 maximum epochs, with early stopping regularization.

Loss function: The predictive model is trained via the Mean Squared Error (MSE) as the loss function. The MSE is a widely employed metric in regression tasks, serving to quantify the average squared difference between the predicted values and the ground truth labels. Mathematically, the MSE is computed as the mean of the squared differences between each prediction and its corresponding actual value, as follow

$$MSE=\frac{1}{N}\sum _{i=1}^N{(y_n-{\widehat{y}}_n)}^2$$ (9)

where $N$ is the number of instances, $y_n$ represents the true value for instance $i$, and ${\widehat{y}}_n$ denotes the predicted value for the same instance.

4. A real-word application

This section presents an in-depth case study to evaluate the prediction performance of our approach. The experiments were implemented with Python 3.9 on desktop computer with Intel® Core^TM i7-12700 CPU, 16 GB memory, and an NVIDIA GeForce RTX3080 GPU. All experiments are imlemented in Pytorch [60].

4.1. Dataset description

The dataset used for our experiments comprises ride requests originating from the Beijing West Railway Station (BWRS), collected at 15-min intervals over a period spanning from January 1, 2021 to July 31, 2022. In time series prediction, determining the optimal lag order is crucial. The Phase Space Reconstruction (PSR) method [61] offers a coordinate delay approach to identify this optimal lag order. This method reconstructs time series into multidimensional vectors with specified delay times, addressing the complexity and nonlinearity inherent in time series data [62]. Using the PSR method, we determined the optimal lag order to be $T=7$. This means that the future ride-hailing demand for a 15-min horizon is predicted using the previous 7 time periods of observations, which is equivalent to setting the sliding window size to 7. For data preprocessing, we identified missing values and filled them by mean imputation. The dataset was then normalized and divided into a training set (75$\%$), a validation set (15$\%$), and a testing set (10$\%$).

Table 2 presents an overview of the concerned features. The calendar, weather, temperature, and congestion index are all obtained from the Internet. Specifically, the feature “Congestion index” is collected from the Baidu Traffic Big Data Platform and represents the congestion degree of the road network surrounding BWRS. The data for “Competitive service” and “Transport capacity” are obtained from the transportation management authorities. The “Internet sentiment” is derived from textual data, and the corresponding data sources along with their quantities are presented in Table 3. To ensure the authenticity of the textual data, all of it, except for social media comment data, which is collected from Micro-blog, has been gathered from government official websites and authoritative media platforms. Additionally, we employ keyword searches to ensure the relevance of the textual data to the research topic. We first conduct keyword searches, such as “ride-hailing service”, “ride-hailing policy”, “influenza”, “traffic policy”, and so on. Subsequently, we collect and process the data based on the results of these searches. The construction process for the feature of “Internet sentiment” is shown in Fig. 6. Since the impact of textual information is subject to temporal lag, we use the sentiment score from the previous day as the input feature.

Table 2.

Description of concerned features.

Feature	Description
Is holiday	Take value 1 if the day is a holiday, 0 otherwise
Is weekend	Take value 1 if the day is a weekend, 0 otherwise
Weekly	Weekly working day (any day from Monday to Friday)
Is raining	Take value 1 if it rains, 0 otherwise
Rainfall	Rainfall of the day
Temperature	Temperature of the day
Competitive service	Take value 1 if the subway is in regular service, 0 otherwise
Congestion index	The daily traffic congestion index of BWRS
Transport capacity	The available quantity of ride-hailing vehicles for the previous time period at BWRS
Internet sentiment	Internet sentiment polarity derived from textual data of the previous day

Table 3.

Textual data sources and quantity.

Textual data sources	Textual data categories	Collected quantities
Government Information Disclosure Website of the Ministry of Transport	Transportation policies	516
Chinese Government’s Official Web Portal
Xinhua News Agency’s Website
Official Website of the Beijing Municipal Commission of Transport
Official Website of the People’s Government of Beijing Municipality
People’s Daily Online Website	Influenza reports	1852
Chinese Government’s Official Web Portal
Official Website of National Health Commission of the People’s Republic of China
Micro-blog
The Website of China Central Television
Micro-blog	Social media comments	6581
People’s Daily Online Website	News articles of ride-hailing services	298
Baidu
SOHU
The Website of China Central Television
Xinhua News Agency’s Website

Fig. 6.

The feature construction process of Internet Sentiment.

4.2. Prediction results and performance comparisons

The proposed CiFormer model is compared with several selected models across different categories: the time series models (ARIMA), ML models (Random Forest, LightGBM, and XGBoost), and DL models (LSTM, BiLSTM, and BiLSTM+Attention, Transformer, DLinear, Informer). Specifically, the Transformer model adopts an encoder-only architecture. The prediction error, defined as the difference between the actual and predicted values, is closely related to the prediction model’s effectiveness. To evaluate the models, we typically predict the test set and then assess the errors using various evaluation criteria. In this study, we use three criteria to evaluate the accuracy of the prediction models: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE). These criteria are defined as follows

$$MAE=\frac{1}{M}\sum _{m=1}^M|y_m-{\widehat{y}}_m|$$ (10)

$$RMSE=\sqrt{\frac{1}{M}{\sum }_{m=1}^M{(y_m-{\widehat{y}}_m)}^2}$$ (11)

$$MAPE=\frac{1}{M}\sum _{m=1}^M\left|\frac{y_m-{\widehat{y}}_m}{y_m}\right|\times 100\%$$ (12)

where $y_m$ represents the actual value, ${\widehat{y}}_m$ represents the predictive value, and $M$ denotes the scale of the testing set.

We examined the prediction results of all models on the real-world dataset under four distinct feature patterns. As delineated in Table 4, Pattern 1 acts as the benchmark for feature selection, with the integration of Transport capacity in Pattern 2. Pattern 3 introduces Internet sentiment, while Pattern 4 consolidates all considered features. To test the performance of the CiFormer model compared to other models, we conduct separate tests within these four patterns.

Table 4.

Four distinct feature patterns.

	Is holiday	Is weekend	Weekly	Is raining	Rainfall	Temperature	Competitive service	Congestion index	Transport capacity	Internet sentiment
Pattern 1	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$
Pattern 2	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$
Pattern 3	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$		$✓$
Pattern 4	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$	$✓$

Table 5 presents the comprehensive prediction results, demonstrating that our CiFormer model achieves the state-of-the-art performance across all feature patterns compared to baseline models. Specifically, across Patterns 1, 2, 3, and 4, the CiFormer model demonstrates predictive performance with MAE values of 10.26, 6.53, 9.88 and 6.31, RMSE values of 16.58, 11.10, 16.28 and 10.57, and MAPE values of 23.72%, 20.52%, 22.85% and 19.74%, respectively. As our approach primarily concentrates on the predictive performance after incorporating Transport capacity and Internet sentiment, we pay particular attention to the results of Pattern 4. In Table 5, Benchmark_mean and Benchmark_min represent the mean and minimum values of the benchmark models, respectively. Improvements_mean and Improvements_min indicate the magnitude of prediction accuracy improvement of the CiFormer model compared to the benchmark models. Compared to the mean values of the benchmark models, the CiFormer model’s prediction accuracy shows improvements of 48.14%, 53.61%, and 68.05% in terms of MAE, RMSE, and MAPE, respectively. Relative to the best-performing benchmark models (i.e., minimum values), the prediction accuracy has improved by 9.21%, 8.25%, and 7.97% in terms of MAE, RMSE, and MAPE, respectively. In addition to its high prediction accuracy, the CiFormer model also demonstrates strong computational efficiency, with both training speed and inference latency at acceptable levels. The model contains 9.55 million trainable parameters, with a per-epoch training time of 110 s and a total training duration of 92 minutes-primarily due to its use of a longer input sequence compared to the standard Transformer. Its inference latency, measured as the time required to process a single test sample, is only 2 milliseconds, making CiFormer highly suitable for practical prediction applications.

Table 5.

Comparison of prediction results of all models.

Model	Pattern 1			Pattern 2			Pattern 3			Pattern 4
	MAE	RMSE	MAPE (%)	MAE	RMSE	MAPE (%)	MAE	RMSE	MAPE (%)	MAE	RMSE	MAPE (%)
ARIMA	39.28	59.96	263.53	39.28	59.96	263.53	39.28	59.96	263.53	39.28	59.96	263.53
Random Forest	14.02	30.89	51.77	8.29	19.55	31.27	13.33	29.84	49.97	8.02	18.64	29.98
LightGBM	11.47	21.11	52.83	7.34	13.68	35.75	11.29	21.01	51.88	7.23	13.52	34.79
XGBoost	12.48	26.08	52.94	7.82	19.54	33.02	11.98	25.92	51.65	7.58	19.16	32.07
LSTM	16.58	31.53	84.29	14.47	30.34	85.98	15.57	30.50	87.30	14.43	30.59	85.60
BiLSTM	16.19	30.77	86.22	13.78	28.55	81.93	15.45	29.45	80.73	13.40	27.57	80.90
BiLSTM+Attention	13.24	22.95	37.32	10.31	19.22	24.85	12.98	21.03	36.94	10.02	18.84	23.91
Transformer	11.29	19.34	26.27	7.25	12.88	22.97	11.15	19.01	26.03	7.08	11.87	22.06
DLinear	10.98	18.11	25.83	7.03	11.96	21.99	10.61	18.06	25.19	6.95	11.52	21.45
Informer	12.41	23.57	29.67	8.11	16.74	24.26	12.02	23.05	29.44	7.69	16.18	23.64
CiFormer	10.26	16.58	23.72	6.53	11.10	20.52	9.88	16.28	22.85	6.31	10.57	19.74
Benchmark_mean	15.79	28.43	71.07	12.37	23.24	62.56	15.37	27.78	70.26	12.17	22.79	61.79
Benchmark_min	10.98	18.11	25.83	7.03	11.96	21.99	10.61	18.06	25.19	6.95	11.52	21.45
Improvements_mean	35.04%	41.68%	66.62%	47.20%	52.24%	67.20%	35.70%	41.40%	67.48%	48.14%	53.61%	68.05%
Improvements_min	6.56%	8.45%	8.17%	7.11%	7.19%	6.68%	6.88%	9.86%	9.29%	9.21%	8.25%	7.97%

In order to investigate the effect of incorporating different features in the ride-hailing demand prediction, we compare the predictive results of the CiFomer model across four feature patterns. As shown in Table 6, the predictive performance under Pattern 2, which includes the feature of Transport capacity, resulted in notable improvements compared to Pattern 1. This is evidenced by reductions of 36.35%, 33.05%, and 13.49% in MAE, RMSE, and MAPE, respectively. Furthermore, the comparison between Pattern 2 and Pattern 4 indicates that incorporating Internet sentiment resulted in further reductions of 3.37%, 4.77%, and 3.80% in MAE, RMSE, and MAPE, respectively. Pattern 4, which encompasses all features, demonstrates the highest prediction accuracy, representing the pinnacle of predictive performance.

Table 6.

Prediction performance of CiFormer model for different feature patterns.

	MAE	RMSE	MAPE (%)		MAE	RMSE	MAPE (%)
Pattern 1	10.26	16.58	23.72	Pattern 2	6.53	11.10	20.52
Pattern 2	6.53	11.10	20.52	Pattern 4	6.31	10.57	19.74
Improvements	36.35%	33.05%	13.49%	Improvements	3.37%	4.77%	3.80%

To further assess the impact of the channel-independent and channel-mixing approaches on ride-hailing demand prediction, we conduct an ablation study. As shown in Table 7, after introducing the channel-independent approach, the prediction errors are significantly reduced across the four feature patterns. Specifically, the MAE decreases by 9.12%, 9.93%, 11.39%, and 10.88%, and the reductions of RMSE are 14.27%, 13.82%, 14.36%, and 10.95%, while the MAPE decreases by 9.71%, 10.67%, 12.22%, and 10. 51%. These improvements highlight the effectiveness of the channel-independent approach in enhancing ride-hailing prediction accuracy.

Table 7.

Ablation experiment of the channel-independent approach.

	Pattern 1			Pattern 2			Pattern 3			Pattern 4
	MAE	RMSE	MAPE (%)	MAE	RMSE	MAPE (%)	MAE	RMSE	MAPE (%)	MAE	RMSE	MAPE (%)
Channel-mixing	11.29	19.34	26.27	7.25	12.88	22.97	11.15	19.01	26.03	7.08	11.87	22.06
Channel-independent	10.26	16.58	23.72	6.53	11.10	20.52	9.88	16.28	22.85	6.31	10.57	19.74
Improvements	9.12%	14.27%	9.71%	9.93%	13.82%	10.67%	11.39%	14.36%	12.22%	10.88%	10.95%	10.51%

5. Conclusion

This paper presents an innovative channel-independent Transformer approach to ride-hailing demand prediction with incorporating Internet sentiment and transport capacity. In order to extract Internet sentiment from textual data, we devise the BERT-BiLSTM-Attention hybrid method. For the ride-hailing demand prediction, we discard the channel-mixing approach and adopt a novel channel-independent Transformer model to enhance the performance of ride-hailing demand prediction. To validate the effectiveness of our approach, we conducted extensive experiments on a real-world dataset. The results indicate that the CiFormer model demonstrates superior performance across all feature patterns compared to the baseline models. By testing the prediction performance under different feature patterns, we conclude that including transport capacity notably reduces prediction errors and enhances the overall effectiveness of ride-hailing demand prediction. Additionally, we observe that incorporating Internet sentiment further improves the accuracy of these predictions.

This study unveils several promising directions for future exploration. First, the prediction model stands to gain substantial improvements through the adoption of more sophisticated Transformer-based architectures. Notable examples include Crossformer [63], Autoformer [64], FEDformer [65] and other cutting-edge models that represent the forefront of current research. Second, extending our current leveraging of textual data, we aspire to undertake an exploration of a multi-modal prediction methodology that incorporates image and video data [66] in ride-hailing demand prediction. Furthermore, given that the primary objective of ride-hailing demand prediction is to facilitate vehicle dispatch, we intend to explore an integrated approach in the future that seamlessly combines ride-hailing demand prediction with decision-making for vehicle allocation.

Declaration of competing interest

The authors declare that they have no conflicts of interest in this work.

Biographies

Xiang Li (BRID: 09916.00.59732) received the PhD degree from Tsinghua University, Beijing, China, in 2008. He is currently a professor with the School of Management, Beijing Institute of Technology, China. He has authored more than 150 articles on international journals, including IEEE Transactions on Evolutionary Computation, IEEE Transactions on Systems, Man, and Cybernetics: Systems, IEEE Transactions on Intelligent Transportation Systems, IEEE Transactions on Fuzzy Systems, IEEE Transactions on Engineering Management, Engineering, Decision Sciences, Transportation Research Part B: Methodological, Transportation Research Part C: Emerging Technologies, European Journal of Operational Research, and International Journal of Production Economics. His research interests include intelligent transportation systems, optimization under uncertainty, and data-driven decision-making. He serves as the Editor-in-Chief for International Journal of General Systems.

Hongguang Ma (BRID: 07712.00.78878) received the PhD degree in control science and engineering from Beijing University of Chemical Technology, China, in 2019. He is currently an associate professor with the School of Economics and Management, Beijing University of Chemical Technology, China. He has authored more than 30 articles in international journals, including IEEE Transactions on Fuzzy Systems, IEEE Transactions on Intelligent Transportation Systems, IEEE Transactions on Engineering Management, Information Sciences, and Computers and Industrial Engineering. His research interests include transportation management, operations research and optimization, and big data analysis. He has served as the Guest Editor for Information Sciences, Soft Computing, Scientific Programming, and Axioms.