3.1

Spatial-temporal embedding

In practice, the evolution of traffic state will be affected by the basic traffic network structure, so it is necessary to build the network structure and input it into the prediction model. As shown in Figure 2, we model spatial dependence by associating traffic flow with diffusion process, which clearly captures the randomness of road network.

Figure 2.

Spatio-temporal embedding.

The characteristic of the diffusion process is a random walk on graph, and the restart probability is α, with a probability matrix of information transfer . D_𝑂 = diag(W). The matrix will restrain itself to an equilibrium probability condition P ∈ R^N×N. Teng et al.⁵ pointed out that the ultimate stable state can be obtained by the following:

where k is the diffusion steps. In this work, we take the truncation of finite k steps and model the spatial dependence by bidirectional diffusion. Hence, we can express spatial embedding in the following form:

The spatial embedding only provides static representation and cannot represent the dynamic correlation. Therefore, we put forward another way, which encodes time dimension as a vector. We divide a day into N parts, encoded the time steps into R^N and R⁷ by one-hot coding and spliced into R^N+7, represented as .

In our model, we both unify these features into R^D through a fully connected neural module and fuse them as spatiotemporal embedding (STE). Therefore, the STE includes both road network structure and time features.

3.2

Multi-head attention

Since the attention mechanism was proposed, it has been applied extensively in many fields. It can find out the relationship between them according to the raw data and extract the most important features. Multi-head Attention is to calculate the attention of the data in different subspace with the total number of parameters unchanged, and the last step is to merge the attention information in different subspace⁷. The dimension of each vector is reduced by this way when calculating the attention of each head and the over-fitting phenomenon is also avoided; because attention has different parameters in different subspace, Multi-head Attention looks for the correlation between sequences relations from different angles in fact.

For the next state of node i at time t, we update it with the sum of the corresponding weights of all nodes can be expressed as follow:

α is the attention score indicating the significance of node,H^{l – 1} indicates the last hidden state and H^l indicates the current state. Adopting the scaled dot-product approach⁷ to learn attention score.

where || indicates the splicing process, represents spatio-temporal embedding vector, f₁ and f₂ are an activation function with two different parameters, d is the dimension after vector splicing. Then, λ^k is normalized to α by SoftMax⁸, by splicing K attention mechanisms with different learning styles, we can get:

f₃ is another activation function. Therefore, we successfully capture the inner spatial relationship among nodes.

3.3

Gate fusion

In order to further integrate the spatio-temporal relationship, we designed a gated fusion module to adaptively fuses spatiotemporal information as shown in Figure 3.

Figure 3.

Fuse spatio attention and temporal attention information together.

After the input passes through the spatial and temporal attention mechanism, the output is represented as and , and merge into:

where W₁ and b₁are different learnable parameters, σ is the sigmoid function, and is the result of spatio and temporal attention.

3.4

Gating

In order to make the final results obtain the characteristics of long-time period, we introduce a gating mechanism to correct the attention results as shown in Figure 4, so as to reduce the prediction error.

Figure 4.

Control information transmission.

In this way, the network model can not only remember the information of the past, but also selectively forget some inessential information and shape long-term relationships, the calculation process is as follows:

where split means separating the output results, tanh is another activation function, the gating mechanism can combine the obtained output with historical information to get more accurately results.

4.1

Datasets

We used our model to make traffic predictions on real data set PeMS-bay. In this data set, we take the traffic speed every five minutes and normalize the data to the interval [0, 1]. In order to build the road network, we calculate the paired road distance and use the threshold to construct adjacency matrix⁹ if dist(v_i, v_j) ≤ δ, otherwise 0, where W_ij represents the weight of the adjacency matrix. dist(v_i, v_j)indicates the distance between sensors. σ is the standard deviation indicates the degree of dispersion of the distance, and δ is the limit value remove unnecessary data.

4.2

Result analysis

In order to make the model results easier to understand, we visualize the experiments results. Figure 5 shows experimental results on actual traffic data set of the model. From these figures, we conclude that when the average road speed fluctuates little, the model can generate a relatively smooth curve to fit the road traffic conditions. Even if the road conditions change suddenly, the model can capture this change according to the spatio-temporal characteristics and generate more accurate prediction curve.

Figure 5.

Result on real data of PeMS-bay.

4.3

Baselines and comparison

We compare our model with other baselines. Table 1 shows the average results of traffic forecasting estimated performance in the next one hour. It can be seen that our STMAGN gets a good form in all aspects of all evaluation indicators, especially in the long-term prediction stage, it has achieved far better results than other models. In addition, we can observe that the results of traditional sequences approaches are not perfect in general, which shows that these methods have limited capacity for increasingly complex transportation systems. Through comparison, the methods on account of deep learning have generally achieved good results.

Table 1.

Comparison of results between STMAGN and the others.

4.4

Effectiveness of each module

In order to study the impact of each module, we evaluate in three different ways, like removing conversion layer or STE or gating from the model. Figure 6 shows the impact after removing these components, thus proving the effectiveness of each module.

Figure 6.

Effectiveness of each part of the model.

The experimental results show that removing any module has an established impact on the final prediction accuracy, especially the accuracy of modules without STE is dropping fast.