2.5 Trajectory prediction based on LSTM network

LSTM, a special neural network structure, can process time-series data by recording and processing the first N frames’ inputs and the network’s intermediate results. Thus, it can integrate multiple frames’ information for classification, regression, and input prediction [20]. LSTM’s form is primarily the same as others, consisting of the input layer, hidden layer, and output layer. A hidden layer can contain multiple LSTM units, a unique design form of a hidden layer node of RNN [21]. The detailed internal structure of the unit is shown in Fig 5. The equation of each unit is as follows:

In (10), f_t and i_t denote the step t forgetting gate and input gate, respectively. In each playing process, the forgetting gate controls each movement process, and the input gate controls each trajectory prediction process.

The Sigmoid function is selected for f_t and i_t, and the value range is [0,1]. The Tanh function’s value is [–1,1]. C_t−1 is the neuron’s state at time t-1, and C_t is the neuron’s state at time t.

In (14) and (15), o_t is the degree that the output gate controls the trajectory, h_t is the output of step t in the trajectory.

An RNN (Recurrent Neural Network) with LSTM units is built, which receives table tennis ball’s 3D spatial position at time t as the input and output at time t+l. The input layer receives the ball’s 3D spatial position at time t and transmits it to the n-th LSTM unit in the hidden layer. Each LSTM unit calculates the current time according to the input and internal state values and outputs the calculation result to the fully-connected layer. Simultaneously, the internal state value is updated and retained until time t+l. The specific structure is shown in Fig 6.