2.5. Model performance measures

While the LSTM model can predict the SSH of the entire GoM, the evaluation of the predictions was focused on the Loop Current System in the region of interest (ROI) outlined in Figure 3. This is the region in which the LC is the most active and where eddy separation occurs and westward drift begins (Chérubin et al., 2005, 2006). It does not include land, which would have to be removed from the SSH matrix otherwise before reduction. Because HYCOM SSH was chosen, the SSH pattern sequences that the deep learning model has learned are intrinsic to the dynamics of the HYCOM model. Therefore, the skill measurement can only be done with HYCOM, which in this case represents the “true” ocean.

Model domain overlaid with SSH in meters. The black rectangle shows the region of interest (ROI) described in the text. The crosses mark the reference points used to measure the SSH prediction accuracy.

The same performance measures of the prediction skills as in Wang et al. (2019) were used in this study. The Correlation Coefficients (CCs) and Root Mean Square Errors (RMSEs) between “observed” and predicted fields were calculated. In order to evaluate the WELL model prediction against previous models, we used the same metric as in Oey et al. (2005), Zeng et al. (2015), and Wang et al. (2019). The Loop Current System frontal distances to 7 reference points, referred to as the Frontal Position Error (FPE) were also evaluated. As in Zeng et al. (2015), the front was defined by the 0.45 m contour line, and the same reference points as in Oey et al. (2005), Zeng et al. (2015), and Wang et al. (2019) were used (Figure 3). The Frontal position Root Mean Square Error (FP_RMSE) is used to measure the accuracy of the predicted LC and LCE positions. This measure consists of the difference between the distance d_p,r from a predicted frontal position to a reference point and the distance d_o,r from the corresponding observed frontal position to the same reference point. FP_RMSE is averaged over all the reference points as follows:

where R is the number of reference points used in the calculation. We also used a contour similarity measure such as the Modified Hausdorff Distance (MHD) (Hiester et al., 2016) to estimate the mismatch between curves. This measure exhibits a high sensitivity to outliers and can be expressed as follows. Given two corresponding sets of points, let

and

The MHD is then the maximum value between them:

where A represents a set of sample points on the observed contour, B represents a set of sample points on the predicted contour, dis(a, b) is the Euclidean distance between point a and point b, and dis(a, B) is the minimum distance between a and each of the points in set B. Simply put, MHD is the larger of two averages, each of which is the average of the distances from individual sampled points on one contour to the other (Hiester et al., 2016).