4.4. Meta‐prediction models

A conventional approach to combining predictions from several different methods is meta‐prediction, where several prediction methods are combined to give a single prediction, usually at the residue level. For comparison with our predictor selection method, we developed several residue‐level meta‐prediction methods based on the 12 individual predictors examined in this study. We used several variations on meta‐predictor construction: two‐different architectures—logistic regression (LR) and support‐vector regression (SVR), and different input predictors—either all 12 predictors or only the best of the predictors. The best predictors were selected based on the data set level performance on our training set (Table ​(Table1).1). Based on these assessments, we selected two prediction methods—SPOT Disorder and Disopred3—as significantly better than the other individual methods. This gave four meta‐predictors: 12Predictor LR, 12Predictor SVR, Top2Predictor LR and Top2Predictor SVR.

Prediction scores from the individual predictors were initially rescaled to be in the same range of 0–1. The rescaling was based on the default thresholds of respective predictors as values from minimum to threshold to be in the range of 0–0.5 and values from threshold to maximum to be in the range of 0.5–1.

The logistic regression model was trained with the threefold cross validation on the training data set with default L2 regularization penalty by balanced class weights according to the proportions of training set using the L‐BFSG optimization algorithm. The SVR models were trained with the threefold cross validation on the training data set after subsampling 10% of each fold randomly to minimize the training time. We used the radial basis function kernel and performed a grid search for the penalty parameter C (between 2⁻⁵ and 2⁵), kernel coefficient gamma (between 0 and 1), and tolerance for stopping criteria (between 10⁻³ and 10³).