3.3. Quality of the feature selection

The (balanced) prediction accuracy of a classifier is the mean of sensitivity and specificity. The stability of the signature was determined as follows: the dataset was split into 10 subsets, each containing 90% of the samples, and the feature selection approach was applied to each subset, resulting in 10 signatures. The stability was the average similarity over all pairs of signatures. We used the similarity measure proposed by Lustgarten and colleagues, since (i) it is adjusted for the commonality of subsets obtained by chance only, and (ii) it allows to compare signatures of different sizes (Lustgarten et al., 2009). The performance-robustness trade-off (hereafter performance) was computed as the harmonic mean of accuracy and stability (Determan, 2015).