3.5 CAESAR-ALE’s Active Learning Methods

CAESAR-ALE includes two AL methods (Exploitation and Combination_XA), which we now describe in detail.

One of the AL methods implemented in CAESAR-ALE is called Exploitation, referred to as such because it exploits the current separating hyperplane to find condition instances that are most likely to be severe. Exploitation has demonstrated efficiency at detecting unknown malicious code content, files (37–40), and documents (41). Exploitation is based on SVM classifier principles and selects examples more likely to be severe, those lying further from the separating hyperplane, as can be seen in Figure 2. Thus, this method aims at boosting the classification capabilities of the model through the acquisition of as many new severe conditions as possible. For every condition x, Exploitation measures its distance from the separating hyperplane using Equation 8, based on the Normal (W) of the separating hyperplane of the SVM classifier. The separating hyperplane of the SVM is represented by W, which is a linear combination of the most important examples (supporting vectors), multiplied by LaGrange multipliers (α) and by the kernel function K that assists in achieving linear separation in higher dimensions. Accordingly, the distance in Equation 8 is calculated between example x and the Normal (W) presented in Equation 3. The distance calculation required for each instance in Exploitation is equal to the time it takes to classify an instance using SVM-Margin.

Acquiring several severe conditions that are highly similar to each other (i.e., which have similar values for all of the meaningful features, and of course, belong to the same target class) would waste labeling resources, while not contributing much to the future classification capabilities (generality) of the induced classifier; therefore, acquiring one representative condition from this set is preferable. In the Exploitation method, conditions are acquired if they are classified as severe and have maximal distance from the separating hyperplane (marked with a red circle in Figure 6.1).

An illustration showing the Exploitation method’s criteria for acquiring new severe conditions.

To enhance the training set as much as possible, we also check the similarity among selected conditions using the kernel farthest-first (KFF) method suggested by Baram et al. (42), enabling us to avoid acquiring similar conditions. Consequently, only potentially informative conditions likely to be labeled as severe are selected. In Figure 6.1, it can be seen that there are several sets of highly similar conditions, based on their distance in the kernel space. However, only representative conditions that are more likely to be severe are acquired. Contrary to SVM-Margin, Exploitation explores the “severe space” to discover potentially more informative severe conditions, a process which enables further detection of severe conditions. Figure 6.1 also illustrates an additional ability of Exploitation, as it sometimes discovers conditions located far inside the severe side (i.e., class) that were ultimately labeled by the expert as mild. Finding such a surprise is highly useful - this type of confusing condition will become a new support vector of the SVM classifier and update the classification model with the new discovery and knowledge; thus, these “surprises” play an important role in increasing the accuracy of the resultant classifier.

The “Combination_XA” method is a hybrid of SVM-Margin and Exploitation. It conducts a cross acquisition (XA) of potentially informative conditions. That means that during the first trial (and all odd-numbered trials) it acquires conditions according to the SVM-Margin method’s criteria, while during the next trial (and all even-numbered trials) it selects conditions using the Exploitation method’s criteria. This strategy alternates between the exploration phases (conditions acquired using SVM-Margin) and the exploitation phase (conditions acquired using Exploitation) to select the most informative conditions, both mild and severe, while boosting the classification model with severe conditions or very informative mild conditions that lie deep inside the severe side of the SVM’s hyperplane.