3.3. Neural Network Classification Models

Discrimination models, including a deep convolution neural network (DCNN) and four back-propagation neural network (BPNN) models, were used to identify the types of residues on T. sinensis leaves based on the THz fingerprint. Figure 6 shows the flow chart for the whole process. This process provides a feasible strategy to rapidly and accurately detect pesticide multi-residues on plant leaves.

The main flow chart for detecting multiple benzimidazole (BZM) pesticide residues in leaves of Toona sinensis using terahertz (THz) imaging and deep learning.

In this paper, a DCNN model consisting of a convolution layer, pooling layer, and fully connected layer was used to identify multi-pesticide residues on leaves. There were two convolution modules and five fully connected layers in the DCNN framework to process nine classes (BG, CK, and seven types of pesticide residues) of input spectral data, of which 70% (15,750) of the data were used for training and 30% (6750) were used for testing. The convolution module contained a convolution layer and a maximum pooling layer. The number of convolution filters in the first and second convolution layers was set to 32 and 64, respectively. In order to learn the local information quickly and reduce the dimensions of the spectral data, the step size of the convolution kernel was set to 1. The kernel size of the convolution layers and the maximum pooling layer was set to 1 × 3. Rectified linear units (ReLUs) were used for the attribution of the non-linear properties of the decision mapping function [22]. Batch normalization (BN) was used before each full connection layer and after each convolution module to improve learning speed and reduce initialization requirements [23]. The Softmax function was used to process the full-connectivity data of the last layer by highlighting the maximum value and limiting the eigenvalues of other nerve units below the maximum value. The classified cross-entropy loss function was used to measure the distance of the probability distributions between the DCNN output labels and the real class labels [24]. An adaptive moment estimation algorithm was used to process the loss function. In order to prevent over-fitting during the training process, 30% of the training set was selected for verification and a drop-out method was used to further prevent over-fitting after batch normalization of all connection layers. The epoch number, learning speed, beta 1, and beta 2 were set to 500, 0.001, 0.9, and 0.99, respectively.

In comparison with DCNN, four learning functions, including the Powell-Beale conjugate gradient algorithm (TrainCGB), the Fletcher-Reeves conjugate gradient algorithm (TrainCGF), the Polak-Ribiere conjugate gradient algorithm (TrainCGP), and Resilient Back-Propagation (TrainRP) were utilized for the back-propagation neural networks (BPNN) models [25]. To achieve a higher classification accuracy of a large amount of spectral data (15,750 spectra for training and 6750 for testing), four-layer networks were constructed, in which the input layer and the two hidden layers all consisted of 40 nodes and the output layer consisted of nine nodes. Links were used to connect the nodes of different layers. Two parameters (weight and bias) in each link were updated or estimated according to the four functions, TrainCGB, TrainCGF, TrainCGP, and TrainRP [26]. The learning rates of 0.1 were set by default, which determined the size of the weight updated at the end of each epoch, thereby influencing the rate of convergence. The number of epochs was set to 5000, which defined the maximum number of iterations for training. The learning goals were set to 0.0001, which determined the accuracy of the output results.