Model validation and cause-of-death screening

The test set, representing 25% of the overall cohort, was left independent to the training set that had been used to build the multivariate models discriminating the five CoD groups. Thus, this independent dataset could be utilised for prediction and model validation testing.

Validation was performed by using a screening model to predict CoD, using the class prediction function in SIMCA software, and comparing predictions to the correct CoD diagnoses provided by the forensic pathologists. This function predicts group classification based on the distribution and variance of metabolite features included in the final OPLS-DA model built from the training set. Initially, the test set was screened, with the setting that every case should be screened into a single CoD group. The sensitivity and specificity of the test det screening were then compared to that of the training set prediction as a reference.

Further testing of the potential of this CoD screening model was performed by selecting new thresholds for CoD group classification, with the aim to access how much of the training set could be correctly predicted with a high level of accuracy. For the training set, receiver-operator-curves (ROC) were plotted, and thresholds were calculated from the steepest region of each CoD group curve. The calculations were performed within the region where the ROC curve had >0.3 true positive rate (TPR) and <0.1 false positive rate (FPR). These calculated thresholds where then applied to a new CoD screening for the test set of cases, and by comparing to the correct CoD diagnoses, updated sensitivity and specificity values were determined for this ROC-screening model.