Sub-challenge 2 team ranking

The algorithms submitted by participants in the final stage of sub-challenge 2 were applied as implemented by the participants without any tuning to the 70 pairs of training and test datasets described in Figure 4A, Table S5, and Video S1. In each of the 7 scenarios in Figure 4A, there were 2 outcomes predicted (sPTD versus Control; and PPROM versus Control), except for proteomic data (scenario DP2), where the feasible comparisons were sPTD versus control and sPTB versus control; the sPTB group was defined as the union of sPTD and PPROM cases. The AUROC and AUPRC were used to assess predictions for each outcome based on predictions on each of the 10 test sets for each scenario, and then were averaged over the 10 tests sets. The resulting 28 prediction performance averages (7 scenarios x 2 outcomes x 2 metrics) for each team were converted into Z-scores by subtracting the mean and dividing by the standard deviation of these metrics obtained from 1,000 random predictions (random uniform posterior probabilities). Further, only the combinations of scenarios and outcomes resulting in a significant prediction performance (False Discovery Rate-adjusted p value derived from Z-scores, q < 0.05) for at least one of the 13 teams, were considered for team ranking, resulting in 20 performance criteria for each team. Teams were ranked by each of the 20 prediction performance criteria, and a final rank was generated based on the sum of the ranks over all criteria (Table S6).