Learning curves cumulative summation test (LC CUSUM test)

The residents’ performance levels were followed using the LC CUSUM test. This test was developed to identify when an intervention has reached an adequate, predefined level of performance. In a LC CUSUM test, the null hypothesis is that “a performance is inadequate” and the alternative hypothesis is that “a performance is adequate.” Therefore, the learning curves in this study are binary variables (adequate or inadequate performance), and there are no measuring units. At each procedure (t), the LC CUSUM test was calculated as follows: St = min(0, St – 1 + Wt); W = log((1-p₀)/(1 - p₀ – d)) in case of success, and W = log (p₀/ p₀ + d) in case of failure; p₀ = acceptable failure rate; d = acceptable deviance from acceptable performance to be detected. The successive procedures were plotted on the x-axis and the St on the y-axis [7]. The performance cannot be considered acceptable as long as the score remains in the continuation region (between the x-axis and the decision limit (h_LC) [7]. The score increases with success and decreases with failure. A statistical threshold, h. Lc, is defined, above which the null hypothesis is rejected, and the performance considered adequate.

An ORR was considered adequate if ≥50%, and all retrievals with a lower rate were considered failures. The clinical performance was considered adequate if at least 60% of retrievals were successful with a standard deviation of 15%. The thresholds used were based on the mean oocyte retrieval rate of the senior physicians in our department for the years 2015–2016. Indeed, the main objective of the training is to allow residents to reach the performance level of senior physicians, thus allowing them to perform the procedure on their own. A h. Lc threshold of 6 was calculated so that the risk of declaring a resident competent when his performance was inadequate was 4.5% (false discovery rate or α error) and the probability of declaring a resident competent when his performance was indeed adequate was 100% (true discovery rate or power). The frequency with which each resident performed the procedures is shown in Fig. 2.

Learning curve cumulative summation test (LC CUSUM) graph for residents performing oocyte retrievals. The score is plotted on the y axis and time on the x-axis (number of days). The red line represents the threshold h.lc, above which the performance level is considered adequate. The prolonged periods without retrievals correspond to the periods where the department was closed (last week of December and 3 weeks in August), or when the residents were on vacation