Cohen’s kappa coefficient

The Cohen’s kappa coefficient (ƙ) is a statistical measure of inter-rater reliability between two raters⁷ and was calculated as following³⁰:

where P_o is the proportion of observed agreements and P_c is the proportion of agreements expected by chance³⁰. Here, P_c=0.5, since our algorithm identifies two possibilities (SWS and non-SWS). Kappa is considered to be a more robust measure than simple percent agreement calculations because ƙ takes into account only the agreements that occur above chance³⁰. Following the sleep staging literature, the interpretation of kappa coefficients is as follows; values of ≤0.00 indicate poor agreement; 0.00–0.20 indicate slight agreement; 0.21–0.40 indicate fair agreement; 0.41–0.60 indicate moderate agreement; 0.61–0.80 indicate substantial agreement; and more than 0.80 indicates excellent agreement¹⁸.