Statistical Analysis

The composite score was calculated for the applicants selected for the external review step; hence, the standardization of the score was only done on the selected applicants. For analyses of pooled data (2014–2017 together), applicants who applied to the same position multiple times were identified, and only the first instance of application was included in the analyses. The total number of applicants included in separate analyses is 360 (207 applications for Assistant Professor and 153 for Senior Researcher). When repeat applicants are removed for pooled analysis of the full datasets for 2014–2017, keeping only the first instance of application, 186 unique applicants for Assistant Professor and 117 unique applicants for Senior Researcher were included, a total of 303 applicants.

Linear regression models were used to quantify the association between the composite bibliometric score (x-variable) and the external reviewer score (y-variable). In the full model, the analysis was performed adjusting for gender and including an interaction term to assess difference in slopes between men and women. In separate analyses, data were stratified by gender and year (including full data without removing duplicated individuals when each year was analyzed separately). In the results, the slope estimate addresses whether there is an association between composite bibliometric scores and external reviewer scores. The gender term addresses whether there is an overall difference in external reviewer scores for men and women. The interaction term addresses whether there is a difference between the slopes for men and women. The R value is the correlation coefficient revealing the strength of the association between composite bibliometric scores and external reviewer scores. The R-square is the proportion of variation of external reviewer scores that can be explained by the model. Df—degrees of freedom—is related to the number of samples, and F = F-statistic, whether the model is a good fit. P-values for estimates are considered statistically significant at p < 0.05. Analyses were performed in R version 3.6.0.