Data collection and analysis

In order to investigate effects of the in situ simulation-based training program, we conducted repeated questionnaire surveys, and utilized data from the Central Denmark Region Business Intelligence (BI) database. Figure ​Figure11 illustrates how and when data was collected. All analyses were performed using Stata/MP 17.0.

Overview of data collection. Illustration of timeline and types of data

Through consultations with hospital administration, a questionnaire was developed by the authors of this study, based on current research on psychological and organizational outcomes [4, 6, 7, 16]. Questions were developed to meet practical needs when moving to a new setting, including readiness to perform in new facilities. Thus, the survey included three items related to readiness to perform, which were worded as follows: (1) ‘To what extent do you feel ready to navigate in your own ward/clinic?’, (2) ‘To what extent do you feel ready to participate in emergency calls at another ward/clinic?’ and (3) ‘To what extent do you feel ready handling emergency situations, for example a cardiac arrest?’. All items were measured using a Likert scale, consisting of (1) very ready, (2) ready, (3) neither nor, (4) not ready, or (5) not ready at all.

Furthermore, the questionnaire items collected background information, including; sex, age, profession, seniority, workplace, and an ID-number.

Questionnaire-data was collected as a pre-, post-, and follow-up measurement (Fig. ​(Fig.1).1). Pre-measurement was collected upon arrival to the new hospital facilities, prior to participating in the in situ simulation-based training program. Post-measurements were collected immediately after the in situ simulation-based training program. Pre- and post-measurements included employees participating in the simulation-based training program. The follow-up measurement was collected six months after the relocation, and included all employees working at the department of psychiatry.

All questionnaires were distributed using SurveyXact [30]. Pre- and post-measurements were collected by participants accessing the questionnaire through a QR-code with their smartphones as illustrated in Appendix 2 (Supplementary material). The follow-up measurement was collected by sending an email invitation to all employees at the Psychiatry Department at Aarhus University Hospital. Two additional reminder emails were sent, and posters reminding employees to answer the questionnaire were placed in common areas across the hospital.

Data from the questionnaires were categorized and analyzed separately in two groups, one consisting of ‘all participants’ (n = 1.199), and the other consisting of participants who completed all three surveys, which are characterized as the ‘complete case group’ (n = 143).

To investigate readiness, Likert scales were converted to range between 0 and 100, in which 1 was equal to 100, 2 = 75, 3 = 50, 4 = 25 and 5 = 0.

Business intelligence (BI) data were extracted from an ongoing administrative Human Resources database, covering all employment related information in the specific region. BI data was accessed by using the unique ID-number obtained from participants in the questionnaire. BI data included detailed individual sick leave registration, which was covered during two time periods, before and after the intervention, respectively. Sick leave was calculated by estimating the relationship between hours of absence and each staff’s proportion of a full-time position. Thus, part-time employment was taken into account. The first time period covered sick leave from January 30, 2018 until September 30, 2018. The second time period covered January 30, 2019 till September 30, 2019.

As in existing literature, analyses included mean scale scores and standard deviation [8, 16]. A change in mean scores indicated an improvement/aggravation in feeling ready and a higher/lower rate of sick leave. Paired sample t tests were applied to compare dependent groups over time, and non-paired sample t tests were used to compare across independent groups. Furthermore, non parametric tests including Wilcoxon signed-rank test (dependent data) and Wilcoxon rank sum test (independent data) were used to compare medians between groups. Finally, histograms in Appendix 3 (Supplementary material) was conducted to illustrate distribution of data.