Qualitative content analysis

A transcription protocol was created with Microsoft Word that includes the indication of the time in the video and audio recording. The statements of the study participants were transcribed into written form in the transcription protocol. The transcription was based on the transcription system of Kuckartz et al. [36]. Additionally, the user's interactions were described to trace which interactions the user performed in the software at any given time. A transcription protocol was created for each TA-Test.

The transcripts were checked for validity and possible errors were corrected (e.g. missing words or sentences). The transcripts were returned to the study participants for validation, whereupon all participants confirmed the content of the transcripts. For data analysis of the transcripts, deductive categories were created to assign text passages from the transcripts to the categories [35]. Deductive categories are used to evaluate a qualitative content analysis and are defined before the study begins. They are divided into main-categories and sub-categories. The category system used in this study is based on the deductive categories, shown in Fig. 1. We defined the categories based on our research questions. To answer our research questions, all relevant software functions and user interfaces are available in the category system. For each category, the sub-categories “information”, "software functionality” and “usability” were also examined according to the research questions. These sub-categories are defined as follows:

Information: The statements of the study participants refer to information presented in the CDSS. Information are presented after the use of a software function.

Software functionality: The statements of the study participants refer to a software function of the CDSS.

Usability: The statements of the study participants refer to the usability of the CDSS.

Final category system for content analysis

The transcribed material was proven in advance using the category system, to determine whether the categories can be applied to the data material. For this step, we used two (n = 2) transcription protocols, as it is recommended to use 10–50% of the transcribed material [37]. Afterwards, the category system was refined and two categories (1.3 and 3.7) were added to allow a more precise subdivision. After that, all transcripts were used and text passages were assigned to the categories. If a text passage could not be assigned to a category, all authors discussed and decided the assignment. Saturation of the study was reached when (1) all participants had successfully completed the study and (2) when the categories were adequately represented in the data after refinement of the category system [38].

While applying text passages to categories, we defined anchor examples and described when a text passage should be applied to a category. After the text passages were assigned to the categories, the respective statements were summarised per category. Finally, all results of the study were distributed to the study participants. All participants agreed to the results. In order to present the results in this paper, quotations were selected that represent the category at its best. The quotations (shown in Additional file: 5) were translated from German into English.