The analysis of the study was based on a qualitative content analysis according to Mayring [35] and a questionnaire. Qualitative content analysis is a structured, qualitative method for evaluating text-based data. The evaluation process is characterised by a rule-guided, fixed procedure [35]. In the following section, the qualitative content analysis approach and the used questionnaire are described in more detail.

A transcription protocol was created with Microsoft Word that includes the indication of the time in the video and audio recording. The statements of the study participants were transcribed into written form in the transcription protocol. The transcription was based on the transcription system of Kuckartz et al. [36]. Additionally, the user's interactions were described to trace which interactions the user performed in the software at any given time. A transcription protocol was created for each TA-Test.

The transcripts were checked for validity and possible errors were corrected (e.g. missing words or sentences). The transcripts were returned to the study participants for validation, whereupon all participants confirmed the content of the transcripts. For data analysis of the transcripts, deductive categories were created to assign text passages from the transcripts to the categories [35]. Deductive categories are used to evaluate a qualitative content analysis and are defined before the study begins. They are divided into main-categories and sub-categories. The category system used in this study is based on the deductive categories, shown in Fig. 1. We defined the categories based on our research questions. To answer our research questions, all relevant software functions and user interfaces are available in the category system. For each category, the sub-categories “information”, "software functionality” and “usability” were also examined according to the research questions. These sub-categories are defined as follows:

Information: The statements of the study participants refer to information presented in the CDSS. Information are presented after the use of a software function.

Software functionality: The statements of the study participants refer to a software function of the CDSS.

Usability: The statements of the study participants refer to the usability of the CDSS.

Final category system for content analysis

The transcribed material was proven in advance using the category system, to determine whether the categories can be applied to the data material. For this step, we used two (n = 2) transcription protocols, as it is recommended to use 10–50% of the transcribed material [37]. Afterwards, the category system was refined and two categories (1.3 and 3.7) were added to allow a more precise subdivision. After that, all transcripts were used and text passages were assigned to the categories. If a text passage could not be assigned to a category, all authors discussed and decided the assignment. Saturation of the study was reached when (1) all participants had successfully completed the study and (2) when the categories were adequately represented in the data after refinement of the category system [38].

While applying text passages to categories, we defined anchor examples and described when a text passage should be applied to a category. After the text passages were assigned to the categories, the respective statements were summarised per category. Finally, all results of the study were distributed to the study participants. All participants agreed to the results. In order to present the results in this paper, quotations were selected that represent the category at its best. The quotations (shown in Additional file: 5) were translated from German into English.

For further analysis in the evaluation of the CDSS, we developed a questionnaire which consists of three parts. In part one, the System Usability Scale (SUS) was used, which is a standardised questionnaire with ten questions (items) to assess the usability of a software system [39]. Since the items are only available in English, they have been translated into German. In the second part of the questionnaire, further questions were created to evaluate the individual functionalities of the CDSS. These questions were answered using a 5-point Likert scale (from “1 = strongly disagree” to “5 = strongly agree”) [40]. In the third part of the questionnaire, participants were asked to provide some personal information [21]. The following information were collected: gender, age group, medical specialization, years of experience in the field of RDs and previous experience with CDSS.

For the data analysis of the SUS, we used the approach of Bangor et al. [39]. The result is a range from “0” to “100”, which isn’t supposed to be interpreted as a percentage and must be normalised. According to Bangor [41], normalisation can be achieved by using the range shown in Table Table1.1. Furthermore, we calculated the mean and standard deviations for each item of the SUS across all participants.

Usability ranges of the SUS according to Bangor [41]

The answers to the questions in the second part of the questionnaire were assigned with numerical values from “1” to “5” according to the Likert scale, whereas mean and standard deviations were calculated for each question across all participants.

Note: The content above has been extracted from a research article, so it may not display correctly.

Please log in to submit your questions online.
Your question will be posted on the Bio-101 website. We will send your questions to the authors of this protocol and Bio-protocol community members who are experienced with this method. you will be informed using the email address associated with your Bio-protocol account.

We use cookies on this site to enhance your user experience. By using our website, you are agreeing to allow the storage of cookies on your computer.