Data Analysis

For step four in the scale development process (i.e., validation), a number of data analysis procedures are necessary, which are mainly used to establish the reliability and validity of the new instrument. In line with psychological and social science practices, the 7-point Likert scales data were treated as interval-scaled data (e.g., Norman, 2010; Wu and Leung, 2017).

The analyses were performed in several phases, as it was likely that the indicators that are part of the instrument would form a higher-order construct. For each level in this higher-order construct (i.e., from indicators to lower-order constructs, from lower-order to higher-order constructs), reliability and validity metrics were first assessed to guarantee internal consistency (initially without any relationships to external variables). Second, the relationships with criterion variables were tested (for further details, please refer to the Confirmatory Composite Analysis proposed by Hair et al., 2020).

In each phase, the directionality of the relationship between indicators and the higher-order construct had to be defined first (i.e., reflective if indicators are manifestations of a common construct or formative if they form the construct; Jarvis et al., 2003). For the first level (i.e., indicators to lower-order constructs), we followed Ragu-Nathan et al. (2008) and hence used reflective specification. We then initially conducted a series of exploratory factor analyses (EFA) as well as parallel analyses and a Velicer’s MAP test (O‘Connor, 2000) to develop insight into the dimensionality of the DSS further. For the resulting factors, we then followed the steps recommended by Hair et al. (2019) to ensure the quality of the measurement model involving the resulting 1st order constructs:

These indicators were used to create a set of 1st order constructs that showed sufficient reliability and convergent validity (constructs that did not fulfill these minimum requirements were removed). The indicators were then used to form a 2nd order construct, and the model specification (reflective vs. formative) was investigated using the criteria proposed by Coltman et al. (2008). For the 2nd order construct, reliability and validity were assessed again, including discriminant validity in relation to the four criterion variables.

These steps concluded the evaluation of the measurement model. Thus, the new instrument as well as other constructs included in this investigation showed sufficient internal consistency and were also sufficiently conceptually different from each other.

The nomological validity of the new instrument was then tested during the structural model evaluation, when its relationships with the four criterion variables and the control variables were tested. For this purpose, a number of regression models were estimated. In addition, the same procedures were implemented using the existing TSC instrument to make possible a direct comparison with our DSS. Moreover, we confirmed that the relationships with other variables found with TSC could also be found with the new instrument.

The psychometric properties of the DSS were predominantly assessed using PLS-SEM (using SmartPLS 3 v. 3.2.8) due to some of the benefits of this analytical approach as compared with covariance-based SEM (CB-SEM). According to recent evidence presented by Hair et al. (2019), PLS-SEM is more robust against non-normality of data and is particularly suited for formative models (formative models are also feasible in CB-SEM using MIMIC models, Diamantopoulos, 2011, though such models can lead to results that are not theoretically sound, Hair et al., 2019). Although CB-SEM is the prime method to investigate higher order constructs, it has also been shown recently that PLS-SEM supports models with higher order constructs (Sarstedt et al., 2019).