Statistical analysis

The first goal was to confirm the factor structure of the C-SHARP and to evaluate the measurement invariance across clinic-referred samples with and without ASD. We used several data preparatory methods prior to analysis. In order to ensure an adequate number of responses per category for each C-SHARP item, rarely endorsed (<5%) response categories were collapsed with the next lowest category. Missing data (<5% for each subject) were imputed in five complete data sets using the Amelia package (Honaker et al. 2011). The imputed data sets were evaluated using the lavaan package (Rosseel 2012), and pooled according to standard multiple imputation rules (Rubin 2004).

All CFA procedures were completed using R software (R Core Team 2014). The general structure of the C-SHARP is shown in Figure 1. We performed a sequential series of tests of increasingly restricted parameters. First, we tested configural (structural) invariance, followed by weak (equivalent factor loadings), strong (weak plus equivalent intercepts), strict 1 (strong plus equivalent residuals), and strict 2 (strict 1 plus equivalent latent factor variance/covariance). In other words, the initial CFA model shown in Figure 1 was iteratively run with increasing levels of restrictions on parameters, setting them to equality between groups. Models were estimated using maximum likelihood with robust standard errors.

General structure of the Children's Scale for Hostility and Aggression: Reactive/Proactive (C-SHARP). All covariances were modeled, but were excluded from the diagram for clarity. This path diagram reflects the factor structure of the C-SHARP (including three cross-loadings), which was tested in the current analyses. The measurement invariance analyses in the current study placed restrictions on some of these paths (and some not illustrated here) to equality between groups.

A combination approach was used to evaluate model fit. Satorra–Bentler (SB-) χ² was used to evaluate absolute model fit, and χ² difference tests were used to compare nested models. Also calculated was the χ² per degrees of freedom for each model, where lower values indicated better fit. However, χ² tests are very sensitive to large sample sizes. As such, other fit statistics were calculated based on their frequency of use in the literature and expert recommendation for use in measurement invariance testing. The root mean square error of approximation (RMSEA) and standardized root mean residual (SRMR) are reported as goodness of fit measures for individual models (though their utility in model comparisons is less clear). Lower values indicate better fit (preferably <0.05 and <0.08, respectively). The Akaike information criterion (AIC) is reported as a measure of nested or nonnested comparative fit, where lower values are preferred. Finally, the comparative fit index (CFI) and McDonald's non-centrality index (McNCI) are reported, as they measure fit of an individual model (with values closer to 1.0 preferred) and have recommended cutoffs for evaluating nested measurement invariance tests. Simulation studies suggest that measurement invariance should be rejected at the p < 0.05 level when ΔCFI <−0.005 or when ΔMcNCI <−0.010 (Cheung and Rensvold 2002; Chen 2007). Although these simulation studies also suggested cutoffs for changes in RMSEA, SRMR, AIC, and other goodness of fit statistics, these were not considered in this article, as the evidence of their utility is less consistent, whereas the greatest support exists for ΔCFI and ΔMcNCI.

Two structural equation modeling reliability estimates were calculated: Raykov's (2001) method uses covariance structure analysis with nonlinear constraints to estimate the reliability of a factor, controlling for another factor. Bentler's lower bound (Bentler 2009) is interpreted as the unconditioned reliability of a factor. This method does not adjust for cross-loadings, and was, therefore, expected to be lower than the Raykov estimate. In both cases, possible values range from 0.0 to 1.0, with higher values indicating greater reliability. Traditional cutoffs are applied: Values >0.70 are acceptable, values >0.80 are good, and values >0.90 are very good.

The remainder of the validity analyses was completed in SAS version 9.3 (SAS Institute 2012). Pearson correlations or one way ANOVA with post-hoc Scheffe tests were used where appropriate. Differences in dependent correlations were tested using the method described by Steiger (1980), implemented using an online calculator (Lee and Preacher 2013). Alpha was set to p < 0.05.