Data analysis

The cut-off point for acceptable value for content value for each item was a rating of 3 or 4.

I-CVI was calculated as the percentage of experts and SA patients rating an item either 3 or 4, respectively. The S-CVI was calculated as the average of the I-CVIs for all items on the scale (S-CV I/Ave) rating the entire questionnaire by 3 or 4, respectively.

Any I-CVI rated greater than or equal to 0.78 by 6 or more experts is considered excellent. Subsequently Polit et al recommend 0.78 as excellent regardless of the number of experts. The recommendation for S-CVI/Ave is 0.90 or higher.15, 26

To counter the limitations of CVI, each I-CVI was adjusted for chance agreement by calculating the modified kappa statistic (κ*).15, 29 To compute the modified kappa, the probability of chance agreement was computed first: P_c = (N/A (N − A) × 0.5^N where N is the number of experts and A is the number of agreement of good relevance (rating 3 and 4). Then, the κ* was calculated using the formula κ* = (I-CVI − P_c)/(1 − P_c).11

According to the standards of Fleiss et al and Cicchetti and Sparrow the value of each κ* was evaluated as: poor (k < 0.40), fair (0.40 < k < 0.59), good (0.60 < k < 0.74) or excellent (k > 0.74).3, 5

In this study, the experts and the SA patients were asked to evaluate the entire WORC Index overall as a QoL measurement for patients with SA using the same four-point scale, defined as S-CVI_total.

The AD index for Likert-type scales was used to measure inter-rater agreement.2 The AD index measures the dispersion of responses about the median. This was calculated as the sum of differences from the median in absolute values divided by the number of experts. At 5% level of significance the cut-off limit was 0.37 for six raters. Values below 0.37 indicated acceptable and statistical significant agreement.2