3.3. Ranking and quantification

In order to provide numerical information about the strength of the relationship, the text fragments were quantitized. More specifically, for each text fragment that refers to one of the three relationships, the coder scores a vague quantifier according to how strongly it represents the relationship. A dichotomous scale is often used for quantitizing (eg, Reference 39)—present or not present, but that solely allows us to ascertain that an effect is present, and therefore would “undercut the ability to capture the nuance and subtlety of particulars in qualitative studies.”25 The reviewer scores the text fragments by judging the vague quantifiers on an ordinal scale. We chose an ordinal scale because this allows us to code the extent to which a relationship is present.

All three coders listed and ranked the vague quantifiers from the studies based on how strongly they represent one of the three relationships, and then constructed a three‐point, five‐point, and seven‐point scale from the ranking. All rankings are specified in the coding manual [Supporting Information]. For validation purposes, a blind peer also independently ranked the quantifiers. In addition, we asked another blind peer who is a native speaker to check our rankings. Both peers validated our findings. (LvG) coded the first six studies to decide which scale would best fit the data. For all six studies, a categorization of five different values to indicate the strength deemed to be sufficient. We chose to use a five‐point scale with the labels “very small” (2), “small” (3), “medium” (4), “large” (5) and “very large” (6).

Negative values could also be added to the scale (−1, −2, −3, −4, and − 5). They would be used when a text fragment is encountered which indicates an effect in the opposite direction. We did not come across these effects, but other researchers applying this approach could come across groups of studies that include contradictions and negative cases.44 The vague qualifier would then be used in the opposite way, for example stating the following sentence (fictional):

“Some women stated that new knowledge of the risks of smoking for the fetus actually made them less likely to successfully quit smoking.”

The labels were then transformed to effect sizes. We chose to use correlation coefficients as “effect sizes,” because our aim is to describe how the scores of one measure relate to the scores of another measure for that sample, indicating a cause of the factors on the effect smoking cessation. Note that we do not imply causality in the strict sense that we consider all alternatives for smoking cessation be ruled out—already given by the fact that we consider three different causes for smoking cessation—but we do assume that the factor precedes the outcome. Therefore, we do use the labels “independent” and “dependent” variables. The scores are, however, not assigned to the independent and dependent variables as their unique variances are not measured. It is merely assumed that we can estimate how strongly these variables are correlated.

In order to come up with meaningful and realistic values to be assigned to the five‐point scale for measuring the correlations, we have asked a researcher with expertise in research in healthcare interventions what an appropriate range of effect sizes is for this field. To determine a range that is firmly grounded in the literature, he made a selection of seven recent systematic reviews in public health that he, based on his knowledge of the field, deemed relevant for our case. We have collected all effect sizes from the primary studies that were reported in these seven recent systematic reviews to get an idea of the range of effect sizes that one can expect from interventions in public health. A total of 69 effect sizes were extracted. We found an average correlation r of .22. The correlation range was .01 to .54. We divided these data into six equal parts, and we selected the values of the effect sizes corresponding with the percentile 16.7, 33.3, 50, 66.7, and 83.3. The values corresponding to these percentiles were .13, .19, .23, .27, and .31, and these numbers were assigned the labels very small, small, medium, large, and very large respectively.