Flashbulb test of recall bias

We suggest here a recall bias test strategy that is based on the concept of flashbulb memories developed in the psychology literature. The original “Flashbulb Memory” concept hypothesises that major events, such as natural tragedies or traumatic events, should be recalled vividly and reliably, and that these events trigger in the subject not only memories of the main event in the long term, but also of other personal circumstances of the subject at the time of such event. Brown and Kulik [5], in their seminal paper, use as an example the US President John F. Kennedy’s assassination, arguing that “Almost everyone can remember, with an almost perceptual clarity, where he was when he heard, what he was doing at the time, who told him, what was the immediate aftermath, how he felt about it, and also one or more totally idiosyncratic and often trivial concomitants” (p. 73). Further work by Wright et al. [27] analysed two major events in the UK, namely Margaret Thatcher’s resignation and the Hillsborough football disaster, in the context of a population survey, finding that in the longer term (three to four years after the events) even these memories can somehow fade, with only a small percentage of the population being able to remember these events vividly.

For this study, our interest does not lie in how accurately survey respondents can recall specific events per se, but rather in whether the patterns of inaccuracies in retrieving such events, or the forgetting process, differ between survey respondents living in areas more and less affected by conflict violence. We build on the Flashbulb Memory concept to construct a simple but novel recall bias test, which we call Flashbulb recall test, to assess how forgetting and memory functioning can affect inference from conflict and health survey responses. More specifically, we propose the use of a well-known past event, that can be corroborated objectively, to test if recall errors differ across areas affected and not affected by conflict, between two time periods generally defined here as pre- and post-peace.

The presence of recall error can be expressed empirically as:

where Zt-1 is a well-known past event occurred at time “t − 1” (pre-peace) and Z^t-1,it denotes the recall of the event by individual i at the time of the post-peace survey “t”. We can then test the conditional independence assumption in a model with covariates, covDi|Xi,ωi=0, by estimating the following regression:

where ωi∈0,1 is an indicator that takes the value of zero if individual i retrieves accurately the past event, and 1 if the individual retrieves the event inaccurately (see Eq. 4),β0 is the intercept, Di∈0,1 is an indicator that takes the value of 1 for conflict zones (zero otherwise),Xi contains the control variables, and φi is the error term.

Coefficient β1 is the parameter of interest for this test of the conditional independence assumption: if we cannot reject the null hypothesis that β1 is equal to zero (i.e. if β1 is not statistically significant at conventional levels), then there is evidence to support that recall errors are distributed randomly across conflict and non-conflict areas, thus mitigating concerns about recall bias affecting the parameter α1 in Eq. (3). Conversely, rejecting the null hypothesis that β1 is equal to zero would point to the presence of recall errors that are non-randomly distributed between conflict and non-conflict zones, and thus to potential recall bias.

This test implicitly assesses the patterns of memory functioning across surveyed populations in conflict and non-conflict zones. We expect that, as time passes, the normal forgetting of events will affect the retrieving process for all survey respondents, but if forgetting was different in conflict zones (say “higher” due to e.g. motivated forgetting, dissociative amnesia or even physical brain damage), then we would expect to find a statistically significant estimate of β1, indicating potential recall bias for the estimation of conflict effects on health outcomes.