Adopting a “better bets” approach to evidence scanning

The intention of the second phase of the evidence review was to both define the interventions with an evidence-base alongside determining what we assessed to be ‘better bet’ interventions for FCV settings. What do we mean by a “better bets” approach? As academics, we understand and appreciate the need for rigorous and evaluative evidence reviews. However, we must also recognize that life and death decisions are constantly being made by policymakers, managers, and frontline providers in FCV settings, often handicapped by inaccessible and incoherent evidence even where directional evidence exists; inadequate resources, infrastructure, and skills to operationalize. Therefore, the existing evidence base needs to be assessed to identify which might be “better bet” interventions to be implemented in austere and adverse health care settings. In this content a “better bets approach” describes the formal and systematic assessment of the existing evidence base to identify interventions most useful and feasible to implement in FCV settings. We acknowledge that the “better bets” approach is a novel method to assess which interventions may be most effective in implementation, summarising the associated biases in the limitations section, but intend to develop the approach and scoring system through further research.

Following completion of the evidence scans, a better bets scoring system was conducted, utilising four criteria as outlined in Table 1. “Generalizability of the studies’ evidence” refers to the extent to which findings can be extrapolated to similar settings[9]. “Feasibility of implementation” addresses whether the interventions are practical and appropriate in the existing or another setting, which is important to establish prior to embarking on implementing a relatively large-scale intervention. “Positive results” as a criterion is important as the literature scan was conducted to find evidence where patient safety interventions were shown to be effective and/or beneficial in some way, especially for FCV settings. Including “positive results” rather than “effectiveness” as a criterion was based on logical rationale, however the limitation is that the rapid evidence scan did not search for the complete universe of articles on the subject, which would encompass a better balance of evidence on what works and what does not work. “Representativeness” refers to the focus on interventions relevant to FCV contexts and health care settings. A single sub-intervention (eg,, ICP) with evidence only representing one health care setting or country, for example, would score lower than an intervention with positive results across multiple countries and/or health care settings. Studies in multiple settings/countries were considered by the authors to be more representative of resource constrained conditions in FCV settings as the interventions were assessed in a range of relevant environments. Again, the limitation of assessing representativeness from results of the rapid evidence scan is that the studies we selected are not necessarily representative of all studies in FCV and low-income settings and thus may be subject to bias.

Definitions of the better bets criteria

FCV – fragile, conflict-affected, and vulnerable

The scoring took place in June 2020. Three reviewers, Niki O’Brien (MSc), Alex Shaw (MSc), and Mike Durkin (FRCA, FRCP, DSc), independently examined each intervention evidence table and applied these four criteria based on a rating scale of 0,1 or 2. Each reviewer independently scored each intervention against these criteria and then scores were totalled to a maximum of 8 points. Comparing the scores, a discrepancy of 2 or more points was considered disagreement. In such cases each reviewer’s rationale for giving the score was presented and each reviewer was able to adjust their score if they considered it necessary based on the rationales presented. The scores were then averaged to produce a mean score. A score of 7 or 8 was anticipated to be a reasonable threshold to identify better bets. The fact that this threshold was not attained cannot be judged as a conclusive assessment of the merit of the interventions but rather may be indicative of the quality and quantity of studies.