Study design and methods

Comparing the datasets involved a two-stage process: first, to ensure comparability by assessing whether individual events met our inclusion criteria, and second, to determine the overlap between datasets, including efforts to determine the extent of underreporting. We used a basic event record comparison (based on record-linkage from the Multiple Systems Estimation or capture/recapture methodology [48]) to determine overlap between the datasets and the extent of under-reporting, as well as a more detailed event comparison to identify patterns in the nature, source, or type of overlap. Even though our data did not meet the strict conditions for a capture-recapture analysis [48], we conducted this analysis to determine a crude estimate of the possible extent of underreporting. Using the Chapman estimate [49] as a conservative parameter, we used the equation N = (r1 + 1)(r2 + 1)/ r12-1 -1, where r1 is the number of events in the first dataset and r2 references the second dataset, and r12 is the number of events captured in both datasets, to arrive at an estimate of for the minimum number of events in these 31 countries ((165+1)(122+1)33-1-1=x). These findings are reported in the Results section.

To prepare the datasets for analysis, we first eliminated from our analysis all events outside of the WHE framework (see Table ​Table1)1) and in the Syrian Arab Republic. The WHE framework designates WHO responses in grade 1–3 emergencies as well as ‘non-graded protracted emergencies’, with grade 3 referring to the most acute emergencies in the 2016 update—the latest date available for our 2017 data. Thirty-one of these were emergency contexts, with an additional 16 protracted crises. Although WHO surveillance concentrated on the countries in Table ​Table1,1, the WHO data did include events outside these countries. To maximise our dataset, our analysis encompassed all 47 countries in Table ​Table11 except Syria. Syria events comprised a significant proportion of all available data for WHO and the SiND, but due to incomplete or missing information it was impossible to eliminate double-counted events or to ensure accuracy in matching event records.

WHE Grade 1–3 emergencies and protracted emergencies (from 2016) [45]

Iraq

Nigeria

South Sudan

Syrian Arab Republic

Yemen

Angola

Cameroon

Central African Republic (CAR)

Democratic Republic of the Congo (DRC)

Ecuador

Ethiopia

Haiti

Libya

Myanmar

Niger

Ukraine

United Republic of Tanzania

Afghanistan

Bangladesh

Democratic People's Republic of Korea (DPRK)

Fiji

Indonesia

Kenya

Mali

Nepal

Pakistan

Papua New Guinea

Sri Lanka

Thailand

The Philippines

West Bank and Gaza Strip

Burkina Faso

Chad

Colombia

Djibouti

Egypt

Guatemala

Honduras

Jordan

Lebanon

Mauritania

Senegal

Somalia

Sudan

The Gambia

Turkey

Zimbabwe

Next, two coders separately examined each individual event by dataset (WHO or SiND) to determine whether the event matched our inclusion criteria. The inclusion criteria required that an event met the WHO definition of attack (defined as ‘any act of verbal or physical violence or obstruction or threat of violence that interferes with the availability, access and delivery of curative and/or preventive health services during emergencies’) and that occurred within one of the 47 WHE programme countries for 2016.

Both coders also separately defined a match status for each individual event (see Table ​Table2;2; see also [9] for another study using this methodology). When an event appeared in both datasets, each coder identified the ID number of the corresponding event in the other dataset based on location and identifying information (location, date of the attack, name or type of facility, name or affiliation of victim(s), and perpetrator type). We then compared our results (see Fig. 1). Where coding discrepancies existed (either based on inclusion criteria or defined as a ‘possible match’), we discussed the event in question and reached a consensus about inclusion and match status, moving all ‘possible matches’ to either the definite match or unique event category. This process identified three duplicate events, out of 290 events, which we excluded from the final analysis. In several cases, we conducted additional web searches based on the event descriptions in order to determine match status.

Basis for determining match status

Events for which one or more of the following appeared to match: (1) the date of the attack, (2) location or (3) the name of the facility or of the victim

For events in this category, the coders discussed and reached consensus, moving all possible match events to either the unique or definite match category

We report the findings of our comparison with descriptive statistics in our Results section below (see Data Comparison).