Annotation

We identified 2,584,607 notes with one or more of the phrases, and annotated 1856 text snippets randomly selected from notes that contained any of the search terms. Each snippet contains a 35-word span before and after a firearm-relevant phrase. The annotation classifications for firearm access were the following: positive (ability to determine that the veteran had current access to at least one firearm); negative (language that the veteran did not have current access to any firearms); and ambiguous (there was insufficient evidence for either a positive or negative classification from the note – an example might be that the veteran owned a firearm but it was somewhere else). Each snippet was annotated by two of the authors and disagreement adjudicated by their consensus. An inter-annotator agreement was calculated. The annotated snippets served as the reference standard in training and testing.