2.11. Performance assessment

For performance assessment, matching results were compared against the manual annotation.

We calculated the following performance metrics to assess the performance of FLAP:

where c is the number of addresses that are correct matches, n is the number of address samples, and p is the number of addresses that can be matched in manual annotation (see Section 2.10).

As described in Section 2.6, FLAP also outputs the probability of match as a score. We use the score to illustrate the usability aspect of FLAP. AccuracyinTop5 was defined as any correct mapping in the five candidates with the highest scores. Given the score s and the threshold of the score t, a cross-tabulation is created for the confidence score and matching correctness:

The following metrics are calculated:

A lift curve was generated to demonstrate the usability of FLAP at different confidence score thresholds (Figure 3).

Performance of FLAP at different thresholds of classifier score.