Development and validation of the threshold values

Step 3 is the definition and validation of the EFSI threshold values to establish a 5-scale score, coined the Enviroscore.

We defined the thresholds based on the distribution pattern of the EFSI. In line with the methodology used in the development of the Nutri-Score^72–75. In order to reflect the environmental impact distribution of the food product of the current and future market, an additional dataset was collated containing 22 representative food items (N₃ = 22). This dataset encompasses the 12 representative food items of the European Food Basket and an additional set of 10 food items including food products with a steep increase the market demand to account for potential changes in food acquisition trends. The EFSI of the additional food items was calculated as described in Step 1 (see SP1).

Afterwards, the accuracy of threshold values was validated., namely the capability of the Enviroscore to obtain assured results. In the absence of a golden standard, we compare the obtained Enviroscore results on the relative environmental impact of the food products with experts’ categorization. To obtain experts' agreement on the relative environmental impact of food items we used the Delphi method. All the information regarding the protocol, questionnaire, and results are available in Method section. In summary, the Delphi method is a technique used to gain insight into a particular topic. It uses an iterative feedback technique with a group of experts, and it has been widely used to gather expert’s opinions and consensus in numerous fields, such as identification of food safety priorities⁷⁶, definition of policies to improve population nutrition⁷⁷ or the identification of key factors affecting e-commerce⁷⁸. For the purpose of the study, we defined ‘expert’ as those who have at least three years of experience in calculating environmental impact of food items, or at least one year of experience working with life-cycle assessment method to calculate environmental footprint of food items and have working experience in European food products. The Delphi method was carried out with seven experts from February till April 2019. It consisted of three rounds of expert’s feedback through a series of semi-anonymized online surveys. Experts categorized 22 food items as very low-, low-, medium-, high-, and very high-impact. Food items were withheld only when 80% of the experts agreed in the categorization. Their categorization was then compared with the Enviroscore results of the representative products (N₃ = 22). We evaluated the agreement between the expert categorization of the food products and the categorization of the Enviroscore by calculating the weighted Kappa, a coefficient that signals the overall agreement⁷⁹. Indeed, we evaluated the accuracy of the categorization between the Enviroscore results and the experts’ opinion by a contingency table that portrays the coincidences between the categorization by Delphi experts and by the Enviroscore when assessing the same product.

Finally, we tested the performance of the categorization, namely the capacity of the Enviroscore to capture reasonable variability within the same food product. To do so, the Enviroscore results of each representative food item (N₃ = 22) was compared with the results of the same hypothetical food item (N₂ = 149). In example, the score of a representative orange is compared with the scores of hypothetical oranges (e.g., orange produced in China and transported by plane). For that comparison, a second contingency table was built to represent the coincidences and variations between the Enviroscore results of the 22 representative food items and the scores of each hypothetical item.