Model validation

We compared the HGF to two simpler learning models, one (RW; Rescorla and Wagner 1972) in which a single learning rate parameter is estimated for each participant and another (SK1;⁴⁵ with a dynamic learning rate that varies trial-to-trial but does not learn about volatility. As a third comparison model (HGF_alt) we fit the same perceptual model for the HGF but changed the linear mapping to responses such that the surprise(t) predictor did not contain an interaction with stimulus noise.

To disambiguate these alternative explanations (models) for the participants’ behavior, we used BMS, which evaluates the relative plausibility of competing models in terms of their log evidences while adjusting for the trade-off between accuracy (fit) and complexity.⁸¹ The three level HGF is shown to be the winning model (Figure S1A)

We also simulated 100 virtual agents using the mean parameters of the propranolol and placebo groups. Categorisation of these RTs according to trial type (E, UE) and stimulus noise (H, M, L) showed that the main behavioral effects can be recapitulated by the model (Figure S1B). Statistical analysis of this simulated data confirmed the same results as the real participant behavior, notably a significant linear main effect of stimulus noise (F(1,197) = 23.06, p < 0.001), and a linear noise ^∗ drug group interaction (F(1,197) = 4.53, p = 0.035).

We inverted the parameters of the winning model using simulated data to determine if we could recover the parameters and the statistical differences reported between the groups in the main text. First, we simulated 200 datasets using mean parameters for the propranolol and placebo groups and fit the model to these data (Figure S1C). A binary logistic regression predicting group (propranolol, placebo) from the recovered ω2 and ω3 parameters, was significant overall (X2 = 270.73, df = 2, r² = 0.65, p < 0.001), with ω2 and ω3 both significantly lower in the propranolol group (b = 0.78, p < 0.001; b = 1.2, p < 0.001). Furthermore, to show that we can also recover parameters across the range of values estimated from real participant data we also simulated data using the parameters from the individual participant model fits reported in the main text, averaging across 20 simulations per participant (Figure S1D). For these parameters, the binary logistic regression predicting group approached significance (X2 = 4.99, df = 2, r² = 0.157, p = 0.08). In this model ω2 was significantly lower in the propranolol group (b = 0.55, p = 0.042), whereas ω3 was numerically, but not statistically lower in the propranolol group (b = 0.09, p = 0.59). Taken together, these analyses broadly recapitulate the primary findings reported in the main text, where ω2 is significantly reduced under propranolol but the reduction in ω3 is less reliable, not reaching significance at a 2-tailed level.

Finally, we examined the two-way mixed effects intra-class correlation coefficients (ICC) for the real ω2 and ω3 parameters estimated for each single subject against the mean of the recovered parameters for each participant. Our ICC analysis sought absolute agreement between measures, and so accounted for systematic differences.⁸² The ICC between the real and recovered values of ω2 was 0.860 (CI: 0.69-0.94; p < 0.001), and the ICC between the real and recovered values of ω3 was 0.870(CI: 0.75-0.93; p < 0.001). ICC coefficients of 0.81–.1.00 are considered ‘almost perfect’⁸³, suggesting that the reliability of the parameter recovery for this model is encouraging.