Computational analyses: Expected vs Realized RPE

We submit this explanation in response to the reader's question: "When calculate expected prediction error, what do you mean by expected and realized RPE? I can understand how expected RPE is calculated(R-Q). But how realized RPE is calculated?"

Our choice of this terminology (i.e., ‘expected’ vs ‘realized reward prediction errors’) is based on the earlier work by Drs. Preuschoff, Quartz, and Bossaerts (2008, 2010; please see below for citations) on the neural mechanisms of risk learning. ‘Expected prediction error’ or ‘expected risk’ is the inferred value of average prediction error learned based on experienced prediction errors across trials: E[(R-Q)²]. ‘Realized prediction error’ or ‘realized risk’ is the prediction error experienced on a given trial. In other words, the ‘realized prediction error’ is the difference between the reward and expected value on a given trial that the reader referred to in the question: R-Q. On tasks in which the level of risk or uncertainty associated with options in not known, but instead needs to be learned thought experience, agents can form expectations about the average size of prediction errors across trials (‘expected prediction error’ or ‘expected risk’). These ‘expected prediction error’ signals are thought to be present at the beginning of the trial; at the end of the trial ‘expected prediction errors’ are compared to ‘realized prediction errors’ to compute ‘risk prediction errors’ (analogous to ‘reward prediction errors). These ‘risk prediction errors’ can then be used to update ‘expected predicted errors across trials’.

Preuschoff K, Quartz SR, Bossaerts P (2008). Human insula activation reflects risk prediction errors as well as risk. J Neurosci 28:2745–2752.  https://doi.org/10.1523/JNEUROSCI.4286-07.2008

Bossaerts, P (2010). Risk and risk prediction error signals in anterior insula. Brain Struct Funct 214, 645–653. https://doi.org/10.1007/s00429-010-0253-1

Category