Loss functions

Both our memorable image generators are designed to use the same loss function, the Wasserstein metric combined with a component which calculates the difference between the desired and generated memorability for a given image. This training mechanism works for both single-score and VMS map memorability training examples.

The loss function is designed to embed a memorability predictor and contains the following components: a generator network G, a discriminator D and memorability predictor network M. Considering the latent code distribution Pz, target VMS distribution Pt, real image distribution Pr, predicted VMS distribution Pv, and generated image distribution Pg based upon the latent code z^ and v^ we define the loss function in Eq. (³). The latent code z^ is drawn from a Gaussian distribution and v from a distribution of target VMS maps, where height, width, and intensity of VMS regions is drawn from a uniform distribution. λLossgp refers to the gradient penalty loss in³⁷. α controls the strength of the memorability loss. Pg represents the probability of the generated data and Pr is the probability of the real data. The additional term controlled by the hyperparameter λ prevents the gradients inside the discriminator from violating Lipschitz continuity, whereas the first two terms evaluate the Earth-Mover distance between the generated and real distributions. The additional memorability loss, combined with the Wasserstein loss, constrains the image generation by both ‘realness’ and memorability simultaneously.