Pairwise coupled network

We considered a network of 12 neurons arranged in a ring. We defined a reward function that was equal to 1 if exactly 4 adjacent neurons were active, and 0 otherwise. We defined the coding cost as described in the previous section, to penalise deviations from the population averaged mean firing rate.

We approximated the value function by a quadratic function,

We optimised the parameters of this value function using the algorithm described previously, in the Methods section entitled ‘Approximate method for larger networks’, with λ = 0.05. We used batches of n_batch = 40 samples, and updated the target parameters after every n_epoch = 100 batches.

Concretely, substituting in the quadratic value function described above into Eq 43, we obtain the following updates for the network couplings:

where,

We inferred the reward function from the inferred network couplings, J and h as described in the Methods section entitled ‘Approximate methods for larger networks’. As described previously, this problem is only well-posed if we assume a low-d parametric form for the reward function, or add additional assumptions. We therefore considered several different sets of assumptions. For our initial ‘sparse model’, we set up a linear programming problem in which we minimised l₁ = ∑_σ r(σ), under the constraint that the reward was always greater than 0 while satisfying the optimality criterion given by Eq 47. For the ‘pairwise model’ we assumed that r = ∑_i,j W_ij σ_i σ_j. We fitted the parameters, W_ij, so as to minimise the squared difference between the left and right hand side of Eq 47. Finally, for the ‘global model’ we assumed that r = ∑_j δ_j,m W_j, where m is the total number of active neurons and δ_ij is the kronecker-delta. Parameters, W_j were fitted to the data as for the pairwise model.

Finally, for the simulations shown in Fig 5, panels B-D, we ran the optimisation with λ = 0.1, 0.01 and 0.1, respectively. For panel 3C we removed connections between neurons separated by a distance of 3 or more on the ring. For panel 3D we forced two of the neurons to be continuously active.