Fluorescein dilution

The scaling factor S_f for relating molecules of fluorescein to arbitrary fluorescent units is computed as one parameter of a fit to a model of systematic pipetting error S_p.

If we ignore pipetting error, then the model for serial dilution has an initial population of calibrant p₀ that is diluted n times by a factor of α at each dilution, such that the expected population of calibrant for the ith dilution level is:

In the case of the specific protocols used here, α = 0.5. For the fluorescein dilution protocol used in 2016 and 2017, p₀ = 3.01e15 molecules of fluorescein and in 2018, p₀ = 6.02e14 molecules of fluorescein.

The model for systematic pipetting error modifies the intended dilution factor α with the addition of an unknown bias β, such that the expected biased population b_i for the ith dilution level is:

We then simultaneously fit β and the scaling factor S_p to minimize the sum squared error over all valid dilution levels:

where ϵ is sum squared error of the fit.

The residuals for this fit are then the absolute ratio of fit-predicted to observed net mean bi/Spμ(Oi)−μ(B) for all valid levels.