2.5. Bayesian Binary Logistic Regression Model

Binary logistic regression model is used to explain the probability of a binary response variable as function of some covariates [33]. Bayesian logistic regression procedure is used to make inference for the parameters of a logistic regression model. Bayesian statistical methods are becoming ever more popular in applied and fundamental research [34]. This estimation allows a detailed inference from parameters for any arbitrary sample size [35]. Based on the previous study, Bayesian estimation is more accurate than the maximum likelihood estimation even under noninformative prior [36], because this estimation also allows for probabilistic interpretations to the parameters. In general, various studies conclude that Bayesian logistic regression performs better in posterior parameter estimation. There is significant bias of maximum likelihood estimation in small samples, and this weakness can be opposed by using Bayesian logistic regression as an alternative method. Since Bayesian approach is flexible, it does not need to conform to challenging assumptions as proposed in the maximum likelihood method. The Bayesian framework is the mixture of the likelihood function and the prior distribution to develop the posterior distribution [37].

Thus, the response variable y_i follows a Bernoulli distribution with probability π. The likelihood function is the probability density function of the data which is seen as a function of the parameter treating the observed data as fixed quantities. For a given sample size n, the likelihood function is given as [37]:

Prior information is the special feature of Bayesian estimation. In some cases, we may not be in possession of enough prior information to aid in drawing posterior inferences. From a Bayesian point of view, this lack of information is still important to consider and incorporate into our statistical specifications [34]. The most common priors for logistic regression parameters are normal, and the prior distribution of these parameters is given by:

The most common choice for prior mean μj is 0 for all the coefficients and large enough prior variance σ_j² to be considered as noninformative [35].

Bayesian estimation can be done from the posterior distribution, which is derived by multiplying the prior distribution of all parameters and the likelihood function of the data. Then, the posterior distribution is given as follows.

Markov Chain Monte Carlo (MCMC) methods are used to make inference for Bayesian logistic regression models to obtain the posterior distribution of estimation based on a prior distribution and the likelihood function [38]. This method becomes a popular and useful method in Bayesian inference to get information from posterior distributions. The strength of MCMC is that it can be used to draw samples from distributions even when all that is known about the distribution is how to calculate the density for different samples [39]. MCMC sampling method is one very useful class of simulation techniques, and it can simulate a series of dependent random draws from models that are often quite complex.

One popular MCMC method for constructing a Markov chain for a target density is Gibbs sampling. Sampling from the multivariate posterior distribution is not feasible but sampling from the conditional distributions of each parameter is possible; in such cases, Gibbs sampling has been found to be quite applicable. To create the Markov chain, Gibbs sampling uses a set of full conditional distributions associated with the target distribution. Gibbs sampling allows us to use the joint densities that have all other parameters set at their current values [40]. The Gibbs sampler is a special case of the Metropolis-Hastings algorithm using the ordered subupdates. All proposed updates are accepted (there is no accept-reject step). MCMC is (currently) the most general technique for obtaining samples from any posterior density [41]. The Gibbs sampler was implemented easily through WinBUGS software to solve approximates the properties of the marginal posterior distributions for each parameter.