The Estimation of Logistic Probability Model of Binomial Data: Simulation Comparison

Properties of various types of estimators of the regression coefficients in linear logistic regression models were considered. The estimators include those based on maximum likelihood, minimum chi-square and weighted least squares. The results of a large scale simulation investigation evaluating the moment properties of the estimators are presented for the case of logistic model with a single explanatory variable.


INTRODUCTION
Assume that y 1 , y 2 ,...,y g represent g independent binomial random variables. For ith group, i = 1, 2,…, g, let x i1 ,x i2 ,...,x ik denote the values on k explanatory variable which are thought to influence the individual trial probability of success, denoted by P i . Then the linear logistic regression model is: The regression coefficients in β are usually unknown and there are a number of well-known methods of estimation of β, such as maximum likelihood, minimum chi-square and weighted least squares.
The kernel of the log-likelihood may be written a: where p i = y i /n i denotes the observed proportion of successes in the i th group. In a matrix form the first and second order derivatives of the log-likelihood are given by: where v1 = diag ((n i P i Q i )) is a (g x g) diagonal matrix and X is the (g x (k+1) design matrix [2] . If we let 1 β ) denote the ML estimate of β and put 1 1 is given by the solution of the (k+1) equations given by D 1 = 0. Applying a standard Newton-Raphson approach [8] to solve these equations, if . The procedure is to calculate the estimates from a linear probability model and to use these as an initial values which begins finding a solution. When the difference between ( 1) is close enough to zero, the process stops. This iteration procedure may be viewed as a method of reweighted least squares.

Minimum chi-squares:
The minimum chi-square (MCS) estimator [2,4] which we denote by 2 β ) is the value of β that minimizes: The first and the second order derivatives of R (β) are: is given by the solution of the (k+1) equations given by: An iterative solution can again be found by using Newton-Raphson approach similar to that outlined for the maximum likelihood estimation procedure.

Weighted least squares:
The ML and MCS estimation methods both require an iteration method of solution, although as we have noted the estimates can be obtained using many packages (straight forward). A none iterative solution can be found by using weighted least squares (WLS), which is sometimes referred to as minimum logit chi-square estimation [9] . Defining the group empirical logits by: we have, and The WLS estimate is the value of β which minimizes: The weights w i = n i p i q i are the reciprocal of the asymptotic variances of the z i 's based on the sample proportions of successes. Denoting the WLS estimate by 3 β ) , the explicit solution is: In the results of a fairly large scale simulation investigation that compare the moment properties of the estimators for a number of sample sizes and parameter configurations when we consider a single explanatory variable. These results considerably extend the findings made by [10] who considered the particular case g = 3, n i = 1,2,3 and showed that the simple WLS method was more efficient than the ML and MCS methods of estimation under a number of success probability configurations.
Numerical results with discussions: In order to investigate the properties of the ML, MCS an WLS estimators, a large scale simulation was made for the case of a single explanatory variable with equally spaced values. Without loss of generality, the linear logistic regression model was taken as: Group numbers g = 5, 10 and sample sizes n = 25, 50, 100 were used, the simulation run size being, 1000 in each case. Three pairs of values (β 0 , β 1 ) were examined for each value of g to give coverage of markedly different configurations for the groups trial probabilities of success, as shown in Table 1. In Table 2, the variances of the estimators of β 0 and β 1 as obtained by simulation are given together with the approximate values given by 23. The results show that the ML estimators gave the largest variances for both β 0 and β 1 in all cases. The variance differences for the MCS and WLS estimators were generally extremely small. The results also show that approximation 23 which is applicable to all the estimators gave satisfactory results especially when n = 100. Table 3 gives the means square errors of the estimators and the efficiencies of the MCS and WLS estimators relative to ML estimator defined as the ratio of the mean square errors. These efficiency results suggest that the ML estimation procedure should not be   used when the groups sample sizes are less than 100. The efficiency differences between the MCS and WLS procedures are consistently small and WLS has the computational advantage of providing a non-iterative fit. Table 3: Mean square errors and percentage efficiencies of estimators relative to the ML estimators for configurations shown in Table 1 Configuration