In this article, we explain how to obtain the MLE (Maximum Likelihood Estimate) for the binomial distribution. Our goal is to obtain an estimate for the parameter ‘p’ which denotes the probability of success in the binomial distribution.
The MLE (Maximum Likelihood Estimate) for the parameter ‘p’ is given as,
p = X̄ / N
Here, X̄ denotes the mean of the sample values.
Suppose that X1, X2,…, and Xn are sample values drawn from a binomial distribution with parameters ‘N’ and ‘p’. That is, X1, X2,…,Xn ~ Binom(N,p).
Step 1: Write the PDF of the Binomial Distribution and take the product to find the Likelihood function.
The probability distribution function of the binomial distribution for each of the random variables Xi (i=1,2,…,n) is given as,
f(xi; N,p) = NCxi pxi (1-p)N-xi
We find the likelihood function L by taking the product of f(xi; N,p) from i=1 to n.
L = ∏ f(xi; a,b) = NCx1 px1 (1-p)N-x1 x NCx2 px2 (1-p)N-x2 x…x NCxnpxn(1-p)N-xn
L = K pΣxi (1-p)Σ(N-xi) = K pΣxi (1-p)nN – Σxi
Here, K = ∏ NCxi , is a constant that does not depend on ‘p’.
Step 2: Take the Logarithm of the Likelihood Function to find the Log-Likelihood Function
The log-likelihood function is given as,
log(L) = log(K) + (Σxi) log(p) + (nN – Σxi) log(1 – p)
Step 3: Take the Derivative of the Log-Likelihood Function
The derivative of the log-likelihood function with respect to ‘p’ is given as,
d/dp [log (L)] = (Σxi)/p – (nN – Σxi)/(1 – p)
Note that since K is a constant not depending on ‘p’, the derivative of log(K) is equal to 0.
The derivative of log(p) is 1/p and the derivative of log(1 – p) is -1/(1-p) which introduces the minus sign in the above expression.
Step 4: Put the expression for the derivative above equal to 0 and solve for p
Since we want to find the maxima for the likelihood function, this means that we must put the derivative calculated above to be equal to zero. We have,
(Σxi)/p – (nN – Σxi)/(1 – p) = 0
⇒ (Σxi)/p = (nN – Σxi)/(1 – p)
⇒ (Σxi)(1-p) = (nN – Σxi)p (By Cross Multiplication)
⇒ Σxi – pΣxi = nNp – pΣxi
Canceling pΣxi from both sides we get,
Σxi = nNp
Dividing by ‘n’ on both sides, since X̄ = Σxi/n we get,
X̄ = Np
⇒ p = X̄ / N
This gives the Maximum Likelihood Estimate for the parameter ‘p’ of the binomial distribution.