MLE of Binomial Distribution (Step by Step Proof)

In this article, we explain how to obtain the MLE (Maximum Likelihood Estimate) for the binomial distribution. Our goal is to obtain an estimate for the parameter ‘p’ which denotes the probability of success in the binomial distribution.

The MLE (Maximum Likelihood Estimate) for the parameter ‘p’ is given as,

p = X̄ / N

Here, X̄ denotes the mean of the sample values.

Suppose that X₁, X₂,…, and X_n are sample values drawn from a binomial distribution with parameters ‘N’ and ‘p’. That is, X₁, X₂,…,X_n ~ Binom(N,p).

Step 1: Write the PDF of the Binomial Distribution and take the product to find the Likelihood function.

The probability distribution function of the binomial distribution for each of the random variables X_i (i=1,2,…,n) is given as,

f(x_i; N,p) = ^NC_xi p^x_i (1-p)^N-x_i

We find the likelihood function L by taking the product of f(x_i; N,p) from i=1 to n.

L = ∏ f(x_i; a,b) = ^NC_x1 p^x₁ (1-p)^N-x₁ x ^NC_x2 p^x₂ (1-p)^N-x₂ x…x ^NC_xnp^x_n(1-p)^N-x_n

L = K p^Σx_i (1-p)^Σ(N-x_i) = K p^Σx_i (1-p)^{nN – Σx_i}

Here, K = ∏ ^NC_xi , is a constant that does not depend on ‘p’.

Step 2: Take the Logarithm of the Likelihood Function to find the Log-Likelihood Function

The log-likelihood function is given as,

log(L) = log(K) + (Σx_i) log(p) + (nN – Σx_i) log(1 – p)

Step 3: Take the Derivative of the Log-Likelihood Function

The derivative of the log-likelihood function with respect to ‘p’ is given as,

d/dp [log (L)] = (Σx_i)/p – (nN – Σx_i)/(1 – p)

Note that since K is a constant not depending on ‘p’, the derivative of log(K) is equal to 0.

The derivative of log(p) is 1/p and the derivative of log(1 – p) is -1/(1-p) which introduces the minus sign in the above expression.

Step 4: Put the expression for the derivative above equal to 0 and solve for p

Since we want to find the maxima for the likelihood function, this means that we must put the derivative calculated above to be equal to zero. We have,

(Σx_i)/p – (nN – Σx_i)/(1 – p) = 0

⇒ (Σx_i)/p = (nN – Σx_i)/(1 – p)

⇒ (Σx_i)(1-p) = (nN – Σx_i)p (By Cross Multiplication)

⇒ Σx_i – pΣx_i = nNp – pΣx_i

Canceling pΣx_i from both sides we get,

Σx_i = nNp

Dividing by ‘n’ on both sides, since X̄ = Σx_i/n we get,

X̄ = Np

⇒ p = X̄ / N

This gives the Maximum Likelihood Estimate for the parameter ‘p’ of the binomial distribution.