Binomial to Poisson

A brief explanation of Poisson and how to derive it from Binomial distribution

As a starting point, Binomial distribution helps us to find the probability of discreet event(s) that are independent of each other. An important instance might be the probability of an error occur in production line, if we assume that these errors are not related to each other. Let’s say we want to calculate the probability of at least one errors appear given that 3 production attempt. Then we can simply represent each case as, which is equal to the permutation of :

\(r = 1 \rightarrow [E,W,W] \;\; [W,E,W] \;\; [W,W,E] = C^3_1\)

\(r=2 \rightarrow [E,E,W] \;\; [E,W,E] \;\; [W,E,E] = C^3_2\)

\(r=3 \rightarrow [E,E,E] = C_3^3\)

These three situations represent the case where only one error occurs among three production phase. If we calculate the probability of each case then we can find our main purpose. However, in order to find that we have to have a priori probability for error occuring. Let’s say it is 0.1. Rather than calculating probabilities of each case, we can find the probability of no one error occurs and then extract it from 1, since the remainings are the ones that we want to calculate

\(r=0 \rightarrow [W,W,W] = C^3_0\)

\(P(r=0) = 0.9^3 =0.729\)

\(P(r\geq1) = 1- P(r=0) = 0.271\)

Thus we can formulate the density function of Binomial distribution as:

\(P(X=r) = C_r^n P^r (1-P)^{n-r}\)

We have to switch to Poisson distribution. The best explanation for Poisson is that it is an approximation of Binomial distribution when the observations (n) have huge values which will make very hard to compute the combinations. Poisson has another parameter that represents the average number of occurences for certain numbers of observations, which is lambda. Though, we can say that this parameter is equal to the number of observations times the probability of event occur.

\(\lambda = np\)

\(P(X=r) = \frac{\lambda^r e^{-\lambda}}{r!}\)

As we have indicated, it is useful when we have data with huge numbers of observations, like number of phone calls in a certain time period. In this case one will encounter with question such that the mean value (lambda parameter) is given based on a certain time or interval and will require you to find the probability of a certain occurence number.

Let’s look at how can we find the density function of Poisson distribution from Binomial distribution. We know that lambda equals to the number of observations times a priori probability. So we can replace probability with lambda divided by n.

\(P(X=r) = \frac{n!}{r!(n-r)!} (\frac{\lambda}{n})^r (1-\frac{\lambda}{n})^n(1-\frac{\lambda}{n})^{-r}\)

Since we have indicated that Poisson is used when huge numbers, we will take the limit of this function when n goes to infinity and we can take constants out :

\(\frac{\lambda^r}{r!} \; \textrm{lim}_{n \rightarrow \infty} \frac{n!}{(n-r)!} (\frac{1}{n^r}) (1-\frac{\lambda}{n})^n(1-\frac{\lambda}{n})^{-r}\)

Let’s look at the first terms. The division of factorials will give us multiplication n from n-r+1, since all the remaining numbers are extracted out by (n-r)!. And also there will be r numbers of multiplication left, thus we can rewrite the equation as

\(\frac{n(n-1)(n-2)...(n-r+1)}{n^r} = \frac{n}{n}\frac{(n-1)}{n}\frac{(n-2)}{n}...\frac{(n-r+1)}{n}\)

Since n goes to infinity, each division will cancel each out, which will give us 1. Then we can look at the last term with the exponent of -r. Since n goes to infinity, the division reaches to 0, so 1 exponent to anything will be equal to 1.

So, we are only left with the term with exponent with n. It resembles to the function when it goes to infinity it equalizes to euler number.

\(e = \textrm{lim}_{n \rightarrow \infty}(1+\frac{1}{x})^x\)

So if we replace x with necessary compenent we will get this function.

\(x = \frac{n}{-\lambda}\)

\((1-\frac{\lambda}{n})^n = (1+\frac{1}{x})^{x(-\lambda)} = e^{-\lambda}\)

We have solved all of the terms in our function. So finally we can write the formula for density function of Poisson distribution ast (do not forget the terms that we took out from limit)

\(P(X=r) = \frac{ \lambda^r e^{-\lambda}}{r!}\)