Bayes Theorem

Proof of Bayes Theorem

Using the basic idea of conditional probability, we can state eq. (1) and (2) below.

P(A | B) = P(A ⋂ B) / P(B) -(1)

P(B | A) = P(A ⋂ B) / P(A) -(2)

Here rearranging eq. (1) and (2) we obtain (3) and (4) as given below.

P(A ⋂ B) = P(A | B) * P(B) -(3)

P(A ⋂ B) = P(B | A) * P(A) -(4)

Using eq. (3) and (4) we get eq. (5)

P(A | B) * P(B) = P(B | A) * P(A) -(5)

P(A | B) = (P(B | A) * P(A)) / P(B) On rearranging eq. (5)

We have obtained our well known Bayes formula using a couple of substitutions and basic knowledge of conditional probability.

Examples Explaining The Application Of This Theorem In Problems

1. A bag B1 contains ten balls, 4 black and 6 white balls. On the other hand, we have another bag B2 containing ten balls, 6 black and 4 white balls. A black ball is drawn at random from the bag. Find the probability that the ball was drawn from bag B2.

Soln.

We have two events, Eb1 (event of choosing bag B1) and Eb2 (event of choosing bag B2), and even X (event of choosing a black ball).

The two bags are identical externally, and there's no bias driving the choice of the bags. Hence, P(Eb1) = P(Eb2) = ½.

P(A|Eb1) = 4 / 10 = 2 / 5 (the probability of choosing a black ball from the bag B1)

P(A|Eb2) = 6 / 10 = 3 / 5 (the probability of choosing a black ball from the bag B2)

We need to find P(Eb2|A), i.e., the probability of the bag being B2 given the ball selected from the bags was black.

P(A) = P(A|Eb1) * P(Eb1) + P(A|Eb2) * P(Eb2)

P(A) = 2 / 5 * 1 / 2 + 3 / 5 * 1 / 2

P(A) = 1 / 2

P(Eb2|A) = (P(A|Eb2) * P(Eb2)) / (P(A)) -using Bayes formula

P(Eb2|A) = (3 / 10) / (1 / 2) = 3 / 5

Ans = 3 / 5

2. A truck can take three lanes, lane L1 has a probability of 10% that the truck will be late for delivery (because of traffic and bad roads), lane L2 has a probability of 20% that the truck will arrive late, and lane L3 has a probability of 5% that the truck will be late. Given that the truck was late find the truck driver's probability of choosing lane L3.

Soln.

There are three events El1 (the event that L1 lane is chosen), El2 (the event that L2 lane is chosen), El3 (the event that L3 lane is chosen). And also the event L (the event that the truck will be late for delivery).

All the lanes are identical, and any of them can be chosen randomly.

P(El1) = P(El2) = P(El3) = 1 / 3.
 

P(L|El1) = 10 / 100 = 1 / 10 (the probability that the truck is late if the lane chosen is L1)

P(L|El2) = 20 / 100 = 1 / 5 (the probability that the truck is late if the lane chosen is L2)

P(L|El3) = 5 / 100 = 1 / 20 (the probability that the truck is late if the lane chosen is L3)
 

P(L) = P(L|El1) * P(El1) + P(L|El2) * P(El2) + P(L|El3) * P(El3)

P(L) = 1 / 10 * 1 / 3 + 1 / 5 * 1 / 3 + 1 / 20 * 1 / 3

P(L) = 7 / 60

We need to find P(El3 | L) = P(L | El3) * P(El3) / P(L), i.e., The probability that the lane L3 is chosen given that truck is late for delivery.

P(El3 | L) = (1 / 20 * 1 / 3) / (7 / 60)

P(El3 | L) = 1 / 7

Ans = 1 / 7

FAQs

1. What are the applications of the Bayes Theorem?

Bayes theorem has applications in various fields such as medicine, engineering, sports, etc. It's also used in the Naive Bayes family of classifier algorithms. These algorithms allow fast predictions, good accuracy, ease of implementation and, in addition, require very small datasets to work.

2. What led to the name 'Bayes' Theorem?

The theorem is named after the Reverend Thomas Bayes. He was the first to use conditional probability to device an algorithm that used evidence to determine the limits of an unknown parameter.

3. What are the advantages of studying the Bayes Theorem?

Questions related to Bayes theorem are often asked in various competitive exams such as GATE. This topic is also foundational before one dives into various streams such as Machine Learning.

4. What is the formula of the Bayes Theorem?

The Bayes formula is as follows,

P(A | B) = (P(B | A) * P(A))/ (P(B))

P(A) and P(B) are non zero probabilities of the event A and B occurring in the sample space S.

5. State a  common use case of Bayes Theorem in problems.

Bayes theorem is commonly used in problems to find the reverse conditional probability of two events, i.e. if P(A | B) is known, then P(B | A) can be found if P(A) and P(B) are also known.

Key Takeaways

In this article, we have extensively discussed the Bayes Theorem and tried to cover the formula, examples and the importance of understanding this topic. Refer to the following articles on probability and conditional probability as pre-requisites for this article. It may also be a good idea to refer to the article on Naive Bayes to get more clarity about the theorem and get an introduction to Machine Learning algorithms. 

Refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. 

Enroll in our courses and refer to the mock test and problems available.

Take a look at the interview experiences and interview bundle for placement preparations.

Do upvote our blog to help other ninjas grow. 

Happy Coding!