Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
What is Bayes Theorem
3.
Proof of Bayes Theorem
4.
Examples Explaining The Application Of This Theorem In Problems
5.
FAQs
6.
Key Takeaways
Last Updated: Mar 27, 2024

Bayes Theorem

Author SHIKHAR SONI
0 upvote
Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

The article discusses and explores the Bayes Theorem, which is one of the most fundamental ideas used in Probability theory as a means of revising predictions in light of new evidence or parameters that impact predictions. The article aims to help you understand the idea behind the Bayes theorem's various use cases and to help you get to practice with a few solved examples.

What is Bayes Theorem

Refer to the article on conditional probability before proceeding with this article. Bayes theorem states that the probability of an event (A) occurring, conditioned on the occurrence of an event (B), is equal to the product of the probability of event (B) given event (A) and the probability of the first event (A).

The Bayes theorem formula is P(A | B) = (P(B | A) * P(A)) / P(B). Here, P(X) is the non-zero probability that event X occurs in the sample space S. The formula for Bayes Theorem can be easily derived using substitution and basic conditional probability. The process to do this is described in the section below.

Bayes Theorem has various applications some of which are:

  1. It is commonly used in diagnosis, where we examine the probability of a particular diagnosis given various factors such as symptoms, signs or test outcomes.
  2. One of the most famous applications of this theorem is in Naive Bayes Classifier Algorithms (a family of algorithms based on Bayes theorem), used in Machine Learning.
Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Proof of Bayes Theorem

Using the basic idea of conditional probability, we can state eq. (1) and (2) below.

P(A | B) = P(A ⋂ B) / P(B) -(1)

P(B | A) = P(A ⋂ B) / P(A) -(2)

Here rearranging eq. (1) and (2) we obtain (3) and (4) as given below.

P(A ⋂ B) = P(A | B) * P(B) -(3)

P(A ⋂ B) = P(B | A) * P(A) -(4)

Using eq. (3) and (4) we get eq. (5)

P(A | B) * P(B) = P(B | A) * P(A) -(5)

P(A | B) = (P(B | A) * P(A)) / P(B) On rearranging eq. (5)

We have obtained our well known Bayes formula using a couple of substitutions and basic knowledge of conditional probability.

Examples Explaining The Application Of This Theorem In Problems

1. A bag B1 contains ten balls, 4 black and 6 white balls. On the other hand, we have another bag B2 containing ten balls, 6 black and 4 white balls. A black ball is drawn at random from the bag. Find the probability that the ball was drawn from bag B2.

Soln.

We have two events, Eb1 (event of choosing bag B1) and Eb2 (event of choosing bag B2), and even X (event of choosing a black ball).

The two bags are identical externally, and there's no bias driving the choice of the bags. Hence, P(Eb1) = P(Eb2) = ½.

P(A|Eb1) = 4 / 10 = 2 / 5 (the probability of choosing a black ball from the bag B1)

P(A|Eb2) = 6 / 10 = 3 / 5 (the probability of choosing a black ball from the bag B2)

 

We need to find P(Eb2|A), i.e., the probability of the bag being B2 given the ball selected from the bags was black.

P(A) = P(A|Eb1) * P(Eb1) + P(A|Eb2) * P(Eb2)

P(A) = 2 / 5 * 1 / 2 + 3 / 5 * 1 / 2

P(A) = 1 / 2

 

P(Eb2|A) = (P(A|Eb2) * P(Eb2)) / (P(A)) -using Bayes formula

P(Eb2|A) = (3 / 10) / (1 / 2) = 3 / 5

Ans = 3 / 5

2. A truck can take three lanes, lane L1 has a probability of 10% that the truck will be late for delivery (because of traffic and bad roads), lane L2 has a probability of 20% that the truck will arrive late, and lane L3 has a probability of 5% that the truck will be late. Given that the truck was late find the truck driver's probability of choosing lane L3.

Soln.

There are three events El1 (the event that L1 lane is chosen), El2 (the event that L2 lane is chosen), El3 (the event that L3 lane is chosen). And also the event L (the event that the truck will be late for delivery).

All the lanes are identical, and any of them can be chosen randomly.

P(El1) = P(El2) = P(El3) = 1 / 3.
 

P(L|El1) = 10 / 100 = 1 / 10 (the probability that the truck is late if the lane chosen is L1)

P(L|El2) = 20 / 100 = 1 / 5 (the probability that the truck is late if the lane chosen is L2)

P(L|El3) = 5 / 100 = 1 / 20 (the probability that the truck is late if the lane chosen is L3)
 

P(L) = P(L|El1) * P(El1) + P(L|El2) * P(El2) + P(L|El3) * P(El3)

P(L) = 1 / 10 * 1 / 3 + 1 / 5 * 1 / 3 + 1 / 20 * 1 / 3

P(L) = 7 / 60

We need to find P(El3 | L) = P(L | El3) * P(El3) / P(L), i.e., The probability that the lane L3 is chosen given that truck is late for delivery.

P(El3 | L) = (1 / 20 * 1 / 3) / (7 / 60)

P(El3 | L) = 1 / 7

Ans = 1 / 7

FAQs

1. What are the applications of the Bayes Theorem?

Bayes theorem has applications in various fields such as medicine, engineering, sports, etc. It's also used in the Naive Bayes family of classifier algorithms. These algorithms allow fast predictions, good accuracy, ease of implementation and, in addition, require very small datasets to work.

2. What led to the name 'Bayes' Theorem?

The theorem is named after the Reverend Thomas Bayes. He was the first to use conditional probability to device an algorithm that used evidence to determine the limits of an unknown parameter.

3. What are the advantages of studying the Bayes Theorem?

Questions related to Bayes theorem are often asked in various competitive exams such as GATE. This topic is also foundational before one dives into various streams such as Machine Learning.

4. What is the formula of the Bayes Theorem?

The Bayes formula is as follows,

P(A | B) = (P(B | A) * P(A))/ (P(B))

P(A) and P(B) are non zero probabilities of the event A and B occurring in the sample space S.

5. State a  common use case of Bayes Theorem in problems.

Bayes theorem is commonly used in problems to find the reverse conditional probability of two events, i.e. if P(A | B) is known, then P(B | A) can be found if P(A) and P(B) are also known.

Key Takeaways

In this article, we have extensively discussed the Bayes Theorem and tried to cover the formula, examples and the importance of understanding this topic. Refer to the following articles on probability and conditional probability as pre-requisites for this article. It may also be a good idea to refer to the article on Naive Bayes to get more clarity about the theorem and get an introduction to Machine Learning algorithms. 

Refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. 

Enroll in our courses and refer to the mock test and problems available.

Take a look at the interview experiences and interview bundle for placement preparations.

Do upvote our blog to help other ninjas grow. 

Happy Coding!

Previous article
Terms related to Probability
Next article
Uniform Distribution
Live masterclass