python/ML

[Probability] Bayes Rule

rudgh99_algo 2025. 8. 3. 13:36

According to the definition of conditional probability,
P(X = x | Y = y) = p(X = x, Y = y) / p(Y = y).

We can rewrite the numerator using the product rule:
p(X = x, Y = y) = p(X = x) × p(Y = y | X = x).

We can also rewrite the denominator using the law of total probability:
p(Y = y) = Σ over x' [ p(X = x') × p(Y = y | X = x') ].

For example, if the event Y = y corresponds to "ate melon",
then the total probability that someone ate melon can be computed by summing:

  • the probability that a man ate melon, and
  • the probability that a woman ate melon.

This is because we are considering all possible values of X (e.g., male, female),
and we weight each conditional probability (like "ate melon given man")
by the prior probability of that group (like "probability of being male").

This is a direct application of the law of total probability.