Joint and conditional probabilities

We are often interested in the case when two events happen at the same time. For example, to get snow falling on the ground, it is necessary that two events, $\{A_1$=``precipitating cloud''} and $\{A_2$=``boundary layer below freezing''} occur at the same time. The probability of two events happening at the same time, $\Pr(A_1$ and $A_2)$, is known as the joint probability of events $A_1$ and $A_2$. For mutually exclusive events that never occur at the same time, the joint probability is zero.

It is also useful to define the probability of an event GIVEN that another event has happened. This approach is very powerful and is known as conditioning. The conditional probability of an event $A_1$ given $A_2$ (i.e. conditioned on $A_2$) is defined as

$$\Pr(A_1\vert A_2)=\frac{\Pr(A_1\mbox{ and } A_2)}{\Pr(A_2)}$$ (3.1)

For example, to estimate the probability of rain during El Niño episodes we could use a conditional probability conditioned on El Niño events (rather than all events).

For independent events, $\Pr(A_1$ and $A_2)=\Pr(A_1)\Pr(A_2)$ and so the conditional probability $\Pr(A_1\vert A_2)=\Pr(A_1)$ - in other words, conditioning on independent events does not change the probability of the event. This is the definition of independence.

By equating $\Pr(A_1$ and $A_2)=\Pr(A_1\vert A_2)\Pr(A_2)$ and $\Pr(A_1$ and $A_2)=\Pr(A_2\vert A_1)\Pr(A_1)$, one can derive the following useful identity

$$\Pr(A_1\vert A_2)=\frac{\Pr(A_2\vert A_1)\Pr(A_1)}{\Pr(A_2)}$$ (3.2)

This is known as Bayes' theorem and provides a useful way of getting from the unconditioned prior probability $\Pr(A_1)$ to the posterior probability $\Pr(A_1\vert A_2)$ conditioned on event $A_2$. Event $A_2$ is invariably taken to be the occurence of the sample of available data. In other words, by conditioning on the available data, it is possible to obtain revised estimates of the probability of event $A_1$.