Alternative hypotheses

So far we have been considering simple situations in which the alternative hypothesis is just the complement of the null hypothesis (e.g. $H_0:\mu=\mu_0$ and $H_1:\mu\neq\mu_0$). The rejection region in such cases includes the tails on both sides of $t=0$, and so the tests are known as two-sided tests or two-tailed tests. However, it is possible to have more sophisticated alternative hypotheses where the alternative hypothesis is not the complement of the null hypothesis. For example, if we wanted to test whether means were not just different but were larger than the population mean, we would use instead these hypotheses $H_0:\mu=\mu_0$ and $H_1:\mu>\mu_0$. The null hypothesis would be rejected in favour of the alternative hypothesis only if the sample mean was significantly greater than the population mean. In other words, the null hypothesis would only be rejected if the test statistic fell in the rejection region to the right of the origin. This kind of test is known as a one-sided test or one-tailed test. One-sided tests take into account more prior knowledge about how the null hypothesis may fail.

Table: The four possible situations that can arise in hypothesis testing

	$H_0$ true	$H_1$ true
$p>\alpha$	Correct non rejection	Missed rejection (Type II error)
don't reject $H_0$	probability $1-\alpha$	probability $\beta$
$p\leq\alpha$ reject	False rejection (Type I error)	Correct rejection
reject $H_0$	probability $\alpha$	probability $1-\beta$

Table 6.1 shows the four possible situations that can arise in hypothesis testing. There are two ways of making a correct decision and two ways of making a wrong decision. The false rejection of a true null hypothesis is known as a Type I error and occurs with a probability exactly equal to the level of significance $\alpha$ for a null hypothesis that is true. In the legal example, this kind of error corresponds to the conviction of an innocent suspect. This kind of error is made less frequent by choosing $\alpha$ to be a small number typically 0.05, 0.01, or 0.001. The missed rejection of a false null hypothesis is known as a Type II error and corresponds to failing to convict a guilty suspect in the legal example. For a true alternative hypothesis, type II errors occur with an unspecified probability $\beta$ determined by the sample size, the level of significance, and the choice of null hypothesis and test statistic. The probability $1-\beta$ is known as the power of the test and this should ideally be as large as possible for the test to avoid missing any rejections. There is invariably a trade off between Type I and Type II errors, since choosing a smaller $\alpha$ leads to fewer overall rejections, and so fewer type I errors but more type II errors. To reduce the number of type II errors it is a good idea to choose the null hypothesis to be the simplest one possible for explaining the population (``principle of parsimony'').