Motivation

Imagine that we assume a certain random variable to be distributed according to some distribution $X\sim f(\theta)$ and that we wish to use a sample of data to estimate the population parameter $\theta$. For example, we may be interested in estimating either the mean $\mu$ or the variance $\sigma^2$ (or both) of a variable that is thought to be normally distributed $X\sim N(\mu,\sigma^2)$. A single value point estimate $\hat{\theta}$ may be obtained by choosing a suitable sample statistic $\hat{\theta}=t(x)$, for example, the sample mean $t(x)=\overline{x}$ provides a simple (yet far from unique) way of estimating the population mean $\mu$. However, because sample sizes are finite, the sample estimate is only an approximation to the true population value - another sample from the same population would give a different value for the same sample statistic. Therefore, rather than give single value point estimates, it is better to use the information in the sample to provide a range of possible values for $\theta$ known as an interval estimate. To take account of the sampling uncertainty caused by finite sample size, it is necessary to consider the probability distribution of sample statistics in more detail.