Sampling distributions

The probability distribution of a sample statistic such as the sample mean is known as a sampling distribution (and should not be confused with the probability distribution of the underlying random variable). For example, it can be shown that the sample mean of $n$ independent normally distributed variables $X\sim N(\mu,\sigma^2)$ has a sampling distribution given by $\overline{X}\sim N(\mu,\sigma^2/n)$. In other words, the sample mean of $n$ normal variables is also normally distributed with the same mean but with a reduced variance $\sigma^2/n$ that becomes smaller for larger samples. Rather amazingly, the sample mean of any variables no matter how distributed has a sampling distribution often tends to normal $\overline{X}\sim N(\mu,\sigma^2/n)$ for sufficiently large sample size. This famous result is known as the Central Limit Theorem and accounts for why we encounter the normal distribution so often for observed quantities such as measurement errors etc.

The sampling distribution $f_T(.)$ of a sample statistic $T(X)$ depends on:

• The underlying probability distribution $X\sim f(.)$ determined by its population parameters $(.)=(\mu,\sigma^2,$ etc).

• The choice of the particular sample statistic. Simple analytic forms for the sampling distribution can only be derived for a few particularly simple sample statistics such as the sample mean and sample variance. In other cases, it is necessary to resort to computer based simulation methods such as resampling.

• The sample size $n$. The larger the sample size, the less the uncertainty (spread) in the sampling distribution. For example, the mean heights of two different samples of meteorologists could be quite different for small samples with $n=11$, whereas the mean heights are unlikely to be very different for large samples with $n=11000$.