Transformation of data

Transformations are widely used in statistics to make patterns easier to see, or to reduce data to standard forms. Some common methods of re-expressing data are as follows:

• Centering The sample mean (column mean) $\overline{x}$ is subtracted from the data values $x_i$ in order to obtain centered ``anomalies'' $x_i-\overline{x}$ having zero mean. All information about mean location is lost.

• Standardizing The data values are centered and then divided by their standard deviations to obtain ``normalised anomalies'' (meteorological notation) having zero mean and unit variance. All knowledge of location and scale is lost and so statistics based on standardised anomalies are unaffected by any shifts or rescaling of the original data. Standardizing makes the data dimensionless and so is useful for defining standard indices. Certain statistics are unaffected by any linear transformations such as standardization (e.g. correlation, see Chapter 3).

• Normalizing

  Normalizing transformations are non-linear transformations often used by statisticians to make data more normal (Gaussian). This can reduce bias caused by outliers, and can also transform data to satisfy normality assumptions that are assumed by many statistical techniques. Note that the transformation just puts the data on a different scale; it needn't change the information content. Note also that meteorologists (and even some statisticians) often confusingly say ``normalizing'' when what they really mean is ``standardizing''!

  A much used class of transformations is the Box-Cox power law transformation $y=(x^\lambda-1)/\lambda$, where $\lambda$ can be optimally tuned. In the limit as $\lambda\rightarrow 0$, one obtains the $y=\log x$ transformation much used to make postively skewed quantities such as stockmarket prices more normal. The $\lambda=0.5$ square root transformation is often a good compromise for postively skewed variables such as rainfall amounts (Stephenson et al. 1999).