Parametric and non-parametric regression

The response can also sometimes depend on a nonlinear function of the explanatory variables e.g. $y=f(x)+\epsilon$. For example, under realistic carbon emission scenarios, predicted future global warming is not expected to be a simple linear function of time and so a linear fit would be inappropriate.

In some cases, the expected form of the non-linear function is known and can be parameterised in terms of basis functions. For example, polynomial regression consists of performing multiple regression with variables $\{x,x^2,x^3,\ldots\}$ in order to find the polynomial coefficients (parameters). Note, however, that strong correlations between $\{x,x^2,x^3,\ldots\}$ can lead to collinearity and poor fits. A better approach is to use a basis set of orthogonal uncorrelated predictor functions such as Fourier modes. These types of regression are known as parametric regression since they are based on models that require the estimation of a finite number of parameters.

In other cases, the functional form is not known and so can not be parameterised in terms of any basis functions. The smooth function can be estimate in such cases using what is known as non-parametric regression. Two of the most commonly used approaches to non-parametric regression are smoothing splines ^8.1and kernel regression. Smoothing splines minimise the sum of squared residuals plus a term which penalizes the roughness of the fit, whereas kernel regression involves making smooth composites by applying a weighted filter to the data. Both methods are useful for estimating the slow trends in climatic time series and avoid spurious features often obtained in simpler smoothing approaches.