Multivariate regression

Multiple regression can easily be extended to deal with situations where the response consists of $p>1$ different variables. Multivariate regression is defined by the General Linear Model

$${\bf Y} = {\bf X}{\boldmath\beta}+{\bf E}$$ (8.8)

where ${\bf Y}$ is a $(n\times p)$ data matrix containing the response variables, ${\bf X}$ is a $(n\times q)$ data matrix containing the factors, ${\boldmath\beta}$ is a $(q\times p)$ data matrix containing the factor coefficients (model parameters), and ${\bf E}$ is a $(n\times p)$ data matrix containing the noise terms.

The least squares estimates for the beta parameters are obtained by solving the normal equations as in multiple regression. To avoid having large uncertainities in the estimates of the beta parameters, it is important to ensure that the matrix ${\bf X}'{\bf X}$ is well-conditioned. Poor conditioning (determinant of ${\bf X}'{\bf X}$ is small) can occur due to collinearity in explanatory variables, and so it is important to select only response variables that are not strongly correlated with one another. To choose the best model, it is vitally important to make a careful selection of variables when choosing the explanatory variables. Semi-automatic methods such as forward, backward, and stepwise selection have been developed to help in this complex process of model identification.