Mean squared prediction error

In statistics the mean squared prediction error of a smoothing or curve fitting procedure is the expected value of the squared difference between the fitted values implied by the predictive function $\widehat {g}$ and the values of the (unobservable) function g. It is an inverse measure of the explanatory power of ${\widehat {g}},$ and can be used in the process of cross-validation of an estimated model.

If the smoothing or fitting procedure has operator matrix (i.e., hat matrix) L, which maps the observed values vector $y$ to predicted values vector ${\hat {y}}$ via ${\hat {y}}=Ly,$ then

\operatorname {MSPE}(L)=\operatorname {E}\left[\left(g(x_{i})-\widehat {g}(x_{i})\right)^{2}\right].

The MSPE can be decomposed into two terms (just like mean squared error is decomposed into bias and variance); however for MSPE one term is the sum of squared biases of the fitted values and another the sum of variances of the fitted values:

\operatorname {MSPE}(L)=\sum _{{i=1}}^{n}\left(\operatorname {E}\left[\widehat {g}(x_{i})\right]-g(x_{i})\right)^{2}+\sum _{{i=1}}^{n}\operatorname {var}\left[\widehat {g}(x_{i})\right].

Note that knowledge of g is required in order to calculate MSPE exactly.

Estimation of MSPE

For the model $y_{i}=g(x_{i})+\sigma \varepsilon _{i}$ where $\varepsilon _{i}\sim {\mathcal {N}}(0,1)$ , one may write

\operatorname {MSPE}(L)=g'(I-L)'(I-L)g+\sigma ^{2}\operatorname {tr}\left[L'L\right].

The first term is equivalent to

\sum _{{i=1}}^{n}\left(\operatorname {E}\left[\widehat {g}(x_{i})\right]-g(x_{i})\right)^{2}=\operatorname {E}\left[\sum _{{i=1}}^{n}\left(y_{i}-\widehat {g}(x_{i})\right)^{2}\right]-\sigma ^{2}\operatorname {tr}\left[\left(I-L\right)'\left(I-L\right)\right].

Thus,

\operatorname {MSPE}(L)=\operatorname {E}\left[\sum _{{i=1}}^{n}\left(y_{i}-\widehat {g}(x_{i})\right)^{2}\right]-\sigma ^{2}\left(n-2\operatorname {tr}\left[L\right]\right).

If $\sigma ^{2}$ is known or well-estimated by $\widehat {\sigma }^{2}$ , it becomes possible to estimate MSPE by

\operatorname {\widehat {MSPE}}(L)=\sum _{{i=1}}^{n}\left(y_{i}-\widehat {g}(x_{i})\right)^{2}-\widehat {\sigma }^{2}\left(n-2\operatorname {tr}\left[L\right]\right).

Colin Mallows advocated this method in the construction of his model selection statistic C_p, which is a normalized version of the estimated MSPE:

C_{p}={\frac {\sum _{{i=1}}^{n}\left(y_{i}-\widehat {g}(x_{i})\right)^{2}}{\widehat {\sigma }^{2}}}-n+2\operatorname {tr}\left[L\right].

where p comes from the fact that the number of parameters p estimated for a parametric smoother is given by $p=\operatorname {tr}\left[L\right]$ , and C is in honor of Cuthbert Daniel.

Mean squared prediction error

Estimation of MSPE

See also