Quadratic form (statistics)
From Wikipedia, the free encyclopedia
If ε is a vector of n random variables, and Λ is an n-dimensional symmetric square matrix, then the scalar quantity ε'Λε is known as a quadratic form in ε.
Contents |
[edit] Expectation
It can be shown that
![\operatorname{E}\left[\epsilon'\Lambda\epsilon\right]=\operatorname{tr}\left[\Lambda \Sigma\right] + \mu'\Lambda\mu](http://upload.wikimedia.org/math/3/4/e/34eef77b1c8a11ed1953c0997bdf4ebc.png)
where μ and Σ are the expected value and variance-covariance matrix of ε, respectively, and tr denotes the trace of a matrix. This result only depends on the existence of μ and Σ; in particular, normality of ε is not required.
[edit] Variance
In general, the variance of a quadratic form depends greatly on the distribution of ε. However, if ε does follow a multivariate normal distribution, the variance of the quadratic form becomes particularly tractable. Assume for the moment that Λ is a symmetric matrix. Then,
![\operatorname{var}\left[\epsilon'\Lambda\epsilon\right]=2\operatorname{tr}\left[\Lambda \Sigma\Lambda \Sigma\right] + 4\mu'\Lambda\Sigma\Lambda\mu](http://upload.wikimedia.org/math/5/8/3/5830ca4558866d7dd70d73009d9f1424.png)
In fact, this can be generalized to find the covariance between two quadratic forms on the same ε (once again, Λ1 and Λ2 must both be symmetric):
![\operatorname{cov}\left[\epsilon'\Lambda_1\epsilon,\epsilon'\Lambda_2\epsilon\right]=2\operatorname{tr}\left[\Lambda _1\Sigma\Lambda_2 \Sigma\right] + 4\mu'\Lambda_1\Sigma\Lambda_2\mu](http://upload.wikimedia.org/math/3/d/a/3da5d699bd6c6dc5703454971e2143d0.png)
[edit] Computing the variance in the non-symmetric case
Some texts incorrectly state the above variance or covariance results without enforcing Λ to be symmetric. The case for general Λ can be derived by noting that
- ε'Λ'ε = ε'Λε
so
But this is a quadratic form in the symmetric matrix 
, so the mean and variance expressions are the same, provided Λ is replaced by 
therein.
[edit] Examples of quadratic forms
In the setting where one has a set of observations y and an operator matrix H, then the residual sum of squares can be written as a quadratic form in y:
For procedures where the matrix H is symmetric and idempotent, and the errors are Gaussian with covariance matrix σ2I, RSS / σ2 has a chi-square distribution with k degrees of freedom and noncentrality parameter λ, where
![k=\operatorname{tr}\left[\left(I-H\right)'\left(I-H\right)\right]](http://upload.wikimedia.org/math/8/8/b/88b39e8a056cfce81cd5e8718de20af2.png)
may be found by matching the first two central moments of a noncentral chi-square random variable to the expressions given in the first two sections. If Hy estimates μ with no bias, then the noncentrality λ is zero and RSS / σ2 follows a central chi-square distribution.
