In applied statistics, total least squares is a type of errors-in-variables regression, a least squares data modeling technique in which observational errors on both dependent and independent variables are taken into account. It is a generalization of Deming regression and also of orthogonal regression, and can be applied to both linear and non-linear models.

The total least squares approximation of the data is generically equivalent to the best, in the Frobenius norm, low-rank approximation of the data matrix.^[1]

Linear model

Background

In the least squares method of data modeling, the objective function, S,

${\displaystyle S=\mathbf {r^{T}Wr} ,}$

is minimized, where r is the vector of residuals and W is a weighting matrix. In linear least squares the model contains equations which are linear in the parameters appearing in the parameter vector ${\displaystyle {\boldsymbol {\beta }}}$, so the residuals are given by

${\displaystyle \mathbf {r=y-X{\boldsymbol {\beta }}} .}$

There are m observations in y and n parameters in β with m>n. X is a m×n matrix whose elements are either constants or functions of the independent variables, x. The weight matrix W is, ideally, the inverse of the variance-covariance matrix ${\displaystyle \mathbf {M} _{y}}$ of the observations y. The independent variables are assumed to be error-free. The parameter estimates are found by setting the gradient equations to zero, which results in the normal equations ^{[note 1]}

${\displaystyle \mathbf {X^{T}WX{\boldsymbol {\beta }}=X^{T}Wy} .}$

Allowing observation errors in all variables

Now, suppose that both x and y are observed subject to error, with variance-covariance matrices ${\displaystyle \mathbf {M} _{x}}$ and ${\displaystyle \mathbf {M} _{y}}$ respectively. In this case the objective function can be written as

${\displaystyle S=\mathbf {r_{x}^{T}M_{x}^{-1}r_{x}+r_{y}^{T}M_{y}^{-1}r_{y}} ,}$

where ${\displaystyle \mathbf {r} _{x}}$ and ${\displaystyle \mathbf {r} _{y}}$ are the residuals in x and y respectively. Clearly^{[further explanation needed]} these residuals cannot be independent of each other, but they must be constrained by some kind of relationship. Writing the model function as ${\displaystyle \mathbf {f(r_{x},r_{y},{\boldsymbol {\beta }})} }$, the constraints are expressed by m condition equations.^[2]

${\displaystyle \mathbf {F=\Delta y-{\frac {\partial f}{\partial r_{x}}}r_{x}-{\frac {\partial f}{\partial r_{y}}}r_{y}-X\Delta {\boldsymbol {\beta }}=0} .}$

Thus, the problem is to minimize the objective function subject to the m constraints. It is solved by the use of Lagrange multipliers. After some algebraic manipulations,^[3] the result is obtained.

${\displaystyle \mathbf {X^{T}M^{-1}X\Delta {\boldsymbol {\beta }}=X^{T}M^{-1}\Delta y} ,}$

or alternatively ${\displaystyle \mathbf {X^{T}M^{-1}X{\boldsymbol {\beta }}=X^{T}M^{-1}y} ,}$ where M is the variance-covariance matrix relative to both independent and dependent variables.

${\displaystyle \mathbf {M=K_{x}M_{x}K_{x}^{T}+K_{y}M_{y}K_{y}^{T};\ K_{x}=-{\frac {\partial f}{\partial r_{x}}},\ K_{y}=-{\frac {\partial f}{\partial r_{y}}}} .}$

Example

When the data errors are uncorrelated, all matrices M and W are diagonal. Then, take the example of straight line fitting.

${\displaystyle f(x_{i},\beta )=\alpha +\beta x_{i}}$

in this case

${\displaystyle M_{ii}=\sigma _{y,i}^{2}+\beta ^{2}\sigma _{x,i}^{2}}$

showing how the variance at the ith point is determined by the variances of both independent and dependent variables and by the model being used to fit the data. The expression may be generalized by noting that the parameter ${\displaystyle \beta }$ is the slope of the line.

${\displaystyle M_{ii}={}_{}^{}}$Zdroj:https://en.wikipedia.org?pojem=Total_least_squares
Text je dostupný za podmienok Creative Commons Attribution/Share-Alike License 3.0 Unported; prípadne za ďalších podmienok. Podrobnejšie informácie nájdete na stránke Podmienky použitia.

---