Part of a series on
Regression analysis
Models
Linear regression Simple regression Polynomial regression General linear model
Generalized linear model Vector generalized linear model Discrete choice Binomial regression Binary regression Logistic regression Multinomial logistic regression Mixed logit Probit Multinomial probit Ordered logit Ordered probit Poisson
Multilevel model Fixed effects Random effects Linear mixed-effects model Nonlinear mixed-effects model
Nonlinear regression Nonparametric Semiparametric Robust Quantile Isotonic Principal components Least angle Local Segmented
Errors-in-variables
Estimation
Least squares Linear Non-linear
Ordinary Weighted Generalized Generalized estimating equation
Partial Total Non-negative Ridge regression Regularized
Least absolute deviations Iteratively reweighted Bayesian Bayesian multivariate Least-squares spectral analysis
Background
Regression validation Mean and predicted response Errors and residuals Goodness of fit Studentized residual Gauss–Markov theorem
Mathematics portal
v t e

Linear least squares (LLS) is the least squares approximation of linear functions to data. It is a set of formulations for solving statistical problems involved in linear regression, including variants for ordinary (unweighted), weighted, and generalized (correlated) residuals. Numerical methods for linear least squares include inverting the matrix of the normal equations and orthogonal decomposition methods.

Basic formulation

Consider the linear equation

$${\displaystyle Ax=b,}$$

(1)

where ${\displaystyle A\in \mathbb {R} ^{m\times n}}$ and ${\displaystyle b\in \mathbb {R} ^{m}}$ are given and ${\displaystyle x\in \mathbb {R} ^{n}}$ is variable to be computed. When ${\displaystyle m>n,}$ it is generally the case that (1) has no solution. For example, there is no value of ${\displaystyle x}$ that satisfies

because the first two rows require that ${\displaystyle x=(1,1),}$ but then the third row is not satisfied. Thus, for ${\displaystyle m>n,}$ the goal of solving (1) exactly is typically replaced by finding the value of ${\displaystyle x}$ that minimizes some error. There are many ways that the error can be defined, but one of the most common is to define it as ${\displaystyle \|Ax-b\|^{2}.}$ This produces a minimization problem, called a least squares problem

$${\displaystyle {\begin{aligned}{\underset {x\in \mathbb {R} ^{n}}{\operatorname {minimize} }}\quad &\|Ax-b\|^{2}.\end{aligned}}}$$

(2)

The solution to the least squares problem (1) is computed by solving the normal equation^[1]

$${\displaystyle A^{\top }Ax=A^{\top }b,}$$

(3)

where ${\displaystyle A^{\top }}$ denotes the transpose of ${\displaystyle A}$.

Continuing the example, above, with

we find and Solving the normal equation gives ${\displaystyle x=(1}$Zdroj:https://en.wikipedia.org?pojem=Linear_least_squares
Text je dostupný za podmienok Creative Commons Attribution/Share-Alike License 3.0 Unported; prípadne za ďalších podmienok. Podrobnejšie informácie nájdete na stránke Podmienky použitia.

---