In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of an assumed probability distribution, given some observed data. This is achieved by maximizing a likelihood function so that, under the assumed statistical model, the observed data is most probable. The point in the parameter space that maximizes the likelihood function is called the maximum likelihood estimate.^[1] The logic of maximum likelihood is both intuitive and flexible, and as such the method has become a dominant means of statistical inference.^[2][3][4]

If the likelihood function is differentiable, the derivative test for finding maxima can be applied. In some cases, the first-order conditions of the likelihood function can be solved analytically; for instance, the ordinary least squares estimator for a linear regression model maximizes the likelihood when the random errors are assumed to have normal distributions with the same variance.^[5]

From the perspective of Bayesian inference, MLE is generally equivalent to maximum a posteriori (MAP) estimation with uniform prior distributions (or a normal prior distribution with a standard deviation of infinity). In frequentist inference, MLE is a special case of an extremum estimator, with the objective function being the likelihood.

Principles

We model a set of observations as a random sample from an unknown joint probability distribution which is expressed in terms of a set of parameters. The goal of maximum likelihood estimation is to determine the parameters for which the observed data have the highest joint probability. We write the parameters governing the joint distribution as a vector ${\displaystyle \;\theta =\left^{\mathsf {T}}\;}$ so that this distribution falls within a parametric family ${\displaystyle \;\{f(\cdot \,;\theta )\mid \theta \in \Theta \}\;,}$ where ${\displaystyle \,\Theta \,}$ is called the parameter space, a finite-dimensional subset of Euclidean space. Evaluating the joint density at the observed data sample ${\displaystyle \;\mathbf {y} =(y_{1},y_{2},\ldots ,y_{n})\;}$ gives a real-valued function,

${\displaystyle {\mathcal {L}}_{n}(\theta )={\mathcal {L}}_{n}(\theta ;\mathbf {y} )=f_{n}(\mathbf {y} ;\theta )\;,}$

which is called the likelihood function. For independent and identically distributed random variables, ${\displaystyle f_{n}(\mathbf {y} ;\theta )}$ will be the product of univariate density functions:

${\displaystyle f_{n}(\mathbf {y} ;\theta )=\prod _{k=1}^{n}\,f_{k}^{\mathsf {univar}}(y_{k};\theta )~.}$

The goal of maximum likelihood estimation is to find the values of the model parameters that maximize the likelihood function over the parameter space,^[6] that is

${\displaystyle {\hat {\theta }}={\underset {\theta \in \Theta }{\operatorname {arg\;max} }}\,{\mathcal {L}}_{n}(\theta \,;\mathbf {y} )~.}$

Intuitively, this selects the parameter values that make the observed data most probable. The specific value ${\displaystyle ~{\hat {\theta }}={\hat {\theta }}_{n}(\mathbf {y} )\in \Theta ~}$ that maximizes the likelihood function ${\displaystyle \,{\mathcal {L}}_{n}\,}$ is called the maximum likelihood estimate. Further, if the function ${\displaystyle \;{\hat {\theta }}_{n}:\mathbb {R} ^{n}\to \Theta \;}$ so defined is measurable, then it is called the maximum likelihood estimator. It is generally a function defined over the sample space, i.e. taking a given sample as its argument. A sufficient but not necessary condition for its existence is for the likelihood function to be continuous over a parameter space ${\displaystyle \,\Theta \,}$ that is compact.^[7] For an open ${\displaystyle \,\Theta \,}$ the likelihood function may increase without ever reaching a supremum value.

In practice, it is often convenient to work with the natural logarithm of the likelihood function, called the log-likelihood:

${\displaystyle \ell (\theta \,;\mathbf {y} )=\ln {\mathcal {L}}_{n}(\theta \,;\mathbf {y} )~.}$

Since the logarithm is a monotonic function, the maximum of ${\displaystyle \;\ell (\theta \,;\mathbf {y} )\;}$ occurs at the same value of ${\displaystyle \theta }$ as does the maximum of ${\displaystyle \,{\mathcal {L}}_{n}~.}$^[8] If $$Zdroj:https://en.wikipedia.org?pojem=Maximum_likelihood_estimation
Text je dostupný za podmienok Creative Commons Attribution/Share-Alike License 3.0 Unported; prípadne za ďalších podmienok. Podrobnejšie informácie nájdete na stránke Podmienky použitia.