In statistics, the bias of an estimator (or bias function) is the difference between this estimator's expected value and the true value of the parameter being estimated. An estimator or decision rule with zero bias is called unbiased. In statistics, "bias" is an objective property of an estimator. Bias is a distinct concept from consistency: consistent estimators converge in probability to the true value of the parameter, but may be biased or unbiased; see bias versus consistency for more.

All else being equal, an unbiased estimator is preferable to a biased estimator, although in practice, biased estimators (with generally small bias) are frequently used. When a biased estimator is used, bounds of the bias are calculated. A biased estimator may be used for various reasons: because an unbiased estimator does not exist without further assumptions about a population; because an estimator is difficult to compute (as in unbiased estimation of standard deviation); because a biased estimator may be unbiased with respect to different measures of central tendency; because a biased estimator gives a lower value of some loss function (particularly mean squared error) compared with unbiased estimators (notably in shrinkage estimators); or because in some cases being unbiased is too strong a condition, and the only unbiased estimators are not useful.

Bias can also be measured with respect to the median, rather than the mean (expected value), in which case one distinguishes median-unbiased from the usual mean-unbiasedness property. Mean-unbiasedness is not preserved under non-linear transformations, though median-unbiasedness is (see § Effect of transformations); for example, the sample variance is a biased estimator for the population variance. These are all illustrated below.

An unbiased estimator for a parameter need not always exist. For example, there is no unbiased estimator for the reciprocal of the parameter of a binomial random variable.^[1]

Definition

Suppose we have a statistical model, parameterized by a real number θ, giving rise to a probability distribution for observed data, ${\displaystyle P_{\theta }(x)=P(x\mid \theta )}$, and a statistic ${\displaystyle {\hat {\theta }}}$ which serves as an estimator of θ based on any observed data ${\displaystyle x}$. That is, we assume that our data follows some unknown distribution ${\displaystyle P(x\mid \theta )}$ (where θ is a fixed, unknown constant that is part of this distribution), and then we construct some estimator ${\displaystyle {\hat {\theta }}}$ that maps observed data to values that we hope are close to θ. The bias of ${\displaystyle {\hat {\theta }}}$ relative to ${\displaystyle \theta }$ is defined as^[2]

${\displaystyle \operatorname {Bias} ({\hat {\theta }},\theta )=\operatorname {Bias} _{\theta }=\operatorname {E} _{x\mid \theta }-\theta =\operatorname {E} _{x\mid \theta },}$

where ${\displaystyle \operatorname {E} _{x\mid \theta }}$ denotes expected value over the distribution ${\displaystyle P(x\mid \theta )}$ (i.e., averaging over all possible observations ${\displaystyle x}$). The second equation follows since θ is measurable with respect to the conditional distribution ${\displaystyle P(x\mid \theta )}$.

An estimator is said to be unbiased if its bias is equal to zero for all values of parameter θ, or equivalently, if the expected value of the estimator matches that of the parameter.^[3] Unbiasedness is not guaranteed to carry over. For example, if ${\displaystyle {\hat {\theta }}}$ is an unbiased estimator for parameter θ, it is not guaranteed that g(${\displaystyle {\hat {\theta }}}$) is an unbiased estimator for g(θ).^[4]

In a simulation experiment concerning the properties of an estimator, the bias of the estimator may be assessed using the mean signed difference.

Examples

Sample variance

The sample variance of a random variable demonstrates two aspects of estimator bias: firstly, the naive estimator is biased, which can be corrected by a scale factor; second, the unbiased estimator is not optimal in terms of mean squared error (MSE), which can be minimized by using a different scale factor, resulting in a biased estimator with lower MSE than the unbiased estimator. Concretely, the naive estimator sums the squared deviations and divides by n, which is biased. Dividing instead by n − 1 yields an unbiased estimator. Conversely, MSE can be minimized by dividing by a different number (depending on distribution), but this results in a biased estimator. This number is always larger than n − 1, so this is known as a shrinkage estimator, as it "shrinks" the unbiased estimator towards zero; for the normal distribution the optimal value is n + 1.

Suppose X₁, ..., X_n are independent and identically distributed (i.i.d.) random variables with expectation μ and variance σ². If the sample mean and uncorrected sample variance are defined as

${\displaystyle {\overline {X}}\,={\frac {1}{n}}\sum _{i=1}^{n}X_{i}\qquad S^{2}={\frac {1}{n}}\sum _{i=1}^{n}{\big (}X_{i}-{\overline {X}}\,{\big )}^{2}\qquad }$

then S² is a biased estimator of σ², because

Zdroj:https://en.wikipedia.org?pojem=Biased_estimator
Text je dostupný za podmienok Creative Commons Attribution/Share-Alike License 3.0 Unported; prípadne za ďalších podmienok. Podrobnejšie informácie nájdete na stránke Podmienky použitia.

---