Biomedical Engineering Reference
In-Depth Information
Parameter Estimation . Presuming that something is a power law, a natural
way of estimating its exponent is to use linear regression to find the line of best
fit to the points on the log-log plot. This is actually a consistent estimator, if the
data really do come from a power law. However, the loss function used in linear
regression is the sum of the squared distances between the line and the points
("least squares"). In general, the line minimizing the sum of squared errors is not
a valid probability distribution, and so this is simply not a reliable way to esti-
mate the distribution .
One is much better off using maximum likelihood to estimate the
parameter. With a discrete variable, the probability function is expressed as
follows: Pr( X = x ) = x - B /[(B), where [(B) =
is the Riemann zeta
function, which ensures that the probability is normalized. Thus the maximum
likelihood estimate of the exponent is obtained by minimizing the negative log-
likelihood, L (B) = ( i B log x i + log [(B), i.e., L (B) is our loss function. In the
continuous case, the probability density is (B -1 ) c B -1 / x B , with x c > 0.
k B
d
=
k
1
Error Estimation . Most programs used to perform linear regression also
provide an estimate of the standard error in the estimated slope, and one some-
times sees this reported as the uncertainty in the power law. This is an entirely
unacceptable procedure. Those calculations of the standard error assume that
measured values having Gaussian fluctuations around their true means. Here
that would mean that the log of the empirical relative frequency is equal to the
log of the probability plus Gaussian noise. However, by the central limit theo-
rem, one knows that the relative frequency is equal to the probability plus Gaus-
sian noise, so the former condition does not hold. Notice that one can obtain
asymptotically reliable standard errors from maximum likelihood estimation.
Validation , R 2 . Perhaps the most pernicious error is that of trying to vali-
date the assumption of a power law distribution by either eye-balling the fit to a
straight line, or evaluating it using the R 2 statistic, i.e., the fraction of the vari-
ance accounted for by the least-squares regression line. Unfortunately, while
these procedures are good at confirming that something is a power law, if it
really is (low Type I error, or high statistical significance), they are very bad at
alerting you to things that are not power laws (they have a very high rate of
Type II error, or low statistical power). The basic problem here is that any
smooth curve looks like a straight line, if you confine your attention to a suffi-
ciently small region—and for some non-power-law distributions, such "suffi-
ciently small" regions can extend over multiple orders of magnitude.
To illustrate this last point, consider Figure 5, made by generating 10,000
random numbers according to a log-normal distribution, with a mean log of 0
and a standard deviation in the log of 3. Restricting attention to the "tail" of ran-
dom numbers 1, and doing a usual least-squares fit, gives the line shown in
Search WWH ::




Custom Search