Appendix A: Likelihood function (LLF)

The likelihood function (aka likelihood/LLF) is a function of the parameters of a statistical model. In other words, the likelihood of model parameters given some observed outcome (i.e., sample) is equal to the probability of those outcomes given the model and its parameter values.

In non-technical parlance, "likelihood" is usually a synonym for "probability," but a clear technical distinction is made in statistical usage.

For many applications involving likelihood functions, it is more convenient to work in terms of the natural logarithm of the likelihood function, called the log-likelihood, than the likelihood function itself. Because the logarithm is a monotonically increasing function, the logarithm of a function achieves its maximum value at the same points as the function itself, and hence, the log-likelihood can be used in place of the likelihood in maximum likelihood estimation and related techniques.

By definition, the likelihood function for a statistical model is described as:

$$ L^* = \prod_{n = 1}^N {f\left( {y_n \left| {y_{n-1},y_{n-2},...,y_1 ,\theta_1, \theta_2,...,\theta_k } \right.} \right)} $$ - OR - $$ \ln L^* = \sum_{n = 1}^N {\ln f\left( {y_n \left| {y_{n-1},y_{n-2},...,y_1 ,\theta_1, \theta_2,...,\theta_k } \right.} \right)} $$ Where:

• $L^*$ is the likelihood function.
• $\ln L^*$ is the log-likelihood function.
• $f\left( \right)$ is the conditional probability density function.
• $\ln f\left( \right)$ is the natural log of the conditional probability density function.
• $y_n$ is the value of the time series at time n.
• $y_{n-1},y_{n-2},...,y_1$ are the past values of the time series at time n.
• $\theta_1, \theta_2,...,\theta_k$ are the parameters of the statistical model.

Remarks

1. In the case of normal or Gaussian probability density function, the log-likelihood function can be simplified as follows:
   $$\ln L^* = \ln L(\mu \,, \sigma \mid Y_1\,,Y_2...Y_N) = -\frac{N}{2}\ln(2\times \pi)- N\times \ln\sigma - \sum_{i=1}^{N}\frac{(Y_i-\mu)^2}{2\times\sigma^2} $$
   
   • $\mu$ is the normal distribution mean
   • $\sigma$ is the standard deviation of the underlying distribution
   • $N$ is the number of observed values in the sample