Generalized Linear Model

The generalized linear model (GLM) is a flexible generalization of ordinary least squares regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable (i.e., $Y$) via a link function (i.e., $g(.)$)and by allowing the magnitude of the variance of each measurement to be a function of its predicted value.

The GLM is described as follows:
$$Y = \mu + \epsilon $$
And
$$ E\left[Y\right]=\mu=g^{-1}(X\beta) = g^{-1}(\eta)$$
Where:

• $\epsilon$ is the residual or deviation from the mean
• $g(.)$ is the link function
• $g^{-1}(.)$ is the inverse-link function
• $X$ is the independent variables or the exogenous factors
• $\beta$ is a parameter vector
• $\eta$ is the linear predictor: the quantity that incorporates the information about the independent variables into the model.
  $$\eta=X\beta$$

Remarks

1. Each outcome of the dependent variables,$Y$, is assumed to be generated from a particular distribution in the exponential family, an extensive range of probability distributions that include the normal, binomial, and Poisson distributions, among others.
2. The distribution mean of the $Y$ variable (i.e., $\mu$) depends solely on the independent variables, $X$.
   
   $$E\left[Y\right]=\mu=g^{-1}(X\beta)$$
3. The conditional variance of the dependent variable, $Y$, is constant:
   $$V(Y\|{X\beta})=c $$
   Where:
   
   • $V(.)$ is the variance function.
   • $c$ is a constant value.
4. The marginal variance of the dependent variable, $Y$, is a function of the mean:
   $$V(Y) = V(\mu) = V(X\beta) $$

Link Function

1. The Link Function provides the relationship between the linear predictor and the mean of the distribution function. Many commonly used link functions are here, and their choice can be somewhat arbitrary. It can be convenient to match the domain of the link function to the range of the distribution function's mean.
2. NumXL supports three canonical link functions: Identity, Logit, and Log. The table below defines the link function and outlines the residual distribution assumption.
   

   Distribution	Name	Link Function
   Normal	Identity	$X\beta=\mu$
   Binomial	Logit	$X\beta = \ln\frac{\mu}{1-\mu}$
   Binomial	Probit	$X\beta = \Phi^{-1}(\mu)$
   Binomial	Log-Log	$X\beta = \ln{\left(-\ln{\left(1-\mu\right)}\right)}$
   Poisson	Log	$X\beta = \ln\mu$

Files Examples

Related Links

• wikipedia: Generalized linear model
• wikipedia: Linear regression

References

• Tim F. (Futing) Liao;Interpreting Probability Models: Logit, Probit, and Other Generalized Linear Models; SAGE Publications, Inc; 1st Edition(Jun 30, 1994), ISBN: 0803949995
• John H. Aldrich, Forrest D. Nelson;Linear Probability, Logit, and Probit Models; SAGE Publications, Inc; 1st Edition(Nov 01, 1984), ISBN: 0803921330
• Hamilton, J .D.; Time Series Analysis , Princeton University Press (1994), ISBN 0-691-04289-6
• Tsay, Ruey S.; Analysis of Financial Time Series John Wiley & SONS. (2005), ISBN 0-471-690740