Generalized Linear Model

The generalized linear model (GLM) is a flexible generalization of ordinary least squares regression. The GLM generalizes linear regression by allowing the linear model to be related to the response variable (i.e. $Y$) via a link function (i.e. $g(.)$)and by allowing the magnitude of the variance of each measurement to be a function of its predicted value.

The GLM is described as follow:
$$Y = \mu + \epsilon $$
And
$$ E\left[Y\right]=\mu=g^{-1}(X\beta) = g^{-1}(\eta)$$
Where:

• $\epsilon$ is the residuals or deviation from the mean
• $g(.)$ is the link function
• $g^{-1}(.)$ is the inverse-link function
• $X$ is the independent variables or the exogenous factors
• $\beta$ is a parameter vector
• $\eta$ is the linear predictor: the quantity which incorporates the information about the independent variables into the model.
  $$\eta=X\beta$$

Remarks

1. Each outcome of the dependent variables,$Y$, is assumed to be generated from a particular distribution in the exponential family, a large range of probability distributions that includes the normal, binomial and Poisson distributions, among others.
2. The distribution mean of the $Y$ variable (i.e. $\mu$) depends solely on the independent variables, $X$.
   
   $$E\left[Y\right]=\mu=g^{-1}(X\beta)$$
3. The conditional variance of the dependent variable, $Y$, is constant:
   $$V(Y\|{X\beta})=c $$
   Where:
   
   • $V(.)$ is the variance function.
   • $c$ is a constant value.
4. The marginal variance of the dependent variable, $Y$, is a function of the mean:
   $$V(Y) = V(\mu) = V(X\beta) $$

Link Function

1. The Link Function provides the relationship between the linear predictor and the mean of the distribution function. Here are many commonly used link functions, and their choice can be somewhat arbitrary. It can be convenient to match the domain of the link function to the range of the distribution function's mean.
2. NumXL supports three canonical link functions: Identity, Logit and Log. In the table below, we define the link function and outline the residuals distribution assumption.
   

   Distribution	Name	Link Function
   Normal	Identity	$X\beta=\mu$
   Binomial	Logit	$X\beta = \ln\frac{\mu}{1-\mu}$
   Binomial	Probit	$X\beta = \Phi^{-1}(\mu)$
   Binomial	Log-Log	$X\beta = \ln{\left(-\ln{\left(1-\mu\right)}\right)}$
   Poisson	Log	$X\beta = \ln\mu$