Autoregressive Integrated Moving Average (ARIMA) Model

The ARIMA model is an extension of the ARMA model that applies to non-stationary time series (the kind of time series with one or more integrated unit roots). By definition, the auto-regressive integrated moving average (ARIMA) process is an ARMA process for the differenced time series: $$(1-\phi_1 L - \phi_2 L^2 -\cdots - \phi_p L^p)(1-L)^d x_t - \phi_o= (1+\theta_1 L+\theta_2 L^2 + \cdots + \theta_q L^q)a_t$$ $$y_t = (1-L)^d x_t$$ Where:

• $x_t$ is the original non-stationary output at time $t$.
• $y_t$ is the observed differenced (stationary) output at time $t$.
• $d$ is the integration order of the time series.
• $a_t$ is the innovation, shock, or error term at time $t$.
• $p$ is the order of the last lagged variables.
• $q$ is the order of the last lagged innovation or shock.
• $a_t$ time series observations are independent and identically distributed (i.e., $i.i.d$) and follow a Gaussian distribution (i.e., $\Phi(0,\sigma^2)$).

Remarks

1. The variance of the shocks is constant or time-invariant.
2. Assuming (i.e., $(1-L)^d x_t$) is a stationary process with a long-run mean of $\mu$, then taking the expectation from both sides, we can express $\phi_o$ as follows:$$\phi_o = (1-\phi_1-\phi_2-\cdots -\phi_p)\mu$$
3. Thus, the ARIMA (p, d, q) process can now be expressed as:$$\eqalign{ & (1 - {\phi _1}L - {\phi _2}{L^2} - \cdots - {\phi _p}{L^p})({y_t} - \mu ) \cr & = (1 + {\theta _1}L + {\theta _2}{L^2} + \cdots + {\theta _q}{L^q}){a_t}{z_t} \cr & = {y_t} - \mu (1 - {\phi _1}L - {\phi _2}{L^2} - \cdots - {\phi _p}{L^p}){z_t} \cr & = (1 + {\theta _1}L + {\theta _2}{L^2} + \cdots + {\theta _q}{L^q}){a_t} \cr}$$
4. In sum, $z_t$ is the differenced signal after we subtract its long-run average.
5. The order of an ARIMA process is solely determined by the order of the last lagged variable with a non-zero coefficient. In principle, you can have a fewer number of parameters than the order of the model. Consider the following ARIMA (12, 2) process: $$(1-\phi_1 L -\phi_{12} L^{12} ) (y_t-\mu) = (1+\theta_2 L^2 ) a_t$$

Related Links

• Wikipedia - Autoregressive moving average model.

References

• James Douglas Hamilton; Time Series Analysis, Princeton University Press; 1st edition(Jan 11, 1994), ISBN: 691042896.
• Tsay, Ruey S.; Analysis of Financial Time Series, John Wiley & SONS; 2nd edition(Aug 30, 2005), ISBN: 0-471-690740.