Autoregressive Moving Average (ARMA (p, q)) Model

In one sentence

An ARMA(p, q) model represents a stationary time series using p autoregressive lags and q moving-average shock terms, and it is commonly used for modeling and forecasting.

Stationarity / invertibility (rule of thumb)

For an ARMA model, a common requirement is that the AR polynomial has roots outside the unit circle (stationarity). For AR(1):

[ |\phi| < 1 ]

ARMA components

    flowchart TD
	  X["Series X_t"] --> AR["AR part\n(lags of X)"]
	  X --> MA["MA part\n(lags of shocks)"]
	  AR --> Fit["Fit/forecast"]
	  MA --> Fit

Background

The Autoregressive Moving Average (ARMA (p, q)) model is a fundamental tool in statistical analysis and econometrics, widely used for understanding and predicting future points in univariate time series data. Combining the autoregressive (AR) and moving average (MA) components, the ARMA model efficiently captures and encapsulates the linear dependencies in stochastic processes.

Historical Context

The ARMA model’s theoretical roots can be traced back to the work of U. Yule in the 1920s who introduced autoregressive formulations, extended later by G.E.P. Box and G.M. Jenkins in the early 1970s through their pioneering development of Box-Jenkins methodology. This methodology established a systematic way to identify, estimate, and check models for time series data, leading to the widespread adoption of ARMA models in econometric analysis.

Definitions and Concepts

The ARMA model is represented as ARMA(p, q), where:

p denotes the number of lagged terms (autoregressive order)
q represents the number of lagged forecast errors or residuals (moving average order)

In the generalized mathematical form, the ARMA model is defined as: \[ X_t = c + \epsilon_t + \sum_{i=1}^{p} \phi_i X_{t-i} + \sum_{j=1}^{q} \theta_j \epsilon_{t-j} \] where:

\(X_t\) is the value at time \(t\)
\(c\) is a constant
\(\phi_i\) are the parameters of the autoregressive terms
\(\theta_j\) are the parameters of the moving averages
\(\epsilon_t\) is white noise (a random error term at time \(t\))

AR (Autoregressive) Model: A model where current values are based linearly on prior values in time series.
MA (Moving Average) Model: A model based on linear regressions of current values with past forecast errors.
ARIMA Model: Differentiation-included ARMA modeling for non-stationary data.
Box-Jenkins Methodology: Structured methodology for identifying, estimating, and checking time series models.
Stationarity: A property where the statistical properties of a time series are constant over time.

Quiz

### Which components are combined in an ARMA(p,q) model? - [x] Autoregressive (AR) and Moving Average (MA) - [ ] Exponential Smoothing and Linear Regression - [ ] Seasonal and Trend - [ ] Exponential Smoothing and Random Walk > **Explanation:** ARMA models combine the Autoregressive (AR) and Moving Average (MA) components to analyze time series data. ### An AR(p) model considers which of the following? - [ ] Lagged error terms - [x] Past values of the series - [ ] Future values of the series - [ ] Constant growth rates > **Explanation:** The AR(p) component accounts for the current value of a time series based on its previous values or lags. ### True or False: MA(q) models consider past values of the series. - [ ] True - [x] False > **Explanation:** MA(q) models consider past error terms or shocks, not the past values of the series themselves. ### In an ARMA(p, q) model, what does 'p' denote? - [x] The number of lagged observations in the AR part - [ ] The number of lagged observations in the MA part - [ ] The forecast error at time t - [ ] The order of integration > **Explanation:** 'p' represents the number of lagged observations used in the AR part of the model. ### What does 'q' in the ARMA(p, q) model signify? - [ ] The number of lagged observations in the AR part - [x] The number of lagged error terms in the MA part - [ ] Seasonal components - [ ] The trend cycle component > **Explanation:** 'q' denotes the number of lagged error terms considered in the MA component of the model. ### Who popularized ARMA models through their influential methodology in the 1970s? - [x] George E. P. Box and Gwilym Jenkins - [ ] Charles Darwin - [ ] Albert Einstein - [ ] John Keynes > **Explanation:** George E. P. Box and Gwilym Jenkins popularized the ARMA model in their groundbreaking methodology for time series forecasting. ### The ARIMA model extends which model? - [ ] Exponential Smoothing - [ ] Linear Regression - [ ] Random Walk - [x] ARMA > **Explanation:** ARIMA extends the ARMA model by incorporating an integration component to handle non-stationary data. ### An MA(q) model considers: - [x] Past error terms - [ ] Past values of the series - [ ] Seasonal and trend components - [ ] Exponential smoothing parameters > **Explanation:** The MA(q) component considers past error terms or shocks. ### ARMA models are particularly useful for: - [ ] Non-time series data - [ ] Univariate time series data - [x] Univariate time series data - [ ] Multivariate time series data > **Explanation:** ARMA models are most effective for analyzing univariate time series data. ### Which book is co-authored by George E. P. Box? - [x] *Time Series Analysis: Forecasting and Control* - [ ] *Introduction to Econometrics* - [ ] *Principles of Economics* - [ ] *Economic Growth* > **Explanation:** The book *Time Series Analysis: Forecasting and Control* is co-authored by George E. P. Box and is a cornerstone in the field of time series analysis.