Traditional Problem Definition - Computational Intelligence in Time Series Forecasting - page 43

exponentially decaying and damped sinusoidal components. In addition, for higher
order autoregressive processes the sample partial autocorrelation function should
also be considered, which becomes zero when more model parameters are involved
than needed.
The partial autocorrelation function is not helpful for identifying the order of
the moving-average process because if the number of model parameters is higher
than required, then the autocorrelation process becomes zero. Nevertheless, the fact
that both the sample autocorrelation functions and the partial autocorrelation
functions are random variables makes the model identification generally difficult,
particularly the identification of a mixed ARMA model. Also, developing time
series models using sample plots of both autocorrelation functions involves
multiple trial-and-error iterations, which is time consuming. Akaike (1974)
proposed the information criterion, known as the Akaike information criterion
(AIC):

AIC n () v log( V ml )

2

2 n ,

e

with V ml = RSS/Q and RSS being the residual sum of squares. By minimizing the
criterion with respect to n , the model order Q can be determined, which helps
automate the model identification process. For instance, in the case of two
equivalent models being found, with both having acceptable residuals, the one
having a lower AIC( n ) value can be taken as the better one.
A similar criterion was proposed by Schwarz, known as the Bayesian
information criterion (BIC), defined as

BIC n () v log( V ml )

2

n log . v

e e

It delivers a lower order model than the AIC, which is an argument for its
preference. But also here, in the initial phase of the model identification process,
the stationarity, seasonality, etc . of the given time series have to be checked and
removed by de-trending and de-seasonalization of time series data.
A successful identification phase of model building is to a great extent a matter
of knowledge and practical experience, rather than the matter of some given rigid
instructions about how to do it. Yet, some recommendations related to the initial
parameter estimation of a pure AR process are still available, relying on the use of
the Yule-Walker approach. Much more difficult is to model the MA part of an
ARMA model, where a system of nonlinear equations has to be solved.

2.8.2 Model Estimation

Once the preliminary time series model has been identified, i.e . the number of
required model parameters has been determined, the actual model parameter values
have to be estimated using the observation data. This is a nonlinear estimation
problem that needs some special statistical procedures, like the maximum
likelihood method or nonlinear least-squares estimation . The parameter values
estimated at this stage of model building should minimize the sum of squared