19.3 Basic Theoretical Aspects of Time Series

A collection of random variables ordered in time is called a stochastic process. The observed value in a given time period is usually a particular realization of the stochastic process. An important concept is so-called stationary of a time series or stochastic process. A time series is stationary if mean, variance, and covariance between any lagged observation are constant:

  • Constant mean: \(E(y_t) = \mu\)
  • Constant variance: \(Var(y_t) = \sigma^2\)
  • Constant covariance: \(Cov(y_t,y_{t-h})\) depends on \(h\) but not on \(t\).

A time series with a trend is usually not stationary (or nonstationary). The concepts behind are explained latter but the issue can be illustrated with a simple simulation.

spurious            = data.frame(x=matrix(0,500,1),y=matrix(0,500,1))
spurious[1,1]       = 1
spurious[1,2]       = 2
for(i in 2:500){
     spurious[i,1] = spurious[i-1,1]+rnorm(1)
     spurious[i,2] = spurious[i-1,2]+rnorm(1)}
bhat = lm(y~x,data=spurious)
summary(bhat)
## 
## Call:
## lm(formula = y ~ x, data = spurious)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -20.983  -4.103   1.663   5.609  15.353 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 19.54973    0.36613   53.40   <2e-16 ***
## x            1.38393    0.07835   17.66   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 8.08 on 498 degrees of freedom
## Multiple R-squared:  0.3852, Adjusted R-squared:  0.384 
## F-statistic:   312 on 1 and 498 DF,  p-value: < 2.2e-16
rm(spurious,bhat)

There is no relationship between \(y\) and \(x\) yet the regression results indicate statistical significance.

Consider the following so-called autoregressive model. The model depends on lagged terms of the dependent variable: \[y_t=\alpha+\phi_1 \cdot y_{t-1}+\epsilon_t\] This is called an AR(1) because the \(y\) is lagged by one period. The requirement for a stationary AR(1) is that \(|\phi_1|<1\). The properties of the AR(1) process are:

  • Mean of \(y_t\): \(\mu = \frac{\alpha}{1-\phi_1}\)
  • Variance: \(Var(x_t) = \frac{\sigma^2_w}{1-\phi_1^2}\)
  • Correlation: \(\rho_h = \phi^h_1\)