21 - Concentration and covariance matrix in an autoregressive model and in a dynamic linear model

Autoregressive model (\(AR(1)\))

Consider this model: \[ x_i = a x_{i-1} + e_i, \quad i=1, \dots, 3 \] and \(x_0=e_0\). All terms \(e_0, \dots, e_3\) are independent and \(N(0,v^2)\) distributed. Let \(e=(e_0, \dots, e_3)\) and \(x=(x_0, \dots x_3)\). Hence \(e \sim N(0, v^2 I)\). Isolating error terms gives \[ e= tex(e) = tex(L1) tex(x) = L_1 x \]

N <- 3
L1 <- diag(4)
L1[cbind(1 + (1:N), 1:N)] <- "-a"
L1 <- as_sym(L1)

Since \(\mathbf{Var}(e)=v^2 I\) we have \(\mathbf{Var}(e)=v^2 I=L \mathbf{Var}(x) L'\) so the covariance matrix of \(x\) is \(V_1=\mathbf{Var}(x) = v^2 L^- (L^-)'\) while the concentration matrix (the inverse covariances matrix) is \(K=v^{-2}L' L\).

def_sym(v2)
L1inv <- inv(L1)
V1 <- v2 * L1inv %*% t(L1inv)
K1 <- (t(L1) %*% L1) / v2

Dynamic linear model

Augment the \(AR(1)\) process above with \(y_i=b x_i + u_i\) for \(i=1,2,3\). Suppose \(u_i\sim N(0, w^2)\) and all \(u_i\) are independent and inpendent of \(e\). Then \((e,u)\) can be expressed in terms of \((x,y)\) as \[ (e,u) = tex(eu) = tex(L2) tex(xy) = L_2 (x,y) \] where

N <- 3
L2 <- diag("1", 1 + 2*N)
L2[cbind(1 + (1:N), 1:N)] <- "-a"
L2[cbind(1 + N + (1:N), 1 + 1:N)] <- "-b"
L2 <- as_sym(L2)

Veu <- diag(1, 7)
diag(Veu)[1:4] <- "v2"
diag(Veu)[5:7] <- "w2"
Veu
Veu <- as_sym(Veu)
Veu
L2inv <- inv(L2) 
V2 <- L2inv %*% Veu %*% t(L2inv) 
K2 <- t(L2) %*% inv(Veu) %*% L2