21 - Concentration and covariance matrix in an autoregressive model and in a dynamic linear model

Autoregressive model (AR(1))

Consider this model: x_i = ax_i − 1 + e_i,  i = 1, …, 3 and x₀ = e₀. All terms e₀, …, e₃ are independent and N(0, v²) distributed. Let e = (e₀, …, e₃) and x = (x₀, …x₃). Hence e ∼ N(0, v²I). Isolating error terms gives e = tex(e) = tex(L1)tex(x) = L₁x

N <- 3
L1 <- diag(4)
L1[cbind(1 + (1:N), 1:N)] <- "-a"
L1 <- as_sym(L1)

Since Var(e) = v²I we have Var(e) = v²I = LVar(x)L′ so the covariance matrix of x is V₁ = Var(x) = v²L⁻(L⁻)′ while the concentration matrix (the inverse covariances matrix) is K = v⁻²L′L.

def_sym(v2)
L1inv <- inv(L1)
V1 <- v2 * L1inv %*% t(L1inv)
K1 <- (t(L1) %*% L1) / v2

Dynamic linear model

Augment the AR(1) process above with y_i = bx_i + u_i for i = 1, 2, 3. Suppose u_i ∼ N(0, w²) and all u_i are independent and inpendent of e. Then (e, u) can be expressed in terms of (x, y) as (e, u) = tex(eu) = tex(L2)tex(xy) = L₂(x, y) where

N <- 3
L2 <- diag("1", 1 + 2*N)
L2[cbind(1 + (1:N), 1:N)] <- "-a"
L2[cbind(1 + N + (1:N), 1 + 1:N)] <- "-b"
L2 <- as_sym(L2)

Veu <- diag(1, 7)
diag(Veu)[1:4] <- "v2"
diag(Veu)[5:7] <- "w2"
Veu
Veu <- as_sym(Veu)
Veu
L2inv <- inv(L2) 
V2 <- L2inv %*% Veu %*% t(L2inv) 
K2 <- t(L2) %*% inv(Veu) %*% L2