The structure of the concentration and covariance matrix in a naive Bayes model

Naive Bayes model

Consider this model: x_i = ax₀ + e_i,  i = 1, …, 4 and x₀ = e₀. All terms e₀, …, e₃ are independent and N(0, 1) distributed. Let e = (e₀, …, e₃) and x = (x₀, …x₃). Isolating error terms gives that e = L₁x where L₁ has the form

L1chr <- diag(4)
L1chr[2:4, 1] <- "-a"
L1 <- ysym(L1chr)
L1

## {{ 1,  0,  0,  0},
##  {-a,  1,  0,  0},
##  {-a,  0,  1,  0},
##  {-a,  0,  0,  1}}

If error terms have variance 1 then Var(e) = LVar(x)L′ so the covariance matrix is V1 = Var(x) = L⁻(L⁻)′ while the concentration matrix (the inverse covariances matrix) is K = L′L.

L1inv <- solve(L1)
K1 <- t(L1) %*% L1
V1 <- L1inv %*% t(L1inv)

cat(
  "\\begin{align} 
    K_1 &= ", tex(K1), " \\\\ 
   V_1 &= ", tex(V1), " 
  \\end{align}", sep = "")

Slightly more elaborate:

L2chr <- diag(4)
L2chr[2:4, 1] <- c("-a1", "-a2", "-a3")
L2 <- ysym(L2chr)
L2

## {{  1,   0,   0,   0},
##  {-a1,   1,   0,   0},
##  {-a2,   0,   1,   0},
##  {-a3,   0,   0,   1}}

Vechr <- diag(4)
Vechr[cbind(1:4, 1:4)] <- c("w1", "w2", "w2", "w2")
Ve <- ysym(Vechr)
Ve

## {{w1,  0,  0,  0},
##  { 0, w2,  0,  0},
##  { 0,  0, w2,  0},
##  { 0,  0,  0, w2}}

L2inv <- solve(L2)
K2 <- t(L2) %*% solve(Ve) %*% L2
V2 <- L2inv %*% Ve %*% t(L2inv)

cat(
  "\\begin{align} 
    K_2 &= ", tex(K2), " \\\\ 
   V_2 &= ", tex(V2), " 
  \\end{align}", sep = "")