Eigendecomposition

How to remember the eigendecomposition formula?

Let $A$ be a semi definite matrix with (positive) eigenvalues $\lambda_1, ..., \lambda_n$ and corresponding column eigenvectors $v_1, ..., v_n$, i.e.
$$Av_i=\lambda_i v_i$$
for $i=1,...,n$. Then we have the eigendecomposition formula
$$A = UDU^T$$
where
- $U=[v_1,..., v_n]$ is a matrix of size $n\times n$.
- $D=\text{diag} (\lambda_1, ..., \lambda_n)$ is a diagonal matrix of size $n\times n$.
- $U^T$ denotes the transpose of $U$.

Proof. We have
$$Av_i=\lambda_i v_i, \hspace{0.5cm} \forall i=1,...,n.$$
Thus,
$$A[ v_1, ..., v_n]=[\lambda_1 v_1, ..., \lambda_nv_n]=[ v_1, ..., v_n]\text{diag}(\lambda_1, ..., \lambda_n)$$
Since each $v_i$ is a column vector.
Hence we obtain
$$AU=UD$$
or
$$A=UDU^{-1}=UDU^T.$$

Note. To remember how to derive $A=UDU^T$ we may use the clue $$A[ v_1, ..., v_n]=[\lambda_1 v_1, ..., \lambda_nv_n],$$
which is a matrix formulation of $Av_i=\lambda_iv_i$.

Code Example.
Simulation

import numpy as np
from math import pi
alpha = pi/12
U = np.array([[np.cos(alpha), -np.sin(alpha)], [np.sin(alpha), np.cos(alpha)]])
D = np.diag([1., 2.])
A = U @ D @ U.T
print(f"U={U}")
print(f"D={D}")
print(f"A=UDU^T={A}")

result

U=[[ 0.96592583 -0.25881905] [ 0.25881905 0.96592583]]
D=[[1. 0.] [0. 2.]]
A=UDU^T=[[ 1.0669873 -0.25 ] [-0.25 1.9330127]]

eigendecomposition with numpy

L, U2 = np.linalg.eig(A)
print(f"L={L}")
print(f"U2={U}")

result

L=[1. 2.]
U2=[[ 0.96592583 -0.25881905] [ 0.25881905 0.96592583]]

We can clearly see that U and U2 are the same, this confirms our formula.