返回 Guide 目录

线性代数 / Linear Algebra

矩阵代数

Matrix Algebra

本页结构

核心概念

  • 非奇异、正交、幂等和半正定矩阵 Nonsingular, orthogonal, idempotent and positive semidefinite matrices
  • 二维旋转和几何解释 2D rotations and geometric interpretation
  • 二次型与协方差矩阵直觉 Quadratic forms and covariance-matrix intuition

学习顺序

  1. 把代数定义和几何含义对应起来。 Link algebraic definitions to geometric meaning.
  2. 讨论协方差矩阵时强调对称性和半正定性。 For covariance matrices, emphasize symmetry and positive semidefiniteness.
  3. 用低维例子快速校验结论。 Use small-dimensional examples to sanity-check claims.

基础知识

Fundamental Knowledge

Let AA and BB be square n×nn \times n matrices. Then all of the following hold:

令 AA 和 BB 为 n×nn \times n 方阵。那么以下所有条件都成立：

非奇异矩阵

Nonsingular Matrices

A nonsingular matrix is invertible. AA (n×nn \times n) is nonsingular if and only if any (and therefore all) of the following hold:

非奇异矩阵是可逆的。 AA (n×nn \times n) 是非奇异的当且仅当以下任何一个（因此全部）成立：

  1. Columns of AA span Rn\mathbb{R}^n, or equivalently, rank(A)=dim(range(A))=n\text{rank}(A) = \text{dim}(\text{range}(A)) = n
  2. A⊺A^\intercal is nonsingular
  3. det⁡(A)≠0\det(A) \neq 0
  4. Ax=0Ax = 0 has only the trivial solution x=0x = 0; dim(nul(A))=0\text{dim}(\text{nul}(A)) = 0
  1. AA 的列跨越 Rn\mathbb{R}^n，或等效地，rank(A)=dim(range(A))=n\text{rank}(A) = \text{dim}(\text{range}(A)) = n
  2. A⊺A^\intercal is nonsingular
  3. det⁡(A)≠0\det(A) \neq 0
  4. Ax=0Ax = 0 has only the trivial solution x=0x = 0; dim(nul(A))=0\text{dim}(\text{nul}(A)) = 0

Note that if A=[abcd]A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}, then A−1=1det⁡(A)[d−b−ca]A^{-1} = \frac{1}{\det(A)} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}. Larger inverses may be found via Gauss-Jordan Elimination: [A∣I]→elementary row operations[I∣A−1][A \mid I] \xrightarrow{\text{elementary row operations}} [I \mid A^{-1}]

请注意，如果 A=[abcd]A = \begin{bmatrix} a & b \\ c & d \end{bmatrix}，则 A−1=1det⁡(A)[d−b−ca]A^{-1} = \frac{1}{\det(A)} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}。更大的逆可以通过高斯约当消元法找到：[A∣I]→elementary row operations[I∣A−1][A \mid I] \xrightarrow{\text{elementary row operations}} [I \mid A^{-1}]

2D 旋转矩阵

2D Rotation Matrices

2D Rotation matrices by θ\theta radians counter-clockwise about the origin are matrices in the form Rθ=[cos⁡θ−sin⁡θsin⁡θcos⁡θ]R_\theta = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}.

绕原点逆时针旋转 θ\theta 弧度的 2D 旋转矩阵是 Rθ=[cos⁡θ−sin⁡θsin⁡θcos⁡θ]R_\theta = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix} 形式的矩阵。

正交矩阵

Orthogonal Matrices

Orthogonal matrices (unitary matrices in the reals) are square with orthonormal row and column vectors. They are nonsingular and satisfy Q⊺=Q−1Q^\intercal = Q^{-1}. Orthogonal matrices can be interpreted as rotation matrices.

正交矩阵（实数酉矩阵）是正交行向量和列向量的平方。它们是非奇异的并且满足 Q⊺=Q−1Q^\intercal = Q^{-1}。正交矩阵可以解释为旋转矩阵。

幂等矩阵

Idempotent Matrices

Idempotent matrices are square matrices satisfying A2=AA^2 = A. In other words, the effect of applying the linear transformation AA twice is the same as applying it once. Projection matrices are Idempotent.

幂等矩阵是满足 A2=AA^2 = A 的方阵。换句话说，应用两次线性变换 AA 的效果与应用一次相同。投影矩阵是幂等的。

正半定矩阵

Positive Semi-definite Matrices

Covariance and correlation matrices are always positive semi-definite and positive definite if there is no perfect linear dependence among random variables. Each of the following conditions is a necessary and sufficient condition for AA to be positive semi-definite/definite:

如果随机变量之间不存在完美的线性相关性，则协方差和相关矩阵始终是正半定和正定矩阵。以下每个条件都是 AA 为正半定/正定的充要条件：

Positive Semi-Definite Positive Definite
z⊺Az≥0z^\intercal Az \ge 0 for all column vectors zz z⊺Az>0z^\intercal Az > 0 for all nonzero column vectors zz
All eigenvalues are nonnegative All eigenvalues are positive
All upper left/lower right submatrices have nonnegative determinants All upper left/lower right submatrices have positive determinants
半正定 正定
z⊺Az≥0z^\intercal Az \ge 0 for all column vectors zz z⊺Az>0z^\intercal Az > 0 for all nonzero column vectors zz
所有特征值都是非负的 所有特征值均为正
所有左上/右下子矩阵都有非负行列式 所有左上/右下子矩阵都有正行列式

Note that if AA has negative diagonal elements, then AA cannot be positive semi-definite.

请注意，如果 AA 具有负对角线元素，则 AA 不能是半正定的。

补充讲解

用映射和子空间理解矩阵

Think in maps and subspaces

矩阵是线性映射。秩、零空间、列空间和可逆性描述这个映射保留或丢失了哪些信息。

A matrix is a linear map. Rank, null space, column space, and invertibility describe what information the map preserves or loses.

半正定是协方差语言

PSD is the covariance language

协方差矩阵必须对称半正定。负特征值通常意味着估计噪声、数据问题或不合法的变换。

Covariance matrices must be symmetric positive semidefinite. Negative eigenvalues usually indicate estimation noise, data issues, or invalid transformations.

条件数影响实现可靠性

Conditioning affects implementation

即使公式在代数上正确，病态矩阵也会让数值求逆不稳定。必要时应优先使用矩阵分解或正则化。

Even when a formula is algebraically correct, ill-conditioned matrices can make numerical inversion unstable. Prefer decompositions or regularization when needed.