【最优化方法】对称矩阵的对角化

正交化方法

设 $R^n$ 中线性无关组 $a_1,a_2,a_3,\dots ,a_n$，令
$$\begin{array}{rl}& {\beta }_1={\alpha }_1\\ & {\beta }_2={\alpha }_2-\frac{[{\alpha }_2{\beta }_1]}{||{\beta }_1||}{\beta }_1\\ & {\beta }_3={\alpha }_3-\frac{[{\alpha }_3{\beta }_1]}{||{\beta }_1||}{\beta }_1-\frac{[{\alpha }_3{\beta }_2]}{||{\beta }_2||}{\beta }_2\\ & {\beta }_n={\alpha }_3-\frac{[{\alpha }_n{\beta }_1]}{||{\beta }_1||}{\beta }_1-\cdots -\frac{[{\alpha }_n{\beta }_{n-1}]}{||{\beta }_{n-1}||}{\beta }_{n-1}\end{array}$$

该方法称为施密特正交化（Gram–Schmidt process）

$[x,y]$ 为向量的内积，$||x||=[x,x]$
$$[x,y]=x_1{y}_1+x_2{y}_2+\cdots +x_n{y}_n$$

示例

将向量组
$$\begin{array}{rl}& {\alpha }_1=(1,1,0,0{)}^T,{\alpha }_2=(1,0,1,0{)}^T\\ & {\alpha }_3=(-1,0,0,1{)}^T,{\alpha }_4=(1,-1,-1,1{)}^T\end{array}$$

标准正交化

解： 先正交化
$$\begin{array}{rl}& {\beta }_1=(1,1,0,0{)}^T\\ & {\beta }_2=(1,0,1,0{)}^T-\frac{1}{2}(1,1,0,0{)}^T=\frac{1}{2}(1,-1,2,0{)}^T\\ & {\beta }_3=(-1,0,0,1{)}^T+\frac{1}{2}(1,1,0,0{)}^T+\frac{1}{6}(1,-1,2,0{)}^T=\frac{1}{3}(-1,1,1,3{)}^T\\ & {\beta }_4=(1,-1,-1,1{)}^T-0-0-0=(1,-1,-1,1{)}^T\end{array}$$

再标准化
$$\begin{array}{rl}& {\beta }_1=\frac{1}{\sqrt{2}}(1,1,0,0{)}^T\\ & {\beta }_2=\frac{1}{\sqrt{6}}(1,-1,2,0{)}^T\\ & {\beta }_3=\frac{1}{2\sqrt{3}}(-1,1,1,3{)}^T\\ & {\beta }_4=\frac{1}{2}(1,-1,-1,1{)}^T\end{array}$$

矩阵正交化

A = ( 0 1 1 − 1 1 0 − 1 1 1 − 1 0 1 − 1 1 1 0 ) A = \begin{pmatrix} 0 & 1 & 1 & -1 \\ 1 & 0 & -1 & 1 \\ 1 & -1 & 0 & 1 \\ -1 & 1 & 1 & 0 \\ \end{pmatrix} A= ​011−1​10−11​1−101​−1110​ ​

求一正交矩阵 $P$，使 $P^{T}AP$ 成对角形。

解：
∣ A − λ E ∣   =   ∣ − λ 1 1 − 1 1 − λ − 1 1 1 − 1 − λ 1 − 1 1 1 − λ ∣   =   ∣ 1 − λ 1 − λ 1 − λ 1 − λ 1 − λ − 1 1 1 − 1 − λ 1 − 1 1 1 − λ ∣ =   ( 1 − λ ) ∣ 1 1 1 1 1 − λ − 1 1 1 − 1 − λ 1 − 1 1 1 − λ ∣   =   ( 1 − λ ) ∣ 1 1 1 1 0 − λ − 1 − 2 0 0 − 2 − λ − 1 0 0 2 2 1 − λ ∣   =   ( 1 − λ ) 2 ( λ 2 + 2 λ − 3 ) = ( λ − 1 ) 3 ( λ + 3 ) \begin{aligned} & |A-\lambda E| ~=~ \begin{vmatrix}-\lambda & 1 & 1 & -1 \\ 1 & -\lambda & -1 & 1 \\ 1 & -1 & -\lambda & 1 \\ -1 & 1 & 1 & -\lambda \\ \end{vmatrix} ~=~ \begin{vmatrix} 1-\lambda & 1-\lambda & 1-\lambda & 1-\lambda \\ 1 & -\lambda & -1 & 1 \\ 1 & -1 & -\lambda & 1 \\ -1 & 1 & 1 & -\lambda \\ \end{vmatrix} \\\\\\ & =~ (1-\lambda) \begin{vmatrix} 1 & 1 & 1 & 1 \\ 1 & -\lambda & -1 & 1 \\ 1 & -1 & -\lambda & 1 \\ -1 & 1 & 1 & -\lambda \\ \end{vmatrix} ~=~ (1-\lambda) \begin{vmatrix} 1 & 1 & 1 & 1 \\ 0 & -\lambda-1 & -2 & 0 \\ 0 & -2 & -\lambda-1 & 0 \\ 0 & 2 & 2 & 1-\lambda \\ \end{vmatrix} \\\\\\\ & =~ (1-\lambda)^2(\lambda^2+2\lambda-3) = (\lambda-1)^3(\lambda+3) \end{aligned}  ​∣A−λE∣ =  ​−λ11−1​1−λ−11​1−1−λ1​−111−λ​ ​ =  ​1−λ11−1​1−λ−λ−11​1−λ−1−λ1​1−λ11−λ​ ​= (1−λ) ​111−1​1−λ−11​1−1−λ1​111−λ​ ​ = (1−λ) ​1000​1−λ−1−22​1−2−λ−12​1001−λ​ ​= (1−λ)2(λ2+2λ−3)=(λ−1)3(λ+3)​

求得 ${\lambda }_1={\lambda }_2={\lambda }_3=1,{\lambda }_4=-3$ ，

把 ${\lambda }_1=1$ （3重）带入齐次方程组，得
A − E = ( − 1 1 1 − 1 1 − 1 − 1 1 1 − 1 − 1 1 − 1 1 1 − 1 ) = ( 1 − 1 − 1 1 0 0 0 0 0 0 0 0 0 0 0 0 ) A - E = \begin{pmatrix}-1 & 1 & 1 & -1 \\ 1 & -1 & -1 & 1 \\ 1 & -1 & -1 & 1 \\-1 & 1 & 1 & -1 \\ \end{pmatrix}= \begin{pmatrix} 1 & -1 & -1 & 1 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ \end{pmatrix} A−E= ​−111−1​1−1−11​1−1−11​−111−1​ ​= ​1000​−1000​−1000​1000​ ​ { x 1 = x 2 + x 3 − x 4 x 2 = x 2 x 3 =           x 3 x 4 =                    x 4 = > x 2 ( 1 1 0 0 ) + x 3 ( 1 0 1 0 ) + x 4 ( − 1 0 0 1 ) \begin{cases} x_1 = x_2 + x_3 - x_4 \\ x_2 = x_2 \\ x_3 = ~~~~~~~~~x_3 \\ x_4 = ~~~~~~~~~~~~~~~~~~x_4 \\ \end{cases} => x_2 \begin{pmatrix} 1 \\ 1 \\ 0 \\ 0 \end{pmatrix}+ x_3 \begin{pmatrix} 1 \\ 0 \\ 1 \\ 0 \end{pmatrix} + x_4 \begin{pmatrix} -1 \\ 0 \\ 0 \\ 1 \end{pmatrix} ⎩ ⎨ ⎧​x1​=x2​+x3​−x4​x2​=x2​x3​=         x3​x4​=                  x4​​=>x2​ ​1100​ ​+x3​ ​1010​ ​+x4​ ​−1001​ ​

得出基础解系 ${\zeta }_1,{\zeta }_2,{\zeta }_3$ ，
ζ 1 = ( 1 1 0 0 ) , ζ 2 = ( 1 0 1 0 ) , ζ 3 = ( − 1 0 0 1 ) \zeta_1 = \begin{pmatrix} 1 \\ 1 \\ 0 \\ 0 \end{pmatrix}, \zeta_2 = \begin{pmatrix} 1 \\ 0 \\ 1 \\ 0 \end{pmatrix}, \zeta_3 = \begin{pmatrix} -1 \\ 0 \\ 0 \\ 1 \end{pmatrix} ζ1​= ​1100​ ​,ζ2​= ​1010​ ​,ζ3​= ​−1001​ ​

将 ${\zeta }_1,{\zeta }_2,{\zeta }_3$ 正交化 : 取 ${\eta }_1={\zeta }_1$，
η 2 = ζ 2 − [ η 1 , ζ 2 ] ∣ ∣ η 1 ∣ ∣ η 1 = ( 1 0 1 0 ) − 1 2 ( 1 1 0 0 ) = 1 2 ( 1 − 1 2 0 ) η 3 = ζ 3 − [ η 3 , ζ 1 ] ∣ ∣ η 1 ∣ ∣ η 1 − [ η 3 , ζ 2 ] ∣ ∣ η 2 ∣ ∣ η 2 = ( − 1 0 0 1 ) + 1 2 ( 1 1 0 0 ) + 1 6 ( 1 − 1 2 0 ) = 1 3 ( − 1 1 1 3 ) \begin{aligned} & \eta_2 = \zeta_2 - \frac{[\eta_1,\zeta_2]}{||\eta_1||}\eta_1 = \begin{pmatrix} 1 \\ 0 \\ 1 \\ 0 \end{pmatrix}- \frac{1}{2} \begin{pmatrix} 1 \\ 1 \\ 0 \\ 0 \end{pmatrix}= \frac{1}{2} \begin{pmatrix} 1 \\ -1 \\ 2 \\ 0 \end{pmatrix} \\ & \eta_3 = \zeta_3 - \frac{[\eta_3,\zeta_1]}{||\eta_1||}\eta_1- \frac{[\eta_3,\zeta_2]}{||\eta_2||}\eta_2 = \begin{pmatrix} -1 \\ 0 \\ 0 \\ 1 \end{pmatrix} + \frac{1}{2} \begin{pmatrix} 1 \\ 1 \\ 0 \\ 0 \end{pmatrix} + \frac{1}{6} \begin{pmatrix} 1 \\ -1 \\ 2 \\ 0 \end{pmatrix} = \frac{1}{3} \begin{pmatrix} -1 \\ 1 \\ 1 \\ 3 \end{pmatrix} \end{aligned} ​η2​=ζ2​−∣∣η1​∣∣[η1​,ζ2​]​η1​= ​1010​ ​−21​ ​1100​ ​=21​ ​1−120​ ​η3​=ζ3​−∣∣η1​∣∣[η3​,ζ1​]​η1​−∣∣η2​∣∣[η3​,ζ2​]​η2​= ​−1001​ ​+21​ ​1100​ ​+61​ ​1−120​ ​=31​ ​−1113​ ​​

将 ${\eta }_1,{\eta }_2,{\eta }_3$ 单位化求得 $p_1,p_2,p_3$
$$\begin{array}{rl}& p_1=\frac{1}{\sqrt{2}}(1,1,0,0{)}^T\\ & p_2=\frac{1}{\sqrt{6}}(1,-1,2,0{)}^T\\ & p_3=\frac{1}{\sqrt{12}}(-1,1,1,3{)}^T\end{array}$$

把 ${\lambda }_4=-3$ 带入齐次方程组，得
A + 3 E = ( 3 1 1 − 1 1 3 − 1 1 1 − 1 3 1 − 1 1 1 3 ) = ( 1 0 0 − 1 0 1 0 1 0 0 1 1 0 0 0 0 ) A + 3E = \begin{pmatrix} 3 & 1 & 1 & -1 \\ 1 & 3 & -1 & 1 \\ 1 & -1 & 3 & 1 \\-1 & 1 & 1 & 3 \\ \end{pmatrix}= \begin{pmatrix} 1 & 0 & 0 & -1 \\ 0 & 1 & 0 & 1 \\ 0 & 0 & 1 & 1 \\ 0 & 0 & 0 & 0 \\ \end{pmatrix} A+3E= ​311−1​13−11​1−131​−1113​ ​= ​1000​0100​0010​−1110​ ​ { x 1 = x 4 x 2 = − x 4 x 3 = − x 4 x 4 = x 4 = > x 4 ( 1 − 1 − 1 1 ) \begin{cases} x_1 = x_4 \\ x_2 = -x_4 \\ x_3 = -x_4 \\ x_4 = x_4 \\ \end{cases} => x_4 \begin{pmatrix} 1 \\ -1 \\ -1 \\ 1 \end{pmatrix} ⎩ ⎨ ⎧​x1​=x4​x2​=−x4​x3​=−x4​x4​=x4​​=>x4​ ​1−1−11​ ​

得出基础解系 ${\zeta }_4$，
ζ 4 = ( 1 − 1 − 1 1 ) \zeta_4 = \begin{pmatrix} 1 \\ -1 \\ -1 \\ 1 \end{pmatrix} ζ4​= ​1−1−11​ ​

将 ${\zeta }_4$ 单位化，得 $p_4$，
$$p_4=\frac{1}{2}(1,-1,-1,1{)}^T$$

将 $p_1,p_2,p_3,p_4$ 构成正交矩阵 $P$
P = ( p 1 , p 2 , p 3 , p 4 ) = ( 1 2 1 6 − 1 12 1 2 1 2 − 1 6 1 12 − 1 2 0 2 6 1 12 − 1 2 0 0 3 12 1 2 ) P = (p_1,p_2,p_3,p_4) = \begin{pmatrix} \frac{1}{\sqrt2} & \frac{1}{\sqrt6} & -\frac{1}{\sqrt{12}} & \frac{1}{2} \\ \frac{1}{\sqrt2} & -\frac{1}{\sqrt6} & \frac{1}{\sqrt{12}} & -\frac{1}{2} \\ 0 & \frac{2}{\sqrt6} & \frac{1}{\sqrt{12}} & -\frac{1}{2} \\ 0 & 0 & \frac{3}{\sqrt{12}} & \frac{1}{2} \\ \end{pmatrix} P=(p1​,p2​,p3​,p4​)= ​2 ​1​2 ​1​00​6 ​1​−6 ​1​6 ​2​0​−12 ​1​12 ​1​12 ​1​12 ​3​​21​−21​−21​21​​ ​

有
P T A P = ( 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 − 3 ) P^{T}AP = \begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & -3 \\ \end{pmatrix} PTAP= ​1000​0100​0010​000−3​ ​