RLS算法-公式初探

RLS算法-公式推导

不带遗忘因子的推导：递推最小二乘法推导（RLS）——全网最简单易懂的推导过程 - 阿Q在江湖的文章 - 知乎
https://zhuanlan.zhihu.com/p/111758532

对于一组观测点$(x_1,y_1)$，$(x_2,y_2)$，$\cdots$，$(x_n,y_n)$，用$\hat{y}=kx+b$进行线性拟合，有如下优化问题：
$$er{r}_{min}=min\sum _{i=1}^n{\zeta }^{n-i}(\hat{y}-y_i{)}^2=min\sum _{i=1}^n{\zeta }^{n-i}(k{x}_i+b-y_i{)}^2$$
记$f(k,b)={\sum }_{i=1}^n{\zeta }^{n-i}(k{x}_i+b-y_i{)}^2$，分别对$k,b$求偏导，令其等于$0$，有
$$\{\begin{array}{l}\frac{\partial f}{\partial k}={\sum }_{i=1}^n{\zeta }^{n-i}(k{x}_i+b-y_i)x_i=0\\ \\ \frac{\partial f}{\partial b}={\sum }_{i=1}^n{\zeta }^{n-i}(k{x}_i+b-y_i)=0\end{array}$$
改写成矩阵形式：
$$\left(\begin{array}{cc}{\sum }_{i=1}^n{\zeta }^{n-i}{x}_i^2& {\sum }_{i=1}^n{\zeta }^{n-i}{x}_i\\ & \\ {\sum }_{i=1}^n{\zeta }^{n-i}{x}_i& {\sum }_{i=1}^n{\zeta }^{n-i}\end{array}\right)\ast \left(\begin{array}{c}k\\ \\ b\end{array}\right)=\left(\begin{array}{c}{\sum }_{i=1}^n{\zeta }^{n-i}{x}_i{y}_i\\ \\ {\sum }_{i=1}^n{\zeta }^{n-i}{y}_i\end{array}\right)\text{\hspace{1em}(1)}$$
细致一点：
$$\left(\begin{array}{cccc}{x}_1& x_2& \cdots & x_n\\ 1& 1& \cdots & 1\end{array}\right)\ast \left(\begin{array}{cccc}{\zeta }^{n-1}& 0& \cdots & 0\\ 0& {\zeta }^{n-2}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\zeta }^0\end{array}\right)\ast \left(\begin{array}{cc}{x}_1& 1\\ x_2& 1\\ ⋮& ⋮\\ x_n& 1\end{array}\right)\ast \left(\begin{array}{c}k\\ b\end{array}\right)=\left(\begin{array}{cccc}{x}_1& x_2& \cdots & x_n\\ 1& 1& \cdots & 1\end{array}\right)\ast \left(\begin{array}{cccc}{\zeta }^{n-1}& 0& \cdots & 0\\ 0& {\zeta }^{n-2}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\zeta }^0\end{array}\right)\ast \left(\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_n\end{array}\right)$$

记 公式$(1)$中的系数矩阵(自相关矩阵)的逆矩阵为 $R(n)$ ，$n$代表观测数据的组数，有：
$$\begin{array}{rl}R(n)={\left(\begin{array}{cc}{\sum }_{i=1}^n{\zeta }^{n-i}{x}_i^2& {\sum }_{i=1}^n{\zeta }^{n-i}{x}_i\\ & \\ {\sum }_{i=1}^n{\zeta }^{n-i}{x}_i& {\sum }_{i=1}^n{\zeta }^{n-i}\end{array}\right)}^{-1}& =(\zeta \left(\begin{array}{cc}{\sum }_{i=1}^{n-1}{\zeta }^{n-1-i}{x}_i^2& {\sum }_{i=1}^{n-1}{\zeta }^{n-1-i}{x}_i\\ & \\ {\sum }_{i=1}^{n-1}{\zeta }^{n-1-i}{x}_i& {\sum }_{i=1}^{n-1}{\zeta }^{n-1-i}\end{array}\right)+\left(\begin{array}{cc}{x}_n^2& x_n\\ & \\ x_n& {\zeta }^0\end{array}\right){)}^{-1}\\ & \\ & =(\zeta R^{-1}(n-1)+\left(\begin{array}{c}{x}_n\\ \\ 1\end{array}\right)\ast \left(\begin{array}{cc}{x}_n& 1\end{array}\right){)}^{-1}\end{array}\text{\hspace{1em}(2)}$$
记$$\varphi (i)=\left(\begin{array}{c}{x}_i\\ \\ 1\end{array}\right)$$
根据矩阵引逆定理，展开上式可得：
$$R(n)=\frac{R(n-1)}{\zeta }-\frac{R(n-1)\varphi (n){\varphi }^T(n)R(n-1)}{{\zeta }^2+\zeta {\varphi }^T(n)R(n-1)\varphi (n)}\text{\hspace{1em}(3)}$$
记公式$(1)$中右边结果值矩阵(互相关矩阵)为 $D(n)$，有
$$\begin{array}{rl}D(n)=\left(\begin{array}{c}{\sum }_{i=1}^n{\zeta }^{n-i}{x}_i{y}_i\\ \\ {\sum }_{i=1}^n{\zeta }^{n-i}{y}_i\end{array}\right)& =\zeta \left(\begin{array}{c}{\sum }_{i=1}^{n-1}{\zeta }^{n-1-i}{x}_i{y}_i\\ \\ {\sum }_{i=1}^{n-1}{\zeta }^{n-1-i}{y}_i\end{array}\right)+\left(\begin{array}{c}{x}_n{y}_n\\ \\ y_n\end{array}\right)\\ & \\ & =\zeta D(n-1)+\varphi (n)\ast \left(\begin{array}{c}{y}_n\end{array}\right)\end{array}\text{\hspace{1em}(4)}$$

---

记
$$\Theta =\left(\begin{array}{c}k\\ \\ b\end{array}\right)$$
根据公式$(1)$可得：
$$\begin{array}{rl}\Theta (n)=R(n)D(n)& =R(n)[\zeta D(n-1)+\varphi (n)\ast \left(\begin{array}{c}{y}_n\end{array}\right)]\\ & =R(n)[\zeta R^{-1}(n-1)\Theta (n-1)+\varphi (n)\ast \left(\begin{array}{c}{y}_n\end{array}\right)]\\ & =R(n)[\zeta \frac{(R^{-1}(n)-\varphi (n){\varphi }^T(n))}{\zeta }\Theta (n-1)+\varphi (n)\ast \left(\begin{array}{c}{y}_n\end{array}\right)]\\ & =\Theta (n-1)+R(n)\varphi (n)[\left(\begin{array}{c}{y}_n\end{array}\right)-{\varphi }^T(n)\Theta (n-1)]\end{array}\text{\hspace{1em}(5)}$$

总结：只需要计算得到$\Theta (n-1)$和$R(n)$，就可以递推出$\Theta (n)$！