[線性代數] 矩陣二次式的等價運算

假設 $A$ 為 對稱 正定矩陣  (亦即 $A^T = A$ 且 $A$ 的 eigenvalue 全為正值)，現在考慮一個矩陣二次函數：
\[
V(x) = x^T A x + c^T x + d
\]上述矩陣二次項為 $x^TAx$ 且 線性項為 $c^T x$ 常數項為 $d$。

注意到上式可改寫為 $ V(x) = (x-v)^T H (x-v) +d$。WHY? 因為改寫成此形式之後，最小值一目了然，亦即 $x=v$ 可得最小值。


現在若考慮兩組矩陣二次式
\[\left\{ \begin{array}{l}
{V_1}(x) = \frac{1}{2}{(x - a)^T}A(x - a)\\
{V_2}(x) = \frac{1}{2}{(x - b)^T}B(x - b)
\end{array} \right.\]且假設 $A >0$ 為 正定矩陣，$B$為半正定矩陣。

===============
FACT:  $V_1, V_2$ 皆為矩陣二次式，其和亦為矩陣二次式；亦即
$$V(x) = \frac{1}{2} (x-v)^T H (x-v) + d = V_1(x) + V_2(x)
$$===============
故現在問題變成如何找出 $d, H, v $ 用 $A,B,a,b$表示?

===============
FACT: 考慮兩組矩陣二次式
\[\left\{ \begin{array}{l}
{V_1}(x) = \frac{1}{2}{(x - a)^T}A(x - a)\\
{V_2}(x) = \frac{1}{2}{(x - b)^T}B(x - b)
\end{array} \right.\]若 $A^T = A$ 且 $B^T = B$，則 $$V(x) = \frac{1}{2} (x-v)^T H (x-v) + d = V_1(x) + V_2(x)$$ 且
\[\left\{ \begin{array}{l}
H = A + B\\
v = {\left( {A + B} \right)^{ - 1}}\left( {Aa + Bb} \right)\\
d = {V_1}\left( v \right) + {V_2}\left( v \right)
\end{array} \right.\]===============

Proof:
注意到
\[V(x) = \frac{1}{2}\left( {{x^T}Hx - 2{x^T}Hv + {v^T}Hv} \right) + d\]對 $V(x)$ 取 一階導數 與 二階導數，可得
\[\left\{ \begin{array}{l}
\frac{d}{{dx}}V(x) = \frac{1}{2}\left( {Hx + {H^T}x - 2Hv} \right) = H\left( {x - v} \right)\\
\frac{{{d^2}}}{{d{x^2}}}V(x) = \frac{d}{{dx}}\left( {\frac{d}{{dx}}V(x)} \right) = \frac{d}{{dx}}\left( {H\left( {x - v} \right)} \right) = {H^T} = H
\end{array} \right.
\]又因為
\[\left\{ \begin{array}{l}
\frac{d}{{dx}}V(x) = \frac{d}{{dx}}{V_1}(x) + \frac{d}{{dx}}{V_2}(x)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{1}{2}(Ax + {A^T}x - 2Aa) + \frac{1}{2}(Bx + {B^T}x - 2Bb)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = A(x - a) + B(x - b)\\
\frac{{{d^2}}}{{d{x^2}}}V(x) = \frac{d}{{dx}}\left( {A(x - a) + B(x - b)} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = A + B
\end{array} \right.\]現在比較手邊結果可得
\[\begin{array}{l}
\left\{ \begin{array}{l}
\frac{d}{{dx}}V(x) = H\left( {x - v} \right) = A(x - a) + B(x - b)\\
\frac{{{d^2}}}{{d{x^2}}}V(x) = H = A + B
\end{array} \right.\\
 \Rightarrow \left\{ \begin{array}{l}
v = {\left( {A + B} \right)^{ - 1}}\left( {Aa + Bb} \right)\\
H = A + B
\end{array} \right.
\end{array}\]上述 $v$ 的求解 需要 $(A+B)^{-1}$ 但因為我們假設 $A$ 為正定 且 $B$ 為半正定，故 $(A+B)$ 為正定矩陣，反矩陣存在。

接著我們計算常數項 $d$：注意到
\[V\left( v \right) = d = {V_1}\left( v \right) + {V_2}\left( v \right)\]


現在我們進一步推廣上述結果：
================
FACT: 
考慮兩組矩陣二次式
\[\left\{ {\begin{array}{*{20}{l}}
{{V_1}(x) = \frac{1}{2}{{(x - a)}^T}A(x - a)}\\
{{V_2}(x) = \frac{1}{2}{{(Cx - b)}^T}B(Cx - b)}
\end{array}} \right.\]
若 $A^T = A$ 且 $B^T = B$，則 $$V(x) = \frac{1}{2} (x-v)^T H (x-v) + d = V_1(x) + V_2(x)$$  且
\[{\left\{ \begin{array}{l}
\begin{array}{*{20}{l}}
{v = {{\left( {A + {C^T}BC} \right)}^{ - 1}}\left( {Aa + {C^T}Bb} \right)}\\
{H = A + {C^T}BC}
\end{array}\\
d = {V_1}\left( v \right) + {V_2}\left( v \right)
\end{array} \right.}\]================

Proof:
證明同前述 FACT，注意到
\[V(x) = \frac{1}{2}\left( {{x^T}Hx - 2{x^T}Hv + {v^T}Hv} \right) + d\]現在分別對 $V(x)$ 取 一階導數 與 二階導數，可得
\[\left\{ \begin{array}{l}
\frac{d}{{dx}}V(x) = \frac{1}{2}\left( {Hx + {H^T}x - 2Hv} \right) = H\left( {x - v} \right)\\
\frac{{{d^2}}}{{d{x^2}}}V(x) = \frac{d}{{dx}}\left( {\frac{d}{{dx}}V(x)} \right) = \frac{d}{{dx}}\left( {H\left( {x - v} \right)} \right) = {H^T} = H
\end{array} \right.
\]又因為
\[\left\{ {\begin{array}{*{20}{l}}
{\frac{d}{{dx}}V(x) = \frac{d}{{dx}}{V_1}(x) + \frac{d}{{dx}}{V_2}(x)}\\
{\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{1}{2}(Ax + {A^T}x - 2Aa) + \frac{1}{2}({C^T}BCx + {C^T}BCx - 2{C^T}Bb)}\\
{\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = A(x - a) + {C^T}B(Cx - b)}\\
{\frac{{{d^2}}}{{d{x^2}}}V(x) = \frac{d}{{dx}}\left( {A(x - a) + {C^T}B(Cx - b)} \right)}\\
{\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = A + {C^T}BC}
\end{array}} \right.\]現在比較手邊結果可得
\[\begin{array}{*{20}{l}}
{\left\{ {\begin{array}{*{20}{l}}
{\frac{d}{{dx}}V(x) = H\left( {x - v} \right) = A(x - a) + {C^T}B(Cx - b)}\\
{\frac{{{d^2}}}{{d{x^2}}}V(x) = H = A + {C^T}BC}
\end{array}} \right.}\\
{ \Rightarrow \left\{ {\begin{array}{*{20}{l}}
{v = {{\left( {A + {C^T}BC} \right)}^{ - 1}}\left( {Aa + {C^T}Bb} \right)}\\
{H = A + {C^T}BC}
\end{array}} \right.}
\end{array}\]上述 $v$ 的求解 需要 $(A+C^TBC)^{-1}$ 但因為我們假設 $A$ 為正定 且 $C^TBC$ 永遠為半正定，故 $(A+C^TBC)$ 為正定矩陣，反矩陣存在。

接著我們計算常數項 $d$：注意到
\[V\left( v \right) = d = {V_1}\left( v \right) + {V_2}\left( v \right)\]