考慮以下問題 令 $A$ 為 任意 $n \times n$ 矩陣，試問下列陳述是否正確
Claim: $ A^2 = I $則 $A = I$?
讀者可能注意到 $A^2 = AA = I$ 表示我們有
\[
A= A^{-1}
\]故上述陳述看來頗為誘人讓人想回答 True 但事實上此陳述為錯誤陳述，因為若我們考慮
\[A = \left[ {\begin{array}{*{20}{c}}
0&1\\
1&0
\end{array}} \right]\]則
\[
A^2 = I_{2 \times 2}
\]但 $A \neq I_{2 \times 2}$
12/14/2015
12/07/2015
[線性代數] 線性算子 與 特徵值/特徵向量(1) - 線性算子的矩陣代表 所表示的等價特徵問題
令 $V$ 為有限維度向量空間配備基底 $S=\{{\bf s}_1,{\bf s}_2,...,{\bf s}_n\}$ 且 $L: V \to V$ 為線性算子，則必存在唯一 的 一個 對應於基底 $S$ 的 $n \times n$ 矩陣代表 $A$ 來 表示 $L$ (我們稱此矩陣代表 $A$ 為 representation of $L$ with respect to $S$) 使得 對任意 ${\bf x} \in V$ 我們有
\[
[L({\bf x})]_S = A [{\bf x}]_S \;\;\;\;\; (\star)
\]其中 $[{\bf x}]_S$ 表示 ${\bf x}$基於 基底 $S$ 的座標向量 (coordinate vector)，亦即 若\[{[{\bf{x}}]_S} = \left[ \begin{array}{l}
{a_1}\\
{a_2}\\
\vdots \\
{a_n}
\end{array} \right] \Leftrightarrow {\bf{x}} = {a_1}{{\bf{s}}_1} + {a_2}{{\bf{s}}_2} + ... + {a_n}{{\bf{s}}_n}\]
現在我們回憶原本 定義在 線性算子 $L$ 之上的特徵問題：亦即我們要 找出一組 特徵值 $\lambda$ 與其對應的 特徵向量 ${\bf x} \neq {\bf 0}$ 且 ${\bf x} \in V$ 滿足
\[
L({\bf x}) = \lambda {\bf x}\;\;\;\;\; (*)
\]
觀察 $(\star)$ 式，我們可得到透過 $A$ 矩陣所描述的等價特徵問題如下
\[\begin{array}{l}
{[L({\bf{x}})]_S} = A{[{\bf{x}}]_S}\;\\
\Rightarrow {[\lambda {\bf{x}}]_S} = A{[{\bf{x}}]_S}\;\\
\Rightarrow \lambda {[{\bf{x}}]_S} = A{[{\bf{x}}]_S}\;
\end{array}
\]則我們的目標變成要找 一組 $\lambda \in \mathbb{R}$ (or $\in \mathbb{C}$) 與 $[{\bf x}]_S \neq {\bf 0}$ 且 $[{\bf x}]_S \in \mathbb{R}^n$ (or $\mathbb{C}^n$) 滿足
\[
\lambda {[{\bf{x}}]_S} = A{[{\bf{x}}]_S}
\]
現在我們考慮以下例子：
Example 1
令 $L: P_1 \to P_1$ 為線性算子滿足
\[
L( at + b) := bt + a
\]另外給定一組 $P_1$ 的 有序基底 $S:=\{t, 1\}$，
(a) 試求透過 基底 $S$ 的矩陣 $A$ 來代表線性算子 $L$
(b) 定義對應於 $A$ 矩陣的等價特徵問題
Solution (a):
令 $A$ 為 線性算子 $L$ 為透過 基底 $S$ 的矩陣代表 ，則 $A$ 必須滿足 對任意 ${\bf x} \in P_1$，
\[
[L({\bf x})]_S = A [{\bf x}]_S
\]注意到我們的基底元素 ${\bf s}_1, {\bf s}_2 \in P_1$ 故我們現在若觀察
\[\left\{ \begin{array}{l}
L\left( {{{\bf{s}}_1}} \right) = L\left( t \right) = 1\\
L\left( {{{\bf{s}}_2}} \right) = L\left( 1 \right) = t
\end{array} \right.\]亦即
\[\left\{ \begin{array}{l}
{\left[ {L\left( {{{\bf{s}}_1}} \right)} \right]_S} = {\left[ 1 \right]_S} = \left[ \begin{array}{l}
0\\
1
\end{array} \right]\\
{\left[ {L\left( {{{\bf{s}}_2}} \right)} \right]_S} = {\left[ t \right]_S} = \left[ \begin{array}{l}
1\\
0
\end{array} \right]
\end{array} \right.\]故我們求得矩陣代表 (基於 $S$) 為
\[A = \left[ {\begin{array}{*{20}{c}}
{{{\left[ {L\left( {{{\bf{s}}_1}} \right)} \right]}_S}}&{{{\left[ {L\left( {{{\bf{s}}_2}} \right)} \right]}_S}}
\end{array}} \right] = \left[ {\begin{array}{*{20}{c}}
0&1\\
1&0
\end{array}} \right]\]
Solution (b)
等價的矩陣代表的特徵問題為要找 一組 $\lambda \in \mathbb{R}$ (or $\in \mathbb{C}$) 與 $[{\bf x}]_S \neq {\bf 0}$ 且 $[{\bf x}]_S \in \mathbb{R}^2$ 滿足
\[
A{[{\bf{x}}]_S} = \lambda {[{\bf{x}}]_S}
\] 或者更簡而言之，我們要找 $\lambda \in \mathbb{R}^1$ (or $\mathbb{C}^1$) 與非零向量 ${\bf v} \in \mathbb{R}^2$ 滿足
\[
A {\bf v} = \lambda {\bf v}
\]
上述討論說明了 對線性算子的特徵問題(Eigenproblem) 可以透過 其矩陣代表 描述，事實上對任意方陣，我們皆可定義其特徵問題如下：
若 $A$ 為 $n \times n$ 方陣 ，定義 線性算子 $L: \mathbb{R}^n \to \mathbb{R}^n$ or ($\mathbb{C}^n \to \mathbb{C}^n$) 滿足 對任意 ${\bf x} \in \mathbb{R}^n$ (or $\mathbb{C}^n$)
\[
L({\bf x}) = A{\bf x}
\]現在，若存在 $\lambda \in \mathbb{R}$ (or $\mathbb{C}$) 且 ${\bf x} \neq {\bf 0}, {\bf x} \in \mathbb{R}^n$ or ($\mathbb{C}^n$) 使得
\[
A {\bf x} = \lambda {\bf x}
\]則我們說 $\lambda$ 為 $A$ 的特徵值 且 ${\bf x}$ 為其 對應於 $\lambda$ 的特徵向量，亦即 $\lambda$ 為 $L$ 的特徵值 且 ${\bf x}$ 為其 對應於 $\lambda$ 的特徵向量
\[
[L({\bf x})]_S = A [{\bf x}]_S \;\;\;\;\; (\star)
\]其中 $[{\bf x}]_S$ 表示 ${\bf x}$基於 基底 $S$ 的座標向量 (coordinate vector)，亦即 若\[{[{\bf{x}}]_S} = \left[ \begin{array}{l}
{a_1}\\
{a_2}\\
\vdots \\
{a_n}
\end{array} \right] \Leftrightarrow {\bf{x}} = {a_1}{{\bf{s}}_1} + {a_2}{{\bf{s}}_2} + ... + {a_n}{{\bf{s}}_n}\]
現在我們回憶原本 定義在 線性算子 $L$ 之上的特徵問題：亦即我們要 找出一組 特徵值 $\lambda$ 與其對應的 特徵向量 ${\bf x} \neq {\bf 0}$ 且 ${\bf x} \in V$ 滿足
\[
L({\bf x}) = \lambda {\bf x}\;\;\;\;\; (*)
\]
觀察 $(\star)$ 式，我們可得到透過 $A$ 矩陣所描述的等價特徵問題如下
\[\begin{array}{l}
{[L({\bf{x}})]_S} = A{[{\bf{x}}]_S}\;\\
\Rightarrow {[\lambda {\bf{x}}]_S} = A{[{\bf{x}}]_S}\;\\
\Rightarrow \lambda {[{\bf{x}}]_S} = A{[{\bf{x}}]_S}\;
\end{array}
\]則我們的目標變成要找 一組 $\lambda \in \mathbb{R}$ (or $\in \mathbb{C}$) 與 $[{\bf x}]_S \neq {\bf 0}$ 且 $[{\bf x}]_S \in \mathbb{R}^n$ (or $\mathbb{C}^n$) 滿足
\[
\lambda {[{\bf{x}}]_S} = A{[{\bf{x}}]_S}
\]
現在我們考慮以下例子：
Example 1
令 $L: P_1 \to P_1$ 為線性算子滿足
\[
L( at + b) := bt + a
\]另外給定一組 $P_1$ 的 有序基底 $S:=\{t, 1\}$，
(a) 試求透過 基底 $S$ 的矩陣 $A$ 來代表線性算子 $L$
(b) 定義對應於 $A$ 矩陣的等價特徵問題
Solution (a):
令 $A$ 為 線性算子 $L$ 為透過 基底 $S$ 的矩陣代表 ，則 $A$ 必須滿足 對任意 ${\bf x} \in P_1$，
\[
[L({\bf x})]_S = A [{\bf x}]_S
\]注意到我們的基底元素 ${\bf s}_1, {\bf s}_2 \in P_1$ 故我們現在若觀察
\[\left\{ \begin{array}{l}
L\left( {{{\bf{s}}_1}} \right) = L\left( t \right) = 1\\
L\left( {{{\bf{s}}_2}} \right) = L\left( 1 \right) = t
\end{array} \right.\]亦即
\[\left\{ \begin{array}{l}
{\left[ {L\left( {{{\bf{s}}_1}} \right)} \right]_S} = {\left[ 1 \right]_S} = \left[ \begin{array}{l}
0\\
1
\end{array} \right]\\
{\left[ {L\left( {{{\bf{s}}_2}} \right)} \right]_S} = {\left[ t \right]_S} = \left[ \begin{array}{l}
1\\
0
\end{array} \right]
\end{array} \right.\]故我們求得矩陣代表 (基於 $S$) 為
\[A = \left[ {\begin{array}{*{20}{c}}
{{{\left[ {L\left( {{{\bf{s}}_1}} \right)} \right]}_S}}&{{{\left[ {L\left( {{{\bf{s}}_2}} \right)} \right]}_S}}
\end{array}} \right] = \left[ {\begin{array}{*{20}{c}}
0&1\\
1&0
\end{array}} \right]\]
Solution (b)
等價的矩陣代表的特徵問題為要找 一組 $\lambda \in \mathbb{R}$ (or $\in \mathbb{C}$) 與 $[{\bf x}]_S \neq {\bf 0}$ 且 $[{\bf x}]_S \in \mathbb{R}^2$ 滿足
\[
A{[{\bf{x}}]_S} = \lambda {[{\bf{x}}]_S}
\] 或者更簡而言之，我們要找 $\lambda \in \mathbb{R}^1$ (or $\mathbb{C}^1$) 與非零向量 ${\bf v} \in \mathbb{R}^2$ 滿足
\[
A {\bf v} = \lambda {\bf v}
\]
若 $A$ 為 $n \times n$ 方陣 ，定義 線性算子 $L: \mathbb{R}^n \to \mathbb{R}^n$ or ($\mathbb{C}^n \to \mathbb{C}^n$) 滿足 對任意 ${\bf x} \in \mathbb{R}^n$ (or $\mathbb{C}^n$)
\[
L({\bf x}) = A{\bf x}
\]現在，若存在 $\lambda \in \mathbb{R}$ (or $\mathbb{C}$) 且 ${\bf x} \neq {\bf 0}, {\bf x} \in \mathbb{R}^n$ or ($\mathbb{C}^n$) 使得
\[
A {\bf x} = \lambda {\bf x}
\]則我們說 $\lambda$ 為 $A$ 的特徵值 且 ${\bf x}$ 為其 對應於 $\lambda$ 的特徵向量，亦即 $\lambda$ 為 $L$ 的特徵值 且 ${\bf x}$ 為其 對應於 $\lambda$ 的特徵向量
12/05/2015
[線性代數] 線性算子 與 特徵值/特徵向量(0)
======================
Definition: Linear Operator
令 $V$ 為 $n$ 維 向量空間 且 $L: V \to V$ 為線性轉換 (Linear transformation)：亦即給定任意兩向量 ${\bf u,v} \in V$ 與 $c \in \mathbb{R}$ (or $c \in \mathbb{C}$)滿足
\[\begin{array}{l}
L\left( {{\bf{u}} + {\bf{v}}} \right) = L\left( {\bf{u}} \right) + L\left( {\bf{v}} \right)\\
L\left( {c{\bf{u}}} \right) = cL\left( {\bf{u}} \right)
\end{array}\]則我們稱該 $L:V \to V$ 為定義在 $V$ 上的線性算子 (Linear Operator)
======================
那麼我們現在想問一個基本問題：是否可以找到 一組非零向量 ${\bf v} \neq {\bf 0}$ 與 純量 $\lambda \in \mathbb{R}$ (or $\in \mathbb{C}$) 使得
\[
L({\bf v}) = \lambda{\bf v}
\]
此問題在工程領域有諸多應用，一般而言上述問題又稱為特徵值問題。
Comments:
1. 上述討論中所提及的 Linear Operator 僅僅表示 domain 與 codomain 都為同一個向量空間 $V$，其餘皆與線性轉換定義相同。也就是說若我們將 domain $V$ 與 codomain $W$ 設為不同的向量空間，且若 $L: V \to W$ 滿足
\[\begin{array}{l}
L\left( {{\bf{u}} + {\bf{v}}} \right) = L\left( {\bf{u}} \right) + L\left( {\bf{v}} \right)\\
L\left( {c{\bf{u}}} \right) = cL\left( {\bf{u}} \right)
\end{array}\]則我們稱 $L$ 為線性轉換 (Linear Transformation)。
2. 若 ${\bf v} = {\bf 0}$ 則 $L({\bf v}) = \lambda{\bf v}$ 自動滿足故我們只需關心 ${\bf v} \neq {\bf 0}$ 的情況
3. 注意到若 $V:= \mathbb{R}^n$ 或者複數向量空間 $V= \mathbb{C}^n$ 則我們可從幾何觀點來看上述特徵值問題，則此問題變成決定是否 $L({\bf v})$ 與 ${\bf v}$ 平行。
4. 給定任意線性算子 $L: V \to V$ ，其(實數)特徵值與其對應的特徵向量不一定存在，比如說考慮 旋轉轉換 $L: \mathbb{R}^2 \to \mathbb{R}^2$ 滿足
\[L\left( {\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]} \right) = \left[ {\begin{array}{*{20}{c}}
{\cos \theta }&{ - \sin \theta }\\
{\sin \theta }&{\cos \theta }
\end{array}} \right]\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]\]其中 $0 < \theta < \pi$ 則此算子為線性算子但不存在(實數)特徵值與特徵向量。但存在 (複數)特徵值與複數特徵向量
Example 1:
令 Linear operator $L : \mathbb{R}^2 \to \mathbb{R}^2$ 滿足
\[L\left( {\bf{v}} \right) = L\left( {\left[ \begin{array}{l}
{v_1}\\
{v_2}
\end{array} \right]} \right): = \left[ \begin{array}{l}
{v_1}\\
- {v_2}
\end{array} \right]\]試求 ${\bf v} \neq {\bf 0}$ 與 $\lambda$ 使得 $L({\bf v}) = \lambda {\bf v}$?
Solution
觀察
\[L\left( {\bf{v}} \right) = L\left( {\left[ \begin{array}{l}
{v_1}\\
{v_2}
\end{array} \right]} \right): = \left[ \begin{array}{l}
{v_1}\\
- {v_2}
\end{array} \right]\]因為我們要求 $L({\bf v}) = \lambda {\bf v}$ 故
\[\begin{array}{*{20}{l}}
{\left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{ - {v_2}}
\end{array}} \right] = \lambda \left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{{v_2}}
\end{array}} \right]}\\
{ \Rightarrow \lambda \left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{{v_2}}
\end{array}} \right] - \left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{ - {v_2}}
\end{array}} \right] = {\bf{0}}}\\
{ \Rightarrow \left[ {\begin{array}{*{20}{c}}
{\lambda - 1}&0\\
0&{\lambda + 1}
\end{array}} \right]\left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{{v_2}}
\end{array}} \right] = \left[ {\begin{array}{*{20}{l}}
0\\
0
\end{array}} \right]}
\end{array}\]由上式可知我們在決定 $\lambda$ 使得並求解 Null Space of ${\left[ {\begin{array}{*{20}{c}}
{\lambda - 1}&0\\
0&{ \left( {\lambda + 1} \right)}
\end{array}} \right]}$ 故若我們選 $\lambda := \lambda_1 = 1 $ 則對應的 ${\bf v}_1 = [s\;\;0]^T$ 其中 $s \in \mathbb{R}^1$。
另外若選 $\lambda := \lambda_2 = -1$ 則對應的 ${\bf v}_2 = [0\;\;t]$ 其中 $t \in \mathbb{R}^1$。 $\square$
由上述討論所求出的 $\lambda$ 與 ${\bf v}$ 即為所謂特徵值與特徵向量，現在我們引入 eigenvalue 與 eigenvector 定義
======================
Definition: Eigenvalue and Eigenvector
令 $V$ 為 $n$ 維度向量空間且 $L: V \to V$ 為定義在 $V$ 上的 線性算子。我們稱 $\lambda$ 為 $L$ 的特徵值( eigenvalue of $L$) 若存在一組非零向量 ${\bf x} \in V$ 使得
\[
L({\bf x}) = \lambda {\bf x}
\]且 任意非零向量 ${\bf x}$ 滿足上式稱為 $L$ 對應於特徵值 $\lambda $ 的特徵向量 (eigenvector of $L$ associated with the eigenvalue $\lambda$)
======================
Remark:
1. 上述定義中的純量 $\lambda$ 與向量 ${\bf x}$皆可為 實數或者複數。有興趣的讀者請看 Example 2
2. 若我們允許 ${\bf x} = {\bf 0}$ 則 任意 $\lambda$ 都可為 eigenvalue 因為
\[
L({\bf x}) = \lambda {\bf x} \Rightarrow L({\bf 0}) = \lambda {\bf 0}
\]則任何 $\lambda$ 都滿足上式。
3. 事實上特徵值與特徵向量在無窮維向量空間 亦可被定義，有興趣讀者請看 Example 3
===================
Fact: 令 $L:V\to V$ 為線性算子，若 ${\bf x}$ 為 eigenvector of $L$ associated with the eigenvalue ${\lambda}$ 則 對任意實數 $c \in \mathbb{R}^1$ 我們有
\[
L(c {\bf x}) = \lambda (c {\bf x})
\]===================
Proof:
由於 ${\bf x}$ 為 eigenvector of $L$ associated with the eigenvalue ${\lambda}$ 我們有 對任意非零向量 ${\bf x} \neq {\bf 0}$
\[
L({\bf x}) = \lambda ({\bf x})
\] 現在觀察
\[\begin{array}{l}
L(c{\bf{x}}) = cL({\bf{x}})\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c\left( {\lambda {\bf{x}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \lambda \left( {c{\bf{x}}} \right)
\end{array}\]讀者應注意到上述第一條等式應用 $L$ 為線性算子的性質。$\square$
以下我們用一個例子來說明何時會發生複數的 eigenvalue $\lambda$ 與 負數特徵向量 ${\bf x}$
Example 2:
考慮線性算子 $L: \mathbb{R}^2 \to \mathbb{R}^2$ 滿足
\[L\left( {\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]} \right) = \left[ \begin{array}{l}
- {x_2}\\
{x_1}
\end{array} \right]\]試求 eigenvalue of $L$ 與對應的 eigenvector
Solution
回憶前述定義，線性算子 $L$ 的特徵值 $\lambda$ 必須滿足 $L\left( {\bf{x}} \right) = \lambda {\bf{x}}$ 故現在觀察
\[\begin{array}{l}
L\left( {\bf{x}} \right) = \lambda {\bf{x}}\\
\Rightarrow L\left( {\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]} \right) = \left[ \begin{array}{l}
- {x_2}\\
{x_1}
\end{array} \right] = \lambda \left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]\\
\Rightarrow \lambda \left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right] - \left[ \begin{array}{l}
- {x_2}\\
{x_1}
\end{array} \right] = \left[ \begin{array}{l}
0\\
0
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
\lambda &1\\
{ - 1}&\lambda
\end{array}} \right]\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right] = \left[ \begin{array}{l}
0\\
0
\end{array} \right]
\end{array}\]觀察上述線性系統方程，求解 Null Space of $\left[ {\begin{array}{*{20}{c}}
\lambda &1\\
{ - 1}&\lambda
\end{array}} \right]$ 可得到以下結果：
若 $\lambda = \lambda_1 = i$ 則對應的特徵向量可透過將 $\lambda = \lambda_1$ 帶回上述線性系統方程並求解 ${\bf x}_1$ 如下 \[
{\bf x}_1 = s[i \;\; 1]^T,\;\; \forall s \in \mathbb{R}^1
\]
若 $\lambda = \lambda_i =-i$ 則對應的特徵向量可透過將 $\lambda = \lambda_2$ 帶回上述線性系統方程並求解 ${\bf x}_2$ 如下
\[
{\bf x}_2 = t[-i \;\; 1]^T,\;\; \forall t \in \mathbb{R}^1
\]
注意到上述結果之中，$L$ 的 特徵值皆為複數：亦即 $\lambda_{i} \in \mathbb{C}$ 對任意 $i=1,2$ 且對應的特徵向量亦為 $\mathbb{C}^2$ 複數向量。且這表示原本題目之中要求 $L: \mathbb{R}^2 \to \mathbb{R}^2$ 並無法找到對應的複數向量除非我們更改 domain 與 codomain 使其變成 $\mathbb{C}^2$
Example 3: Eigenvalue Problem in Function Space
考慮 $V:=C^\infty (\mathbb{R})$ 亦即 $V$ 為所有 單變數實數函數所成的向量空間 且我們假設其上的函數任意階導數存在。現在令 $L: V \to V$ 為線性算子滿足
\[
L(f) := f'
\]
則我們想問是否可找到 常數 $\lambda$ 與 函數 $f \neq 0$ 且 $f \in V$ 使得 $L(f) = \lambda f$ 成立？
Solution
令 $f \in V$ 為單變數函數，注意到 $L(f) = f'$ 為線性算子(why?)，現在觀察 $L(f) = f' = \lambda f $ 亦即 我們要找出 $\lambda$ 與對應的 $f \neq 0, f \in V$ 滿足
\[
f' = \lambda f
\] 由於我們要找的 $f$ 必須無窮維導數存在 且一次導數必須滿足 $f' = \lambda f$，故我們猜 $f(t) = e^{\lambda t}$ 則
\[\frac{d}{{dt}}\left( {{e^{\lambda t}}} \right) = \lambda {e^{\lambda t}} = \lambda f\left( t \right)\]故現在我們找到一組 $f$ 滿足該方程，但是否有其他人選？答案是肯定的，比如說我們改令
\[
f(t) = K e^{\lambda t}
\]仍為該方程 $f' = \lambda f$ 的解 （讀者可自行驗證），因此我們有以下結果，對任意 特徵值 $\lambda \in \mathbb{R}^1$，其特徵向量 $f(t) = K e^{\lambda t} $ 其中 $K$ 為任意非零常數。$\square$
Comment:
1. 上述例子中顯示微分方程 $f' = \lambda f$ 的解 $f(t) = exp(\lambda t)$ 剛好為該 $L(f) = f'$ 的對應於特徵值 $\lambda$ 的特徵向量。
2. 若我們只關心 $f>0$ 上述微分方程可直接求解不必猜測，解法如下：
\[\begin{array}{l}
f' = \lambda f\\
\Rightarrow \frac{{df\left( t \right)}}{{dt}} = \lambda f\left( t \right)\\
\Rightarrow \frac{{df\left( t \right)}}{{f\left( t \right)}} = \lambda dt\\
\Rightarrow \int {\frac{1}{{f\left( t \right)}}df\left( t \right)} = \int {\lambda dt} \\
\Rightarrow \log f\left( t \right) + C = \lambda t + D\\
\Rightarrow f\left( t \right) = {e^{\lambda t}}{e^{D-C}}: = K{e^{\lambda t}}
\end{array}\]
Definition: Linear Operator
令 $V$ 為 $n$ 維 向量空間 且 $L: V \to V$ 為線性轉換 (Linear transformation)：亦即給定任意兩向量 ${\bf u,v} \in V$ 與 $c \in \mathbb{R}$ (or $c \in \mathbb{C}$)滿足
\[\begin{array}{l}
L\left( {{\bf{u}} + {\bf{v}}} \right) = L\left( {\bf{u}} \right) + L\left( {\bf{v}} \right)\\
L\left( {c{\bf{u}}} \right) = cL\left( {\bf{u}} \right)
\end{array}\]則我們稱該 $L:V \to V$ 為定義在 $V$ 上的線性算子 (Linear Operator)
======================
那麼我們現在想問一個基本問題：是否可以找到 一組非零向量 ${\bf v} \neq {\bf 0}$ 與 純量 $\lambda \in \mathbb{R}$ (or $\in \mathbb{C}$) 使得
\[
L({\bf v}) = \lambda{\bf v}
\]
此問題在工程領域有諸多應用，一般而言上述問題又稱為特徵值問題。
Comments:
1. 上述討論中所提及的 Linear Operator 僅僅表示 domain 與 codomain 都為同一個向量空間 $V$，其餘皆與線性轉換定義相同。也就是說若我們將 domain $V$ 與 codomain $W$ 設為不同的向量空間，且若 $L: V \to W$ 滿足
\[\begin{array}{l}
L\left( {{\bf{u}} + {\bf{v}}} \right) = L\left( {\bf{u}} \right) + L\left( {\bf{v}} \right)\\
L\left( {c{\bf{u}}} \right) = cL\left( {\bf{u}} \right)
\end{array}\]則我們稱 $L$ 為線性轉換 (Linear Transformation)。
2. 若 ${\bf v} = {\bf 0}$ 則 $L({\bf v}) = \lambda{\bf v}$ 自動滿足故我們只需關心 ${\bf v} \neq {\bf 0}$ 的情況
3. 注意到若 $V:= \mathbb{R}^n$ 或者複數向量空間 $V= \mathbb{C}^n$ 則我們可從幾何觀點來看上述特徵值問題，則此問題變成決定是否 $L({\bf v})$ 與 ${\bf v}$ 平行。
4. 給定任意線性算子 $L: V \to V$ ，其(實數)特徵值與其對應的特徵向量不一定存在，比如說考慮 旋轉轉換 $L: \mathbb{R}^2 \to \mathbb{R}^2$ 滿足
\[L\left( {\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]} \right) = \left[ {\begin{array}{*{20}{c}}
{\cos \theta }&{ - \sin \theta }\\
{\sin \theta }&{\cos \theta }
\end{array}} \right]\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]\]其中 $0 < \theta < \pi$ 則此算子為線性算子但不存在(實數)特徵值與特徵向量。但存在 (複數)特徵值與複數特徵向量
Example 1:
令 Linear operator $L : \mathbb{R}^2 \to \mathbb{R}^2$ 滿足
\[L\left( {\bf{v}} \right) = L\left( {\left[ \begin{array}{l}
{v_1}\\
{v_2}
\end{array} \right]} \right): = \left[ \begin{array}{l}
{v_1}\\
- {v_2}
\end{array} \right]\]試求 ${\bf v} \neq {\bf 0}$ 與 $\lambda$ 使得 $L({\bf v}) = \lambda {\bf v}$?
Solution
觀察
\[L\left( {\bf{v}} \right) = L\left( {\left[ \begin{array}{l}
{v_1}\\
{v_2}
\end{array} \right]} \right): = \left[ \begin{array}{l}
{v_1}\\
- {v_2}
\end{array} \right]\]因為我們要求 $L({\bf v}) = \lambda {\bf v}$ 故
\[\begin{array}{*{20}{l}}
{\left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{ - {v_2}}
\end{array}} \right] = \lambda \left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{{v_2}}
\end{array}} \right]}\\
{ \Rightarrow \lambda \left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{{v_2}}
\end{array}} \right] - \left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{ - {v_2}}
\end{array}} \right] = {\bf{0}}}\\
{ \Rightarrow \left[ {\begin{array}{*{20}{c}}
{\lambda - 1}&0\\
0&{\lambda + 1}
\end{array}} \right]\left[ {\begin{array}{*{20}{l}}
{{v_1}}\\
{{v_2}}
\end{array}} \right] = \left[ {\begin{array}{*{20}{l}}
0\\
0
\end{array}} \right]}
\end{array}\]由上式可知我們在決定 $\lambda$ 使得並求解 Null Space of ${\left[ {\begin{array}{*{20}{c}}
{\lambda - 1}&0\\
0&{ \left( {\lambda + 1} \right)}
\end{array}} \right]}$ 故若我們選 $\lambda := \lambda_1 = 1 $ 則對應的 ${\bf v}_1 = [s\;\;0]^T$ 其中 $s \in \mathbb{R}^1$。
另外若選 $\lambda := \lambda_2 = -1$ 則對應的 ${\bf v}_2 = [0\;\;t]$ 其中 $t \in \mathbb{R}^1$。 $\square$
由上述討論所求出的 $\lambda$ 與 ${\bf v}$ 即為所謂特徵值與特徵向量，現在我們引入 eigenvalue 與 eigenvector 定義
======================
Definition: Eigenvalue and Eigenvector
令 $V$ 為 $n$ 維度向量空間且 $L: V \to V$ 為定義在 $V$ 上的 線性算子。我們稱 $\lambda$ 為 $L$ 的特徵值( eigenvalue of $L$) 若存在一組非零向量 ${\bf x} \in V$ 使得
\[
L({\bf x}) = \lambda {\bf x}
\]且 任意非零向量 ${\bf x}$ 滿足上式稱為 $L$ 對應於特徵值 $\lambda $ 的特徵向量 (eigenvector of $L$ associated with the eigenvalue $\lambda$)
======================
Remark:
1. 上述定義中的純量 $\lambda$ 與向量 ${\bf x}$皆可為 實數或者複數。有興趣的讀者請看 Example 2
2. 若我們允許 ${\bf x} = {\bf 0}$ 則 任意 $\lambda$ 都可為 eigenvalue 因為
\[
L({\bf x}) = \lambda {\bf x} \Rightarrow L({\bf 0}) = \lambda {\bf 0}
\]則任何 $\lambda$ 都滿足上式。
3. 事實上特徵值與特徵向量在無窮維向量空間 亦可被定義，有興趣讀者請看 Example 3
===================
Fact: 令 $L:V\to V$ 為線性算子，若 ${\bf x}$ 為 eigenvector of $L$ associated with the eigenvalue ${\lambda}$ 則 對任意實數 $c \in \mathbb{R}^1$ 我們有
\[
L(c {\bf x}) = \lambda (c {\bf x})
\]===================
由於 ${\bf x}$ 為 eigenvector of $L$ associated with the eigenvalue ${\lambda}$ 我們有 對任意非零向量 ${\bf x} \neq {\bf 0}$
\[
L({\bf x}) = \lambda ({\bf x})
\] 現在觀察
\[\begin{array}{l}
L(c{\bf{x}}) = cL({\bf{x}})\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c\left( {\lambda {\bf{x}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \lambda \left( {c{\bf{x}}} \right)
\end{array}\]讀者應注意到上述第一條等式應用 $L$ 為線性算子的性質。$\square$
以下我們用一個例子來說明何時會發生複數的 eigenvalue $\lambda$ 與 負數特徵向量 ${\bf x}$
Example 2:
考慮線性算子 $L: \mathbb{R}^2 \to \mathbb{R}^2$ 滿足
\[L\left( {\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]} \right) = \left[ \begin{array}{l}
- {x_2}\\
{x_1}
\end{array} \right]\]試求 eigenvalue of $L$ 與對應的 eigenvector
Solution
回憶前述定義，線性算子 $L$ 的特徵值 $\lambda$ 必須滿足 $L\left( {\bf{x}} \right) = \lambda {\bf{x}}$ 故現在觀察
\[\begin{array}{l}
L\left( {\bf{x}} \right) = \lambda {\bf{x}}\\
\Rightarrow L\left( {\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]} \right) = \left[ \begin{array}{l}
- {x_2}\\
{x_1}
\end{array} \right] = \lambda \left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right]\\
\Rightarrow \lambda \left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right] - \left[ \begin{array}{l}
- {x_2}\\
{x_1}
\end{array} \right] = \left[ \begin{array}{l}
0\\
0
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
\lambda &1\\
{ - 1}&\lambda
\end{array}} \right]\left[ \begin{array}{l}
{x_1}\\
{x_2}
\end{array} \right] = \left[ \begin{array}{l}
0\\
0
\end{array} \right]
\end{array}\]觀察上述線性系統方程，求解 Null Space of $\left[ {\begin{array}{*{20}{c}}
\lambda &1\\
{ - 1}&\lambda
\end{array}} \right]$ 可得到以下結果：
若 $\lambda = \lambda_1 = i$ 則對應的特徵向量可透過將 $\lambda = \lambda_1$ 帶回上述線性系統方程並求解 ${\bf x}_1$ 如下 \[
{\bf x}_1 = s[i \;\; 1]^T,\;\; \forall s \in \mathbb{R}^1
\]
若 $\lambda = \lambda_i =-i$ 則對應的特徵向量可透過將 $\lambda = \lambda_2$ 帶回上述線性系統方程並求解 ${\bf x}_2$ 如下
\[
{\bf x}_2 = t[-i \;\; 1]^T,\;\; \forall t \in \mathbb{R}^1
\]
注意到上述結果之中，$L$ 的 特徵值皆為複數：亦即 $\lambda_{i} \in \mathbb{C}$ 對任意 $i=1,2$ 且對應的特徵向量亦為 $\mathbb{C}^2$ 複數向量。且這表示原本題目之中要求 $L: \mathbb{R}^2 \to \mathbb{R}^2$ 並無法找到對應的複數向量除非我們更改 domain 與 codomain 使其變成 $\mathbb{C}^2$
Example 3: Eigenvalue Problem in Function Space
考慮 $V:=C^\infty (\mathbb{R})$ 亦即 $V$ 為所有 單變數實數函數所成的向量空間 且我們假設其上的函數任意階導數存在。現在令 $L: V \to V$ 為線性算子滿足
\[
L(f) := f'
\]
則我們想問是否可找到 常數 $\lambda$ 與 函數 $f \neq 0$ 且 $f \in V$ 使得 $L(f) = \lambda f$ 成立？
Solution
令 $f \in V$ 為單變數函數，注意到 $L(f) = f'$ 為線性算子(why?)，現在觀察 $L(f) = f' = \lambda f $ 亦即 我們要找出 $\lambda$ 與對應的 $f \neq 0, f \in V$ 滿足
\[
f' = \lambda f
\] 由於我們要找的 $f$ 必須無窮維導數存在 且一次導數必須滿足 $f' = \lambda f$，故我們猜 $f(t) = e^{\lambda t}$ 則
\[\frac{d}{{dt}}\left( {{e^{\lambda t}}} \right) = \lambda {e^{\lambda t}} = \lambda f\left( t \right)\]故現在我們找到一組 $f$ 滿足該方程，但是否有其他人選？答案是肯定的，比如說我們改令
\[
f(t) = K e^{\lambda t}
\]仍為該方程 $f' = \lambda f$ 的解 （讀者可自行驗證），因此我們有以下結果，對任意 特徵值 $\lambda \in \mathbb{R}^1$，其特徵向量 $f(t) = K e^{\lambda t} $ 其中 $K$ 為任意非零常數。$\square$
Comment:
1. 上述例子中顯示微分方程 $f' = \lambda f$ 的解 $f(t) = exp(\lambda t)$ 剛好為該 $L(f) = f'$ 的對應於特徵值 $\lambda$ 的特徵向量。
2. 若我們只關心 $f>0$ 上述微分方程可直接求解不必猜測，解法如下：
\[\begin{array}{l}
f' = \lambda f\\
\Rightarrow \frac{{df\left( t \right)}}{{dt}} = \lambda f\left( t \right)\\
\Rightarrow \frac{{df\left( t \right)}}{{f\left( t \right)}} = \lambda dt\\
\Rightarrow \int {\frac{1}{{f\left( t \right)}}df\left( t \right)} = \int {\lambda dt} \\
\Rightarrow \log f\left( t \right) + C = \lambda t + D\\
\Rightarrow f\left( t \right) = {e^{\lambda t}}{e^{D-C}}: = K{e^{\lambda t}}
\end{array}\]
11/26/2015
[線性代數] Orthonormal Basis 與 Gram-Schmidt Process (1)
延續前篇 [線性代數] Orthonormal Basis 與 Gram-Schmidt Process (0) 的問題，以下我們正式引入 Gram-Schmidt Process
Theorem: Gram-Schmidt Process
令 $V$ 為 有限維度內積空間 且 令 $W\neq \{ {\bf 0}\}$ 為 $V$中的 $m$-維子空間。則此子空間 $W$ 存在一組正交基底 $T =\{{\bf w}_1,...{\bf w}_m\}$
Proof:
我們首先建構一組 orthogonal basis $T^* :=\{{\bf v}_1,{\bf v}_2...,{\bf v}_m\}$ for $W$。由於 $W$ 為 $V$ 的子空間，故我們可在 $W$ 其上選取一組基底，令 $S=\{{\bf u}_1,...,{\bf u}_m\} $ 接著我們選取其中任意一個向量，比如說 ${\bf u}_1 \in S$ 並稱此向量為 ${\bf v}_1$ 亦即我們重新定義
\[
{\bf v}_1 := {\bf u}_1
\]注意到此 ${\bf v}_1 \in W_1:=span\{ v_1 \}$ 其中 $W_1$ 為 $W$ 的子空間
接著我們要尋找 ${\bf v}_2$，我們希望此向量 ${\bf v}_2$ 落在 $W$ 子空間 $W_2 = span\{ {\bf u}_1, {\bf u}_2\} $ 且 ${\bf v}_2$ 與 ${\bf v}_1$ 彼此 orthogonal。但注意到我們有 ${\bf v}_1 := {\bf u}_1 $ 故 ${\bf v}_2 \in W_2 = span\{ {\bf u}_1 , {\bf u}_2\} = span\{ {\bf v}_1, {\bf u}_2\} $ 也就是說 ${\bf v}_2$ 可透過 ${\bf v}_1$ 與 ${\bf u}_2$ 做線性組合
\[
{\bf v}_2 = a_1 {\bf v}_1 + a_2 {\bf u}_2
\]其中 $a_1, a_2$ 待定。 注意到由於我們要讓 ${\bf v}_2$ 與 ${\bf v}_1$ 彼此 orthogonal 故 $\langle {\bf v}_2, {\bf v}_1 \rangle = 0$ 故現在觀察
\[\begin{array}{l}
\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow \langle {a_1}{{\bf{v}}_1} + {a_2}{{\bf{u}}_2},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow {a_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle + {a_2}\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow {a_1} = - {a_2}\frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }} \;\;\;\; (*)
\end{array}\]注意到上式中 $\langle {\bf v}_1, {\bf v}_2 \rangle \neq 0$ 因為 ${\bf v}_1 = {\bf u}_1 \in S$ 且 $S$ 為 非零子空間 $W$ 的基底 。注意到 $(*)$ 為一條方程式兩個未知數 $a_1,a_2$ 故可任意令 $a_2 \in \mathbb{R}^1$ 為自由變數解得 $a_1$ 。為了計算方便起見我們選 $a_2 :=1$ 則
\[{a_1} = - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}
\]此說明了
\[\begin{array}{l}
{{\bf{v}}_2} = {a_1}{{\bf{v}}_1} + {a_2}{{\bf{u}}_2}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} + {{\bf{u}}_2} = {{\bf{u}}_2} - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1}
\end{array}\]至此我們有了一組 orthogonal subset $\{{\bf v}_1, {\bf v}_2\}$ for $W$。
接著我們尋找 ${\bf v}_3 \in W_3 := span\{{\bf u}_1, {\bf u}_2, {\bf u}_3\}$ 且 ${\bf v}_3$ 與 ${\bf v}_1, {\bf v}_2$ 為 orthogonal。注意到
\[{W_3}: = span\{ {{\bf{u}}_1},{{\bf{u}}_2},{{\bf{u}}_3}\} = span\{ {{\bf{v}}_1},{{\bf{v}}_2},{{\bf{u}}_3}\} \]故
\[\begin{array}{l}
{{\bf{v}}_3} \in span\{ {{\bf{v}}_1},{{\bf{v}}_2},{{\bf{u}}_3}\} \\
\Rightarrow {{\bf{v}}_3} = {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3}
\end{array}\]且又因為我們要求 ${\bf v}_3$ 與 ${\bf v}_1, {\bf v}_2$ 為 orthogonal故
\[\begin{array}{l}
{{\bf{v}}_3},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow \langle {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow {b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle = 0\\
\\
\langle {{\bf{v}}_3},{{\bf{v}}_2}\rangle = 0\\
\Rightarrow \langle {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3},{{\bf{v}}_2}\rangle = 0\\
\Rightarrow {b_1}\langle {{\bf{v}}_1},{{\bf{v}}_2}\rangle + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle = 0
\end{array}\]也就是說我們有一組聯立方程
\[\begin{array}{l}
\left\{ {\begin{array}{*{20}{l}}
{\langle {{\bf{v}}_3},{{\bf{v}}_1}\rangle = 0}\\
{\langle {{\bf{v}}_3},{{\bf{v}}_2}\rangle = 0}
\end{array}} \right.\\
\Rightarrow \left\{ {\begin{array}{*{20}{l}}
{{b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle + {b_2}\underbrace {\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle }_{ = 0} + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle = 0}\\
{{b_1}\underbrace {\langle {{\bf{v}}_1},{{\bf{v}}_2}\rangle }_{ = 0} + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle = 0}
\end{array}} \right.\\
\Rightarrow \left\{ {\begin{array}{*{20}{l}}
{{b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle = 0}\\
{{b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle = 0}
\end{array}} \right.
\end{array}\]注意到 ${\bf v}_2 \neq {\bf 0}$ 因為 ${\bf v}_2$ 需與 ${\bf v}_1$ 正交，故兩條方程三個未知數 $b_1,b_2,b_3$，可指定一自由變數，故選 $b_3 :=1 \in \mathbb{R}^1$ 則我們可解得 $b_1, b_2$ 如下
\[\left\{ {\begin{array}{*{20}{l}}
{{b_1} = - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}}\\
{{b_2} = - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}}
\end{array}} \right.\]
也就是說
\[\begin{array}{l}
{{\bf{v}}_3} = {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3}\\
\Rightarrow {{\bf{v}}_3} = - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}{{\bf{v}}_2} + {{\bf{u}}_3}\\
\Rightarrow {{\bf{v}}_3} = {{\bf{u}}_3} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}{{\bf{v}}_2}
\end{array}
\]至此我們有了一組 orthogonal subset $\{{\bf v}_1,{\bf v}_2,{\bf v}_3\} $ for $W$
重複上述步驟 (by induction)我們可建構一組 orthogonal basis $T^* :=\{{\bf v}_1,{\bf v}_2,...,{\bf v}_m\}$ 最後我們對每一組 ${\bf v}_i$ 做正規化，定義
\[
{\bf w}_i := \frac{1}{||{\bf v}_i||} {\bf v}_i
\]則我們得到一組 orthonormal basis $T := \{{\bf w}_1,...,{\bf w}_m\}$
讀者可用以下幾個例子做練習
Example 1: 令 $S=\{[1\;\;2]^T, [-3\;\;4]^T\}$ 為 ordered basis for $ V:= \mathbb{R}^2$ 且其上內積為標準內積。
(a) 試利用 Gram-Schmidt process 找出 orthogonal basis
(b) 試利用 Gram-Schmidt process 找出 orthonormal basis
Example 2: 令 $V := P_3$ 且其上的內積定義為
\[
\langle p(t), q(t) \rangle := \int_0^1 p(t) q(t) dt
\] 現在令 $W$ 為 $P_3$ 子空間且基底為 $\{t,t^2\}$ 試求 orthonormal basis for $W$
Theorem: Gram-Schmidt Process
令 $V$ 為 有限維度內積空間 且 令 $W\neq \{ {\bf 0}\}$ 為 $V$中的 $m$-維子空間。則此子空間 $W$ 存在一組正交基底 $T =\{{\bf w}_1,...{\bf w}_m\}$
Proof:
我們首先建構一組 orthogonal basis $T^* :=\{{\bf v}_1,{\bf v}_2...,{\bf v}_m\}$ for $W$。由於 $W$ 為 $V$ 的子空間，故我們可在 $W$ 其上選取一組基底，令 $S=\{{\bf u}_1,...,{\bf u}_m\} $ 接著我們選取其中任意一個向量，比如說 ${\bf u}_1 \in S$ 並稱此向量為 ${\bf v}_1$ 亦即我們重新定義
\[
{\bf v}_1 := {\bf u}_1
\]注意到此 ${\bf v}_1 \in W_1:=span\{ v_1 \}$ 其中 $W_1$ 為 $W$ 的子空間
接著我們要尋找 ${\bf v}_2$，我們希望此向量 ${\bf v}_2$ 落在 $W$ 子空間 $W_2 = span\{ {\bf u}_1, {\bf u}_2\} $ 且 ${\bf v}_2$ 與 ${\bf v}_1$ 彼此 orthogonal。但注意到我們有 ${\bf v}_1 := {\bf u}_1 $ 故 ${\bf v}_2 \in W_2 = span\{ {\bf u}_1 , {\bf u}_2\} = span\{ {\bf v}_1, {\bf u}_2\} $ 也就是說 ${\bf v}_2$ 可透過 ${\bf v}_1$ 與 ${\bf u}_2$ 做線性組合
\[
{\bf v}_2 = a_1 {\bf v}_1 + a_2 {\bf u}_2
\]其中 $a_1, a_2$ 待定。 注意到由於我們要讓 ${\bf v}_2$ 與 ${\bf v}_1$ 彼此 orthogonal 故 $\langle {\bf v}_2, {\bf v}_1 \rangle = 0$ 故現在觀察
\[\begin{array}{l}
\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow \langle {a_1}{{\bf{v}}_1} + {a_2}{{\bf{u}}_2},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow {a_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle + {a_2}\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow {a_1} = - {a_2}\frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }} \;\;\;\; (*)
\end{array}\]注意到上式中 $\langle {\bf v}_1, {\bf v}_2 \rangle \neq 0$ 因為 ${\bf v}_1 = {\bf u}_1 \in S$ 且 $S$ 為 非零子空間 $W$ 的基底 。注意到 $(*)$ 為一條方程式兩個未知數 $a_1,a_2$ 故可任意令 $a_2 \in \mathbb{R}^1$ 為自由變數解得 $a_1$ 。為了計算方便起見我們選 $a_2 :=1$ 則
\[{a_1} = - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}
\]此說明了
\[\begin{array}{l}
{{\bf{v}}_2} = {a_1}{{\bf{v}}_1} + {a_2}{{\bf{u}}_2}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} + {{\bf{u}}_2} = {{\bf{u}}_2} - \frac{{\langle {{\bf{u}}_2},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1}
\end{array}\]至此我們有了一組 orthogonal subset $\{{\bf v}_1, {\bf v}_2\}$ for $W$。
接著我們尋找 ${\bf v}_3 \in W_3 := span\{{\bf u}_1, {\bf u}_2, {\bf u}_3\}$ 且 ${\bf v}_3$ 與 ${\bf v}_1, {\bf v}_2$ 為 orthogonal。注意到
\[{W_3}: = span\{ {{\bf{u}}_1},{{\bf{u}}_2},{{\bf{u}}_3}\} = span\{ {{\bf{v}}_1},{{\bf{v}}_2},{{\bf{u}}_3}\} \]故
\[\begin{array}{l}
{{\bf{v}}_3} \in span\{ {{\bf{v}}_1},{{\bf{v}}_2},{{\bf{u}}_3}\} \\
\Rightarrow {{\bf{v}}_3} = {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3}
\end{array}\]且又因為我們要求 ${\bf v}_3$ 與 ${\bf v}_1, {\bf v}_2$ 為 orthogonal故
\[\begin{array}{l}
{{\bf{v}}_3},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow \langle {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3},{{\bf{v}}_1}\rangle = 0\\
\Rightarrow {b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle = 0\\
\\
\langle {{\bf{v}}_3},{{\bf{v}}_2}\rangle = 0\\
\Rightarrow \langle {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3},{{\bf{v}}_2}\rangle = 0\\
\Rightarrow {b_1}\langle {{\bf{v}}_1},{{\bf{v}}_2}\rangle + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle = 0
\end{array}\]也就是說我們有一組聯立方程
\[\begin{array}{l}
\left\{ {\begin{array}{*{20}{l}}
{\langle {{\bf{v}}_3},{{\bf{v}}_1}\rangle = 0}\\
{\langle {{\bf{v}}_3},{{\bf{v}}_2}\rangle = 0}
\end{array}} \right.\\
\Rightarrow \left\{ {\begin{array}{*{20}{l}}
{{b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle + {b_2}\underbrace {\langle {{\bf{v}}_2},{{\bf{v}}_1}\rangle }_{ = 0} + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle = 0}\\
{{b_1}\underbrace {\langle {{\bf{v}}_1},{{\bf{v}}_2}\rangle }_{ = 0} + {b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle = 0}
\end{array}} \right.\\
\Rightarrow \left\{ {\begin{array}{*{20}{l}}
{{b_1}\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle = 0}\\
{{b_2}\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle + {b_3}\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle = 0}
\end{array}} \right.
\end{array}\]注意到 ${\bf v}_2 \neq {\bf 0}$ 因為 ${\bf v}_2$ 需與 ${\bf v}_1$ 正交，故兩條方程三個未知數 $b_1,b_2,b_3$，可指定一自由變數，故選 $b_3 :=1 \in \mathbb{R}^1$ 則我們可解得 $b_1, b_2$ 如下
\[\left\{ {\begin{array}{*{20}{l}}
{{b_1} = - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}}\\
{{b_2} = - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}}
\end{array}} \right.\]
也就是說
\[\begin{array}{l}
{{\bf{v}}_3} = {b_1}{{\bf{v}}_1} + {b_2}{{\bf{v}}_2} + {b_3}{{\bf{u}}_3}\\
\Rightarrow {{\bf{v}}_3} = - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}{{\bf{v}}_2} + {{\bf{u}}_3}\\
\Rightarrow {{\bf{v}}_3} = {{\bf{u}}_3} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_1}\rangle }}{{\langle {{\bf{v}}_1},{{\bf{v}}_1}\rangle }}{{\bf{v}}_1} - \frac{{\langle {{\bf{u}}_3},{{\bf{v}}_2}\rangle }}{{\langle {{\bf{v}}_2},{{\bf{v}}_2}\rangle }}{{\bf{v}}_2}
\end{array}
\]至此我們有了一組 orthogonal subset $\{{\bf v}_1,{\bf v}_2,{\bf v}_3\} $ for $W$
重複上述步驟 (by induction)我們可建構一組 orthogonal basis $T^* :=\{{\bf v}_1,{\bf v}_2,...,{\bf v}_m\}$ 最後我們對每一組 ${\bf v}_i$ 做正規化，定義
\[
{\bf w}_i := \frac{1}{||{\bf v}_i||} {\bf v}_i
\]則我們得到一組 orthonormal basis $T := \{{\bf w}_1,...,{\bf w}_m\}$
讀者可用以下幾個例子做練習
Example 1: 令 $S=\{[1\;\;2]^T, [-3\;\;4]^T\}$ 為 ordered basis for $ V:= \mathbb{R}^2$ 且其上內積為標準內積。
(a) 試利用 Gram-Schmidt process 找出 orthogonal basis
(b) 試利用 Gram-Schmidt process 找出 orthonormal basis
Example 2: 令 $V := P_3$ 且其上的內積定義為
\[
\langle p(t), q(t) \rangle := \int_0^1 p(t) q(t) dt
\] 現在令 $W$ 為 $P_3$ 子空間且基底為 $\{t,t^2\}$ 試求 orthonormal basis for $W$
[線性代數] Orthonormal Basis 與 Gram-Schmidt Process (0)
首先引入 一組向量彼此互為標準正交的定義
===================
Definition: Orthonormal Set
令 $V$ 為有限維度的內積空間 且 令 $S$ 為 $V$ 上的一組 集合滿足 $S =\{{\bf v}_1,..{\bf v}_n\}$ 。則我們稱 $S$ 為 orthonormal set 若
\[\left\langle {{{\bf{v}}_i},{{\bf{v}}_j}} \right\rangle = \left\{ \begin{array}{l}
0\begin{array}{*{20}{c}}
{}&{}
\end{array}i \ne j\\
1\begin{array}{*{20}{c}}
{}&{}
\end{array}i = j
\end{array} \right.\]其中 $\left\langle {{{\bf{v}}_i},{{\bf{v}}_j}} \right\rangle $ 為 $V$ 上的內積運算。
====================
Comment:
1. 給定一個向量空間我們如果有 orthonormal basis 則其上的任意向量將可以被非常容易地表示 (why?) 比如說我們考慮 $V:= \mathbb{R}^2$ 且 具備一組標準基底 $S:=\{{\bf s}_1, {\bf s}_2\} = \{[1 \;0]^T, [0\;1]^T\}$ 則此基底為 orthonormal 。現在若給訂任意向量 ${\bf v} := [100, -99]^T\in V$ 則此向量可以非常容易透過 基底 $S$ 做線性組合來組出 ${\bf v}$亦即
\[\underbrace {\left[ \begin{array}{l}
100\\
- 99
\end{array} \right]}_{ = {\bf{v}}} = 100\underbrace {\left[ \begin{array}{l}
1\\
0
\end{array} \right]}_{ = {{\bf{s}}_1}} + \left( { - 99} \right)\underbrace {\left[ \begin{array}{l}
0\\
1
\end{array} \right]}_{ = {{\bf{s}}_2}}\]
2. 上述觀點事實上到無窮維仍然成立，也就是說我們可以將正交的概念推廣到函數空間上面，並且說明什麼叫做兩個"函數" 彼此正交。以下我們看個無窮維函數空間的例子：
Example (Infinite-dimension Case)
令 $V:= C[0, 2 \pi]$ 且配備內積 \[
(f(t),g(t)) := \frac{1}{2 \pi}\int_0^{2 \pi} f(t) \bar{g} (t) dt
\]則 下列集合
\[
S :=\{f_n(t): f_n(t) := e^{jnt} =\cos nt + j \sin nt, \; n \in \mathbb{Z}\}
\]為 orthonormal set
Proof:
取 $f_n(t), g_m(t) \in S$ 觀察
\[\begin{array}{l}
({f_n}(t),{g_m}(t)) = \frac{1}{{2\pi }}\int_0^{2\pi } {{f_n}} (t){{\bar g}_m}(t)dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {{e^{jnt}}} {e^{ - jmt}}dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {{e^{j\left( {n - m} \right)t}}} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {\left[ {\cos \left( {\left( {n - m} \right)t} \right) + j\sin \left( {\left( {n - m} \right)t} \right)} \right]} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {\cos \left( {\left( {n - m} \right)t} \right)} dt + \frac{j}{{2\pi }}\int_0^{2\pi } {\sin \left( {\left( {n - m} \right)t} \right)} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \left\{ \begin{array}{l}
1,\begin{array}{*{20}{c}}
{}&{}
\end{array}n = m\\
0,\begin{array}{*{20}{c}}
{}&{}
\end{array}n \ne m
\end{array} \right. \;\;\;\;\;\;\;\;\; \square
\end{array}\]
=====================
Definition: Orthonormal Basis
若 $S$ 為 內積空間$V$上的一組有序基底，且 $S$ 為 orthnormal set 則我們稱此 $S$ 為 Orthnormal basis。
=====================
以下定理給出了 orthonormal basis 可以快速決定任意向量用該基底做線性組合的係數。
====================
Theorem:
令 $S = \{{\bf u}_1, {\bf u}_2,...,{\bf u}_n\}$ 為一組 orthonormal basis 對有限維度向量空間 $V$ 且令 ${\bf v} \in V$ 則
\[
{\bf v} = c_1{\bf u}_1 + c_2 {\bf u}_2 + ... + c_n {\bf u}_n
\]其中 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle \;\;\;\; \forall i=1,2,...,n$
====================
Comment:
上述定理中提及的 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle$ 在幾何意義上為 ${\bf v}$ 在 ${\bf u}_i$ 上的分量，且 $c_i$ 在數學上又稱為 Fourier Coefficient
以下我們給出證明：
Proof:
由於 ${\bf v} \in V$故此向量 ${\bf v}$ 可透過 $V$ 上的基底作唯一線性組合表示。
\[
{\bf v} = c_1{\bf u}_1 + c_2 {\bf u}_2 + ... + c_n {\bf u}_n
\]
故我們只需證明 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle \;\;\;\; \forall i=1,2,...,n$
現在固定任意 $i$ ，並且觀察
\[\begin{array}{l}
\left\langle {{\bf{v}},{{\bf{u}}_i}} \right\rangle = \left\langle {{c_1}{{\bf{u}}_1} + {c_2}{{\bf{u}}_2} + ... + {c_n}{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left\langle {{c_1}{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle + \left\langle {{c_2}{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle + ... + \left\langle {{c_i}{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle + ... + \left\langle {{c_n}{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_1}\left\langle {{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle + {c_2}\left\langle {{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle + ... + {c_i}\left\langle {{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle + ... + {c_n}\left\langle {{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_1}\underbrace {\left\langle {{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle }_{ = 0} + {c_2}\underbrace {\left\langle {{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle }_{ = 0} + ... + {c_i}\underbrace {\left\langle {{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle }_{ = 1} + ... + {c_n}\underbrace {\left\langle {{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle }_{ = 0}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_i}
\end{array}
\]注意到上述結果使用了 $S$ 基底為 orthonormal 故當 $i\neq j$ 時候， $\langle {\bf u_i}, {\bf u}_j\rangle =0$ $\square$
以下我們看個例子
Example 1:
令 $S = \{{\bf u}_1, {\bf u}_2\}$ 為 $\mathbb{R}^2$ 的一組基底，其中
\[{{\bf{u}}_1} = \frac{1}{\sqrt{2} }\left[ {\begin{array}{*{20}{c}}
1\\
1
\end{array}} \right];{{\bf{u}}_1} = \frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right]\]
(a) 確認 $S$ 為一組 orthonormal basis
(b) 令 ${\bf v} := [3\;\;4]^T$ 試決定其透過 $S$ 基底所構成的線性組合
Proof:
(a) 注意到 $\mathbb{R}^2$ 為內積空間，我們可在其上定義內積運算為
\[
\langle {\bf u}, {\bf v} \rangle := {\bf u}^T {\bf v}
\]
現在我們檢驗內積 $\langle {\bf u}_1, {\bf u}_2 \rangle$
\[\langle {{\bf{u}}_1},{{\bf{u}}_2}\rangle = \frac{1}{2} \left[ {\begin{array}{*{20}{c}}
1&1
\end{array}} \right] \left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right] = 0\]故此說明了 ${\bf u}_1, {\bf u}_2$ 為 orthogonal 接著我們驗證此基底具有 unit length
\[\left\| {{{\bf{u}}_1}} \right\| = \sqrt {\langle {{\bf{u}}_1},{{\bf{u}}_1}\rangle } = 1;\begin{array}{*{20}{c}}
{}&{}
\end{array}\left\| {{{\bf{u}}_2}} \right\| = \sqrt {\langle {{\bf{u}}_2},{{\bf{u}}_2}\rangle } = 1\]
綜上所述， $S$ 為 orthonormal basis。
(b) 現在令 ${\bf v}:= [3\;\;4]^T \in \mathbb{R}^2$ 故此向量可透過 $S$ 基底做線性組合表示
\[
{\bf v} = c_1 {\bf u}_1 + c_2 {\bf u}_2
\] 又因為 $S$ 為 orthonormal basis 故由前述定理可知 上式中的係數可透過內積求得
\[\begin{array}{l}
{c_1} = \langle {\bf{v}},{{\bf{u}}_1}\rangle = {{\bf{v}}^T}{{\bf{u}}_1} = \left[ {\begin{array}{*{20}{c}}
3&4
\end{array}} \right]\left( {\frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
1\\
1
\end{array}} \right]} \right) = \frac{7}{\sqrt{2}}\\
{c_2} = \langle {\bf{v}},{{\bf{u}}_2}\rangle = {{\bf{v}}^T}{{\bf{u}}_2} = \left[ {\begin{array}{*{20}{c}}
3&4
\end{array}} \right]\left( {\frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right]} \right) = \frac{1}{\sqrt{2}}\;\;\;\;\; \square
\end{array}\]
現在我們可以考慮以下問題：
若給定一個有限維度向量空間 $V$ 伴隨一組基底 $S$。那麼我們想進一步詢問是否可從這組基底 $S$ 中找出另外一組基底 $T$ 且 $T$ 基底元素彼此互相正交 且 單位長度為 $1$ ? 亦即我們想問是否可以從一組不是 orthonormal basis $S$ 來建構一組 orthonormal basis $T$ ，答案是肯定的，此構造方法稱為 Gram-Schmidt Process 我們之後會再行介紹。
===================
Definition: Orthonormal Set
令 $V$ 為有限維度的內積空間 且 令 $S$ 為 $V$ 上的一組 集合滿足 $S =\{{\bf v}_1,..{\bf v}_n\}$ 。則我們稱 $S$ 為 orthonormal set 若
\[\left\langle {{{\bf{v}}_i},{{\bf{v}}_j}} \right\rangle = \left\{ \begin{array}{l}
0\begin{array}{*{20}{c}}
{}&{}
\end{array}i \ne j\\
1\begin{array}{*{20}{c}}
{}&{}
\end{array}i = j
\end{array} \right.\]其中 $\left\langle {{{\bf{v}}_i},{{\bf{v}}_j}} \right\rangle $ 為 $V$ 上的內積運算。
====================
Comment:
1. 給定一個向量空間我們如果有 orthonormal basis 則其上的任意向量將可以被非常容易地表示 (why?) 比如說我們考慮 $V:= \mathbb{R}^2$ 且 具備一組標準基底 $S:=\{{\bf s}_1, {\bf s}_2\} = \{[1 \;0]^T, [0\;1]^T\}$ 則此基底為 orthonormal 。現在若給訂任意向量 ${\bf v} := [100, -99]^T\in V$ 則此向量可以非常容易透過 基底 $S$ 做線性組合來組出 ${\bf v}$亦即
\[\underbrace {\left[ \begin{array}{l}
100\\
- 99
\end{array} \right]}_{ = {\bf{v}}} = 100\underbrace {\left[ \begin{array}{l}
1\\
0
\end{array} \right]}_{ = {{\bf{s}}_1}} + \left( { - 99} \right)\underbrace {\left[ \begin{array}{l}
0\\
1
\end{array} \right]}_{ = {{\bf{s}}_2}}\]
2. 上述觀點事實上到無窮維仍然成立，也就是說我們可以將正交的概念推廣到函數空間上面，並且說明什麼叫做兩個"函數" 彼此正交。以下我們看個無窮維函數空間的例子：
Example (Infinite-dimension Case)
令 $V:= C[0, 2 \pi]$ 且配備內積 \[
(f(t),g(t)) := \frac{1}{2 \pi}\int_0^{2 \pi} f(t) \bar{g} (t) dt
\]則 下列集合
\[
S :=\{f_n(t): f_n(t) := e^{jnt} =\cos nt + j \sin nt, \; n \in \mathbb{Z}\}
\]為 orthonormal set
Proof:
取 $f_n(t), g_m(t) \in S$ 觀察
\[\begin{array}{l}
({f_n}(t),{g_m}(t)) = \frac{1}{{2\pi }}\int_0^{2\pi } {{f_n}} (t){{\bar g}_m}(t)dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {{e^{jnt}}} {e^{ - jmt}}dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {{e^{j\left( {n - m} \right)t}}} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {\left[ {\cos \left( {\left( {n - m} \right)t} \right) + j\sin \left( {\left( {n - m} \right)t} \right)} \right]} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_0^{2\pi } {\cos \left( {\left( {n - m} \right)t} \right)} dt + \frac{j}{{2\pi }}\int_0^{2\pi } {\sin \left( {\left( {n - m} \right)t} \right)} dt\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}
\end{array} = \left\{ \begin{array}{l}
1,\begin{array}{*{20}{c}}
{}&{}
\end{array}n = m\\
0,\begin{array}{*{20}{c}}
{}&{}
\end{array}n \ne m
\end{array} \right. \;\;\;\;\;\;\;\;\; \square
\end{array}\]
=====================
Definition: Orthonormal Basis
若 $S$ 為 內積空間$V$上的一組有序基底，且 $S$ 為 orthnormal set 則我們稱此 $S$ 為 Orthnormal basis。
=====================
以下定理給出了 orthonormal basis 可以快速決定任意向量用該基底做線性組合的係數。
====================
Theorem:
令 $S = \{{\bf u}_1, {\bf u}_2,...,{\bf u}_n\}$ 為一組 orthonormal basis 對有限維度向量空間 $V$ 且令 ${\bf v} \in V$ 則
\[
{\bf v} = c_1{\bf u}_1 + c_2 {\bf u}_2 + ... + c_n {\bf u}_n
\]其中 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle \;\;\;\; \forall i=1,2,...,n$
====================
Comment:
上述定理中提及的 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle$ 在幾何意義上為 ${\bf v}$ 在 ${\bf u}_i$ 上的分量，且 $c_i$ 在數學上又稱為 Fourier Coefficient
以下我們給出證明：
Proof:
由於 ${\bf v} \in V$故此向量 ${\bf v}$ 可透過 $V$ 上的基底作唯一線性組合表示。
\[
{\bf v} = c_1{\bf u}_1 + c_2 {\bf u}_2 + ... + c_n {\bf u}_n
\]
故我們只需證明 $c_i = \left \langle {\bf v}, {\bf u}_i \right \rangle \;\;\;\; \forall i=1,2,...,n$
\[\begin{array}{l}
\left\langle {{\bf{v}},{{\bf{u}}_i}} \right\rangle = \left\langle {{c_1}{{\bf{u}}_1} + {c_2}{{\bf{u}}_2} + ... + {c_n}{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left\langle {{c_1}{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle + \left\langle {{c_2}{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle + ... + \left\langle {{c_i}{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle + ... + \left\langle {{c_n}{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_1}\left\langle {{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle + {c_2}\left\langle {{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle + ... + {c_i}\left\langle {{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle + ... + {c_n}\left\langle {{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle \\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_1}\underbrace {\left\langle {{{\bf{u}}_1},{{\bf{u}}_i}} \right\rangle }_{ = 0} + {c_2}\underbrace {\left\langle {{{\bf{u}}_2},{{\bf{u}}_i}} \right\rangle }_{ = 0} + ... + {c_i}\underbrace {\left\langle {{{\bf{u}}_i},{{\bf{u}}_i}} \right\rangle }_{ = 1} + ... + {c_n}\underbrace {\left\langle {{{\bf{u}}_n},{{\bf{u}}_i}} \right\rangle }_{ = 0}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {c_i}
\end{array}
\]注意到上述結果使用了 $S$ 基底為 orthonormal 故當 $i\neq j$ 時候， $\langle {\bf u_i}, {\bf u}_j\rangle =0$ $\square$
以下我們看個例子
Example 1:
令 $S = \{{\bf u}_1, {\bf u}_2\}$ 為 $\mathbb{R}^2$ 的一組基底，其中
\[{{\bf{u}}_1} = \frac{1}{\sqrt{2} }\left[ {\begin{array}{*{20}{c}}
1\\
1
\end{array}} \right];{{\bf{u}}_1} = \frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right]\]
(a) 確認 $S$ 為一組 orthonormal basis
(b) 令 ${\bf v} := [3\;\;4]^T$ 試決定其透過 $S$ 基底所構成的線性組合
Proof:
(a) 注意到 $\mathbb{R}^2$ 為內積空間，我們可在其上定義內積運算為
\[
\langle {\bf u}, {\bf v} \rangle := {\bf u}^T {\bf v}
\]
現在我們檢驗內積 $\langle {\bf u}_1, {\bf u}_2 \rangle$
\[\langle {{\bf{u}}_1},{{\bf{u}}_2}\rangle = \frac{1}{2} \left[ {\begin{array}{*{20}{c}}
1&1
\end{array}} \right] \left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right] = 0\]故此說明了 ${\bf u}_1, {\bf u}_2$ 為 orthogonal 接著我們驗證此基底具有 unit length
\[\left\| {{{\bf{u}}_1}} \right\| = \sqrt {\langle {{\bf{u}}_1},{{\bf{u}}_1}\rangle } = 1;\begin{array}{*{20}{c}}
{}&{}
\end{array}\left\| {{{\bf{u}}_2}} \right\| = \sqrt {\langle {{\bf{u}}_2},{{\bf{u}}_2}\rangle } = 1\]
綜上所述， $S$ 為 orthonormal basis。
(b) 現在令 ${\bf v}:= [3\;\;4]^T \in \mathbb{R}^2$ 故此向量可透過 $S$ 基底做線性組合表示
\[
{\bf v} = c_1 {\bf u}_1 + c_2 {\bf u}_2
\] 又因為 $S$ 為 orthonormal basis 故由前述定理可知 上式中的係數可透過內積求得
\[\begin{array}{l}
{c_1} = \langle {\bf{v}},{{\bf{u}}_1}\rangle = {{\bf{v}}^T}{{\bf{u}}_1} = \left[ {\begin{array}{*{20}{c}}
3&4
\end{array}} \right]\left( {\frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
1\\
1
\end{array}} \right]} \right) = \frac{7}{\sqrt{2}}\\
{c_2} = \langle {\bf{v}},{{\bf{u}}_2}\rangle = {{\bf{v}}^T}{{\bf{u}}_2} = \left[ {\begin{array}{*{20}{c}}
3&4
\end{array}} \right]\left( {\frac{1}{\sqrt{2}}\left[ {\begin{array}{*{20}{c}}
{ - 1}\\
1
\end{array}} \right]} \right) = \frac{1}{\sqrt{2}}\;\;\;\;\; \square
\end{array}\]
現在我們可以考慮以下問題：
若給定一個有限維度向量空間 $V$ 伴隨一組基底 $S$。那麼我們想進一步詢問是否可從這組基底 $S$ 中找出另外一組基底 $T$ 且 $T$ 基底元素彼此互相正交 且 單位長度為 $1$ ? 亦即我們想問是否可以從一組不是 orthonormal basis $S$ 來建構一組 orthonormal basis $T$ ，答案是肯定的，此構造方法稱為 Gram-Schmidt Process 我們之後會再行介紹。
[數學分析] 三角多項式 與 三角級數 (1)
三角多項式表示一個函數可以透過 多個三角函數 方式表示，具體定義如下。
============================
Definition: Trigonometric polynomial
我們說 $f(x)$ 為一個 三角多項式( trigonometric polynomial) 若 $f$ 具有下列形式：
\[
f(x) := \sum_{n=0}^N a_n \cos(nx) + b_n \sin (nx) \ \ \ \ \ (*)
\]其中 $a_n, b_n \in \mathbb{C}$ 且 $x \in \mathbb{R}$。；或者上式可等價寫為 複數形式
\[
f(x) := \sum_{n=-N}^N c_n e^{i n x}
\]對任意 $c_n \in \mathbb{C}$ 與 $x \in \mathbb{R}$
============================
Proof: 亦即我們要證明 $f(x+2 \pi) = f(x)$，故
\[\begin{array}{l}
f(x + 2\pi ): = \sum\limits_{n = - N}^N {{c_n}} {e^{in\left( {x + 2\pi } \right)}} = \sum\limits_{n = - N}^N {{c_n}} {e^{in\left( x \right)}}{e^{in\left( {2\pi } \right)}}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \sum\limits_{n = - N}^N {{c_n}} {e^{in\left( x \right)}}\underbrace {\left[ {\cos n2\pi + i\sin n2\pi } \right]}_{ = 1} = \sum\limits_{n = - N}^N {{c_n}} {e^{in\left( x \right)}} = f\left( x \right) \ \ \ \ \square
\end{array}\]
\[\frac{1}{{2\pi }}\int_{ - \pi }^\pi {{e^{imx}}} {e^{ - inx}}dx = \left\{ \begin{array}{l}
0,\begin{array}{*{20}{c}}
{}
\end{array}n \ne m\\
1,\begin{array}{*{20}{c}}
{}
\end{array}n = m
\end{array} \right.\]==========================
\[
c_n = \frac{1}{2 \pi}\int_{-\pi}^\pi f(x) e^{-inx}dx
\]==========================
Proof:
\[\begin{array}{*{20}{l}}
{\frac{1}{{2\pi }}\int_{ - \pi }^\pi f (x){e^{ - inx}}dx = \frac{1}{{2\pi }}\int_{ - \pi }^\pi {\sum\limits_{m = - N}^N {{c_m}} {e^{imx}}} {e^{ - inx}}dx}\\
{\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\sum\limits_{m = - N}^N {{c_m}} \int_{ - \pi }^\pi {{e^{imx}}} {e^{ - inx}}dx}
\end{array}\]利用 FACT 2 可得
\[\frac{1}{{2\pi }}\int_{ - \pi }^\pi f (x){e^{ - inx}}dx = \frac{1}{{2\pi }}\sum\limits_{m = - N}^N {{c_m}} \underbrace {\int_{ - \pi }^\pi {{e^{imx}}} {e^{ - inx}}dx}_{ = 1\begin{array}{*{20}{c}}
{}
\end{array}if\begin{array}{*{20}{c}}
{}
\end{array}n = m} = {c_n} \ \ \ \ \square\]
===================
FACT 4: Trigonometric polynomial $f$ 為 實數函數 若且唯若 $c_n^* = c_{-n}$ ( 其中$( \cdot )^*$) 表示 complex conjugate。
===================
Proof:
$(\Rightarrow)$ 假設 Trigonometric polynomial $f$ 為 Real-valued 函數，我們要證明 $c_n^* = c_{-n}$ 。故由於 $f$ 為 Real-valued 函數，我們有 $f^* = f$；亦即
\[\begin{array}{l}
c_n^* = {\left( {\frac{1}{{2\pi }}\int_{ - \pi }^\pi f (x){e^{ - inx}}dx} \right)^*} = \frac{1}{{2\pi }}\int_{ - \pi }^\pi {{f^*}} (x){e^{inx}}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_{ - \pi }^\pi f (x){e^{inx}}dx = \frac{1}{{2\pi }}\int_{ - \pi }^\pi {\sum\limits_{m = - N}^N {{c_m}} {e^{imx}}} {e^{inx}}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\sum\limits_{m = - N}^N {{c_m}} \int_{ - \pi }^\pi {{e^{imx}}} {e^{inx}}dx = {c_{ - n}}.
\end{array}\]
$(\Leftarrow)$ 假設 $c_n^* = c_{-n}$ ，我們要證 Trigonometric polynomial $f$ 為 Real-valued 函數。亦即要證明 $f^* = f$。現在觀察
\[{f^*}(x) = {\left( {\sum\limits_{n = - N}^N {{c_n}} {e^{inx}}} \right)^*} = \sum\limits_{n = - N}^N {c_n^*} {e^{ - inx}} = \sum\limits_{n = - N}^N {c_{ - n}^{}} {e^{ - inx}}\]令 $m:=-n$ 可得
\[{f^*}(x) = \sum\limits_{n = - N}^N {c_n^*} {e^{ - inx}} = \sum\limits_{m = - N}^N {c_m^{}} {e^{imx}} = f\left( x \right) .\ \ \ \ \square
\]
接著我們可以定義 Trigonometric Series 如下：
========================
Definition: Trigonometric Series
我們說 Trigonometric Series 為具有下列形式的無窮級數
\[
\sum_{n=-\infty}^\infty c_n e^{-inx}
\]且由先前 Trigonometric polynomial 的係數定法，我們可以一樣定義對一個週期函數 $f$ 的 $m$-th Fourier Coefficient ：
\[
c_m := \frac{1}{2 \pi} \int_{-\pi}^\pi f(x) e^{-imx}dx
\]========================
========================
Definition: Fourier Series
Fourier Series 為一個 Trigonometric Series 且其係數為 Fourier coefficient of $f$，我們將 Fourier Series 記做
\[
f \sim \sum_{n = -\infty}^\infty c_n e^{inx}
\]========================
注意：上述並非等號；單純表示 $c_n$ 是來自 $f$ 的 Fourier Series coefficient。
故我們想問 "何時可以讓 $f$ 與 Fourier Series 等號成立? " 或者說 基於怎樣的測量基準之下，此兩者可以被適當的逼近?
我們將回答此問題於更廣義的 Fourier Series 之上，在後面的文章會在做介紹。
============================
Definition: Trigonometric polynomial
我們說 $f(x)$ 為一個 三角多項式( trigonometric polynomial) 若 $f$ 具有下列形式：
\[
f(x) := \sum_{n=0}^N a_n \cos(nx) + b_n \sin (nx) \ \ \ \ \ (*)
\]其中 $a_n, b_n \in \mathbb{C}$ 且 $x \in \mathbb{R}$。；或者上式可等價寫為 複數形式
\[
f(x) := \sum_{n=-N}^N c_n e^{i n x}
\]對任意 $c_n \in \mathbb{C}$ 與 $x \in \mathbb{R}$
============================
Comment:
注意到對於 式子 $(*)$ 可改寫為
\[f(x) = \sum\limits_{n = 0}^N {{a_n}} \cos (nx) + {b_n}\sin (nx) = {a_0} + \sum\limits_{n = 1}^N {{a_n}} \cos (nx) + {b_n}\sin (nx)\]
==========================
FACT 1: Trigonometric polynomial $f$ 為週期函數且週期為 $2 \pi$。
==========================
\[\begin{array}{l}
f(x + 2\pi ): = \sum\limits_{n = - N}^N {{c_n}} {e^{in\left( {x + 2\pi } \right)}} = \sum\limits_{n = - N}^N {{c_n}} {e^{in\left( x \right)}}{e^{in\left( {2\pi } \right)}}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \sum\limits_{n = - N}^N {{c_n}} {e^{in\left( x \right)}}\underbrace {\left[ {\cos n2\pi + i\sin n2\pi } \right]}_{ = 1} = \sum\limits_{n = - N}^N {{c_n}} {e^{in\left( x \right)}} = f\left( x \right) \ \ \ \ \square
\end{array}\]
==========================
FACT 2: 下列等式成立：\[\frac{1}{{2\pi }}\int_{ - \pi }^\pi {{e^{imx}}} {e^{ - inx}}dx = \left\{ \begin{array}{l}
0,\begin{array}{*{20}{c}}
{}
\end{array}n \ne m\\
1,\begin{array}{*{20}{c}}
{}
\end{array}n = m
\end{array} \right.\]==========================
Proof: omitted.
==========================
FACT 3: Trigonometric polynomial $f$ 的係數 $c_n$ 可由下列積分決定：\[
c_n = \frac{1}{2 \pi}\int_{-\pi}^\pi f(x) e^{-inx}dx
\]==========================
Proof:
\[\begin{array}{*{20}{l}}
{\frac{1}{{2\pi }}\int_{ - \pi }^\pi f (x){e^{ - inx}}dx = \frac{1}{{2\pi }}\int_{ - \pi }^\pi {\sum\limits_{m = - N}^N {{c_m}} {e^{imx}}} {e^{ - inx}}dx}\\
{\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\sum\limits_{m = - N}^N {{c_m}} \int_{ - \pi }^\pi {{e^{imx}}} {e^{ - inx}}dx}
\end{array}\]利用 FACT 2 可得
\[\frac{1}{{2\pi }}\int_{ - \pi }^\pi f (x){e^{ - inx}}dx = \frac{1}{{2\pi }}\sum\limits_{m = - N}^N {{c_m}} \underbrace {\int_{ - \pi }^\pi {{e^{imx}}} {e^{ - inx}}dx}_{ = 1\begin{array}{*{20}{c}}
{}
\end{array}if\begin{array}{*{20}{c}}
{}
\end{array}n = m} = {c_n} \ \ \ \ \square\]
===================
FACT 4: Trigonometric polynomial $f$ 為 實數函數 若且唯若 $c_n^* = c_{-n}$ ( 其中$( \cdot )^*$) 表示 complex conjugate。
===================
$(\Rightarrow)$ 假設 Trigonometric polynomial $f$ 為 Real-valued 函數，我們要證明 $c_n^* = c_{-n}$ 。故由於 $f$ 為 Real-valued 函數，我們有 $f^* = f$；亦即
\[\begin{array}{l}
c_n^* = {\left( {\frac{1}{{2\pi }}\int_{ - \pi }^\pi f (x){e^{ - inx}}dx} \right)^*} = \frac{1}{{2\pi }}\int_{ - \pi }^\pi {{f^*}} (x){e^{inx}}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\int_{ - \pi }^\pi f (x){e^{inx}}dx = \frac{1}{{2\pi }}\int_{ - \pi }^\pi {\sum\limits_{m = - N}^N {{c_m}} {e^{imx}}} {e^{inx}}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}&{}&{}&{}&{}
\end{array} = \frac{1}{{2\pi }}\sum\limits_{m = - N}^N {{c_m}} \int_{ - \pi }^\pi {{e^{imx}}} {e^{inx}}dx = {c_{ - n}}.
\end{array}\]
$(\Leftarrow)$ 假設 $c_n^* = c_{-n}$ ，我們要證 Trigonometric polynomial $f$ 為 Real-valued 函數。亦即要證明 $f^* = f$。現在觀察
\[{f^*}(x) = {\left( {\sum\limits_{n = - N}^N {{c_n}} {e^{inx}}} \right)^*} = \sum\limits_{n = - N}^N {c_n^*} {e^{ - inx}} = \sum\limits_{n = - N}^N {c_{ - n}^{}} {e^{ - inx}}\]令 $m:=-n$ 可得
\[{f^*}(x) = \sum\limits_{n = - N}^N {c_n^*} {e^{ - inx}} = \sum\limits_{m = - N}^N {c_m^{}} {e^{imx}} = f\left( x \right) .\ \ \ \ \square
\]
========================
Definition: Trigonometric Series
我們說 Trigonometric Series 為具有下列形式的無窮級數
\[
\sum_{n=-\infty}^\infty c_n e^{-inx}
\]且由先前 Trigonometric polynomial 的係數定法，我們可以一樣定義對一個週期函數 $f$ 的 $m$-th Fourier Coefficient ：
\[
c_m := \frac{1}{2 \pi} \int_{-\pi}^\pi f(x) e^{-imx}dx
\]========================
Definition: Fourier Series
Fourier Series 為一個 Trigonometric Series 且其係數為 Fourier coefficient of $f$，我們將 Fourier Series 記做
\[
f \sim \sum_{n = -\infty}^\infty c_n e^{inx}
\]========================
注意：上述並非等號；單純表示 $c_n$ 是來自 $f$ 的 Fourier Series coefficient。
故我們想問 "何時可以讓 $f$ 與 Fourier Series 等號成立? " 或者說 基於怎樣的測量基準之下，此兩者可以被適當的逼近?
我們將回答此問題於更廣義的 Fourier Series 之上，在後面的文章會在做介紹。
[數學分析] 三角多項式 與 三角級數 (2)- Generalized Fourier Series
現在我們定義廣義 Fourier Series ：
=================
Definition: (Orthogonal System of Functions)
令 $\{\phi_n\}$, $n \in \mathbb{N}$ 為在 $[a,b]$ 上 的 Complex-valued 函數 sequence 且滿足下列積分
\[
\int_a^b \phi_n(x) \phi_m^*(x) dx =0, \;\; \text{ if $n \neq m$}
\]那麼我們稱 $\{\phi_n\}$ 為在 $[a,b]$ 上 orthogonal 或稱 (orthogonal system of functions on $[a,b]$) 。除此之外，若積分
\[
\int_a^b \phi_n(x) \phi_n^*(x) dx =1
\]我們稱 $\{\phi_n\}$ 為在 $[a,b]$ 上 orthonormal 或稱 (orthonormal system of functions on $[a,b]$) 。
=====================
Comments:
一般而言，若我們取 $\{\phi_n\}$ $n \in \mathbb{N}$ 為在 $[0, 2\pi]$ 上 的 Complex-valued 函數 sequence 且滿足 $\phi_n(x):= exp(inx)$ 則讀者可自行驗證此 函數 sequence 為 orthogonal
=====================
Definition: (n-th Fourier Coefficient of $f$)
若 $\{\phi_n\}$ 為 orthonormal on $[a,b]$ 且 對任意 $n \in \mathbb{N}$，
\[
c_n:=\int_a^b f(x) \phi_n^*(x) dx
\]我們稱 $c_n$ 為 $n$-th Fourier coefficient of $f$ (relative to $\{\phi_n\}$)
=====================
Definition: Generalized Fourier Series
Generalized Fourier Series of $f$ (relative to $\{\phi_n\}$) 定義為
\[
f(x) \sim \sum_{n=1}^\infty c_n \phi_n(x)
\]其中 $c_n=\int_a^b f(x) \phi_n^*(x) dx $。
====================
Theorem: Bessel's inequality
若 $\{\phi_n\}$ 為 orthnormal on $[a,b]$ 且若 $f(x) \sim \sum_{n=1}^\infty c_n \phi_n(x)$ 則 \[
\sum_{n=1}^\infty |c_n|^2 \le \int_a^b |f(x)|^2 dx
\]
====================
Theorem: Best Approximation of Fourier Series
令 $\{\phi_n\}$ 為在 $[a,b]$ 上 的 Complex-valued 函數 sequence ，且 $\phi_n$ 為 orthogonal。現在定義 $n$-th partial sum of Fourier Series of $f$ 如下
\[
s_n (x):= \sum_{m=1}^n c_m \phi_m(x)
\]且令 $t_n$ 為任意 series 如下
\[
t_n(x) := \sum_{m=1}^n \gamma_m \phi_m(x)
\]其中 $\gamma_n \in \mathbb{C}$ 則
\[
\int_a^b |f (x)- s_n(x)|^2 dx \le \int_a^b |f(x) - t_n(x)|^2 dx
\] 且 上式等式成立 若且為若 $\gamma_n = c_n$ 對任意 $n$。
==================
Proof:
令
\[
s_n (x):= \sum_{m=1}^n c_m \phi_m(x);\;\;\; t_n(x) := \sum_{m=1}^n \gamma_m \phi_m(x)
\]
我們首先證明下列不等式成立
\[
\int_a^b |f (x)- s_n(x)|^2 dx \le \int_a^b |f(x) - t_n(x)|^2 dx
\] 首先觀察
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \int_a^b {\left( {f(x) - {t_n}(x)} \right){{\left( {f(x) - {t_n}(x)} \right)}^*}} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\left( {f(x){f^*}(x) - f(x){t_n}^*(x) - {t_n}(x){f^*}(x) + {t_n}(x){t_n}^*(x)} \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx}_{term1} - \underbrace {\int_a^b {f(x){t_n}^*(x)} dx}_{term2} - \underbrace {\int_a^b {{t_n}(x){f^*}(x)} dx}_{term3} + \underbrace {\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx}_{term4} \ \ \ \ \ \ (*)
\end{array}\]接著對上式逐項觀察，首先檢驗 term 2：
\[\begin{array}{l}
\int_a^b {f(x){t_n}^*(x)} dx = {\int_a^b {f(x)\left[ {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)} \right]} ^*}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {f(x)\sum\limits_{m = 1}^n {{\gamma _m}^*} {\phi _m}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}^*} \underbrace {\int_a^b {f(x){\phi _m}^*(x)} dx}_{ = {c_m}}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}^*} {c_m}
\end{array}\]
接著我們檢驗 term 3:
\[\begin{array}{l}
\int_a^b {{t_n}(x){f^*}(x)} dx = \int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x){f^*}(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} \underbrace {\int_a^b {{f^*}(x){\phi _m}(x)} dx}_{ = c_m^*}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} c_m^*
\end{array}\]
最後檢驗 term 4：
\[\begin{array}{l}
\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx = \int_a^b {{t_n}(x)t_n^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = {\int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)\left[ {\sum\limits_{k = 1}^n {{\gamma _k}} {\phi _k}(x)} \right]} ^*}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)\sum\limits_{k = 1}^n {{\gamma _k}^*} {\phi _k}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} \sum\limits_{k = 1}^n {{\gamma _k}^*} \underbrace {\int_a^b {{\phi _m}(x){\phi _k}^*(x)} dx}_{ = 1\begin{array}{*{20}{c}}
{}
\end{array}if\begin{array}{*{20}{c}}
{}
\end{array}m = k}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m= 1}^n {{\gamma _m}{\gamma _m}^*} = \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}}
\end{array}\]
故現在將上述結果 帶回 $(*)$ 可得
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx}_{term1} - \underbrace {\int_a^b {f(x){t_n}^*(x)} dx}_{term2} - \underbrace {\int_a^b {{t_n}(x){f^*}(x)} dx}_{term3} + \underbrace {\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx}_{term4}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {\gamma _m^*c_m^{}} - \sum\limits_{m = 1}^n {{\gamma _m}c_m^*} + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}} \ \ \ \ (\star)
\end{array}\]
注意到下列 FACT：
\[\begin{array}{l}
\sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} = \sum\limits_{m = 1}^n {\left( {{c_n} - {\gamma _m}} \right){{\left( {{c_n} - {\gamma _m}} \right)}^*}} = \sum\limits_{m = 1}^n {\left( {{c_n}{c_n}^* - {c_n}{\gamma _m}^* - {\gamma _m}{c_n}^* + {\gamma _m}{\gamma _m}^*} \right)} \\
\Rightarrow \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} = \sum\limits_{m = 1}^n {{c_n}{c_n}^*} - \sum\limits_{m = 1}^n {{c_n}{\gamma _m}^*} - \sum\limits_{m = 1}^n {{\gamma _m}{c_n}^*} + \sum\limits_{m = 1}^n {{\gamma _m}{\gamma _m}^*} \\
\Rightarrow \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}} = - \sum\limits_{m = 1}^n {{c_n}{\gamma _m}^*} - \sum\limits_{m = 1}^n {{\gamma _m}{c_n}^*} + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}}
\end{array}\]與
\[\begin{array}{l}
\int_a^b | f(x) - {s_n}(x){|^2}dx = \int_a^b {\left( {f(x) - {s_n}(x)} \right){{\left( {f(x) - {s_n}(x)} \right)}^*}} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\left( {f(x){f^*}(x) - f(x){s_n}^*(x) - {s_n}(x){f^*}(x) + {s_n}(x){s_n}^*(x)} \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {f(x){f^*}(x)} dx - \int_a^b {f(x){s_n}^*(x)} dx - \int_a^b {{s_n}(x){f^*}(x)} dx + \int_a^b {{s_n}(x){s_n}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \int_a^b {\left( {f(x)\sum\limits_{m = 1}^n {{c_m}^*{\phi _m}^*\left( x \right)} } \right)} dx - \int_a^b {\left( {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} {f^*}(x)} \right)} dx + \int_a^b {\left( {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} \sum\limits_{k = 1}^n {{c_k}^*{\phi _k}^*\left( x \right)} } \right)} dx\\
{\rm{since}}\begin{array}{*{20}{c}}
{}
\end{array}{s_n}(x): = \sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} \Rightarrow s_n^*(x): = {\left[ {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} } \right]^*} = \sum\limits_{m = 1}^n {{c_m}^*{\phi _m}^*\left( x \right)} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} \int_a^b {\left( {f(x){\phi _m}^*\left( x \right)} \right)} dx - \sum\limits_{m = 1}^n {{c_m}} \int_a^b {{f^*}(x){\phi _m}\left( x \right)} dx + \sum\limits_{m = 1}^n {{c_m}} \sum\limits_{k = 1}^n {{c_k}^*} \int_a^b {{\phi _m}\left( x \right){\phi _k}^*\left( x \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} {c_m} - \sum\limits_{m = 1}^n {{c_m}} c_m^* + \sum\limits_{m = 1}^n {{c_m}{c_m}^*} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} {c_m}\\
\Rightarrow \int_a^b | f(x) - {s_n}(x){|^2}dx = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{{\left| {{c_m}} \right|}^2}}
\end{array}\]
故 $\star$ 可進一步改寫
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {\gamma _m^*c_m^{}} - \sum\limits_{m = 1}^n {{\gamma _m}c_m^*} + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx + \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}} }_{ = \int_a^b | f(x) - {s_n}(x){|^2}dx} + \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b | f(x) - {s_n}(x){|^2}dx + \underbrace {\sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} }_{ \ge 0}\\
\Rightarrow \int_a^b | f(x) - {t_n}(x){|^2}dx \ge \int_a^b | f(x) - {s_n}(x){|^2}dx\\
\end{array}\]
且若 $c_m = \gamma_m$ 則等式成立。 $\square$
=================
Definition: (Orthogonal System of Functions)
令 $\{\phi_n\}$, $n \in \mathbb{N}$ 為在 $[a,b]$ 上 的 Complex-valued 函數 sequence 且滿足下列積分
\[
\int_a^b \phi_n(x) \phi_m^*(x) dx =0, \;\; \text{ if $n \neq m$}
\]那麼我們稱 $\{\phi_n\}$ 為在 $[a,b]$ 上 orthogonal 或稱 (orthogonal system of functions on $[a,b]$) 。除此之外，若積分
\[
\int_a^b \phi_n(x) \phi_n^*(x) dx =1
\]我們稱 $\{\phi_n\}$ 為在 $[a,b]$ 上 orthonormal 或稱 (orthonormal system of functions on $[a,b]$) 。
=====================
Comments:
一般而言，若我們取 $\{\phi_n\}$ $n \in \mathbb{N}$ 為在 $[0, 2\pi]$ 上 的 Complex-valued 函數 sequence 且滿足 $\phi_n(x):= exp(inx)$ 則讀者可自行驗證此 函數 sequence 為 orthogonal
=====================
Definition: (n-th Fourier Coefficient of $f$)
若 $\{\phi_n\}$ 為 orthonormal on $[a,b]$ 且 對任意 $n \in \mathbb{N}$，
\[
c_n:=\int_a^b f(x) \phi_n^*(x) dx
\]我們稱 $c_n$ 為 $n$-th Fourier coefficient of $f$ (relative to $\{\phi_n\}$)
=====================
上述 $^*$ 為 complex conjugate。
Definition: Generalized Fourier Series
Generalized Fourier Series of $f$ (relative to $\{\phi_n\}$) 定義為
\[
f(x) \sim \sum_{n=1}^\infty c_n \phi_n(x)
\]其中 $c_n=\int_a^b f(x) \phi_n^*(x) dx $。
====================
Theorem: Bessel's inequality
若 $\{\phi_n\}$ 為 orthnormal on $[a,b]$ 且若 $f(x) \sim \sum_{n=1}^\infty c_n \phi_n(x)$ 則 \[
\sum_{n=1}^\infty |c_n|^2 \le \int_a^b |f(x)|^2 dx
\]
====================
Theorem: Best Approximation of Fourier Series
令 $\{\phi_n\}$ 為在 $[a,b]$ 上 的 Complex-valued 函數 sequence ，且 $\phi_n$ 為 orthogonal。現在定義 $n$-th partial sum of Fourier Series of $f$ 如下
\[
s_n (x):= \sum_{m=1}^n c_m \phi_m(x)
\]且令 $t_n$ 為任意 series 如下
\[
t_n(x) := \sum_{m=1}^n \gamma_m \phi_m(x)
\]其中 $\gamma_n \in \mathbb{C}$ 則
\[
\int_a^b |f (x)- s_n(x)|^2 dx \le \int_a^b |f(x) - t_n(x)|^2 dx
\] 且 上式等式成立 若且為若 $\gamma_n = c_n$ 對任意 $n$。
==================
令
\[
s_n (x):= \sum_{m=1}^n c_m \phi_m(x);\;\;\; t_n(x) := \sum_{m=1}^n \gamma_m \phi_m(x)
\]
我們首先證明下列不等式成立
\[
\int_a^b |f (x)- s_n(x)|^2 dx \le \int_a^b |f(x) - t_n(x)|^2 dx
\] 首先觀察
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \int_a^b {\left( {f(x) - {t_n}(x)} \right){{\left( {f(x) - {t_n}(x)} \right)}^*}} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\left( {f(x){f^*}(x) - f(x){t_n}^*(x) - {t_n}(x){f^*}(x) + {t_n}(x){t_n}^*(x)} \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx}_{term1} - \underbrace {\int_a^b {f(x){t_n}^*(x)} dx}_{term2} - \underbrace {\int_a^b {{t_n}(x){f^*}(x)} dx}_{term3} + \underbrace {\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx}_{term4} \ \ \ \ \ \ (*)
\end{array}\]接著對上式逐項觀察，首先檢驗 term 2：
\[\begin{array}{l}
\int_a^b {f(x){t_n}^*(x)} dx = {\int_a^b {f(x)\left[ {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)} \right]} ^*}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {f(x)\sum\limits_{m = 1}^n {{\gamma _m}^*} {\phi _m}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}^*} \underbrace {\int_a^b {f(x){\phi _m}^*(x)} dx}_{ = {c_m}}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}^*} {c_m}
\end{array}\]
接著我們檢驗 term 3:
\[\begin{array}{l}
\int_a^b {{t_n}(x){f^*}(x)} dx = \int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x){f^*}(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} \underbrace {\int_a^b {{f^*}(x){\phi _m}(x)} dx}_{ = c_m^*}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} c_m^*
\end{array}\]
最後檢驗 term 4：
\[\begin{array}{l}
\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx = \int_a^b {{t_n}(x)t_n^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = {\int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)\left[ {\sum\limits_{k = 1}^n {{\gamma _k}} {\phi _k}(x)} \right]} ^*}dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\sum\limits_{m = 1}^n {{\gamma _m}} {\phi _m}(x)\sum\limits_{k = 1}^n {{\gamma _k}^*} {\phi _k}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m = 1}^n {{\gamma _m}} \sum\limits_{k = 1}^n {{\gamma _k}^*} \underbrace {\int_a^b {{\phi _m}(x){\phi _k}^*(x)} dx}_{ = 1\begin{array}{*{20}{c}}
{}
\end{array}if\begin{array}{*{20}{c}}
{}
\end{array}m = k}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \sum\limits_{m= 1}^n {{\gamma _m}{\gamma _m}^*} = \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}}
\end{array}\]
故現在將上述結果 帶回 $(*)$ 可得
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx}_{term1} - \underbrace {\int_a^b {f(x){t_n}^*(x)} dx}_{term2} - \underbrace {\int_a^b {{t_n}(x){f^*}(x)} dx}_{term3} + \underbrace {\int_a^b {{{\left| {{t_n}(x)} \right|}^2}} dx}_{term4}\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {\gamma _m^*c_m^{}} - \sum\limits_{m = 1}^n {{\gamma _m}c_m^*} + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}} \ \ \ \ (\star)
\end{array}\]
注意到下列 FACT：
\[\begin{array}{l}
\sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} = \sum\limits_{m = 1}^n {\left( {{c_n} - {\gamma _m}} \right){{\left( {{c_n} - {\gamma _m}} \right)}^*}} = \sum\limits_{m = 1}^n {\left( {{c_n}{c_n}^* - {c_n}{\gamma _m}^* - {\gamma _m}{c_n}^* + {\gamma _m}{\gamma _m}^*} \right)} \\
\Rightarrow \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} = \sum\limits_{m = 1}^n {{c_n}{c_n}^*} - \sum\limits_{m = 1}^n {{c_n}{\gamma _m}^*} - \sum\limits_{m = 1}^n {{\gamma _m}{c_n}^*} + \sum\limits_{m = 1}^n {{\gamma _m}{\gamma _m}^*} \\
\Rightarrow \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}} = - \sum\limits_{m = 1}^n {{c_n}{\gamma _m}^*} - \sum\limits_{m = 1}^n {{\gamma _m}{c_n}^*} + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}}
\end{array}\]與
\[\begin{array}{l}
\int_a^b | f(x) - {s_n}(x){|^2}dx = \int_a^b {\left( {f(x) - {s_n}(x)} \right){{\left( {f(x) - {s_n}(x)} \right)}^*}} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {\left( {f(x){f^*}(x) - f(x){s_n}^*(x) - {s_n}(x){f^*}(x) + {s_n}(x){s_n}^*(x)} \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {f(x){f^*}(x)} dx - \int_a^b {f(x){s_n}^*(x)} dx - \int_a^b {{s_n}(x){f^*}(x)} dx + \int_a^b {{s_n}(x){s_n}^*(x)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \int_a^b {\left( {f(x)\sum\limits_{m = 1}^n {{c_m}^*{\phi _m}^*\left( x \right)} } \right)} dx - \int_a^b {\left( {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} {f^*}(x)} \right)} dx + \int_a^b {\left( {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} \sum\limits_{k = 1}^n {{c_k}^*{\phi _k}^*\left( x \right)} } \right)} dx\\
{\rm{since}}\begin{array}{*{20}{c}}
{}
\end{array}{s_n}(x): = \sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} \Rightarrow s_n^*(x): = {\left[ {\sum\limits_{m = 1}^n {{c_m}{\phi _m}\left( x \right)} } \right]^*} = \sum\limits_{m = 1}^n {{c_m}^*{\phi _m}^*\left( x \right)} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} \int_a^b {\left( {f(x){\phi _m}^*\left( x \right)} \right)} dx - \sum\limits_{m = 1}^n {{c_m}} \int_a^b {{f^*}(x){\phi _m}\left( x \right)} dx + \sum\limits_{m = 1}^n {{c_m}} \sum\limits_{k = 1}^n {{c_k}^*} \int_a^b {{\phi _m}\left( x \right){\phi _k}^*\left( x \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} {c_m} - \sum\limits_{m = 1}^n {{c_m}} c_m^* + \sum\limits_{m = 1}^n {{c_m}{c_m}^*} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{c_m}^*} {c_m}\\
\Rightarrow \int_a^b | f(x) - {s_n}(x){|^2}dx = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{{\left| {{c_m}} \right|}^2}}
\end{array}\]
故 $\star$ 可進一步改寫
\[\begin{array}{l}
\int_a^b | f(x) - {t_n}(x){|^2}dx = \int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {\gamma _m^*c_m^{}} - \sum\limits_{m = 1}^n {{\gamma _m}c_m^*} + \sum\limits_{m = 1}^n {{{\left| {{\gamma _m}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b {{{\left| {f(x)} \right|}^2}} dx + \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \underbrace {\int_a^b {{{\left| {f(x)} \right|}^2}} dx - \sum\limits_{m = 1}^n {{{\left| {{c_n}} \right|}^2}} }_{ = \int_a^b | f(x) - {s_n}(x){|^2}dx} + \sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}
\end{array} = \int_a^b | f(x) - {s_n}(x){|^2}dx + \underbrace {\sum\limits_{m = 1}^n {{{\left| {{c_n} - {\gamma _m}} \right|}^2}} }_{ \ge 0}\\
\Rightarrow \int_a^b | f(x) - {t_n}(x){|^2}dx \ge \int_a^b | f(x) - {s_n}(x){|^2}dx\\
\end{array}\]
且若 $c_m = \gamma_m$ 則等式成立。 $\square$
[數學分析] 內積空間的不等式 Cauchy-Schwarz Inequality 與 Triangular Inequality
==============================
Theorem: (Cauchy-Schwarz Inequality)
令 $V$ 為實數內積空間，且 ${\bf u}, {\bf v} \in V$ 則\[
|({\bf u}, {\bf v})| \le ||{\bf u}|| \; ||{\bf v}||
\]==============================
先看幾個例子
Example 1: 歐幾里德平面空間對應的 柯西不等式
$V:=\mathbb{R}^2$ 且配備標準內積 $({\bf u}, {\bf v}) := {\bf u}^T {\bf v}$，現令 ${\bf u}:=[u_1\;\;u_2]^T; \; {\bf v}:=[v_1\;\;v_2]^T$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{l}
\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|\\
\Rightarrow {\left| {{{\bf{u}}^T}{\bf{v}}} \right|^2} \le \left( {{{\bf{u}}^T}{\bf{u}}} \right)\left( {{{\bf{v}}^T}{\bf{v}}} \right)\\
\Rightarrow {\left| {{u_1}{v_1} + {u_2}{v_2}} \right|^2} \le \left( {u_1^2 + u_2^2} \right)\left( {v_1^2 + v_2^2} \right)
\end{array}\]
Example 2: 有限維歐幾里德空間對應的 柯西不等式
$V:=\mathbb{R}^n$ 且配備標準內積 $({\bf u}, {\bf v}) := {\bf u}^T {\bf v}$，現令 ${\bf u}:=[u_1,...,\;\;u_n]^T; \; {\bf v}:=[v_1,...,\;\;v_n]^T$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{*{20}{l}}
{\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|}\\
{ \Rightarrow {{\left| {{{\bf{u}}^T}{\bf{v}}} \right|}^2} \le \left( {{{\bf{u}}^T}{\bf{u}}} \right)\left( {{{\bf{v}}^T}{\bf{v}}} \right)}\\
{ \Rightarrow {{\left| {{u_1}{v_1} + {u_2}{v_2} + ... + {u_n}{v_n}} \right|}^2} \le \left( {u_1^2 + u_2^2 + ... + u_n^2} \right)\left( {v_1^2 + v_2^2 + ... + v_n^n} \right)}
\end{array}\]
Example 3: 無窮維 實數連續函數空間 對應的 柯西不等式
$V:=C[0,1]$ 且配備內積 $(f(t), g(t)) := \int_0^1 f(t) g(t) dt$，現令 ${\bf u}:=f(t); \; {\bf v}:=g(t)$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{l}
\left| {\left( {f\left( t \right),g\left( t \right)} \right)} \right| \le \left\| {f\left( t \right)} \right\|\left\| {g\left( t \right)} \right\|\\
\Rightarrow {\left| {\int_0^1 {f\left( t \right)g\left( t \right)dt} } \right|^2} \le \left( {\int_0^1 {{f^2}\left( t \right)dt} } \right)\left( {\int_0^1 {{g^2}\left( t \right)dt} } \right)
\end{array}\]
Example 4: 實數矩陣空間對應的柯西不等式
令 $V:= M_{n \times n}$ 且配備內積 $(A, B) := tr(B^T A)$ 現令 ${\bf u}:=A; \; {\bf v}:=B$ 為 $n \times n$ 矩陣，則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{*{20}{l}}
{\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|}\\
{ \Rightarrow {{\left| {tr\left( {{B^T}A} \right)} \right|}^2} \le tr\left( {{A^T}A} \right)tr\left( {{B^T}B} \right)}
\end{array}\]
Example 5: 隨機變數所成的 $L^2$ 空間之柯西不等式：
令 $V:= L^p :=\{X: E[|X|^2] < \infty\}$ 且配備內積 $(X,Y) := E[XY]$，其中 $X,Y$ 為隨機變數，$E[\cdot]$ 表期望值。現令 ${\bf u} := X$ 且 ${\bf v} := Y$ 則上述的 Cauchy-Schwarz Inequality 可表為
\begin{align*}
& \left| {\left( {{\mathbf{u}},{\mathbf{v}}} \right)} \right| \leq \left\| {\mathbf{u}} \right\|\left\| {\mathbf{v}} \right\| \hfill \\
& \Rightarrow \left| {E\left[ {XY} \right]} \right| \leq \left\| X \right\|\left\| Y \right\| \hfill \\
& \Rightarrow \left| {E\left[ {XY} \right]} \right| \leq \sqrt {E\left[ {{X^2}} \right]} \sqrt {E\left[ {{Y^2}} \right]} \hfill \\
\end{align*}
Comments:
1.上述幾個例子展示了儘管所表現的樣式非常不同，但從抽象化觀點而言是同一件事情。
2. 柯西等式何時成立？
以下我們給出 Cauchy-Schwarz Inequality 的證明，此證明頗具巧思有興趣的讀者可細細品味。
Proof of Cauchy-Schwarz Inequality
令 $c \in \mathbb{R}^1$ 現在觀察
\[\begin{array}{l}
\underbrace {\left( {{\bf{u}} - c{\bf{v}},{\bf{u}} - c{\bf{v}}} \right)}_{ = {{\left\| {{\bf{u}} - c{\bf{v}}} \right\|}^2}} = \left( {{\bf{u}},{\bf{u}}} \right) + \left( { - c{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{u}}, - c{\bf{v}}} \right) + \left( { - c{\bf{v}}, - c{\bf{v}}} \right)\\
= \left( {{\bf{u}},{\bf{u}}} \right) - c\left( {{\bf{v}},{\bf{u}}} \right) - c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
= \left( {{\bf{u}},{\bf{u}}} \right) - 2c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right) \;\;\;\; (*)
\end{array}\]
若 ${\bf{v}} \ne 0$ 則 $({\bf v},{\bf v})>0$ 我們可取 \[c: = \frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}\]將此 $c$ 帶入 $(*)$ 可得
\[\begin{array}{l}
{\left\| {{\bf{u}} - c{\bf{v}}} \right\|^2} = \left( {{\bf{u}},{\bf{u}}} \right) - 2c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
= \left( {{\bf{u}},{\bf{u}}} \right) - 2\frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}\left( {{\bf{u}},{\bf{v}}} \right) + {\left( {\frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}} \right)^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
= \left( {{\bf{u}},{\bf{u}}} \right) - \frac{{{{\left( {{\bf{u}},{\bf{v}}} \right)}^2}}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}
\end{array}\]但由於 ${\left\| {{\bf{u}} - c{\bf{v}}} \right\|^2} \ge 0$ 故
\[\left( {{\bf{u}},{\bf{u}}} \right) - \frac{{{{\left( {{\bf{u}},{\bf{v}}} \right)}^2}}}{{\left( {{\bf{v}},{\bf{v}}} \right)}} \ge 0 \Rightarrow \left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right) \ge {\left( {{\bf{u}},{\bf{v}}} \right)^2}\]
上式結果說明在 ${\bf v} \neq 0$ 時 Cauchy-Schwarz Inequality 成立。另外我們回頭檢驗 ${\bf v} = 0$ 的情況，則此時 $\left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right) \ge {\left( {{\bf{u}},{\bf{v}}} \right)^2}$ 自動滿足。故不論如何我們都有
\[{\left( {{\bf{u}},{\bf{v}}} \right)^2} \le \left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right)\]至此證畢。$\square$
Comments:
1. 注意到若 ${\bf u} = c {\bf v} $ 則 Cauchy-Schwarz 等式成立。此結果背後蘊含最小平方的最佳化觀點但我們在此不作贅述。
2. 上述 Cauchy-Schwarz Inequality 可引出 Triangular Inequality
Corollary: Triangular Inequality
令 $V$ 為實數內積空間，若 ${\bf u}, {\bf v} \in V $ 則
\[
||{\bf u} + {\bf v}|| \le ||{\bf u}|| + ||{\bf v}||
\]
Proof:
觀察
\[\begin{array}{l}
||{\bf{u}} + {\bf{v}}|{|^2} = \left( {{\bf{u}} + {\bf{v}},{\bf{u}} + {\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{u}}} \right) + \left( {{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{u}},{\bf{v}}} \right) + \left( {{\bf{v}},{\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{u}}} \right) + 2\left( {{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{v}},{\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {\left\| {\bf{u}} \right\|^2} + 2\left( {{\bf{v}},{\bf{u}}} \right) + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left\| {\bf{u}} \right\|^2} + 2\left| {\left( {{\bf{v}},{\bf{u}}} \right)} \right| + {\left\| {\bf{v}} \right\|^2}
\end{array}\]由 Cauchy-Schwarz Inequality 我們有 $
|({\bf u}, {\bf v})| \le ||{\bf u}|| \; ||{\bf v}||$ 故
\[\begin{array}{l}
||{\bf{u}} + {\bf{v}}|{|^2} \le {\left\| {\bf{u}} \right\|^2} + 2\left| {\left( {{\bf{v}},{\bf{u}}} \right)} \right| + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left\| {\bf{u}} \right\|^2} + 2||{\bf{u}}||\;||{\bf{v}}|| + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left( {\left\| {\bf{u}} \right\| + \left\| {\bf{v}} \right\|} \right)^2}
\end{array}\]對兩邊同開根號我們得到
\[
||{\bf u} + {\bf v}|| \le ||{\bf u}|| + ||{\bf v}||
\]即為所求 $\square$
Comment:
若 ${\bf u}, {\bf v}$ 互為正交，亦即 $({\bf u},{\bf v}) = 0$ 則我們有
\[
||{\bf u} + {\bf v}||^2 = ||{\bf u}||^2 + ||{\bf v}||^2
\]上述等式 可視為 畢氏定理 在向量空間中的推廣。
Theorem: (Cauchy-Schwarz Inequality)
令 $V$ 為實數內積空間，且 ${\bf u}, {\bf v} \in V$ 則\[
|({\bf u}, {\bf v})| \le ||{\bf u}|| \; ||{\bf v}||
\]==============================
先看幾個例子
Example 1: 歐幾里德平面空間對應的 柯西不等式
$V:=\mathbb{R}^2$ 且配備標準內積 $({\bf u}, {\bf v}) := {\bf u}^T {\bf v}$，現令 ${\bf u}:=[u_1\;\;u_2]^T; \; {\bf v}:=[v_1\;\;v_2]^T$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{l}
\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|\\
\Rightarrow {\left| {{{\bf{u}}^T}{\bf{v}}} \right|^2} \le \left( {{{\bf{u}}^T}{\bf{u}}} \right)\left( {{{\bf{v}}^T}{\bf{v}}} \right)\\
\Rightarrow {\left| {{u_1}{v_1} + {u_2}{v_2}} \right|^2} \le \left( {u_1^2 + u_2^2} \right)\left( {v_1^2 + v_2^2} \right)
\end{array}\]
Example 2: 有限維歐幾里德空間對應的 柯西不等式
$V:=\mathbb{R}^n$ 且配備標準內積 $({\bf u}, {\bf v}) := {\bf u}^T {\bf v}$，現令 ${\bf u}:=[u_1,...,\;\;u_n]^T; \; {\bf v}:=[v_1,...,\;\;v_n]^T$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{*{20}{l}}
{\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|}\\
{ \Rightarrow {{\left| {{{\bf{u}}^T}{\bf{v}}} \right|}^2} \le \left( {{{\bf{u}}^T}{\bf{u}}} \right)\left( {{{\bf{v}}^T}{\bf{v}}} \right)}\\
{ \Rightarrow {{\left| {{u_1}{v_1} + {u_2}{v_2} + ... + {u_n}{v_n}} \right|}^2} \le \left( {u_1^2 + u_2^2 + ... + u_n^2} \right)\left( {v_1^2 + v_2^2 + ... + v_n^n} \right)}
\end{array}\]
Example 3: 無窮維 實數連續函數空間 對應的 柯西不等式
$V:=C[0,1]$ 且配備內積 $(f(t), g(t)) := \int_0^1 f(t) g(t) dt$，現令 ${\bf u}:=f(t); \; {\bf v}:=g(t)$ 則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{l}
\left| {\left( {f\left( t \right),g\left( t \right)} \right)} \right| \le \left\| {f\left( t \right)} \right\|\left\| {g\left( t \right)} \right\|\\
\Rightarrow {\left| {\int_0^1 {f\left( t \right)g\left( t \right)dt} } \right|^2} \le \left( {\int_0^1 {{f^2}\left( t \right)dt} } \right)\left( {\int_0^1 {{g^2}\left( t \right)dt} } \right)
\end{array}\]
Example 4: 實數矩陣空間對應的柯西不等式
令 $V:= M_{n \times n}$ 且配備內積 $(A, B) := tr(B^T A)$ 現令 ${\bf u}:=A; \; {\bf v}:=B$ 為 $n \times n$ 矩陣，則上述的 Cauchy-Schwarz Inequality 可表為
\[\begin{array}{*{20}{l}}
{\left| {\left( {{\bf{u}},{\bf{v}}} \right)} \right| \le \left\| {\bf{u}} \right\|\left\| {\bf{v}} \right\|}\\
{ \Rightarrow {{\left| {tr\left( {{B^T}A} \right)} \right|}^2} \le tr\left( {{A^T}A} \right)tr\left( {{B^T}B} \right)}
\end{array}\]
Example 5: 隨機變數所成的 $L^2$ 空間之柯西不等式：
令 $V:= L^p :=\{X: E[|X|^2] < \infty\}$ 且配備內積 $(X,Y) := E[XY]$，其中 $X,Y$ 為隨機變數，$E[\cdot]$ 表期望值。現令 ${\bf u} := X$ 且 ${\bf v} := Y$ 則上述的 Cauchy-Schwarz Inequality 可表為
\begin{align*}
& \left| {\left( {{\mathbf{u}},{\mathbf{v}}} \right)} \right| \leq \left\| {\mathbf{u}} \right\|\left\| {\mathbf{v}} \right\| \hfill \\
& \Rightarrow \left| {E\left[ {XY} \right]} \right| \leq \left\| X \right\|\left\| Y \right\| \hfill \\
& \Rightarrow \left| {E\left[ {XY} \right]} \right| \leq \sqrt {E\left[ {{X^2}} \right]} \sqrt {E\left[ {{Y^2}} \right]} \hfill \\
\end{align*}
Comments:
1.上述幾個例子展示了儘管所表現的樣式非常不同，但從抽象化觀點而言是同一件事情。
2. 柯西等式何時成立？
以下我們給出 Cauchy-Schwarz Inequality 的證明，此證明頗具巧思有興趣的讀者可細細品味。
Proof of Cauchy-Schwarz Inequality
令 $c \in \mathbb{R}^1$ 現在觀察
\[\begin{array}{l}
\underbrace {\left( {{\bf{u}} - c{\bf{v}},{\bf{u}} - c{\bf{v}}} \right)}_{ = {{\left\| {{\bf{u}} - c{\bf{v}}} \right\|}^2}} = \left( {{\bf{u}},{\bf{u}}} \right) + \left( { - c{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{u}}, - c{\bf{v}}} \right) + \left( { - c{\bf{v}}, - c{\bf{v}}} \right)\\
= \left( {{\bf{u}},{\bf{u}}} \right) - c\left( {{\bf{v}},{\bf{u}}} \right) - c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
= \left( {{\bf{u}},{\bf{u}}} \right) - 2c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right) \;\;\;\; (*)
\end{array}\]
若 ${\bf{v}} \ne 0$ 則 $({\bf v},{\bf v})>0$ 我們可取 \[c: = \frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}\]將此 $c$ 帶入 $(*)$ 可得
\[\begin{array}{l}
{\left\| {{\bf{u}} - c{\bf{v}}} \right\|^2} = \left( {{\bf{u}},{\bf{u}}} \right) - 2c\left( {{\bf{u}},{\bf{v}}} \right) + {c^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
= \left( {{\bf{u}},{\bf{u}}} \right) - 2\frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}\left( {{\bf{u}},{\bf{v}}} \right) + {\left( {\frac{{\left( {{\bf{u}},{\bf{v}}} \right)}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}} \right)^2}\left( {{\bf{v}},{\bf{v}}} \right)\\
= \left( {{\bf{u}},{\bf{u}}} \right) - \frac{{{{\left( {{\bf{u}},{\bf{v}}} \right)}^2}}}{{\left( {{\bf{v}},{\bf{v}}} \right)}}
\end{array}\]但由於 ${\left\| {{\bf{u}} - c{\bf{v}}} \right\|^2} \ge 0$ 故
\[\left( {{\bf{u}},{\bf{u}}} \right) - \frac{{{{\left( {{\bf{u}},{\bf{v}}} \right)}^2}}}{{\left( {{\bf{v}},{\bf{v}}} \right)}} \ge 0 \Rightarrow \left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right) \ge {\left( {{\bf{u}},{\bf{v}}} \right)^2}\]
上式結果說明在 ${\bf v} \neq 0$ 時 Cauchy-Schwarz Inequality 成立。另外我們回頭檢驗 ${\bf v} = 0$ 的情況，則此時 $\left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right) \ge {\left( {{\bf{u}},{\bf{v}}} \right)^2}$ 自動滿足。故不論如何我們都有
\[{\left( {{\bf{u}},{\bf{v}}} \right)^2} \le \left( {{\bf{u}},{\bf{u}}} \right)\left( {{\bf{v}},{\bf{v}}} \right)\]至此證畢。$\square$
Comments:
1. 注意到若 ${\bf u} = c {\bf v} $ 則 Cauchy-Schwarz 等式成立。此結果背後蘊含最小平方的最佳化觀點但我們在此不作贅述。
2. 上述 Cauchy-Schwarz Inequality 可引出 Triangular Inequality
Corollary: Triangular Inequality
令 $V$ 為實數內積空間，若 ${\bf u}, {\bf v} \in V $ 則
\[
||{\bf u} + {\bf v}|| \le ||{\bf u}|| + ||{\bf v}||
\]
Proof:
觀察
\[\begin{array}{l}
||{\bf{u}} + {\bf{v}}|{|^2} = \left( {{\bf{u}} + {\bf{v}},{\bf{u}} + {\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{u}}} \right) + \left( {{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{u}},{\bf{v}}} \right) + \left( {{\bf{v}},{\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{u}}} \right) + 2\left( {{\bf{v}},{\bf{u}}} \right) + \left( {{\bf{v}},{\bf{v}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = {\left\| {\bf{u}} \right\|^2} + 2\left( {{\bf{v}},{\bf{u}}} \right) + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left\| {\bf{u}} \right\|^2} + 2\left| {\left( {{\bf{v}},{\bf{u}}} \right)} \right| + {\left\| {\bf{v}} \right\|^2}
\end{array}\]由 Cauchy-Schwarz Inequality 我們有 $
|({\bf u}, {\bf v})| \le ||{\bf u}|| \; ||{\bf v}||$ 故
\[\begin{array}{l}
||{\bf{u}} + {\bf{v}}|{|^2} \le {\left\| {\bf{u}} \right\|^2} + 2\left| {\left( {{\bf{v}},{\bf{u}}} \right)} \right| + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left\| {\bf{u}} \right\|^2} + 2||{\bf{u}}||\;||{\bf{v}}|| + {\left\| {\bf{v}} \right\|^2}\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} \le {\left( {\left\| {\bf{u}} \right\| + \left\| {\bf{v}} \right\|} \right)^2}
\end{array}\]對兩邊同開根號我們得到
\[
||{\bf u} + {\bf v}|| \le ||{\bf u}|| + ||{\bf v}||
\]即為所求 $\square$
Comment:
若 ${\bf u}, {\bf v}$ 互為正交，亦即 $({\bf u},{\bf v}) = 0$ 則我們有
\[
||{\bf u} + {\bf v}||^2 = ||{\bf u}||^2 + ||{\bf v}||^2
\]上述等式 可視為 畢氏定理 在向量空間中的推廣。
11/19/2015
[機率論] 非負連續隨機變數 的期望值
令 $Y $ 為 任意非負 連續隨機變數 配備機率密度 $f_Y$，則我們有以下非常簡潔的結果來描述 $Y$ 的期望值 $E[Y]$。
============
Lemma:
\[
E[Y] = \int_0^\infty P(Y>y) dy
\]============
Proof:
首先觀察等式右方，由於 $P\left( {Y > y} \right) = \int_y^\infty {{f_Y}\left( x \right)dx} $ 故
\[\begin{array}{l}
\int_0^\infty {P\left( {Y > y} \right)dy} = \int_0^\infty {\left( {\int_y^\infty {{f_Y}\left( x \right)dx} } \right)dy} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {\left( {\int_0^\infty {{f_Y}\left( x \right){1_{\left\{ {x \ge y} \right\}}}\left( x \right)dx} } \right)dy} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {\left( {\int_0^\infty {{f_Y}\left( x \right){1_{\left\{ {y \le x} \right\}}}\left( y \right)dx} } \right)dy}
\end{array}\]由於 integrand 非負，由 Fubini Theorem 我們可互換積分順序並得到如下結果
\[\begin{array}{l}
\int_0^\infty {P\left( {Y > y} \right)dy} = \int_0^\infty {\int_0^\infty {{f_Y}\left( x \right){1_{\left\{ {y \le x} \right\}}}\left( y \right)dydx} } \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {{f_Y}\left( x \right)\left( {\int_0^\infty {{1_{\left\{ {y \le x} \right\}}}\left( y \right)dy} } \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {{f_Y}\left( x \right)\left( {\int_0^x {1dy} } \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {{f_Y}\left( x \right)x} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {x{f_Y}\left( x \right)} dx = E[Y] \;\;\;\;\;\;\; \square
\end{array}\]
Comments:
1. 前述假設 非負隨機變數是指 $Y \ge 0$ almost surely, 亦即 $ P(Y \ge 0) = 1$
2. 上述證明中採用的符號 $1_{A} (x)$ 表示 指示函數 (indicator function)，我們給出定義如下：令 $X$ 為任意集合 則我們可定義對其上的任意子集 $A \subset X$ 所對應的指示函數( indicator function of a subset $A$ of a set $X$ ) 為 $1_A: X \to \{0,1\}$ 滿足 \[{1_A}\left( x \right): = \left\{ \begin{array}{l}
1,\begin{array}{*{20}{c}}
{}&{x \in A}
\end{array}\\
0,\begin{array}{*{20}{c}}
{}&{x \notin A}
\end{array}
\end{array} \right.\]
3. 上述結果可用 distribution function 改寫，令 $F_Y(y) := P(Y \leq y)$則
\[
E[Y] = \int_0^\infty P(Y>y) dy = \int_0^\infty (1 - F_Y(y)) dy
\]這個結果可以使得我們在計算期望值的同時，不用再困擾需要先求出 pdf ，只要有分配函數 即可計算期望值。
============
Lemma:
\[
E[Y] = \int_0^\infty P(Y>y) dy
\]============
Proof:
首先觀察等式右方，由於 $P\left( {Y > y} \right) = \int_y^\infty {{f_Y}\left( x \right)dx} $ 故
\[\begin{array}{l}
\int_0^\infty {P\left( {Y > y} \right)dy} = \int_0^\infty {\left( {\int_y^\infty {{f_Y}\left( x \right)dx} } \right)dy} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {\left( {\int_0^\infty {{f_Y}\left( x \right){1_{\left\{ {x \ge y} \right\}}}\left( x \right)dx} } \right)dy} \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {\left( {\int_0^\infty {{f_Y}\left( x \right){1_{\left\{ {y \le x} \right\}}}\left( y \right)dx} } \right)dy}
\end{array}\]由於 integrand 非負，由 Fubini Theorem 我們可互換積分順序並得到如下結果
\[\begin{array}{l}
\int_0^\infty {P\left( {Y > y} \right)dy} = \int_0^\infty {\int_0^\infty {{f_Y}\left( x \right){1_{\left\{ {y \le x} \right\}}}\left( y \right)dydx} } \\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {{f_Y}\left( x \right)\left( {\int_0^\infty {{1_{\left\{ {y \le x} \right\}}}\left( y \right)dy} } \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {{f_Y}\left( x \right)\left( {\int_0^x {1dy} } \right)} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {{f_Y}\left( x \right)x} dx\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \int_0^\infty {x{f_Y}\left( x \right)} dx = E[Y] \;\;\;\;\;\;\; \square
\end{array}\]
Comments:
1. 前述假設 非負隨機變數是指 $Y \ge 0$ almost surely, 亦即 $ P(Y \ge 0) = 1$
2. 上述證明中採用的符號 $1_{A} (x)$ 表示 指示函數 (indicator function)，我們給出定義如下：令 $X$ 為任意集合 則我們可定義對其上的任意子集 $A \subset X$ 所對應的指示函數( indicator function of a subset $A$ of a set $X$ ) 為 $1_A: X \to \{0,1\}$ 滿足 \[{1_A}\left( x \right): = \left\{ \begin{array}{l}
1,\begin{array}{*{20}{c}}
{}&{x \in A}
\end{array}\\
0,\begin{array}{*{20}{c}}
{}&{x \notin A}
\end{array}
\end{array} \right.\]
3. 上述結果可用 distribution function 改寫，令 $F_Y(y) := P(Y \leq y)$則
\[
E[Y] = \int_0^\infty P(Y>y) dy = \int_0^\infty (1 - F_Y(y)) dy
\]這個結果可以使得我們在計算期望值的同時，不用再困擾需要先求出 pdf ，只要有分配函數 即可計算期望值。
11/14/2015
[線性代數] 淺論有限維 實數內積空間 (0)
這次要介紹 有限維度的內積空間 (Inner Product Space)，簡而言之就是有限維度向量空間 $(V, \oplus, \odot)$ 上額外定義內積運算，則我們稱此類空間為 內積空間。
Comments:
1. 有限維度內積空間稱為 歐式空間 (Euclidean Space)
2. 若為無窮維度的內積空間我們稱為 Pre-Hilbert Space，若此無窮維度內積空間為完備空間，則稱之為 Hilbert Space
3. 為何好好的向量空間不用還要多此一舉另外又定一個 內積空間？主因是向量空間本身只定義了加法 與純量乘法的運算，如果我們想討論在向量空間中某元素的大小 或者 某兩元素之間的關係則無從得知。但是如果我們引入 內積運算 到向量空間中，則可以在原本的向量空間上將 代數 與 幾何 的概念做直接的連結，也就是我們可以透過內積引入 其上的兩元素是否 垂直 (正交) 的概念，亦可針對某元素來探討其 長度與大小 概念 。
4. 讀者可回憶 高中所學習過的 點積 (dot product)，此文所探討的內積 即為 點積 的推廣。
首先定義內積
==================
Definition: Inner Product on Vector Space
令 $V$ 為實數向量空間，則 Inner Product on $V$ 為函數 $(\cdot, \cdot): V\times V \to \mathbb{R}$ 滿足下列條件
(a) $({\bf u}, {\bf u}) \ge 0$：且 $({\bf u}, {\bf u}) = 0$ 若且唯若 ${\bf u} = {\bf 0}_V$
(b) $({\bf u}, {\bf v}) = ({\bf v}, {\bf u}), \; \forall {\bf u,v} \in V$
(c) $({\bf u} + {\bf v}, {\bf w}) = ({\bf u}, {\bf w}) + ({\bf u}, {\bf v}), \; \forall {\bf u,v,w} \in V$
(d) $(c {\bf u}, {\bf v}) = c({\bf u}, {\bf v}),\; \forall {\bf u,v} \in V, c \in \mathbb{R}$
==================
Comment:
1. 透過內積我們亦可定義 ${\bf u}$ 的大小，記作 $||{\bf u}|| = \sqrt{ ({\bf u}, {\bf u} )} $
2. 透過內積我們亦可定義在內積空間中兩向量是否垂直：亦即 若 ${\bf u}, {\bf v} \in V$ 稱為 垂直 或 正交 若 $({\bf u}, {\bf v}) = 0$
2. 前述內積定義可立即有以下衍生結果：
Fact 1: $\left( {{\bf{u}},c{\bf{v}}} \right) = c\left( {{\bf{u}},{\bf{v}}} \right)$
Fact 2: $\left( {{\bf{u}},{\bf{v}} + {\bf{w}}} \right) = \left( {{\bf{u}},{\bf{v}}} \right) + \left( {{\bf{u}},{\bf{w}}} \right)$
讀者可自行驗證上述兩個結果。
=========
Claim 1: 給定 ${\bf v}, {\bf w} \in V$ ，若對任意 ${\bf u} \in V$ 我們有 $
({\bf u},{\bf v}) = ({\bf u}, {\bf w})$ 則 ${\bf v} = {\bf w}$
=========
Proof:
由於我們要證明 ${\bf v} = {\bf w}$，故我們僅需證明 $\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = 0$ 現在觀察兩者之差的內積
\[\begin{array}{l}
\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) + \left( {{\bf{v}} - {\bf{w}}, - {\bf{w}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) - \left( {{\bf{v}} - {\bf{w}},{\bf{w}}} \right)
\end{array}\]注意到 ${\bf v} - {\bf w} \in V$ 故 若我們令 ${\bf u}:= {\bf v} - {\bf w} $ 則 由已知條件可知
\[\begin{array}{l}
\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) - \left( {{\bf{v}} - {\bf{w}},{\bf{w}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{v}}} \right) - \left( {{\bf{u}},{\bf{w}}} \right) = 0. \;\;\; \square
\end{array}\]
由上述結果我們有以下衍生定理
=========
Corollary of Claim 1:
若 對任意 ${\bf u} \in V$ 我們有 $({\bf u}, {\bf v})=0$ 則 ${\bf v} = 0$
=========
Proof: omitted.
=========
Claim: 令向量空間 $ V := \mathbb{R}^n$ 現在定義下列函數 $f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ 滿足
\[
f({\bf u},{\bf v}) := {\bf u}^T {\bf v}
\]其中 ${\bf u} := [u_1\;u_2\;...,u_n]^T $ 與 ${\bf v} := [v_1\;\;v_2\;...,v_n]^T $ 為任意 $\mathbb{R}^n$ 空間中的向量， 則此運算為一種內積。
=========
Proof:
我們現在驗證 $f$ 確實為內積，故我們需驗證其滿足前述四項 (a,b,c,d)條件：首先驗證 $(a)$:
\[f({\bf{u}},{\bf{u}}) = {{\bf{u}}^T}{\bf{u}} = u_1^2 + ...u_2^2 \ge 0\]且 $f({\bf{u}},{\bf{u}}) = 0$ 若且唯若 ${\bf u} = [0,...,0]^T$
$(b)$ 令 ${\bf u,v} \in \mathbb{R}^n$ 現在我們觀察
\[\begin{array}{l}
f({\bf{v}},{\bf{u}}) = {{\bf{v}}^T}{\bf{u}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {v_1}{u_1} + ... + {v_n}{u_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {u_1}{v_1} + ... + {u_n}{v_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right] = {{\bf{u}}^T}{\bf{v}} = f\left( {{\bf{u}},{\bf{v}}} \right)
\end{array}
\]
$(c)$ 令 $\bf u,v,w$$\in \mathbb{R}^n$ 接著我們觀察
\[\begin{array}{l}
f({\bf{u}} + {\bf{v}},{\bf{w}}) = {\left( {{\bf{u}} + {\bf{v}}} \right)^T}{\bf{w}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1} + {v_1}}&{{u_2} + {v_2}}&{...}&{{u_n} + {v_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{u_1} + {v_1}} \right){w_1} + ...\left( {{u_n} + {v_n}} \right){w_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{u_1}{w_1} + ... + {u_n}{w_n}} \right) + \left( {{v_1}{w_1} + ... + {v_n}{w_n}} \right)\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right] + \left[ {\begin{array}{*{20}{c}}
{{v_1}}&{{v_2}}&{...}&{{v_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {\left( {\bf{u}} \right)^T}{\bf{w}} + {\left( {\bf{v}} \right)^T}{\bf{w}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = f({\bf{u}},{\bf{w}}) + f({\bf{v}},{\bf{w}})
\end{array}
\]
$(d)$ 同理我們觀察
\[\begin{array}{l}
f(c{\bf{u}},{\bf{v}}) = {\left( {c{\bf{u}}} \right)^T}{\bf{v}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{c{u_1}}&{c{u_2}}&{...}&{c{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c\left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c{\left( {\bf{u}} \right)^T}{\bf{v}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = cf({\bf{u}},{\bf{v}})
\end{array}\]
Claim 2: 令向量空間 $ V := \mathbb{R}^n$ 現在定義下列函數 $f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ 滿足
\[
f({\bf u},{\bf v}) := {\bf u}^T {\bf v}
\]其中 ${\bf u} := [u_1\;u_2\;...,u_n]^T $ 與 ${\bf v} := [v_1\;\;v_2\;...,v_n]^T $ 為任意 $\mathbb{R}^n$ 空間中的向量， 則此運算為 $\mathbb{R}^n$ 一種內積。
Proof: omitted
Claim 3: 令向量空間 $ V := C[a,b]$ ，若 $f,g \in V$ 令
\[\left( {f,g} \right): = \int_a^b {f\left( t \right)g\left( t \right)dt} \] 則 $(f,g)$ 為 $C[a,b]$ 上的一種內積。
Proof: omitted
以下我們接著介紹對任意有限維度向量空間，則其上的內積可以用一個透過基底表示的矩陣 $C$ 來完全決定。
==================
Theorem: 令 $S=\{{\bf u}_1,...,{\bf u}_n\}$ 為 向量空間 $V$ 的 ordered basis 且假設我們可在 $V$ 上定義內積，現在令 $c_{ij} = ({\bf u}_i, {\bf u}_j)$ 且 $C=[c_{ij}]$ 矩陣 則
對任意 ${\bf v,w} \in V$， 存在 $C = [c_{ij}]$ 矩陣 使得 $({\bf v},{\bf w}) = [{\bf v}]_S^T C [{\bf w}]_S$
==================
Proof : 給定 ${\bf v,w} \in V$ 則我們有
\[\begin{array}{l}
{\bf{v}} = {a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n}\\
{\bf{w}} = {b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}
\end{array}
\]
現在我們觀察內積 \[\begin{array}{l}
({\bf{v}},{\bf{w}}) = ({a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n},{\bf{w}})\\
= ({a_1}{{\bf{u}}_1},{\bf{w}}) + ({a_2}{{\bf{u}}_2},{\bf{w}}) + ... + ({a_n}{{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{\bf{w}}) + {a_2}({{\bf{u}}_2},{\bf{w}}) + ... + {a_n}({{\bf{u}}_n},{\bf{w}})
\end{array}\]又因為 ${\bf{w}} = {b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}$ 故
\[\begin{array}{l}
({\bf{v}},{\bf{w}}) = ({a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n},{\bf{w}})\\
= ({a_1}{{\bf{u}}_1},{\bf{w}}) + ({a_2}{{\bf{u}}_2},{\bf{w}}) + ... + ({a_n}{{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{\bf{w}}) + {a_2}({{\bf{u}}_2},{\bf{w}}) + ... + {a_n}({{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}) + {a_2}({{\bf{u}}_2},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}) + ... + {a_n}({{\bf{u}}_n},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n})\\
= \left[ {{a_1}({{\bf{u}}_1},{b_1}{{\bf{u}}_1}) + {a_1}({{\bf{u}}_1},{b_2}{{\bf{u}}_2}) + ... + {a_1}({{\bf{u}}_1},{b_n}{{\bf{u}}_n})} \right] + \\
\begin{array}{*{20}{c}}
{}&{}
\end{array}... + \left[ {{a_n}({{\bf{u}}_n},{b_1}{{\bf{u}}_1}) + {a_n}({{\bf{u}}_n},{b_2}{{\bf{u}}_2})... + {a_n}({{\bf{u}}_n},{b_n}{{\bf{u}}_n})} \right]\\
= \left[ {{a_1}{b_1}({{\bf{u}}_1},{{\bf{u}}_1}) + {a_1}{b_2}({{\bf{u}}_1},{{\bf{u}}_2}) + ... + {a_1}{b_n}({{\bf{u}}_1},{{\bf{u}}_n})} \right] + \\
\begin{array}{*{20}{c}}
{}&{}
\end{array}... + \left[ {{a_n}{b_1}({{\bf{u}}_n},{{\bf{u}}_1}) + {a_n}{b_2}({{\bf{u}}_n},{{\bf{u}}_2})... + {a_n}{b_n}({{\bf{u}}_n},{{\bf{u}}_n})} \right]\\
= \sum\limits_{j = 1}^n {{a_1}{b_j}({{\bf{u}}_1},{{\bf{u}}_j})} + \sum\limits_{j = 1}^n {{a_2}{b_j}({{\bf{u}}_2},{{\bf{u}}_j})} + ... + \sum\limits_{j = 1}^n {{a_n}{b_j}({{\bf{u}}_n},{{\bf{u}}_j})} \\
= \sum\limits_{j = 1}^n {\left( {{a_1}{b_j}({{\bf{u}}_1},{{\bf{u}}_j}) + {a_2}{b_j}({{\bf{u}}_2},{{\bf{u}}_j}) + ... + {a_n}{b_j}({{\bf{u}}_n},{{\bf{u}}_j})} \right)} \\
= \sum\limits_{j = 1}^n {\sum\limits_{i = 1}^n {{a_i}{b_j}({{\bf{u}}_i},{{\bf{u}}_j})} }
\end{array}\]
現在令 $c_{ij} = (a_i, b_j)$ ，則我們有
\[({\bf{v}},{\bf{w}}) = \sum\limits_{j = 1}^n {\sum\limits_{i = 1}^n {{a_i}{b_j}{c_{ij}}} } =[{\bf v}]_S C [{\bf w}]_S\]
Comments:
1. 前述的 $C$ 矩陣亦稱為 matrix of the inner produce with respect to the ordered basis $S$
2. 由於我們有 $c_{ij} = ({\bf u}_i, {\bf u}_j)$ 且利用內積定義 $ ({\bf u}_i, {\bf u}_j)=({\bf u}_j, {\bf u}_i) $ 我們得到 $c_{ij} = c_{ji}$，故 前述的 $C = [c_{ij}]$ 矩陣為對稱矩陣。
上述 inner product 不但可用來引出 norm 的概念，更保持了連續性，以下我們給出相關結果。
============================
Lemma: (Continuity of Inner Product)
令 $u_n \to u$ 且 $v_n \to v$ 且收斂在某內積空間 (或 Pre-Hibert Space) ，則
\[
(u_n,v_n) \to (u,v)
\] ============================
Proof: 直接觀察
\begin{align*}
\left| {({u_n},{v_n}) - (u,v)} \right| &= \left| {({u_n},{v_n}) - \left( {{u_n},v} \right) + \left( {{u_n},v} \right) - (u,v)} \right| \hfill \\
& \leqslant \left| {({u_n},{v_n}) - \left( {{u_n},v} \right)} \right| + \left| {\left( {{u_n},v} \right) - (u,v)} \right| \hfill \\
&= \left| {({u_n},{v_n} - v)} \right| + \left| {\left( {{u_n} - u,v} \right)} \right| \hfill \\
\end{align*} 回憶 Cauchy Schwarz Inequality $|(x,y)| \leq ||x|| ||y||$，我們有
\[\left| {({u_n},{v_n}) - (u,v)} \right| \leqslant \left\| {{u_n}} \right\|\left\| {{v_n} - v} \right\| + \left\| {{u_n} - u} \right\|\left\| v \right\|\;\;\;\; (*)
\]注意到因為 $\{u_n\}$數列為收斂數列，故 $||u_n|| < \infty$，另外 $v$ 為 收斂數列 $\{v_n\}$ 的極限，故 $||v|| < \infty$。除此之外，由假設 $u_n \to u$ 且 $v_n \to v$ 可知
\[
||u_n - u|| \to 0;\;\;\;\;\; ||v_n \to v|| \to 0
\]因此，上式 $(*)$ 取極限後可得
\[\left| {({u_n},{v_n}) - (u,v)} \right| \leqslant \underbrace {\left\| {{u_n}} \right\|}_{ < \infty }\underbrace {\left\| {{v_n} - v} \right\|}_{ \to 0} + \underbrace {\left\| {{u_n} - u} \right\|}_{ \to 0}\underbrace {\left\| v \right\|}_{ < \infty } \to 0 + 0 = 0\]
Comments:
1. 有限維度內積空間稱為 歐式空間 (Euclidean Space)
2. 若為無窮維度的內積空間我們稱為 Pre-Hilbert Space，若此無窮維度內積空間為完備空間，則稱之為 Hilbert Space
3. 為何好好的向量空間不用還要多此一舉另外又定一個 內積空間？主因是向量空間本身只定義了加法 與純量乘法的運算，如果我們想討論在向量空間中某元素的大小 或者 某兩元素之間的關係則無從得知。但是如果我們引入 內積運算 到向量空間中，則可以在原本的向量空間上將 代數 與 幾何 的概念做直接的連結，也就是我們可以透過內積引入 其上的兩元素是否 垂直 (正交) 的概念，亦可針對某元素來探討其 長度與大小 概念 。
4. 讀者可回憶 高中所學習過的 點積 (dot product)，此文所探討的內積 即為 點積 的推廣。
首先定義內積
==================
Definition: Inner Product on Vector Space
令 $V$ 為實數向量空間，則 Inner Product on $V$ 為函數 $(\cdot, \cdot): V\times V \to \mathbb{R}$ 滿足下列條件
(a) $({\bf u}, {\bf u}) \ge 0$：且 $({\bf u}, {\bf u}) = 0$ 若且唯若 ${\bf u} = {\bf 0}_V$
(b) $({\bf u}, {\bf v}) = ({\bf v}, {\bf u}), \; \forall {\bf u,v} \in V$
(c) $({\bf u} + {\bf v}, {\bf w}) = ({\bf u}, {\bf w}) + ({\bf u}, {\bf v}), \; \forall {\bf u,v,w} \in V$
(d) $(c {\bf u}, {\bf v}) = c({\bf u}, {\bf v}),\; \forall {\bf u,v} \in V, c \in \mathbb{R}$
==================
1. 透過內積我們亦可定義 ${\bf u}$ 的大小，記作 $||{\bf u}|| = \sqrt{ ({\bf u}, {\bf u} )} $
2. 透過內積我們亦可定義在內積空間中兩向量是否垂直：亦即 若 ${\bf u}, {\bf v} \in V$ 稱為 垂直 或 正交 若 $({\bf u}, {\bf v}) = 0$
2. 前述內積定義可立即有以下衍生結果：
Fact 1: $\left( {{\bf{u}},c{\bf{v}}} \right) = c\left( {{\bf{u}},{\bf{v}}} \right)$
Fact 2: $\left( {{\bf{u}},{\bf{v}} + {\bf{w}}} \right) = \left( {{\bf{u}},{\bf{v}}} \right) + \left( {{\bf{u}},{\bf{w}}} \right)$
讀者可自行驗證上述兩個結果。
=========
Claim 1: 給定 ${\bf v}, {\bf w} \in V$ ，若對任意 ${\bf u} \in V$ 我們有 $
({\bf u},{\bf v}) = ({\bf u}, {\bf w})$ 則 ${\bf v} = {\bf w}$
=========
Proof:
由於我們要證明 ${\bf v} = {\bf w}$，故我們僅需證明 $\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = 0$ 現在觀察兩者之差的內積
\[\begin{array}{l}
\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) + \left( {{\bf{v}} - {\bf{w}}, - {\bf{w}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) - \left( {{\bf{v}} - {\bf{w}},{\bf{w}}} \right)
\end{array}\]注意到 ${\bf v} - {\bf w} \in V$ 故 若我們令 ${\bf u}:= {\bf v} - {\bf w} $ 則 由已知條件可知
\[\begin{array}{l}
\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) - \left( {{\bf{v}} - {\bf{w}},{\bf{w}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{v}}} \right) - \left( {{\bf{u}},{\bf{w}}} \right) = 0. \;\;\; \square
\end{array}\]
由上述結果我們有以下衍生定理
=========
Corollary of Claim 1:
若 對任意 ${\bf u} \in V$ 我們有 $({\bf u}, {\bf v})=0$ 則 ${\bf v} = 0$
=========
Proof: omitted.
=========
Claim: 令向量空間 $ V := \mathbb{R}^n$ 現在定義下列函數 $f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ 滿足
\[
f({\bf u},{\bf v}) := {\bf u}^T {\bf v}
\]其中 ${\bf u} := [u_1\;u_2\;...,u_n]^T $ 與 ${\bf v} := [v_1\;\;v_2\;...,v_n]^T $ 為任意 $\mathbb{R}^n$ 空間中的向量， 則此運算為一種內積。
=========
Proof:
我們現在驗證 $f$ 確實為內積，故我們需驗證其滿足前述四項 (a,b,c,d)條件：首先驗證 $(a)$:
\[f({\bf{u}},{\bf{u}}) = {{\bf{u}}^T}{\bf{u}} = u_1^2 + ...u_2^2 \ge 0\]且 $f({\bf{u}},{\bf{u}}) = 0$ 若且唯若 ${\bf u} = [0,...,0]^T$
$(b)$ 令 ${\bf u,v} \in \mathbb{R}^n$ 現在我們觀察
\[\begin{array}{l}
f({\bf{v}},{\bf{u}}) = {{\bf{v}}^T}{\bf{u}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {v_1}{u_1} + ... + {v_n}{u_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {u_1}{v_1} + ... + {u_n}{v_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right] = {{\bf{u}}^T}{\bf{v}} = f\left( {{\bf{u}},{\bf{v}}} \right)
\end{array}
\]
$(c)$ 令 $\bf u,v,w$$\in \mathbb{R}^n$ 接著我們觀察
\[\begin{array}{l}
f({\bf{u}} + {\bf{v}},{\bf{w}}) = {\left( {{\bf{u}} + {\bf{v}}} \right)^T}{\bf{w}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1} + {v_1}}&{{u_2} + {v_2}}&{...}&{{u_n} + {v_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{u_1} + {v_1}} \right){w_1} + ...\left( {{u_n} + {v_n}} \right){w_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{u_1}{w_1} + ... + {u_n}{w_n}} \right) + \left( {{v_1}{w_1} + ... + {v_n}{w_n}} \right)\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right] + \left[ {\begin{array}{*{20}{c}}
{{v_1}}&{{v_2}}&{...}&{{v_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {\left( {\bf{u}} \right)^T}{\bf{w}} + {\left( {\bf{v}} \right)^T}{\bf{w}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = f({\bf{u}},{\bf{w}}) + f({\bf{v}},{\bf{w}})
\end{array}
\]
$(d)$ 同理我們觀察
\[\begin{array}{l}
f(c{\bf{u}},{\bf{v}}) = {\left( {c{\bf{u}}} \right)^T}{\bf{v}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{c{u_1}}&{c{u_2}}&{...}&{c{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c\left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c{\left( {\bf{u}} \right)^T}{\bf{v}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = cf({\bf{u}},{\bf{v}})
\end{array}\]
Claim 2: 令向量空間 $ V := \mathbb{R}^n$ 現在定義下列函數 $f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ 滿足
\[
f({\bf u},{\bf v}) := {\bf u}^T {\bf v}
\]其中 ${\bf u} := [u_1\;u_2\;...,u_n]^T $ 與 ${\bf v} := [v_1\;\;v_2\;...,v_n]^T $ 為任意 $\mathbb{R}^n$ 空間中的向量， 則此運算為 $\mathbb{R}^n$ 一種內積。
Proof: omitted
Claim 3: 令向量空間 $ V := C[a,b]$ ，若 $f,g \in V$ 令
\[\left( {f,g} \right): = \int_a^b {f\left( t \right)g\left( t \right)dt} \] 則 $(f,g)$ 為 $C[a,b]$ 上的一種內積。
Proof: omitted
以下我們接著介紹對任意有限維度向量空間，則其上的內積可以用一個透過基底表示的矩陣 $C$ 來完全決定。
==================
Theorem: 令 $S=\{{\bf u}_1,...,{\bf u}_n\}$ 為 向量空間 $V$ 的 ordered basis 且假設我們可在 $V$ 上定義內積，現在令 $c_{ij} = ({\bf u}_i, {\bf u}_j)$ 且 $C=[c_{ij}]$ 矩陣 則
對任意 ${\bf v,w} \in V$， 存在 $C = [c_{ij}]$ 矩陣 使得 $({\bf v},{\bf w}) = [{\bf v}]_S^T C [{\bf w}]_S$
==================
\[\begin{array}{l}
{\bf{v}} = {a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n}\\
{\bf{w}} = {b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}
\end{array}
\]
現在我們觀察內積 \[\begin{array}{l}
({\bf{v}},{\bf{w}}) = ({a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n},{\bf{w}})\\
= ({a_1}{{\bf{u}}_1},{\bf{w}}) + ({a_2}{{\bf{u}}_2},{\bf{w}}) + ... + ({a_n}{{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{\bf{w}}) + {a_2}({{\bf{u}}_2},{\bf{w}}) + ... + {a_n}({{\bf{u}}_n},{\bf{w}})
\end{array}\]又因為 ${\bf{w}} = {b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}$ 故
\[\begin{array}{l}
({\bf{v}},{\bf{w}}) = ({a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n},{\bf{w}})\\
= ({a_1}{{\bf{u}}_1},{\bf{w}}) + ({a_2}{{\bf{u}}_2},{\bf{w}}) + ... + ({a_n}{{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{\bf{w}}) + {a_2}({{\bf{u}}_2},{\bf{w}}) + ... + {a_n}({{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}) + {a_2}({{\bf{u}}_2},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}) + ... + {a_n}({{\bf{u}}_n},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n})\\
= \left[ {{a_1}({{\bf{u}}_1},{b_1}{{\bf{u}}_1}) + {a_1}({{\bf{u}}_1},{b_2}{{\bf{u}}_2}) + ... + {a_1}({{\bf{u}}_1},{b_n}{{\bf{u}}_n})} \right] + \\
\begin{array}{*{20}{c}}
{}&{}
\end{array}... + \left[ {{a_n}({{\bf{u}}_n},{b_1}{{\bf{u}}_1}) + {a_n}({{\bf{u}}_n},{b_2}{{\bf{u}}_2})... + {a_n}({{\bf{u}}_n},{b_n}{{\bf{u}}_n})} \right]\\
= \left[ {{a_1}{b_1}({{\bf{u}}_1},{{\bf{u}}_1}) + {a_1}{b_2}({{\bf{u}}_1},{{\bf{u}}_2}) + ... + {a_1}{b_n}({{\bf{u}}_1},{{\bf{u}}_n})} \right] + \\
\begin{array}{*{20}{c}}
{}&{}
\end{array}... + \left[ {{a_n}{b_1}({{\bf{u}}_n},{{\bf{u}}_1}) + {a_n}{b_2}({{\bf{u}}_n},{{\bf{u}}_2})... + {a_n}{b_n}({{\bf{u}}_n},{{\bf{u}}_n})} \right]\\
= \sum\limits_{j = 1}^n {{a_1}{b_j}({{\bf{u}}_1},{{\bf{u}}_j})} + \sum\limits_{j = 1}^n {{a_2}{b_j}({{\bf{u}}_2},{{\bf{u}}_j})} + ... + \sum\limits_{j = 1}^n {{a_n}{b_j}({{\bf{u}}_n},{{\bf{u}}_j})} \\
= \sum\limits_{j = 1}^n {\left( {{a_1}{b_j}({{\bf{u}}_1},{{\bf{u}}_j}) + {a_2}{b_j}({{\bf{u}}_2},{{\bf{u}}_j}) + ... + {a_n}{b_j}({{\bf{u}}_n},{{\bf{u}}_j})} \right)} \\
= \sum\limits_{j = 1}^n {\sum\limits_{i = 1}^n {{a_i}{b_j}({{\bf{u}}_i},{{\bf{u}}_j})} }
\end{array}\]
現在令 $c_{ij} = (a_i, b_j)$ ，則我們有
\[({\bf{v}},{\bf{w}}) = \sum\limits_{j = 1}^n {\sum\limits_{i = 1}^n {{a_i}{b_j}{c_{ij}}} } =[{\bf v}]_S C [{\bf w}]_S\]
Comments:
1. 前述的 $C$ 矩陣亦稱為 matrix of the inner produce with respect to the ordered basis $S$
2. 由於我們有 $c_{ij} = ({\bf u}_i, {\bf u}_j)$ 且利用內積定義 $ ({\bf u}_i, {\bf u}_j)=({\bf u}_j, {\bf u}_i) $ 我們得到 $c_{ij} = c_{ji}$，故 前述的 $C = [c_{ij}]$ 矩陣為對稱矩陣。
上述 inner product 不但可用來引出 norm 的概念，更保持了連續性，以下我們給出相關結果。
============================
Lemma: (Continuity of Inner Product)
令 $u_n \to u$ 且 $v_n \to v$ 且收斂在某內積空間 (或 Pre-Hibert Space) ，則
\[
(u_n,v_n) \to (u,v)
\] ============================
\begin{align*}
\left| {({u_n},{v_n}) - (u,v)} \right| &= \left| {({u_n},{v_n}) - \left( {{u_n},v} \right) + \left( {{u_n},v} \right) - (u,v)} \right| \hfill \\
& \leqslant \left| {({u_n},{v_n}) - \left( {{u_n},v} \right)} \right| + \left| {\left( {{u_n},v} \right) - (u,v)} \right| \hfill \\
&= \left| {({u_n},{v_n} - v)} \right| + \left| {\left( {{u_n} - u,v} \right)} \right| \hfill \\
\end{align*} 回憶 Cauchy Schwarz Inequality $|(x,y)| \leq ||x|| ||y||$，我們有
\[\left| {({u_n},{v_n}) - (u,v)} \right| \leqslant \left\| {{u_n}} \right\|\left\| {{v_n} - v} \right\| + \left\| {{u_n} - u} \right\|\left\| v \right\|\;\;\;\; (*)
\]注意到因為 $\{u_n\}$數列為收斂數列，故 $||u_n|| < \infty$，另外 $v$ 為 收斂數列 $\{v_n\}$ 的極限，故 $||v|| < \infty$。除此之外，由假設 $u_n \to u$ 且 $v_n \to v$ 可知
\[
||u_n - u|| \to 0;\;\;\;\;\; ||v_n \to v|| \to 0
\]因此，上式 $(*)$ 取極限後可得
\[\left| {({u_n},{v_n}) - (u,v)} \right| \leqslant \underbrace {\left\| {{u_n}} \right\|}_{ < \infty }\underbrace {\left\| {{v_n} - v} \right\|}_{ \to 0} + \underbrace {\left\| {{u_n} - u} \right\|}_{ \to 0}\underbrace {\left\| v \right\|}_{ < \infty } \to 0 + 0 = 0\]
11/01/2015
[MATLAB] 將 symbolic expression 轉成 latex 程式碼
一般而言在 MATLAB 使用中不免會碰到使用 symbolic toolbox 情況，但有時表示式非常繁雜，如果又想要把該表示方程式轉寫成 latex 貼到論文中該怎麼辦？
MATLAB 提供一個非常方便的功能
latex(.)
可以幫助我們直接轉換 MATLAB symbolic expression 成為 LATEX code
MATLAB 提供一個非常方便的功能
latex(.)
可以幫助我們直接轉換 MATLAB symbolic expression 成為 LATEX code
10/30/2015
[線性代數] 座標轉換矩陣
考慮 $V$ 為 $n$ 維向量空間，且 ${\bf v} \in V$。現在考慮兩組 有序基底 (ordered basis)
\[\begin{array}{l}
S: = \{ {{\bf{v}}_1},{{\bf{v}}_2},...,{{\bf{v}}_n}\} \\
T: = \{ {{\bf{w}}_1},{{\bf{w}}_2},...,{{\bf{w}}_n}\}
\end{array}\]
則 我們可將 ${\bf v}$ 用上述有序基底做線性組合唯一表示，比如說
\[
{\bf v} = c_1 { {\bf w}_1} + ... c_n {\bf w}_n
\]且其對應於 $T$ 基底的 座標向量 (coordinate vector) 與 對 $S$ 基底的 coordinate vector 我們定義如下
{\bf v} = c_1 { {\bf w}_1} + ... c_n {\bf w}_n
\]且其對應於 $T$ 基底的 座標向量 (coordinate vector) 與 對 $S$ 基底的 coordinate vector 我們定義如下
\[{\left[ {\bf{v}} \right]_T} := \left[ {\begin{array}{*{20}{l}}
{{c_1}}\\
{{c_2}}\\
\vdots \\
{{c_n}}
\end{array}} \right]; \;\;{[{\bf{v}}]_S} := \left[ \begin{array}{l}
{a_1}\\
{a_2}\\
\vdots \\
{a_n}
\end{array} \right]
\]注意到事實上 上述 對 $T$ 基底的 coordinate vector 可看成是函數，比如說令 $L: V \to \mathbb{R}^n$ 滿足
\[
L({\bf v}) = [{\bf v}]_T
\]同理，對 $S$ 基底的 coordinate vector 令 $L': V \to \mathbb{R}^n$ 滿足
\[
L'({\bf v}) = [{\bf v}]_S
\]
Comment:
1. 上述 $L, L'$ 統稱為 coordinate mapping
2. Coordinate mapping 為 bijective linear transformation 或稱 isomorphism。
現在若我們想建構 對於 $S$ 基底的座標向量 與 $T$ 基底座標向量之間兩者的關係，利用 coordinate mapping $[{\bf v}]_S$ 為 linear transformation 性質 ，我們有
\[\begin{array}{l}
{[{\bf{v}}]_S} = {[{c_1}{{\bf{w}}_1} + ...{c_n}{{\bf{w}}_n}]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {[{c_1}{{\bf{w}}_1}]_S} + ... + {[{c_n}{{\bf{w}}_n}]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {c_1}{[{{\bf{w}}_1}]_S} + ... + {c_n}{[{{\bf{w}}_n}]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
|&|&{}&|\\
{{{[{{\bf{w}}_1}]}_S}}&{{{[{{\bf{w}}_2}]}_S}}&{...}&{{{[{{\bf{w}}_n}]}_S}}\\
|&|&{}&|
\end{array}} \right]\left[ \begin{array}{l}
{c_1}\\
{c_2}\\
\vdots \\
{c_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {P_{S \leftarrow T}}\left[ \begin{array}{l}
{c_1}\\
{c_2}\\
\vdots \\
{c_n}
\end{array} \right]
\end{array}\]其中 ${P_{S \leftarrow T}}: = \left[ {\begin{array}{*{20}{c}}
|&|&{}&|\\
{{{[{{\bf{w}}_1}]}_S}}&{{{[{{\bf{w}}_2}]}_S}}&{...}&{{{[{{\bf{w}}_n}]}_S}}\\
|&|&{}&|
\end{array}} \right]$ 稱為 從 $T$ 基底到 $S$ 基底的座標轉換矩陣 (Transition Matrix from T-basis to the S basis )且對於個別的 coordinate vector; e.g., $[{\bf w}_j]_S$ 我們有
\[{[{{\bf{w}}_j}]_S} = \left[ \begin{array}{l}
{a_{1j}}\\
{a_{2j}}\\
\vdots \\
{a_{nj}}
\end{array} \right]\]
故我們有以下結果
\[
[{\bf v}]_S = P_{S \leftarrow T} [{\bf v}]_T
\]
以下是一些關於 Transition Matrix 的性質：
令 $S,T$ 為向量空間的兩組 ordered basis 則
FACT 1. $P_{T \leftarrow T} = I$
FACT 2. $P_{S \leftarrow T}$ 為 nonsingular
現在看幾個例子：
Example 1.
令 \[S: = \left\{ {{{\bf{v}}_1},{{\bf{v}}_2}} \right\} = \left\{ {\left[ \begin{array}{l}
1\\
2
\end{array} \right],\left[ \begin{array}{l}
0\\
1
\end{array} \right]} \right\};T: = \left\{ {{{\bf{w}}_1},{{\bf{w}}_2}} \right\} = \left\{ {\left[ \begin{array}{l}
1\\
1
\end{array} \right],\left[ \begin{array}{l}
2\\
3
\end{array} \right]} \right\}\]為 ordered bases for $\mathbb{R}^2$。現在令 ${\bf v} = [1 \;\; 5]^T$ 與 ${\bf w} := [5 \;\; 4]^T$ 。
(a) 試求 coordinate vectors $[{\bf v}]_T$ 與 $[{\bf w}]_T$
(b) 試求 $P_{S \leftarrow T}$
(c) 試求 coordinate vectors $[{\bf v}]_S$ 與 $[{\bf w}]_S$
Solution(a)
由 $[{\bf v}]_T$ 的定義可知 ${\left[ {\bf{v}} \right]_T} = \left[ \begin{array}{l}
{a_1}\\
{a_2}
\end{array} \right]$且
\[\begin{array}{l}
\left[ \begin{array}{l}
1\\
5
\end{array} \right] = {a_1}\left[ \begin{array}{l}
1\\
1
\end{array} \right] + {a_2}\left[ \begin{array}{l}
2\\
3
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
1&2\\
1&3
\end{array}} \right]\left[ \begin{array}{l}
{a_1}\\
{a_2}
\end{array} \right] = \left[ \begin{array}{l}
1\\
5
\end{array} \right]\\
\Rightarrow \left\{ \begin{array}{l}
{a_1} = - 7\\
{a_2} = 4
\end{array} \right.
\end{array}\]同理 ${\left[ {\bf{w}} \right]_T} = \left[ \begin{array}{l}
{b_1}\\
{b_2}
\end{array} \right]$
\[\begin{array}{l}
\underbrace {\left[ \begin{array}{l}
5\\
4
\end{array} \right]}_{ = {\bf{w}}} = {b_1}\left[ \begin{array}{l}
1\\
1
\end{array} \right] + {b_2}\left[ \begin{array}{l}
2\\
3
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
1&2\\
1&3
\end{array}} \right]\left[ \begin{array}{l}
{b_1}\\
{b_2}
\end{array} \right] = \left[ \begin{array}{l}
5\\
4
\end{array} \right]\\
\Rightarrow \left\{ \begin{array}{l}
{b_1} = 7\\
{b_2} = - 1
\end{array} \right.
\end{array}\]
Solution (b)
我們要求 $P_{S \leftarrow T}$，由 part (a) 可知我們有 $[{\bf v}]_T = [-7\;\;4]$ 故
\[\begin{array}{l}
{[{\bf{v}}]_S} = {\left[ { - 7{{\bf{w}}_1} + 4{{\bf{w}}_2}} \right]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = - 7{\left[ {{{\bf{w}}_1}} \right]_S} + 4{\left[ {{{\bf{w}}_2}} \right]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \underbrace {\left[ {\begin{array}{*{20}{c}}
|&|\\
{{{\left[ {{{\bf{w}}_1}} \right]}_S}}&{{{\left[ {{{\bf{w}}_2}} \right]}_S}}\\
|&|
\end{array}} \right]}_{ = {P_{S \leftarrow T}}}\underbrace {\left[ \begin{array}{l}
- 7\\
4
\end{array} \right]}_{ = {{[{\bf{v}}]}_T}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \underbrace {\left[ {\begin{array}{*{20}{c}}
{{b_{11}}}&{{b_{12}}}\\
{{b_{21}}}&{{b_{22}}}
\end{array}} \right]}_{{P_{S \leftarrow T}}}\left[ \begin{array}{l}
- 7\\
4
\end{array} \right]
\end{array}\]
現在我們分別求取 $[{\bf w}_1]_S$ 與 $[{\bf w}_2]_S$如下:
\[\begin{array}{l}
{\left[ {{{\bf{w}}_1}} \right]_S} = \left[ \begin{array}{l}
{b_{11}}\\
{b_{21}}
\end{array} \right]\\
\Rightarrow {b_{11}}\left[ \begin{array}{l}
1\\
2
\end{array} \right] + {b_{21}}\left[ \begin{array}{l}
0\\
1
\end{array} \right] = \underbrace {\left[ \begin{array}{l}
1\\
1
\end{array} \right]}_{ = {{\bf{w}}_1}}\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
1&0\\
2&1
\end{array}} \right]\left[ \begin{array}{l}
{b_{11}}\\
{b_{21}}
\end{array} \right] = \left[ \begin{array}{l}
1\\
1
\end{array} \right]\\
\Rightarrow \left\{ \begin{array}{l}
{b_{11}} = 1\\
{b_{21}} = - 1
\end{array} \right.
\end{array}\]
且
\[\begin{array}{l}
{\left[ {{{\bf{w}}_2}} \right]_S} = \left[ \begin{array}{l}
{b_{12}}\\
{b_{22}}
\end{array} \right]\\
\Rightarrow {b_{12}}\left[ \begin{array}{l}
1\\
2
\end{array} \right] + {b_{22}}\left[ \begin{array}{l}
0\\
1
\end{array} \right] = \underbrace {\left[ \begin{array}{l}
2\\
3
\end{array} \right]}_{ = {{\bf{w}}_2}}\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
1&0\\
2&1
\end{array}} \right]\left[ \begin{array}{l}
{b_{11}}\\
{b_{21}}
\end{array} \right] = \left[ \begin{array}{l}
2\\
3
\end{array} \right]\\
\Rightarrow \left\{ \begin{array}{l}
{b_{12}} = 2\\
{b_{21}} = - 1
\end{array} \right.
\end{array}\]故
\[{P_{S \leftarrow T}} = \left[ {\begin{array}{*{20}{c}}
1&2\\
{ - 1}&{ - 1}
\end{array}} \right]\]
Solution (c)
\[\begin{array}{l}
{[{\bf{v}}]_S} = {P_{S \leftarrow T}}{[{\bf{v}}]_T}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
1&2\\
{ - 1}&{ - 1}
\end{array}} \right]\left[ \begin{array}{l}
- 7\\
4
\end{array} \right] = \left[ \begin{array}{l}
1\\
3
\end{array} \right]\\
{[{\bf{w}}]_S} = {P_{S \leftarrow T}}{[{\bf{w}}]_T}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
1&2\\
{ - 1}&{ - 1}
\end{array}} \right]\left[ \begin{array}{l}
7\\
- 1
\end{array} \right] = \left[ \begin{array}{l}
5\\
- 6
\end{array} \right]
\end{array}\]
Example 2.
令 $V:= \mathbb{R^3}$ 且令 $S:=\{{\bf v}_1,{\bf v}_2, {\bf v}_3\}$ 且 $T = \{{\bf w}_1, {\bf w}_2, {\bf w}_3\}$ 為 $\mathbb{R}^3$ 的ordered basis，其中
\[\begin{array}{l}
{{\bf{v}}_1} = \left[ \begin{array}{l}
2\\
0\\
1
\end{array} \right];{{\bf{v}}_2} = \left[ \begin{array}{l}
1\\
2\\
0
\end{array} \right];{{\bf{v}}_3} = \left[ \begin{array}{l}
1\\
1\\
1
\end{array} \right]\\
{{\bf{w}}_1} = \left[ \begin{array}{l}
6\\
3\\
3
\end{array} \right];{{\bf{w}}_2} = \left[ \begin{array}{l}
4\\
- 1\\
3
\end{array} \right];{{\bf{w}}_3} = \left[ \begin{array}{l}
5\\
5\\
2
\end{array} \right]
\end{array}\]
(a) 試計算 $P_{S \leftarrow T}$
(Exercise) 令 ${\bf v} = [4 \;\; -9 \;\; 5]$ 驗證 $[{\bf v}]_S = P_{S \leftarrow T} [{\bf v}]_T$
Solution (a):
由前面討論可知
\[{P_{S \leftarrow T}}: = \left[ {\begin{array}{*{20}{c}}
|&|&|\\
{{{[{{\bf{w}}_1}]}_S}}&{{{[{{\bf{w}}_2}]}_S}}&{{{[{{\bf{w}}_3}]}_S}}\\
|&|&|
\end{array}} \right]\]故我們需分別求出 $[{\bf w}_1]_S, [{\bf w}_2]_S$ 與 $[{\bf w}_3]_S$：
首先求 ${[{{\bf{w}}_1}]_S} = \left[ \begin{array}{l}
{a_{11}}\\
{a_{21}}\\
{a_{31}}
\end{array} \right]$ 如下：由於 ${\bf w}_1$ 可用 $S$ 有序基底作唯一線性組合表示，故
\[\begin{array}{l}
\left[ \begin{array}{l}
6\\
3\\
3
\end{array} \right] = {a_{11}}\left[ \begin{array}{l}
2\\
0\\
1
\end{array} \right] + {a_{21}}\left[ \begin{array}{l}
1\\
2\\
0
\end{array} \right] + {a_{31}}\left[ \begin{array}{l}
1\\
1\\
1
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
2&1&1\\
0&2&1\\
1&0&1
\end{array}} \right]\left[ \begin{array}{l}
{a_{11}}\\
{a_{21}}\\
{a_{31}}
\end{array} \right] = \left[ \begin{array}{l}
6\\
3\\
3
\end{array} \right] \Rightarrow \left\{ \begin{array}{l}
{a_{11}} = 2\\
{a_{21}} = 1\\
{a_{31}} = 1
\end{array} \right.
\end{array}
\]
接著我們求 ${[{{\bf{w}}_2}]_S} = \left[ \begin{array}{l}
{a_{12}}\\
{a_{22}}\\
{a_{32}}
\end{array} \right]$ ，由 ${\bf w}_2$ 可用 $S$ 有序基底作唯一線性組合表示，我們可得
\[\begin{array}{l}
\left[ \begin{array}{l}
4\\
- 1\\
3
\end{array} \right] = {a_{12}}\left[ \begin{array}{l}
2\\
0\\
1
\end{array} \right] + {a_{22}}\left[ \begin{array}{l}
1\\
2\\
0
\end{array} \right] + {a_{32}}\left[ \begin{array}{l}
1\\
1\\
1
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
2&1&1\\
0&2&1\\
1&0&1
\end{array}} \right]\left[ \begin{array}{l}
{a_{11}}\\
{a_{21}}\\
{a_{31}}
\end{array} \right] = \left[ \begin{array}{l}
4\\
- 1\\
3
\end{array} \right] \Rightarrow \left\{ \begin{array}{l}
{a_{12}} = 2\\
{a_{22}} = - 1\\
{a_{32}} = 1
\end{array} \right.
\end{array}\]
最後求 ${[{{\bf{w}}_3}]_S} = \left[ \begin{array}{l}
{a_{13}}\\
{a_{23}}\\
{a_{33}}
\end{array} \right]$，同前述方法，利用 ${\bf w}_3$ 可透過 $S$ 有序基底作唯一線性組合表示，我們可得
\[\begin{array}{l}
\left[ \begin{array}{l}
5\\
5\\
2
\end{array} \right] = {a_{13}}\left[ \begin{array}{l}
2\\
0\\
1
\end{array} \right] + {a_{23}}\left[ \begin{array}{l}
1\\
2\\
0
\end{array} \right] + {a_{33}}\left[ \begin{array}{l}
1\\
1\\
1
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
2&1&1\\
0&2&1\\
1&0&1
\end{array}} \right]\left[ \begin{array}{l}
{a_{11}}\\
{a_{21}}\\
{a_{31}}
\end{array} \right] = \left[ \begin{array}{l}
5\\
5\\
2
\end{array} \right] \Rightarrow \left\{ \begin{array}{l}
{a_{13}} = 1\\
{a_{23}} = 2\\
{a_{33}} = 1
\end{array} \right.
\end{array}\]
\[\begin{array}{l}
{P_{S \leftarrow T}}: = \left[ {\begin{array}{*{20}{c}}
|&|&|\\
{{{[{{\bf{w}}_1}]}_S}}&{{{[{{\bf{w}}_2}]}_S}}&{{{[{{\bf{w}}_3}]}_S}}\\
|&|&|
\end{array}} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{a_{11}}}&{{a_{12}}}&{{a_{13}}}\\
{{a_{12}}}&{{a_{22}}}&{{a_{23}}}\\
{{a_{13}}}&{{a_{32}}}&{{a_{33}}}
\end{array}} \right] = \left[ {\begin{array}{*{20}{c}}
2&2&1\\
1&{ - 1}&2\\
1&1&1
\end{array}} \right]
\end{array}\]
{{c_1}}\\
{{c_2}}\\
\vdots \\
{{c_n}}
\end{array}} \right]; \;\;{[{\bf{v}}]_S} := \left[ \begin{array}{l}
{a_1}\\
{a_2}\\
\vdots \\
{a_n}
\end{array} \right]
\]注意到事實上 上述 對 $T$ 基底的 coordinate vector 可看成是函數，比如說令 $L: V \to \mathbb{R}^n$ 滿足
\[
L({\bf v}) = [{\bf v}]_T
\]同理，對 $S$ 基底的 coordinate vector 令 $L': V \to \mathbb{R}^n$ 滿足
\[
L'({\bf v}) = [{\bf v}]_S
\]
Comment:
1. 上述 $L, L'$ 統稱為 coordinate mapping
2. Coordinate mapping 為 bijective linear transformation 或稱 isomorphism。
現在若我們想建構 對於 $S$ 基底的座標向量 與 $T$ 基底座標向量之間兩者的關係，利用 coordinate mapping $[{\bf v}]_S$ 為 linear transformation 性質 ，我們有
\[\begin{array}{l}
{[{\bf{v}}]_S} = {[{c_1}{{\bf{w}}_1} + ...{c_n}{{\bf{w}}_n}]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {[{c_1}{{\bf{w}}_1}]_S} + ... + {[{c_n}{{\bf{w}}_n}]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {c_1}{[{{\bf{w}}_1}]_S} + ... + {c_n}{[{{\bf{w}}_n}]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
|&|&{}&|\\
{{{[{{\bf{w}}_1}]}_S}}&{{{[{{\bf{w}}_2}]}_S}}&{...}&{{{[{{\bf{w}}_n}]}_S}}\\
|&|&{}&|
\end{array}} \right]\left[ \begin{array}{l}
{c_1}\\
{c_2}\\
\vdots \\
{c_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {P_{S \leftarrow T}}\left[ \begin{array}{l}
{c_1}\\
{c_2}\\
\vdots \\
{c_n}
\end{array} \right]
\end{array}\]其中 ${P_{S \leftarrow T}}: = \left[ {\begin{array}{*{20}{c}}
|&|&{}&|\\
{{{[{{\bf{w}}_1}]}_S}}&{{{[{{\bf{w}}_2}]}_S}}&{...}&{{{[{{\bf{w}}_n}]}_S}}\\
|&|&{}&|
\end{array}} \right]$ 稱為 從 $T$ 基底到 $S$ 基底的座標轉換矩陣 (Transition Matrix from T-basis to the S basis )且對於個別的 coordinate vector; e.g., $[{\bf w}_j]_S$ 我們有
\[{[{{\bf{w}}_j}]_S} = \left[ \begin{array}{l}
{a_{1j}}\\
{a_{2j}}\\
\vdots \\
{a_{nj}}
\end{array} \right]\]
故我們有以下結果
\[
[{\bf v}]_S = P_{S \leftarrow T} [{\bf v}]_T
\]
以下是一些關於 Transition Matrix 的性質：
令 $S,T$ 為向量空間的兩組 ordered basis 則
FACT 1. $P_{T \leftarrow T} = I$
FACT 2. $P_{S \leftarrow T}$ 為 nonsingular
現在看幾個例子：
Example 1.
令 \[S: = \left\{ {{{\bf{v}}_1},{{\bf{v}}_2}} \right\} = \left\{ {\left[ \begin{array}{l}
1\\
2
\end{array} \right],\left[ \begin{array}{l}
0\\
1
\end{array} \right]} \right\};T: = \left\{ {{{\bf{w}}_1},{{\bf{w}}_2}} \right\} = \left\{ {\left[ \begin{array}{l}
1\\
1
\end{array} \right],\left[ \begin{array}{l}
2\\
3
\end{array} \right]} \right\}\]為 ordered bases for $\mathbb{R}^2$。現在令 ${\bf v} = [1 \;\; 5]^T$ 與 ${\bf w} := [5 \;\; 4]^T$ 。
(a) 試求 coordinate vectors $[{\bf v}]_T$ 與 $[{\bf w}]_T$
(b) 試求 $P_{S \leftarrow T}$
(c) 試求 coordinate vectors $[{\bf v}]_S$ 與 $[{\bf w}]_S$
Solution(a)
由 $[{\bf v}]_T$ 的定義可知 ${\left[ {\bf{v}} \right]_T} = \left[ \begin{array}{l}
{a_1}\\
{a_2}
\end{array} \right]$且
\[\begin{array}{l}
\left[ \begin{array}{l}
1\\
5
\end{array} \right] = {a_1}\left[ \begin{array}{l}
1\\
1
\end{array} \right] + {a_2}\left[ \begin{array}{l}
2\\
3
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
1&2\\
1&3
\end{array}} \right]\left[ \begin{array}{l}
{a_1}\\
{a_2}
\end{array} \right] = \left[ \begin{array}{l}
1\\
5
\end{array} \right]\\
\Rightarrow \left\{ \begin{array}{l}
{a_1} = - 7\\
{a_2} = 4
\end{array} \right.
\end{array}\]同理 ${\left[ {\bf{w}} \right]_T} = \left[ \begin{array}{l}
{b_1}\\
{b_2}
\end{array} \right]$
\[\begin{array}{l}
\underbrace {\left[ \begin{array}{l}
5\\
4
\end{array} \right]}_{ = {\bf{w}}} = {b_1}\left[ \begin{array}{l}
1\\
1
\end{array} \right] + {b_2}\left[ \begin{array}{l}
2\\
3
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
1&2\\
1&3
\end{array}} \right]\left[ \begin{array}{l}
{b_1}\\
{b_2}
\end{array} \right] = \left[ \begin{array}{l}
5\\
4
\end{array} \right]\\
\Rightarrow \left\{ \begin{array}{l}
{b_1} = 7\\
{b_2} = - 1
\end{array} \right.
\end{array}\]
Solution (b)
我們要求 $P_{S \leftarrow T}$，由 part (a) 可知我們有 $[{\bf v}]_T = [-7\;\;4]$ 故
\[\begin{array}{l}
{[{\bf{v}}]_S} = {\left[ { - 7{{\bf{w}}_1} + 4{{\bf{w}}_2}} \right]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = - 7{\left[ {{{\bf{w}}_1}} \right]_S} + 4{\left[ {{{\bf{w}}_2}} \right]_S}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \underbrace {\left[ {\begin{array}{*{20}{c}}
|&|\\
{{{\left[ {{{\bf{w}}_1}} \right]}_S}}&{{{\left[ {{{\bf{w}}_2}} \right]}_S}}\\
|&|
\end{array}} \right]}_{ = {P_{S \leftarrow T}}}\underbrace {\left[ \begin{array}{l}
- 7\\
4
\end{array} \right]}_{ = {{[{\bf{v}}]}_T}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \underbrace {\left[ {\begin{array}{*{20}{c}}
{{b_{11}}}&{{b_{12}}}\\
{{b_{21}}}&{{b_{22}}}
\end{array}} \right]}_{{P_{S \leftarrow T}}}\left[ \begin{array}{l}
- 7\\
4
\end{array} \right]
\end{array}\]
現在我們分別求取 $[{\bf w}_1]_S$ 與 $[{\bf w}_2]_S$如下:
\[\begin{array}{l}
{\left[ {{{\bf{w}}_1}} \right]_S} = \left[ \begin{array}{l}
{b_{11}}\\
{b_{21}}
\end{array} \right]\\
\Rightarrow {b_{11}}\left[ \begin{array}{l}
1\\
2
\end{array} \right] + {b_{21}}\left[ \begin{array}{l}
0\\
1
\end{array} \right] = \underbrace {\left[ \begin{array}{l}
1\\
1
\end{array} \right]}_{ = {{\bf{w}}_1}}\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
1&0\\
2&1
\end{array}} \right]\left[ \begin{array}{l}
{b_{11}}\\
{b_{21}}
\end{array} \right] = \left[ \begin{array}{l}
1\\
1
\end{array} \right]\\
\Rightarrow \left\{ \begin{array}{l}
{b_{11}} = 1\\
{b_{21}} = - 1
\end{array} \right.
\end{array}\]
且
\[\begin{array}{l}
{\left[ {{{\bf{w}}_2}} \right]_S} = \left[ \begin{array}{l}
{b_{12}}\\
{b_{22}}
\end{array} \right]\\
\Rightarrow {b_{12}}\left[ \begin{array}{l}
1\\
2
\end{array} \right] + {b_{22}}\left[ \begin{array}{l}
0\\
1
\end{array} \right] = \underbrace {\left[ \begin{array}{l}
2\\
3
\end{array} \right]}_{ = {{\bf{w}}_2}}\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
1&0\\
2&1
\end{array}} \right]\left[ \begin{array}{l}
{b_{11}}\\
{b_{21}}
\end{array} \right] = \left[ \begin{array}{l}
2\\
3
\end{array} \right]\\
\Rightarrow \left\{ \begin{array}{l}
{b_{12}} = 2\\
{b_{21}} = - 1
\end{array} \right.
\end{array}\]故
\[{P_{S \leftarrow T}} = \left[ {\begin{array}{*{20}{c}}
1&2\\
{ - 1}&{ - 1}
\end{array}} \right]\]
Solution (c)
\[\begin{array}{l}
{[{\bf{v}}]_S} = {P_{S \leftarrow T}}{[{\bf{v}}]_T}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
1&2\\
{ - 1}&{ - 1}
\end{array}} \right]\left[ \begin{array}{l}
- 7\\
4
\end{array} \right] = \left[ \begin{array}{l}
1\\
3
\end{array} \right]\\
{[{\bf{w}}]_S} = {P_{S \leftarrow T}}{[{\bf{w}}]_T}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
1&2\\
{ - 1}&{ - 1}
\end{array}} \right]\left[ \begin{array}{l}
7\\
- 1
\end{array} \right] = \left[ \begin{array}{l}
5\\
- 6
\end{array} \right]
\end{array}\]
Example 2.
令 $V:= \mathbb{R^3}$ 且令 $S:=\{{\bf v}_1,{\bf v}_2, {\bf v}_3\}$ 且 $T = \{{\bf w}_1, {\bf w}_2, {\bf w}_3\}$ 為 $\mathbb{R}^3$ 的ordered basis，其中
\[\begin{array}{l}
{{\bf{v}}_1} = \left[ \begin{array}{l}
2\\
0\\
1
\end{array} \right];{{\bf{v}}_2} = \left[ \begin{array}{l}
1\\
2\\
0
\end{array} \right];{{\bf{v}}_3} = \left[ \begin{array}{l}
1\\
1\\
1
\end{array} \right]\\
{{\bf{w}}_1} = \left[ \begin{array}{l}
6\\
3\\
3
\end{array} \right];{{\bf{w}}_2} = \left[ \begin{array}{l}
4\\
- 1\\
3
\end{array} \right];{{\bf{w}}_3} = \left[ \begin{array}{l}
5\\
5\\
2
\end{array} \right]
\end{array}\]
(a) 試計算 $P_{S \leftarrow T}$
(Exercise) 令 ${\bf v} = [4 \;\; -9 \;\; 5]$ 驗證 $[{\bf v}]_S = P_{S \leftarrow T} [{\bf v}]_T$
Solution (a):
由前面討論可知
\[{P_{S \leftarrow T}}: = \left[ {\begin{array}{*{20}{c}}
|&|&|\\
{{{[{{\bf{w}}_1}]}_S}}&{{{[{{\bf{w}}_2}]}_S}}&{{{[{{\bf{w}}_3}]}_S}}\\
|&|&|
\end{array}} \right]\]故我們需分別求出 $[{\bf w}_1]_S, [{\bf w}_2]_S$ 與 $[{\bf w}_3]_S$：
首先求 ${[{{\bf{w}}_1}]_S} = \left[ \begin{array}{l}
{a_{11}}\\
{a_{21}}\\
{a_{31}}
\end{array} \right]$ 如下：由於 ${\bf w}_1$ 可用 $S$ 有序基底作唯一線性組合表示，故
\[\begin{array}{l}
\left[ \begin{array}{l}
6\\
3\\
3
\end{array} \right] = {a_{11}}\left[ \begin{array}{l}
2\\
0\\
1
\end{array} \right] + {a_{21}}\left[ \begin{array}{l}
1\\
2\\
0
\end{array} \right] + {a_{31}}\left[ \begin{array}{l}
1\\
1\\
1
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
2&1&1\\
0&2&1\\
1&0&1
\end{array}} \right]\left[ \begin{array}{l}
{a_{11}}\\
{a_{21}}\\
{a_{31}}
\end{array} \right] = \left[ \begin{array}{l}
6\\
3\\
3
\end{array} \right] \Rightarrow \left\{ \begin{array}{l}
{a_{11}} = 2\\
{a_{21}} = 1\\
{a_{31}} = 1
\end{array} \right.
\end{array}
\]
接著我們求 ${[{{\bf{w}}_2}]_S} = \left[ \begin{array}{l}
{a_{12}}\\
{a_{22}}\\
{a_{32}}
\end{array} \right]$ ，由 ${\bf w}_2$ 可用 $S$ 有序基底作唯一線性組合表示，我們可得
\[\begin{array}{l}
\left[ \begin{array}{l}
4\\
- 1\\
3
\end{array} \right] = {a_{12}}\left[ \begin{array}{l}
2\\
0\\
1
\end{array} \right] + {a_{22}}\left[ \begin{array}{l}
1\\
2\\
0
\end{array} \right] + {a_{32}}\left[ \begin{array}{l}
1\\
1\\
1
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
2&1&1\\
0&2&1\\
1&0&1
\end{array}} \right]\left[ \begin{array}{l}
{a_{11}}\\
{a_{21}}\\
{a_{31}}
\end{array} \right] = \left[ \begin{array}{l}
4\\
- 1\\
3
\end{array} \right] \Rightarrow \left\{ \begin{array}{l}
{a_{12}} = 2\\
{a_{22}} = - 1\\
{a_{32}} = 1
\end{array} \right.
\end{array}\]
最後求 ${[{{\bf{w}}_3}]_S} = \left[ \begin{array}{l}
{a_{13}}\\
{a_{23}}\\
{a_{33}}
\end{array} \right]$，同前述方法，利用 ${\bf w}_3$ 可透過 $S$ 有序基底作唯一線性組合表示，我們可得
\[\begin{array}{l}
\left[ \begin{array}{l}
5\\
5\\
2
\end{array} \right] = {a_{13}}\left[ \begin{array}{l}
2\\
0\\
1
\end{array} \right] + {a_{23}}\left[ \begin{array}{l}
1\\
2\\
0
\end{array} \right] + {a_{33}}\left[ \begin{array}{l}
1\\
1\\
1
\end{array} \right]\\
\Rightarrow \left[ {\begin{array}{*{20}{c}}
2&1&1\\
0&2&1\\
1&0&1
\end{array}} \right]\left[ \begin{array}{l}
{a_{11}}\\
{a_{21}}\\
{a_{31}}
\end{array} \right] = \left[ \begin{array}{l}
5\\
5\\
2
\end{array} \right] \Rightarrow \left\{ \begin{array}{l}
{a_{13}} = 1\\
{a_{23}} = 2\\
{a_{33}} = 1
\end{array} \right.
\end{array}\]
\[\begin{array}{l}
{P_{S \leftarrow T}}: = \left[ {\begin{array}{*{20}{c}}
|&|&|\\
{{{[{{\bf{w}}_1}]}_S}}&{{{[{{\bf{w}}_2}]}_S}}&{{{[{{\bf{w}}_3}]}_S}}\\
|&|&|
\end{array}} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{a_{11}}}&{{a_{12}}}&{{a_{13}}}\\
{{a_{12}}}&{{a_{22}}}&{{a_{23}}}\\
{{a_{13}}}&{{a_{32}}}&{{a_{33}}}
\end{array}} \right] = \left[ {\begin{array}{*{20}{c}}
2&2&1\\
1&{ - 1}&2\\
1&1&1
\end{array}} \right]
\end{array}\]
10/29/2015
[線性代數] 伴隨矩陣 $adj(A)$ 的一些性質
給定 $n \times n$ 的非奇異方陣 $A$，則下列性質成立
Claim: $adj A$ 為非奇異矩陣
Proof: 由於 $A$ 為 nonsingular，我們有
\[
A^{-1} = \frac{1}{detA} adj (A) \]
注意到等號左方 $A^{-1}$ 亦為 nonsingular (why? 由 $A$ 為 nonsingular 的定義出發，我們可知$A A^{-1} = I$ 等價說明 $A^{-1}$ 為 nonsingular)，且由於 $det A$ 僅為常數，故 $adj A$ 必定為 nonsingular。 $\square$
Claim: $\det (adj A) = (\det A)^{n-1}$
Proof: 由 $A^{-1}$ 可知
\[\begin{array}{l}
{A^{ - 1}} = \frac{1}{{\det A}}adj\left( A \right)\\
\Rightarrow \left( {\det A} \right)\left( {{A^{ - 1}}} \right) = adj\left( A \right)\\
\Rightarrow \det \left( {\left( {\det A} \right)\left( {{A^{ - 1}}} \right)} \right) = \det \left( {adj\left( A \right)} \right)\\
\Rightarrow {\left( {\det A} \right)^n}\det \left( {{A^{ - 1}}} \right) = \det \left( {adj\left( A \right)} \right)\\
\Rightarrow {\left( {\det A} \right)^n}\frac{1}{{\det A}} = \det \left( {adj\left( A \right)} \right)\\
\Rightarrow {\left( {\det A} \right)^{n - 1}} = \det \left( {adj\left( A \right)} \right)
\end{array}\]
Claim: $(adj A)^{-1} = adj (A^{-1}) = \frac{1}{det A} A$
首先注意到
\[\begin{array}{l}
{A^{ - 1}} = \frac{1}{{\det A}}adj\left( A \right)\\
\Rightarrow \left( {\det A} \right)I = adj\left( A \right)A
\end{array}\]由 Claim 1 可知 $adj A$ 為 nonsingular 故 $(adj (A))^{-1}$ 存在，亦即我們可改寫上式如下
\[\begin{array}{l}
{\left( {adj\left( A \right)} \right)^{ - 1}}\left( {\det A} \right)I = {\left( {adj\left( A \right)} \right)^{ - 1}}adj\left( A \right)A\\
\Rightarrow \left( {\det A} \right){\left( {adj\left( A \right)} \right)^{ - 1}} = A\\
\Rightarrow {\left( {adj\left( A \right)} \right)^{ - 1}} = \frac{1}{{\det A}}A
\end{array}\]
接著我們證明 $adj (A^{-1}) = \frac{1}{det A} A$
我們觀察
\[\begin{array}{l}
{A^{ - 1}} = \frac{1}{{\det A}}adj\left( A \right)\\
\Rightarrow A = \frac{1}{{\det \left( {{A^{ - 1}}} \right)}}adj\left( {{A^{ - 1}}} \right)\\
\Rightarrow A = \left( {\det A} \right)adj\left( {{A^{ - 1}}} \right)\\
\Rightarrow \frac{1}{{\det A}}A = adj\left( {{A^{ - 1}}} \right)
\end{array}\]
故綜合以上所述，我們可得 $(adj A)^{-1} = adj (A^{-1}) = \frac{1}{det A} A$ $\square$
Claim: $adj A$ 為非奇異矩陣
Proof: 由於 $A$ 為 nonsingular，我們有
\[
A^{-1} = \frac{1}{detA} adj (A) \]
注意到等號左方 $A^{-1}$ 亦為 nonsingular (why? 由 $A$ 為 nonsingular 的定義出發，我們可知$A A^{-1} = I$ 等價說明 $A^{-1}$ 為 nonsingular)，且由於 $det A$ 僅為常數，故 $adj A$ 必定為 nonsingular。 $\square$
Claim: $\det (adj A) = (\det A)^{n-1}$
Proof: 由 $A^{-1}$ 可知
\[\begin{array}{l}
{A^{ - 1}} = \frac{1}{{\det A}}adj\left( A \right)\\
\Rightarrow \left( {\det A} \right)\left( {{A^{ - 1}}} \right) = adj\left( A \right)\\
\Rightarrow \det \left( {\left( {\det A} \right)\left( {{A^{ - 1}}} \right)} \right) = \det \left( {adj\left( A \right)} \right)\\
\Rightarrow {\left( {\det A} \right)^n}\det \left( {{A^{ - 1}}} \right) = \det \left( {adj\left( A \right)} \right)\\
\Rightarrow {\left( {\det A} \right)^n}\frac{1}{{\det A}} = \det \left( {adj\left( A \right)} \right)\\
\Rightarrow {\left( {\det A} \right)^{n - 1}} = \det \left( {adj\left( A \right)} \right)
\end{array}\]
Claim: $(adj A)^{-1} = adj (A^{-1}) = \frac{1}{det A} A$
首先注意到
\[\begin{array}{l}
{A^{ - 1}} = \frac{1}{{\det A}}adj\left( A \right)\\
\Rightarrow \left( {\det A} \right)I = adj\left( A \right)A
\end{array}\]由 Claim 1 可知 $adj A$ 為 nonsingular 故 $(adj (A))^{-1}$ 存在，亦即我們可改寫上式如下
\[\begin{array}{l}
{\left( {adj\left( A \right)} \right)^{ - 1}}\left( {\det A} \right)I = {\left( {adj\left( A \right)} \right)^{ - 1}}adj\left( A \right)A\\
\Rightarrow \left( {\det A} \right){\left( {adj\left( A \right)} \right)^{ - 1}} = A\\
\Rightarrow {\left( {adj\left( A \right)} \right)^{ - 1}} = \frac{1}{{\det A}}A
\end{array}\]
接著我們證明 $adj (A^{-1}) = \frac{1}{det A} A$
我們觀察
\[\begin{array}{l}
{A^{ - 1}} = \frac{1}{{\det A}}adj\left( A \right)\\
\Rightarrow A = \frac{1}{{\det \left( {{A^{ - 1}}} \right)}}adj\left( {{A^{ - 1}}} \right)\\
\Rightarrow A = \left( {\det A} \right)adj\left( {{A^{ - 1}}} \right)\\
\Rightarrow \frac{1}{{\det A}}A = adj\left( {{A^{ - 1}}} \right)
\end{array}\]
故綜合以上所述，我們可得 $(adj A)^{-1} = adj (A^{-1}) = \frac{1}{det A} A$ $\square$
10/18/2015
[測度論] 淺論 Ring 與 Algebra 及其性質 (0)
令 $X$ 為一集合，且 $\mathcal{P}(X)$ 表示為 $X$ 的 Power Set。
=================
Definition: 我們稱 $R \subset \mathcal{P}(X)$ 為一個 ring 若下列條件成立
1. $\emptyset \in R$
2. 若 $A,B \in R$ 則 $A\setminus B \in R$
3. 若 $A, B \in R$ 則 $A \cup B \in R$
================
以下我們看幾個 Ring 的性質：
令 $R$ 為一個 ring
=========
Property 1: 若 $A_1,...,A_n \in R$ 則 $\cup_{i=1}^n A_i \in R$
Proof:
由 $R$ 定義 (3) 可知 若 $A_1, A_2 \in R \Rightarrow A_1 \cup A_2 \in R$ ，故透過數學歸納法，假定 $A_1,...,A_n \in R \Rightarrow \cup_{i=1}^n A_i \in R$ 我們要證明
\[
A_1,...,A_n, A_{n+1} \in R \Rightarrow \cup_{i=1}^{n+1} A_i \in R
\]故 令 $B := \cup_{i=1}^n A_i $ 則
\[
\cup_{i=1}^{n+1} A_i = B\cup A_{n+1}
\]由於 $A_{n+1} \in R$ ，利用 $R$ 的定義 (3) 可知 $B \cup A_{n+1} \in R$ 故得證。 $\square$
========
Property 2: 若 $A, B \in R $ 則 $A \cap B \in R$
Proof:
注意到 $A \cap B = A \setminus (A\setminus B)$ ，又因為 $A \in R$ 且 $A\setminus B \in R$ 故利用 $R$ 的定義 (2) 可知
\[
A \setminus (A\setminus B) \in R \;\;\;\; \square
\]
=======
Property 3: 若 $A_1,...,A_n \in R$ 則 $\cap_{i=1}^n A_i \in R$
Proof:
此證明雷同於 Property 1，
首先回顧 Property 2 ，我們知道 $A_1, A_2 \in R \Rightarrow A_1 \cap A_2 \in R$ 故透過數學歸納法，假定 $A_1,...,A_n \in R \Rightarrow \cap_{i=1}^n A_i \in R$ 我們要證明
\[
A_1,...,A_n, A_{n+1} \in R \Rightarrow \cap_{i=1}^{n+1} A_i \in R
\]故 令 $B := \cap_{i=1}^n A_i $ 則
\[
\cap_{i=1}^{n+1} A_i = B\cap A_{n+1}
\]由於 $A_{n+1} \in R$ ，利用 Property (2) 可知 $B \cap A_{n+1} \in R$ 故得證。 $\square$
Property 4: $\mathcal{P}(X)$ 為 Ring
有了 Ring 的概念，我們便可以接著引入 algebra 的概念，以下給出定義：
=================
Definition: 我們稱 $R \subset \mathcal{P}(X)$ 為一個 algebra 若下列條件成立
1. $\emptyset \in R$
2. 若 $A,B \in R$ 則 $A\setminus B \in R$
3. 若 $A, B \in R$ 則 $A \cup B \in R$
4. $X \in R$
================
Comments:
1. 讀者可發現 algebra 僅僅比 Ring 多了第四個條件 $X \in R$。
2. 在抽象代數中所定義的 Ring 與這邊介紹的 Ring 可以互通有無，考慮向量空間 $(R, \oplus, \odot)$ ，若任取兩集合 $A,B \in R$ 我們定義其上的 加法 為 $A \oplus B := A \Delta B := (A \setminus B) \cup (B \setminus A)$ 且其上的 乘法 定為 $A \odot B := A \cap B$ 則可看出其滿足 抽象代數中所要求的 Ring 的加法與乘法的封閉性，亦即 $A \oplus B \in R$ 且 $A \odot B \in R$。有興趣的讀者可驗證剩餘的性質。
3. 前述對於 Ring 與 Algebra 是用來做之後推廣到 $\sigma$-Ring 與 $\sigma$-Algebra 的準備。我們介紹如下。
=================
Definition: 我們稱 $R \subset \mathcal{P}(X)$ 為一個 $\sigma$-ring 若下列條件成立
1. $\emptyset \in R$
2. 若 $A,B \in R$ 則 $A\setminus B \in R$
3. 若 $A_1, A_2,... \in R$ 則 $\cup_{n=1}^\infty A_n \in R$
================
=================
Definition: 我們稱 $R \subset \mathcal{P}(X)$ 為一個 $\sigma$-algebra 若下列條件成立
1. $\emptyset \in R$
2. 若 $A,B \in R$ 則 $A\setminus B \in R$
3. 若 $A_1, A_2,... \in R$ 則 $\cup_{n=1}^\infty A_n \in R$
4. $X \in R$
================
Comment:
1. 前述兩個定義所稱的 $\sigma$ 表達 可數多個 "countably many"
2. $\sigma$-algebra/ring 有多個等價定義，我們之後會再行介紹。
3. $\sigma$-algebra 必定為 $\sigma$-ring
4. 機率論中，$\sigma-$algebra 表示所有 事件 所形成的集合
接著我們考慮以下情況：通常我們不一定能得到 $\sigma$-algebra 或者 $\sigma$-ring 而是只有 $X$ 中的某些子集合 比如說 $D \subset P(X)$，那我們該如何得到 $\sigma$-algebra 或者 $\sigma$-ring?
令 $D \subset \mathcal{P}(X)$
Definition: 我們稱 Ring generated by $D$ 為如下定義 $\mathcal{R}(D) := \{\text{ intersection of all rings in $\mathcal{P}(X)$ which contains $D$}\}$
上述透過子集合產生的 Ring 可推廣到產生 $\sigma$-algebra 或者 $\sigma$-ring 。
=================
Definition: 我們稱 $R \subset \mathcal{P}(X)$ 為一個 ring 若下列條件成立
1. $\emptyset \in R$
2. 若 $A,B \in R$ 則 $A\setminus B \in R$
3. 若 $A, B \in R$ 則 $A \cup B \in R$
================
以下我們看幾個 Ring 的性質：
令 $R$ 為一個 ring
=========
Property 1: 若 $A_1,...,A_n \in R$ 則 $\cup_{i=1}^n A_i \in R$
Proof:
由 $R$ 定義 (3) 可知 若 $A_1, A_2 \in R \Rightarrow A_1 \cup A_2 \in R$ ，故透過數學歸納法，假定 $A_1,...,A_n \in R \Rightarrow \cup_{i=1}^n A_i \in R$ 我們要證明
\[
A_1,...,A_n, A_{n+1} \in R \Rightarrow \cup_{i=1}^{n+1} A_i \in R
\]故 令 $B := \cup_{i=1}^n A_i $ 則
\[
\cup_{i=1}^{n+1} A_i = B\cup A_{n+1}
\]由於 $A_{n+1} \in R$ ，利用 $R$ 的定義 (3) 可知 $B \cup A_{n+1} \in R$ 故得證。 $\square$
========
Property 2: 若 $A, B \in R $ 則 $A \cap B \in R$
Proof:
注意到 $A \cap B = A \setminus (A\setminus B)$ ，又因為 $A \in R$ 且 $A\setminus B \in R$ 故利用 $R$ 的定義 (2) 可知
\[
A \setminus (A\setminus B) \in R \;\;\;\; \square
\]
=======
Property 3: 若 $A_1,...,A_n \in R$ 則 $\cap_{i=1}^n A_i \in R$
Proof:
此證明雷同於 Property 1，
首先回顧 Property 2 ，我們知道 $A_1, A_2 \in R \Rightarrow A_1 \cap A_2 \in R$ 故透過數學歸納法，假定 $A_1,...,A_n \in R \Rightarrow \cap_{i=1}^n A_i \in R$ 我們要證明
\[
A_1,...,A_n, A_{n+1} \in R \Rightarrow \cap_{i=1}^{n+1} A_i \in R
\]故 令 $B := \cap_{i=1}^n A_i $ 則
\[
\cap_{i=1}^{n+1} A_i = B\cap A_{n+1}
\]由於 $A_{n+1} \in R$ ，利用 Property (2) 可知 $B \cap A_{n+1} \in R$ 故得證。 $\square$
有了 Ring 的概念，我們便可以接著引入 algebra 的概念，以下給出定義：
=================
Definition: 我們稱 $R \subset \mathcal{P}(X)$ 為一個 algebra 若下列條件成立
1. $\emptyset \in R$
2. 若 $A,B \in R$ 則 $A\setminus B \in R$
3. 若 $A, B \in R$ 則 $A \cup B \in R$
4. $X \in R$
================
Comments:
1. 讀者可發現 algebra 僅僅比 Ring 多了第四個條件 $X \in R$。
2. 在抽象代數中所定義的 Ring 與這邊介紹的 Ring 可以互通有無，考慮向量空間 $(R, \oplus, \odot)$ ，若任取兩集合 $A,B \in R$ 我們定義其上的 加法 為 $A \oplus B := A \Delta B := (A \setminus B) \cup (B \setminus A)$ 且其上的 乘法 定為 $A \odot B := A \cap B$ 則可看出其滿足 抽象代數中所要求的 Ring 的加法與乘法的封閉性，亦即 $A \oplus B \in R$ 且 $A \odot B \in R$。有興趣的讀者可驗證剩餘的性質。
3. 前述對於 Ring 與 Algebra 是用來做之後推廣到 $\sigma$-Ring 與 $\sigma$-Algebra 的準備。我們介紹如下。
=================
Definition: 我們稱 $R \subset \mathcal{P}(X)$ 為一個 $\sigma$-ring 若下列條件成立
1. $\emptyset \in R$
2. 若 $A,B \in R$ 則 $A\setminus B \in R$
3. 若 $A_1, A_2,... \in R$ 則 $\cup_{n=1}^\infty A_n \in R$
================
=================
Definition: 我們稱 $R \subset \mathcal{P}(X)$ 為一個 $\sigma$-algebra 若下列條件成立
1. $\emptyset \in R$
2. 若 $A,B \in R$ 則 $A\setminus B \in R$
3. 若 $A_1, A_2,... \in R$ 則 $\cup_{n=1}^\infty A_n \in R$
4. $X \in R$
================
Comment:
1. 前述兩個定義所稱的 $\sigma$ 表達 可數多個 "countably many"
2. $\sigma$-algebra/ring 有多個等價定義，我們之後會再行介紹。
3. $\sigma$-algebra 必定為 $\sigma$-ring
4. 機率論中，$\sigma-$algebra 表示所有 事件 所形成的集合
接著我們考慮以下情況：通常我們不一定能得到 $\sigma$-algebra 或者 $\sigma$-ring 而是只有 $X$ 中的某些子集合 比如說 $D \subset P(X)$，那我們該如何得到 $\sigma$-algebra 或者 $\sigma$-ring?
令 $D \subset \mathcal{P}(X)$
Definition: 我們稱 Ring generated by $D$ 為如下定義 $\mathcal{R}(D) := \{\text{ intersection of all rings in $\mathcal{P}(X)$ which contains $D$}\}$
上述透過子集合產生的 Ring 可推廣到產生 $\sigma$-algebra 或者 $\sigma$-ring 。
10/17/2015
[線性代數] 應用行列式計算三角形面積
考慮 $\mathbb{R}^2$空間 中的 頂點分別為 $(x_1, y_1), (x_2, y_2)$ 與 $(x_3,y_3)$ 的三角形 如下圖所示
則我們可以計算此三角形 $P_1P_2P_3$ 面積為
三角形$P_1P_2P_3$ 面積 = 梯形$AP_1P_2B$ 的面積 + 梯形 $BP_2P_3C$ 的面積 - 梯形 $A P_1 P_3 C$的面積
現在回憶 國/高中數學，梯形面積 $=$(上底 $+$ 下底) $\times$ 高 $/$ 2，故我們有
\[\begin{array}{l}
Area\left( {{P_1}{P_2}{P_3}} \right) = Area\left( {A{P_1}{P_2}B} \right) + Area\left( {B{P_2}{P_3}C} \right) - Area\left( {A{P_1}{P_3}C} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{1}{2}\left( {{y_1} + {y_2}} \right)\left( {{x_2} - {x_1}} \right) + \frac{1}{2}\left( {{y_3} + {y_2}} \right)\left( {{x_3} - {x_2}} \right) - \frac{1}{2}\left( {{y_3} + {y_1}} \right)\left( {{x_3} - {x_1}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{1}{2}\left( {{x_1}{y_3} - {x_1}{y_2} + {x_2}{y_1} - {x_2}{y_3} + {x_3}{y_2} - {x_3}{y_1}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = - \frac{1}{2}\left( {\left( {{x_2}{y_3} - {x_3}{y_2}} \right) - \left( {{x_1}{y_3} - {x_3}{y_1}} \right) + \left( {{x_1}{y_2} - {x_2}{y_1}} \right)} \right)
\end{array}\]但上述結果事實上剛好為 對下列矩陣的行列式 (讀者可自行驗證)
\[\left[ {\begin{array}{*{20}{c}}
{{x_1}}&{{y_1}}&1\\
{{x_2}}&{{y_2}}&1\\
{{x_3}}&{{y_3}}&1
\end{array}} \right]\]
注意到由於行列式有正負之分，故若我們在計算面積時，需加上絕對值保證其恆為正數，故對於 $\mathbb{R}^2$ 空間三角形 $\Delta$ 面積可透過下式計算：
\[Area\left( \Delta \right) = \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
{{x_1}}&{{y_1}}&1\\
{{x_2}}&{{y_2}}&1\\
{{x_3}}&{{y_3}}&1
\end{array}} \right]} \right)} \right|\]
Example 1:
試計算下圖中的三角形面積
Solution:
利用前述結果可得
\[\begin{array}{l}
Area\left( \Delta \right) = \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
{{x_1}}&{{y_1}}&1\\
{{x_2}}&{{y_2}}&1\\
{{x_3}}&{{y_3}}&1
\end{array}} \right]} \right)} \right|\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
2&6&1\\
{ - 1}&4&1\\
3&1&1
\end{array}} \right]} \right)} \right| = \frac{{17}}{2}
\end{array}\]
Example 2:
試計算下圖四邊形面積
Solution
注意到圖中四邊形面積可視為兩個三角形面積之和，故
\[\begin{array}{l}
Area = \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
2&6&1\\
{ - 1}&4&1\\
3&1&1
\end{array}} \right]} \right)} \right| + \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
2&6&1\\
6&3&1\\
3&1&1
\end{array}} \right]} \right)} \right|\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{{17}}{2} + \frac{{17}}{2} = 17
\end{array}\]
則我們可以計算此三角形 $P_1P_2P_3$ 面積為
三角形$P_1P_2P_3$ 面積 = 梯形$AP_1P_2B$ 的面積 + 梯形 $BP_2P_3C$ 的面積 - 梯形 $A P_1 P_3 C$的面積
現在回憶 國/高中數學，梯形面積 $=$(上底 $+$ 下底) $\times$ 高 $/$ 2，故我們有
\[\begin{array}{l}
Area\left( {{P_1}{P_2}{P_3}} \right) = Area\left( {A{P_1}{P_2}B} \right) + Area\left( {B{P_2}{P_3}C} \right) - Area\left( {A{P_1}{P_3}C} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{1}{2}\left( {{y_1} + {y_2}} \right)\left( {{x_2} - {x_1}} \right) + \frac{1}{2}\left( {{y_3} + {y_2}} \right)\left( {{x_3} - {x_2}} \right) - \frac{1}{2}\left( {{y_3} + {y_1}} \right)\left( {{x_3} - {x_1}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{1}{2}\left( {{x_1}{y_3} - {x_1}{y_2} + {x_2}{y_1} - {x_2}{y_3} + {x_3}{y_2} - {x_3}{y_1}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = - \frac{1}{2}\left( {\left( {{x_2}{y_3} - {x_3}{y_2}} \right) - \left( {{x_1}{y_3} - {x_3}{y_1}} \right) + \left( {{x_1}{y_2} - {x_2}{y_1}} \right)} \right)
\end{array}\]但上述結果事實上剛好為 對下列矩陣的行列式 (讀者可自行驗證)
\[\left[ {\begin{array}{*{20}{c}}
{{x_1}}&{{y_1}}&1\\
{{x_2}}&{{y_2}}&1\\
{{x_3}}&{{y_3}}&1
\end{array}} \right]\]
注意到由於行列式有正負之分，故若我們在計算面積時，需加上絕對值保證其恆為正數，故對於 $\mathbb{R}^2$ 空間三角形 $\Delta$ 面積可透過下式計算：
\[Area\left( \Delta \right) = \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
{{x_1}}&{{y_1}}&1\\
{{x_2}}&{{y_2}}&1\\
{{x_3}}&{{y_3}}&1
\end{array}} \right]} \right)} \right|\]
Example 1:
試計算下圖中的三角形面積
利用前述結果可得
\[\begin{array}{l}
Area\left( \Delta \right) = \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
{{x_1}}&{{y_1}}&1\\
{{x_2}}&{{y_2}}&1\\
{{x_3}}&{{y_3}}&1
\end{array}} \right]} \right)} \right|\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
2&6&1\\
{ - 1}&4&1\\
3&1&1
\end{array}} \right]} \right)} \right| = \frac{{17}}{2}
\end{array}\]
Example 2:
試計算下圖四邊形面積
Solution
注意到圖中四邊形面積可視為兩個三角形面積之和，故
\[\begin{array}{l}
Area = \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
2&6&1\\
{ - 1}&4&1\\
3&1&1
\end{array}} \right]} \right)} \right| + \frac{1}{2}\left| {\det \left( {\left[ {\begin{array}{*{20}{c}}
2&6&1\\
6&3&1\\
3&1&1
\end{array}} \right]} \right)} \right|\\
\begin{array}{*{20}{c}}
{}&{}&{}
\end{array} = \frac{{17}}{2} + \frac{{17}}{2} = 17
\end{array}\]
10/12/2015
[轉載] How to Study Math? -Paul R. Halmos
Don't just read it; fight it! Ask your own questions, look for your own examples, discover your own proofs. Is the hypothesis necessary? Is the converse true? What happens in the classical special case? What about the degenerate cases? Where does the proof use the hypothesis?
9/30/2015
[線性代數] 淺論座標
令 $V$ 為 $n$ 維向量空間，則我們知道 $V$ 有基底 (basis) $S$ 且其元素為 $n$ 維向量。現在我們定義 $S :=\{{\bf v}_1,...{\bf v}_n\}$ 為向量空間 $V$ 的一組有序基底 (ordered basis) 則任意向量 ${\bf v} \in V$ 可由上述有序基底唯一表示成以下的線性組合形式：
\[
{\bf v} = a_1 {\bf v}_1 + a_2 {\bf v}_2 + ... + a_n {\bf v}_n
\]其中 $a_1,...a_n \in \mathbb{R}^1$
Definition: Coordinate Vector
定義 ${\bf v}$ 對應於有序基底 $S$ 的座標向量 (coordinate vector) 為
\[{[{\bf{v}}]_S}: = \left[ \begin{array}{l}
{a_1}\\
{a_2}\\
\vdots \\
{a_n}
\end{array} \right]\] 且其中 $[{\bf v}]_S$ 的元素 $a_i$ 稱之為 ${\bf v}$ 對應於有序基底的座標。
Example 1:
考慮向量空間 \[
V:= P_2 := \{p(t) = a_2t^2 + a_1t + a_0: a_2,a_1,a_0 \in \mathbb{R}^2\}
\]且令基底 $S= \{t^2, t, 1\}$ 現考慮 ${\bf v}:= p(t) = \alpha t^2 + \alpha t^1 + \alpha$ 求 $[{\bf v}]_S = ?$
Solution
注意到 ${\bf v}:= p(t) = \alpha t^2 + \alpha t^1 + \alpha$，暫稱此式為 $(*)$ 又因為 ${\bf v} \in V$ 故由 $\bf v$ 可由 $S$ 的有序基底 $\{ t^2, t,1\}$ 透過線性組合唯一表示：也就是說
\[{\bf{v}} = p\left( t \right) \in {P_2} \Leftrightarrow {\bf v} = {a_2}{t^2} + {a_1}t + {a_0} \;\;\;\;\; (\star)
\]
故比較 $(*)$ 與 $(\star)$ 兩式
\[{a_2}{t^2} + {a_1}t + {a_0} = \alpha {t^2} + \alpha t + \alpha
\] 可得 $a_2 = \alpha$， $a_1 = \alpha$ 與 $a_0 = \alpha$ 故
\[{[{\bf{v}}]_S}: = \left[ \begin{array}{l}
{a_2}\\
{a_1}\\
{a_0}
\end{array} \right] = \left[ \begin{array}{l}
\alpha \\
\alpha \\
\alpha
\end{array} \right]\]
Example 2:
考慮向量空間 \[
V:= P_2 := \{p(t) = a_2t^2 + a_1t + a_0: a_2,a_1,a_0 \in \mathbb{R}^2\}
\]且令基底 $S= \{t^2-t+1, t+1, 1^2+1\}$ 現考慮 ${\bf v}:= p(t) = 4 t^2 -2 t^1 + 3$ 求 $[{\bf v}]_S = ?$
Solution:
令 \[{\left[ {\bf{v}} \right]_S} = \left[ \begin{array}{l}
{a_1}\\
{a_2}\\
{a_3}
\end{array} \right]\]且 $\bf v$ 可透過基底 $S$ 做線性組合
\[\begin{array}{l}
{\bf{v}} = {a_1}{{\bf{v}}_1} + {a_2}{{\bf{v}}_2} + {a_3}{{\bf{v}}_3}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {a_1}\left( {{t^2} - t + 1} \right) + {a_2}\left( {t + 1} \right) + {a_3}\left( {{t^2} + 1} \right)\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{a_1} + {a_3}} \right){t^2} + \left( {{a_2} - {a_1}} \right)t + \left( {{a_1} + {a_2} + {a_3}} \right)\;\;\;\; (*)
\end{array}\]又因為
\[
{\bf v} = 4t^2 -2t +3 \;\;\;\; (\star)
\]故比較 $(\star)$ 與 $(*)$ 係數可得
\[\begin{gathered}
4{t^2} - 2t + 3 = \left( {{a_1} + {a_3}} \right){t^2} + \left( {{a_2} - {a_1}} \right)t + \left( {{a_1} + {a_2} + {a_3}} \right) \hfill \\
\Rightarrow \left\{ \begin{gathered}
{a_1} + {a_3} = 4 \hfill \\
{a_2} - {a_1} = - 2 \hfill \\
{a_1} + {a_2} + {a_3} = 3 \hfill \\
\end{gathered} \right. \hfill \\
\end{gathered} \]
將上式改寫成矩陣求解 $a_1, a_2, a_3$如下
\[\begin{gathered}
\left[ {\begin{array}{*{20}{c}}
1&0&1 \\
{ - 1}&1&0 \\
1&1&1
\end{array}} \right]\left[ \begin{gathered}
{a_1} \hfill \\
{a_2} \hfill \\
{a_3} \hfill \\
\end{gathered} \right] = \left[ \begin{gathered}
4 \hfill \\
- 2 \hfill \\
3 \hfill \\
\end{gathered} \right] \hfill \\
\Rightarrow \left\{ \begin{gathered}
{a_1} = 1 \hfill \\
{a_2} = - 1 \hfill \\
{a_3} = 3 \hfill \\
\end{gathered} \right. \hfill \\
\end{gathered} \]
\[
{\bf v} = a_1 {\bf v}_1 + a_2 {\bf v}_2 + ... + a_n {\bf v}_n
\]其中 $a_1,...a_n \in \mathbb{R}^1$
Definition: Coordinate Vector
定義 ${\bf v}$ 對應於有序基底 $S$ 的座標向量 (coordinate vector) 為
\[{[{\bf{v}}]_S}: = \left[ \begin{array}{l}
{a_1}\\
{a_2}\\
\vdots \\
{a_n}
\end{array} \right]\] 且其中 $[{\bf v}]_S$ 的元素 $a_i$ 稱之為 ${\bf v}$ 對應於有序基底的座標。
Example 1:
考慮向量空間 \[
V:= P_2 := \{p(t) = a_2t^2 + a_1t + a_0: a_2,a_1,a_0 \in \mathbb{R}^2\}
\]且令基底 $S= \{t^2, t, 1\}$ 現考慮 ${\bf v}:= p(t) = \alpha t^2 + \alpha t^1 + \alpha$ 求 $[{\bf v}]_S = ?$
Solution
注意到 ${\bf v}:= p(t) = \alpha t^2 + \alpha t^1 + \alpha$，暫稱此式為 $(*)$ 又因為 ${\bf v} \in V$ 故由 $\bf v$ 可由 $S$ 的有序基底 $\{ t^2, t,1\}$ 透過線性組合唯一表示：也就是說
\[{\bf{v}} = p\left( t \right) \in {P_2} \Leftrightarrow {\bf v} = {a_2}{t^2} + {a_1}t + {a_0} \;\;\;\;\; (\star)
\]
故比較 $(*)$ 與 $(\star)$ 兩式
\[{a_2}{t^2} + {a_1}t + {a_0} = \alpha {t^2} + \alpha t + \alpha
\] 可得 $a_2 = \alpha$， $a_1 = \alpha$ 與 $a_0 = \alpha$ 故
\[{[{\bf{v}}]_S}: = \left[ \begin{array}{l}
{a_2}\\
{a_1}\\
{a_0}
\end{array} \right] = \left[ \begin{array}{l}
\alpha \\
\alpha \\
\alpha
\end{array} \right]\]
Example 2:
考慮向量空間 \[
V:= P_2 := \{p(t) = a_2t^2 + a_1t + a_0: a_2,a_1,a_0 \in \mathbb{R}^2\}
\]且令基底 $S= \{t^2-t+1, t+1, 1^2+1\}$ 現考慮 ${\bf v}:= p(t) = 4 t^2 -2 t^1 + 3$ 求 $[{\bf v}]_S = ?$
Solution:
令 \[{\left[ {\bf{v}} \right]_S} = \left[ \begin{array}{l}
{a_1}\\
{a_2}\\
{a_3}
\end{array} \right]\]且 $\bf v$ 可透過基底 $S$ 做線性組合
\[\begin{array}{l}
{\bf{v}} = {a_1}{{\bf{v}}_1} + {a_2}{{\bf{v}}_2} + {a_3}{{\bf{v}}_3}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {a_1}\left( {{t^2} - t + 1} \right) + {a_2}\left( {t + 1} \right) + {a_3}\left( {{t^2} + 1} \right)\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{a_1} + {a_3}} \right){t^2} + \left( {{a_2} - {a_1}} \right)t + \left( {{a_1} + {a_2} + {a_3}} \right)\;\;\;\; (*)
\end{array}\]又因為
\[
{\bf v} = 4t^2 -2t +3 \;\;\;\; (\star)
\]故比較 $(\star)$ 與 $(*)$ 係數可得
\[\begin{gathered}
4{t^2} - 2t + 3 = \left( {{a_1} + {a_3}} \right){t^2} + \left( {{a_2} - {a_1}} \right)t + \left( {{a_1} + {a_2} + {a_3}} \right) \hfill \\
\Rightarrow \left\{ \begin{gathered}
{a_1} + {a_3} = 4 \hfill \\
{a_2} - {a_1} = - 2 \hfill \\
{a_1} + {a_2} + {a_3} = 3 \hfill \\
\end{gathered} \right. \hfill \\
\end{gathered} \]
將上式改寫成矩陣求解 $a_1, a_2, a_3$如下
\[\begin{gathered}
\left[ {\begin{array}{*{20}{c}}
1&0&1 \\
{ - 1}&1&0 \\
1&1&1
\end{array}} \right]\left[ \begin{gathered}
{a_1} \hfill \\
{a_2} \hfill \\
{a_3} \hfill \\
\end{gathered} \right] = \left[ \begin{gathered}
4 \hfill \\
- 2 \hfill \\
3 \hfill \\
\end{gathered} \right] \hfill \\
\Rightarrow \left\{ \begin{gathered}
{a_1} = 1 \hfill \\
{a_2} = - 1 \hfill \\
{a_3} = 3 \hfill \\
\end{gathered} \right. \hfill \\
\end{gathered} \]
訂閱：
文章 (Atom)
-
數學上的 if and only if ( 此文不討論邏輯學中的 if and only if，只討論數學上的 if and only if。) 中文翻譯叫做 若且唯若 (or 當且僅當) ， 記得當初剛接觸這個詞彙的時候，我是完全不明白到底是甚麼意思，查了翻譯也是愛...
-
這次要介紹的是數學上一個重要的概念： Norm： 一般翻譯成 範數 (在英語中 norm 有規範的意思，比如我們說normalization就是把某種東西/物品/事件 做 正規化，也就是加上規範使其正常化)，不過個人認為其實翻譯成 範數 也是看不懂的...這邊建議把 No...
-
半導體中的電流是由電子(electron)及電洞(hole)兩種載子(carrier)移動所產生 載子移動的方式： 擴散(diffusion) $\Rightarrow$ 擴散電流 (不受外力電場作用) 飄移(drift) $\Rightarrow$ 飄移電流 (受外...