這次要介紹 有限維度的內積空間 (Inner Product Space)，簡而言之就是有限維度向量空間 $(V, \oplus, \odot)$ 上額外定義內積運算，則我們稱此類空間為 內積空間。
Comments:
1. 有限維度內積空間稱為
歐式空間 (Euclidean Space)
2. 若為無窮維度的內積空間我們稱為
Pre-Hilbert Space，若此無窮維度內積空間為完備空間，則稱之為
Hilbert Space
3. 為何好好的向量空間不用還要多此一舉另外又定一個 內積空間？主因是向量空間本身只定義了加法 與純量乘法的運算，如果我們想討論在向量空間中某元素的大小 或者 某兩元素之間的關係則無從得知。但是如果我們引入 內積運算 到向量空間中，則可以在原本的向量空間上將 代數 與 幾何 的概念做直接的連結，也就是我們可以透過內積引入 其上的兩元素是否 垂直 (正交) 的概念，亦可針對某元素來探討其 長度與大小 概念 。
4. 讀者可回憶 高中所學習過的 點積 (dot product)，此文所探討的內積 即為 點積 的推廣。
首先定義內積
==================
Definition: Inner Product on Vector Space
令 $V$ 為實數向量空間，則 Inner Product on $V$ 為函數 $(\cdot, \cdot): V\times V \to \mathbb{R}$ 滿足下列條件
(a) $({\bf u}, {\bf u}) \ge 0$：且 $({\bf u}, {\bf u}) = 0$ 若且唯若 ${\bf u} = {\bf 0}_V$
(b) $({\bf u}, {\bf v}) = ({\bf v}, {\bf u}), \; \forall {\bf u,v} \in V$
(c) $({\bf u} + {\bf v}, {\bf w}) = ({\bf u}, {\bf w}) + ({\bf u}, {\bf v}), \; \forall {\bf u,v,w} \in V$
(d) $(c {\bf u}, {\bf v}) = c({\bf u}, {\bf v}),\; \forall {\bf u,v} \in V, c \in \mathbb{R}$
==================
Comment:
1. 透過內積我們亦可定義 ${\bf u}$ 的大小，記作 $||{\bf u}|| = \sqrt{ ({\bf u}, {\bf u} )} $
2. 透過內積我們亦可定義在內積空間中兩向量是否垂直：亦即 若 ${\bf u}, {\bf v} \in V$ 稱為 垂直 或 正交 若 $({\bf u}, {\bf v}) = 0$
2. 前述內積定義可立即有以下衍生結果：
Fact 1: $\left( {{\bf{u}},c{\bf{v}}} \right) = c\left( {{\bf{u}},{\bf{v}}} \right)$
Fact 2: $\left( {{\bf{u}},{\bf{v}} + {\bf{w}}} \right) = \left( {{\bf{u}},{\bf{v}}} \right) + \left( {{\bf{u}},{\bf{w}}} \right)$
讀者可自行驗證上述兩個結果。
=========
Claim 1: 給定 ${\bf v}, {\bf w} \in V$ ，若對任意 ${\bf u} \in V$ 我們有 $
({\bf u},{\bf v}) = ({\bf u}, {\bf w})$ 則 ${\bf v} = {\bf w}$
=========
Proof:
由於我們要證明 ${\bf v} = {\bf w}$，故我們僅需證明 $\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = 0$ 現在觀察兩者之差的內積
\[\begin{array}{l}
\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) + \left( {{\bf{v}} - {\bf{w}}, - {\bf{w}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) - \left( {{\bf{v}} - {\bf{w}},{\bf{w}}} \right)
\end{array}\]注意到 ${\bf v} - {\bf w} \in V$ 故 若我們令 ${\bf u}:= {\bf v} - {\bf w} $ 則 由已知條件可知
\[\begin{array}{l}
\left( {{\bf{v}} - {\bf{w}},{\bf{v}} - {\bf{w}}} \right) = \left( {{\bf{v}} - {\bf{w}},{\bf{v}}} \right) - \left( {{\bf{v}} - {\bf{w}},{\bf{w}}} \right)\\
\begin{array}{*{20}{c}}
{}&{}&{}&{}&{}&{}
\end{array} = \left( {{\bf{u}},{\bf{v}}} \right) - \left( {{\bf{u}},{\bf{w}}} \right) = 0. \;\;\; \square
\end{array}\]
由上述結果我們有以下衍生定理
=========
Corollary of Claim 1:
若 對任意 ${\bf u} \in V$ 我們有 $({\bf u}, {\bf v})=0$ 則 ${\bf v} = 0$
=========
Proof: omitted.
=========
Claim: 令向量空間 $ V := \mathbb{R}^n$ 現在定義下列函數 $f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ 滿足
\[
f({\bf u},{\bf v}) := {\bf u}^T {\bf v}
\]其中 ${\bf u} := [u_1\;u_2\;...,u_n]^T $ 與 ${\bf v} := [v_1\;\;v_2\;...,v_n]^T $ 為任意 $\mathbb{R}^n$ 空間中的向量， 則此運算為一種內積。
=========
Proof:
我們現在驗證 $f$ 確實為內積，故我們需驗證其滿足前述四項 (a,b,c,d)條件：首先驗證 $(a)$:
\[f({\bf{u}},{\bf{u}}) = {{\bf{u}}^T}{\bf{u}} = u_1^2 + ...u_2^2 \ge 0\]且 $f({\bf{u}},{\bf{u}}) = 0$ 若且唯若 ${\bf u} = [0,...,0]^T$
$(b)$ 令 ${\bf u,v} \in \mathbb{R}^n$ 現在我們觀察
\[\begin{array}{l}
f({\bf{v}},{\bf{u}}) = {{\bf{v}}^T}{\bf{u}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {v_1}{u_1} + ... + {v_n}{u_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {u_1}{v_1} + ... + {u_n}{v_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right] = {{\bf{u}}^T}{\bf{v}} = f\left( {{\bf{u}},{\bf{v}}} \right)
\end{array}
\]
$(c)$ 令 $\bf u,v,w$$\in \mathbb{R}^n$ 接著我們觀察
\[\begin{array}{l}
f({\bf{u}} + {\bf{v}},{\bf{w}}) = {\left( {{\bf{u}} + {\bf{v}}} \right)^T}{\bf{w}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1} + {v_1}}&{{u_2} + {v_2}}&{...}&{{u_n} + {v_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{u_1} + {v_1}} \right){w_1} + ...\left( {{u_n} + {v_n}} \right){w_n}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left( {{u_1}{w_1} + ... + {u_n}{w_n}} \right) + \left( {{v_1}{w_1} + ... + {v_n}{w_n}} \right)\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right] + \left[ {\begin{array}{*{20}{c}}
{{v_1}}&{{v_2}}&{...}&{{v_n}}
\end{array}} \right]\left[ \begin{array}{l}
{w_1}\\
{w_2}\\
\vdots \\
{w_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = {\left( {\bf{u}} \right)^T}{\bf{w}} + {\left( {\bf{v}} \right)^T}{\bf{w}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = f({\bf{u}},{\bf{w}}) + f({\bf{v}},{\bf{w}})
\end{array}
\]
$(d)$ 同理我們觀察
\[\begin{array}{l}
f(c{\bf{u}},{\bf{v}}) = {\left( {c{\bf{u}}} \right)^T}{\bf{v}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = \left[ {\begin{array}{*{20}{c}}
{c{u_1}}&{c{u_2}}&{...}&{c{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c\left[ {\begin{array}{*{20}{c}}
{{u_1}}&{{u_2}}&{...}&{{u_n}}
\end{array}} \right]\left[ \begin{array}{l}
{v_1}\\
{v_2}\\
\vdots \\
{v_n}
\end{array} \right]\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = c{\left( {\bf{u}} \right)^T}{\bf{v}}\\
\begin{array}{*{20}{c}}
{}&{}
\end{array} = cf({\bf{u}},{\bf{v}})
\end{array}\]
Claim 2: 令向量空間 $ V := \mathbb{R}^n$ 現在定義下列函數 $f: \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ 滿足
\[
f({\bf u},{\bf v}) := {\bf u}^T {\bf v}
\]其中 ${\bf u} := [u_1\;u_2\;...,u_n]^T $ 與 ${\bf v} := [v_1\;\;v_2\;...,v_n]^T $ 為任意 $\mathbb{R}^n$ 空間中的向量， 則此運算為 $\mathbb{R}^n$ 一種內積。
Proof: omitted
Claim 3: 令向量空間 $ V := C[a,b]$ ，若 $f,g \in V$ 令
\[\left( {f,g} \right): = \int_a^b {f\left( t \right)g\left( t \right)dt} \] 則 $(f,g)$ 為 $C[a,b]$ 上的一種內積。
Proof: omitted
以下我們接著介紹對任意有限維度向量空間，則其上的內積可以用一個透過基底表示的矩陣 $C$ 來完全決定。
==================
Theorem: 令 $S=\{{\bf u}_1,...,{\bf u}_n\}$ 為 向量空間 $V$ 的 ordered basis 且假設我們可在 $V$ 上定義內積，現在令 $c_{ij} = ({\bf u}_i, {\bf u}_j)$ 且 $C=[c_{ij}]$ 矩陣 則
對任意 ${\bf v,w} \in V$， 存在 $C = [c_{ij}]$ 矩陣 使得 $({\bf v},{\bf w}) = [{\bf v}]_S^T C [{\bf w}]_S$
==================
Proof : 給定
${\bf v,w} \in V$ 則我們有
\[\begin{array}{l}
{\bf{v}} = {a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n}\\
{\bf{w}} = {b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}
\end{array}
\]
現在我們觀察內積 \[\begin{array}{l}
({\bf{v}},{\bf{w}}) = ({a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n},{\bf{w}})\\
= ({a_1}{{\bf{u}}_1},{\bf{w}}) + ({a_2}{{\bf{u}}_2},{\bf{w}}) + ... + ({a_n}{{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{\bf{w}}) + {a_2}({{\bf{u}}_2},{\bf{w}}) + ... + {a_n}({{\bf{u}}_n},{\bf{w}})
\end{array}\]又因為 ${\bf{w}} = {b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}$ 故
\[\begin{array}{l}
({\bf{v}},{\bf{w}}) = ({a_1}{{\bf{u}}_1} + ... + {a_n}{{\bf{u}}_n},{\bf{w}})\\
= ({a_1}{{\bf{u}}_1},{\bf{w}}) + ({a_2}{{\bf{u}}_2},{\bf{w}}) + ... + ({a_n}{{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{\bf{w}}) + {a_2}({{\bf{u}}_2},{\bf{w}}) + ... + {a_n}({{\bf{u}}_n},{\bf{w}})\\
= {a_1}({{\bf{u}}_1},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}) + {a_2}({{\bf{u}}_2},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n}) + ... + {a_n}({{\bf{u}}_n},{b_1}{{\bf{u}}_1} + ... + {b_n}{{\bf{u}}_n})\\
= \left[ {{a_1}({{\bf{u}}_1},{b_1}{{\bf{u}}_1}) + {a_1}({{\bf{u}}_1},{b_2}{{\bf{u}}_2}) + ... + {a_1}({{\bf{u}}_1},{b_n}{{\bf{u}}_n})} \right] + \\
\begin{array}{*{20}{c}}
{}&{}
\end{array}... + \left[ {{a_n}({{\bf{u}}_n},{b_1}{{\bf{u}}_1}) + {a_n}({{\bf{u}}_n},{b_2}{{\bf{u}}_2})... + {a_n}({{\bf{u}}_n},{b_n}{{\bf{u}}_n})} \right]\\
= \left[ {{a_1}{b_1}({{\bf{u}}_1},{{\bf{u}}_1}) + {a_1}{b_2}({{\bf{u}}_1},{{\bf{u}}_2}) + ... + {a_1}{b_n}({{\bf{u}}_1},{{\bf{u}}_n})} \right] + \\
\begin{array}{*{20}{c}}
{}&{}
\end{array}... + \left[ {{a_n}{b_1}({{\bf{u}}_n},{{\bf{u}}_1}) + {a_n}{b_2}({{\bf{u}}_n},{{\bf{u}}_2})... + {a_n}{b_n}({{\bf{u}}_n},{{\bf{u}}_n})} \right]\\
= \sum\limits_{j = 1}^n {{a_1}{b_j}({{\bf{u}}_1},{{\bf{u}}_j})} + \sum\limits_{j = 1}^n {{a_2}{b_j}({{\bf{u}}_2},{{\bf{u}}_j})} + ... + \sum\limits_{j = 1}^n {{a_n}{b_j}({{\bf{u}}_n},{{\bf{u}}_j})} \\
= \sum\limits_{j = 1}^n {\left( {{a_1}{b_j}({{\bf{u}}_1},{{\bf{u}}_j}) + {a_2}{b_j}({{\bf{u}}_2},{{\bf{u}}_j}) + ... + {a_n}{b_j}({{\bf{u}}_n},{{\bf{u}}_j})} \right)} \\
= \sum\limits_{j = 1}^n {\sum\limits_{i = 1}^n {{a_i}{b_j}({{\bf{u}}_i},{{\bf{u}}_j})} }
\end{array}\]
現在令 $c_{ij} = (a_i, b_j)$ ，則我們有
\[({\bf{v}},{\bf{w}}) = \sum\limits_{j = 1}^n {\sum\limits_{i = 1}^n {{a_i}{b_j}{c_{ij}}} } =[{\bf v}]_S C [{\bf w}]_S\]
Comments:
1. 前述的 $C$ 矩陣亦稱為 matrix of the inner produce with respect to the ordered basis $S$
2. 由於我們有 $c_{ij} = ({\bf u}_i, {\bf u}_j)$ 且利用內積定義 $ ({\bf u}_i, {\bf u}_j)=({\bf u}_j, {\bf u}_i) $ 我們得到 $c_{ij} = c_{ji}$，故 前述的 $C = [c_{ij}]$ 矩陣為對稱矩陣。
上述 inner product 不但可用來引出 norm 的概念，更保持了連續性，以下我們給出相關結果。
============================
Lemma: (Continuity of Inner Product)
令 $u_n \to u$ 且 $v_n \to v$ 且收斂在某內積空間 (或 Pre-Hibert Space) ，則
\[
(u_n,v_n) \to (u,v)
\] ============================
Proof: 直接觀察
\begin{align*}
\left| {({u_n},{v_n}) - (u,v)} \right| &= \left| {({u_n},{v_n}) - \left( {{u_n},v} \right) + \left( {{u_n},v} \right) - (u,v)} \right| \hfill \\
& \leqslant \left| {({u_n},{v_n}) - \left( {{u_n},v} \right)} \right| + \left| {\left( {{u_n},v} \right) - (u,v)} \right| \hfill \\
&= \left| {({u_n},{v_n} - v)} \right| + \left| {\left( {{u_n} - u,v} \right)} \right| \hfill \\
\end{align*} 回憶 Cauchy Schwarz Inequality $|(x,y)| \leq ||x|| ||y||$，我們有
\[\left| {({u_n},{v_n}) - (u,v)} \right| \leqslant \left\| {{u_n}} \right\|\left\| {{v_n} - v} \right\| + \left\| {{u_n} - u} \right\|\left\| v \right\|\;\;\;\; (*)
\]注意到因為 $\{u_n\}$數列為收斂數列，故 $||u_n|| < \infty$，另外 $v$ 為 收斂數列 $\{v_n\}$ 的極限，故 $||v|| < \infty$。除此之外，由假設 $u_n \to u$ 且 $v_n \to v$ 可知
\[
||u_n - u|| \to 0;\;\;\;\;\; ||v_n \to v|| \to 0
\]因此，上式 $(*)$ 取極限後可得
\[\left| {({u_n},{v_n}) - (u,v)} \right| \leqslant \underbrace {\left\| {{u_n}} \right\|}_{ < \infty }\underbrace {\left\| {{v_n} - v} \right\|}_{ \to 0} + \underbrace {\left\| {{u_n} - u} \right\|}_{ \to 0}\underbrace {\left\| v \right\|}_{ < \infty } \to 0 + 0 = 0\]
