SOME NOTES ON THE SPECTRAL THEOREM

1. Introduction

These are notes for an introductory graduate algebra class explaining a spectral theorem: a self adjoint operator on a real inner product space or a normal operator on a complex Hermitian space is always diagonalizable and the eigenspaces are orthogonal.

2. Inner Product Spaces and Hermitian Spaces

The theory has one formulation in terms of real inner product spaces and one in terms of complex Hermitian spaces. In fact the real case is naturally encompassed by the complex case. But since the real case is so often used, we will give both formulations starting with the real case.

Let V be a real vector space. A real pairing ⟨·, ·⟩ on V,

⟨·, ·⟩ : V × V → ℝ, (v⃗, w⃗) ↦ ⟨v⃗, w⃗⟩

is symmetric if for every v⃗, w⃗ in V,

⟨w⃗, v⃗⟩ = ⟨v⃗, w⃗⟩.

A pairing which is symmetric turns out to be bilinear if and only if for every v⃗_{1}, v⃗_{2}, w⃗ in V and for every scalar c in ℝ,

⟨cv⃗_{1} + v⃗_{2}, w⃗⟩ = c · ⟨v⃗_{1}, w⃗⟩ + ⟨v⃗_{2}, w⃗⟩.

For every v⃗ in V, define ∥v⃗∥^{2} by

∥v⃗∥^{2} := ⟨v⃗, v⃗⟩.

A symmetric, bilinear pairing is positive definite if for every nonzero v⃗ in V, ∥v⃗∥^{2} is a positive real number. In this case define ∥v⃗∥ by

∥v⃗∥ := \sqrt{∥v⃗∥^{2}} = \sqrt{⟨v⃗, v⃗⟩}.

Definition 2.1. A real inner product space is a pair (V, ⟨·, ·⟩) of a real vector space V and a symmetric, bilinear, positive definite pairing ⟨·, ·⟩ on V.

Next let V be a ℂ-vector space. A complex pairing ⟨·, ·⟩ on V,

⟨·, ·⟩ : V × V → ℂ, (v⃗, w⃗) ↦ ⟨v⃗, w⃗⟩

is conjugate symmetric if for every v⃗, w⃗ in V,

\overline{⟨w⃗, v⃗⟩} = ⟨v⃗, w⃗⟩.

A pairing which is conjugate symmetric turns out to be sesquilinear (or “halflinear”) if and only if for every v⃗_{1}, v⃗_{2}, w⃗ in V and for every scalar c in ℝ,

⟨cv⃗_{1} + v⃗_{2}, w⃗⟩ = c · ⟨v⃗_{1}, w⃗⟩ + ⟨v⃗_{2}, w⃗⟩.

Date: November 5, 2009.

As above define ∥v⃗∥^{2} by

∥v⃗∥^{2} := ⟨v⃗, v⃗⟩.

A conjugate symmetric, sesquilinear pairing is positive definite if for every nonzero v⃗ in V, ∥v⃗∥^{2} is a positive real number. In this case define ∥v⃗∥ by

∥v⃗∥ := \sqrt{∥v⃗∥^{2}} = \sqrt{⟨v⃗, v⃗⟩}.

Definition 2.2. A complex Hermitian space is a pair (V, ⟨·, ·⟩) of a complex vector space V and a conjugate symmetric, sesquilinear, positive definite pairing ⟨·, ·⟩ on V.

In both the case of a real inner product space and a complex Hermitian space, the (conjugate) symmetry of the pairing and the additivity of the first argument imply the additivity of the second argument,

⟨v⃗, w⃗_{1} + w⃗_{2}⟩ = ⟨v⃗, w⃗_{1}⟩ + ⟨v⃗, w⃗_{2}⟩.

And in the case of a real inner product space, symmetry and scaling in the first argument imply scaling in the second argument,

⟨v⃗, c · w⃗⟩ = c · ⟨v⃗, w⃗⟩.

Thus a real symmetric pairing which satisfies the condition above is ℝ-bilinear in the usual sense. However, in the case of a complex Hermitian space, complex conjugation intervenes,

⟨v⃗, c · w⃗⟩ = \overline{c}⟨v⃗, w⃗⟩.

A pairing which is linear in the first argument but “conjugate linear” in the second is sesquilinear. For a conjugate symmetric pairing, linearity in the first argument implies sesquilinearity.

Proposition 2.3 (Cauchy-Bunyakovsky-Schwarz inequality). For every pair of elements v⃗, w⃗ of V,

|⟨v⃗, w⃗⟩|^{2} ≤ ∥v⃗∥^{2} · ∥w⃗∥^{2}

and equality holds if and only if v⃗, w⃗ are linearly dependent.

Proof. If v⃗ and w⃗ are linearly dependent, i.e., if one is a scalar multiple of the other, then it is straightforward to verify that the inequality is an equality. Similarly, if ⟨v⃗, w⃗⟩ equals 0, then the proposition trivially follows. Therefore assume that v⃗, w⃗ are linearly independent and assume that ⟨v⃗, w⃗⟩ is nonzero.

Define c to the be the scalar (complex in the Hermitian case) of unit length defined by

c = \frac{|⟨v⃗, w⃗⟩|}{⟨v⃗, w⃗⟩}.

In particular, the scalar c⟨v⃗, w⃗⟩ is a positive real number. Consider the quadratic polynomial in the real variable t,

q(t) = ⟨cv⃗ + tw⃗, cv⃗ + tw⃗⟩.

On the one hand, expanding out gives

q(t) = ∥w⃗∥^{2}t^{2} + 2t|⟨v⃗, w⃗⟩| + ∥v⃗∥^{2}.

Since v⃗ and w⃗ are linearly independent, cv⃗ + tw⃗ is never the zero vector. Thus, by positive definiteness, q(t) is never 0. Since q(t) has no real roots, the discriminant of the quadratic polynomial is negative, i.e.,

(2|⟨v⃗, w⃗⟩|)^{2} < 4∥v⃗∥^{2} · ∥w⃗∥^{2}.

Dividing by 4 gives the inequality. □

Corollary 2.4. The function ∥ · ∥ : V → ℝ is a norm, i.e., it satisfies each of the following.

(i). Homogeneity. For every v⃗ in V and every scalar c, ∥c · v⃗∥ equals c · ∥v⃗∥. (ii). Positive definite. For every nonzero v⃗ in V, ∥v⃗} is positive.

(iii). Triangle inequality. For every v⃗, w⃗ in V,

∥v⃗ + w⃗∥ ≤ ∥v⃗∥ + ∥w⃗∥.

Proof. The first two items are obvious. The triangle inequality follows from the previous result. By definition,

∥v⃗ + w⃗∥^{2} = ⟨v⃗, v⃗⟩ + 2Re(⟨v⃗, w⃗⟩) + ⟨w⃗, w⃗⟩.

Applying the Cauchy-Bunyakovsky-Schwarz inequality, this is bounded by

∥v⃗ + w⃗∥^{2} ≤ ∥v⃗∥^{2} + 2∥v⃗∥∥w⃗∥ + ∥w⃗∥^{2} = (|v⃗∥ + ∥w⃗∥)^{2}.

Taking square roots gives the result. □

Examples. (1). Let n ≥ 0 be an integer. Let V be the real vector space ℝ^{n} with its standard ordered basis (e_{1}, …, e_{n}). Let ⟨·, ·⟩ be the Euclidean inner product,

⟨∑_{i=1}^{n} x_{i}e_{i}, ∑_{j=1}^{n} y_{j}e_{j}⟩ = ∑_{k=1}^{n} x_{k}y_{k}.

This is a real inner product space. It is usually called the real Euclidean n-space.

Examples. (2). Similarly, let V be the complex vector space ℂ^{n} with its standard ordered basis (e_{1}, …, e_{n}). Let ⟨·, ·⟩ be the pairing

⟨∑_{i=1}^{n} x_{i}e_{i}, ∑_{j=1}^{n} y_{j}e_{j}⟩ = ∑_{k=1}^{n} x_{k}\overline{y_{k}}.

This is a complex Hermitian space. It is sometimes also called a Euclidean n-space.

Examples. (3). Let ℓ_{ℝ}^{2}, respectively ℓ_{ℂ}^{2}, denote the real vector space, resp. complex vector space, of all sequences (a_{n})_{n≥0} of real numbers, resp. complex numbers, such that the series

∑_{n=0}^{∞} |a_{n}|^{2}

is convergent. Define a pairing by the series

⟨(a_{n})_{n≥0}, (b_{n})_{n≥0}⟩ = ∑_{n=0}^{∞} a_{n}\overline{b_{n}}.

Applying the Cauchy-Bunyakovsky-Scwarz inequality to the finite sequences (a_{n})_{n≤N}, (b_{n})_{n≤N} considered as elements of the Euclidean N-space, it follows that the series above is bounded by the product of the convergent sequences ∥(a_{n})∥ · ∥(b_{n})∥.

Therefore the series is absolutely convergent. It is straightforward to verify that this defines a real inner product space, resp. a complex Hermitian space.

Examples. (4). Let I be a closed interval in ℝ (possibly unbounded), and let C_{ℝ}^{2}(I), resp. C_{ℂ}^{2}(I), denote the real vector space, resp. complex vector space, of continuous functions f on I which are real-valued, resp. complex-valued, and such that the integral

∫_{I} |f(t)|^{2}dt

is convergent. Notice the convergence is automatic when I is a bounded interval. Define an inner product by

⟨f, g⟩ = ∫_{I} f(t)\overline{g(t)}dt.

For every bounded interval J, the restrictions of f and g to J satisfy the Cauchy-Bunyakovsky-Schwarz inequality, i.e.,

|∫_{J} f(t)\overline{g(t)}dt|^{2} ≤ |∫_{I} |f(t)|^{2}dt| · |∫_{I} |g(t)|^{2}dt| .

Since the product on the right is convergent, it follows the supremum over all J of the integeral on the left is also defined. Thus the inner product of f and g is welldefined. This pairing defines a real inner product space, resp. a complex Hermitian space. (However, it is usually not complete with respect to the corresponding norm.)

Examples. (5). Let (V, ⟨·, ·⟩) be a real inner product space. Denote by V_{ℂ} the associated complex vector space,

V_{ℂ} := V ⊗_{ℝ} ℂ.

Every element in this space has a unique decomposition of the form v⃗_{R} + iv⃗_{I} where v⃗_{R}, v⃗_{I} are elements of V. Define a pairing

⟨·, ·⟩ : V_{ℂ} × V_{ℂ} → ℂ,

by the rule

⟨v⃗_{R} + iv⃗_{I}, w⃗_{R} + iw⃗_{I}⟩ := (⟨v⃗_{R}, w⃗_{R}⟩ + ⟨v⃗_{I}, w⃗_{I}⟩) + i(⟨v⃗_{I}, w⃗_{R}⟩ − ⟨v⃗_{R}, w⃗_{I}⟩).

It is straightforward to verify that this is a Hermitian product. Thus associated to every real inner product space there is a complex Hermitian space. This is the beginning of many parallels between the theory of real inner product spaces and complex Hermitian spaces.

3. Orthonormal Bases and Gram-Schmidt

Let (V, ⟨·, ·⟩) be a real inner product space or complex Hermitian space. An element v⃗ of V is normal if ∥v∥ equals 1. A subset S of elements in V is orthogonal if for every pair v⃗, w⃗ of distinct elements in S, ⟨v⃗, w⃗⟩ equals 0. An orthogonal subset of normal vectors is an orthonormal subset. When (V, ⟨·, ·⟩) is a complex Hermitian space, the word “unitary” is sometimes also used to mean “orthonormal”. Every orthonormal set is linearly independent.

A collection (U_{i})_{i∈I} of subspaces of V is orthogonal if for every pair i, j of distinct elements of I, every element in U_{i} is orthogonal to every element in U_{j}. Given any subspace U of V, the set of all vectors v⃗ in V which are orthogonal to every element of U is called the orthogonal complement of U and denote U^{⊥}.

Lemma 3.1. Every collection of orthogonal subspaces is linearly independent. In particular, for every linear subspace U of V, U and U^{⊥} are linearly independent.

Proof. The definition of linear independence involves finite linear combinations. Thus it suffices to verify the first assertion for finite collections of orthogonal subspaces. This is proved by induction on the number of subspaces in the collection. The main case is when there are two subspaces, say U′ and U′′. And U′, U′′ are linearly independent if and only if U′ ∩ U′′ is {0}. Let u⃗ be an element in U′ ∩ U′′. Since u⃗ is in U′ it is orthogonal to every element of U′′. Since also u⃗ is in U′′, u⃗ is orthogonal to itself, i.e., ∥u⃗∥^{2} equals 0. By positive definiteness, u⃗ equals 0. Therefore U′, U′′ are linearly independent.

Now suppose that n ≥ 3 and let (U_{1}, …, U_{n}) be an orthogonal collection of vectors. Denote by U′ the sum U_{1} + ⋯ + U_{n−1} and denote by U′′ the space U_{n}. By the induction hypothesis, (U_{1}, …, U_{n}) is linearly independent. Thus to prove that (U_{1}, …, U_{n−1}, U_{n}) is linearly independent, it suffices to prove that U′ and U′′ are linearly independent. Since (U_{1}, …, U_{n}) is orthogonal, U′ and U′′ are orthogonal. Therefore, by the last paragraph, U′ and U′′ are linearly independent. □

Proposition 3.2 (Gram-Schmidt). Let (v⃗_{1}, …, v⃗_{n}) be a linearly independent set of elements in V. There exists a unique orthonormal set (û_{1}, …, û_{n}) such that for every r = 1, …, n, span(v⃗_{1}, …, v⃗_{r}) equals span(û_{1}, …, û_{r}) and ⟨v⃗_{r}, û_{r}⟩ is a positive real number.

Proof. This is proved by induction on n. For n = 1, û_{1} := v⃗_{1}/∥v⃗_{1}∥ is the unique normal vector in span(v⃗_{1}) such that ⟨v⃗_{1}, û_{1}⟩ is a positive real number. Thus, by way of induction, assume that n > 1 and assume the result is proved for n − 1.

Denote span(v⃗_{1}, …, v⃗_{n}) by U. Denote by U′ the subspace span(v⃗_{1}, …, v⃗_{n−1}). And denote by U′′ the orthogonal complement of U′ in U. Since v⃗_{n} is linearly independent from v⃗_{1}, …, v⃗_{n−1}, v⃗_{n} is not contained in U′.

By the induction hypothesis, there exists an orthonormal set (û_{1}, …, û_{n−1}) satisfying the conditions of the proposition for r = 1, …, n − 1. Denote

v′_{n} = ∑_{m=1}^{n−1}⟨v⃗_{n}, û_{m}⟩û_{m}

and denote v′′_{n} := v⃗_{n} − v′_{n}. By construction v′_{n} is in U′. For every i = 1, …, n − 1,

⟨v′_{n}, û_{i}⟩ = ⟨v⃗_{n}, û_{i}⟩.

Therefore ⟨v′′_{n}, û_{i}⟩ equals 0, i.e., v′′_{n} is in U′′. By the previous lemma U′ and U′′ are linearly independent. Thus v′_{n} and v′′_{n} are the unique elements in U′ and U′′ such that v⃗_{n} equals v′_{n} + v′′_{n}. In particular, note that

⟨v⃗_{n}, v′′_{n}⟩ = ∥v′′_{n}∥^{2}

which is a nonnegative real number.

In fact, since v⃗_{n} is not contained in U′, also v′′_{n} is not contained in U′, i.e., v′′_{n} is nonzero. So the nonnegative real number above is a positive real number. Also, since U′ and U′′ are linearly independent and since U′ + U′′ is contained in U, dim(U′) + dim(U′′) is at most dim(U). Thus dim(U′′) is at most 1. Therefore v′′_{n} is a basis for U′′. Since any vector û_{n} satisfying the conditions of the proposition lies in U′′, û_{n} must be a scalar multiple of v′′_{n}. But since û_{n} must be normal and since lrv⃗_{n}, û_{n} must be a positive real number, there is no choice about the scalar multiple. The only scalar multiple satisfying these conditions is

û_{n} := \frac{v′′_{n}}{∥v′′_{n}∥}.

Since (û_{1}, …, û_{n−1}) is a basis for U′ and since û_{n} is a basis for U′′, (û_{1}, …, û_{n−1}, û_{n}) is a linearly independent set of n elements in the n-dimensional space U. Therefore this is a basis for U.

Since v′′_{n} is a normal vector which is orthogonal to U′, and since (û_{1}, …, û_{n−1}) are orthonormal by the induction hypothesis, also the set (û_{1}, …, û_{n−1}, û_{n}) is orthonormal. By the induction hypothesis (û_{1}, …, û_{n−1}) is the unique set satisfying the conditions for i = 1, …, n − 1. And, as shown, û_{n} is the unique normal vector in span(v⃗_{1}, …, v⃗_{n}) which is orthogonal to (û_{1}, …, û_{n−1}) and such that ⟨v⃗_{r}, û_{r}⟩ is a positive real number. Thus (û_{1}, …, û_{n}) is the unique orthonormal set satisfying the conditions of the proposition. Thus the proposition is proved by induction on n. □

In particular, if V is finite dimensional, then it follows that V has a basis which is orthonormal. When V is complex Hermitian, an orthonormal basis is sometimes also called a unitary frame.

It also follows from Gram-Schmidt that if V is a finite dimensional vector space, then for every linear subspace U, (U, U^{⊥}) gives a direct sum decomposition of V. Since by the lemma (U, U^{⊥}) is linearly independent, it suffices to prove that every vector v⃗ in V is a sum of elements in U and U^{⊥}. If v⃗ is in U, this is obvious. If v⃗ is not in U, first use Gram-Schmidt to produce an orthnormal basis (û_{1}, …, û_{m}) for U. Next apply Gram-Schmidt to (û_{1}, …, û_{m}, v⃗) to produce (û_{1}, …, û_{m}, û′′). Then û′′ is in U^{⊥} and v⃗ is a sum of a linear combination of û_{1}, …, û_{m}, which is in U, and a scalar multiple of û′′, which is in U^{⊥}. This proves that V = U ⊕U^{⊥}. Incidentally, this fails when V is infinite dimensional. One characterization of Hilbert spaces, i.e., real inner product spaces or complex Hermitian spaces where the metric induced by ∥ · ∥ is complete, is that for every closed subspace U, the pair (U, U^{⊥}) does give an orthogonal decomposition of V.

4. Adjoint Operators and Normal Operators

For a real vector space V, resp. a complex vector space V, a linear functional on V is a linear transformation from V to ℝ, resp. ℂ. The set of linear functionals is denoted V^{∨} and is naturally a real vector space, resp. complex vector space, as discussed in lecture. A linear functional χ is bounded (with respect to ∥ · ∥) if there exist a positive real number M such that for every v⃗ in V,

|χ(v⃗)| ≤ M∥v⃗∥.

In this case, for every nonzero v⃗ in V, |χ(v⃗)|/∥v⃗∥ is defined and bounded by M. Therefore the supremum of |χ(v⃗)|/∥v⃗∥ over nonzero v⃗ is a well-defined nonnegative real number which is positive if χ is nonzero. This is the operator norm of χ,

∥χ∥ = sup_{v⃗≠0} \frac{|χ(v⃗)|}{∥v⃗∥}.

In particular, for every vector w⃗ in V, define ϕw⃗ to be the linear functional

ϕw⃗(v⃗) = ⟨v⃗, w⃗⟩.

Then the Cauchy-Schwarz inequality implies that ϕw⃗ is bounded and

∥ϕw⃗∥ = ∥w⃗∥.

In all that follows we will assume that V is finite dimensional. In fact there are infinite dimensional analogues which are even more important than the finite dimensional results. But this is more the topic of an analysis course rather than an algebra course.

Lemma 4.1 (Riesz Representation Theorem). Assume that V is finite dimensional. Every linear functional on V is bounded. The map Φ : V → V^{∨} defined by Φ(w⃗) = ϕw⃗ is an ℝ-linear isomorphism of ℝ-vector spaces. Moreover, ∥Φ(w⃗)∥ equals ∥w⃗∥ so that ∥ · ∥ is a norm on V^{∨} and Φ is an isometry.

Proof. As we have seen, when V is finite dimensional then V^{∨} is also finite dimensional and has the same dimension as V. If V and V^{∨} are complex vector spaces, then they are also both finite dimensional real vector spaces whose real dimension is twice the complex dimension, via the inclusion ℝ ⊂ ℂ. It is straightforward that Φ is ℝ-linear. Thus Φ is an ℝ-linear transformation between ℝ-vector spaces of the same finite dimension. It follows from the rank-nullity theorem that Φ is surjective if and only if Φ is an ℝ-linear isomorphism if and only if Φ is injective.

Let w⃗ be any nonzero element of V. Since ϕw⃗(w⃗) = ∥w⃗∥^{2} and since ⟨·, ·⟩ is positive definite, ϕw⃗(w⃗) is nonzero. Thus ϕw⃗ is nonzero. Thus Φ is injective. Therefore Φ is an ℝ-linear isomorphism. A posteriori it follows that every element in V^{∨} is bounded (since they are each of the form ϕw⃗) and that ∥ · ∥ is a norm on V^{∨} (since ∥ · ∥ is a norm on V). □

Remarks. (1). It is important to remark that when V is a complex Hermitian space, Φ is not ℂ-linear. In fact Φ(c · w⃗) = \overline{c} · Φ(w).

(2). In analysis classes it is proved that if V is complete with respect to the norm ∥ · ∥ (which is automatic if V is finite dimensional), then Φ is an isometric ℝ-linear isomorphism from V onto the subspace of bounded linear functionals. This is the usual statement of the Riesz representation theorem.

Now let (V, ⟨·, ·⟩_{V}) and (W, ⟨·, ·⟩_{W}) be finite dimensional real inner product spaces, resp. finite dimensional complex Hermitian spaces. Let T : V → W be a linear transformation. Recall that the linear transformation T^{†} : W^{∨} → V^{∨} is defined by

(T^{†}(χ⃗))(v⃗) = χ(T (v⃗)).

By the Riesz Representation Theorem there exists a unique linear transformation

T^{∗} : W → V, such that Φ_{V} ∘ T^{∗} = T^{†} ∘ Φ_{W} .

In other words, for every v⃗ in V and for every w⃗ in W,

⟨T (v⃗), w⃗⟩_{W} = ⟨v⃗, T^{∗}(w⃗)⟩_{V} .

The linear transformation T^{∗} is called the adjoint of T.

The most important case is when (W, ⟨·, ·⟩_{W}) equals (V, ⟨·, ·⟩_{V}), i.e., T is a linear operator on V. In this case T^{∗} is also a linear operator on V.

Definition 4.2. The linear transformation T is self-adjoint if T^{∗} equals T. The linear transformation T is anti-self-adjoint if T^{∗} equals −T. The linear transformation is normal if T^{∗}T equals T T^{∗}, i.e., T commutes with its adjoint. And the linear transformation T is orthogonal, resp. unitary in the complex case, if T^{∗} and T are inverse operators.

Examples. (1). Let (V, ⟨·, ·⟩) be the real Euclidean space ℝ^{n}. Let T be a linear operator on V. And let M be the matrix representative of T. Then the matrix representative of T^{∗} is M^{†}, the transpose matrix of M, (M^{†})_{i,j} = M_{j,i}. Thus T is self-adjoint if and only if the associated matrix M is symmetric, M_{i,j} = M_{j,i}. Also T is anti-self-adjoint if and only if M is skew-symmetric, −M_{i,j} = M_{j,i}. And T is orthogonal if and only if the column vectors of M are an orthonormal basis for ℝ^{n}. In fact, for every finite dimensional real inner product space V, for every linear operator T and for every ordered orthonormal basis B = (û_{1}, …, û_{n}), T is self-adjoint, resp. anti-self-adjoint, orthogonal, if and only if the matrix representative [T]_{B,B} is symmetric, resp. skew-symmetric, orthogonal. This is because B determines a linear, isometric isomorphism between (V, ⟨·, ·⟩) and the real Euclidean space.

Examples. (2). Let (V, ⟨·, ·⟩) be the complex Euclidean space ℂ^{n}. Let T be a linear operator on V. And let M be the matrix representative of T. Then the matrix representative of T^{∗} is \overline{M^{†}}, the conjugate transpose matrix of M, (\overline{M^{†}})_{i,j} = \overline{M_{j,i}}. Thus T is self-adjoint if and only if M is “conjugate symmetric”, \overline{M_{i,j}} = M_{j,i}. Also T is anti-self-adjoing if and only if M is “conjugate skew-symmetric”, −\overline{M_{i,j}} = M_{j,i}. And T is unitary if and only if the column vectors of M are a unitary frame (i.e., orthonormal basis) for ℂ^{n}. For the same reason as above, for every finite dimensional complex Hermitian space V, for every linear operator T and for every ordered orthonormal basis B = (û_{1}, …, û_{n}), T is self-adjoing, resp. antiself-adjoing, unitary, if and only if the matrix representative [T]_{B,B} is conjugate symmetric, resp. conjugate skew-symmetric, unitary.

The point of normal operators is that self-adjoint, anti-self-adjoint and orthogonal, resp. unitary, operators are all examples of normal operators. So the class of normal operators is the correct one in which to consider all three types of operators. The basic fact about normal operators is the following.

Lemma 4.3. For a normal operator T, the kernel of T^{∗} equals the kernel of T. And the kernel and image of T are orthogonal subspaces of V.

Proof. Let v⃗ be in the kernel of T. Then T^{∗}T (v⃗) equals 0. Since T is normal, i.e., T T^{∗} equals T^{∗}T, also T T^{∗}(v⃗) equals 0. Thus the square norm of T^{∗}(v⃗) equals 0 by

∥T^{∗}(v⃗)∥^{2} = ⟨T^{∗}(v⃗), T^{∗}(v⃗)⟩ = ⟨T T^{∗}(v⃗), v⃗⟩ = ⟨0, v⃗⟩ = 0.

Since ⟨·, ·⟩ is positive definite, T^{∗}(v⃗) equals 0. Thus v is in the kernel of T^{∗}. Since (T^{∗})^{∗} equals T, the same argument proves that the kernel of T is contained in the kernel of T^{∗}. Therefore the kernel of T equals the kernel of T^{∗}.

Now let v⃗ be in the kernel of T. Then T^{∗}(v⃗) equals 0. Therefore, for every element w⃗ of V,

⟨v⃗, T (w⃗)⟩ = ⟨T^{∗}(v⃗), w⃗⟩ = ⟨0, w⃗⟩ = 0.

Thus v⃗ is orthogonal to every element T (w⃗) in the image of T. So the kernel of T is orthogonal to the image of T. □

A basic fact about commutativity of operators is that it extends to polynomial expressions in those operators (and in analysis, even to holomorphic expressions on open neighborhoods of the spectra of the operators). For every linear operator T and for every polynomial with coefficients in the scalar field,

p(t) = c_{n}t^{n} + ⋯ + c_{1}t + c_{0},

the linear operator p(T) is defined by

p(T) = c_{n}T^{n} + ⋯ + c_{1}T + c_{0}Id_{V} .

Exercise. Let S and T be linear operators that commute, ST = T S. Prove that for all nonnegative integers m, n, S^{m} and T^{n} commute. Conclude that for all polynomials p(s) and q(t) with coefficients in the scalar field, p(S) and q(T) commute. In fact, for every pair of 2-variable operators u(s, t) and v(s, t), the operators u(S, T) and v(S, T) are well-defined and commute.

Lemma 4.4. Let T be a normal operator on a complex Hermitian space V. Then for every polynomial u(s, t) with complex coefficients, u(T ∗, T) is also normal. Also a vector v⃗ is an eigenvector of T with eigenvalue λ if and only if v⃗ is an eigenvector of T^{∗} with eigenvalue \overline{λ}.

Proof. The first assertion follows immediately from the exercise since u(T ∗, T) is well-defined and (u(T ∗, T))^{∗} equals v(T^{∗}, T) where v(s, t) is the complex polynomial v(s, t) = \overline{v(\overline{t}, \overline{s})}. Because T is normal, and using the previous lemma, also T − λId is normal. And (T − λId)^{∗} equals T^{∗} − \overline{λ}Id. By Lemma 4.3, the kernel of T − λId equals the kernel of (T −λId)^{∗}. Thus the λ-eigenspace of T equals the \overline{λ}-eigenspace of T^{∗}. □

Theorem 4.5 (The Spectral Theorem). Let T be a normal operator on a finite dimensional complex Hermitian space. Then T is diagonalizable and the eigenspaces are pairwise orthogonal. If T is self-adjoint, then the eigenvalues are all real. Similarly, every self-adjoint operator on a finite dimensional real inner product space is diagonalizable with orthogonal eigenspaces.

Proof. First we prove that a normal operator on a complex Hermitian space is diagonalizable with orthogonal eigenspaces. For every complex number λ, denote by E_{T,λ} the λ-eigenspace, i.e.,

E_{T,λ} := ker(λId_{V} − T).

By Lemma 4.4, E_{T,λ} equals E_{T^{∗},\overline{λ}} and λId_{V} − T is normal. And for every complex number μ with μ ≠ λ, λId_{V} − T maps E_{T,μ} isomorphically back to itself via the scaling (λ − μ), i.e., for v⃗ in E_{T,μ},

(λId_{V} − T)(v⃗) = λv⃗ − T (v⃗) = (λ − μ)v⃗).

Thus E_{T,μ} is in the image of λId_{V} − T. Therefore, by Lemma 4.3, the eigenspaces (E_{T,λ_{1}}, …, E_{T,λ_{r}}) are orthogonal. So, denoting by λ_{1}, …, λ_{r} the distinct eigenvalues of T, (E_{T,λ_{1}}, …, E_{T,λ_{r}}) is an orthogonal set of subspaces of V. It remains only to prove that the subspace U′_{T} = E_{T,λ_{1}} + ⋯ + E_{T,λ_{r}} equals all of V.

Denote by U′′_{T} the orthogonal complement of U′_{T}. Of course T maps each eigenspace E_{T,λ} back into itself via the scaling λ. And by Lemma 4.4, for every v⃗ in V and every w⃗ in E_{T,λ},

⟨T (v⃗), w⃗⟩ = ⟨v⃗, T^{∗}(w⃗)⟩ = ⟨v⃗, \overline{λ}w⃗⟩ = λ⟨v⃗, w⃗⟩.

In particular, if T (v⃗) is orthogonal to w⃗, then also T (v⃗) is orthogonal to w⃗. Therefore T maps U′′_{T} back into itself. Also by Lemma 4.4 once more, U′_{T^{∗}} equals U′_{T}, and thus also U′′_{T^{∗}} equals U′′_{T}. Therefore, by the same argument as above, T^{∗} also maps U′′_{T} back into itself.

But this means that the restriction

T |_{U′′} : U′′_{T} → U′′_{T}

is a normal operator on the complex Hermitian space U′′_{T} with the restricted Hermitian product. Every linear operator S on a nonzero, finite dimensional complex vector space U has a nonzero eigenvector by the Fundamental Theorem of Algebra applied to the characteristic polynomial

χ_{S}(x) = det(xId_{U} − S).

Therefore, if U′′_{T} is nonzero, then there exists an eigenvalue λ and a nonzero λeigenvector v⃗ for T |_{U′′}. Then v⃗ is in E_{T,λ} which is contained in U′_{T}. Since U′_{T} and U′′_{T} are orthgonal, their intersection equals {0} by Lemma 3.1. So there cannot exist a nonzero vector v⃗ in U′′_{T} and in E_{T,λ}. This proves that U′′_{T} does equal {0} (by contradiction). Therefore V equals the sum of the T -eigenspaces, i.e., T is diagonalizable on V.

Next assume that T is self-adjoint. Let λ be an eigenvalue with a nonzero λeigenvector v⃗. By Lemma 4.4, v⃗ is also a \overline{λ}-eigenvector of T^{∗}. But since T = T^{∗}, it follows that λ = \overline{λ}, i.e., λ is real. Therefore every eigenvalue of T is real.

Finally, consider the real case. By Gram-Schmidt, every real inner product space has an orthonormal basis. Thus we may reduce to the case when the real inner product space is the Euclidean space ℝ^{n}. Let

T : ℝ^{n} → ℝ^{n}

be an ℝ-linear operator. Define a ℂ-linear operator T_{ℂ} on ℂ^{n} by

T_{ℂ} : ℂ^{n} → ℂ^{n}, T_{ℂ}(v⃗_{R} + iv⃗_{I}) := T (v⃗_{R}) + iT (v⃗_{I}).

It is straightforward to verify that (T_{ℂ})^{∗} equals (T^{∗})_{ℂ}. In particular, if T is selfadjoint then also T_{ℂ} is self-adjoint. Then by the previous case, the ℂ^{n} is a direct sum of eigenspace E_{T_{ℂ},λ} for real numbers λ. But now let v⃗_{R} + iv⃗_{I} be any element in E_{T_{ℂ},λ}. Then we have the identity

T (v⃗_{R}) + iT (v⃗_{I}) = T_{ℂ}(v⃗_{R} + iv⃗_{I}) = λ(v⃗_{R} + iv⃗_{I}) = (λv⃗) + i(λv⃗_{I}).

It follows that both v⃗_{R} and v⃗_{I} are in E_{T,λ}, i.e., E_{T_{ℂ},λ} = E_{T,λ} ⊗_{ℝ} ℂ. Therefore the λ-eigenspaces E_{T,λ} of T on ℝ^{n} also span ℝ^{n}. Orthogonality of the distinct eigenspaces of T_{ℂ} implies orthogonality of the corresponding eigenspaces of T. Thus the eigenspaces of T are orthogonal and give a direct sum decomposition of ℝ^{n}. Therefore T is diagonalizable on ℝ^{n}. □