Blog

The Eigenvalue Problem

(Chapter 11 in the textbook, Lecture notes.)

A square matrix \(\mA\) acts on vectors: it transforms a vector \(\xx\) into the vector \(\mA\xx\), a map from \(\C^n\) to \(\C^n\). Most vectors are turned and stretched by this map. Some special vectors keep their direction and are only scaled. These are the eigenvectors, and the eigenvalue problem is to find them: for a square matrix \(\mA\), find the vectors \(\xx\ne\nul\) and scalars \(\lambda\) with

To solve it, move everything to one side: \[\mA\xx=\lambda\xx\Rightarrow\mA\xx-\lambda\xx=\nul\Rightarrow(\mA-\lambda\mI)\xx=\nul.\] For a given \(\lambda\) this is a homogeneous system, so it is always consistent, and \(\xx=\nul\) is always a solution. But we are interested in non-zero solutions. A homogeneous system has non-zero solutions exactly when its matrix is not invertible, that is, when \[ \underbrace{\det(\mA-\lambda\mI)=0}_{\text{a polynomial equation in }\lambda}. \] So non-trivial solutions of \((\mA-\lambda\mI)\xx=\nul\) exist only for those \(\lambda\) that solve this equation.

Definition  A pair \((\lambda,\xx)\) with \(\xx\ne\nul\) that solves \(\mA\xx=\lambda\xx\) is an eigenpair: \(\lambda\) is an eigenvalue of \(\mA\) and \(\xx\) is a corresponding eigenvector.

Definition  The polynomial \(\det(\mA-\lambda\mI)\) is called the characteristic polynomial of \(\mA\). Its roots are exactly the eigenvalues of \(\mA\).

Example Find the eigenvalues and the corresponding eigenvectors (the eigenspaces) of \[ \mA=\col{2&2&1\\1&3&1\\1&2&2}. \] The characteristic polynomial factors as \[ \det(\mA-\lambda\mI)=\begin{vmatrix}2-\lambda&2&1\\1&3-\lambda&1\\1&2&2-\lambda\end{vmatrix} =-\lambda^3+7\lambda^2-11\lambda+5=-(\lambda-5)(\lambda-1)^2. \] So \(\lambda_1=5\) is a simple eigenvalue, and \(\lambda_2=1\) is a repeated eigenvalue with multiplicity \(2\). For each eigenvalue we solve \((\mA-\lambda\mI)\xx=\nul\).

For \(\lambda_1=5\): \[ (\mA-5\mI)\xx=\col{-3&2&1\\1&-2&1\\1&2&-3}\xx=\nul. \] Gauss elimination gives \[ \col{-3&2&1\\0&1&-1\\0&0&0}\col{x_1\\x_2\\x_3}=\col{0\\0\\0}\Rightarrow \begin{gathered}x_1\ \&\ x_2\text{ are pivot variables,}\\x_3\text{ is a free variable.}\end{gathered} \] Setting \(x_3=\alpha\) gives \(x_2=\alpha\) and then \(x_1=\alpha\), so \(\xx=\col{\alpha\\\alpha\\\alpha}=\alpha\col{1\\1\\1}\). The eigenspace of \(\lambda_1=5\) is \[ \Sp\left\{\col{1\\1\\1}\right\}. \] For \(\lambda_2=1\): \[(\mA-\mI)\xx=\col{1&2&1\\1&2&1\\1&2&1}\xx=\nul.\] Gauss elimination leaves a single non-zero row: \[ \col{1&2&1\\0&0&0\\0&0&0}\col{x_1\\x_2\\x_3}=\col{0\\0\\0}\Rightarrow x_1\text{ is a pivot variable, and }x_2\ \&\ x_3\text{ are free variables.} \] Setting \(x_3=\alpha\) and \(x_2=\beta\) gives \(x_1=-2\beta-\alpha\), so \[ \xx=\col{-2\beta-\alpha\\\beta\\\alpha}=\alpha\col{-1\\0\\1}+\beta\col{-2\\1\\0}. \] The eigenspace associated with \(\lambda_2=1\) is two-dimensional: \[ \Sp\left\{\col{-1\\0\\1},\col{-2\\1\\0}\right\}. \] Notes

1. If \(\xx\) is an eigenvector, then so is \(c\xx\) for any \(c\ne0\). Eigenvectors are only determined up to a scale factor.

2. If \(\lambda\) is an eigenvalue with multiplicity \(k\), its eigenspace may have a dimension lower than \(k\). The next example shows this.

Example Let \(\mA=\col{1&0&1\\1&1&0\\0&0&1}\). Then \[ \det(\mA-\lambda\mI)=\begin{vmatrix}1-\lambda&0&1\\1&1-\lambda&0\\0&0&1-\lambda\end{vmatrix}=(1-\lambda)^3=0. \] So \(\lambda_1=1\) is the only eigenvalue, with multiplicity \(3\). Its eigenspace is the set of solutions of \[ (\mA-\mI)\xx=\nul\Rightarrow\col{0&0&1\\1&0&0\\0&0&0}\xx=\col{0\\0\\0}. \] Here \(x_1\) and \(x_3\) are pivot variables and \(x_2\) is free. The equations force \(x_1=0\) and \(x_3=0\), and \(x_2=\alpha\) is arbitrary, so \(\xx=\col{0\\\alpha\\0}=\alpha\col{0\\1\\0}\). The eigenspace is \[ \Sp\left\{\col{0\\1\\0}\right\}, \] which is only one-dimensional even though the eigenvalue has multiplicity three.

A Special Case: Symmetric Matrices

Symmetric matrices have especially nice eigenvalues and eigenvectors. We collect four facts about them, each building on the one before.

Proof  Let \((\lambda,\xx)\) be an eigenpair of \(\mA\), possibly with complex entries, and write \(\xx^H\) for the conjugate transpose of \(\xx\). Multiply \(\mA\xx=\lambda\xx\) on the left by \(\xx^H\). Because \(\mA\) is real and symmetric, \(\mA^H=\mA\), so \(\xx^H(\mA\xx)=(\mA\xx)^H\xx\). Therefore \begin{align*} \mA\xx=\lambda\xx&\Rightarrow\xx^H(\mA\xx)=\lambda\xx^H\xx\\ &=(\mA\xx)^H\xx=(\lambda\xx)^H\xx=\lambda^*\xx^H\xx. \end{align*} Comparing the two ends gives \[ \Rightarrow\lambda\xx^H\xx=\lambda^*\xx^H\xx\Rightarrow(\lambda-\lambda^*)\xx^H\xx=0. \] Since \(\xx\ne\nul\), we have \(\xx^H\xx>0\), so \(\lambda=\lambda^*\), and \(\lambda\) is real. \(\blacksquare\)

Proof  This is proved after Theorem 11.4.4.

Proof  Let \((\lambda_1,\xx_1)\) and \((\lambda_2,\xx_2)\) be eigenpairs of \(\mA\) with \(\lambda_1\ne\lambda_2\). As before, move \(\mA\) across the product using \(\mA^H=\mA\), and recall that \(\lambda_2\) is real by Theorem 11.3.1, so \(\lambda_2^*=\lambda_2\): \begin{align*} \mA\xx_1=\lambda_1\xx_1&\Rightarrow\xx_2^H(\mA\xx_1)=\lambda_1\xx_2^H\xx_1\\ &=(\mA\xx_2)^H\xx_1=\lambda_2^*\xx_2^H\xx_1=\lambda_2\xx_2^H\xx_1. \end{align*} Comparing the ends, \[ \Rightarrow\lambda_1\xx_2^H\xx_1=\lambda_2\xx_2^H\xx_1\Rightarrow(\lambda_1-\lambda_2)\xx_2^H\xx_1=0. \] Since \(\lambda_1-\lambda_2\ne0\), we must have \(\xx_2^H\xx_1=0\). \(\blacksquare\)

Proof  There are two cases.

1. If \(\mA\) has \(n\) distinct eigenvalues \(\lambda_1,\ldots,\lambda_n\), their eigenvectors are orthogonal by Theorem 11.3.3, so they form an orthogonal basis.

2. If \(\mA\) has repeated eigenvalues, say distinct eigenvalues \(\lambda_1,\ldots,\lambda_m\) where \(\lambda_i\) has multiplicity \(k_i\), then each eigenspace is \(k_i\)-dimensional by Theorem 11.3.2. So each eigenspace has an orthogonal basis \(\{\uu_{i1},\ldots,\uu_{ik_i}\}\) (for example by Gram–Schmidt). Eigenspaces belonging to different eigenvalues are orthogonal to each other by Theorem 11.3.3, so putting all these bases together gives an orthogonal basis of \(\R^n\): \[ \bigcup_{i=1}^m\{\uu_{i1},\ldots,\uu_{ik_i}\}. \] There are exactly \(n\) vectors in total, because \(\sum_{i=1}^m k_i=n\). \(\blacksquare\)

Diagonalization

A diagonal matrix is the easiest kind of matrix to work with. Diagonalization tries to turn a given matrix into one by changing coordinates. If we can find a matrix \(\mQ\) such that

then we say that \(\mA\) is diagonalizable and that \(\mQ\) diagonalizes \(\mA\). The next theorem says which matrices are diagonalizable and what \(\mQ\) and \(\mD\) are.

Theorem 11.4.1

Let \(\mA\) be an \(n\times n\) matrix.

Proof  (i) We show the two directions separately.

\((\Rightarrow)\) Suppose \(\mA\) is diagonalizable, so \(\mQ^{-1}\mA\mQ=\mD\) for some \(\mQ=[\ev_1\ \cdots\ \ev_n]\). Multiply by \(\mQ\) on the left and compare columns: \[ \mQ^{-1}\mA\mQ=\mD\Rightarrow\mA\mQ=\mQ\mD=[\ev_1\ \cdots\ \ev_n]\col{d_1&&\\&\ddots&\\&&d_n} \] \[\Rightarrow\mA[\ev_1\ \cdots\ \ev_n]=[d_1\ev_1\ d_2\ev_2\ \cdots\ d_n\ev_n].\] So \(\mA\ev_i=d_i\ev_i\) for every \(i\). No column of an invertible matrix is zero, so \(\ev_i\ne\nul\), and therefore \(\ev_i\) is an eigenvector with eigenvalue \(d_i\). Since \(\mQ\) is invertible, its columns \(\ev_i\) are linearly independent. This gives \(n\) linearly independent eigenvectors. \(\blacksquare\)

\((\Leftarrow)\) Suppose \(\mA\) has \(n\) linearly independent eigenvectors \(\{\ev_1,\ldots,\ev_n\}\). Put them in the columns of \(\mQ\) and the eigenvalues on the diagonal of \(\mLambda\). The statements \(\mA\ev_i=\lambda_i\ev_i\) for all \(i\) together say \[ \mA[\ev_1\ \cdots\ \ev_n]=\underbrace{[\ev_1\ \cdots\ \ev_n]}_{\mQ}\underbrace{\col{\lambda_1&&\\&\ddots&\\&&\lambda_n}}_{\mLambda}. \] The columns of \(\mQ\) are linearly independent, so \(\mQ\) is invertible, and \[\Rightarrow\mQ^{-1}\mA\mQ=\mLambda\longleftarrow\text{diagonal}\Rightarrow\mA\text{ is diagonalizable}.\quad\blacksquare\]

Theorem 11.4.2

When are there enough eigenvectors? One easy case is when all the eigenvalues are different.

Proof  Suppose \(\alpha_1\ev_1+\cdots+\alpha_n\ev_n=\nul\). We will show that this forces \(\alpha_1=\cdots=\alpha_n=0\), which means that \(\{\ev_1,\ldots,\ev_n\}\) is LI. Written with a column of ones, the assumption says \[[\alpha_1\ev_1\ \cdots\ \alpha_n\ev_n]\col{1\\1\\\vdots\\1}=\nul.\] Applying \(\mA\) to the assumption multiplies each term by its own eigenvalue, \(\mA(\alpha_1\ev_1+\cdots+\alpha_n\ev_n)=\lambda_1\alpha_1\ev_1+\cdots+\lambda_n\alpha_n\ev_n=\nul\), and applying \(\mA\) again and again gives, for every power up to \(n-1\), \[\mA^{n-1}(\alpha_1\ev_1+\cdots+\alpha_n\ev_n)=\lambda_1^{n-1}\alpha_1\ev_1+\cdots+\lambda_n^{n-1}\alpha_n\ev_n=\nul.\] Collecting all of these equations into one matrix equation, \[ [\alpha_1\ev_1\ \cdots\ \alpha_n\ev_n] \underbrace{\col{1&\lambda_1&\cdots&\lambda_1^{n-1}\\1&\lambda_2&\cdots&\lambda_2^{n-1}\\\vdots&\vdots&&\vdots\\1&\lambda_n&\cdots&\lambda_n^{n-1}}}_{\text{Vandermonde matrix}} =[\nul\ \nul\ \cdots\ \nul]=\mO_{n\times n}. \] A Vandermonde matrix has a non-zero determinant when the \(\lambda_i\) are all different (a standard fact about these matrices), so it is invertible. Multiplying on the right by its inverse gives \[\Rightarrow[\alpha_1\ev_1\ \cdots\ \alpha_n\ev_n]=\mO.\] Each eigenvector is non-zero, \(\ev_i\ne\nul\), so each \(\alpha_i=0\). Hence \(\{\ev_1,\ldots,\ev_n\}\) is LI. \(\blacksquare\)

Theorem 11.4.3

Proof  \(n\) distinct eigenvalues give \(n\) linearly independent eigenvectors (Theorem 11.4.2), and \(n\) linearly independent eigenvectors make \(\mA\) diagonalizable (Theorem 11.4.1). \(\blacksquare\)

Theorem 11.4.4

Symmetric matrices are always diagonalizable, even when eigenvalues repeat, and moreover the matrix \(\mQ\) can be chosen orthogonal.

Proof  We give a proof that does not use Theorem 11.3.2; this is what makes the proof of Theorem 11.3.2 below legitimate.

The proof is by induction on the size \(n\) of the matrix.

For a \(1\times1\) symmetric matrix, \(\mA=[a_{11}]\) is already diagonal, so it is certainly diagonalizable (with \(\mQ=1\)).

Now assume that every symmetric matrix of size \((n-1)\times(n-1)\) is diagonalizable in the following way: \[ \mQ^T\mA\mQ=\mD\quad\longleftarrow\text{diagonal}\qquad\text{where }\mQ^T\mQ=\mI\text{ or }\mQ^{-1}=\mQ^T. \] We show that every \(n\times n\) symmetric matrix \(\mA_{n\times n}\) then also has an orthogonal \(\mQ_{n\times n}\) that diagonalizes it.

Let \((\lambda,\xx)\) be an eigenpair of \(\mA_{n\times n}\). The eigenvalue \(\lambda\) is real by Theorem 11.3.1, and we choose the eigenvector real, with length one: \(\xx^T\xx=1\). Extend \(\xx\) to an orthonormal basis \(\{\yy_1,\ldots,\yy_{n-1},\xx\}\) of \(\R^n\), and collect the new vectors in the \(n\times(n-1)\) matrix \(\mY=[\yy_1\ \cdots\ \yy_{n-1}]\). It has the following properties:

1. \(\mY^T\mY=\mI_{n-1}\), since \(\yy_i^T\yy_j=\delta_{ij}\).

2. \(\xx^T\mY=\nul^T\), since \(\xx^T\yy_i=0\).

3. \(\mY^T\mA\mY\) is symmetric, because \(\mA\) is symmetric.

So \(\mY^T\mA\mY\) is a symmetric matrix of size \((n-1)\times(n-1)\). By the induction hypothesis there is an orthogonal \(\Qs\) that diagonalizes it: \[\Qs^T\mY^T\mA\mY\Qs=\mD.\qquad(\star1)\] Now define \(\mQ_{n\times n}=[\xx\ \mY\Qs]\). We show that it is orthogonal and that it diagonalizes \(\mA\). First, orthogonality:

\begin{align*} \mQ_{n\times n}^T\mQ_{n\times n} &=[\xx\ \mY\Qs]^T[\xx\ \mY\Qs]\\ &=\col{\xx^T\\\Qs^T\mY^T}[\xx\ \mY\Qs]\\ &=\col{\xx^T\xx&\xx^T\mY\Qs\\\Qs^T\mY^T\xx&\Qs^T\mY^T\mY\Qs}. \end{align*} The four blocks are:

• \(\xx^T\xx=1\).

• \(\xx^T\mY\Qs=\nul^T\), since \(\xx^T\mY=\nul^T\).

• \(\Qs^T\mY^T\xx=\nul\), since \(\mY^T\xx=\nul\).

• \(\Qs^T\mY^T\mY\Qs=\Qs^T\underbrace{(\mY^T\mY)}_{\mI_{n-1}}\Qs=\Qs^T\Qs=\mI_{n-1}\).

So \[\Rightarrow\mQ_{n\times n}^T\mQ_{n\times n}=\col{1&\nul^T\\\nul&\mI_{(n-1)\times(n-1)}}=\mI_{n\times n}.\] Second, we compute \(\mQ^T\mA\mQ\): \begin{align*} \mQ_{n\times n}^T\mA_{n\times n}\mQ_{n\times n}&=[\xx\ \mY\Qs]^T\mA_{n\times n}[\xx\ \mY\Qs]\\ &=\col{\xx^T\mA\xx&\xx^T\mA\mY\Qs\\(\xx^T\mA\mY\Qs)^T&(\mY\Qs)^T\mA\mY\Qs}. \end{align*} Its blocks are:

• \(\xx^T\mA\xx=\xx^T\lambda\xx=\lambda\).

• \(\underbrace{\xx^T\mA}_{\lambda\xx^T}\mY\Qs=\lambda\underbrace{\xx^T\mY}_{\nul^T}\Qs=\nul^T\).

• \(\Qs^T(\mY^T\mA\mY)\Qs=\mD\), by equation \((\star1)\).

Therefore \[ \Rightarrow\mQ_{n\times n}^T\mA_{n\times n}\mQ_{n\times n}=\col{\lambda&\nul^T\\\nul&\mD}\text{ is diagonal},\quad\mQ^T_{n\times n}\mQ_{n\times n}=\mI. \] Thus \(\mA_{n\times n}\) is diagonalizable by an orthogonal matrix. \(\blacksquare\)

Revisit Theorem 11.3.2

Now that we know every real symmetric matrix is diagonalizable, we can prove the fact that was postponed earlier.

Proof By Theorem 11.4.4, a real symmetric \(\mA_{n\times n}\) is diagonalizable by an orthogonal matrix: \[\mQ^T\mA\mQ=\mQ^{-1}\mA\mQ=\mD.\] Since \(\mQ^T(\mA-\lambda\mI)\mQ=\mD-\lambda\mI\) and \(\det\mQ\det\mQ^T=1\), the characteristic polynomial is \(\det(\mA-\lambda\mI)=\prod_{i=1}^n(d_i-\lambda)\). So an eigenvalue of multiplicity \(k\) appears exactly \(k\) times on the diagonal of \(\mD\). The \(k\) matching columns of \(\mQ\) are eigenvectors (because \(\mA\mQ=\mQ\mD\)) and they are linearly independent, so the eigenspace has dimension at least \(k\). It cannot be larger: eigenspaces of different eigenvalues are linearly independent of each other, and the multiplicities add up to \(n\), so the dimensions cannot add up to more than \(n\). Hence the eigenspace has dimension exactly \(k\). \(\blacksquare\)

Closing Note

Even if \(\mA\) is not diagonalizable, it can still be brought to its Jordan form (over \(\C\)), which is upper triangular, with the eigenvalues on the diagonal and only \(0\)’s and \(1\)’s directly above the diagonal: \[\mP^{-1}\mA\mP=\underbrace{\mJ}_{\text{Jordan form for }\mA}.\] The textbook discusses this on page 582 (exercise 11.4.10).

Cayley–Hamilton Theorem

A matrix satisfies its own characteristic equation. Let \(\mA_{n\times n}\) have the characteristic equation \[\lambda^n+\alpha_1\lambda^{n-1}+\cdots+\alpha_{n-1}\lambda+\alpha_n=0.\]

Proof We prove the theorem for the case where \(\mA\) has a full set of eigenvectors \(\vv_1,\ldots,\vv_n\) (the theorem is true in general, but this case shows the idea). Collect the eigenvectors in \(\mV\) and the eigenvalues on the diagonal of \(\mLambda\): \[\mA\col{\vv_1&\vv_2&\cdots&\vv_n}=\underbrace{\col{\vv_1&\vv_2&\cdots&\vv_n}}_{\mV}\underbrace{\col{\lambda_1&&&\\&\lambda_2&&\\&&\ddots&\\&&&\lambda_n}}_{\mLambda}.\] Some of the \(\lambda_i\) may be equal, but the columns of \(\mV\) are linearly independent, so \(\mV\) is invertible, and \[\mV^{-1}\mA\mV=\mLambda.\] Powers of \(\mA\) are easy to follow through this change of coordinates, because the \(\mV\) and \(\mV^{-1}\) in the middle cancel: \[(\mV^{-1}\mA\mV)^2=\mV^{-1}\mA\underbrace{\mV\mV^{-1}}_{\mI}\mA\mV=\mV^{-1}\mA^2\mV=\mLambda^2\] and in general \[(\mV^{-1}\mA\mV)^k=\mV^{-1}\mA^k\mV=\mLambda^k=\col{\lambda_1^k&&\\&\ddots&\\&&\lambda_n^k}.\] Each eigenvalue \(\lambda_i\) satisfies the characteristic equation, so the diagonal matrix of those polynomial values is zero: \[\col{\lambda_1^n+\alpha_1\lambda_1^{n-1}+\cdots+\alpha_{n-1}\lambda_1+\alpha_n&&\\&\ddots&\\&&\lambda_n^n+\alpha_1\lambda_n^{n-1}+\cdots+\alpha_{n-1}\lambda_n+\alpha_n}=\mO.\] This diagonal matrix is the same as the polynomial evaluated at \(\mLambda\). Rewriting each power of \(\mLambda\) as \(\mV^{-1}\mA^k\mV\) gives \begin{align*} &=\mLambda^n+\alpha_1\mLambda^{n-1}+\cdots+\alpha_{n-1}\mLambda+\alpha_n\mI=\mO\\ &\Rightarrow\mV^{-1}\mA^n\mV+\alpha_1\mV^{-1}\mA^{n-1}\mV+\cdots+\alpha_{n-1}\mV^{-1}\mA\mV+\alpha_n\mV^{-1}\mI\mV=\mO\\ &\Rightarrow\mV^{-1}(\mA^n+\alpha_1\mA^{n-1}+\cdots+\alpha_{n-1}\mA+\alpha_n\mI)\mV=\mO. \end{align*} Multiplying on the left by \(\mV\) and on the right by \(\mV^{-1}\) removes the outer factors: \[\mA^n+\alpha_1\mA^{n-1}+\cdots+\alpha_{n-1}\mA+\alpha_n\mI=\mO.\qquad\blacksquare\]

Cayley–Hamilton is useful because it lets us reduce high powers of a matrix to low ones.

Example Let \(\mA=\col{0&1\\-2&-3}\). Its characteristic equation is \[\begin{vmatrix}-\lambda&1\\-2&-3-\lambda\end{vmatrix}=0\Rightarrow\lambda^2+3\lambda+2=0,\qquad\lambda_1=-1,\quad\lambda_2=-2.\] We can check the theorem directly: \[\mA^2=\col{-2&-3\\6&7},\qquad\mA^2+3\mA+2\mI=\col{-2&-3\\6&7}+\col{0&3\\-6&-9}+\col{2&0\\0&2}=\mO.\] This says \(\mA^2=-3\mA-2\mI\). So \(\mA^2\) is a combination of \(\mA\) and \(\mI\), and every higher power can be reduced in the same way: \[\mA^k=\beta_1\mA+\beta_0\mI\qquad\text{where $\beta_1$ and $\beta_0$ depend on $k$}.\] For an \(n\times n\) matrix in general, \(\displaystyle\mA^k=\sum_{i=0}^{n-1}\beta_i\mA^i\).

How do we find the coefficients \(\beta_i\)? Let \(\lambda_j\) be an eigenvalue of \(\mA\) with eigenvector \(\vv_j\). Applying both sides of \(\mA^k=\sum_i\beta_i\mA^i\) to \(\vv_j\), and using \(\mA^i\vv_j=\lambda_j^i\vv_j\), \[\underbrace{\mA^k\vv_j}_{\lambda_j^k\vv_j}=\left(\sum_{i=0}^{n-1}\beta_i\mA^i\right)\vv_j \quad\Rightarrow\quad\lambda_j^k\vv_j=\left(\sum_{i=0}^{n-1}\beta_i\lambda_j^i\right)\vv_j.\] Since \(\vv_j\ne\nul\), the scalars must agree. So every eigenvalue gives one equation for the unknown \(\beta_i\):

Example For the same matrix, compute \(\mA^5\). Write \(\mA^5=\beta_1\mA+\beta_0\mI\) with \(\lambda_1=-1\) and \(\lambda_2=-2\). The two equations for \(\beta_0,\beta_1\) are \[\begin{gathered}\lambda_1^5=\beta_1\lambda_1+\beta_0\\\lambda_2^5=\beta_1\lambda_2+\beta_0\end{gathered}\quad\Leftrightarrow\quad\col{1&\lambda_1\\1&\lambda_2}\col{\beta_0\\\beta_1}=\col{\lambda_1^5\\\lambda_2^5}.\] Plugging in the numbers and solving, \[\col{1&-1\\1&-2}\col{\beta_0\\\beta_1}=\col{-1\\-32}\Rightarrow\col{\beta_0\\\beta_1}=-\col{-2&1\\-1&1}\col{-1\\-32}=\col{30\\31}.\] Hence \[\mA^5=31\mA+30\mI=\col{0&31\\-62&-93}+\col{30&0\\0&30}=\col{30&31\\-62&-63}.\]

Example A repeated eigenvalue. Let \(\mA=\col{0&1\\-1&-2}\), which has \(\lambda_1=\lambda_2=-1\). Find \(\mA^{10}\). Write \(\mA^{10}=\beta_1\mA+\beta_0\mI\). The eigenvalue equation gives one condition: \[\lambda_1^{10}=\beta_1\lambda_1+\beta_0\Rightarrow1=-\beta_1+\beta_0.\] Since \(\lambda_2=\lambda_1\) this gives no second independent equation for \(\beta_1\) and \(\beta_0\). But \(\lambda_1\) is a double root of the characteristic polynomial, so differentiating both sides of \(\lambda^{10}=\beta_1\lambda+\beta_0\) with respect to \(\lambda\) also holds at \(\lambda_1\), and that gives the missing equation: \[\left.\frac{d}{d\lambda}\lambda^{10}\right|_{\lambda=\lambda_1}=\left.\frac{d}{d\lambda}(\beta_1\lambda+\beta_0)\right|_{\lambda=\lambda_1}\quad\Rightarrow10\lambda_1^9=\beta_1\Rightarrow\beta_1=-10.\] Together, \[\col{1&-1\\0&1}\col{\beta_0\\\beta_1}=\col{1\\-10}\Rightarrow\beta_1=-10,\quad\beta_0=-9.\] Hence \[\mA^{10}=-10\mA-9\mI=\col{0&-10\\10&20}-\col{9&0\\0&9}=\col{-9&-10\\10&11}.\] The same idea computes the matrix exponential \(e^{\mA t}\), which is important for differential equations later on.

Example Find \(e^{\mA t}\) for \(\mA=\col{1&-1\\0&1}\). Since \(\mA\) is \(2\times2\) with \(\lambda_1=\lambda_2=1\), write \(e^{\mA t}=\beta_1\mA+\beta_0\mI\). The eigenvalue condition and its derivative with respect to \(\lambda\) at \(\lambda=1\) give \[e^{\lambda_1 t}=\beta_1\lambda_1+\beta_0\Rightarrow e^t=\beta_1+\beta_0,\qquad\left.\frac{d}{d\lambda}e^{\lambda t}\right|_{\lambda=1}=\left.\frac{d}{d\lambda}(\beta_1\lambda+\beta_0)\right|_{\lambda=1}\Rightarrow te^t=\beta_1.\] So \(\beta_1=te^t\) and \(\beta_0=(1-t)e^t\), and \[e^{\mA t}=te^t\col{1&-1\\0&1}+(1-t)e^t\col{1&0\\0&1}=\col{e^t&-te^t\\0&e^t}.\] Alternatively, if \(\mA\) is diagonalizable, any polynomial in \(\mA\) can be computed from the eigenvalues. Let \(f(\mA)=\sum_{r=0}^{p}\beta_r\mA^r\) and \(\mQ^{-1}\mA\mQ=\mLambda\), where \(\mLambda\) has the eigenvalues on its diagonal. Then \[\mQ^{-1}f(\mA)\mQ=\sum_{r=0}^{p}\beta_r\mQ^{-1}\mA^r\mQ.\]

Because \(\mQ^{-1}\mA^r\mQ=(\mQ^{-1}\mA\mQ)^r=\mLambda^r\), the change of coordinates carries every power of \(\mA\) to the same power of \(\mLambda\). So \[\Rightarrow\mQ^{-1}f(\mA)\mQ=\sum_{r=0}^p\beta_r\mLambda^r,\qquad\text{since }\underbrace{(\mQ^{-1}\mA\mQ)^r}_{\mLambda^r}=\mQ^{-1}\mA^r\mQ.\] The right-hand side is diagonal, and its entries are the polynomial evaluated at each eigenvalue: \[\Rightarrow\mQ^{-1}f(\mA)\mQ=\col{\sum_{r=0}^p\beta_r\lambda_1^r&&\\&\ddots&\\&&\sum_{r=0}^p\beta_r\lambda_n^r}=\col{f(\lambda_1)&&\\&\ddots&\\&&f(\lambda_n)}.\] Solving for \(f(\mA)\) gives a recipe for applying a function to a diagonalizable matrix: apply it to the eigenvalues, then change coordinates back.

Example Calculate \(e^{\mA t}\) for \(\mA=\col{0&1\\-2&-3}\), which has \(\lambda_1=-1\) and \(\lambda_2=-2\). This time we also need the eigenvectors. Solving \((\mA-\lambda_i\mI)\ev_i=\nul\) gives \[\begin{gathered}(\mA-\lambda_1\mI)\ev_1=\nul\Rightarrow\ev_1=\col{1\\-1}\\(\mA-\lambda_2\mI)\ev_2=\nul\Rightarrow\ev_2=\col{1\\-2}\end{gathered}\quad\left\{\begin{gathered}\mQ=[\ev_1\ \ev_2]=\col{1&1\\-1&-2}\\\mQ^{-1}=\col{2&1\\-1&-1}\end{gathered}\right.\] Applying \(f(\lambda)=e^{\lambda t}\) to the eigenvalues, we get \[e^{\mA t}=\mQ\col{e^{-t}&0\\0&e^{-2t}}\mQ^{-1}=\col{2e^{-t}-e^{-2t}&e^{-t}-e^{-2t}\\-2e^{-t}+2e^{-2t}&-e^{-t}+2e^{-2t}}.\]