Eigenvectors and Eigenvalues

Eigenvectors are simple once the ideas before them are solid: matrices as transformations, determinants, null spaces, and change of basis. If any of those feel shaky, open the refresher below; otherwise, read on. One matrix $A$ drives every figure on the page. Change it in any panel (type the entries, or pick an example) and every figure follows. All numbers are rounded to the nearest tenth.

Most vectors get knocked off their span (the line through them) by a transformation. A few special vectors stay on it, so $A$ only stretches or squishes them, as if multiplying by a scalar. In Figure 2, drag $\mathbf{v}$: the dashed line is its span and the arrow is $A\mathbf{v}$. For the 3Blue1Brown example, $\mathbf{e}_1$ is stretched by $3$ and $(-1, 1)$ is stretched by $2$, and by linearity so is every vector on their lines. Toggle on \emph{Eigenlines} to see both, or press \emph{Snap} to move $\mathbf{v}$ onto one. \begin{definition} A \textbf{nonzero} vector $\mathbf{v}$ is an \textbf{eigenvector} of $A$ if $A\mathbf{v} = \lambda\mathbf{v}$ for some scalar $\lambda$, its \textbf{eigenvalue}. The eigenvectors for $\lambda$, together with $\mathbf{0}$, form the \textbf{eigenspace} $E_\lambda$. \end{definition}

A negative eigenvalue flips a vector but still keeps it on its line: try the \emph{Diagonal} example, where $\mathbf{e}_1$ has eigenvalue $-1$. In 3D, an eigenvector of a rotation is its axis, with eigenvalue $1$. Either way, eigenvectors describe what $A$ actually does, independent of the coordinates we chose.

To find eigenvectors, write $\lambda\mathbf{v}$ as $\lambda I\mathbf{v}$ and move it across: $$A\mathbf{v} = \lambda\mathbf{v} \quad\Longleftrightarrow\quad (A - \lambda I)\mathbf{v} = \mathbf{0}.$$ So $\mathbf{v}$ is a nonzero vector in the null space of $A - \lambda I$, and by \Cref{thm:det-null} one exists exactly when $$\det(A - \lambda I) = 0.$$ Figure 3 turns $\lambda$ into a knob. The left graph is the grid under $A - \lambda I$; the right graph plots $\det(A - \lambda I)$ against $\lambda$. Drag the orange point along the axis. For the 3Blue1Brown example, at $\lambda = 2$ the grid collapses onto a line and the green null space $\mathcal{N}(A - 2I)$ appears: those are exactly the eigenvectors with eigenvalue $2$. The same happens at $\lambda = 3$.

\begin{theorem} $\lambda$ is an eigenvalue of $A$ if and only if $\det(A - \lambda I) = 0$, and then $E_\lambda = \mathcal{N}(A - \lambda I)$. For a $2 \times 2$ matrix, $\det(A - \lambda I) = \lambda^2 - (a + d)\lambda + (ad - bc)$, the \textbf{characteristic polynomial}. \end{theorem} So finding eigenvectors takes two familiar steps: find the roots of $\det(A - \lambda I)$, then row reduce $A - \lambda I$ for each root. \begin{example} For the 3Blue1Brown example, $\det(A - \lambda I) = (3 - \lambda)(2 - \lambda)$, so $\lambda = 2$ or $3$. Then $$A - 2I = \begin{bmatrix} 1 & 1 \\ 0 & 0 \end{bmatrix} \;\Longrightarrow\; E_2 = \operatorname{span}\left\{\begin{bmatrix} -1 \\ \phantom{-}1 \end{bmatrix}\right\}, \qquad A - 3I = \begin{bmatrix} 0 & \phantom{-}1 \\ 0 & -1 \end{bmatrix} \;\Longrightarrow\; E_3 = \operatorname{span}\left\{\begin{bmatrix} 1 \\ 0 \end{bmatrix}\right\}.$$ \end{example}

Not every transformation has eigenvectors, or enough of them. Figure 4 applies two fixed matrices to a ring of vectors; the ones that stay on their span are green. A rotation by $90^\circ$ knocks every vector off its span, and indeed $\det(A - \lambda I) = \lambda^2 + 1$ has no real roots. A shear has the single root $\lambda = 1$, and only the horizontal vectors stay put.

If the basis vectors are themselves eigenvectors, the matrix is \textbf{diagonal}, with the eigenvalues on the diagonal (this is the \emph{Diagonal} example). Diagonal matrices are easy to work with: applying one $n$ times just raises each eigenvalue to the $n$^th power. When $A$ has enough eigenvectors to span the plane, we can change coordinates so that they become the basis. \begin{theorem} If $\mathbf{v}_1, \mathbf{v}_2$ are eigenvectors of $A$ forming a basis (an \textbf{eigenbasis}), with eigenvalues $\lambda_1, \lambda_2$, and $P = \begin{bmatrix} \mathbf{v}_1 & \mathbf{v}_2 \end{bmatrix}$, then $$P^{-1}AP = D = \begin{bmatrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{bmatrix} \qquad\text{and}\qquad A^n = PD^nP^{-1}.$$ \end{theorem} \begin{proof} $AP = \begin{bmatrix} \lambda_1\mathbf{v}_1 & \lambda_2\mathbf{v}_2 \end{bmatrix} = PD$, and $P$ is invertible since its columns form a basis. In $A^n = (PDP^{-1})\cdots(PDP^{-1})$, each inner $P^{-1}P$ cancels. \end{proof} In Figure 5, the green and purple lines are the grid built from the eigenbasis; under $A$ each grid line only stretches along itself. Toggle on \emph{β-coordinates} to see the same transformation as $D$ acting on a square grid. The \emph{Powers} slider plots $\mathbf{x}, A\mathbf{x}, A^2\mathbf{x}, \dots$, which swing toward the line of the larger eigenvalue. Try the \emph{Rotation} or \emph{Shear} example to see a matrix with no eigenbasis.

Eigenvalues return in the differential equations half of the course, where an eigenbasis splits a system $\mathbf{x}' = A\mathbf{x}$ into independent pieces. Slides are on the LADE page.