Constant-Coefficient Differential Equations

Exponentials make a consistent appearance in the study of differnetial equations, particularly when we model rates of change in proportion to a current amount. In the late seventeenth century, Jacob Bernoulli was wondering how money grows under this model, under what we call compound interest. Starting with a dollar and doubling it each year, after one year he would earn a dollar. This is 100 percent compounded interest, but applied in a sparse and discrete fashion, with a growth only once per year. Instead, he could apply 50 percent interest, twice in a year. Or 25 percent four times. He could break this up monthly, weekly, daily, all the way until we get the continuously limiting behavior. In this limit, we truly model that at any point in time, the \textit{rate of change} is proportional (in this case equal with $r = 1$) to the \textit{amount invested} $$y' = ry.$$ Solving this equation amounts to the function that models the limit below: $e^{rx}$

In Lecture 30 we said that solutions to a linear, $n$th-order, homogeneous differential equation $L[y] = 0$ form a subspace $n$-dimensional subspace $\mathcal{N}(L) \subseteq C^0(S)$ (continuous functions over some domain $S \subseteq \R$). As with finding solutions to the homogeneous system in linear algebra $A\mathbf{x} = \mathbf{0}$, finding a basis is sufficient to finding all possible solutions. We can reduce the search from an infinite set to a finite one, as any solution is just a linear combination of our basis vectors. For general coefficients $a_k(x)$ this is hard, often impossible. However, when the coefficients are constants, $$a_ny^{(n)} + a_{n-1}y^{(n-1)} + \cdots + a_1y' + a_0y = 0, \qquad a_n, \dots, a_1, a_0 \in \R, \quad a_n \neq 0,$$ the whole problem collapses to finding the roots of a single polynomial. We will assume that $a_n = 1$, without loss of generality, to make computations simpler.

Recall the example above about compound interest: $y' = ry$ which can equivalently be written as $y'-ry = 0$. We know the solution is the exponential $y = e^{rx}$. By linearity, and the fact the differential equation is of order one, we actually know all solution must be of the form $y = C_1e^{rx}$. Let's see if we can use this idea to guess other solutions to higher order, constant-coefficient, linear differential equations. Suppose we have the equation $$y^{(n)} + a_{n-1}y^{(n-1)} + \cdots + a_1y' + a_0y = 0$$ and we think $e^{rx}$ is a solution, for some $r$. For now, $r$ is some free scalar, possible comple. Note that each derivative just adds another multipliciatve factor of $r$, i.e. $$y' = re^{rx}, \quad r'' = r^2e^{rx}, \quad ..., \quad y^{(k)} = r^ke^{rx}.$$ Following this pattern and plugging these values in our differential equation, we get $$L[e^{rx}] = \left(r^n + a_{n-1}r^{n-1} + \cdots + a_1r + a_0\right)e^{rx} = p(r)\,e^{rx} = 0.$$ That is, $L$ acts on an exponential by simply multiplying it by the number $p(r)$. Since $e^{rx}$ is never zero, $e^{rx}$ solves $L[y] = 0$ exactly when $p(r) = 0$. \begin{remark} Let's think a little closer about the relationships outlined above. Here, we have $$L[e^{rx}] = p(r)e^{rx}.$$ With functions replacing vectors and linear operators (like $L$) replacing matrices, we have a way to equate the \textit{output of a linear operator} to the \textit{scaling of the input}. This is precisely an eigenvector-eigenvalue relationship, and this highlights a core technique to solving (ordinary and partial) differential equations. Note that in the simplest case, where $L[y] = y'$ is just applying the derivative, the eigenfunctions are any $e^{rx}$ with eigenvalue $r$. Since $L$ is built out of powers of $\tfrac{d}{dx}$, it is $p\left(\tfrac{d}{dx}\right)$, and so $e^{rx}$ is an eigenfunction of $L$ with eigenvalue $p(r)$. Compare with the eigenvectors notebook, where $A\mathbf{v} = \lambda\mathbf{v}$ gives $A^k\mathbf{v} = \lambda^k\mathbf{v}$. \end{remark} \begin{definition} The \textbf{characteristic polynomial} of the differential equation above is $$p(r) = r^n + a_{n-1}r^{n-1} + \cdots + a_1r + a_0,$$ the polynomial with the same coefficients, where the $k$th derivative $y^{(k)}$ is replaced by the power $r^k$. \end{definition} Since $p$ has degree $n$, it has exactly $n$ roots, possibly repeated, possibly complex. Because the coefficients are real, the complex roots come in conjugate pairs, so the roots are always symmetric about the real axis. This is why, in the figure below, only one root of each complex pair can be dragged: its conjugate is forced to follow.

Recall from the end of Lecture 31 that the non-homogeneous equation $L[y] = b(x)$ has the same structure as a linear system $A\mathbf{x} = \mathbf{b}$. If $y_p$ is any one solution, then the general solution is $$y(x) = y_c(x) + y_p(x), \qquad y_c \in \mathcal{N}(L),$$ exactly like $\mathbf{x}_p + \mathcal{N}(A)$ on the null space and range page (and as we saw for first order equations). We already know how to find $y_c$, so what is left is to find a \textit{single} particular solution $y_p$. For constant coefficients and the right kind of $b(x)$, the same identity $L[e^{sx}] = p(s)e^{sx}$ does the work. The key observation is this: if $b(x) = e^{sx}$ and $p(s) \neq 0$, then $$L\left[\frac{e^{sx}}{p(s)}\right] = \frac{p(s)e^{sx}}{p(s)} = e^{sx}, \qquad \text{so} \qquad y_p = \frac{e^{sx}}{p(s)}.$$ Just one division! For a non-homogeneous term like $e^{\alpha x}\cos(\beta x)$, we take $s = \alpha + i\beta$, solve with $e^{sx}$, and keep the real part, since (as in Case (3)) $L$ with real coefficients keeps real and imaginary parts apart. For $\sin(\beta x)$ we keep the imaginary part instead. \begin{example} For $y'' + y = \cos(2x)$, we have $p(r) = r^2 + 1$ and $s = 2i$, so $p(2i) = -4 + 1 = -3$. Then $y_p = \operatorname{Re}\left(\tfrac{e^{2ix}}{-3}\right) = -\tfrac{1}{3}\cos(2x)$, and the general solution is $y = c_1\cos x + c_2\sin x - \tfrac{1}{3}\cos(2x)$. \end{example} The method of \textbf{undetermined coefficients} is this idea, extended to polynomials times exponentials times sines and cosines. Rather than dividing, we guess the form of $y_p$ with unknown constants, plug it in, and match coefficients. \begin{theorem}\label{thm:uc} Let $b(x) = P(x)e^{\alpha x}\cos(\beta x)$ or $b(x) = P(x)e^{\alpha x}\sin(\beta x)$, where $P$ is a polynomial of degree $m$. Set $s = \alpha + i\beta$, and let $k$ be the multiplicity of $s$ as a root of $p$ (with $k = 0$ if $p(s) \neq 0$). Then $L[y] = b(x)$ has a particular solution of the form $$y_p(x) = x^ke^{\alpha x}\left[\left(A_0 + A_1x + \cdots + A_mx^m\right)\cos(\beta x) + \left(B_0 + B_1x + \cdots + B_mx^m\right)\sin(\beta x)\right].$$ When $\beta = 0$ there is no sine or cosine, and the guess is just $x^ke^{\alpha x}\left(A_0 + \cdots + A_mx^m\right)$. \end{theorem} Two things are worth noting. First, we need both the $\cos$ and the $\sin$ in the guess even when $b$ only has one of them, since derivatives turn one into the other. Second, the factor $x^k$ is there because of the homogeneous solutions. If $s$ is a root of $p$, then $e^{sx}$ is already in $\mathcal{N}(L)$, so $L$ sends it (and any multiple of it) to zero, and no such guess can ever produce $b$. This is the same ladder as in Case (2): the first function up the ladder that $L$ does not kill is $x^ke^{sx}$, where $k$ is the multiplicity. Precisely, the derivative trick from before gives $L\left[x^ke^{sx}\right] = p^{(k)}(s)e^{sx}$ when $p(s) = \cdots = p^{(k-1)}(s) = 0$, so for $b = e^{sx}$, $$y_p = \frac{x^ke^{sx}}{p^{(k)}(s)},$$ which is the formula $\tfrac{e^{sx}}{p(s)}$ with $p(s)$ replaced by the first derivative that does not vanish. \begin{example} For $y'' + y = \cos x$, now $s = i$ \textit{is} a simple root of $p(r) = r^2 + 1$, so $k = 1$. With $p'(r) = 2r$, $$y_p = \operatorname{Re}\left(\frac{xe^{ix}}{2i}\right) = \operatorname{Re}\left(-\frac{i}{2}x\left(\cos x + i\sin x\right)\right) = \frac{1}{2}x\sin x.$$ This is \textbf{resonance}: pushing a spring at its natural frequency, the amplitude grows linearly in $x$, rather than staying bounded. \end{example} In the figure below, the non-homogeneous term is $b(x) = x^me^{\alpha x}\cos(\beta x)$ or $x^me^{\alpha x}\sin(\beta x)$, set with the sliders for $m$, $\alpha$, $\beta$ and the $\cos$/$\sin$ toggle. The orange point is $s = \alpha + i\beta$ (with its conjugate, faded), and dragging it moves $\alpha$ and $\beta$ as well. On the right, $b(x)$ is the dashed curve and $y_p$ is the orange curve. When dragged, $s$ snaps onto a root of $p$ when it gets close, and the guess picks up its factor $x^k$. The roots can be dragged here as well. Starting from $y'' + y = \cos(2x)$, try lowering $\beta$ toward $1$ and watch $\left|\tfrac{1}{p(s)}\right|$ in the computation. As $s$ approaches the root $i$, the amplitude of $y_p$ blows up, and once it lands, the bounded solution is replaced by the growing $\tfrac{1}{2}x\sin x$. For $y''' - y''$, try $\alpha = \beta = 0$ (a constant $b$): $0$ is a double root, so $k = 2$ and the guess starts at $x^2$.

\begin{remark} When $b$ is a sum of such terms, say $b = b_1 + b_2$, we find a particular solution for each piece separately and add them, since $L[y_1 + y_2] = L[y_1] + L[y_2] = b_1 + b_2$ by linearity. For example, $y'' + y = \cos x + e^x$ has $y_p = \tfrac{1}{2}x\sin x + \tfrac{1}{2}e^x$, where the second piece is $\tfrac{e^{x}}{p(1)}$ with $p(1) = 2$. \end{remark} To reiterate, the full recipe for $L[y] = b(x)$ with constant coefficients: \begin{itemize} \item find the roots of $p(r)$ and their multiplicities, which give the basis $\beta$ and $y_c = c_1y_1 + \cdots + c_ny_n$; \item find one particular solution $y_p$, by guessing its form from $s$ and $k$ and matching coefficients; \item write $y = y_c + y_p$, and if there are initial conditions, solve for $c_1, \dots, c_n$ \textit{last}, using $y$ and not $y_c$ alone. \end{itemize} The last step is a linear system for the $c_k$ (an $n \times n$ one for $n$ initial conditions), so we end up back in linear algebra once more.