Q&A: Why are Eigenvalues and Singular Values Important?

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

You can often use eigenvalues/eigenvectors to “diagonalize” a square matrix, i.e., represent it in the form $A = P D P^{-1}$ where $D$ is a diagonal matrix (with the eigenvalues on the diagonal) and the columns of $P$ are the corresponding eigenvectors.

If you think of the matrix as a transformation, the eigenvectors represent the perpendicular directions in which the matrix stretches or compresses space. The eigenvalues are the corresponding stretch factors. $P^{-1}$ reframes the coordinate space so that the eigenvectors are the axes, $D$ applies the stretches/compressions along those axes, and then $P$ maps you back to your original space.

This is useful whenever you need to represent your matrix in a simpler way. Some examples: If your matrix contains data, you might want to see the independent attributes that your data is spread out along. Those are the eigenvectors.

• For instance, if we collect a bunch of information about cars (price, size, color, safety rating, top speed, whatever else, for a bunch of different models) and compute the covariance matrix, then one eigenvector of that matrix might be the direction of increasing "luxury," and another eigenvector might be the direction of increasing "masculinity."
• In various places throughout math, it's sometimes helpful to find a formula for each entry of $A^n$ in terms of $n.$ You can do this by diagonalizing A and then exponentiating the diagonalized representation.
• For instance, you can use this to find a closed-form (non-recursive) formula for e.g. the Fibonacci sequence.
• You can also use this to define $e^A,$ that is, the number e raised to the power of the matrix $A,$ which you can use to generalize differential equations solution formulas to systems of differential equations.
• Unfortunately, lots of matrices can't be diagonalized. In particular, you can't diagonalize non-square matrices (and there are even some square matrices that you can't diagonalize).
• Singular value decomposition is like eigenvector diagonalization, but it works on rectangular matrices too. So, you can always use it.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.