The Value of Foundational Math Knowledge in Machine Learning

If you start to flail (or, more subtly, doubt yourself and lose interest) after jumping into ML without a baseline level of foundational knowledge, then you need to put your ego aside and re-allocate your time into shoring up your foundations.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

There is tremendous value in having a baseline level of foundational knowledge in whatever domain you’re in. And in machine learning, most of that foundational knowledge is math.

If you want to get into serious machine learning, beyond just “copy-paste an example from a ML library’s documentation and change the input data to your own,” then at a bare minimum, these are the math subjects you need to know for ML:

• single- and multi-variable calculus – e.g., so that you can understand, reason about, implement, and debug various forms of gradient descent
• linear algebra – not just the matrix multiplication procedure but also the theory and mechanics underlying more serious properties like eigenvalues/eigenvectors & various matrix factorizations
• probability & statistics – including continuous/conditional probability distributions, likelihood functions, Bayesian inference, ideally some information theory... this is the biggest of the 3 categories.

One of the most common - yet also most preventable - traps that people fall into, that leads them to give up on their ML aspirations, is that lots of people try to jump into serious machine learning headfirst without the above math background, thinking they can just learn whatever math they need as they go.

There are some people who can pull that off, but if you don’t already have solid evidence indicating that you’re one of them (or you don’t know what I mean by that), then you’re almost certainly not one of them.

For the vast, vast majority of people, jumping in headfirst without a baseline level of foundational knowledge is not a viable approach. It can be exciting at first, until you come to the realization that you’re spending all your time flailing around and being confused.

It’s like going to the gym, getting super hyped up to lift the heaviest weight, and repeatedly trying & failing to lift it. Not only are you not getting any stronger, but you’re also wasting valuable time that you could be using to successfully develop your strength.

Now, I want to be clear: I’m not saying that you should avoid trying to jump into things that interest you. You’ve gotten really excited about machine learning and you want to see how far you can pursue your interest jumping in headfirst? Go for it!

All I’m saying is that you need to be honest with yourself about whether you’re making progress – and if you’re starting to flail (or, more subtly, doubt yourself and lose interest), then you need to put your ego aside and re-allocate your time into shoring up your foundations.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.