Plan Your Broad-Strokes Journey Top-Down, But Carry Out the Granular Steps Bottom-Up

by Justin Skycak (@justinskycak) on

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

The top-down approach can be useful for planning a broad-strokes learning journey towards a goal. For instance, if you want to learn machine learning, then you can think top-down to figure out what fields of math you need to learn in order for machine learning to become accessible to you. You’ll find that you absolutely need to learn calculus, linear algebra, and prob/stats, and you can skip stuff like abstract algebra, number theory, etc.

However, the granular steps of the journey, the actual learning, needs to be carried out bottom-up. For instance, are you really going master computing neural net weight gradients via backpropagation by asking “what does that squiggly ‘d’ mean,” “why do you have to chain-multiply the derivatives like that,” “how do you calculate the derivative of any activation function,” etc., all the way down to the depths of whatever is the last piece of math you’ve mastered?

No, all you’re going to do with those questions is create a roadmap of what you need to learn. Which is essentially a calculus course. Except your roadmap will be terrible because you don’t actually know the subject yourself – it will have all sorts of gaps that you don’t even realize are missing because, which is to be expected given that you don’t actually know the subject.

You’ll try to climb back up the skill tree implied by your incomplete roadmap and you’ll repeatedly get stuck trying to climb up to the next branch that you can’t reach because there are prerequisites that you don’t realize you’re missing.

Most people in this situation will eventually just give up due to all the friction. Only those who have extremely outsized perseverance and generalization ability have any chance of fighting through and making it to the other side. And even then, it will take longer (and they’ll likely end up with more holes in their knowledge) than if they just sucked it up and worked through a well-sequenced calculus course.


Want to get notified about new posts? Join the mailing list and follow on X/Twitter.