Link to Podcast

The most comprehensive 2h overview of my thoughts on serious upskilling, to date. Not just how to train efficiently, but also how to find your mission. Not just the microstructure, but also the metagame. We covered tons of bases ranging from the micro level (science of learning & training efficiently) to the macro level (broader journey of finding, developing, and exploiting your personal talents).
[~0:30] What is Bloom's two-sigma problem, how did Bloom attempt to solve it, why does it remain unsolved, and what is Math Academy's approach to solving it?
[~9:00] Efficient learning feels like exercise. The point is to overcome a challenge that strains you. It is by definition unpleasant.
[~13:30] Knowledge graphs are vital when constructing efficient learning experiences. They allow you to systematically organize a learner's performance data to identify their edge of mastery (the boundary between what they know and don't know), what previously learned topics below the edge are in need of review, and what new topics on the edge will maximize the amount of review that's knocked out implicitly.
[~18:00] None of this efficiency stuff matters if you don't show up consistently. Progress equals volume times efficiency. If either of those factors are low then you don't make much progress.
[~21:30] Getting excited about the idea of getting good provides an initial activation energy, but seeing yourself improve is what fuels you to keep playing the long game, and efficiency is vital for that.
[~26:30] Your training doesn't have to be super efficient at the beginning. You can gradually nudge yourself into higher efficiency training even if you don't have a whole lot of intrinsic motivation to begin with. However, there's often a skill barrier you need to break through to really get to the fun part, and it's advisable to do that in a timely manner so you don't stall out. But at the same time, don't rush it and fall off the rails.
[~34:30] A common failure mode: being unwilling to identify, accept, and start at the level you're at.
[~41:30] Center your identity on a mission that speaks to you, that you can contribute to, and do whatever else is needed to further it, regardless of whether you perceive these other things to be "you" or not. You'll be surprised what capabilities you develop, that you hadn't previously perceived to be a part of your identity.
[~48:30] How to find your mission: sample wide to figure out what activities speak to you, then filter down and pick one (or a couple) that you're willing to seriously invest your time and effort climbing up the skill tree and going on "quests". You may not understand this early on, but skill trees branch out, and quests beget follow-up quests, and the act of climbing to these branch-points will imbue you with perspective that you can leverage to keep filtering down. If you iterate this process enough, it gradually converges into a single area that you can describe coherently and uniquely. That's your mission.
[~55:30] Every stage in the journey to your mission is hard work, and the earlier you get to putting in that work, the better off you're going to be. It's never too late, but the longer you wait, the rougher it gets. At the same time, don't make a rash decision, don't tear the house down and build up a new house that you don't even like. But don't underestimate how fast you can progress when your internal motivation is aligned with your external incentives.
[~1:12:00] Focus on what matters. That's obvious, but it's so easy to mess up lose focus and not realize it until after you've wasted a bunch of time.
[~1:15:30] How to get back on the horse after you've fallen off. How to avoid feeling bad when something outside of your control temporarily knocks you off your horse. A good social environment can push you to get back on your horse.
[~1:26:30] If you're a beginner, don't feel like you have to be advanced to join a community of learners. You can do this right away. And don't shy away from posting your progress -- it's not about where you are, it's about where you're going and how fast. It's only people who are insecure who will make fun of you. Most people, especially advanced people, will be supportive.
[~1:31:30] There are numerous cognitive learning strategies that 1) can be used to massively improve learning, 2) have been reproduced so many times they might as well be laws of physics, and 3) connect all the way down to the mechanics of what's going on in the brain. The biggest levers: active learning (as opposed to passive consumption), direct/explicit instruction (as opposed to discovery learning), the spacing effect, mixed practice (a.k.a. interleaving), retrieval practice (a.k.a. the testing effect).

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

Link to Podcast

This podcast is most comprehensive 2h overview of my thoughts on serious upskilling, to date. Not just how to train efficiently, but also how to find your mission. Not just the microstructure, but also the metagame. We covered tons of bases ranging from

• the micro level: science of learning & training efficiently
• to the macro level: broader journey of finding, developing, and exploiting your personal talents.

Not to mention, it was genuinely one of the most enjoyable podcast conversations I’ve ever had. Daniel is a top-notch host with a knack for pulling serious alpha out of people. If you enjoy this episode then I highly recommend checking out his other episodes of The Metagame!

The transcript below is provided with the following caveats:

1. There may be occasional typos and light rephrasings. Typos can be introduced by process of converting audio to a raw word-for-word transcript, and light rephrasings can be introduced by the process of smoothing out natural speech patterns to be more readable via text.
2. The transcript has been filtered to include my responses only. I do not wish to infringe on another speaker's content or quote them with the possibility of occasional typos and light rephrasings.

Justin: Thanks, I’m excited to be here.

Justin: Bloom’s 2 Sigma problem was coined in the 1980s by an educational psychologist named Benjamin Bloom. There was a particular study of his that really kicked it off and framed this problem. It compared the effectiveness of one-on-one tutoring versus traditional classroom teaching. Bloom found that the average tutored student in this study performed better than 98% of students in the traditional class. This was measured in standard deviations, or sigmas, as a 2 Sigma effect size. The idea is that it’s a huge effect. Is it one and a half sigmas, two and a half sigmas? It depends on how you measure it—they’re going to get different results based on who the tutors are and the overall context. But the idea is that there’s so much learning being left on the table, a lot of human potential going unrealized. How do we capture it? That’s the 2 Sigma problem. You can massively elevate student learning outcomes with properly individualized pedagogy, but society can’t afford to equip every student with a human tutor. So what do you do? That’s the 2 Sigma problem.

When it comes to solutions, Bloom tried to solve this himself. He didn’t just frame the problem; he actually had an entire research program aimed at solving it. His approach was that the benefit of the human tutor could be captured by combining a bunch of evidence-based learning strategies that a tutor was using. The idea was to take what the human tutor does and try to have the teacher do some approximation of that—as much as possible—and compound all these things the tutor is doing to try to reach that 2 Sigma effect.

Justin: Exactly. As you might expect, this search—Bloom found some strategies that seriously elevated student learning outcomes, like mastery learning, for instance. Depending on how you measure that and in what context, you can get about a one Sigma effect. Bloom reported a one Sigma effect in his study. Other studies have reported about a half Sigma effect. Either way, it’s a huge gain, but it’s not the whole thing. Bloom was compounding a lot of these other strategies, trying to turn all the levers possible for a single teacher with many students.

Justin: Exactly. It’s one of those things you would think—well, isn’t that just learning?

Justin: What else would you do? But this is violated so often in the classroom. It’s not because the teacher is walking in saying, “Hey, you guys don’t know anything about math. Let’s solve integrals.” What happens is each student comes in with a wildly different knowledge profile. Unless you’re trying to measure what each student individually knows and figure out what the whole class knows, chances are if you choose a problem that feels like it should be appropriate for students entering a class, it’s probably not going to be appropriate for most of the class because they all have missing foundational knowledge in different areas.

Bloom’s approach with mastery learning was to do this more systematically, where the human teacher intentionally and carefully tried to find the overlap of student knowledge profiles as much as possible and make sure the class mastered the prerequisites before going to more advanced material. But it’s a kind of spectrum—there’s an attempt at mastery learning for the whole class, and then there’s actual true mastery learning, which is completely individualized to a particular student. The closer you get to this platonic ideal of mastery learning, the better the learning and the greater the increase in outcomes.

Basically, the problem was that Bloom restricted his search space to strategies that could be implemented manually by a single human teacher with many students. He was checking the box on a lot of the important stuff you’re supposed to do if you want to maximize learning outcomes, but he wasn’t able to push it to the fullest extent of individualization you get with one-on-one tutoring. That, at least to me, seems like the fatal flaw.

His approach did increase learning outcomes a lot by compounding these strategies, but it didn’t get all the way up to the two Sigma personal tutor effect. He set it up as an incredibly shoot-for-the-moon, land-among-the-stars kind of thing. That’s basically the inspiration for Math Academy: take all these evidence-based learning strategies and remove the constraint that you have to implement them manually—one teacher with thirty students—and instead implement this programmatically in the computer.

We try to individualize as much as we can for every individual student’s knowledge profile. That’s what I and Math Academy have been working on for the past decade or so. Our entire purpose has been to construct a solution to the two Sigma problem that no longer takes the one human teacher as a constraint and removes that constraint. Let’s see how much of this we can code up into a system that does it all automatically.

Justin: Exactly. Bloom was doing this back in the 80s, and you think of computers back in the 80s as just bricks, gigantic. I can’t remember what the stats are on how many schools had computers or how many kids did, but kids were not walking around with their Chromebooks like they are nowadays. Compute power was so limited. It’s taken a while for technology to catch up.

Justin: Learning—efficient learning—should feel like exercise. Most people don’t want their learning experiences to feel like exercise. When I say it feels like exercise, it comes down to deliberate practice.

To name a handful of features of deliberate practice off the top of my head: the tasks you need to be working on must be right at the edge of your ability. Deliberate practice is about working at the edge of your ability and pushing that ability forward. You don’t want to be doing the stuff that’s easy over and over, even though that’s the most comfortable. You have to be straining, because the point is to strain. Your body—or in this case, your brain, which is part of your body—adapts to overcome the challenge that’s been presented.

The idea is that you’re putting strain on your body, your body adapts to that strain, and the strain becomes less straining. Then you increase the strain by doing even more challenging things and continue pushing yourself forward.

Of course, this is unpleasant. There are many studies on deliberate practice showing that expert performers do not feel pleasant during training. It’s not a feeling of, “Wow, this is a nice, pleasant experience.” It’s satisfying after the fact—the feeling of overcoming a challenge is satisfying—but your body does not feel pleasant. This is not a massage on your brain. Watching a YouTube video and getting dopamine hits from cool images about math is not deliberate practice.

Many people just don’t want their learning to feel like that, even though that’s the most efficient way. That’s one side. Another side where people often get this wrong is taking on things that are too challenging. It’s a balance—you need to work on things that are right at your level.

If you go too challenging, it’s like walking into the gym, putting 500 pounds on the barbell, and trying to pick it up even though you can only lift 200. You’re straining the whole time and it never budges. The challenge only helps you if it’s one you can overcome. That’s the key—you have to be able to overcome it.

It’s the middle way of deliberate practice, and there are failure modes on both sides of it.

Justin: The knowledge graph is super important because it provides a way to organize information—what a student knows and what they don’t know. You can think of all the stuff we’re talking about visually and physically when you overlay it on a knowledge graph.

There’s this idea of a student’s knowledge frontier. That’s kind of an abstract thing—the edge of their knowledge—but when you put their knowledge on a knowledge graph, you’ve got all the topics linked up. This is a prerequisite of that, that’s a prerequisite of this. It forms a directed acyclic graph that looks kind of like a tree, and you can trace out their knowledge frontier. They have a cohesive body of knowledge, and then they have things they don’t know. It’s not that there are holes in their knowledge profile so much as a jagged knowledge profile. There are two bodies, and the boundary between them is their edge of mastery.

The knowledge profile becomes the bookkeeping system for everything that’s going on—what they’ve done, what they need to work on, how recently they’ve done everything. Especially with math, or any kind of skill tree in general, you often get implicit review whenever you learn a new thing. You’re often reviewing or implicitly practicing subskills that are part of that new thing.

If you’re solving a quadratic equation by factoring, you’re also getting implicit practice on factoring quadratics and solving the linear equations that emerge. If you solve a two-step linear equation, you get implicit practice on solving a one-step equation because to solve a two-step, you have to transform it into a one-step. If you’re doing algebra with fractions—solving a linear equation with fractional numbers—you’re getting implicit review on fraction arithmetic.

If you really want to optimize the learning process, you have to track all this credit that’s flowing through the graph. You learn something, you need to review it at some point in the future. Then you learn another new skill—oh, that new skill actually has you practicing the other thing you were supposed to review, so you don’t have to review it for a while. You can wait a bit longer.

This leads to the question: if people are getting implicit review on all these subskills, why are we leaving it to chance—randomly asking them to learn new stuff and tracking things backward? Why not choose new material intentionally, so it covers all the review they need? The goal is to have you move as fast as possible learning new things that knock out all the implicit review.

The knowledge graph organizes all this thinking and makes it possible to do algorithmic treatments—to treat it like a physical or computational problem, instead of just a nebulous idea in your mind.

Justin: You can train with the best trainer in the world, and they can have you doing the perfect exercises for you. But if you only show up one day a month for 30 minutes, what’s going to happen?

There are two factors: the efficiency of your training and the volume of your training. Your progress is basically the product of those factors. If either of those factors is low, you don’t make progress. One part of it is solving for efficiency—trying to make your training as effective as possible. But the other part is that you have to show up consistently. You’re not going to make progress otherwise.

There’s no magic pill where you just have a 30-minute intervention once and now you know all of math, or you’re suddenly a bodybuilder or an Olympic gymnast. It’s a long game. You have to be consistent, and consistent in the sense that it becomes a many-times-a-week habit. You need to be serious about intentionally improving your abilities at something. You’re probably spending several hours on it per week, spread over multiple days.

There are groups of students—you can split a student population in many ways—but one particular split is between students who actually want to show up and learn and students who don’t. This often causes teachers to change their techniques, giving up some efficiency just to get students through the door who otherwise wouldn’t come.

Back to deliberate practice—it’s hard. It’s not particularly pleasant. You have to want to do it because it improves performance. If you have a class full of students with two different perspectives—some who want to engage in deliberate practice and learn, and others who don’t—it’s hard enough just to get them to do the exercise. You have to sell them on the idea of actually wanting to get better first.

These two groups are at different stages in their talent development journey, and it becomes challenging to find the right setup for both. The question is: if someone hasn’t built consistency yet, hasn’t built the habit yet, how do you go about building that?

One thing that comes up often is: what actually causes people to want to continue improving at something? Sometimes people think you can just sell someone on the idea of being really good at something—that if you make it sound exciting enough, that’ll fuel them through their talent development journey. But what really happens is that this gives them an initial activation energy that gets them started practicing. From there, the big lever is how much they’re improving through practice. Seeing yourself improve is what fuels you to keep going.

Back to efficiency—yes, exactly. The key to getting consistent is getting results. That makes you want to come back and stay consistent. But initially, there’s activation energy. You have to start before you see any results. Especially in something like fitness or math, you’re not going to see big results after the first or second session. It might take a couple of weeks, maybe a couple of months, before you really start to notice.

You think you’re going to the gym to get fitter, but when do you actually notice a change in your physique? It probably doesn’t happen for a while. You can’t be discouraged because you didn’t get six-pack abs after the first weekend at the gym.

Justin: I like what you said about putting skin in the game. You make a promise to somebody that you’re going to do something, or you sign up—you create a forcing function for yourself.

I think that’s one thing that sometimes gets overlooked, especially in education. There’s often a focus on intrinsic motivation. While it’s always good to try to build intrinsic motivation for whatever you’re doing or learning, or to guide yourself into areas where you have more natural intrinsic motivation, you can always step it up a level by adding extrinsic incentives. Even if you’re already intrinsically motivated, layering on additional incentives will get you more motivated.

That’s especially helpful at the beginning of a learning journey, when you don’t have a sense of your progress or haven’t gotten far enough into the discipline to build much intrinsic motivation. A lot of intrinsic motivation comes from playing a game that becomes fun—but the game isn’t fun until you learn how to play it. These things can feel almost like playing a game later on, but initially, getting the subskills in place to reach that point takes time.

It’s like playing a sport—playing hockey is fun once you know how to skate. The elements that make hockey fun aren’t necessarily present in the process of learning to skate. Sometimes you just have to break through a barrier. Beginners often experience this: at the beginning, it’s not going to be as fun as it’ll get later. You just have to push through that initial barrier and set up some incentives, some kind of social structure.

At the beginning, it doesn’t even have to be perfectly optimal learning. You just need to optimize for getting yourself in the door, doing the thing, and seeing yourself improve at it. Once you reach a level of comfort—where this is part of what you do, where you see yourself as good at it, enjoy it, and it becomes part of your identity—then you can start turning the dial on deliberate practice. At that point, you don’t have to protect your feelings of pleasure during the activity as much anymore.

Justin: I don’t think that sounds stupid at all. That’s a great idea. I mean, it’s anything that you want to try to get yourself to do, you just need to lower the friction enough that you start. It’s about lowering this distance between you and the thing that you’re trying to get yourself to do.

Whatever it takes, it can always start small. It can always start very small to kick a habit off of going to the gym and exercising. You don’t necessarily have to be doing what you’re doing on day 30 on day one. Day one can be just the smallest bit of intentional action towards this goal.

You just get more comfortable. Initially, it’s a new thing, and then it becomes familiar. Then you layer on some more element of newness to it. It’s almost like a deliberate practice, building your habit. You’re working at the edge of familiarity and what is part of your routine.

Okay, this is part of my routine. Let’s add more to it. Add more to it until you’re kind of doing what it is you want to get yourself to do.

Justin: I think that’s a really good question, actually. I’ve not had enough heart-to-heart conversations with people on this to really have a confident answer, but I can speculate, and I’d be interested to know what you think too.

If I were to just speculate, I think one element is—well, okay, I’ll just name some things that I’ve seen happen to math students. I don’t know the relative frequencies of these things, but some of the failure modes are:

Failure mode number one is somebody who sets the bar too high from day one. They say, I want to learn math, and then they crack open—well, okay, somebody gets excited about math because they saw all this machine learning stuff happening nowadays, and they’re like, wow, machine learning, that’s really cool. I want to learn how to do this. Everyone says it’s a bunch of math under the hood. Well, I guess I want to learn math now. Let’s see what I need to do. Oh, you need multivariable calculus and linear algebra and probability and statistics.

They go attack these high-level university courses, and meanwhile, their algebra is shaky. They forgot how to add fractions. Their high school and elementary school math is kind of rotting away because they haven’t been using that for so long. But they get so tied up in, no, I can’t go back, I don’t have time to rebuild the foundations, I just need to—they just get their ego too caught up in this goal. They don’t want to drop down to a level where it’s possible. They feel like the effort they put forth toward their goal doesn’t count unless it’s right within striking distance of the thing.

In your case, when you were going to the gym each day, imagine if you had said it doesn’t count unless I actually exercise, and my exercise doesn’t count unless I’m putting a ton of weight on the bar and also being there for 90 minutes. You put all these conditions on it, like it doesn’t count unless I do a certified Arnold Schwarzenegger workout. If you’re going to hold yourself to that level of standard right from the beginning, then you’re not going to do it.

That happens all the time, even when people know they should tone it down. But they get into the mindset of, well, lifting light weights is for weak people, and I want to be a strong person. I’m not going to lift light weights; I’m going to power through it. They overestimate their willingness to stick with something. They don’t see a lot of improvement because they’re practicing at trying to pull off skills that are too high level.

It’s just a lack of willingness to start where they are. It really comes down to a lack of willingness to start where one is—or sometimes even identify where one is. Sometimes people think they’re at a certain level, and they’re not. I can’t tell you how many people sign up for a calculus course or a math or machine learning course and think, Oh, I got A’s in high school math. They don’t realize how much they’ve forgotten and how incomprehensible their high school math courses were. They think they’re taking off right at this level, but then they actually start working on the system and realize it’s a very rude awakening to see how many prerequisites they’re actually missing.

Justin: Yeah. Especially if you’ve spent a lot of time with this kind of self-deception. Now everyone thinks you’re the nerd in your friend group who’s really good at math. Well, if you’re really good at math, why are you working on cleaning up your algebra foundations? It becomes this false perception that becomes part of your identity. Dropping down to a level that’s appropriate for you shatters this perception, this facade. You put yourself in a position where you just can’t budge.

Justin: The thing about having an identity built around being the math guy or the X guy or insert-thing-here guy is it kind of forces you to—it impedes your focus on the things that really matter to you. Most of the time, the things that really touch one’s soul are not necessarily about being well known for something. It’s more of a mission, an action—like I want to do something, as opposed to I want to be known for this or that.

In my case, during high school and college, I had some element of that—being the math guy, being knowledgeable about math, physics, and research stuff. These things were a part of me that had a similar effect. I kind of wanted to be really good at them, but why? I think this happens to a lot of people as they figure out more of who they are and what their goals are in life.

For me, it morphed into a different goal—a mission of optimizing math education. Identifying a problem, an area of the world that makes me upset—there are so many inefficiencies in math education. There are so many students who could be learning a lot more math and doing a lot more cool things in life, but there’s so much friction that gets them off the rails.

Now, I feel like my identity is more centered around this mission. As a result, it’s broadened the range of things that I do. This has led to some surprising results. For instance, I never thought of myself as a writer. I’m not sure I even think of myself as a writer now. But I’ve written quite a bit in service of this mission, and it’s gone really well. Thank you—I appreciate it. I guess I’ve just been surprised. I never thought of myself as particularly good at writing, just something where I could check the boxes on basic communication. But in service of this larger goal, I’ve been putting my reps in and thinking really hard about certain things, and this has happened as a result.

Likewise, when I started my career in tech as a data scientist, I didn’t think of myself as a software engineer who works on real-time systems. I thought, I only do the stuff that’s mathy, that you can fit in a script or a Python notebook. Deploying stuff on a server or having some real-time validations—whatever—that was never something I thought I’d do.

But then I got involved working on a mission, and the needs came up. I thought, well, I guess I’ll do this, I’ll do that. Gotta figure out how to solve this problem, solve that problem. Then years later, I looked up and realized I had accumulated a lot of software engineering skills.

I would agree with keeping your identity small. If you center your identity around something, you just take something off the shelf—this is what I’m good at, this is what I’m going to become amazing at. You simultaneously overestimate how willing or talented you are at that thing while underestimating all these other things you could be good at if you just gave them a shot.

When you have more of a mission, it’s easier to just let go of, “Am I X?” and instead think, “I’m whatever I need to be today to make this thing happen.”

Justin: That’s a good question. We talk—it’s always, you want a mission, but how do you get a mission? This is a common thing that comes up in chats with people. I want these things, but how do I get them? That’s often the less obvious part.

When it comes to finding a mission, you’re not going to point at the first thing that comes to you in life and be confident that’s your mission. There’s a wide sampling that needs to occur first for you to know what’s out there, and that stage can’t be bypassed.

Benjamin Bloom, the Two Sigma guy, did a study on talent development. He interviewed a large number of highly accomplished people across many talent fields and found that the journey to developing their talent tended to proceed along the same few stages.

Stage one was the sampling years, where you try a bunch of stuff and don’t get a lot of critical feedback on it. You just get rewarded with praise and encouragement for trying things, playing around, and doing a lot. Discovery. The point isn’t to upskill in the most efficient way—it’s to find out what quest you want to go on. You try a large number of things.

Stage two was when you pick one particular thing, or maybe two, but you’re going down the funnel, excluding things, filtering what you’re willing to invest your time into. Then you start taking your time more seriously and engage in deliberate practice to really upskill.

Justin: Even at this point, you don’t necessarily know what your mission is. You might have a goal in mind, like, Oh, I want to win this or win that. But it’s still not necessarily part of a bigger mission. It might still be a single-minded or narrowly scoped identity.

You might not even realize what your full mission is until you’ve climbed high enough up the skill tree in this domain to see the bigger picture and how you might contribute to it.

Stage three is once you’re upskilled, you’ve chosen your area of specialization, and you’ve gained serious skills in that area. Now it’s time to do something big—be a world-class performer in this area. To do that, you need to find your individual style—what makes you you, and what you’re really trying to accomplish. It’s kind of like a return to stage one, but with playfulness. You’re back to play, but now with the serious skills you’ve built up.

To put this in concrete terms, for someone who’s thinking, Okay, I’ve got all these interests—how do I know what my mission is going to be? How do I find a mission? I don’t think there’s a way to bypass these stages. You can’t just be told which one of your interests will become your mission. You have to spend a lot of time doing these interests until you get a sense of what you like and dislike.

You have to approach it with the mindset that, on one hand, you’re playing around with your various interests, but it’s also work. The point is to do things far enough to understand if you really like them or not—not just judging based on first impressions. Get a real sense: does something speak to you or not?

It’s kind of like dating. Some people don’t want to go on dates because they think, What’s the point? The odds of marrying the first person I go out with are so small. Then they never go on dates, and they never find anyone. They’re not taking it seriously.

It comes down to putting in a large volume of work. It’s going to be work at every stage—there’s no escaping it. If you don’t put in the volume of work, you won’t find the thing. If you don’t put in the volume of work, you won’t upskill in the area. And if you don’t do that, you won’t be able to play like a master in the field and find your individual contribution.

It’s work all the way. That’s my take.

Justin: I hear that a lot from people who are not late in their careers but have some kind of homeostasis in life. There’s some stability, and they’re worried about giving that up.

I’ve got two takes on this. Take number one: I think it is definitely possible to change up your life in a stable way. You don’t have to quit your job tomorrow and then ask yourself, what do I want to do? You can plan this out more, so it’s a little less rough riding. But the earlier you can figure this out in life, the easier it is to change course. The longer you commit to something, the harder it is. You’ve built your life like a piece of furniture or a house, and everything is now installed. Now you’re wanting to make adjustments to it—it’s going to require a lot of change. It would have been easier to just build it the way you wanted in the first place.

I’m not saying there’s no way to salvage the situation, but if somebody earlier in life is listening to this, my advice would be: figure this out now. Don’t just wait, don’t just go through the motions and think, oh, I’ll figure out what I want to do when I’m 30. In my 20s, I’ll just get a stable job. No—you need to start thinking about this now.

But let’s say somebody has a stable job, a fairly high salary, the golden handcuffs. What do you do in that situation?

I think the way it starts is having a hobby that you take progressively more and more seriously. On one hand, it’s interesting to explore the idea of just jumping ship right away—quit tomorrow and ask yourself the next day, what do I want to do? But let’s see what goes wrong with that scenario and how to avoid those issues.

The things that go wrong: number one, you gave up your salary. Now you have a forcing function—I need to find another paying job really fast. That’s probably going to force you to make a suboptimal decision, to commit to something that maybe doesn’t touch your soul but just pays the bills. In that case, why not just stay with your current situation?

You don’t want to make rash decisions. You don’t want to tear the house down and build up another house you don’t even like. You want to make sure it’s being built the right way the second time. Start with the room.

Justin: I think start with a hobby, but take the hobby more seriously. Go on the side outside of work. I know this is going to be more work and more taxing, but there’s no avoiding it. Go through your sampling years—your first dates with all these different things you might be interested in—and put in the work.

This is going to involve a lot of time. This is not just playing around for 10 minutes a day. Ten hours a week sounds about right. My gut says one hour a week is not serious, and 100 hours a week is obviously not possible if you have another job. Ten hours seems about the right order of magnitude.

Go through your sampling, find a thing you’re willing to commit to, and then start upskilling in that area. If this is really a thing you want to take seriously, something you’re going to recenter your life around, then you should be able to find it within yourself to commit to building it up on the side.

If you can’t really commit to building this up on the side of what you’re already doing, then I don’t know that it speaks to you enough. If you make a leap without that, you might end up in the same situation.

You want to build it up, and eventually you hit a point where you’ve built this up, you’ve become skilled in some area, and you can make money in a professional capacity in this new area. That’s when you take the leap—when you can start sustaining yourself professionally.

This might involve a pay cut because you’re not going to be amazing at it at first. But once you can reasonably sustain yourself, you take the leap, you take the pay cut, you suck it up, and then you start improving again.

To be honest, I haven’t seen enough of these things play out to be 100% confident in all this advice. This is just what I would do, and what I did with my own career. But I don’t know—what do you think? Have you seen this play out? What works? What doesn’t?

Justin: That makes sense. I totally agree. You get the alignment right, and suddenly you can think deeper about everything you’re doing, come up with more solutions. It’s aligning all the incentives—the external and internal motivations.

Justin: Totally. I think a large part of that too is having all the context spun up in your mind of what you’re doing. If you only do something for a couple hours a day, or you’re not thinking about it all the time, you turn your brain off for a long time and come back to it later. You can tell yourself you’ll deal with it later, but when you come back, you have to spin up all your mental resources again.

If you’re constantly context switching, it’s like you never fully get into it. But if you have this one mission or problem you’re working on at the forefront of your mind all the time—even when you’re taking a shower—it’s just there in the background. It never fully goes away; it only moves slightly aside to let other things occupy your mind.

For me, it feels almost like being able to hold more in working memory. That’s not exactly true, but it’s as if every neural connection in your brain is ready to fire. It’s like, I dare you to poke me the littlest amount, and I’ll fire and alert my buddies, and we’ll come up with some pattern that manifests as a solution. Everything is trigger-happy in a good way.

You get tipped by the smallest thing. You’re thinking about a problem, and later in the day you think about something random that cascades into, wait, I know what I should do—because you had everything primed and ready to go. But if you don’t have everything primed, if you’re not thinking about it much, you’ve got to get the ball rolling again. It’s like rebooting the computer to turn it on again.

Justin: Keep your hands on the boulder. It’s the idea that when you’re on a mission, trying to solve some problem, you’re pushing a boulder in some direction. It’s a really heavy boulder, but it’s moving.

This is almost embarrassingly obvious, but if you want the boulder to keep moving, your goal is to push it over to that area. Pretty much any action, any mission, can be summed up in some kind of action that involves doing the thing. Let’s just say this thing is pushing a boulder.

The minute you stop—this boulder is easy to keep moving while you’re pushing it, you’ve got momentum on your side—but the minute you stop pushing and just hope it’ll roll itself all the way over, or you take your hands off it, it’s not going to move.

This is stupid obvious, but people often get distracted by other things instead of pushing the boulder. The more clever insight is that when you’re pushing the boulder a lot, you increase your surface area of things you could be doing. You solve one problem, that opens up more possibilities. It’s easy to get pulled off by fascinating distractions. You get pulled off course—you’re doing this one thing, then, oh, that thing’s cool, let me do that, let me do that.

Before you know it, you look up and think, wait, what am I even doing? What have I been doing for the past couple of days? Then you realize you’re pushing some rocks in some direction, but it’s not the boulder you were trying to push.

You always have to realign yourself and ask, am I actually pushing the boulder? The obvious part is: push the boulder, don’t take your hands off the boulder. The part where people often mess up is they think they’re pushing the boulder, but they don’t realize they’ve transitioned to pushing something else.

Justin: Exactly. Sometimes I think it’s important to say this doesn’t mean there’s only one boulder you always have to be pushing. Sometimes you might realize that this new object you’re pushing is actually the thing you want to be pushing. This is now your boulder.

But that has to be intentional. You don’t want to get nerd-sniped by, oh, that’s cool, that’s cool—and suddenly you’re solving some problem that doesn’t actually contribute toward your solution. You want to intentionally make sure you always identify what thing is the thing you’re supposed to be pushing, and then make sure you’re pushing it.

Justin: I think that’s a really good question. That’s the double-edged nature of streaks. They help you build something up, but once you lose the streak, you might feel so bad that you just say screw it and rage quit the whole thing. You want to avoid that situation.

There are a couple of ways you can do this. The two things that come to mind are:

Thing number one is building in some kind of allowed off-day frequency. Depending on the person, this might be, okay, I always take Sundays off. Maybe that alleviates some stress, and you can say, if I screw up during the week, then I’ll make it up on Sunday because you have a buffer. Or maybe it’s once every two weeks. It depends on the person, but you give yourself a little slack in the system.

This doesn’t mean you have to take the day off. You can go through your week, and if it’s Sunday, you have the option to take the day off. You can if you want to and not feel guilty, or you can keep going and keep adding to your streak—whatever you feel like that day. Of course, this can’t turn into, I have one day off a week, two days off a week, three days off. You can’t bank them. You can’t have too frequent days off. But if you give yourself a little slack in the system, that can help you get back on the horse without feeling bad.

Because really, it’s just the feeling bad that keeps you from getting back on the horse. It’s not that you’ve broken the habit. The habit takes more days off to really break. It’s just that you’re feeling bad.

For me personally, what’s worked best is I don’t even think of it in terms of off days. Every day I want to be on. And if there’s a day I don’t get as much done as I wanted, or something knocks me off the horse, I’ve realized it’s not worth feeling bad. I would simply choose not to feel bad.

If it’s possible for someone to get themselves to not feel bad, there are a couple of ways to do that. Way number one: think about the outcome of feeling bad. If I feel bad about this, it’s going to be cognitively taxing today. I’m not going to have a good day. I might not get as much done, and that might spiral into more issues in the future—kick me off the horse more.

Another way that really works for me is just acceptance. Things are not going to go your way all the time, and you just have to roll with the punches.

Justin: I think that sort of acceptance is particularly useful to build up because it helps you in so many other areas. When we talked about sampling a bunch of different things you could be doing, the idea is that probably the next thing you try isn’t going to be the thing you do for the rest of your life, but you have to try it.

If you can go do a thing, invest effort into something, and not have that effort be fully realized, just accept that you made a bet worth making. It’s not like you have some limited amount of effort capacity to spend. You can wake up tomorrow with new effort to spend on new things. You’re not trying to conserve effort here.

If you spend your effort freely like that and don’t expect too much immediate return when you’re in these sampling years, you can’t feel bad about it. Consider it a win that you did it, even if it didn’t pan out.

I think it’s less about the result and more about knowing you made the right decision with your time. If you could play the situation over again, would you do the same things? If so, then why feel bad about it? You can make all the right decisions in a day and still not get the outcome you want based on the information you had. What are you going to do?

If you did make a wrong decision, then learn from it. Apply that in the future. Get back on course so that next time you run into the same situation, it doesn’t derail you. That’s the way I tend to think of it.

Justin: Those are cool insights. I’d agree with you on all fronts. One thing related to this that we haven’t addressed is another part of getting back on the horse: the social environment.

If you have a community of people, you can think of it like working out with a personal trainer. If you fall off your horse and don’t show up, what are they going to do? A good personal trainer is going to send you a text—Hey, am I going to see you tomorrow?—or try to get you to commit to coming again, to pull you back on the horse.

Even on X, a lot of people post their progress and get others asking, what course are you on now? How’s your progress going? If you can get involved in a community of learners or people who are working out, that sense of belonging and having others reach out to pull you back on the rails can really help.

Beginners in particular often shy away from this, even though they’re the ones who need it most. They’re embarrassed because they’re not jacked yet, or not doing university-level math yet. Maybe they’re overweight and don’t want to make friends at the gym because they think everyone’s going to laugh at them, or they don’t want to post their math progress because they’re working on adding fractions or solving linear equations. They think everyone else is going to say, wow, you’re such a moron, I can’t believe you need to fill in your high school math foundations.

But that’s not what happens in a good community. People who are working on upskilling aren’t going to make fun of you for this. The only people who would are the ones at your level or lower who are insecure about their own situation. The people who are actually skilled—the ones who are jacked in the gym or seriously knowledgeable in mathematics—they’re going to be happy for you.

When you see somebody getting their beginner gains and progressing so quickly, it’s incredibly motivating.

All this to say, if you’re a beginner, don’t feel like you have to be advanced to join a community of learners or post your progress. You can do this right away and have meaningful support, even from more advanced people. They’ll get excited seeing your progress. It’s not about where you are—it’s about where you’re going and how much velocity you have.

Justin: Sure. I actually do have a list I can go through. I remember there was a Stack Exchange question about which cognitive psychology findings are solid and can be used to help students learn better.

I initially meant to write a short response, but then somebody else wrote an answer that basically said, oh, everything’s a replication crisis, you can’t trust anything. That got me all fired up, and I just went crazy on it.

At the beginning of this, I went down a laundry list of these findings. Maybe I can go through them really quick, and if there’s anything I miss, we can talk about it—or any particular common myths or misconceptions we can expand on.

The idea is that when it comes to the science of learning, there are quite a few findings in psychology that don’t hold up—there’s a replication crisis—but at the same time, many findings do. The challenge is separating out what’s credible.

If something’s new, sounds weird, and someone just published a paper on it, you probably shouldn’t trust it much. But if something has been replicated over and over for decades or even a century, that’s a different situation—and there are many things in the science of learning that have been replicated like that.

I’ll go down the list of these things, starting with the most obvious ones and then moving into the less obvious.

First of all, actively solving problems produces more learning than passively watching a video or lecture or rereading notes. This is pretty obvious and has been tested scientifically numerous times—completely replicable, might as well be a law of physics at this point. Active learning beats passive learning.

One nuance is that active learning doesn’t mean students never watch or listen. It just means that students actively solve problems as soon as possible, following a minimum effective dose of explanation, and they spend the vast majority of their time actively solving problems.

Active learning does not mean unguided learning—that’s a common misconception.

Justin: Something interesting about that—no videos—is that when I first came on with Math Academy, I was actually developing tutorial videos. We used to have some videos, and what we realized is that they were not as helpful as we thought they would be for students.

What typically happens when a student watches a video is they click play, look at it, but they’re not actually thinking about it. They’re just staring at the screen, thinking they’re watching, but they’re really just watching the pen move around. It bypasses the critical thought of what’s going on.

Also, videos are hard to refine over time. We refine all our content—we do content analytics. Whenever some knowledge point in our scaffolding sequence for a lesson has an alarmingly large drop-off rate, where people fail the lesson at that point, we split that up. We’re always tweaking our content and refining and improving it.

But with videos, that’s harder. It’s a lot harder to do. It’s hard enough to edit a video in the first place.

Going down the line with more findings: if you don’t review information, you’re going to forget it. You can model this precisely, mathematically, using a forgetting curve. It’s basically a law of physics—the only real difference is that we’ve gone up several levels of scale and are dealing with noisier stochastic processes.

There are a number of other findings that are just as well replicated as these obvious things but are less obvious.

The first one related to this—you need to review things—is called the spacing effect. Anyone familiar with spaced repetition knows this. The idea is that more long-term retention occurs when you space out your practice, even if it’s the same amount of total practice.

If you’re going to practice for three hours on something this week, don’t do it all today. Spread it out over multiple days. The more you practice a particular skill, the longer you can go before you need to practice it again.

What really increases your memory retention is having to recall it when it’s fuzzy. That’s the skill you’re training. You’re not just training doing the thing—you’re training overcoming the challenge of doing the thing.

One type of challenge is waiting a while, letting your memory decay, and then pulling it from memory again. This act of waiting is almost like lifting a weight from your memory.

Spaced repetition is weight lifting—W-A-I-T lifting. The more you do this, the longer you can go before your memory decays, because your forgetting slows. The weight (W-A-I-T) duration creates the weight you’re lifting from your brain, and you get better at that.

Justin: Another way that you can make something harder for your brain to retrieve—in a good way that actually increases your ability to retrieve it if you overcome it—is called interleaving, or mixed practice.

That’s the idea that after you get your arms around a new skill you’re trying to learn and kick it off with some initial repetitive, consecutive practice, what you want to do once you get into the memory-building or review phase is mix it up with other skills.

For example, after you learn to shoot a basketball from the free-throw line, you practice that until you’re hitting baskets. But now start moving around the court, shooting the basketball from different places. Don’t just shoot them all from the same spot.

You want to avoid making the learning artificial. If you’re doing the same thing over and over, you already have the context spun up in your brain—you’re not training the movement from having to pull that context from memory.

What you’re training is pulling the context from memory. You want to wipe your working memory of that context, do something else, then come back and have to re-pull what you initially had there. If you do the same thing over and over again, you’ve really done one rep of what you were supposed to do—pulling from memory—and now you’re just using it repeatedly because it’s already there.

Of course, that’s after you’ve built up some initial proficiency with the skill, because the first goal is just to get yourself to the point where you can perform the skill, which sometimes takes repetitive practice at the beginning.

Justin: Interleaving.

Justin: That could qualify as an example of it. You do a jazz scale, and then once you can play the scale, instead of playing the same one over and over again fifty times in a row, maybe switch between multiple scales. The transition is what you’re training.

Play one of them once or twice, then switch to another, then another, then back to the original. You’ll probably think, wait, I had it, I was doing it just fine, and then I switched and messed up. But if that messed you up, you didn’t have it down pat. You thought you did because your fingers were in the right position, but you weren’t training all the skills involved in actually playing that scale.

That same thing happens in math learning all the time. Someone’s taking calculus, and you give them some limits problems. They say, okay, I’m getting this, this is good. Then you change it up—you move on to derivatives. They work on that, and then you bring them back to limits problems, and they say, wait, you made me screw up, you made me forget my limits stuff because you had me work on derivatives. You made me lose learning.

No—you just exposed the fact that you didn’t have that learning to begin with. We’re going to help you build it up by making you retrieve it from memory. You’re just complaining because you don’t want to lift the heavy weight off the floor—you just want to hold the weight after it’s already lifted off the rack. No, put it down and lift it back up.

Classes often get this wrong. Courses, even textbooks read cover to cover, often don’t work out well. What happens is, when you do one unit at a time and finish it in full, you’re essentially doing massed practice on one thing. You develop comfortable fluency while you’re doing it, and you think you have the skill. Then you move on to the next unit, spend weeks or a month on that, and forget much of the original.

You’re not reviewing, not interleaving—you’re just doing another round of massed practice. Because you’re not practicing the original skills, you don’t realize how shaky you’ve become on them. You don’t realize you can’t context switch back to them quickly. You don’t understand that you’ve lost that learning—or that you never really had it in the first place.

From unit one, your perception of learning was completely artificial. You go on to unit two, and you’re not testing—you’re just trusting that unit one is still there. It’s not. You keep moving on, get to unit ten, and by the end you think, I’ve mastered calculus. Then someone gives you a limits problem, and you think, I should know how to do this—but you don’t. It’s a rude awakening.

You want to be interleaving, going breadth-first through the knowledge graph, not depth-first in one direction and then another. Go breadth-first, layer by layer by layer. This also helps you build connections, layering on new skills that pull together a wide variety of lower-level ones. You’ll form more mental connections in what you’re learning if you go this breadth-first route.

Justin: Right, exactly. You’ve got more in your mind that you can potentially form connections to. When you’re doing depth-first, you’ve just got a smaller volume of area in the knowledge graph that you’re covering.

After interleaving, there’s the testing effect. The idea behind that is if you want to maximize the amount by which your memory is extended when solving problems—when solving review problems—you can’t be looking back at the reference material all the time unless you are completely stuck and can’t remember how to proceed.

You need to engage in retrieval practice, which means trying to pull the memory from your brain unassisted. You can’t rely on reference material by default. The reference material is like your spotter at the gym. Your spotter can’t be lifting the weight for you. If you’re struggling on a rep and just can’t get the weight up, your spotter can assist you, but only just enough to help you get the weight up.

In the context of reference material, it’s the same. You can’t be solving a problem alongside a worked example. You can look at a worked example, but whenever you do, read through it once at the beginning when you’re learning, then move on to the practice problem. Don’t solve the practice problem alongside it—try to solve it independently.

Maybe you get to a point where you just can’t remember what to do next. Then you flip back to the worked example and look only at the place where you’re getting stuck to remind yourself—just the minimum amount of priming. Then flip back to your problem and try to solve the rest unassisted.

When you solve review problems days or weeks later, solve those review problems cold. Don’t prime yourself by going back for a pre-review on the lesson, because that’s robbing you. Try to recall as much as possible.

These are the main ones: mastery learning, learning the prerequisites, spaced repetition, the testing effect or retrieval practice effect, and interleaving or mixed practice.

There are also other elements, like gamification, and more technical things such as non-associative interference—the idea that you should avoid learning a bunch of really similar things on the same day. For example, if you’re learning trigonometry, don’t try to cover all five different trig identities on the same day, because those are going to get mixed up in your head.

Ideally, do one trig identity, then maybe some algebra and some geometry—again, breadth-first. Minimize the interference between all these new concepts you’re learning. These are pretty much the ones I talked about just now—the biggest levers here.

Justin: At a fundamental level, upskilling—climbing a skill tree, any skill tree, whether it’s physical or intellectual—comes down to retrieving lower-level skills independently. There are so many ways people try to cheat this process and convince themselves they’re executing these skills independently when they’re not.

It all comes down to what you said—you have to pull information from long-term memory into working memory, and that’s the pull you’re trying to train. You’re also trying to form more connections between things in long-term memory.

The thing people screw up is they don’t understand that this pull is what they’re training. It’s like going to the gym—you need to lift the weights off the ground. Sitting on a bench scrolling on your phone for an hour isn’t working out. Trying to lift a weight that’s way too heavy and getting zero reps in for the day isn’t working out. Doing one rep with a really light weight doesn’t count as a workout.

As a general rule, when you’re learning, ask yourself: if this were physical exercise in the gym, what would it look like? Then you can get a sense of whether what you’re doing is real or not.

Justin: A lot of the writing we’ve discussed is on my site, justinmath.com. This was a great conversation—it was really fun talking to you. It’s always fun when we have these conversations where we go deeper and realize even deeper insights into things that were probably on both of our minds originally. I’ve definitely got some new takeaways from our conversation.

Prompt

The following prompt was used to generate this transcript.

You are a grammar cleaner. All you do is clean grammar, remove single filler words such as “yeah” and “like” and “so”, remove any phrases that are repeated consecutively verbatim, and make short paragraphs separated by empty lines. Do not change any word choice, or leave any information out. Do not summarize or change phrasing. Please clean the attached text. It should be almost exactly verbatim. Keep all the original phrasing. Do not censor.

I manually ran this on each segment of a couple thousand characters of text from the original transcript.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.