Link to Podcast

The primary key to motivation, goal-setting, understanding how to apply all the mad skills you’ve learned... it seems like it's all coming down to multisteps.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

Link to Podcast

Summary

“When/how am I going to get to use these cool skills that I’m learning in some legitimately awesome real-life context?”

This has been a common theme in questions I’ve gotten and discussions I’ve seen this week so I wanted to talk a little bit about our ideas on that and what we’ve already started working on in that direction. Summary below:

We’ve got this particular task type called a multistep task. It’s where we take some really cool context, kind of like a mini-project, and we break it up into very scaffolded pieces, roughly 8-12 questions. Each question leverages some technical skills that this student has learned before attempting the multistep. And the goal of multisteps is to just pull all of this knowledge together into a more complex and authentic problem-solving context.

Originally, the way we started out having multisteps in the system is we first created them to just emulate questions in the free response section of the AP Calculus exam. Those questions go beyond the scope of just a standard minute-long “compute this integral, compute the slope of the tangent line to this function,” and present a more complicated problem scenario. They pull together a lot of what you’ve learned from calculus in order to work your way through it.

As usual, when we created these problems, we wanted to make it very scaffolded and try our best not to frustrate students. A complicated real-world context is cool when you can actually do it, when you actually know what to do each step of the way and you’re like “wow, this is awesome, I can’t believe I did this.” But one thing that turns it from cool to uncool is when you just can’t figure out how to go about it and you’re just struggling and it’s no fun.

So we scaffolded out a ton of these multistep tasks to walk you through the process of solving problems like what you’d see on the free response section of the AP Calculus exam. that was our initial conception of the multistep task. They’re meant to prep students for the AP Calculus exam.

But recently our understanding of what these multisteps can be has started to expand.

We recently started working on coding projects, and it turns out they fit perfectly into the multistep framework where we’ve got a kind of a cool problem context that we’re breaking down into steps. The idea is that we’re just teaching you how to take what you’ve learned and apply it into some particular context that’s really cool.

@exojason brought it up a week ago that “hey, these really cool problem scenarios, they shouldn’t just be for the ML course. We should have these everywhere. We need to pepper our entire curriculum with them. This is not just an advanced level calculus or machine learning thing. We need them in algebra. In prealgebra. In grade school math.”

And we need to really lean into the cool factor. When people learn math they often do so because it’s going to get them closer to doing cool things. What cool things? It depends on the person. It might be launching rockets, modeling bioinformatic data, building a chatbot AI, mapping out how to colonize a planet… these can all be problem scenarios for multisteps.

So we want to start creating tons of these multisteps and leaning into them as a big front-and-center thing in the system.

Right now we don’t have that many multisteps, the ones we do have are mainly concentrated in AP Calculus BC and lower-grade courses, and they’re often not about the most exciting problem contexts. But if we make the problem contexts cool enough and have enough of them that students are receiving multistep tasks frequently, they can be a beacon for learners who are trying to summon up motivation to learn due to questioning why they are learning this or that.

And as Jason pointed out to me last week, people also get super excited about combining math with coding – not just in ML, but in general. Coding opens up the door to do lots of even cooler things with math and coding can be done in lower grades if students have mastered the fundamentals. (After the ML course, we’re also going to be putting together an Introduction to Programming course where students – even younger students learning lower-level math – could learn those coding fundamentals.)

And if we have tons of these multisteps, it can totally take the focus off of “ugh, when am I finally going to be done with the course” and put the focus on “wow, I can’t wait to get to that one multistep on Mars colonization tomorrow.” When you have frequent milestones like that, it’s less of a struggle to maintain long-term motivation because you’re feeling short-term payoffs so frequently. You just rush towards the next short-term payoff and we make sure that by doing that, you are effectively playing the long game correctly as well.

The primary key to motivation, goal-setting, understanding how to apply all the mad skills you’ve learned… it seems like it’s all coming down to multisteps.

Transcript

The transcript below is provided with the caveat that there may be occasional typos and light rephrasings. Typos can be introduced by process of converting audio to a raw word-for-word transcript, and light rephrasings can be introduced by the process of smoothing out natural speech patterns to be more readable via text.

I’ve seen a common theme in a lot of questions about Math Academy this past week. One thing that keeps coming up is people wondering, when am I going to get to use these cool skills I’m learning in a legitimately awesome real-life context?

I wanted to talk a little about our ideas and what we’ve already started working on in that direction.

We’ve got this particular task type called a multi-step task. It’s where we take a really cool project-like problem and break it up into various scaffolded pieces, roughly 8 to 12 questions. Each question leverages technical skills the student has learned before attempting the multi-step. The goal is to pull all this knowledge together into a cool context that gets you excited about it.

Originally, we started out having multi-steps in the system to emulate AP Calculus BC questions. On the AP Calculus exam, there’s a free-response section with questions that go beyond standard multiple choice. Instead of just computing an integral or finding the slope of a tangent line, it presents a more complicated problem scenario where you have to pull together what you’ve learned from calculus to work through it.

We created a bunch of multi-step questions in our AP Calculus BC course to emulate these multi-part free-response questions. They walk students through the process of applying calculus to solve real-world problems. As usual, we wanted to make it very scaffolded and not frustrate students.

A cool problem is cool when you can actually do it—when you know what to do at each step, and you’re thinking, “Wow, this is awesome.” But one thing that turns a cool problem into a frustrating problem is when, even if the learner has all the necessary knowledge, they just can’t figure out how to approach it. If they’re stuck struggling, it’s no fun.

So we scaffolded out a ton of these multi-step problems. In Calculus BC, we’ve got problems similar to what you’d see on the AP Calc BC exam. There are standard optimization problems involving boxes, stock and commodity prices, and modeling epidemics using exponential and logistic growth with differential equations.

This was our initial conception of multi-steps—they were meant to prep students for the AP Calc BC exam. But recently, our understanding of what these multi-steps can be has started to expand.

We recently started working on coding projects for this upcoming machine learning course. These coding projects fit perfectly into the multi-step framework, where we’ve got a cool problem context broken down into steps.

For instance, this is a blueprint for a notebook we’re working on—using linear regression to fit data that represents housing in a city. The goal is to figure out house prices and rental yields based on features like distance from the city center, year built, and square footage. You do a bunch of interesting tasks—generate graphs, fit a model, and perform data transformations to make the model fit well.

The idea is to teach you how to take what you’ve learned and apply it to a particular context. That’s opened the door for us to integrate these coding problems into the multi-step framework.

Jason brought it up several days ago, maybe a week ago, that these cool problem scenarios shouldn’t just be for the BC Calc exam or the machine learning course. We should have them everywhere. We need to integrate them throughout our curriculum, not just at the advanced level.

We need them in pre-algebra, algebra one, fifth-grade math, and even fourth-grade math. We have some in the lower grades, but we haven’t leaned into the cool factor as much as we need to. We also need to create a lot more of these multi-steps.

As Jason, Sandy, Alex, and I have been discussing what else we need in the system, there’s a long list. But one of the most common wishes people have is, “The system’s cool, but I wish it showed me how I could apply these superhero skills I’m training.”

This is essentially superhero training, but we don’t tell you how to actually be a superhero with the skills you acquire. We’re making you really strong, and the natural question is, “What can I do with that super strength?”

In a superhero movie, it’s obvious—you save the world from an asteroid or fight villains. But in math, it’s less obvious. If you’re highly technically capable with your math skills, what do you get to do that’s super awesome?

Typically, people learn about these applications by chance. Maybe they read a science book or article, or watch a video that captures their interest. They think, “Wow, that looks awesome. How did they do that?” They look it up and realize it involved a lot of math.

Maybe you know somebody in your family or meet someone who does something that seems really cool to you. Maybe they’re solving a data science problem, coding a system, or working in bioinformatics. It captures your attention and interest.

Then you start to realize there’s a lot of math behind it. Launching a rocket is probably the most obvious example. It’s captivating to see a rocket go into space and perform incredible feats. Once you start looking into how to launch or build a rocket, you realize there’s a massive amount of math involved. That can be very motivating and can also give you a sense of direction in your mathematical journey.

You’re trying to go from point A to point B. What’s point B for you? It might be launching rockets, modeling bioinformatics data, vaccine and drug discovery, or building an AI chatbot. Maybe it’s mapping out how to colonize a planet. Jason always uses that example—suppose you’re on Mars. What materials do you need? How do you construct a civilization given the different conditions from Earth? There’s so much interesting math that goes into that.

We want to start making multi-steps a front-and-center feature in the system. Right now, we have them in the AP Calculus course and in some lower grades, maybe involving tiling areas with shapes or fencing a yard.

In the past, we’ve underestimated how cool these multi-steps can be, how engaging we can make these problem contexts, and how powerful they can be for learners. For students trying to summon motivation to learn math or questioning why they are learning certain topics, these can serve as a bright beacon.

Not just in terms of motivation, but also in the capability of doing cool things—how do you do cool things? You practice doing cool things. All this skill practice on Math Academy, building up these component skills, leads to the big payoff of applying those skills to do cool things. That’s what we need to lean more into.

The thing that has brought this most immediately to our attention is when we started doing these machine learning notebooks. I posted some of these—we’ve got a number of them done. They’re about implementing machine learning algorithms, identifying failure modes, and reasoning about them.

We’re going to have neural net multi-steps where you build a neural net and use it for a digit classification task, image classification, or training it to play tic-tac-toe. Just cool stuff. People get super excited about that, especially the coding aspect.

Jason, of course, noticed this pretty quickly. Naturally, he said, “We need to have this everywhere in the curriculum.” Cool projects lean on coding. Isn’t that why everybody learns math and computer science? Because they want to do cool stuff with it?

There’s a smaller subset of people who find the subject beautiful, and that’s their primary motivation. But most people go into math and computer science because they want to come out the other end with improved capability—to be an agent in the world and do cool things.

I know I’m using this phrase—do cool stuff, do cool things, do awesome shit—but really, that’s what it’s all about. That’s why most people learn math and computer science: to do cool stuff.

We can lean into this so much, even in lower grades. Imagine a Mars colonization multi-step problem where you’re figuring out how inhabiting Mars will work. Maybe you’re spec’ing out the physics of a building or determining how to do agriculture on Mars.

There’s math and coding involved in figuring this out. Maybe you have to simulate a setup under Martian conditions. You take a system we have on Earth, model it mathematically, and then adapt it to Mars.

For example, consider the physics of a building—what makes it resistant to minor earthquakes and strong winds on Earth? What are those conditions like on Mars, and how do they change the physics required for stability?

You mathematically model a building on Earth, project it onto Mars’ physics, run a simulation, and realize, “Oh crap, the building is going to collapse. This isn’t going to work. How do we fortify it?”

Or maybe there are different materials available to build with. How do you construct a workable building under these different constraints?

Maybe we step you through the process of modeling the problem, coming up with a solution, and simulating how that solution will withstand the conditions on Mars. Then, you realize there’s a shortcoming—maybe the building is going to crumble. The resolution comes when you mathematically figure out how to patch the problem.

I’m just trying to give as concrete an example as possible of a really cool problem context that can generalize across different areas.

That’s a Mars colonization problem, but how about something like modeling the ecology of a rainforest? It doesn’t all have to be tech-related—rockets and Mars and that kind of thing. It could also be in the natural sciences.

For example, imagine subpopulations of animals and plants in a rainforest. What happens if they get a little out of balance? Maybe one population is too low. If it’s at the top of the food web, then the animals at the bottom start growing unchecked. Understanding how the entire ecological network works together is a fascinating problem.

We could list hundreds of these cool problem contexts. You could even bring in specialists—scientists who actually work on these problems—to design truly engaging, real-world problem contexts.

This could be a big deal—not just having these contexts, but actually scaffolding learners up to the point where they can do these things.

One of the problems with a lot of project-based learning as it’s currently practiced is that the projects themselves are cool, but students’ foundational skills aren’t developed enough for them to successfully complete the problems and have a meaningful learning experience.

It’s not enough for them to just feel good about the project—they need to actually be increasing their skills, not walking away with misconceptions or struggling to complete the problem and saying, “Well, I guess that was fun.” They need to actually pull it off.

This is also going to be a huge part of the puzzle in terms of motivation and mapping out a learning journey.

There are so many questions we’ve gotten that I think are answered by multi-steps.

One question is, “What course should I enroll in if I want to do this cool applied thing?” The natural answer, when you have a bunch of really cool multi-steps in place, is to look at the multi-steps available in each course. Which projects excite you the most? That’s the direction you should move in.

There’s also the question of, “Why am I learning any particular topic?” Previously, we considered implementing a way to look up the knowledge graph and see, for example, that linear equations are needed to understand quadratic equations, which are needed to do more advanced math.

But that’s not a very satisfying answer when it comes to skill-building. If you think about it in terms of martial arts, just learning more technical moves isn’t the exciting part. The real motivation comes from understanding what you’ll be able to do with those moves. It’s not about learning one punch to learn another punch—it’s about learning the punch that lets you defeat a particular kind of opponent.

Multi-steps provide that kind of motivation. They offer big, applied problem contexts—authentic use cases that connect to what you’re interested in. If we integrate them throughout the curriculum, they can create a clearer sense of purpose.

Instead of saying, “Learn linear equations so you can eventually do differential equations and get an engineering job,” which feels too distant, we can say, “Learn to solve linear equations so you can tackle this cool Mars colonization multi-step.” It puts the goal right in front of you, making it feel more achievable.

When the goal feels too far away, it becomes abstract. You don’t always understand why you’re going where you’re going. But if we scatter multi-step projects throughout the curriculum, with students tackling an exciting multi-step every day or every few days, it keeps the motivation alive.

Right now, I think we have it set at 100 XP per multi-step. Every time you complete 100 XP on the system, you get served a multi-step—at least in the courses that have enough of them.

We can use these as motivational gravity.

Learn this math topic, learn linear equations, because you get to apply them, because you get to use them as a tool to solve another really cool applied problem that you find super interesting.

Anytime you do a topic, we can tell you which multi-steps are connected to it. Maybe we can even let you choose the multi-steps you’re most excited about. You check off the ones that interest you the most, and if a course has 100 potential projects, you select the 15 that excite you the most. Then, we can guide your learning path to get you to those multi-steps as quickly as possible.

There will also be intermediate goals. Previously, we thought about goal setting in terms of XP—how much XP to complete a course, what courses you need to complete to get where you want to go, and how long it will take. But again, that goal is too far away.

Imagine having a view where you see all the cool projects you want to do and the remaining topics standing between you and each multi-step project. You could even see how many XP away you are from each one.

Suddenly, it’s not about making a multi-month journey where we celebrate 10%, 20%, and 30% progress milestones. Instead, every few days, you’re thinking, “Oh, cool, I can’t wait to do this project or that project.”

You’re still on a long journey, but every couple of days, you get to stop and experience something exciting—whether it’s a cool attraction, a great restaurant, or Niagara Falls. These moments break up the monotony of the longer journey and make the experience more satisfying and enriching.

When there’s always something exciting just ahead, you naturally put in more effort. You race toward each goal because there’s something that excites you just two days away, rather than months in the future.

Of course, you’re going to sprint faster to the two-day checkmark. Then, if there’s another two-day checkmark after that with another cool project, you just stack these over and over.

Eventually, you get to the point where you’re not even thinking about the big picture. You don’t have to motivate yourself by saying, “I want to come out the other end able to do serious engineering or serious bioinformatics.” You don’t have to continually realign yourself with the idea that there’s going to be a long period of struggle before you get where you want to go.

Instead, you’re just continually focused on, “Oh, this is cool. I can’t wait for that. I can’t wait for this,” every couple of days. You’re excited the whole time.

We make sure you are continually moving in the right direction—the one that will take you where you want to go long term—but that’s not your focus. That’s our focus. We take care of the long term. You just get excited about the short term, about these projects we’ve scattered along the way.

Anyway, I meant for this to be a quick three-minute video, but I don’t even know when I started—probably half an hour ago by now.

I just wanted to communicate that if you’re feeling like you’re missing some level of excitement about the skills you’re learning and what they will make you capable of, if you feel like it would be really motivating to see applications for the topics you’re studying, I want to say this is on our roadmap.

We’ve started pushing the boulder in this direction, but recently, it’s gotten to the point where it feels like a eureka moment. We see that multi-steps are going to be the solution to this missing piece—the fun, motivational, “How do I get to use this in real life?” side of the system.

Of course, we’ve always wanted that to be a part of the system, but initially, it wasn’t totally clear how to go about it.

We prioritized foundational skill building first because you have to earn the right to think about these cool problem contexts. If you don’t build up foundational skills, these projects don’t have the same effect. They aren’t motivating because you try them and can’t do them. You don’t learn anything from them because, again, you try and fail. You just end up confused, wasting time flailing over things that should be easy if you’ve built the right foundational skills along the way.

It feels like we’ve reached the point where we’ve nailed down foundational skill building enough. There’s still plenty to improve, like adding coaching feedback to the process or measuring when students struggle because they’re not reading worked examples. But we’ve gotten to where we can start thinking about these exciting, big-payoff, wrap-it-all-together, do-cool-stuff learning experiences.

That is very much on our roadmap—not just the “someday” roadmap, but right now. We’re starting with the machine learning course, building out coding projects, and then creating an introduction to programming course after that. This will allow us to put a bunch of really cool math and coding projects throughout the curriculum, not just in high-level courses but also in lower-level ones.

That’s going to be one of our key priorities for the near future—peppering in these multi-steps.

Going a little meta here—why am I so excited about this? Because it’s such a near-term thing. We don’t need years of additional infrastructure to make it happen. We can start right now.

It’s the same thing I mentioned earlier about doing a course or a sequence of courses. If you view it in terms of months or years, you know you’ll come out the other side with skills you can use. But when you have a near-term goal, it pulls you toward it. That’s what we’re experiencing with multi-steps being so front and center in the system. It’s pulling us in that direction.

Anyway, this has just turned into a rambly brain dump.

If you made it to the end of this video, I’m surprised—but we will have multi-steps for you in the near future. It’s going to be great.

Prompt

The following prompt was used to generate this transcript.

You are a grammar cleaner. All you do is clean grammar, remove single filler words such as “yeah” and “like” and “so”, remove any phrases that are repeated consecutively verbatim, and make short paragraphs separated by empty lines. Do not change any word choice, or leave any information out. Do not summarize or change phrasing. Please clean the attached text. It should be almost exactly verbatim. Keep all the original phrasing. Do not censor.

I manually ran this on each segment of a couple thousand characters of text from the original transcript.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.