Watch on YouTube or Listen on Spotify

[0:02:10] My background: growing up in a non-technical family and finding math on my own.
[0:05:45] Self-studying 3,000 hours of college math in high school: starting with calculus the summer after 10th grade and continuing through undergraduate-level math for the rest of high school.
[0:16:10] Whether the same ground could have been covered more efficiently — and how being responsible for other people's learning eventually crystallized the underlying principles.
[0:29:55] How having math foundations in place paid off in research: getting into Fermilab and CERN research projects at university labs.
[0:43:10] What the Math Academy learning system looks like: adaptive diagnostic, custom knowledge graph, minimum effective doses of instruction followed immediately by problem-solving, mastery before advancing.
[0:47:34] How we built the knowledge graph: years of manual work by domain experts, refined with analytics for nearly a decade.
[1:10:46] How the FIRE (Fractional Implicit REpetition) algorithm works: solving a harder problem implicitly reviews the sub-skills it encompasses, compressing the review pile significantly.
[1:35:50] Math and sport: cognitive science principles — mastery before advancing, spaced practice, interleaving — are often easier to see in sport than in math.
[1:42:00] Does doing math well require different skills than teaching it well?
[1:56:25] Automaticity as a prerequisite for deeper understanding.
[2:05:35] The anatomy of "aha" moments.
[2:14:11] Learning math as an adult: the amount of work doesn't change, only your free time does. Math Academy's Mathematical Foundations sequence covers the prerequisite stack for university math in roughly 15,000 minutes.
[2:24:10] Balancing fundamentals and exploration: exploration pays off most at the frontier of a subject.
[2:33:55] Is it ever too late?
[2:46:00] Bottom-up versus top-down learning.
[2:56:30] Students with ADHD often feel the effects of inefficient pedagogy more strongly. Interleaving minimum effective doses of guided instruction and active problem-solving is better for everyone.
[3:06:20] AI tools as a multiplier on existing ability: the more you know, the more useful they are; the less you know, the harder it is to detect when they've gone wrong.
[3:14:37] What I'm most focused on right now: taking Math Academy from workshop to factory — producing courses at scale without sacrificing quality.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

Watch on YouTube or Listen on Spotify

The transcript below is provided with the following caveats:

1. There may be occasional typos and light rephrasings. Typos can be introduced by process of converting audio to a raw word-for-word transcript, and light rephrasings can be introduced by the process of smoothing out natural speech patterns to be more readable via text.
2. The transcript has been filtered to include my responses only. I do not wish to infringe on another speaker's content or quote them with the possibility of occasional typos and light rephrasings.

Justin Skycak: Sure. My name is Justin Skycak. I’m the chief quantum director of analytics at Math Academy. We’re an adaptive online math learning platform, elementary school through university. The goal is to cover all of math and math-adjacent stuff, and we take a full nerd approach to learning.

Just trying to optimize it every moment, make sure the student is working on the learning tasks that are the most efficient use of their time. We’re all about learning efficiency.

And what I’m working on right now, what we’re all working on, and has consumed my life for the past several months of nine years, is taking us from a workshop to a factory transition where we can really turn out a lot of amazing content at the same level of quality that we have been turning out for years.

In the podcast, we’ll get into a lot of stories about our own, probably your learning background, my learning background, how we found where we’re at right now. But nowadays it’s really startup early stage company, grind mode.

And that’s just my life, totally engulfed by this tornado. In a good way, it brings together a lot of stuff, but nevertheless, there’s not a whole lot of free time for learning just for fun anymore. It’s like, let’s go, let’s go, let’s go.

Justin Skycak: Well, there’s that saying, right, nothing succeeds like success, which, I mean, on the surface sounds kind of dumb, but really it’s whether it’s building a company, working on a project, learning math, learning anything.

The more that as you experience these small wins, you get a lot of motivation to continue doing it, and that motivation, at least for me, that’s been one of the best sources of motivation, just knowing that, hey, this is working and it can work even better if I keep doing it and do it more efficiently and just see what it looks like.

Justin Skycak: What was the main driver here? This was kind of a chance event. And it was kind of like, you know, I always liked math class growing up in middle school and it was my favorite class. But am I going to go learn math outside of class? That thought just never really occurred to me. I was not that much into math, but at some point in high school actually, the summer after 10th grade, I had just finished up pre-calculus and it was a summer. And I was actually getting kind of excited about learning calculus because calculus is like, you know, when you watch a sci-fi movie and there’s always a scientist on the board. There’s got to be the integral signs, the derivatives, I mean, just at a very surface level. It’s like, what’s advanced math for somebody who’s going in through high school or learning pre-calc or algebra? It’s calculus. That’s the first thing, at least for me, that came to mind.

And knowing that, I was just, you know, hey, I’m going to learn this next year. I just finished pre-calculus. Calculus is next up. We did a little bit of calculus at the end of pre-calculus and I was getting ready to go. I was really interested in it. I knew I was going to go into STEM in some form. And it was just like, I was going to learn ANSI. So I’m like, why don’t I, I don’t know how this is going to go, but why don’t I just try and teach it myself? What’s the way, I mean, I guess I’ll learn next year at school, but if I could just teach it to myself over the summer, that’d be awesome.

It was just this idea. Nobody really planted that in my head. It was just one of those things where I was just thinking one day, why don’t I just try and do this? But that ended up turning into this kind of chance event spark that led to the trajectory that I’m in now in my obsession with efficient learning.

And so how it ended up is I found this pretty well-sequenced calculus course. It was almost like a proto Math Academy type thing. It was in some online course collection, the National Repository of Online Courses. You’d go in and there’d be a couple, it was broken up into maybe 50 or 60 or more lessons, the H.P. calculus course. You go in, you read a little bit on some slide of information and then you solve some problems. And they were very concrete problems. You got a lot of reps and stuff. So it was a great course.

Unfortunately, it no longer exists. It was sold to somebody and revamped and whatever. So you can’t just go learn it from there anymore. But this ended up working really, really well. Actually way better than I thought it was going to go. I thought I was going to spend the whole summer just grinding calculus and maybe one hour a day and come out and have learned the first third or something. But actually, just like we were talking about, success breeds success, having this small win stack up, got through lesson one, got through lesson two, solving problems. Hey, this is not a big deal. This is actually, I like it. This is fun.

And it was actually so fun that I used to play more video games and so math took the place of video games. And it was just, it almost felt like progressing through levels in a video game. And so I just kind of leaned into it and I just pulled up in my room working on calculus problems for eight hours a day or something. So I was obsessed.

And so my parents are just like, my mom’s an art major in school. My dad, business major, just logistics for health care company. They don’t, and nobody in my family really looks technical or mathematical or anything. So it’s like talking with other parents and other like, you know, what’s your kid doing over the summer? Oh, he’s just in his room learning calculus. It’s like, oh, almost as if I was in the basement playing a video game. And nobody says like, oh, yeah, my kid’s in the basement playing a video game.

And I was in the air. The song is great. I was trying to tell you that to go out and the other stuff. What did, what did we really confused, honestly, it was because it’s like, I mean, it started out like, wow, my kid is learning, taking this education into his own hands and he’s prepping for the upcoming school year. That’s great. And then just like, where is he still, he’s still deserving. It’s just like working on this problem. It’s kind of weird.

And I mean, they had never learned this kind of stuff themselves and nobody had STEM careers or anything. But so it was one of those things that was like, well, it seems like it’s a good thing. But it also seems like he’s addicted and obsessed to it. But it also seems like a good thing. So what do you do? They didn’t really intervene or anything, which I think ended up being a good thing.

I was very, very unbalanced that summer.

Justin Skycak: The 3000 hours. After that summer, I spent maybe 30 days, eight hours a day, just grinding this calculus course. And by the end of this, I was just so sucked in. I want math. Give me more math. What’s next? I can’t, it’s like, it just felt like I had found this hack or something. Nobody told me, you can learn outside of class. And I’m like, okay, let’s do this. This is working out amazingly well. I’m having fun. This is going to be great for the future. I don’t really know exactly what it’s going to do, but it seems like it’s going to do good things.

And I was like, okay, let’s keep this going. Let’s learn what’s after calculus. I finished one month into the summer and I’m like, okay, I don’t know what’s after calculus. I need to look up online. What course comes after calculus? What’s math above that? Is there math above that? I’m sure there’s math above that. I don’t know what it is.

I stumbled into MIT OpenCourseWare because that has a nice, you know, sequencing of courses. It’s like, well, what comes after calculus? We’ll just look at the MIT undergrad math major. That kind of tells you what to do. And so that’s when I started learning linear algebra and multivariable calculus. And I would just work through the, I mean, it was much less structured than this original AP calculus course that I had found. Because MIT OpenCourseWare, at the time, that was really just a collection of lecture videos and some problem sets.

Now, I know they’ve built out some of their foundational courses a little more since then, augmented it with more learning materials, but at that time, it was very, very rough writing. Here’s 20 lectures and here’s all the problem sets. Go. And so I just did that and I found some PDFs of the textbooks and I was just doing that.

I did that the rest of summer into the school year when I got to school. That following, like, 11th grade, this whole self-study experience was making me realize how much time I was wasting in school, just kind of sitting there in class. And so I was like, well, I’m not going to go back to that. I’m going to keep leaning into this thing. So I would take the math books with me to all my classes and just be doing that instead, while trying to look like I’m paying attention and stuff. Not going to confiscate stuff like that.

But yes, I kept that going the following year. And, you know, this 3000 hours thing, it was way more time than I really needed if I were working efficiently on learning this stuff. But I was, I mean, as most teenagers, I think I know everything. I know how to learn. I did not do my research in how to learn efficiently. I’m just like, okay, I’m going to go do it. And so I ended up hitting my head on basically every sort of ledge there was to hit my head on along the way.

I was kind of just discovering a lot of these principles. In hindsight, really obvious principles of learning science. But I remember there was a day or so that I initially came into it thinking, why am I working all these problems? I can just watch all the videos and learn it that way. And then I took a day and I just blew through what I thought was a third of the course watching videos. I’m like, this is such a hack. I don’t have to do the problems. I can just do the videos.

And then the following day, I just tried to work out some problems and realized I don’t retain anything from it. And so that was just the learning of like, okay, I have to do problems. And so every, basically every single cognitive science principle, I violated an issue during my self-study and kind of learned from that.

Justin Skycak: You know, I mean, on one hand, I think there was a little bit of value to having tried some inefficient forms of learning and only to realize, hey, this doesn’t work. And I think in any discipline that you are seriously going into, that’s going to be your main thing that you’re doing. It’s worth exploring the space of, you know, try things out, what works, what doesn’t.

But I just think about that spending 3,000 hours to get from calculus all the way up there. I think I got through about real analysis, abstract algebra, you know, junior level math major courses and also some physics and stuff. If I had, you know, there’s just a lot of time that I’d waste just kind of being confused, doing inefficient techniques that did not contribute to this holistic view of learning.

A little taste of, you know, did it wrong. Okay, correct to this. There’s maybe some small amount, maybe a couple percent of the time, I would say was kind of useful in exploring the space. But that came to a point where I was really like, I just want to go fast and learn this stuff. And it was trudging through mud. And the higher I got, the more things would come back to bite me, like not doing a proper review system and just working on problems that were way too hard.

A lot of this stuff, I mean, some stuff I figured out on the fly, but a lot of it, I really did not learn until years later how to do this really, really efficiently. And if there’s one thing that I wish I had more during my teenage years and just as a student, it would have been guidance on just how, just tell me, just tell me how to do it.

I just want to learn math and I’m fighting my way through getting my head on all these ledges. Maybe some amount, a small amount of that is kind of shaping my view of understanding what works and what doesn’t. But most, the vast majority of that time, for me at least, felt like it had just gone to waste.

And there was some instances of, I also, part of my goal in this was, I want to just learn the core body of math so I can contribute to do some cool research projects. And there came a point around maybe 2000 hours in. I’m just like, you know, I think I’ve learned enough and I’m going to do research. And what was this based on? This was just based on the sheer volume of time that went into learning, not based on how far I had gone.

And so I ended up doing a research project, which was just kind of on my own. It had to do with finding a generalized formula for partial for actions to become. And I spent a whole summer on it. And I used, it was kind of fun. But what ended up happening is I was kind of groping around in the dark in a direction that I would later find out was just made completely trivial by some results in complex analysis.

I mean, it’s a high-level undergrad thing, but it’s not exactly super advanced math. And so it’s like, well, if I just. You’re trying to be discovered stuff that we’re already known.

Justin Skycak: Cognitive science actually was not really on my radar until, well, I guess after high school, sort of at the beginning of college, I was sort of interested in neuroscience, computational neuroscience, and that led to kind of interest in AI and stuff. And that, I mean, it was kind of cognitive science adjacent.

But what really flipped the switch for me was once I was responsible for other people’s work. Starting out, I did a lot of tutoring in high school also and I really liked tutoring and just math education in general. At one point, actually during college, I had a full-time data science job and after graduating college, I was like, you know, screw it. We’re not doing data science anymore. We’re doing math education and we’re going to find some way to incorporate all of the coding and whatever that I had enjoyed into math education.

But it was really once I had a classroom of students who were kind of depending on me to set up some kind of structure that is going to result in them learning this material. That’s when I realized a lot of things that were, there’s some of the less obvious stuff, like the importance of spaced review and interleaving and stuff.

And that’s about the time that I also met Jason Roberts, founder of Math Academy. He and his wife Sandy founded it together. And I was teaching in the original school program there. He had done a lot of, you know, just reading around the subject of cognitive science. It was incorporating those principles.

And I was like, oh, wow, this is really interesting. And for once, it’s like, okay, this is the guidance that I was saying that I didn’t have. I was like, I was groping around, discovered maybe a third of it on my own. It was like two thirds at least that was just like, oh my goodness, that makes total sense. I go like, okay, do that, do that, do that.

And it was working in the classroom too. The more that we would leverage these cognitive science principles, the better the students would learn the material. And then at some point after that, actually around maybe three years ago or so, Jason was like, hey, we need to write up about how the science of Math Academy works. Can you just make a little information for the web page? And I started on that and looked up about 400 pages later that we had the Math Academy, what it was big book and how we apply the cognitive science learning, for instance.

Justin Skycak: Oh, I was just gonna say, I totally agree with, I’ve had that same experience with, you know, high-level professors. You sign up for the course. You’re like, wow, this professor’s really good at research. I’m going to learn so much.

And then you get in there and then you get served up these slam dunk problems that are supposed to, the professor thinks, oh, I’m going to create these really just a few really nice, really interesting problems and give them to the students. And you get those problems. You’re like, what is this? I don’t even know where to begin. And you just are flailing around and the professor is, you know, not really helping bridge the gap from you to them. And I’ve seen this too in a lot of students I’ve tutored as well. It’s a lot of engineering.

Justin Skycak: We’re just kind of accepted that the class moves in lockstep and there’s a normal distribution of grades at the end. And some of these grades are failing. Some of them are, just barely understands the material. A few grades are, okay, they get it. We’re just going to accept this system. But really, if you have every student working on what is the most efficient use of this.

Justin Skycak: I’ll talk a little bit about that. The Fermi Lab and CERN projects, these were projects that I did with local universities. Fermi Lab was with Indiana University South Bend, where I grew up, in the professor Ulan Levine’s lab. And then CERN Project was with a lab at Notre Dame. That’s a Core Connect program which connected some of the top high school students in the areas with working on projects that were going on with some labs that are at Notre Dame.

And the math, it was kind of fun. It helped in a lot of big ways, get my foot in the door in terms of these projects because, you know, you learn a bunch of the advanced math. You stand out. In my high school, I was like, okay, who are we going to pick? They could pick one student to send to this Core Connect program at Notre Dame. And they’re like, well, who are we going to pick? Well, Justin is just crushing the calculus course as a grade earlier or whatever. I guess, and he seems to be learning physics. I don’t know, I don’t know, send him.

That was just an obvious decision. Because that’s what happens when you get really, you know, you find your obsession. You get really ahead. People take notice and you get a lot of opportunities to pursue that further.

And with the other project, my junior year, I was working on the Fermilab project. And that was actually just, I was taking this research class in my high school where the point was to reach out to local professors and just try to get your foot in the door working on a research project with them. And part of the thing that made me seem like, hey, this kid is going to actually be able to help contribute a little and not just be a total waste of time was the fact that I already have my math foundations in place for all this stuff. And I’d taken it from myself to learn.

And when I got to these, you know, started working with these labs, math was just a nice thing. Math was just a non-issue. I remember for so many other students in my high school research class, they would go off to their labs as well during the school day. And just at the end of the school day, for an hour is the last period of the school. You go out and leave early.

A lot of other people would be struggling with some of the math involved sometimes. You know, maybe somebody would get into a physics lab and they would not only have to spin off on the context that the lab is operating and what research are we doing? What are the materials? But there’s just a big mathematical gap that was placing hard limits on this type of stuff that they could do.

Now I didn’t have that gap. And that was good. Though I did have some gaps in other areas. I mean, most of my projects had to do with experimental physics. There was, you know, math was one side of it. But there was also, you know, the Fermilab project, I was trying to make a material to better transmit sound. And I had never, I was not a very hands-on lab kind of person. That was definitely a point of, oh man, this is challenging.

But in terms of, you know, math just became a not problem. And I think to anyone who wants to do research, especially high school research, removing skill gaps is such a big advantage there. Because you can, you know, just imagine, say there’s some lab working in some setting, some research setting. If you have all your foundational skills in place, then you can just take off running and be a serious contributing member of lab.

Every little pillar of foundational skills that you don’t have in place is going to slow you down. I would say in my case, okay, the math was a pillar that I just knocked out of a park. That was a non-issue. Some of the more hands-on experimental physics stuff was definitely more of a drag factor for me because I never trained up, you know, that kind of hands-on.

Justin Skycak: You know, I could have leaned into this a lot more. I came in wanting to do theoretical physics. Okay. I was like, I don’t want to do experimental physics. I want to do physics, and I mean, you’re a high schooler. You’re like, hey, can I get involved in this physics stuff? Then they’re like, yeah, you’ll take whatever. You’ll take whatever this project.

And I’m like, I’m not about to turn down the experimental physics opportunity. And it’s like, no, I want to do theoretical, that or nothing. Get out of here, kid. What are you talking about?

I was a little, you know, doing these, I had not quite found my really good fit with these physics projects. It was a halfway good fit, right? That math pillar, okay, that was good. But the experiments, I think this is a pillar. I was just, you know, I wasn’t really interested in that kind of stuff. And I did not take it upon myself to learn this stuff outside of lab.

And it was kind of like, I just did one of the mill projects there. And this was enough to, I ended up making junior year to the Intel International Science and Engineering. The project was good enough to get to that level, but it was not good enough to get place, get a medal or result in a paper or stuff like that. Or even kind of really set a direction for me to, I want to continue looking into this more.

Justin Skycak: That’s interesting. I actually have a question for you on that.

You joined this molecular biology lab and you mentioned you had strong math and physics foundations. When you were joining the lab, were you leveraging the math and physics in your stuff?

Absolutely not. Or okay, it was just very zero biology methodology. Okay. That’s interesting.

Justin Skycak: That was fun. That’s you going and test during those people with all your questions, which I guess initially were a little bit novice questions. But then you build up this storm master that really takes, you know, put ego aside and I’m going to ask the dumb questions because I have to learn this stuff because you don’t get results.

Justin Skycak: It’s very real, you know, once he has something of value there.

Justin Skycak: Starts out with the diagnostic test. Figure out what you know, what you don’t know in whatever course you’re trying to take. It’s going to look back for missing prerequisites many years back into NASA. If you’re placing into calculus, it’s going to test you on a whole bunch of algebra. You don’t know completing the square? Well, guess what? You’re going to learn completing the square. I’m going to add that to your learning plan.

Makes this custom course for you. Now you got a custom course that you’re going to go through. This is kind of your knowledge graph, and we know what you know, what you’re ready to learn, and what you don’t know. And we’re going to serve you stuff that you are ready to learn. This is going to be under your knowledge frontier. We call it your edge of mastery. And we’re going to build up your understanding of these topics.

Now what does a single learning experience on a topic look like? Well, this is a lesson and it’s kind of a sequence of minimum effective doses of instruction and you actually solving problems. Think this whole situation is very similar to if you were to go with a personal trainer. What are they going to do? Well, they’re going to first just see, okay, what are you trying to do? What sport are you prepping for? Or are you trying to increase your vertical jump? What are you trying to do? Where are your weak points and what can we do to get you, you know, better at those?

They’re going to have exercises for you and each exercise they’re going to, you know, I’ll demonstrate it once. We’ll talk, this is what it is. Here’s how you do it. Okay, now you practice. And that’s what a lesson is like. It starts out with just a brief introduction. Then we immediately launch into problem solving. You see, okay. You got your introduction. This is the type of problem that we’re going to focus on solving. Here’s an example of how you can solve the type of problem. Okay, now you do a few problems.

If you’re just knocking it out of the park, you answer the first two questions in a row, it’s like, wait, you got it. Let’s move on to the next more challenging version. If you’re struggling a little bit, you’re like, oh, first question wrong, next one right, we’re going to make you solve a few more problems just to, you know, you got to end on two successes in a row before advancing.

Justin Skycak: Was this inspired by that sort of, this is born out of our hands-on experience teaching. And just, you know, the tutor is really, I’m glad you brought that up because the tutor is our model organism basically. We’re trying to emulate the decisions of an expert tutor who knows everything about your knowledge profile, what you know, what you don’t know, and has unlimited computational bandwidth to have you, you know, it can just make up problems, but it’s got a whole inventory of problems that can match you with. And every single answer that you do, it’s going to kind of take that into consideration to get you working on

Justin Skycak: That stuff, that whole gap. We started this stuff, I mean, it’s pretty little arms, right. There’s just no way to generate any of this information. We start the stuff back in 2016, I think-ish, is when Jason and our director of curriculum, Alex Smith, started working on this thing. I joined around in 2018. And a lot of stuff, we’re hand crafting it manually.

And to this day, everything is very, you know, it comes from a human expert just paying very close attention to, based on their experiences teaching and what they know about max efficiency learning and how to get students through material in a way that’s guaranteed to work. How do you do it?

And we built a lot of tooling over the years to make this process more and more efficient. We’re leveraging all the tooling that we can. But ultimately, it comes from a human domain expert is making sure that all of this is to their standards and this is all mapped out by them.

This is Alex Smith, our director of curriculum. He builds the forwards knowledge graph of, okay, what are the topics that we want to have students learn? What is this course, this calculus course? What are all the topics in it? We got about 300-something topics in our calculus course. Each one is an atomic unit of knowledge consisting of roughly three or four knowledge points.

It starts out with the simplest version of the problem and then layers on additional complexity as you evidence your ability to solve problems in each successive knowledge point. And he maps all this out. He connects everything up, the prerequisites. What topics do you need to have learned before we serve you this new thing?

And I kind of encode this backwards graph we call the encompassing graph. And it says what skills encompass what other skills, what sub-skills are you exercising as components of more advanced skills. And this helps with our spaced repetition algorithm, which is always, you know, when you do a review on an advanced skill, you’re kind of implicitly practicing a lot of skills.

We’re always trying to, you know, something that we do is we’re always trying to maximize your learning efficiency. If we can get you practicing your sub-skills that are, you kind of do for some review, we can get you practicing those by learning something new. Learn a new lesson that knocks us out, then the algorithm serves.

Justin Skycak: Exactly. And something about just the whole, you know, approaching this as, why not just generate this, just try to generate all the content with an LLM or something?

Well, the thing is, a lot of this content, the reason why it works so well is because it’s based on all of our learnings throughout the years of doing this manually. And while LLMs are, I mean, they are really good at traversing this whatever knowledge that they’ve been trained on, right? Everything that’s been written down on the internet, and often applying sequencing of that knowledge to kind of expand beyond the initial knowledge base, there’s a lot of stuff that you learn by just working with students hands-on and just kind of almost extracting data from reality itself. Not just a compressed version of, this is reality according to all the text that’s been written on the internet. There’s a lot of stuff that’s kind of off the

Justin Skycak: I mean, that’s one of the things that, you know, the design of minimum effective doses of instruction and practice. I mean, part of this is just inspired by, you know, cognitive science’s active practice. Let’s maximize the amount of time that you’re actively solving problems with guidance.

But another part is also just inspired by Jason Roberts, founder. He always talks about how, when he was sitting in math class, it’s not that he hated math, it’s that he hated being talked at for an hour. And so this, so many, we’ve realized that so many students, especially students with ADHD or just sort of really can’t sit still type of students, they don’t want to watch a video for an hour. They don’t want to watch an hour, hour video.

And if they do, it’s often because they just want to stare off into space and not think about it. What really keeps the students engaged with the process is actively solving problems that are at the right level of difficulty, kind of entering that flow state, right at the edge of their ability.

But I want to say, in addition to just designing the lessons internally to try to keep the experience maybe a little challenging but very achievable for students, the other part is there’s also the whole connectivity of the knowledge graph. This is stuff that you probably would not think about if you’re just trying to maybe get an LLM to dump out a knowledge graph or just ask somebody else to design a knowledge graph.

There’s a lot that comes into play in terms of structuring the knowledge graph to make students successful. And we have to think about things like quantifying the cognitive congestion of various nodes in the knowledge graph. If you have a topic that has 10 different prerequisites that are all being pulled in that you’ve never practiced pulling in together before, that’s going to inflate the cognitive load a ton.

And there’s just a million of these little optimizations that we put into to make the knowledge graphs really easy to learn from.

Justin Skycak: Exactly. It’s kind of, I think that’s a good analogy. If you take a textbook, and let’s say that this textbook was constructed not just by one author just doing a brain dump in isolation, but an author and an expert on the subject who’s also looking at all the other textbooks, all the other treatments of the subject, trying to just hit all the bases, a comprehensive course.

And they write this textbook. And then, instead of presenting it to the student in this linear sequence, just, here is the unit on, I’ll just use algebra as an example. Here’s the unit on linear equations. Okay, now let’s move up to quadratic equations. Okay, now let’s do some trigonometry.

Instead of presenting by units, you really chop it up into the atomic bits of knowledge and that way you can really trace out the student knowledge as a contiguous area in the knowledge graph. Exactly what they’re ready to learn. You can deliver the next piece of knowledge. You can kind of adapt the number of problems to the student.

And you’re also, you keep it interesting, right? You don’t have to just do one unit for an entire week or two weeks. You know, at school, I often, this is, this quarter, we’re doing trigonometry. Nothing but trigonometry for three months.

Justin Skycak: Three months. I know. You got sick of it. In addition to just getting sick of it, that’s not even an efficient way to learn the material. Ideally you want to be spacing it out, coming back, revisiting it. You don’t want to be blocking or having your practice on the consecutive material all the time. Interleaves are all the different units.

Justin Skycak: It’s an impossible task for one. That’s a good, because 700 students, how are you gonna, you can’t, one teacher cannot deliver an optimal learning experience to some hundred students simultaneously.

Justin Skycak: Well, I think it’s an area that is, it’s not a radar worth paying attention to. At this moment, there’s a lot of challenges with that sort of set. It’s one of those things that’s like, I mean, there’s a pull, right? You want to imagine, you think of efficient learning, you think of this personalization, you often imagine some tutor having a personalized conversation with the student around their interests, relating the material to their interests and stuff like that.

Now, that can potentially work really well in some cases. There’s a lot of surface area for things to go off the rails. And just to name a few cases. Sometimes students want to kind of get out of scope of a topic. You know, you’re teaching one thing and they’re like, oh, what did you do this? What did you do that? Oh, blah, blah, blah, blah.

And important to this can enrich the learning experience, but there also comes time when it’s like, you know, kid, I know you’re excited about this stuff, but we’re just talking in the abstract right now. I used to, back when I was teaching, I would have some students sometimes who would ask me tons of questions about, we’d be learning introductory calculus stuff. And then they start asking me questions about, what’s the hardest integral to solve? Or are all integrals solvable? How do you know or how do you not know?

And pretty quickly you get into a realm where you’re no longer really doing skills-based teaching. You’re just kind of talking around the subject, almost as if they were watching a YouTube video, some visualization and what’s the hardest math problem or that kind of stuff. And it can be, there’s some moment of, well, okay, you got to kind of scope it down to know, there’s a balance to be had, and you don’t want to allow the scope of what you’re doing to delete too much.

Another part is that, in addition to trying, making it non-gameable and stuff like that, there’s a challenge in that generating content on the fly is always, I mean, at all arms, there’s stochastic generators, right? And you don’t, I mean, you can, it’s gotten a lot better at all the hallucinations and stuff is definitely an order of magnitude improvement in that. If you don’t have static content underlying, it’s a little harder to, here, I dig the gamble.

And additionally, just to name one other thing. One thing we do a lot with our knowledge graph is, that’s an advantage of having this kind of static content, is analytics. And every student is being served the same exact step-by-step lesson that we have been optimizing for years to be the most, assuming the students have mastered the prerequisite leading into that lesson. This is the most efficient, most robust way of getting a student to learn the material on that topic.

And a lot of these optimizations, maybe we had three knowledge points and knowledge point number two was actually a little too aggressive and instead of 98% of the students getting through it on the first try, it’s only 80% or 70%. And then we break that up into two steps and we do all this content has gone through so much refinement, analytics refinement.

And once you kind of get rid of this idea of static content, it makes it a lot more difficult to do analytics. Not to say that this could never work, just to say there’s still lots of challenges.

Justin Skycak: I mean, it’s definitely something that I think will remain on our radar. It’s like, you know, the idea of, I mean, it’d be cool to, I mean, we talked about the idea of, you know, some people, when content is presented in a way, making analogies, or using word problems in a setting that they’re just really interested in. Like, oh, they’re interested in soccer. Okay, let’s make this word problem. We’re doing all our word problems about soccer. Let’s just make a fun, that can definitely elevate the learning experience.

There’s other elements, just to say, not that you’ve suggested this, but a common confusion, though, on this point is some people think that students need a million different explanations of a topic until one clicks. And that’s what I’m used to hearing in terms of suggestions for using other lines of the content.

But what we found is that if you just make sure students have all of their prerequisites in place, leading into the lesson that you’re asking them to learn, then they’re just prepared for it. And oftentimes, what you experience as a tutor, when you have to explain things a million different ways for a student to get it, is really, there’s something missing and you’re just trying to cover the space. What is this missing thing? I don’t know. I mean, I’m just rapid fire to try to hit it, which, if you don’t have a mastery learning system that is guaranteed for you, because it’s simply in place, that’s not a bad heuristic.

And we are just on the topic of framing things in ways that are really exciting to students. One of the things that we actually have coming out in one of our upcoming courses is projects that are just based in really interesting scenarios, at least that we think are very interesting. It was a little interesting to people who are interested in astrophysics, modeling, planning out civilization on Mars, translating writings from long lost civilizations, stuff like that.

We got a lot of cool projects like that. But the most, where this path leads, is kind of matching, okay, what kind of projects are you really interested in, to the student?

Justin Skycak: Imagine that you wanna learn math. You know you gotta do some form of spaced repetition because the spacing effect is one of the few free lunches you get in learning. If you space out the material and serve it at the right time, you can kind of minimize the amount of review that you have to do while maintaining a certain level of retention.

How do you do that? Well, option one, this is a thought process that kind of leads to this algorithm. Maybe you say, okay, I’m doing all these math problems and I gotta make up Anki formulas for them, or Anki flashcards. And the first thing I think about is, well, what do I put on my flashcard?

On one hand, there’s some formulas and stuff in math, but so much of math is, if you just memorize the formulas, that doesn’t mean you know how to apply them or how to solve problems with them, right? It’s like, well, how do you do Anki for actual problem solving?

Well, okay, the solution to that is just put problems on flashcards and just have a system that, instead of serving you the same problem, it just replaces it with a slightly different problem. And those are kind of your repetitions that you’re doing.

Now what you run into pretty quickly in the setting is this thing, the review hell basically. You already get this with flashcards, right? The ones that they can go through in two seconds, three seconds, they pile up. You got so many flashcards to do. And even though you can get through them so quickly, now you’ve just made this mathematical problem solving, right? Now it takes you a minute and two minutes to go through each flashcard. And now you’ve got this massive backlog of review to do.

What do you do? You start thinking about it. Okay, say you’re in this situation, you got this massive backlog, right? And you’ve got to practice solving a linear equation. You’ve got to practice solving a quadratic equation. You’ve got to practice fact. Wait a second. If I solve this quadratic by factoring it into linear pieces and then solving each equation arising from the linear piece, I have effectively practiced solving linear equations.

If I solve a linear equation, 2x plus three equals seven, I am practicing subtracting and dividing or that kind of stuff. You’re like, wait, I don’t have to do all these cards. If I do this card, if you just take a card at random, okay, I did this card and I can remove these other cards from the pile because I’ve effectively reviewed those as sub-skills.

That starts to get into compressing your reviews, because the next thing that you want to do after that is like, wait, I don’t want to just draw cards at random from the pile that I have to do. I don’t even want to necessarily draw the card that is most overdue. I want to draw the card that hits the biggest number of due repetitions with the one problem. What’s the one golden problem that is going to make the biggest dent in my review pile?

And that’s where you kind of get into the knowledge graph. FIRE, it was an acronym for fractional implicit repetition. The repetitions, you’re getting implicit repetitions on sub-skills that you’re practicing as a component of each more advanced skill. And oftentimes these sub-skills are fractional in the sense that, well, you solved this problem, it encompassed a hundred percent of the cases of the sub-skill.

Maybe quadratic equation, you factor it, you solve the linear components. Okay, you’ve handled the Ax plus B equals C equations. Or maybe it’s a quadratic equation, x squared plus, leading coefficient of one. You’ve handled the linear equations that are like x plus two equals three, stuff like that. But what about when there’s a coefficient on the x?

Maybe there’s some cases that you don’t cover, so you kind of track what skills are, what fraction of the sub-skills are being encompassed. And then as you do each review problem, these repetitions, they propagate through the graph and trickle down. Almost like you can visualize it as strikes of lightning, trying to just, this thing covers a bunch of stuff below.

Justin Skycak: That’s one of the secret ingredients in this stew of maximum efficiency learning. I mean, each of these things, you know, make sure the student has prerequisites in place before you ask them to learn the new thing. Optimize the new thing into a sequence of steps that are very bite-sized for the student to learn.

Move the student, make this minimum effective doses of guided instruction and problem solving. Don’t make them do 30 problems of the same thing when you know, okay, they just nailed the first two, ready to move on. But at the same time, give them more practice. That’s the spaced repetition, periodic quizzes. All of this kind of combines together. Each of these is a multiplier on the learning efficiency.

Justin Skycak: Well, I think, absolutely, there are knowledge graphs for any subject that is hierarchical. Computer science, physics, anything math-adjacent, for sure is a really clear candidate for a knowledge graph.

Now, this is actually on our, you know, we just released a mathematical methods for physics course. And we have a machine learning course that is almost ready. We are actively expanding out our knowledge graph beyond math proper.

Now, how far can we expand it, how into biology or history? I mean, some just get flatter and flatter the further you move from math. But my gut says you can take this probably area, things that transfer over to some degree.

I think it’s kind of like in math, the hierarchy is kind of strict, whereas you have to learn linear equations before you learn to solve a quadratic equation by factoring. There’s just no, you have to, it’s a practice sub-skill, there’s no way around it.

And I think in other subjects like history, there’s sometimes not necessarily one right answer to, you have to learn this before that, but there may be ways that work out nicely. I don’t know, I have not really thought a whole lot about beyond hierarchical subjects too much, but I know if a subject is hierarchical, you can teach it with a knowledge graph.

Justin Skycak: You had mentioned just the experience of learning biology and archaeology and the different experience. Last fall, I started kind of dipping my toe into learning biology, trying to leverage these cognitive science principles, mastery learning, this kind of stuff. And just kind of saying, what does that even look like?

And I don’t know that I figured out the most efficient way just yet, but I guess I can tell you a little bit about what I kind of stumbled into. And I’d be curious to know how that matches or does not match with your own experience and what’s worked for you.

With biology, I took a similar approach. My wife is doing a PhD in virology, and a lot of that kind of comes down to genetics. And I wanted to be able to talk to her about that kind of work. And that just have it be a tutoring session, which is like, oh, it isn’t Justin, this is how genetics 101. No, I want to actually be able to talk at an intelligent level in the conversation and understand her work on a higher level.

A lot of this was just, you know, it kind of came down to understanding, identified this model of, okay, I need to understand what exactly is a cell and what’s going on inside of it that allows it to function in the body. A lot of this, like you said, was just compressing. There’s just a lot of stuff, a lot of shit going on inside the cell, right?

It’s like, how do you, on all these things that the cell is interacting with other cells or manufacturing proteins or whatever, a lot of it comes down to being automatic, understanding this is about being automatic in the process of it. What’s doing what, who’s doing what, where.

And I kind of turned this into a problems style. Just minimum effective dose of this part of the process of the cell and then, okay, ask me questions about this. What happens if this happens in the ribosome while it’s producing a protein? What ends up happening? Does the protein still get produced or is there a failure? Things like that. A lot of these kind of, almost the equivalent of mathematical problems, questions.

And that ended up working pretty well and got, now it worked pretty well when I was doing it and then I kind of fell off the wagon and have not done my spaced repetition. So I’m very rusty on it now.

But at my peak, I was actually able to have a lot of conversations with my wife about how the genetic variants of whatever are causing things to happen, viruses infecting the cell in this way, launching this kind of attack. Oh, this screws up that mechanism inside the cell. That’s why it’s like that. That’s why this happens.

The one thing that I was not doing though, I’d be interested to hear what you think on this, is I was learning this from an all along and I was asking it to, it started initially trying to ask me, what are your interests? Tell me more about this and that. And I was just like, my interest is in learning this stuff as fast and efficiently as possible. I don’t care.

It was like, because it was trying to, I knew what it was doing. This is what you kind of default to sometimes as a tutor. Oh, what are you interested in? Why don’t we make this relevant? And I’m like, I already have it relevant. I want to be able to talk to my wife about this stuff at a high level. Let’s go. Just tell me stuff.

I want this to just burn it into my brain. I’m not going to become a biologist. I don’t need to know all the little things of, I don’t want to explore the space. Just tell me how it goes, what works. And I want to capture this automaticity as quickly as possible.

And so I was periodically having to tell it, don’t make, I don’t want to hear, and don’t ask me, and don’t offer analogies unless I specifically ask for it. Because it’s kind of slowing down the experience.

Justin Skycak: Through information at a lower level of scale. But it kind of broke your abstraction almost.

Justin Skycak: Definitely. It makes, one of the nice things about developing math curriculum and mostly carries over to computer science and physics is that things don’t really change. In math, it’s just evergreen. There is new stuff that you find that builds on top in directions I hadn’t gone before, but on the foundations, generally it’s locked in place. But I guess, right, as you kind of go out into the fuzzier subjects, there’s these complete overhauls of mental models of what’s happening.

And I remember I was

Justin Skycak: You know, one of my favorite, I think one of the most quoteable figures in the study of talent development, John Wooden, legendary basketball coach. He has so many quotes about sports that apply perfectly to learning in general.

And just any sort of skills-based learning, at least mathematics, physics, coding, where you actually, you know, you’re having to perform manipulations on whether it’s objects of information or physical objects or whatever. And you’re performing these actions, and you’re having to build up to more and more and more complex actions. This all kind of follows similar suit.

Words like, you know, all these cognitive science principles of mastery learning, get your prerequisites actions in place before you do. Well, a lot of these are kind of, they’re actually more obvious in sports, I think. I often like to go to figure skating for sports analogies, because it’s just so dependent on underlying skills.

Before we get you doing skating backwards, let’s have you skating forwards. Before we get you skating on one leg, let’s get you standing on one leg. That’s mastery learning. It’s also spaced repetition. Just because you were able to stand on one leg last week does not mean that you still know how to do it. We have to continue practicing this.

Things like layering more advanced skills to lock in the fundamental skills. It’s like, well, you’re good at standing on one leg, bouncing on one leg on your skate. Your ability to have that has saturated because you’ve been practicing in an easy environment. But once you can, if somebody tries to push you a little bit, or you’re trying to spin around on one leg, this is going to force you to develop robustness in the underlying skills.

Same thing happens in math. In calculus, there’s a joke that calculus is where students actually learn algebra because they are held to account for applying all the algebraic techniques.

And there’s interleaving. Instead of, you know, if you’re going to do a skating practice, let’s not just practice stopping with your right leg the whole time. Let’s also practice stopping with your left leg. And not just do stopping, let’s do a whole bunch of other techniques.

There’s a kind of retrieval practice, even in sports, I think. It’s like, well, you think of it in learning, it’s like on a test or quiz, you’re retrieving something from memory with no reference. You’re not getting a warm start to it by solving some easier problems leading up to it. You’re just right on the dot, under time constraints, you got to pull it out of your brain.

Well, in sports, similar thing. You got to develop a level of, you know, maybe you can do a spin on your figure skate if you practice a few times beforehand, and then you can do the spin as you get your balance. But really, you want to get to the point where you don’t need that practice leading up, stuff like that. I think this all carries over really, really well.

Justin Skycak: Exactly. If you’re playing hockey and basketball, basketball veteran, now it’s because if you’re having to think about how to dribble or how to AOB spin, you can’t, you’re not going to be seeing all your teammates around.

Justin Skycak: Absolutely. I picked that. We touched on this last time, talking about how in college, you can get a professor who’s just widely recognized in their field and you get really excited and you’re like, oh my God, I’m going to learn so much. And then they come and they’re just like proof there and proof on the chalkboard. You don’t even know what they look like because they don’t learn to face it. It’s on the board.

And they don’t really grade the quizzes or there are no quizzes. Homework sets are graded. Did you do the homework? Okay, good. Good enough. Let me get back to my research. Don’t bother me, kid.

That could totally, I think it totally holds. And I think a lot of times the people who, I mean, we’re talking kind of about in generalizations here, but often the people who rise to the top of the discipline, they’ve often gotten a lot of stuff for free. A lot of cognitive machinery or just life situation or whatever. There are things that they often did not have to figure out a solution for.

You can think of a mathematician who just, you know, I never really had challenges writing my work down. It always came naturally. What do you mean, you struggled to write your work down? You just do it. Or stuff like, you know, I never, these concepts, what do you mean, you can’t just read a textbook and then it clicks for you? And then you do the probably just read the chapter and do the problems. But what do you mean the problem is hard?

A lot of these people can take bigger leaps in generalization of things like that. Some got a GPU in their head, right? Instead of a CPU, they’re just running on some different machinery. Sometimes not necessarily different machinery, but just more optimized in different ways. Individual variation and working memory capacity and your attention rates, this. Unfortunately, this is a reality of learning that people are kind of built with different specs like this.

And a lot of learning how to teach effectively comes from, I think, having to really push the limits of your own specs and make you kind of punch above your weight in terms of where you would otherwise be. Anyway, I would totally agree.

Justin Skycak: Well, I actually have two kind of funny, concrete stories of watching this play out. One is very brief, and it’s just, you know, I saw a differential equations class, intro differential equations class, being like, the guy wanted to teach the differential equation, introduction to it, is like, well, the solution to a differential equation is just the kernel of the linear operator. That’s all it is.

This is like, the students have not done differential equations before. They have not really gone over kernels and linear operators aside from maybe a little bit in linear algebra. Just trying to jump to the highest level of abstraction. And the students are all just like, what?

And I was actually tutoring a bit for this class. And the circuit was so out of touch with thinking, oh, the students are working on their proofs. They’re learning a lot. Meanwhile, everybody’s just like, what is this stuff? And I’m trying to fill in all the gaps. And it’s just terrible.

Another instance is, you know, I actually worked with my sister-in-law a couple of years ago, and she’s taking real analysis. And real analysis is the stereotypical math major record course, right? Because that’s where it transitions from a lot of, you’re used to, you know, linear algebra, multivariable, tau, differential equations, it’s kind of concrete and maybe taking methods of proof course, kind of dip your toe in the water of abstract proof writing.

But real analysis comes and it’s like, if you’re not 100% solid on this proof writing, you’re going to struggle. And the instructor would basically, he would serve up maybe five homework problems every week. And those five homework problems were at the highest level of abstraction.

It’s like, we’re not going to skip over proving that a set is closed or open or stuff. We’re going to jump right into theorems of, prove this closure property of the n-dimensional sphere under these conditions. And man, can we just work in one dimension first with some actual numbers, not stuff like that?

And so, Chris, the homework would not even get really graded. It’s kind of a for-completion thing. If you attempt it, it’ll look like you attempted it, then you have points. And then there were no quizzes, there were no assessments, except for a midterm and the fun is this. The midterm was given in the middle of the year, and it was handed back, I think, a week before the final. So there’s no signal.

And I’m told that during class, the way he would run it is he was like, write at the room up on the board, write down the roof, okay, you guys have any questions? He would mistake the student’s silence. He would say, oh, great. Okay, this is easy. You guys already know this. Okay, let’s move on. Meanwhile, everybody’s like, we don’t even know what questions that. And otherwise he was a nice guy, like you have said. They’re nice. Very nice.

Justin Skycak: Know who you’re teaching it to. Absolutely. Another thing that strikes me about how you’re sitting in the front row, asking a bunch of questions. In addition to kind of identifying these missing prerequisites for, you know, a lot of these prerequisites shouldn’t even be prerequisites, like a notation change or whatever. It’s like, why would you do that?

But I’m sure there’s other parts that are legitimate missing prerequisites that are just kind of skimmed over, forcing you to backbuild. I think another thing that probably made this approach work really well is that you’re effectively simulating minimum effective doses of instruction and active, you know, you’re generating what you’re thinking about it, kind of answer it, trying to get your own question answered.

It’s like, right. I’m checking the class. I pay for this. I pay for this. I’m here.

Justin Skycak: Get the knowledge out of them, forcefully almost. This is a broken model of reality, how learned. You don’t teach a student by talking at them for an hour. That’s going to overload any, unless the student already knows the material already, they’re going to be overloaded with their working memory, their cognitive load.

You got to be breaking this stuff up, I mean, filling in the productions, but also having students, why would you talk at a student for even 10 minutes or more before having them do an exercise? And some classes do a kind of very, try to move in this direction by doing clicker exercises and stuff, but that’s only baby steps in this direction to what it needs to be.

Justin Skycak: Here’s, I think this, I think a lot of people who would clash with this, or if there’s a line drawn in the sand on this, there’s a battle between two sides, I think the point of contention really boils down to what does it mean to understand something?

Okay, there’s a kind of conceptual, you know, you watch a video on, say, neural networks, right? You don’t know really how they, you’ve heard neural networks kind of underpin a lot of modern AI, and you’re interested in it, and so you watch a video on YouTube, and you learn about back propagation, and you learn about regularization, and some of these things, and training tests, whatever.

And so you get a little familiar with this stuff. Do you understand it? You definitely don’t have automaticity. You haven’t worked any problems in it. You haven’t set up anything on your own. Do you understand it? Well, there’s some kind of, I guess, you’re familiar with some of the terms.

I personally, whenever I talk about understanding, I mean understanding to the level of conceptual, understanding to the highest degree, conceptualizing. There’s a difference between how maybe a basketball player conceptualizes strategy versus somebody who just watches basketball games.

And because the basketball player knows at the fundamental level these strategies that work or don’t work because of these movements that happen. When they’re thinking about these strategies and stuff, the parts of their brain that are actually activating movements and things. It’s almost like simulating a simulation in their head of all the underlying things versus somebody who’s just seen this at a high level.

And so I think, I mean, you can acquire maybe some familiarity with just the, okay, backdrop is something that involves hardcore multivariable calculus and you somehow figure out how to adjust the weights of the network. I mean, if you call that conceptual understanding, then I guess you can get that without having automaticity on the concrete computations.

But if you talk about backdrop in terms of, how do I optimize? Say this issue is happening in the neural network, this is what the loss curve looks like and this is what layer three reads as, what’s wrong with it. I would say that’s a level of conceptual understanding that you can’t get unless you have actually worked through the computations.

Justin Skycak: I’ve got a concrete example for this again in the machine learning setting. It comes from when I was teaching this really advanced applied math and computer science elective sequence within Math Academy’s original Pasadena program.

We got students from the point of, you know, they came in having their core engineering math in place. They learned linear algebra, multivariable calc, and everything. And we scaffold them all the way up through coding exercises, building machine learning models from scratch, no libraries. It’s like re-implementing papers in 90s artificial intelligence.

And one of the things that I remember just really stood out to me as validating this kind of, you have to learn the movements in order to really understand what’s going on, is when we were talking about exploding gradients and vanishing gradients in neural networks.

I remember having talked about this previously with some people who are interested in machine learning but didn’t really let the math actually go through the computations. And it was always a kind of fuzzy area. It’s like, what do you mean? Why do they explode or vanish? Okay, I guess the activation function, the derivative of the activation function, if it is that the slope is too high or it’s too low or is it if it’s unbounded, they just kind of, something about the activation function, then it’s bad and then that stuff happens. Alright, I understand it, let’s move on. Let’s move on to some cool stuff.

But with these kids, I made them work out some simplified scenarios, maybe a two, three layer neural net of two nodes in each layer with some nice numbers. The computation took maybe three, four minutes or whatever, but they could see very clearly the effect of different activation functions.

And so I would like, okay, what if we use, let’s just say we use an exponential function as the exit or just even a linear function with, you know, quadratic or something where the derivative is continually increasing. And then they would see in their computations, wait, when I chain rule it and I take the derivative of this thing and then I multiply it, it just keeps getting bigger and bigger.

And then I’m like, what do you mean? Why did you give me a bad activation function? Why did you do that? And then I was like, ah, alright, you see the problem. Why don’t I give you an activation problem that doesn’t have that property?

And then it’ll just go flat and they’re like, okay, great, thanks. And then when they do that, they work computations. They’re, wait, wait a second. They just get one round in, and they already anticipate how it’s going to play out. They’re like, wait, this thing is already basically the derivative is saturating. It’s going to zero and all the terms are going to go to zero. I don’t even have to work out the rest. I already know how it’s going to play out because you just chain rule, multiply, chain rule, multiply, goes to zero, goes to zero.

And then at that point, I guess they get it. They get the exploding and the vanishing.

Justin Skycak: I think the experience can vary a lot depending on whether you’re kind of skilling up on known knowledge that’s just, this is figured out, just learning, versus kind of at the edge of the frontier. I think it’s looking really fuzzy and what’s even true and stuff like that.

I think what the confusion represents, that moment where it clicks in place and you just have that insight, you know, oh my goodness, this is it. And then it clicks in place.

I think when you are going through the knowledge graph of just stuff that is known as the foundational material, I mean, I think what that amounts to is really you are missing some prerequisites and you build up. Maybe there’s this topic sits a bunch, or sits atop maybe five different prerequisites, or maybe say 50, 100 ancestor topics, prerequisites and stuff.

And there’s one learning path in there that you haven’t really, you know, it’s just missing. You haven’t filled out. See, you got most of it filled out, but there’s one pillar that’s kind of incomplete and you’re banging your head on the stop. What is wrong? And then at some point, whether you kind of stumble into it or somebody directs you to it or whatever, you kind of realize, oh my goodness, wait, that’s the, oh, this works that way. And that’s how it works. And then it kind of clicks more into place. You kind of fill in that missing pillar.

And it feels amazing, right? It’s the click, it’s the aha moment. But when you’re in that foundational body of knowledge, it’s like, well, there’s a lot of time that you had kind of wasted being confused. If you just had that pillar in place beforehand, you wouldn’t have had to sit there for hours or days just ruminating over this thing. And then you would have the pillar, you wouldn’t have experienced a click and aha moment because it would have just been like, it was a deal. We just put these prerequisites together. What’s hard, man? That’d be it.

And I think when you’re in the foundational body of knowledge, these aha moments are actually something you want to try to avoid. I mean, if you’re in a well-structured print, you won’t actually have, I know it sounds a little counterintuitive.

But I remember somebody, we actually had at Math Academy, we were talking to, this was years ago, there was somebody we were talking to about recommendations for growth on social media and stuff. And they had recommended something like, well, they had to give it a nice name to these aha moments or snap moments, and talk about these things. And then I was just like, wait, that’s the opposite. We tried to smooth it out so you don’t have that, like, you’re climbing out of a hole and you’re like, I made it. I made it. No, it just makes it smooth.

Now, eventually you run out of those knowledge that’s known, right? And you get to the end of the frontier where things are fuzzy. And that’s kind of, I think, this is more what you’re talking about. It’s like you’re kind of in a setting where this has not been fully mapped out.

You’re going to run into situations where you just have missing prerequisites, because everybody has missing prerequisites because nobody knows what are the prerequisites. That’s kind of what we’re trying to figure out. We’re trying to map out how to extend this body of knowledge further.

And part of this mapping it out means identifying what reframes are prerequisites are necessary. And I think as a researcher, that sounds like it would be a very valuable moment where you’re kind of sitting in confusion. Why am I confused? It’s because there’s some missing prerequisite. Holy crap. If I figure out what this is, resolve this, everybody around is confused too. If I can figure this out, that’s going to be a high-value contribution.

That’s my take.

Justin Skycak: It’s like, generally doing hard things is good, right? It’s good to push yourself to do hard things, but also when things don’t need to be hard, why make it harder? You’ll get to the hard things that really matters to spend your time, spend your confusion on things that, when resolving, it’s like, spend your confusion on things that have high ROI and that confusion.

If you’re confused, you get an aha moment, a click moment, how about make it something that actually is a contribution to a field instead of just like, oh, this is a result in this math course that I skipped. Don’t spend, okay.

Justin Skycak: We actually came up with the solution to this problem at Math Academy, because we were getting a lot of adult learners asking this class, basically immediately as we kind of emerged on the scene on X, Twitter.

A lot of adult learners are all asking the same question. It’s like, what do I do? Do I need to kind of start up? I’m not even at calculus. I took pre-calculus in high school and I forgot all of it. Should I just start with fractions, fourth grade math? What do I do?

And then our answer to that was basically, why don’t we design a sequence that is the, you know, we’ll start it with fractions. You can go down as far low as you need. You can place into the sequence. You can place out of the material already know.

But it’s going to strip away all the stuff that is there for, you know, in schools, school, Common Core standards, but does not actually really get leveraged in university math.

Just to name an example, in geometry, there’s this inscribed circle theorems, where you’re like, okay, I got a circle. I got an angle in the circle. And what’s the, if the angle, the vertex lies on the circumference of the circle. What’s the measurement? If you know the arc intercepted by this thing. It’s one of those things. It’s like, okay, maybe it’ll come up a little bit if you do the physics of lenses and stuff, but aside from that, you’re just never going to see this kind of. Not relevant to machine learning, not relevant to most physics, not relevant to basically anything else.

And there’s a ton of these topics that just, they’re not very foundational for high school math. I think it was maybe a quarter or a third of the traditional school sequence. You can just kind of strip away and it’s really not a big deal.

But you get your foundations, the really important stuff that underpins a lot of the serious math and physics and computers and machine learning, whatever people want to do.

And we made this three-course sequence. We call it the Mathematical Foundation sequence. And our director of curriculum, Alex Smith, was really the one who put all this together and made it happen.

But the sequence is all together, it’s this three-course sequence. You take the three courses and you emerge on the other side ready for university math, like linear algebra and multivariable calculus, or a composite course like Math for Machine Learning that stitches stuff together.

And to go from this math foundation sequence, the very bottom to the very top. Assuming that you really forgot how to add fractions, you are starting from almost zero. To fill it out all the way takes about 15,000 minutes.

Now it might seem like a lot, but you actually do it. How long is that actually for days? Well, if you just do every weekday for an hour, for a year, you can fill in all your foundations, if you’re starting from the very bottom.

And now if you decide to do, I’m going to do just two hours per weekday. Guess what? Only half a year. Three hours per weekday. Maybe you want to split that up into a morning session and evening session or a couple of times a day. But if you get really, really serious, guess what? Just a few months.

And that’s assuming that you’re starting from the very bottom. Most people don’t start just chromatic fractions. They remember some algebra at least, how to solve a linear equation, stuff like that. Really, if you’re serious about this, you can get it done in several months.

The path is there and we’ve had people do this and come out the other side. Actually, the person I referred to earlier who had signed up that got me course because he wanted to read this robotics book and do robotics. And he just had no, way above him. He went through that sequence and he came up to the other end and it was, it’s just world changing. Well, this is good.

Justin Skycak: Definitely ties into what we were talking about earlier and the end of the pipeline where I mentioned that we were talking about having projects as just really cool projects as this anchoring goal for learners.

Where, you know, you can look and do, wait, I just take these math courses and then I get to training a convolutional neural network to classify digits, really? Or even cooler applications, like I use a neural net to approximate some lengthy physics computation and run it really fast, or the clustering of language symbols and then a long loss of whatever. We got a ton of different projects like that coming out in our upcoming machine learning course.

And we want to kind of extend this to the whole curriculum, not just machine learning, but actually do it in just your routine calculus course. Why not have calculus with a bunch of really cool projects that involve, you know, be integrated with a little physics and have one of the projects be figuring out the viability of some situation for setting up a colony on Mars or something, or figuring out what type of fuel is best for a boosted race car or figuring out what ingredients are going to work or not going to work for a certain recipe given chemical property, whatever, stuff like that.

A goal is that if we can cover the space with enough really, really cool stuff and then allow people to anchor on particular projects that they want to do. And hopefully not just the highest level projects, but also intermediate projects leading up to there. Have some projects in algebra, have some projects in calculus, some projects in linear algebra, multivariable calculus, math, or machine learning, all this sort of stuff. There’s kind of a real application that seems really, really interesting.

Justin Skycak: This is the only thing. The only thing. There was one paper that we actually are, is going to form the basis of one of the machine learning projects, which, I remember, I mentioned that I had, at least high school, when I was teaching this advanced machine learning crack within Math Academy’s original school program, the kids reproduce papers and there was one paper back in the nineties artificial intelligence.

Like you say, it’s a cool result. And it’s surprisingly, you know, it’s just evolving neural networks to play games, like tic-tac-toe or checkers or whatever. And you actually don’t even need to do back propagation because it’s an evolutionary procedure. And if you can just forward propagate, I mean, that’s really just the hard part is not that much math involved. And then you can do it.

I think having paper reproduction is going to be a really cool thing as well. And I’m sure there’s tons, tons of those papers.

Justin Skycak: I guess one of those things where, as you’re kind of going, we’ve made this distinction between when you’re in the knowledge graph, the known knowledge graph, versus when you’re building on top of the knowledge, right? You’re at the frontier of things.

And I think, just from a pure efficiency standpoint, when you’re going through the known knowledge graph, any sort of creative thing that you will do, unfortunately, it’s not actually that creative because it can be rendered trivial by a lot of results. You could study the statistical properties of random walks as a solo creative project or stuff, but I guess that’s already figured out. Whatever result you’re going to have on that is probably known already.

Now not to say that there is zero value to this because there is motivational value to it, right? If the choices are, either I’m going to lock in, and if you’re going to draw a dichotomy for yourself, it’s like, okay, either go monk mode for all my studies until I reach the frontier and only then will I allow myself any sort of creative exploration. That’s just mechanically going to be the most efficient way to get the frontier.

But if that’s going to force you off the rails or you’re just like, man, I quit. I’m done. Well, okay, just do what you need to do to stay on the rails. And if that involves going down some rabbit hole or whatever, just have some kind of fun with it, experience some cases of the rewards that you want to get once you get to the frontier of it, then okay, do that.

Do what you need to do to make it fun. And everyone’s going to be different in this capacity and how much of that they feel like they want to meet or what type of rabbit holes they want to go down or whatever. But I think the real point where this exploration and everything becomes very valuable, a high ROI sort of thing, is really once you reach the frontier.

Justin Skycak: This approach of doing a, go to some kind of interesting project at the frontier and do what you can and then backfill to try to unlock more creativity or more ideas or whatever to do with this thing to understand it better.

I totally see how this can work for people who have some amount, a large chunk of foundational knowledge already in place. And I think it kind of depends on the distance between what they’re trying to do and what they’re doing.

Because this may seem like a caricature, but this happens a lot where there’s somebody who doesn’t even know how to solve a linear equation. And they want to do like, wow, this AI stuff is cool. How do I do something? And it’s like, well, I mean, I guess you can download and run a transformer model or something.

But if the gap is that big, when it comes into backfilling prerequisites, you’re going to be like, well, okay, I need to learn back propagation. What is that? That’s squiggly simple. Oh, is that calculus? It’s like, if the gap to fill in is not very, very big, then you can be efficient in terms of backfill. Oh, I just need to learn, learn, learn. Okay, new unlock.

But the gap is gigantic, where you’re essentially missing all of algebra, all of calculus, then I think it gets to a point where it can be inefficient to try to trace your way all the way back down to fractions or algebra.

Justin Skycak: I’m like, oh my god, I’m going to level. And it’s like, well, guess what? You’re going to need to learn this stuff anyway. It’s not like there’s three secret topics in all of 300 algebra topics that you need to learn to unlock machine learning. It’s like, no, you got to learn basically all of the algebra. It’s foundational.

And I think for these foundational, when a body is foundational like that, if you try to just go and back, you’re going to be ultimately just kind of trying to construct your own algebra graph. And this is a known thing. Just take the algebra. Take an algebra course, and you can get through it a lot faster.

But I agree, as you reach closer to, okay, you’re kind of within striking distance of this thing, you can kind of see the full path between the project that you’re doing and the math that you need, the backfilling strategy becomes less

Justin Skycak: That’s, I think that’s right. There’s also the variable, how much does person understand how to learn effectively? That plays the same role as a gap in knowledge.

Justin Skycak: I have a very concrete example. But before I say the example, I will say there have been a number of adult students who I’ve seen go from, even in their thirties, just go from, you know, I don’t really, I want to get involved in some kind of cutting edge thing, or maybe not even cutting edge. I just want to become a software engineer.

And they start this in the thirties or even their forties, or I’m sure there’s people who do it even later in life. If you’re willing to put in the work, it’s the same amount of work. The amount of work does not change whether you start this at the age of 15 and emerge where you want to be at the age of 20 or 25 or whatever. It’s the same amount of work if you started at 25 or if you started at 35 or whatever.

And some of the things that makes it more challenging for, well, I think sometimes people think, oh, it’s too late for me to do this kind of thing. And I think that tends to be more of just like, I’m not willing to put in the work given that I have less free time.

You know, you go through life, you got a family, you got kids, you got more responsibility. But it’s just harder to make time to do things aside from it. Especially, you need to have a job too, a separate job. It’s like, where’s the time of the day to learn? When you were a teenager, you’re just like, you just come from school. What are you doing? Play video games or go to the basketball court with friends. You got all the time in the world to do this stuff.

But the amount of work does not change, even if your time to do it. But the time that you devote to doing the work is within your control. You can choose to be more efficient in various areas of your life to open up time. And it’s not going to be easy. And there’s always going to be lots of constraints and stuff. And probably not going to open up as much time as you did when you were younger. But the takeaway, if you put in the work, you will get to the result. If you find some way to put in that work.

And I’ve seen this happen in particular. Back when I was working in data science, I started out as an intern for six months before converting to full time. And there was another guy who was actually interning with me. And he was, I think he was actually in his thirties. Yes, somewhere in his thirties, or maybe 30 years old at the time.

His story was kind of that in high school, he wasn’t really paying attention to what he wanted to really get out of career-wise. He was more concerned about, I just want to have kids, have a family. And that was his main goal. And that’s a fine main goal. And he did that. He accomplished that. Had kids, had a family, and he was working as a receiver at an emergency center, if you 911 who answers the phone.

And he wanted to get into a kind of career that would offer more money and better balance and stuff and just to get more to his family and stuff like that. So he was like, I want to go into software engineering, want to get into data science. And this was, you know, he was in his late 20s when he started even thinking about this. And he didn’t know any math, no coding background, none of the stuff.

And so he just signed up at his community college. And in his late 20s, one of the oldest students there. And he’s like, whatever, this is, I got to learn the stuff. So I’m going to learn the stuff.

And at the same time, he’s got his job that he’s holding down as well. He’s got his family and stuff. This is not an easy sort of, he’s pressed for time. But he’s making it happen. And so he goes there, he gets his computer science degree from community college. And then gets an internship at this company. And so he and I interned around the same time. And he also converted to full time as well.

And he took it very seriously and really got good at the kinds of software engineering that are doing the data flows and stuff. He got the nickname the Lord of the Data Flows over there because he was so into it.

I remember talking to him. He’s like, man, I really need this to work. There’s like, he was at a point where there was no, what’s the alternative plan? He wasn’t thinking about alternative plans. He’s like, no, this is what I’m going to make happen. There’s no hedging of, oh, this doesn’t work out, I guess I’ll do this or whatever. I’m sure there were, he planned these, but he’s not thinking about, focusing on the thing in front of him. Making it happen.

And one thing that I think is true is, initially, the heart of this journey, especially if you’re embarking on it later in life, the hardest part is kind of the upskilling phase when you’re trying to prepare yourself to get paid to do something that’s more aligned with what you want. Because at that moment, you’re not good enough to get paid for it yet. So probably you have to be doing a different thing to get paid. And that’s going to eat up a lot of your time.

You have to spend time doing this other thing and spend time preparing for this new thing. It’s like, you’re just pressed for time. Once you can make that transition to actually earn a living from the thing that’s in the direction that you want to do, it’s like a total phase change because now you have maybe eight hours of your life every week that you’re actually doing the thing that’s closer to what you want to do, if you’re improving in that direction.

And it’s like, you get a rock after this. And if you can just make it to that point, then

Justin Skycak: It’s kind of like, you let life ossify around you. You let the concrete harden. You got to break the concrete. Yes. But once the concrete is, okay, it’s the prize, you have more concrete.

There was a student that we had, well, that we had, started last year. He was talking about how he posted some video talking about the experience of just getting a habit, sitting, doing that. And he was talking about how when he started, he just was not really into it. Math is, I think he wants to go into software engineering or data science. Something in that direction, in the tech direction.

He was doing a degree in computer science. And there was a bunch of math courses that he was just like, I can’t do it. I’ve got it through all my dimensions. And he started on this and initially, he was saying, every day I wake up and my brain just tells me, don’t do it, man. Don’t do it. Don’t start working on the course. This is not going to be fun. You should run away from it.

But it’s every day. And he just, there was, at the beginning, there was just a lot of, shut up, brain, I’m doing this. We’re going to do it in the morning, where you’re really awake enough. I’m going to give you to, before you start yelling, I mean, I don’t like this. We’re just already going to be doing this.

And he said that this continued for weeks. I can’t remember. He said 30 or 60 days or something like that. And he was like, man, I just accepted that this feeling is probably just never going to go away. And then a week after he just accepted that, I’m just, this is what we’re doing. He said he started just not having all those bad thoughts.

But that, just way to, it’s like, you wake up and you’d be waiting for the brain to go, do it, man, don’t do it. Don’t you dare start us on math problems. And it’s like the brain, it just given in. It was just like, okay, I guess this is what we’re doing. Fine.

If you get yourself, you just go through the habit enough to get your brain to just accept that this is happening. This is the path of least resistance is really, you’re going to be

Justin Skycak: Not that, though? I’m a little bit familiar with it. I have the book on my reading list, ultra learning book. I’ve read through a blog for the big students journey through this. Rusty on all the details of it.

But this comes out very very quickly when I talk to people. I’m like, oh, this is very similar to the ultra learning thing. Oh, my God. That’s right. I need to leave this thing again.

Justin Skycak: When I was doing my own version of the ultra-learning marathon, 3000-hour study, very fun. I was actually not really going in my own interest areas. This was, I think, a difference between me and my program and the ultra-learning program, and I started out some life.

And I also did not have the very, my goal was actually not to go through and retire. It was not a specific project that I wanted to do. I started with realizing that, like I said, earth-shattering realization that the pace of school is common. And if I just take this into my own hands, then I can wait back.

That paired with the idea of all this, I’m sorry, initially it was like, I started out, I was like, I’m going to learn confidence. That was the mission goal. Once I got through that, and I was like, I’m just going to, all this math stuff, it seems like it’s going to be really useful for me in the future. And I’m enjoying it. It’s a lot of fun. I’m just improving on the stuff.

I didn’t have a free air. I was just like, okay, I just want to learn. This is a subject, this, this, this, because I can. And so it’s not like I was sitting there, like, okay, I learned calculus, I learned knowledge, but I want to go into different ways. I don’t really have that.

Oh, that’s cool. It’s like, what about the cross pollination between different equations and multivariable and the partials? I wasn’t going out in Wikipedia, going down the rabbit hole and stuff and that. It was a very structured kind of like, I’m gonna, just tell me what to do. Just find a resource that just tells me what to do and what to do and what’s the mission way possible.

And initially, I mean, I was coming into it, I wanted to do this, but I wasn’t always going on that. I was getting my head, my head ball was, I wanted to learn this stuff really, really, I want to get as far as I can.

Justin Skycak: One thing I want to say is that sometimes, if anyone has a particular goal, like machine learning, I want to reimplement this paper, whatever, I never want to say, don’t even try it. Go, try it, you know, give it a whirl. If you are close enough, within striking distance, that you can do it, then what might I say?

I’m just saying, if somebody is like, man, I tried it, but I can’t do this thing, since I don’t know what the hell do I do, it moved in my just to help to do it, and that’s what I’m like, okay, no, stop, stop. You’re doing it. Get your foundations in place. This is going to make all the difference.

Justin Skycak: That’s interesting. When I was doing the biology, I had kind of viewed it as the Math Academy approach to just what you do is, you literally go in for a certain term. And so I kind of do that as, okay, I just want to spin up on biology as fast as possible, which is a mastery of the subject.

But now that you bring it up, it’s true. I kind of scope it down in particular to genetics, genetics, that’s the lay area. And I think overall, there is a reasonable amount of scoping and top-down plan whenever you have it.

And so that’s something that we’ve actually created courses on math that specifically works like Math for Machine Learning. What we’ve got, that’s for physics. It’s like, if somebody wants to do machine, you’re going to have to learn linear algebra, all type of work out of this, probably the statistics, but you don’t need to learn all of these things. There’s a lot of multivariable calculus, like the use of the there, the average is there, it’s there. At the end, that’s not the type of, the most empirical, the CPA.

You really need up to the change of hyperplaning, stuff like that. Which is really, I mean, it’s a large, tropical, but this is not the whole thing. There’s a lot of reasonable scoping to have in any people. That’s kind of almost a hybrid, I guess. It sounds almost like a midway between higher subjects, mastery learning versus the only prerequisite chaining, ultra-learning, just kind of that. Scope it down, top down, and then fill it up.

Justin Skycak: The interesting thing is, a lot of these people with ADHD think that, you know, just a lot of the education system is not cut out, and the way I’d like, you know, that there’s not a hearing from, that’s not, and only one of those things is true. The education system, as it stands, is not threatening to people with attention challenges. People who can’t sit still, actually, I mean, it’s really not friendly to anyone, but especially with the potential for the challenges.

And it’s like, nobody likes being talked about in a subject, but a lot of, I’m kind of similar to you with that, I think. I can try it up, six hours of writing, I just don’t know how much I’ve been talking about. And it’s like, what if you can take somebody who has, you know, more, it is not like you and me as a opposite way. It, just the hint at the talk and the power of how it just kind of limits the amount of, you know, how much can you communicate with somebody before they start doing something?

And initially that might seem like something, some kind of disadvantage. Well, I just keep the system, the power it takes to go through the electronic, you know, quite really. If you look into, you start going down the rabbit hole, it’s not as a learning, it’s like, well, that’s not even a good thing to do in the first place, to be talked about for now.

It’s just people have, some other people have a higher talent, but it’s not good for any. What’s good is these bit of effective doses. Just like, okay, just tell me a little bit of what to do and then let me start doing it, and then active, active, active. Or you stay in a sort of passive state or just not doing anything. It’s like, you’re not, if you go to tennis, it wasn’t so much a question to talk about for you right now, or I don’t care if you have ADHD or not, you’re not getting better at tennis.

If you have ADHD, then you might just be expressing or to this company, come on, what’s going on? Just do something. But either way, the learning’s not happening.

I think a lot of times, the neurodivergence, it’s not just, it makes more sensitive to things that are, there’s an issue. If you’re able to, the issue more, or earlier on.

I mentioned, I have very similar to you in the, the passage of time is just, it’s almost hard to keep on attractive and just to sync with this company. Our founder of Math Academy, Jason Roberts, he is actually humbling the exact opposite of his ADHD perspective. So a lot of Math Academy is built in terms of what would Jason have won? What did Jason hate about the application growing up?

And those other things, like being talked at for a long time, or actually doing problems, hated him. He’s got these stories, I like about him, just being a descendant of the class, who never sits still. He’s just like, Jason, hate, mention, what are you doing? It’s like, well, the problem isn’t Jason. The problem is that he’s being talked at for too long without doing anything.

And so Jason actually has three kids who are all, one was just very highly computationally, mathematically gifted, and just blows through all this stuff. And he has another kid with a discount, he’ll be up, and then another kid who is just kind of using more, give you a typical honors math.

And so the kind of approach that we take is kind of making sure that this efficient learning cycle is working for all the students. And all these kids, he and I, you and I are so different too, and we just agree with all the same things. It’s just whether you come at it from, okay, kid with ADHD, it can’t, it doesn’t want to sit still for 15 minutes for an explanation, or you come at it from just calculate the learning efficiency of this setting, this is that set, you get to the same answer.

Justin Skycak: Absolutely. It’s just kind of, right, the feeling of, it’s like, the feeling is often the real kind of, the being done. Often the issue is not that you’re done, it’s just that it’s some other settings that are not, you know, causing the information to flow, whether that’s a prerequisite gap or just not wanting to participate in the process, or whether it’s you’re just feeling that this is not, you don’t identify with this stuff, this is not something good at. It’s all this kind of blockage that’s gonna be put up, and it’s separate from the actual, how good you

Justin Skycak: That’s a really good question. All right, I’m not a research mathematician, so I don’t want to overstep in my answer to this, making claim that I don’t actually know what I’m talking about. I think this question would be best posed to somebody who has a track record of mathematical publications.

Well, let me generalize the scope here a little bit about amateurs contributing to the edge of the ills in general. One thing, at least in my experience, and that I hear from a lot of the people who confirmed, is that this tooling that exists nowadays, LLM stuff, OpenAI, or whatever, all these kinds of tools, they’re kind of multipliers on what you’re able to do in initiative.

And now, if you’re not able to do a whole bunch, you don’t have technical chops, and then you ask some stuff, ask questions, the LLMs have to code up stuff for you that you couldn’t otherwise do. This can feel sort of like equalizing, like, oh, hey, I’m on the same caliber as everybody. I don’t have to learn how to code. I can alternative vibe to everything. I don’t have to learn how to go to goal, hack into science and stuff. I can just tell the LLM to generate course stuff like that.

It’s like, well, okay, that is, I guess, level you. But now think about how much it’s leveling up people who actually know these foundations of what’s going on. And the thing is, when you have a high level of mastery in something, at least in my experience, these tools can scale you. You are the zero to one, and then these tools put the zeros behind them, like 10, 100, 2000.

And if you don’t really know what you’re doing, you start being

Justin Skycak: The other, the edge, absolutely. A lot of the efficiency gains that I’ve seen personally, you see, these tools have been, it’s more about delegating work that I already know how to do, because that I can automate more and just free me up to do additional stuff.

But it’s not a way to get needed to awful at work that I don’t know how to do because it has the same failure modes. I’ve done this before, I’ve made this mistake before, trying to, I have a junior employee, human employee, and you got a project that come out of this, I was pretty cool. You haven’t really smoked it, you haven’t gotten a piece of it or anything, and it’s like, this sounds like a good thing for you to do.

And then you look up a day later, a week later, just, wait, what the hell was happening? This is not the whole. And then you have,

Justin Skycak: Here’s our options here, what do you think? And then you can, because evaluate on a totally little delivered.

One of the big unlocks, I’d say, that I’ve experienced recently, and this was the agentic coding stuff, was kind of just scoping stuff down enough that I can get it into, that it fits into my own working memory of what we’re doing here. That was one of the challenges that I experienced before.

It had to have this whole plan for, okay, here’s the feature that we’re gonna build, here’s the thing that we’re gonna do. You have to get to do too much, and then it’s gonna go off in directions that you don’t anticipate, because you have not verified everything that’s gonna be doing, because you can’t keep it in your working memory.

But if you split this up into, if you are able to sign up, get the expert feedback, and I’m gonna do it. Okay. This all looks good. I seriously looked at every little thing, it looks good. You can keep the reins tight like that, then it’s amazing.

Justin Skycak: You know, there’s one thing that I think about as soon as I wake up, late into the night, all the time. It’s what I’m working on, and I think I mentioned at the very beginning, when he asked, what am I working on right now, it’s taking Math Academy from workshop to a factory transition.

It’s a huge team-lift. Me, Jason, Alex, Dr. Coagulant, just figuring out how to take the, it’s like, it interfaces with learning with all the AI, everything, how to deliver this learning experience, and in particular, this content, a whole courses at a much higher level of scale. How to turn out more courses, factory style, but not just like, hey, I’ll build a course of work, and then, then it’s like, how do you make the experts, take the human expertise and then apply these tools as multipliers to just speed up that level of expertise without a traffic standards.

And we’ve made so much progress on this in the past few months, and I’m really excited about this. But this is really our kind of, you know, the Tesla, Elon sleeping on the factory floor, kind of stuff. But it’s just production hell of, oh my god, this is, we got ourselves into this game, and now we have to scale on and really deliver.

And just, it’s gonna suck for the next however long it takes to make this factory work. That’s kind of where we’re kind of doing the people. That’s where we are. That’s what I’m doing at every waking hour, basically.

Justin Skycak: Well, fantastic. Thank you, Assistant. Thank you for having me for two days. This was great. This was a lot of fun talking to you. And, you know, like I said earlier, I came in with reflections today, and that only happens when you have a really good conversation with somebody. It’s the testament to you as an interviewer, man.

This is too much effort into this. Kind of have great questions, and also answer, including so many people’s questions that he said to you. I never imagined that we would be addressing this many people’s questions that are possible, that we were able to.

Prompt

The following prompt was used to generate this transcript.

You are a grammar cleaner. All you do is clean grammar, remove single filler words such as “yeah” and “like” and “so”, remove any phrases that are repeated consecutively verbatim, and make short paragraphs separated by empty lines. Do not change any word choice, or leave any information out. Do not summarize or change phrasing. Please clean the attached text. It should be almost exactly verbatim. Keep all the original phrasing. Do not censor.

I manually ran this on each segment of a couple thousand characters of text from the original transcript.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.