Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

PODCAST 7

Improving the SAT Fundamentals course with missing middle topics

Jason (00:00) So the original version of the SAT fundamentals just didn’t cover everything. I mean, what it covered, was everything that was in the standard curriculum

that you’d need to know do well in the SAT. But it didn’t cover everything you need to do well in the SAT. There’s a sort of exists.

And it would probably be enough in the course to get to a certain level score, get in the 700s to an 800, need to know more.

After I get the students’ foundations up to speed, there’s this whole series of skills they need to have to really do well. What are these golden topics?

Justin Skycak (00:37) Stuff that’s

of the standard curriculum, stuff that you’re not going to learn in your school class, this privileged knowledge that these experienced test prep tutors have that really it’s basically just the families who can pay an arm and a leg for test prep get this sort of information.

Integrating SAT topics throughout the math curriculum to spread out the work

Alex Smith (00:00) We added like 120 or so missing middle topics to the SAT fundamentals course, but one of the plans for 2026 is to integrate that into the standard high school curriculum. a student is doing our standard

complete it up to a certain point maybe say algebra 2 or integrated math 2, they’re ready for the SAT. They haven’t got to do some additional SAT fundamentals course in order to kind of get access to those extra stuff. They’re going to be hard baked into our mainstream curriculum.

Justin Skycak (00:31) If you just go through the standard curriculum and you’re not doing these missing middle topics, then by the time the SAT comes around, you’re like, ⁓ gosh, I have 150, topics that are like just off of the standard curriculum.

And yeah, so if you’re like, oh, I’m taking the SAT in two months. Maybe I should prep for it. you got like 200 topics to learn. This is not a good situation. Even if you did go really crazy and you’re like, okay, I’m doing 10 topics a this week.

you’re not going to have time to really consolidate it. And you’re just going to be doing so much that it’s really difficult. It’s like going from I barely know how to cartwheel to I can do a backflip in like two months. Like that’s not going to happen. You got to space that out over more time.

Free response questions don't prime you, making you think harder

Jason (00:00) The reason we do free response is because you’re not primed with potential answers, right? Sometimes you could ask a question and you see some potential choices. You’re like, ⁓ right. It’s this kind of an answer. But when it’s just, there’s nothing to prime you there. All you have is an empty box. like,

Is this a number or expression I’m supposed to enter? You have to think a little more about it.

Alex Smith (00:17) Well.

Justin Skycak (00:21) So just a concrete example of say the question is, find the area of the circle. If you see π in a lot of the answer choices, that kind of tips you off as like, ⁓ right, the result has π, π r squared. Like it kind of helps prime you into it. But if it’s just a free response box, you have to actually remember that this has something to do with π. You have to just pull that from scratch out of your head. There’s nothing, there’s no priming.

Alex Smith (00:45) And one thing as well, especially younger children, they will resort to like non-optimal strategies learn the content. So for example,

trial and error. If it’s like, what’s the solution to this equation, for example, it’s like a really simple linear equation, it’s very easy to take every single answer, just plug it in. it’s that one that works. That’s the correct answer. But of course, that’s not the strategy you want them to be doing. You want them to be solving the and getting the correct answer that Now, more disciplined students would do that anyway, but we want to discourage that kind of behavior as much as possible.

Optimizing learning using new question types

Alex Smith (00:00) Now obviously when Math Academy first started, multiple choice was all we had. So we had to make the best the tools we had. But one of the things I want to do is kind of go back to some topics where we’ve used multiple choice and ask ourselves,

should we remove multiple choice entirely from those topics? Because there should be no priming, no guessing by looking at the unit or whatever.

doing a first pass of the free response stuff and then we’re to kind of circle back and then do like a really hard analysis of, choice really bad thing to do? We’ve got these other options now. We’ve got free response. We’ve got like static select and dynamic select questions, like drop downs. At what point, what knowledge point should we be removing those multiple choice questions from where.

A variety of question types is good

Justin Skycak (00:00) Free response can be really helpful for a lot of these topics. But all topics need free response. We still have multiple choice questions in a lot of topics because they are use quite often. For example, say you go through a really long computation,

and the numbers don’t really prime you to what the result is. And there’s no way to like, guess and check based on the answer choices. You don’t need to necessarily have free response question there. And in fact, a free response question can be less forgiving to small errors. Like,

you know what you’re doing, but you added two numbers or you dropped a negative sign or something, and then your answer doesn’t come out to one of the choices. That kind of tells you in the moment, okay, Hey, check your work. If you really know this, you’ll be able to figure out. If you don’t, then, then you won’t,

like penalize you too much for like a little mistake.

It’s like most things like often like a combined strategy, like you got two parts of the strategy that are, are, that are good at different things. And when you combine them, like have a, have a variety of questions. Generally a variety of question types is good, right? Variety of, of tasks is good.

Limiting the complexity of free response questions to limit learner discomfort

Alex Smith (00:00) I’ve actually also given like some instructions to my content team about limiting like the amount of expressions we, you know, or how big the expressions should be in free response. If it’s like a four or five term polynomial in a fifth degree polynomial or something like that, that’s gonna be real pain for students to enter, especially if it’s like one after another, after another.

Even numbers, if it’s like one over root two, like that’s totally fine. But if it’s like 51 over the square root of 353 π, that’s unnerving for the student. They’re sort of entering a number like that. And it’s like, is this right? Is it right? Doesn’t feel right. Doesn’t feel too nice.

We’ve just got to calculus and calculus is the point in the curriculum where you start encountering expressions like that, like derivatives using the chain rule and product rule and stuff. It’s like, don’t go crazy with the free response for these very complicated expressions because it’s unnerving for the student. It’s a pain in the backside to type. Just, just, you know, apply it where it’s necessary. And you can’t really

back engineer a derivative if it’s multiple choice, for example. So yeah, it’s like, again, it’s that intelligent use, using the best tool for the job at any given moment.

The free response parser is the epitome of the long tail of weird edge cases

Justin Skycak (00:00) I remember when we first started out with the free response parser, we were like, well, okay, this isn’t going to be too bad. We just like read the LaTeX and then evaluate the expression on a few sample points and then see if comes out to the right number. I mean, that’s kind of the general principle use, but there’s lots and lots of edge cases. So many.

Jason (00:17) vastly more complex and that

to handle more functions and stuff or be more affected by domain and range issues and error sensitivities to things that can really mess things up. We have this sort of algebraic equation parser, but

there was so much pre-processing that has to go in to handle the huge range of what expressions can be entered because it’s not just algebra, it’s all the trig and exponential and

everything else and every combination of those and roots and exponents. mean, crazy, right? But Justin, you kind of took it over time. Some kind of complaint from the content team was like, well, this

expression isn’t evaluating. What is going on? And you would get in there and have to write all this pre-processing to say, OK, well, what this really means, we have to transform it into more of an algebraic form so that the system can handle it.

Justin Skycak (01:08) Yeah,

it really turned into like an algorithmic sort of quant like task. Yeah. It’s a long, cause it’s the long tail of weird edge cases and weird behavior.

Jason (01:13) It did. It did.

Machine Learning has been the hardest course to develop

Alex Smith (00:00) The machine learning 1 course,

I mean, almost certainly the most challenging course to date in terms of how to deliver that kind of content.

It’s quite difficult to find people who’ve got both the machine learning knowledge, but at the level we need, which is the math chops essentially, like those who really understand the mathematics behind this algorithm at a very, very fundamental granular level. It’s hard to find people like that. Machine learning engineers do seem to be kind of 2 a penny.

But people that really understand the math, they’re much rarer.

Justin's free book on algorithms and machine learning

Justin Skycak (00:00) Introduction to Algorithms and Machine Learning. This is the book for the Eurisko sequence, the super advanced computer science Jason and I were running within the in-school program.

Jason (00:11) So the thing about what’s great about that book though is, Justin was just making this stuff up on the fly. He and I would

brainstorm and scheme like, maybe we should teach this. And he would write up some stuff and pull stuff together and come up with some lessons and or some exercises for the kids to do. And then, but he battle tests that with kids.

It’s a kind ⁓ of a high bar because you’re teaching stuff that is advanced undergraduate and sometimes even graduate level concepts 15, 16 year old kids.

Justin Skycak (00:35) Yeah, we went all the way up to re-implementing actual research papers in artificial intelligence, style artificial intelligence, the Blondie 24 program or training neural nets using evolution to play games. I mean, started off tic-tac-toe, but then checkers and more advanced stuff.

Jason (00:54) Yeah, was crazy. I mean, it was such cool stuff. you went through three different classes, three different years. And by the third one, you had

kind of fine tuned the process, the introduction of the content, the exercises, so by the time you get to the third one, the third year, you kind of like, here’s the PDF, this is the whole thing.

Justin Skycak (01:07) Yeah, pretty much. Yeah.

Designing the minimum viable problem that students can do by hand to get in their reps

Justin Skycak (00:00) Sounds like kind of

a balance of like, okay, students have to do the reps, right? They have to work things out by hand. They can’t just like, oh here’s a this method or that method. Here’s the neural net back propagation formula. Now transcribe it into your Python code. Great, now you know backprop, no, you just transcribed it. You don’t actually know it. You actually have to work through with some numbers, like see how everything flows, get a sense of

just what things look like, intermediate steps. But at the same there’s a way to make it like hell for the student because every problem takes like 20, 30 minutes because you’re doing so much. You need to avoid that also. So how do you get students to do their reps, but at the same time, not just crush them with the computation burden, right?

Alex Smith (00:43) Exactly. That is the hardest bit. So you’ve got to think of like the sort of almost like the minimum viable problem that they can do every single time. If you’re teaching like back prop, it’s going to be like the smallest neural net you can get away with with specially picked numbers so that the numbers can work out right.

It's basically impossible to find machine learning problems in the sweet spot of difficulty

Alex Smith (00:00) I’m not a machine learning expert, I learned a lot from doing this course. We were creating problems that

It was just obvious the student was going to be struggling with them like 20 minutes, 30 minutes on one problem. This is not good enough. So that’s what I spent most of my time doing. Just simplifying the problems, modifying the the scope in such a way you’re getting genuine reps on the algorithm, but not in a way that’s going to kind of crush you.

Justin Skycak (00:25) Yeah. And that’s like basically nowhere else online, right? Like you look up these algorithms online, like, can you just give me an example of something simple, like just how do you fit a decision tree? Just give me a concrete example. And like, basically nobody has a good concrete example online. It’s either something like that is so overly simplified that it becomes trivial and you don’t actually get to work through the algorithm.

Alex Smith (00:38) Hmm.

Justin Skycak (00:45) Or it’s just somebody gives you a notebook And then you just copy the code over. And if you want, you can inspect the outputs and stuff, but it’s so awful and messy. There’s nothing, there’s it’s so hard to find middle ground stuff.

Dynamic select questions are good for derivations

Alex Smith (00:00) got like one question topic which is dynamic select. It’s almost like a multi-part problem in a regular topic.

So you start off with like the first part of the calculation, and then if you get that right it unfolds to the next part of the calculation, and then if they get that right you unfold to next part of the calculation. If you do that right, you’re done. It’s a wonderful pedagogical tool because it really allows you to almost like force students down

path you want them to go.

Like the Laplace transform e to the KT. The answer is always the same. It’s like, well, how do you test that students know that? Well, what you can do is you can force them through the steps to derive that using these kind of dynamic select questions.

And through that, they can get lots of reps on exactly that process.

Scaling content production is the goal for 2026

Justin Skycak (00:00) This coming year, another thing that we need to do is scaling out factory level production.

We’ve been doing a good job of continually level up the quality of the courses. And now we need to expand also rate of production of what we figured out how to do. You can’t really do that until you know what you’re doing, until you get the prototype site, get the V1s. But then you kind of scale it

Alex Smith (00:27) We need to transition from being like a workshop to a This kind of workshop setup where it’s me and few guys just there creating all these courses, it’s super fun. It’s gotten us to where we are now, but if we’re hopefully become a truly household name,

then I obviously need to stop being the workshop manager and be like the factory manager. So how do we go about doing that?

When scaling a team too quickly backfires

Alex Smith (00:00) We need to transition from being like a workshop to a factory.

So how do we go about doing that? So without dropping quality? exactly.

Jason (00:04) without dropping quality, continuing to innovate. And that’s a tough ask.

tried to do this. I mean, I initially brought up this conversation the end of 2024. I was like, got to get this machine learning course, and we got to speed things up, and we got to go.

But I think one of the things we ran into is that it’s really, really difficult to find people

who have the subject matter expertise and have the pedagogical instincts. Also you just can’t hire that quickly. always a disaster. You can’t just hire four or five people

in like six weeks to do something that isn’t like a common skill that everybody knows how to do and expect that to go well. And it did not go well. It caused us a lot of inefficiency and lot of frustration.

Alex Smith (00:44) Yeah. So I mean, one thing I’ve realized: You need to be very, very careful about how you hire. So you don’t hire en masse.

But if you’re going to do that, you really need to have like the systems and tools in place to just monitor performance. When you’ve got like just a team of a handful of people, you can kind of keep an eye on what people are doing. and that’s probably not the right time to sort of invest in all these tooling and analytics to check that everybody is doing well. But once you start scaling,

I can’t stay necessarily on top of every single one in the same way I could when it was three or four

You need to measure a team's output to ensure efficiency

Justin Skycak (00:00) It really seems almost a software system just made out of You need to invest time into like setting up the

right abstractions, the the right work allocation, everything, and monitor that that’s getting the results that you want.

We can measure performance every of the way, and have it gamified a little bit in terms production doing how much week? How many work putting out? good or bad? How does that compare to what everyone else is doing?

Why we teach concrete computations before abstract proofs

Jason (00:01) is that we kind of take a of a course before you do a proof version of a course.

Because the problem is when you do a proof-based course and the students don’t actually have any real intuition on how these operations work and they haven’t done any problems and they’re just proving stuff about objects that they don’t really understand deeply, the student isn’t getting as much out of it. They’re struggling.

Linear algebra have any proofs. So if we had a second linear algebra course, it would be entirely proof-based.

And likewise, I think you do complex variables, and then you’d have a complex analysis. But if you were like a engineer physics person, you would say, I don’t need the proof-based. I’ll just do complex variables and I’m good to go.

Why we separate university-level courses into computational vs proof-based

Justin Skycak (00:00) seems to be confusing for a lot of people who are not so familiar with university course offerings that like There are levels to things like linear algebra, complex variables, complex analysis.

Sometimes I would get asked like, well, how come your calculus 1 course doesn’t have epsilon delta limits in it? Well, that’s what you do in Real Analysis. It’s not necessarily in a calculus 1 course, unless it’s like a sort of honors calculus at like an elite university, which is really more like a Real Analysis course. They just call it calculus because their

real analysis course is actually like advanced real analysis. So there’s kind of a fuzzy labeling on these things. So I think once we have kind of the second more advanced version of each of these courses out, it’ll be very clear. Like, okay, here’s what linear algebra is. Advanced linear algebra, that’s more like Axler’s linear algebra.

The best textbooks for beginners are NOT the most advanced ones

Jason (00:00) You’ll hear these ridiculous conversations in places like Hacker News, these discussion of math books and they’ll say, well, I think this is the best: Axler or whatever. They might be have some fond memories of this more advanced course, but they’re not really appreciating how much foundational knowledge a student needs to have to come into it to be successful.

And so if you don’t know any better and then you try use that book, the first one, it’s a flame

Alex Smith (00:21) says that it’s just the curse of knowledge, isn’t it? Oh, I remember this advanced linear algebra. That was really cool. And this is the book we use. You should take it. like, as you say, just no appreciation for how they actually probably got to that point in the first place. Some people can go straight to

Axler’s book, but they’re pretty rare.

Computations build intuition for proofs

Jason (00:00) At the University of Chicago, if you’re a math major, the linear algebra was embedded in abstract algebra, and it was mostly proofs. And it was terrible.

I remember our textbook was Hungerford, which was an abstract, it was a graduate It was just so much harder than it needed to be. I guess if you’re dealing with some absolutely brilliant students, and they just work like crazy, and they work in groups, you can kind of

them to fight their way through it, but it’s so inefficient, and it’s so and it’s just unnecessary. That is not the way, and so what we do is take a concrete, straightforward, let’s get your skills, your intuition up. And then once really understand how to do these calculations, you really understand what eigenvectors and eigenvalues and row echelon form all this stuff is just like

instinct to you, then we can make proofs about it. And writing proof about that is not harder than writing proofs about divisibility and parity because you just understand it intrinsically, and then it’s just a matter of having the insight through the proof.

Intuition comes from repetition and hard work

Justin Skycak (00:00) As you always say, the intuition comes from the repetition, right? And then the proofs it’s really represents intuition, but that’s not how you absorb intuition. That’s the product of intuition. is not the product of that.

Jason (00:13) Yeah. It’s, it’s a, that’s just so frustrating when people say, Oh, I just, just give me the concept of intuition. It’s like, I can’t just get you the intuition. Intuition a product of a lot of hard work. That’s where the intuition comes from.

You can't learn philosophy without lived experience

Jason (00:00) intuition is like,

Justin Skycak (00:00) The analogy I

sometimes like to make is like suppose that you just you had a book of life quotes right from your hard-won experience of life and you have a kid and when the kid is years old, you hand them the book. They have no idea what you’re talking about. You can be completely right, the quotes, in your mind they can represent so much valuable knowledge, but

the kid’s not gonna know.

if they say like, I get it, they don’t know.

Jason (00:20) a teenager, even someone in their

I even sometimes think reading Plato and all the stuff really is more useful when you’re like in your 30s or 40s. Gotta have some mileage on the road, come back to stuff like, what is truth fairness? What is justice? You have to see a lot of examples these people struggling to come to any agreement, consensus on these things. And then you can come back, like, let’s go down to the basics.

You gotta get the details before abstracting

Jason (00:00) All these life lessons and stuff, you got to suffer. You got to make lots of mistakes.

You have to get the mileage on the road. And that’s the same with math. You know, as with anything.

Justin Skycak (00:08) You got to get the details, the microstructure before abstract Otherwise you just end up feeling like you have just been pushing symbols around in some logically And you’re like, what’s an eigenvector? What’s an eigenvalue? I don’t know, but it has this symbolic property that helped me complete this proof. So I guess that’s what it is. It’s like, well, yeah, you don’t really know what it is.

Jason (00:12) use.

Yeah.

Earning the right to scale

Justin Skycak (00:00) A lot of infrastructure built, just system infrastructure to deal with When you 10 users, 100 users, it’s kind dumb to worry about like, oh, will this work great when we than that? It’s like,

Don’t worry about it. Just like focus on getting more users and then you earn the right to solve the problems that come with that.

How small companies accumulate infrastructure debt

Jason (00:00) When you’re a small company, you’re just trying to do this 80-20 solutions on everything and get stuff working. Sometimes you put a little more love into something because you know it’s really important, but a of times it’s like, okay, that’s a solid B, B plus, whatever, time to move on. I got 30 other things I got to do. I can’t just sit there and tweak this thing for the next three days.

But there’s some things that you do even little less than 80-20, you’re just just good enough, like, okay, it works, I’m moving on. But this starts to accumulate, and then it can run into

You accumulate technical debt or infrastructure debt.

found an AWS expert.

So, terms of the infrastructure,

this guy is going to help me like upgrade all the things that I probably should have been upgrading before.

Wisdom is gained by trying, failing, and learning

Jason (00:00) in order to build towards something good, whatever it is, you’re going have lots of failures along the way. So you’re going wisdom the years.

You actually do things. run experiments like let’s try this. Some things work. Some things don’t. You learn from those. You go, ah that hurt. That felt good. That worked. can we take away from this? We learn and you run another experiment. But all this honest lessons that you are

incorporating into a compressed sort of

so that you’re operating in as optimal way as possible at every stage you’re at.

You learn the most important problems to solve through first-hand pain

Justin Skycak (00:00) We always talk about like these kind of made up problems that you can envision into the future, but if you have suffered at the expense of a problem in the past, and you know that problem is going to come back shortly, if you don’t stay on top of it, then this is not a made up problem. This is actually concrete thing that punched you in the face before. And

Jason (00:18) If I had just told you about this stuff, it wouldn’t have made as much sense to you as like you suffered through it. You had to help us detect what were the inefficiencies where it come from. You had to help debug queries. going forward, you’d be like, okay, we got to be careful with, with some things.

Math Academy is like a personalized expert tutor

Jason (00:00) Ideally, Math Academy should work an expert tutor, that’s really our mental model. What should the system do? It’s like, okay, well, what would you do if you were tutoring? What would you do right in this situation?

You can’t you can’t get there the tutor and just say, well, I want them to cover this topic right now. The tutor be like, I can’t teach them that because they don’t know all these other things that are needed. But what I can do is I can get them there and you’d be like, I respect your expertise as a tutor, please just get her there as soon as possible because she can’t afford to not do well on this midterm, you know.

Learning rate is limited by working memory capacity

Jason (00:00) We know from cognitive science is long-term memory you’re sleeping. Which means there’s only so much you can do in a certain day. Why?

you have a certain amount of working memory. The average person has four to seven slots of working memories. Think of it like the size of your desk and each part of the problem is paper and you can only have so many papers on the desk before you just can’t look at them all at the same

If there’s a lot of things that have to do with this problem taking slots up in working memory, you can’t see everything in the problem, can’t problem solve, can’t think your way.

If you just learned one to two things at most a day, you let this stuff consolidate to long-term memory, you’re operating with your full with your working memory.

Too much customization makes a product frustrating

Jason (00:00) The reality is that you wanna give

to operate the system in a way that really fits what they’re trying to achieve, but you gotta put limits on it so doesn’t allow them to do something that’s gonna be detrimental. Otherwise, they become frustrated, you lose them as a customer, and they blame you.

There’s a danger, there’s a downside to giving people too much ability to customize, giving them too much control because they can tie themselves up in knots, which

increases your customer support load because then they’re confused, and then they’re upset at you, and it’s just this whole thing.

More users, more customer support problems

Justin Skycak (00:00) At the beginning when you have users, zero users, you don’t have to worry about customer support. But once you get to

a certain level of scale, you can really shoot yourself in the foot by releasing something that is half-baked to

Jason (00:13) have to try to think ahead and be like, is this going create customer support load? If so, how do we get ahead of

You get someone sends you email. It’s all

And then they’re yelling at you. Even though you can develop a little bit of a thick skin to it, it’s really not enjoyable. And so you try and do everything you can to make this stuff clear, but then you send emails to explain stuff, people get mad. Why are you sending me

So all you can do is chip away at different sizes and do the best you can.

Justin Skycak (00:34) So not.

Manually onboarding customers is necessary to understand your customers

Justin Skycak (00:00) just reminds me back when we were just getting our first you would onboard every single customer a video call for like half an hour, like an hour,

Jason (00:08) Hour,

Justin Skycak (00:09) A full hour.

Jason (00:09) It was never less than an hour. Almost always an hour, sometimes more.

It was a critical thing to do when you’re doing a startup initially, you’re really trying to do customer development, which means you’re really trying to understand who your customer is, what the problems are that need solved. Because you have this idea in your head of that. And even if it’s sort of

just a lot of detail that you don’t

The quickest solution is manual until it no longer scales

Jason (00:00) I remember listening to an interview years ago with Brian Chesky, one of the founders of Airbnb. And he was talking about how initially they just had a big spreadsheet of photographers that lived in these different cities where people were trying to rent their places out and on the spreadsheet they would assign it, go take photos of this place.

But that’s how you start. Everything was like manual, grind through it that way until it gets to a point where like, okay, can’t work anymore. We have too many photographers, too many cities, and then it sort of forces you to do it. Now, what ends up happening, at least it seems to happen in my case, is you do the manual thing just a little too long.

Right, because you don’t want to have to bite the bullet because OK, do I spend two, three, four hours setting up which is really annoying, or do I spend, a day, day and a half or whatever it is, building out this UI stuff and all the back end functionality, and you’re just like, I don’t have time for this.

Our mission is to make effective learning available to everyone

Jason (00:00) We want people to use it benefit from the system because that’s our mission is to make effective learning to everybody.

You know, there’s someone could say well, Jason, if you just niche down, you just really hyper-focus to the one group no to all these other people. That’s one approach. Some companies might do that. I don’t want to do that.

Justin Skycak (00:17) don’t want to be just the math system for homeschools. And that’s it. No, anyone who wants to learn math, who’s serious about learning math. That’s our customer.

Jason (00:21) or

Math Academy goes to college

Jason (00:00) had our first university want to use it. And he’s like,

if I try and get this through the math department, it’s going to take forever. But university students or college these courses, you can say, you’re going to use certain resources textbooks that this is the requirement for the course.

that created a whole different flow because he’s setting up at the class. It’s calc two. He’s not paying for it.

So they’re paying for it separately, but they need to immediately get funneled into this class. They’re not doing a subscription. They’re paying for semester long or a quarter long thing. It’s from this date to this date. And each student is paying with their own credit card.

so I bit the bullet and spent couple days working on because then the sign up flow is all different. now that it works, professor comes to say, hey, you know, we’d like to use Math Academy for

It’s like, great, here’s what just up your class, and you paste in a bunch of emails addresses like you do with any of these kinds of oriented apps. And boom, takes them, they have a signup page,

the whole thing.

Build what customers actually need

Jason (00:00) There’s a dance that between the entrepreneur the customer.

You can’t just go off and build a ton of stuff and not get feedback from customers because you just build the wrong stuff. Building an initial version of your product and you say, I’m just going to work on this for years and then release it, that tends to be pretty dangerous because if you miss the mark, you could just have wasted years of time and a ton of money.

It works a lot better you can have a general direction, but you let the customer lead to some degree.

All this stuff, building stuff by hand, doing stuff manually, doing

tons of zoom calls, talking to people, you’re getting to know your customer’s problems and needs, and you’re solving their problems manually. A lot of times it’s really painful, sort informs you about the shape of the problem really is cause sometimes it’s more complex or nuanced what you initially imagined.

The privilege of grinding on something you love

Jason (00:00) I’m incredibly grateful that we’re in a position where we have these problems.

day I’m just like, this is so great, though it can be overwhelming at times and exhausting and frustrating when you customer support load.

I work seven days a week

working my ass off.

It’s like someone who’s maybe like a in the NFL.

On the one hand, they have to grind. They are in physical pain the time from how hard it is on their body to train their ass off year round, but they’re incredibly grateful. I’m in the National Football League, man. I am paid to play football. I mean, this is awesome, but it’s also super, super hard.

There's no substitute for real-world experience

Jason (00:00) It’s harder to a lot of insight in how something should work until you see all the failure modes and you’d experience them personally many times. And you’re just like, this is how this is not gonna work.

Justin Skycak (00:09) Do that enough times then I mean, what’s remaining is the truth, right? It’s what’s going to work. But the great part though is like, you don’t have to depend on anyone else telling you, this is how it’s going to work. You don’t have to on trusting somebody else like, this person knows what they’re talking about. They say this is what generally works. I guess that’s what I’ll do.

Jason (00:13) Here’s the

You need to do things yourself, but a mentor can accelerate your progress

Jason (00:00) Ideally in life, you need to do things yourself, but you need to have somebody who’s maybe done it before who can just sort of help you recognize and frame what’s going on so that you can kind of speed run the lessons you take from it.

Using free response and multiple choice questions to the maximum advantage

Jason (00:00) Free response, depending on how involved the expression is that you’re entering can be sort of a tax on you. I it’s exhausting do a lot of that. Right. So if they had to that all the time, and especially it was longer algebraic expressions, like, Oh, you know, you do it all the time. You know, they just, it’s, it’s, you’re, you’re creating this sort of effortful tax.

And it adds up where he’s like, we want you to keep going. Right. So we want to kind of multiple choice where we can get away with it. and then, and do free response where we can get away with it. Like let’s use each to the maximum advantage. Like we need to sometimes prime you. You need to do the whole thing and enter it other times. The multiple choice little more robust against silly arithmetic errors, but it just, it’s, you can go a little faster. And we’re trying to get as much out of the student

as we possibly can in amount of time they’re gonna put in the system.

It's hard to find people who know machine learning, math, and how to teach

Jason (00:00) So you can find a lot of people who have experience with pytorch and tensor flow and can talk at a high level how these algorithms work and what circumstances tend to work. And they’re, they’re good. They’re practitioners, right?

A practitioner, an engineer needs to have certain types of skills. Other skills, really understanding the mathematics at deep level isn’t always necessary. I mean, depending on what they’re kind of doing. And so if they’re not using that stuff, or they didn’t get a graduate degree in this stuff, or even if they did and they haven’t used it in a while, it’s just atrophied, right? So you, you talked to a lot of people who just didn’t quite have the

skills.

Alex Smith (00:31) Yeah, I mean,

by all accounts, like, you know, decent machine learning engineers in their own right. But the understanding of the math, which as you say, they don’t necessarily use day to day, or they might’ve studied it years ago and forgot or whatever. And then there’s like the pedagogical side of it. It’s like, how do you, okay, you do understand the maths on a very fundamental level, but

how do you present in our kind of setting?

The Eurisko book was the best resource for developing the Machine Learning course

Alex Smith (00:00) The machine learning 1 course,

almost certainly the most challenging course to date in terms of how to deliver that kind of content.

Now, we’re really lucky because Justin wrote a book.

It was super helpful. I mean, I’ve got all the big machine learning books, but is the one I use the most to actually help with this course, at least in terms of the philosophy of how this stuff is delivered. You know, it’s like you’re presenting an algorithm and you kind of work through the algorithm kind of step by step.

You you write down the expression for the iterative procedure. Do one successful iteration, do two successful iterations. Now do the whole thing until you get it to converge. It was, was the book plus a little bit of my own sort of experience in teaching like numerical methods and stuff. The task really was to take

this idea and just put it into our system.

The challenge of picking numbers so that manual calculations are not overwhelming

Alex Smith (00:00) So you’ve got to think of like the sort of almost like the minimum viable problem that they can do every single time. If you’re teaching like back prop, it’s going to be like the smallest neural net you can get away with with specially picked numbers so that the numbers can work out right.

Justin Skycak (00:12) The number picking is kind of interesting too, because sometimes that’s counterintuitive, right? Like you might think like, well, I give you inputs of zero. And then you run that through and suddenly by step two, you’ve got these nasty decimals because everything’s been like log transformed or whatever. In order to get nice numbers, you say like, okay, your inputs are natural log of 2, and natural log of 3, and 1. And then those look so bad like initially, but

Alex Smith (00:33) Yeah.

Justin Skycak (00:38) by step two, it’s like, well, they’ve converted into very nice whole numbers that you can just kind take with you.

Challenges of keeping a large, globally distributed team on the rails

Alex Smith (00:00) you’ve got team of a handful of people, four or five people, you can kind of keep an eye on what people are doing. But once you start scaling,

if it’s 15 people, 20 people all contributing content, I can’t stay necessarily on top of every single one in the same way I could when it was three or four

And of course, we’re obviously truly international. I mean, my team are kind of based all over the world. And so it’s not in the same office. This is even harder. You know, it’s like, you need kind of proper reports. What has this person done

today, this week, this month? And so all these kinds of analytics need to be in place if you’re going to kind of scale up successfully. And it might not necessarily even be that person’s fault. They’ve read the training material. They think they’re doing a good job. But it’s just my job to have access to that data and go, actually, that guy isn’t doing good. But let’s find out why. And

perhaps just one conversation is all it takes to boost that performance. But unless you’ve got the data, you just don’t know really.

Math Academy is a machine that optimizes learning

Jason (00:00) Math Academy is a machine you’re trying to optimize. It kind of reminds me of that joke mathematicians are machines that turn coffee or weak coffee into lemmas, something like that. And by that similar analogy, we’re a machine that turns

money, subscriptions into successful mathematical or educational progress. How do we optimize this machine? So part of it, we have a lot of analytics that we’ve built over the past few years to analyze pass rates and where kids are struggling and everything. So you can really fine tune, be alerted to say, okay, well, we’ve got some low pass rates on some topics.

are those, where are they failing? What’s the number, how to fix it? And then you can use that to fine tune. so to continually which has really, really helped.

Measuring then optimizing content production

Jason (00:00) We need more metrics about how the company is doing.

far our biggest center is content. like, OK, we getting our money’s worth? Are we moving as fast as we can? Like, when is stuff behind? Is it on And so what we really need, and which we’ve started working on, is a lot more

metrics into understanding how this process is going, you know, how it’s going overall, how a course is progressing, how individual people are doing, how efficient our spend is. Because if we’re gonna create two three four or five times the amount of courses that we did in the past without dropping quality, we better have a really granular of insight into every dollar that we’re spending on this on the content.

Axler's linear algebra textbook is not meant for beginners

Justin Skycak (00:00) There are levels to things like linear algebra, complex variables, complex analysis.

So I think once we have the second more advanced version of each of these courses out, it’ll be very clear. Like, okay, here’s, here’s what linear algebra is. Advanced linear algebra, that’s more like Axler’s linear algebra. And you know, Axler even says in his book, like this is meant for somebody who was

basically already taken a first course, who has already had exposure to it. But a lot of people just kind not read that or just don’t want to that don’t straight into Axler unless you’re just incredibly gifted and can all the missing prerequisite knowledge on the fly. But most students, yeah, they need to of go up these levels rung at a time.

This is just performative education

Jason (00:00) My middle daughter was in ninth grade and they were reading Adam Smith.

I was like, she doesn’t know barely little supply and demand, she doesn’t know about economic free markets, she has no intuition for how commerce and economies and any of that stuff works. And you’re talking about the Adam Smith level theory It was like performative. It was like to show off to the parents, look how sophisticated our kids are. It’s like this is just it’s just dumb.

Wisdom is the abstract compression of lived experiences

Jason (00:00) intuition is like,

Justin Skycak (00:00) The analogy I

sometimes like to make is like suppose that you had a book of life quotes right from your hard-won experience of life and you have a kid and when the kid is 5, 10 years old, you hand them the book. They have no idea what you’re talking about. You can be completely right, the quotes, in your mind they can represent so much valuable knowledge, but

the kid’s not gonna

Jason (00:20) You can’t take this like consolidated compressed knowledge and put it in someone’s head and it makes any sense because it just doesn’t, you know, they can mimic it and they can just repeat it to you. And you’re like, oh, that’s amazing. They know this stuff. It’s like, they don’t really know it. All these life lessons and stuff, you got to suffer. You got to make lots of mistakes.

You can talk to a very, very bright, open minded kid who respects what you say, but they’re just, don’t get it. You have to get the mileage on the road. And that’s the same with math. You know, as with anything.

Startups and life are better when they improve gradually rather than suddenly

Jason (00:00) There’s a lot of benefits to organic growth opposed to artificial growth. So when you go and raise a lot of capital from a venture firm and

hire a ton of people and you just spend a lot of money in advertising and you just like go, right? It’s sort of like the similar problem, like when somebody wins a lottery and they have a lot of money and they didn’t really earn it step by step so they don’t know how to manage it, they don’t know how to appreciate it and they kind of blow a lot of it and it creates all kinds of problems in their life, their relationships with other people because it was just thrust on them as opposed to if you spent 30, 40, 50 years building your wealth, it’s

you know how to manage it. You know how to interact with other people in regards to it. Life usually goes better when you go in a nice, smooth upward gradient, as opposed to just bam, you’re here. You don’t have the appreciation for not only the benefits, but also the pitfalls and issues.

Wisdom is the result of years of failures

Jason (00:00) Wisdom is a result of failures.

And so in order to build towards something good, whatever it is, you’re going have lots of failures along the way. So you’re going wisdom the years. And that wisdom will allow you operate effectively at higher levels when you get there. That’s why you take a 23 or 25 year old, you make them president of the company or something. OK, well, that’s going to be a disaster,

unless it was like a little startup that person built themselves. And even then if it happens too fast, they didn’t develop the skills. They don’t know how to manage people. They don’t know how a lot of things It doesn’t mean that you can’t run stuff to some degree, but you shouldn’t skip steps.

Justin Skycak (00:34) Yeah, you can speed run by, by compressing the work by making it more efficient, but you can’t speed run by skipping. There’s no shortcut. There’s just running on long

Want more users? Build a better product

Jason (00:00) One of the reasons that I’ve wanted to approach Math Academy the way we have, which is that we have not taken outside investment. It’s all completely self-funded. We don’t pay for advertising. We simply put things on hard It’s you want users, create a better product.

The hard way means that you have to solve problems. You don’t get to postpone the problems or ignore the problems or paper over the problems. By doing things slowly and things ourselves, we learn

how to improve the pedagogy. We learn about how to create more efficient, more cleaner code, but you gotta learn all those Do things as efficiently as possible, but gain the wisdom along the way so that you’re operating in as optimal way as possible at every stage you’re at.

Do it manually until you understand what the heck you're doing

Justin Skycak (00:00) if you had to build out all that infrastructure just at the beginning, you know, before doing this manually, before going through all the reps. And you had some vision of like, this is how things should work. Doesn’t this make sense in theory? And then you built all that out. It’s like, not only would that be time wasted, but you’d also have to choose between undoing and rebuilding it all or, just trying to like contort it into some way that it should be, or trying to force customers

to use it the way that you which is not way that it in practice. And it just would have been a bad situation either way. So just do it manually until you understand what the heck you’re doing.

You can't eat a whole elephant in one bite

Justin Skycak (00:00) If you want to cover a large swath of knowledge in a single day or two days, like, sorry, that’s, that’s just not going to happen. Cause there’s a limit how fast you can build upon this knowledge. We can’t go not knowing any calculus to knowing a topic halfway in the course over like two days. Even if you were to put in the work to try to build up all those missing prerequisites along that path,

you build too much on unstable, unconsolidated you struggle. And so you’re not really going to get through topics successfully. Most likely, you’re going to get to a point where you’re just failing stuff because you haven’t allowed the knowledge to consolidate. You haven’t done any on that. You’re just trying to like eat the whole elephant in one

Why you should learn breadth-wise, not depth-wise

Justin Skycak (00:00) There’s a portion of people who putting hundreds of XP per day. They’re learning

per day. Instead of taking those 10, 15 topics and spreading them out wise along knowledge frontier, if you allow them to go wise, now they are going to overload their working memory. when you go breadth wise,

you’ve got the prerequisite knowledge consolidated for all these you want to Cause any one of those, only material But when you start like, let’s go 10, 15 topics stacked on top of each other, well,

every single one of those topics you’ve learned that day has not had time to consolidate yet. So it’s really your working putting pressure on your working memory much more than if you had spread that

PODCAST 6

1 in 12 UCSD freshmen don't know middle school math

Justin Skycak (00:00) 1 in students entering UCSD in 2025, they don’t know middle school math.

We’re talking below algebra 1.

Fractions, order of operations, what’s an exponent.

It’s so ridiculous.

They had a remedial math course for students who were missing high school math foundations. And that was too advanced for a lot of the students coming in. So they had to make a remedial, remedial math course, two layers back.

So the crazy part is you don’t actually get a good signal of who’s going to be in remedial math based on their high school grades.

“In fact, for more than two decades, the mathematics department has found that out of all available student data, the single best predictor for math placement has been the SAT math section score, with the ACT score being an equally good predictor.” And guess what happened in recent years? They didn’t require those scores to be set.

happens, there’s a ton of learning loss. And now we’re just going to say like, okay, we’re going to do like this holistic admissions without test scores. We’re going to remove things that actually measure actual learning well. And we’re going to rely on high school grades,

are known to be inflated now, especially after the whole pandemic thing. So now we have this kind of shit show that’s happening.

The massive increase in GPA during the COVID-19 pandemic coincided with less work

Jason Roberts (00:00) know?

Justin Skycak (00:00) Well, the

really hilarious part is that if you were to just look at a graph of gradual creep of GPA over time, and you actually thought that that indicated students are getting better, then you would have to come to the conclusion that over the pandemic, just got way, way, way smarter and more studious and everything because there’s a massive jump in the GPA there. And yeah.

Jason Roberts (00:21) Which is the exact opposite of what happened, right?

Justin Skycak (00:24) Yeah, exactly. It’s the exact opposite,

Jason Roberts (00:26) Kids stopped

working, doing nearly as much was many, many fewer assessments if there were any real assessments. Teachers were kind of throwing their hands Kids were not participating half the time. Teachers were really struggling to real learning experience happening, much less holding them accountable.

Justin Skycak (00:31) Mm-hmm.

Jason Roberts (00:45) You know, there’s always excuses. Well, you know, this is hard and that’s hard and everybody’s stressed and like, well, okay, that’s fine. But there, there are always times in the world where there are disasters, wars going for years. It’s like the the students have to learn and you still figure out a way to make that happen because it’s

Eliminating standardized tests made grades a target, rather than an objective measure of performance

Jason Roberts (00:00) When you take away standardized the only thing to go is the grades and recommendations, Right. So it’s like it counts for more which means parents are

hyper focused on it.

So then I think teachers get put in a bad situation where they feel this pressure: we’re going to ruin my kids’ That C or D is going to ruin my kids’ future. And teachers don’t want to be the bad guy. And they certainly don’t want to have endless grief from the parent.

and the grade grubbing from a lot of the kids and the complaining so teachers are just like, fine, everybody gets a B plus or above. don’t, whatever.

Now, and it may only be a minority of students. It probably is, a minority of parents. It’s brutally painful. It makes it horrible experience.

It’s not like, teachers are just doing a bad job. Teachers live within a framework, within an ecosystem. Principals live within this If parents can complain, complaints with the school board, then everybody is like held accountable to this endless

negotiating and nitpicking and complaining and accusations.

And then of course that coupled with social media, where parents can just take to a higher authority, being the authority of the internet,

blow it

Harvard had to add remedial support to its calculus course

Justin Skycak (00:00) having to add a ton of remedial support to its calculus courses. Cause they’re getting the same kind of situation of people coming in, and they are nowhere close being able a calculus course. They got tons and tons of missing high school and even

middle school, Like at Harvard, really? It’s just incredible.

Eliminating standardized tests puts a target on teachers' backs for incessant grade-grubbing

Eliminating standardized tests allowed colleges to admit students based only on subjective criteria

Jason Roberts (00:00) the SAT and the ACT sort of got removed… So here’s the thing, There were a lot of people in education who were trying to remove these, and the was their excuse that they needed.

There are a lot of people who have been pushing hard for this for a long time.

And then what happened is, OK, if we don’t have standardized tests, and nobody really has to prep for things, make sure they know this stuff. Teachers don’t really have to worry as much about holding kids accountable for learning what they need to learn so they can do well on these exams.

So it’s just like, okay, we can change how we admit people, who we admit. And, and that combined with the grade inflation, the lack of accountability, the lack of signal,

The whole thing is a sham, complete and utter

sham.

Without objective criteria, standards fall to rock bottom

Jason Roberts (00:01) don’t have some kind of objective criteria, and you don’t have…

a way of holding people to certain standards and there’s no incentive to hold them to standards, the standards fall through the floor. They don’t just drop a little bit, they fall through the floor. They just absolutely degrade to absurdity.

s

If you cheat in Justin's class, he will call you out on it

Justin Skycak (00:00) So I had a kid

I just noticed that like he was completing problems really, really quickly. He’d do an integration by parts in like 15 seconds.

And we’re not talking like after having done tons of practice on it.

Which is weird because he did like really terribly on the diagnostic when he placed into the course. And it’s also especially weird because he’s doing most of this work outside of class. Actually, during class, He’s one of the kids that I have to like constantly like, hey, back on task.

So I’m like, okay, something’s not adding up.

I called him out on it. I went up to him during class and I was like, this is amazing. Like, you got to tell me how are you solving these problems so quickly? He’s silent. He’s just like,

looking at me. like, ⁓ shit. And so I made it pretty painful for him where I was like, okay, why don’t you just solve this problem for me and I’ll watch as you do it. And then so we sat there for like minutes.

So eventually he just comes out and he’s I’m sorry, I was using PhotoMath outside of class. PhotoMath is an app where you hold your phone screen up to the problem, and then it figures out the answer for you.

Peak laziness.

Well, I sent an email to his parents and him just basically stating what I’m supposed to do is report you the principal for all this, has a big impact on grades detentions. But as long as you

it honestly going forward, and I’m gonna be watching, then not make this blow up in your face.

I caught it pretty early and kind of got back on the rails.

And yeah, at the end of the day, this kid not only learned the integration by parts, but also learned a very good lesson

LLMs have made it so easy to cheat in school, and teachers have to work harder to make sure kids learn

Justin Skycak (00:00) LLMs, chat GPT. It’s so easy to cheat. It’s so easy to turn homework does not reflect your true level. And if you’re, if you’re a teacher who’s getting tons of pressure from parents to like give my kid an A, and you’re getting homework

that is completed correctly and everything, I mean, it takes effort to really go them out on it and collect evidence to what’s happening.

Kids are going to be knuckleheads, right? They’re going to be knuckleheads all the time. And ultimately as a teacher, you want the best thing for your kid. And you want them to learn the material. You don’t want to like just them into trouble.

But at the same time, you can’t just let this keep happening. Well, you’re like, how are you going to get the kid back on track? So sometimes there’s like a middle road. I think it’s for me is like it was always like you get one strike, one offense that where I notice and I call you out on

there were a couple of times where somebody made a strike one 4 times over the course of years teaching in Math Academy And only one of those times they do it again. And the one time that they did it again, yeah, blew up into this whole thing.

Cheating is a slippery slope

Jason Roberts (00:00) It’s like borrowing money from the mob or something. You’re never out. Like now you’re in. You get involved with criminals, there’s no way out. Once you start cheating, it’s kind of until someone says, I’m going to help get off this track, Like you’re addicted to a drug and I’m going to help you detox and I’m going to help you.

find a path where you can be successful and be in a good place in life. If kids can cheat and get away with it, they’re learning that cheating pays. Crime pays.

And the teachers and parents just look the other way and make excuses for it. And then you just basically undermine their whole You’ve undermined a lot of their sort of ethical development. You’ve distorted reality a little bit. think reality is this thing that you can just kind of fake and stuff and that’s just gonna keep working.

Bring back blue books

Jason Roberts (00:00) I don’t ever believe in take home exams. Definitely when LLMs coming, you can’t write papers.

went to school in college, University Chicago,

the final exam were with those blue books, you those little paper blue books, and you would write those in class, and it was a 2 or 3 hour

final exam period and it was just like you’d fill up, you know, 6, 7, 8, 10 pages of a blue book for each essay question, you just write on the fly. And that’s what you gotta do for all of these more paper writing classes, history and English and things like that

because it’s too easy to fake.

Teachers have to be firm to make sure kids take them and their work seriously

Jason Roberts (00:00) You kind of have to do that as a teacher. You kind of have to scare the kids, you don’t mess with this teacher. You do a good job and otherwise it’s gonna be a horrible experience.

If you come in, you’re like, we’re all going to be friends. And I just think of me as your study buddy, and I’m going to help you. Kids are like, oh, this is a joke. And they just have no respect for you. And then you never get it back. And then if you try and tighten the reins later

they just can’t cause kids are like, And so you set the tone early.

then you lighten up.

Unfortunately, any teacher thinks you can come and say, well, I don’t do that. I just I’m just happy all the time. I like, guarantee you that kids are not doing anything close to what they’re supposed to be doing.

The only way to verify the integrity of a student's writing is to watch them write it

Justin Skycak (00:00) I don’t think there a way other than just having students do stuff in front of you.

Once you allow it to be a take-home thing, especially with the LLMs, you lose the integrity of the data. You just don’t know anymore. If you’ve read, interacted with LLMs a there are things that you kind of notice,

where you get a vibe like, ⁓ this feels bad. But that does muster when trying to argue that a student has cheated a serious way. there’s all these gray lines, right? There is a situation in which you can use an LLM to, in good

like genuinely help you produce a better paper. But there’s a line between actually cheating off the LLM. And

And if you’re trying to argue that a kid has used LLM in a cheating off of way that’s over the line. And you’re just saying like, wow, this kind of sounds like it’s written, just doesn’t pass the sniff test. so if and or their parents really want to fight against you,

they can turn this into a whole situation of like, because you don’t have this really dispositive evidence, they can dig their heels in and create a whole situation.

If you don't correct students' behavior early, it will compound into an unsolvable problem

Jason Roberts (00:00) Unfortunately, any teacher thinks you can come and say, well, I don’t do that. I just I’m just happy all the time. I like, guarantee you that kids are not doing anything close to what they’re supposed to be doing.

Justin Skycak (00:07) Yeah, because it’s like What happens if you’re if you just play cop like all the time 100 % of the time, gonna be slightly off day, they’re drifting off course, and they’re gonna end up miles away. You don’t have to play bad cop all the time. But it’s just whenever there is a slight deviation off of behavior that needs to happen, you have to call it out. Otherwise, it’s gonna compound into a problem.

Jason Roberts (00:20) I’ll talk.

Justin Skycak (00:32) to play Bad Cop now, you’re going to end up having to play it 100 % of the time months you’re just trying to do damage control on But if you just say, OK.

Jason Roberts (00:36) That’s right.

100 % sure.

Justin Skycak (00:47) going to be bad cop in those moments when, uh-oh kid has not done their homework. Like give them a stern, you know, like, this is a serious thing. I don’t want to see this happen again.

It doesn’t have to be like incredibly painful experience for the kid. It just has to be like, oh, uh-oh, I’m on the teacher’s bad side for these five minutes. yeah.

Jason Roberts (01:05) No, you don’t You don’t have to raise your voice. You just say,

this is not I expect more out of you.

You’d much rather be teachers, be a little bit intimidated by you than not respect you. If they don’t respect you, and then you try tighten the reins, then they hate you. They resent you and they hate And that is a just, like you said, it’s an unsolvable problem. The year is kind of lost.

An ounce of prevention is worth a pound of cure

Justin Skycak (00:00) What happens if you’re if you just play cop like all the time 100 % of the time, like, gonna be slightly off day, they’re drifting off course, and they’re gonna end up miles away. You don’t have to play bad cop all the time. But it’s just whenever there is a slight deviation off of behavior that needs to happen, you have to call it out. Otherwise, it’s gonna compound into a problem.

If you refuse to play Bad Cop now, you’re going to end up having to play it 100 % of the time months later. And it’s like, you’re just trying to do damage control on You’re going to lose. It’s just a matter of how bad do you lose. You’re trying to limit your losses. But if you just say, OK.

5 % of the time, I’m going to be bad cop in those moments when, uh-oh kid has not done their homework. Like give them a stern, you know, like, this is a serious thing. I don’t want to see this happen again.

Yeah, and doesn’t take a lot. It doesn’t have to be like incredibly painful experience for the kid. It just has to be like, oh, uh-oh, I’m on the teacher’s bad side for these five minutes.

Effective teachers have to be a little intimidating

Jason Roberts (00:00) You kind of have to do that as a teacher. You kind of have to scare the kids, to the, you don’t mess with this teacher. You do a good job and otherwise it’s gonna be a horrible experience.

Unfortunately, any teacher thinks you can come and say, well, I don’t do that. I just I’m just happy all the time. I like, guarantee you that kids are not doing anything close to what they’re supposed to be doing.

You’d much rather be teachers, be a little bit intimidated by you than not respect you. If they don’t respect you, and then you try tighten the reins, then they hate you. They resent you and they hate And that is a just, an unsolvable problem. The year is kind of lost.

Short problems focus your learning on the specific components that need to be trained up

Justin Skycak (00:00) Why are our a minute, two minutes?

why do we scope them down so much?

Jason Roberts (00:05) If I have a math problem, takes 10 minutes or 20 minutes, then how many reps can I get? And

in any particular problem, there are very specific components, really want them focused in on. There’s other stuff in there that’s like lot of arithmetic and stuff. Well, this turns into, okay, well, after I do this integration or differentiation, I get all this fraction arithmetic and I gotta, you know, it’s like, I know you know fractions. Like, I don’t want you spending seven minutes on the arithmetic.

Justin Skycak (00:33) Yeah, we need to concentrate your practice on the stuff that is new, that you’re struggling most with, that you don’t know yet. that out of the skill. I mean, we’ll practice putting it back in, doing the whole skill altogether. And yes, you do need to keep doing stuff with fractions, but that’ll happen naturally. We don’t need to go crazy on it every single time. We need to really

down new movement that we’re trying to get you to learn.

Designing short problems to get more reps on targeted skills

Jason Roberts (00:00) How can we frame these problems so that What we’re asking for is I don’t need you to complete the entire computation. That’s going to take

10 minutes

Justin Skycak (00:09) Yeah, not only would it take you 10 minutes, but there’s a larger likelihood that you’re gonna, like, you’re gonna fall off the rail somewhere in there. Especially if this is your first encounter.

It’s better to just like, let’s just start by focusing on the new thing that you’re most likely to mess up. Let’s get you strong on that. Maybe you get the first problem wrong and then that draws your attention to like, oh, I get it. I should have done this. And then only one minute has passed. Now you fix that on the next minute. Two minutes have passed. You get a few reps doing that correctly. And now we do that again on another part of

of the problem and now after that we kind of pull it all together in the longer.

Jason Roberts (00:46) Right, you’re

making them do the ugly of the time, but on different Sometimes you have to do the finishing part, sometimes you do the setup part, sometimes whatever. But going to focus our the part that really matters, the tricky, the new part, the part that the student doesn’t get.

Justin Skycak (00:53) Yeah.

The front and the back end need to talk to each other from day 1

Jason Roberts (00:00) So my very first startup, and I was like 20, you know, I started a year after I graduated high school and I did it with my good friend Phil, who we were math majors at University of Chicago.

And I’m the whole engine, elaborate financial simulation engine that simulated equities and futures and derivatives behaviors, and all this stuff’s really complex.

You know, we’d sit down and we’d work next to each other talking every day and he’s building the grizzle stuff.

And this is back before the day we even had like a ⁓ network. We had like a three point inch floppy disk that we would fling across the room to back and forth, to each other like a frisbee.

And then it was this whole thing that took us, I think it took a few days of kind of like figuring out how to get all his UI talk to my engine we were so green.

And then we learned the hard way: the front and the back end need to be talking to each other from the day one. You need to be like writing a system and getting everything to compile and in shaping and melding

the front and the back ends to work together from day one in defining a user and interface. This is how you put information. It’s like information out, know, this is what it looks like.

And we were working on, think it months. And that was it was all just mismatch. it was a hard thing to overcome.

Small impedance mismatches compound into chasms

Jason Roberts (00:00) And I this just happens with human nature. If you don’t get systems to integrate from day one, and so it kind of has built-in accountability.

So I’m doing this. You’re expecting that. Let’s see if we’re on the same page. Teacher and student, two courses, mowed together back end front end.

Justin Skycak (00:12) you

Yeah, because otherwise, impedance mismatch, these defects just compound over and over and over until you look up and you expect to be able to reach out and grab the other thing and it turns out you’re like 100 miles.

Jason Roberts (00:26) That’s right.

That’s right.

College math courses give you a few very difficult problems and expect you to teach yourself the intermediate steps

Justin Skycak (00:00) in college, real analysis.

this is typically how it goes, you get like, 5, 10 problems on a weekly set or something. And these problems are all like really advanced. Like you’re not getting reps on standard definitions, proving that something simple is a closed set. No, we’re gonna give you some really

crazy situations, And you’re supposed to infer that it’s a closed set or an open set or whatever. and because of that, you’re also supposed to infer that, well, like you can use these theorems for closed or open sets I mean, yeah, you gotta build up to that, of course, but

as your first like, no, like just strike the ball. Like we’re just you just got to kick the ball first. That’s what we’re going to that’s what we should do first. But that’s often not what happens.

happens in that sort of situation, the students who succeed in that this in a better practice conditions elsewhere, or they have figured out how to kind of scaffold themselves through reps that need to happen.

It’s just, everyone just assumes like, well, of course, like students should know

how to scaffold themselves up to this level. And if they don’t, then they’re just lazy, which is not true.

Proof methods are not explicitly taught in college

Justin Skycak (00:00) I did with some of her math in college, real analysis.

something that was not explicitly taught

in the class, but is like the number one technique for all sorts of proofs: you’ve got these two ends of the rope. The closer you can move these two ends to each other, the easier time you’re going to have having the insight of how to bridge the gap.

if you don’t see how to approach it immediately, then why don’t we just take the input assumptions and are there any theorems that we can apply to them to sort of get in the direction of the output result? And then you start at the opposite end. Here’s the output result. Let’s work backwards from that.

and you just keep doing this until you complete it. Anyway, so this was like one of those meta principles

With the types of proofs that she was doing, that you really have to be good at this. But it’s something, it was not scaffolded up and it was not, this meta-heuristic was not taught explicitly.

These were just like these kind of golden prerequisites.

She didn’t have those prerequisites in place and that was causing some struggle. Of course, no, of no fault of her own. She was just, this was never taught explicitly. And you don’t really figure this out much. And unless you’re really like, you just look into this stuff, solve problems for fun. Like you just, you learn, you pick it up elsewhere.

And but once we filled those in, it’s like, well, okay, she doesn’t need my help anymore. Like she’s, she’s good.

Making sure students are prepared for proof-based math courses

Jason Roberts (00:00) It’s almost like in terms of accountability, either one, you’d have to have like someone senior in the math department really defining

done in the course, really emphasizing these are the specific things that they need to be able to do by the end of the quarter. Because when they start real analysis or abstract algebra, they’re going to have to do proof by induction and proof by contradiction, and they’re going have to do proofs with sets, and they have to do all these things. And so that the person teaching the course say, okay, can they do these things at the end of the course?

Right? And another way to do it is say, well,

anybody who’s teaching real analysis or abstract algebra, they have to teach methods of proofs the quarter before. So you gotta eat your own dog food. So if these kids come in to real analysis and they can’t prove, you know, statements of open and closed sets because you didn’t properly scaffold them the previous quarter with, you know, the techniques,

that’s how you’re going to suffer. And then you’re have a watered down course or frustrated course, or you have this massive curve where everybody gets like a 30 % on the midterm and final because nobody knows anything. And you’re going pretend that a 37 % is an

Founders have to get into the weeds to successfully scale their companies

Jason Roberts (00:00) There’s a really good article came out like a year ago. There was a talk given at startup school by ⁓ Brian Chesky, who’s one the founders of Airbnb called “Founder Mode,”

The gist of it was a lot of well-meaning people would tell young founders who have growing scaling companies, look, the way you scale a company beyond 5, 10, 15, 20 people is you can hire great people, empower them.

give them the direction, give them the resource and get out of way. and then Brian is like that almost killed Airbnb, almost utterly destroyed All these other founders learned the same lesson. They got all the same advice from all these hire great people, give them direction, out of the way, don’t micromanage, and then you’re good.

And it was a failure and they all had to learn the same lesson that Brian learned is have to get into the details. You got to roll your sleeves up You don’t just have your direct reports. You got to meet down with the people in the engineering part of the marketing department, the sales, the will fulfill whatever. And you got to get in there and you needed to be seeing what they’re doing. You need to be talking to You don’t just wait till the end product and go yes or no.

Founders need to be involved in every step from the beginning

Justin Skycak (00:00) If the founder comes in and has

on top of like what’s happening and actually is invested in the details and they are giving feedback on that, like this is all good. The failure mode that happens is when the founder has gone out of touch for a long time, and lot of momentum has been built into building something that is not aligned with what the founder wants to see.

Or the founder has not been keeping on top of the details, doesn’t really understand the process. And there’s a communication barrier. Like the, the maker is, is like building their thing. And then the founder comes in and then says some stuff and the maker’s like, ⁓ they don’t really understand like how difficult this is or like what that’s what they ask for is impossible. Like just, you let it get so out of touch cause you’re not syncing up

then yeah, it can spiral out of control. But if you just start from the beginning, just tight together, that’s the way.

The same advice doesn't work for both big and small companies

Jason Roberts (00:00) There’s a really good article came out like a year ago. There was a talk given at startup school by ⁓ Brian Chesky, who’s one the founders of Airbnb called “Founder Mode,” which Paul Graham then wrote a you know, widely circulated essay by the same name “Founder Mode.”

The gist of it was a lot of well-meaning would tell young founders who have growing scaling companies, look, the way you scale a company beyond just 10 or 5, 10, 15, 20 people is you can hire great people, empower them.

give them the direction, give them the resource and get out of way.

And Paul Graham’s his takeaway was that the advice everybody was giving really applied to the professional managers. When you have a skilled company, the founder’s gone and we hired some MBA type CEO takes over and he hires people who are supposedly experts in their area and they do it.

Also, a lot of these people in the executive are just really good at bullshit. But these companies are often so big and successful just because they have so much momentum and scale that they’re going to almost going to succeed no matter what. smaller scaling companies can just totally run off the

Steve Jobs & Johnny Ive's relationship

Jason Roberts (00:00) Jony Ive, who was the renowned industrial designer for So if you think of all the cool iPads, iPhones, MacBooks,

Airs, anything you can think of, he designed. don’t think Jony Ive would ever have accused Steve Jobs of micromanaging.

He would stop by Jony Ive’s workshop like pretty much every single day. What’s going on? Just because he loved it, loved the design process. He loved talking

Jony, like what he’s thinking, what are the problems he’s trying to solve? What are some models? You know, sometimes he’d have, they’d have stuff to show off and sometimes they wouldn’t. They would just talk and they would just walk. so Steve was to the design process, which is why in part, this stuff was so great. I mean, yeah, you had brilliant.

designer in Jony Ive, but

Steve was deeply in a very positive, relationship with Jony Ive.

In highly coordinated teams, the error is usually under communication

Jason Roberts (00:00) Whenever you processes that need to be in sync, and it could be just two human beings working on something, it could be two departments, could be backend front end developers, could be whatever it is. Whenever those things are supposed to be in sync,

the error is rarely over communication. The error is usually under communication.

Early frequent communication with students and parents makes resolving conflicts easier

Jason Roberts (00:00) one thing I used to say to you when you first were teaching math academy. say communicate with parents early and often. Get a positive line of communication. Tell them your plans for the year. Tell them you’re excited. Tell them this is gonna be a great year. then,

First couple things that students do well, immediately tell the parents that they’ve done a great job, a great start, I’m really excited. think she’s gonna do a great year, I think she has a lot of thing. And as soon as these things start to go off the rails, say, know, I’m a little concerned, she hasn’t done homework the last couple of days, and then she’s not focused on classes, and the parents are synced up. You guys are synced up. But if you wait, and you end up a C minus at the end of the quarter, she’s like, oh my God, what happened? Oh my God, you know. It’s disaster.

Justin Skycak (00:36) Yeah, that’s right.

Yeah, right. Because like, building some kind of like positive relationship with the parents that you can break bad news later on if it happens and have them on your side, they trust you and they’re going to try and help rectify the situation. And also, even if it’s bad news, like pretty much all the way out, like, at least they hear about it early. And and it’s not like exactly. Yeah.

Jason Roberts (00:58) You have a paper trail. Like I’ve been telling you, we’ve been over this every day of the day, not doing the homework,

not trying, not focusing. And so when it goes bad and you have to talk to the principal like, look, I don’t think this is the right class for her because she’s not doing the work and she’s not trying. that’s where they go, yeah, let’s find a better

placement where the student can be successful.

Jason and Justin talk multiple times every day to keep things on the rails

Jason Roberts (00:00) I think why think we’ve been successful at what we’ve been trying to do is that you and I talk.

I mean, I, I mean, we.

Justin Skycak (00:06) Yeah, every

day, sometimes twice a day, yeah.

Jason Roberts (00:10) Least. I think on average we talk twice a Probably usually there’s a Zoom call when we, I’ll be working on something and be like, hey, can we jump on a call? Let’s talk about this, how about that? And lot of times I’ll call you when I’m on my drive to lunch. And I’ll call you on walking the If I didn’t call you while I was walking my dog, which is like 5 o’clock, I’ll call you when I head to the

I would say two to two and a half times a day would be the average. And there’s probably some times just like, well, you know, maybe you guys are wasting time. But.

Justin Skycak (00:38) On average, those pay off though, big time. It’s like the repeated, whenever there’s a problem or a question that we don’t really understand or have a satisfying answer to, it’s like a spaced repetition sort of thing. It’s like, let’s just think about it a bit. You ask a question today, we kind of brainstorm a little bit, just talk about it a little bit. That’s consolidated overnight, day later, ask the same question.

Jason Roberts (00:51) Mm-hmm.

Justin Skycak (01:04) We are not there to a solution yet, but we’ve gotten further than the previous discussion. Eventually, no matter what the hard problem is, eventually it kind of cracks and we figure out some way to it.

Jason Roberts (01:14) we see our way to a.

Math Academy is a globally distributed company

Jason Roberts (00:00) So what’s interesting about this whole syncing up is like, we are a fully distributive You’re in Boston, Sandy and I are in Pasadena, Alex is in the UK, then the rest of the content team is spread throughout the world.

But we’ve learned to be very, I think, efficient and effective with this model. think it’s funny, they talk that struggle with this. Either you’re fully distributed or you’re It’s tough if you a hub of people who are working together and then other people are kind of off on their

because there’s a lot of communication that happens between these people that’s not written down and therefore it’s not easily shareable.

The importance of letting conversations flow out of scope

Justin Skycak (00:00) One thing in particular that works out well for us is allowing conversations to flow out of scope. Actually I have an uncle, who

works for a big company and his team was remote for a while and it was not working so good. And he’s trying to get people back into the office. But one of the problems he cited is that there’s just like a lack of like spontaneous idea generation, where you just like talk to your at the other desk about some thing that just crossed your mind. Then you come up with an idea, you know. But what we

do a lot is we allow to flow out of scope. We don’t put a scope on the conversation.

you haven’t allowed conversations to flow out of scope before, then you kind of feel like, what are we talking about? What’s to do list, check everything off like, like, when you focus your thoughts too much,

you lose all the kind of background brainstorming that you were doing. It’s like the balloons kind of fly away.

within an organization, there are these kind of like spontaneous thoughts or actions or problems or noticing things. that’s weird. wonder why like just like that, not drilled down fully, but they’re just kind of happening in the background. And

if it comes up in a conversation and you drill down deep into it, you you come away with something valuable. But if every conversation is scoped down really hard, every meeting has has a name on it, this is the meeting to discuss thing x, y, z. We’re not discussing C, then you run

into a problem you’re just exploiting what you already know how to do and you’re not really exploring at all the

If you like the work and who you're working with, you're set up to succeed

Jason Roberts (00:00) A couple of key characteristics that make it work is, one, you gotta like the person you’re talking to.

If you’re not speaking with the other person, then you’re going to scope it down because you want to get it done and get it out of here.

you know, my boss wants to go over this stuff for me or once we have a team, three people, we’ll be like, okay, another meeting, right? And we’re just gonna grind through this stuff. It’s just, it’s not fun. Nobody really wants to be there. Some people may not hate it, but most people it’s not like on their top 10 list of things they’d be like to be doing.

That’s why I think, know, they talk about culture. What’s the culture of your company? I still don’t understand what half that means, and I don’t know if most people do, but it’s like, at the end of day, with people who you really

and if you do that and everybody’s really excited about what they’re working on, and you could you’re you’re setup to succeed.

Jason thinks better when he talks

Jason Roberts (00:00) I think better when I’m talking things out loud. you know, it’s like, well, you need to write to know what you think. It’s like, I kinda need to talk to know what I think. And it’s kinda like my mental whiteboard is just talking. And so like when you and I talk, can kind of, I can be like, okay, so, okay, so let’s, this, this, this,

Let’s think about this again. Okay, so this happens, right? And then, okay, then it’s like I’m writing it down to whiteboard, okay, right, and then this, and then this, and you’ll go, well, actually, da, da, da, okay, right, right, right, okay, right, right, then scratch it up with that. So we’re kind of like, we kind of have this shared whiteboard that’s part of our conversation that allows me to kind of think through things more, you know? And I think that helps. So.

For me personally, the conversation is an aid in clarifying my thinking. I think also it’s just sort of the, you know, we’ve kind of set the habit of just the norm of having lots of conversations. So it’s like, okay, Jason’s gonna call me and as long as, you know, I can get off for dinner or whatever, then you know what?

whatever and then we do and a lot of times we solve really good problems and come to lots more concrete solutions better understanding things sometimes it’s it’s it’s there are some times where it’s not as productive but it’s never really painful and if I feel like we’re not really getting anywhere I’m like okay well I think that’s it then I’ll just kind of like let it go I’m like all right well I’ll let you get back to it

You need much more context than you think to solve problems

Justin Skycak (00:00) One of the things we have found and over that’s incredibly important to getting good results out of anything, whether it’s like somebody who’s working on something or even an LLM that’s trying to do something for you is context. If you’re scoping down your conversations all the only giving somebody the context that you think they need solve the

the thing that you’re asking them to do, which is often not nearly enough context. We’ve learned this over and over, right? ask an LLM to do something or even you ask another person to do something and you think that you’ve given them of background information, but once you really dig down deeply into it, maybe they come back with their first attempt and then it just dawns on you that like, my God,

there’s a lot of foundational things that I forgot to say that they’re not spun up on. You have that happen enough you just kind of like come to accept you have a very tight loop with somebody and you are constantly feeding them just overall context, even if it doesn’t seem like a hundred percent the nose relevant to the thing they’re doing at the moment.

they need that. the only way that they’re going to have enough background context to pull from when needed.

Knowledge is developed over many conversations with lots of back and forth

Jason Roberts (00:00) The English language is so general and

which is why, like, when you see on comment thread, anywhere on the internet, everybody’s arguing about all this stuff. And I think a large percentage of the things they’re arguing about, if they were sitting in the same room they may be arguing because they could go back and forth clarifying really what they’re talking about.

Justin Skycak (00:21) Mm-hmm.

And another thing is it’s like, It’s not just one conversation of a context dump. Like this conversations because Often the context is, it’s so big. It does not fit in your working memory. You have to spread it out. You have to consolidate context and like to make room for more. I mean, that’s Even in LLM usage. That’s like one of the big frustrations, right? Is that everyone knows context is important.

And LLMs now have, have larger context windows than they used to, but it’s still not, to the amount of context that is often required to really perform a task to the level that you need. Like the context window is still tiny, a thousand tokens, a million tokens. Like that’s still I don’t know, Maybe you can dump of a book in there, but like you really need like a library’s worth of information to complete the task.

I mean, humans do this better. Like they can actually consolidate all this into long-term memory right? And yeah.

Jason Roberts (01:18) Yeah, because we have a trial and error ideally a little piece from this book, a little piece from that book

and back and forth and it’s just this constant feedback and error correction about what we’re dealing with. So we can pull context from lots as opposed to just saying, I’m going to try ahead of time to get everything you need to know and put it in here

Both humans and LLMs need tons of context to understand a problem

Jason Roberts (00:00) People really underestimate the amount of context that they relative to what little bit that they’re sharing. Cause you’d be saying things to somebody, but you’re selecting the context that you think is relevant. And a lot of your assumptions are They need more. And they may understand the words that are coming out of your and may they have some understanding of it, but it’s still.

doesn’t have the level of concreteness or connection to other things or whatever to really work for them. And so then they’re kind of cargo-culting in a way, right? They’re kind of, you said these things, I’m doing these things, I don’t really know the why and where this came from in the history, so I don’t have this intuitive sense of all these things. So I’m just gonna go off what these words mean in English or something.

Justin Skycak (00:47) often not enough information, right?

Jason Roberts (00:49) Well,

I think when you were doing some experiments with some LLM you kept going and have to put more and more and more and more context and really fine And I mean, like this, like, oh, I wrote like a one second, one or two sentence prompt. And then it turned out I went to five and 10. It’s like, you went 20 pages of detail.

Justin Skycak (01:08) Yeah, I broke it down into like,

originally I thought this was just a, basically one paragraph worth of information that I have to give. And then it turns out after I spent a couple of days on it, just like drilling down on what are its failure modes, try to scaffold it better through the process. It turns into this five step thing and each step is itself like some paragraph of information. And I have to give, like, I have to enumerate all the, the structure of each step and like.

For instance, this qualifies as blah, blah. This other thing does not. have to give some examples of that because it’s not enough Often you think of words in a slightly different way than they might be used just generally like the dictionary definition.

AI models can only operate like humans if they have all the information that humans have

Jason Roberts (00:00) Friend of mine from college, really, really smart guy.

said, one thing he said to when you’re building a machine learning system, the machine learning algorithm has to have all the information that a human has about that problem.

if you withhold anything, don’t expect it to operate at the level that a human can operate. And so when you’re dealing with a machine learning very quantitative. concrete thing, it’s a little easier to figure out what the human knows or doesn’t know when you’re talking about like an LLM type of situation where it’s much more nebulous. It’s really hard to figure out

what the human knows that may be relevant. And you try and make some hard cuts, maybe I’ll include this or maybe I won’t. And then it turns out later a lot of information was important that the know. And the same goes to talk with humans.

Context problem for hiring someone to do the videos

Jason Roberts (00:00) We’ve been recently trying to hire someone for a position and we interviewed tons of people and.

is very clear that a lot of them lacked sufficient context in a lot of areas.

Justin Skycak (00:18) It’s like

you can talk to them at a high level about these things. You can get faked out because they’re, I don’t know if they’re cargo culting or they’re talking from their own experience, which kind of maps to similar high level principles, but doesn’t actually come back down into our specific context. ⁓ Yeah, but you run into a similar problem of you’re conversing in abstraction land.

And then it turns out there’s a lot of information in the fine details of things that you only get really by doing the reps.

Jason Roberts (00:55) And

Insufficient context leads to misunderstandings

Jason Roberts (00:00) The English language is so general and

which is why, like, when you see on comment thread, anywhere on the internet, everybody’s arguing about all this stuff. And I think a large percentage of the things they’re arguing about, if they were sitting in the same room they may be arguing because they could go back and forth clarifying really what they’re talking about.

They’re not synced up. So if you can’t have this real time inter thing you’re saying something, but I can kind of see the tone of your voice, your face, like concrete this thing is you’re saying, or where it’s just kind of off the cuff or you’re like, this is a definitive statement. You gotta share that information, share that context so much.

LLMs' context windows are still not enough for complex tasks

Justin Skycak (00:00) Often the context is, it’s so big. It does not fit in your working memory. Even in LLM usage. That’s like one of the big frustrations, right? Is that everyone knows context is important.

And LLMs now have, have larger context windows than they used to, but it’s still not, to the amount of context that is often required to really perform a task to the level that you need. Like the context window is still tiny, a thousand tokens, a million tokens. Maybe you can of a book there, but like you really need like a library’s worth of information to complete the task.

I mean, humans do this better. Like they can actually consolidate all this into long-term memory

to make room for more.

Don't scope down conversations to allow for spontaneous idea generation

Justin Skycak (00:00) One thing in particular that works out well for us is allowing conversations to flow out of scope. Actually I have an uncle, who

works for a big company and his team was remote for a while and it was not working so good. And he’s trying to get people back into the office. But one of the problems he cited is that there’s just like a lack of like spontaneous idea generation, where you just like talk to your at the other desk about some thing that just crossed your mind. Then you come up with an idea, you know. But what we

do a lot is we allow to flow out of scope. We don’t put a scope on the conversation.

Letting conversations flow out of scope allows you to explore more of the space of possibilities

Justin Skycak (00:00) what we

do a lot is we allow to flow out of scope. We don’t put a scope on the conversation.

you haven’t allowed conversations to flow out of scope before, then you kind of feel like, what are we talking about? What’s to do list, check everything off like, like, when you focus your thoughts too much,

you lose all the kind of background brainstorming that you were doing. It’s like the balloons kind of fly away.

within an organization, there are these spontaneous or actions or problems or noticing things. that’s weird. wonder why like just like that, not drilled down fully, but they’re just kind of happening in the background. And

if it comes up in a conversation and you drill down deep into it, away with something valuable. But if every conversation is really hard, meeting has a name on it, this is the meeting to discuss thing x, y, z. We’re not discussing C, then you run

into a problem you’re just exploiting what you already know how to do and you’re not really exploring all the

Integrating course phases into the system

Justin Skycak (00:00) We got to integrate course phases. the instruction phase where you’re doing like your lessons and reviews and everything, you’re going through the course. But if you’re taking a final exam, which is something that we’re starting to integrate into the system more,

then you have a pure review period before the final exam. then you take the final. then you get below, say, like 90 % on it, after you do your remediation based on the things that you missed, you’re going to take the final again.

I mean, previously we just had this concept of like, oh hey, a hundred percent means you completed the course. Right. But now there’s really like, well, that means you completed the instruction phase of the course. There’s like, gradations of what it means to complete a course now.

Aptitude and conscientiousness determine a student's accuracy rate

Justin Skycak (00:00) are students have astronomically high accuracy. Maybe their accuracy is 80%, that’s solid. But you can have students like that who pass % of their tasks.

what that indicates is that average an aptitude level, but in terms of diligence, conscientiousness, doing what they’re supposed to, aligning themselves with the learning process. They’re really good. you can also have students % accuracy, but their pass rate

tasks is maybe like % or 90%. These are kind of like of the knucklehead smart really fast on the things that come easy to them and a lot comes easy to them. But then once they struggle, rage quit or just like fall off the rails,

It’s like one of those quadrant memes, like aptitude and like conscientiousness.

Working memory capacity limits learning speed along a single learning path

Jason Roberts (00:00) If B depends on A and C depends on B and D depends on how far along the path can you go in a single day? thing of it is that your memories consolidate when you’re sleeping. And until you’re sleeping, when you just kind of learn something that day, it’s not in your long-term memory.

new information is moved to long-term memory, then it does not take slots in your working memory. And everybody has a very number of slots in the memory, between four and seven for most people.

If you’re allowed to do three or four copies in a single day and you’re if they have enough working memory, they’re high enough aptitude, then maybe they can, the skills that they kind of learned in A and B and C earlier in the day, they have slots in their working memory.

They can still do D. A lot more people will be like, can’t remember. They need to get those long-term memories. This needs to solidify over two or three days so that it doesn’t take up working memory. So their working memory is entirely for the new skills in topic D and the intermediate results of the problem, things like that.

The Math Academy system holds you accountable for learning every skill

Jason Roberts (00:00) that’s what we try and do in math: the Math Academy system will not allow the student to until they’ve mastered the material. And you got to have a jump shot and you have to go dribble left and dribble right and take left hand layups and right hand. You know, you got to do everything. You know, you don’t get that you don’t get the cheat and make

you know, bullshit compensations to fake your way through. It’s like, you can’t do graphs, so you’re just really good at algebra and you kind of come up with some solution when anyone who’s remotely competent at this stuff could just look at the graph and to tell you that’s where the, you know, the singularity is,

You can't avoid learning how to complete the square

Justin Skycak (00:00) The concrete example comes to my mind of a compensatory skill in math would be students who never really learned how to complete the square. They struggle with it when it comes back in calculus, when you’re doing

integrals that require completing square, particular, inverse trig function And so the completing the square is one of the techniques you learn in algebra. You quickly move on to just using the quadratic after

Any time students are asked to solve an equation by completing the square, oftentimes the problems are structured in a way so that ultimately just amounts to solving a quadratic equation. And the kids are always just like, I don’t like completing the square. I’m just going to do quadratic formula. And then they use that all the time.

But then in calculus, when it comes to using completing the square in calculus, this is not a thing that you can use the quadratic formula for. You are restructuring expression so that you can actually integrate it. You’re not solving the equation. And so there’s no way to do it other than to complete the square. And that causes a lot of struggle.

If you just use Desmos and never learn how to graph functions, it will limit your ability to solve complex problems

Justin Skycak (00:00) When classes allow to just use graph everything, students don’t actually the graphs of different functions look like. Their compensatory strategy for everything is just like plug it into Desmos, jig the parameters a bit until the graph looks like the desired.

But then they don’t actually understand geometrically what these transformations do. They have figured out how to use the resource provided to them obtain the answer of what is the effect of the geometric know this in their head what the geometric transformation is. But when you’re talking about like advanced

reasoning about and stuff that pull that in as a component skill, you need to be able to do this the fly in your head quickly as part of a compound thinking move.

Why you need to know your multiplication facts

Justin Skycak (00:00) Why do you have to learn your multiplication facts? Well, for instance, when you’re factoring a quadratic equation, you need to be able to invert that process. And if you know is how to like multiplication by repeatedly adding a thing, you can’t easily invert. Like given the result of a multiplication, what are the factors that went into it? You don’t know it, so you can’t invert it. So it it’s no longer a tool in your

Failure to do the work now will close of future opportunities

Jason Roberts (00:00) The development of compensations is often a lack of oversight from the adults, parents or and a lack of willingness to do the overcome that weakness. They’re just like, oh, I just let them use Desmos. Oh, I just let them use a calculator. Oh, I don’t really give them tests because it’s hard. Oh, know, it’s kind a pain from the adults.

Whether they’re in college and doing real analysis or they’re in third grade learning their multiplication tables, you know, learning is effortful. It doesn’t have to be painful, but it’s effortful and it takes some time. And sometimes it’s not always what you wanted to do. Like I didn’t want to work on my jump shot. Nobody made me work on my jump shot. So guess what? I never got a good enough three point jump shot. So therefore my college basketball career

before it started. That’s the result. That’s how it ends. And so you might as well do the hard work now, do it earlier. And it, it opens up your game.

Now you can do all this Otherwise you start prematurely shutting down future avenues of growth, future opportunities. It just shuts it down. You don’t learn math, well guess what science is shut off to you, technology. Lot of technology jobs certainly would be shut off.

The Math Academy system doesn't get tired of training difficult skills

Jason Roberts (00:00) The Math Academy system it’s not a human, it doesn’t get bored. Math Academy’s like, we can do this all day. Like, we’re gonna work on your jump shot until you’ve got a killer jump And trust me, when you have that killer jump shot, you’re gonna thank me.

Trust me, when know your multiplication tables, you’re going to thank me. You may thank me now, but you’ll be like, thank God can do this.

Missing foundational skills is the drag that stops mathematical progress

Justin Skycak (00:00) We hear about this all the time from parents who have seen their kid grow so confident filling in their foundational skills, right? Like it’s like this level of

in their mathematical ability that they never even expected or thought was possible to have. It’s like, where did that come from? My kid is actually good, crushing their math class. It’s like, well, they just filled in their

skills, they’re actually prepared for it now.

Jason Roberts (00:26) So much,

so much, because that’s the biggest drag on them is those missing foundational skills. It’s the drag. It’s what keeps you from making forward progress. You accumulate too many missing and all progress pretty much stops. And then you have to resort to faking, getting all kind of extra help,

trying to make up for a really lagging grade with extra credit and other

Assessment vs. non-assessment accuracies

Justin Skycak (00:00) So the accuracy metric, we actually split that out into two accuracies that we’re keeping track of. There’s their assessment accuracy and there’s non-assessment accuracy. Assessment accuracy is under timed conditions, you don’t have reference material. It’s a, it’s a harder situation.

We measure those separately and then average them together. If you were to measure a flat out accuracy from all their they’re submitting fewer quiz answers than lesson answers and stuff. you would get overwhelmed by the non-assessment accuracy.

addition to having an aggregate, general across all topics we also measure student on every topic individually and have that propagate through the graph to get an accuracy estimate. Cause sometimes it happens that you have a student of the really

kids in the class, but for whatever reason, there’s this one, this one thing that they are struggling with a little bit. And they’re more like an average student in that category. And then you got to slow it down a little

Accuracy rate is the best metric of student performance and tailors the system to them

Justin Skycak (00:00) A good proxy for well is a student able to stomach load is just how well are they doing on the system in general?

The number one metric turns out to be just what is the student’s accuracy on the system? That more of the signal than even their pass rate because you can pass a lesson get like of it right. Versus pass a lesson and get like 100 % of it right.

And there’s a difference between that.

Whiz kid

is at the top of the accuracy list. don’t need like or four each question per knowledge point. pretty much one shot it. We can give a second one just to be really sure that they didn’t just get the first one chance. But yeah,

learning rate incredibly fast. On the other hand, the students who more or need more repetitions, also the students who are more forgetful of stuff, I mean, that shows up in their accuracy. Because of all these factors, they’re less to get a question right.

We kind of take this into account in various places in the system. It’s part of the glove fit on a student.

Spreading out topics along different learning paths reduces cognitive load

Justin Skycak (00:00) You got to the cognitive load by allowing it to consolidate more. In particular, in skill hierarchies because when you are climbing the skill you’re pulling context from the lower topics as well. If those are not consolidated, then it’s going to be eating up more of your working memory capacity, likely to put you in a state of cognitive overload.

safer if you spread things out, you’re not building a ton same path in a single day. Instead of doing like five topics that stack on top of each other getting that whole tower cognitive load, you spread that out horizontally along the graph, each topic

its prerequisites are much better consolidated in your memory. So it’s a much lower level of cognitive load.

If you're honest with kids early, there's time to fix things

Jason Roberts (00:00) The whole thing is a disaster by adults that they were being when all they were doing is allowing people to lie and fake and cheat or

negotiate or lobby or whatever, any kind of compensation to overcome the fact that people aren’t doing the work and learning the material, learning the core skills that they need to do whatever it is they want to do.

Justin Skycak (00:22) The situation turns into too afraid to tell kids that they’re not on track for their goals. so you just keep telling them that they are everything’s fine. well, you just keep saying, everything’s fine. Everything’s fine. If you had told them least they could have rectified the situation. There’s time to fix this

Jason Roberts (00:31) And there are the parents. Are there

Justin Skycak (00:42) and kind of change the future trajectory. keep the smoke screen up long enough and it just gets to a point where it’s kind of locked in. It’s too pain. Yeah.

Jason Roberts (00:52) It’s an unfixable problem. It’s an unsolvable

problem.

Justin Skycak (00:54) so whatever happens, there’s a cost to be paid and somebody’s going to pay it.

Jason Roberts (00:57) There is a, someone’s going to build, the

check’s going to come due and it’s probably going to be the student is going to pay the biggest because they can’t do these things.

The mathematical rot starts early and goes all the way up

Jason Roberts (00:00) The rot

can start early and start in elementary middle school. It’s like, we don’t teach multiplication tables. And because they don’t know multiplication, the kids are struggling with algebra. So we have very watered down algebra. And we let them use Desmos and calculators and stuff because they can’t actually graph things. They can’t actually solve things. So that stuff’s weak.

And so then the rot goes all the way

And so you just got to nip this stuff in the bud. And you got to hold standards early. You got to fix core skills early. You can’t allow students to progress up the ladder and carry along all of these weaknesses and then come up with all kind of

ways to put up smoke screens and hide it by not giving standardized tests and giving inflated grades and glowing recommendations and whatever nonsense you’re doing.

These are adults created this problem. You know, it’s no one person’s fault. It’s not the teacher’s fault. It’s not the school administrators. It’s not the school board. It’s not the parents. It’s not the politicians. It’s everybody. We all created this together. sometimes

You know, you just have to look at a system and say, OK, this thing is broken. There’s rot and we got to dig out the rot and we got to start fixing these things and so that we have a healthy educational system so students able to be successful and learn that they want to learn and be able to do things they want to do and become the kind of people live the lives they want to lead. That starts early and you got to

stay on it their whole career.

PODCAST 5

When to take notes

Justin Skycak (00:00) don’t take notes when you’re learning. no point, just work exercises. the nuance here is that when there is information that’s not part of reference material, you can’t go look it up after, you’re brainstorming something with someone or a thought occurs to you, and you don’t want to miss it, write it down.

Write it down, because otherwise you don’t have the reference material. There’s a big difference between these two things.

The maker and the manager: you need both

Jason Roberts (00:00) Paul Graham wrote great essay that the maker, has to just get in the zone and think really, really hard about something where it could take hours sometimes to really get into the zone. Whereas,

the manager is like the email and the phone call, the meeting, and let me look this up and check this and just lots and lots of little tasks, both of these things are important, right? In life

and in business, you don’t get to just like, I’m just going to build stuff and then I’m not going to do these other things. Cause you, you get nothing and you can’t, if you spend all your time, just set up meetings and coordinating and stuff like nothing serious gets done. it’s like, gotta have both.

Find your complement, someone who makes up for your weaknesses

Jason Roberts (00:00) It’s really important you find someone who’s like a complement so like you make up for each other’s weaknesses and shortcomings as opposed to you’re both like the same person. You have the same strength. That tends to not work. Right.

And I think that’s what makes really good relationships of any kind and marriage business partnerships, is that you have this complementary skill set and you appreciate what the other person can do. You’re like, thank God you’re doing this. Thank God. you know, and then you respect it and then you appreciate it and then you just like, I got this, you got that.

What happens when you get pulled out of deep work

Jason Roberts (00:00) Once I’m like in the deep work, I resent.

being pulled out of it I don’t want to do the meeting. I don’t want to do the paperwork. I don’t want to return these emails. And even other people I like, it’s interesting stuff. It’s like, I’m like, damn it, I’m in the zone, you know? And then it’s like, I can’t.

get myself to focus, I have like this ADHD and I can’t lock in on something. And it’s sort of like for me, it’s almost like a day thing. Once you get in a certain mode for a day, it can be really hard to switch. Now, sometimes maybe it’s like, well, you go to lunch and you have this like break in the day and maybe you can reset. And some people are probably better at this than others. I struggle with it.

Finding your team takes luck

Jason Roberts (00:00) any successful endeavor requires a great team of people. And getting the right

group of people who not only are individually highly capable, that you all kind of like each other, work together, and trust each other, and all that kind of stuff. Sometimes it just comes together and you’re like, damn, we just did it. You what I mean? You run into people who, their roommates from freshman year in college are their best friends for life. You’re like, that is just luck.

Sometimes the girl you date when you’re in eighth or ninth grade and you marry, she’s like the most amazing girl. was like, that is just luck, man.

EXPORTED (w revisions) - You can't give somebody responsibility without giving them control

Jason Roberts (00:00) You can’t give somebody responsibility for something without giving them control. Right? That’s not fair. And it’s just, it’s not reasonable.

Justin Skycak (00:09) you want somebody who you can hold responsible for the output of the team, right? And the only way to have that person is to give them control over the team that they build. There’s no way otherwise. That’s the only way.

Trying to solve tomorrow's problems today is usually a waste of time

Jason Roberts (00:00) You cannot solve tomorrow’s problems today because you don’t really understand what those problems are, and you don’t even know if they’re real.

It’s mostly a waste of time. I’m not gonna say you can’t think at all about what the future might look like, but you got to be careful not to get caught going down that rabbit hole and spending too much time thinking that you can make all these predictions because you’re probably going to be wrong.

Or at least you’re going to be wrong about the relative importance. You kind of like, okay, here are all these things.

that were interesting or could be important. Let’s figure out what’s really the most important one, and let’s just kind of do that, and then we’ll pick our head up and go, okay, based on what we’ve learned from doing that, now we can talk about the next thing.

We all want to avoid problems and catastrophe. You want to try and get ahead of some of these things, but you just got to be careful about overestimating your ability to really pinpoint what they are. Cause what’ll happen is you’ll over-engineer things.

You came up with this vastly complex solution to a non-problem, to an imaginary problem.

The importance of complementary skillsets

Jason Roberts (00:00) I can pretty good at compartmentalizing and what I mean is living in a state of denial about things that have to get done. And you know that fire is burning. And you’re just like, I’m just going to pretend it’s not burning.

You know, and you’re kind of thinking, you’re wondering to yourself, like, I wonder how long I can let this burn. it’s not great, right? I’m lucky in that I have Sandy, my wife and co-founder, because she is very operational.

just get shit done. like, all right, make a list, collect the data, execute good enough, done, bang, bang, bang.

She treats me a little bit like the mad scientist, like, OK, you go to lab. I’ll take care. I’ll run interference. But I will call you every once in while. And you’re going to have to do certain things.

she’s so good at that And I think that’s what makes really good relationships is that you have this complementary skill set and you appreciate what the other person can do. You’re like, thank God you’re doing this. Thank God.

EXPORTED (w revisions) - The meta-work is not the work

Jason Roberts (00:00) what are you doing? meta work like on a corporate environment where you cover your ass to show you’re doing stuff. Like, well, I’m writing these briefs I feel like there’s just a lot of brief writing.

Like, why do you spend half a day writing a fricking document?

I’ve always to bureaucracy and bullshit and meta work.

And it doesn’t impress me. I don’t give a shit. In fact, I’m annoyed.

The meta work just overwhelmed the process. And I think you see it a lot of companies.

The focus becomes that because

the politics and bureaucracy and it’s like nobody’s doing like one guy who’s actually doing anything. Everybody else is just

talking about who’s doing what and the plan and when this is going to do. meetings and approval And I can only talk about something for so long before we actually have to do it.

If you don't keep people focused on the thing they're supposed to be doing, there will be a compounding of misalignment

Justin Skycak (00:00) Everyone’s excited

but their excitement is leading them to go off in all these different directions. there’s the compounding of misalignment. And you gotta be constantly keeping people focused on the thing they’re supposed to be doing. And if you don’t do that, then somebody is one degree off course, two degrees off course, and you didn’t correct them

for the few days. And then you look up and you’re like, wait, where, where are you? Like, why are you all the way over there? It’s like, well, you let them go that far without pulling them back in.

Predicting start-up revenue early on is nonsense

Jason Roberts (00:00) you know, like when startups would go and try and raise venture capital and they have these pitch decks and they’d like in our growth chart and we’re going to do this. everybody knows this bullshit. The venture capital ists know it’s bullshit. You know, it’s bullshit. They kind of have to go through it just to, it’s just, that’s just part of the.

Justin Skycak (00:12) Yeah, the projecting revenue

like couple of years out from zero. We have zero right now. We’re going to pick up this many customers. Like, really? You can predict.

Jason Roberts (00:16) And we’re going to make a billion dollars.

Everybody’s like, great, okay, sounds

great. It’s like, is a big enough market, and then the profit margin and the cost of the, There’s big opportunity here. That’s all you need to say. But it’s nonsense because there’s just so much between that has to be done that’s just gonna dictate what’s gonna happen. You can’t make these predictions.

Keep it simple stupid

Justin Skycak (00:00) You do need to think hard about the stuff that you’re doing, but you don’t want to make it more complex. You want to think hard about how do I make it simple so that I’m not putting a straight jacket on

Jason Roberts (00:12) Buddy of mine in my first startup, used to say like, you better be careful what code you write because going to be supporting it for the rest of your life. That is going to be a thing that you have to deal with.

And if you’re solving problems that aren’t real problems, now you have all this extra code that you’ve got to lug around that are putting constraints on these other solutions. And now it takes five times as long to build. So there’s just so many reasons.

to try to avoid caught up building stuff that doesn’t really need to be built and just focus on the really critical stuff that you know how to solve now.

Everything takes longer than you think

Jason Roberts (00:00) my inability to really imagine all the conditional levels of complexity that are involved in this thing that has to get done.

And I know this about myself and I’m still always underestimating how long it takes to do stuff.

Justin Skycak (00:09) Yeah.

is kind of funny because,

on one hand, you wanna just focus your field of vision on what you’re doing right now. But then at the same time, when it comes to like, actually like, okay, realistically, how much work is this? How much is it gonna take?

The way I kind of think of it is like, you don’t always know exactly what are the things that you’re going to do after what you see in front of you, but Like historically, maybe what you see in front of you is maybe the first 30%, 40%, 50 % maybe. like, just put a, put an extra two X or.

2.5x factor on there, even though you don’t know exactly what it’s representing, it’s gonna be there in some way. You’ll figure out what it is later, but it is there.

Jason Roberts (00:45) In my case, buyback.

Entrepreneurs need to be self-delusional

Jason Roberts (00:00) Entrepreneurs have to be sort of self-deluding. It’s the only way that you can get started on an actual project because if you knew the actual, was really gonna take, you wouldn’t do it. It’s just, oh my God. You’re like, it’s gonna take me two months, three months, max,

and then you get going. And then, of course, it takes 18 months. In for a dime, in for a dollar, you made enough progress, you’re excited, okay. You need delusional people sometimes at the helm just to get everybody.

You distort reality around you because you believe so strongly that something is possible. And it’s not only possible, it’s possible in a relatively short period of time. And then you get everybody moving in the right direction. If you’re going to finish it, you’re going to have to do that. And then a lot of times, once you start making enough progress and learning enough, you get enough momentum and everybody’s, OK, yeah, yeah, we got this. It’s going to take more work, but we’re in it. So you’ve got to have

that level of self

More features means more customer problems

Jason Roberts (00:00) Every time you build new features to allow more control, understand that you’re going to open up more

potential problems because you’ve made the system more complex because there’s a surface area of things that are happening to the system has increased, and they’re always going to be disappointed because they want to do more than what you’ve even added. You’re never gonna make everybody happy. But of course you add more control the more people get confused and then you get more email support and you’re just like, ⁓ man why are they even doing that, you know?

If everything is a priority, nothing is

Jason Roberts (00:00) If I what are your life goals? You’re like, I got 100 life goals. Okay, so basically you’re gonna do nothing. Because you can’t do everything. So you say, no, my goal is this. Have one or two priorities, maybe three. You get beyond a few priorities and then you have to spread your time and effort among them all so much that nothing’s

An overly complex product will confuse users

Jason Roberts (00:00) It’s like looking at Photoshop. You’re like, pfft.

I gotta take like a semester long course to understand how to use this, even get started. I mean there’s tons of software like this AutoCAD and 3D modeling and rendering and they’re all like that and you’re just like oh my god.

Because every feature you add is putting walls down. It’s making stuff more complex. New users come in and are intimidated. They can’t even get started, and it’s just not a beginner product. So now you open up space for a competitor to come in and have a slimmed downed

version of what you do and say, well, we do the really important 80 %, it has 20 % of the features. It’s like something like we’re video editing. It’s like Cap Cut instead of Adobe Premiere. It’s super powerful, but like I need to just cut this video in like 15 minutes. Well yeah it’s probably not the thing. So it’s just it’s all trade offs.

You gotta get people working together. Some type of synchronization is critical.

Jason Roberts (00:00) you got to get people kind of.

working together rowing, you know, it’s like, how hard is it to keep these guys all pulling together? smaller person in the front is like, pull, pull, whatever. It’s like, do they really need that? I guess, yeah. Cause if they didn’t, they wouldn’t have that person, right? Cause it’s extra weight.

Or conductor in orchestra.

I don’t really know what they’re doing, but apparently they’re keeping everybody moving together and doing their part until everything syncs up and creates its effect. This seems to be required in human nature when you have multiple people working together type is critical.

It's hard synchronizing people who are all working part-time or remote

Jason Roberts (00:00) This seems to be required in human nature when you have multiple people working together type is critical. when you’re working with people who are all working part-time or remote, it’s even harder than if they all show up to an office

and they kind of sort of can self coordinate. When they’re kind of doing this 10 hours a week and they’re working, they’re taking grad classes or working on a dissertation. I mean, it’s so easy for them to be like, well, what are we doing? You know, like do this, come

You better be careful what code you write because you're going to be supporting it for the rest of your life

Jason Roberts (00:00) Buddy of mine in my first startup, used to say like, you better be careful what code you write because going to be supporting it for the rest of your life.

And if you’re solving problems that aren’t real problems, now you have all this extra code that you’ve got to lug around that are putting constraints on these other solutions. It’s like, oh, jeez, but there’s the thing, and it’s going to impact this thing. And now it takes five times as long to build. So there’s just so many reasons.

to try to avoid caught up building stuff that doesn’t really need to be built and just focus on the really critical stuff that you know how to solve now.

Smart people can invent an infinite number of imaginary problems

Jason Roberts (00:00) there’s just so many reasons.

to try to avoid caught up building stuff that doesn’t really need to be built and just focus on the really critical stuff that you know how to solve now.

smart people can invent an infinite number of problems themselves. They can really and they can make a great job convincing you and themselves that this absolutely has to be done. and I think sometimes smart people struggle

setting limitations, constraints on their own hubris and their ability to just think through anything. you just have to remember the times that you did that and you screwed it up

It's a fantasy to believe that kids will naturally fall in love with school

Jason Roberts (00:00) of the problems is is there’s a little bit of a fantasy that parents and some teachers play into. It’s like, just want them to fall in love with the subject.

Little kids or younger kids are enthusiastic about most things, especially if their parents are excited about it too and engaging with them. But you’re talking about middle school and up. It’s just not, it’s not a reliable thing.

Justin Skycak (00:18) Most of the time, that’s not going to work. If you are being held to account for kids learning the material, that’s not the strategy to lean

Like try to make that happen

for the kids who are receptive to it, but just understand that most kids are not going to be like that. So you have to actually focus on the mechanics of incentivizing people to do work. And a lot of kids maybe they go into software and later realize, Hey, math is actually pretty cool. They, might realize this later downstream. And they’re going to have to go through a little bit of, I don’t love it at the moment.

in order to get to a place where they do have a greater connection to it.

Jason Roberts (00:49) Yeah, it’s just

Gamification gets kids excited about drills and practice

Jason Roberts (00:00) I would try and gamify everything that I did when I would teach and coach. So I remember when I would my son Colby when he was five, six, seven years old in soccer,

I want to take advantage of every minute because we only practice like once a week or maybe twice a week like for an hour. So what are you gonna get done? It’s hard to get anything done.

I want a ball at everyone’s foot the entire practice. We’re going to be doing stuff. I’m going explain as quickly as I can to get it across, and then I’m going to quickly turn it into a game to make it fun. And I would come up with a cool name like Zombie Attack.

And they would, I want to play Zombie Attack, right? You give it a cool name, and it worked. And it was a great way to get the most out of them. So it’s make things as fun as and as engaging as you can.

Incentives can work overnight to correct kids' behavior

Jason Roberts (00:00) So I remember in second grade, we get a conversation or something from his teacher.

He walks up and he says, you know, Colby is really having a hard time and he’s really being disruptive or he’s not following directions. And so then he goes, so what I’m gonna do is I’m gonna send home like a

behavior card where it’s like seven categories and I’m gonna rank them a one through five and so the first few days it’s like threes and twos. And Sandy’s like, okay if it’s all fours and fives, I’ll give you a dollar.

Within the week,

He’s racking up a dollar almost every day. And then it’s like a month or five weeks go by and then we didn’t get the card I’m like, what’s going on with the card? And so with the teachers, they’re walking the kids. So he kind of came up to us and he’s like, no, it’s good. It’s fine.

No need anymore. That was end of it. But that incentive system made all the difference

A little gamification goes a long way

Justin Skycak (00:00) It’s funny how minimal these incentives and gamifications have to be to work.

Like kids have imaginations, right? You can say something and they will run with it. And it’s just the idea of framing the thing that you’re doing. So when it comes to math,

Jason Roberts (00:08) It doesn’t take

Justin Skycak (00:14) when we talk about gamifying, we don’t mean you have to make everything into like an actual video game where everything is on the screen. Like you have a joystick and stuff doing math. You just need a little dose that makes it kind of interesting in that sense. Right.

Jason Roberts (00:30) You’re

framing it as a game. This is supposed to be fun. They’re like oh, so it isn’t work. It’s a game. Yeah, yeah, yeah, yeah, it’s a game. They’re like, oh, OK, game. And as long as it’s gamey enough, then it’s a game. It may not be the best game, but it’s a

When you train for speed, you get fast

Jason Roberts (00:00) When Colby and the original cohort of Math academy students

in seventh grade,

I was teaching calculus.

I would introduce a concept, do a couple on the board, and then everybody on the board. And everything was through a competition.

And it was about speed as well as getting it. And immediately they loved it.

They love being up at the board because they didn’t have to sit down. They’re all like move around and

you know, doing the thing. And of course, they enjoy competing.

We had a lot of people would come visit cause couldn’t believe it. I tell them and they’re like, you’re doing what?

I said, yeah, seventh graders are doing calculus. They’re like, And I remember this friend of mine, Gary, who was a mathematician at Caltech.

And he’s watching it and afterwards he’s just like, that was incredible.

But what’s really amazing is how fast they are. And I said, well, Gary, when you train for speed, you get fast.

The incentive structure was about winning the game, and they got really, really good and really, really fast.

High school is a pressure cooker

Jason Roberts (00:00) middle school, kids can be really motivated, really excited to do things. There’s a lot to just do almost anything. Seventh grade, eighth grade, it starts to wane. You get to high school, they’re really…

And One of the problems that was really frustrating about high school is how the system is set up. They have a lot of pressure on them In the US, it’s about like,

you got to take advanced courses or AP courses and you got to get A’s because you got to have this great GPA. Because if you don’t have that, there’s no way you’re going to get into a decent college.

And then you gotta be in all of these extracurriculars and you can’t just be in them, no, you gotta be the student class president.

The thing is though, that creates a lot of stress and a lot of pressure. And then it becomes, I just got to get through this stuff. I don’t have time to love anything anymore.

Because any time there’s stress on you, or pressure on you, it’s hard to enjoy stuff.

It just takes the fun out of everything. Once you get to high school, it really gets to that.

Parenting reality: motivation, incentives, and follow-through

Jason Roberts (00:00) For younger kids who are not

wanting to do Math Academy every day or you know as frequently,

you need to create incentive structures that they feel okay

Find what they’re excited about and set a nice carrot for them.

You find something that’s personally important to them, that they want, and you use that as an incentive to get them to doing it. Because they’re kids, they’re like, I wanna play band, and then by a month in, they don’t wanna practice anymore.

Or I want to be on soccer, but they don’t want to go to soccer practice because they want to play video games with their friends. Like, no, you’re on the soccer team, you got to do soccer practice. Even things that are fun, that they said they wanted to do, their motivation starts to flag because they’re kids.

As parents, you don’t always get to just be like Mr. Nice Guy. You have to say, like, no you’re doing this.

Now you can try and make it as painless as possible by setting up incentives. Like they have to understand, I pretty much have to do this anyway. But now I got this other incentive. You kind of got this sense of a stick

in the background. I mean, I’m going to have to do it, but I have this little carrot I’m excited about. So let’s focus more on the carrot.

Playing the game: when students chase easy A's instead of challenging themselves

Jason Roberts (00:00) of the problems that was really frustrating about high school is just how the system is set up. They have a lot of pressure on them

to get into a decent college.

Their stress is too high for them to be able to fall in love with some of this stuff.

I remember talking to some parents before Math Academy was in the high school, and I was

talking about, we could do a whole science thing and this and they’re like, listen, Jason, these kids have all these AP classes, they have all this stuff and they don’t have the time for all that.

And that started happening to me in the Math Academy program in the high school. We started to see kids who were good at it and liked it. It was just the pressure because their Math Academy classes were typically their hardest were like, what AP classes can I take to get an A? We had some kids who would switch out of Eurisko and just take AP Computer Science for the second semester because it was a joke, comparative they could just get an A.

They weren’t going to learn would get an easy A. And these are a couple of kids who went to MIT and stuff. They’re just like, I’m playing the game, man.

The most mathematically gifted student I ever worked with still needed to be pushed to learn calculus.

Justin Skycak (00:00) The most gifted kid that I’ve ever worked with, he was actually resistant to learning calculus around seventh or eighth grade.

had kind of learned a lot of arithmetic on his own and I had found some puzzles online that he wanted to work on.

And was a good use of time at the time. Eventually, you get to a point you’re not really progressing a whole lot in your mathematical development. I mean, you’re

way ahead compared to grade level, but like at some point you gotta make the leap.

Like there’s levels of math that you have to climb in order to just get further along in the talent domain in order to unlock new things for you to do.

Adults think like that. Kids don’t think like that cause kids don’t have the longer perspective. They don’t know what the long game is. They haven’t seen the long game play out.

And so what ended up pushing this kid over into okay, fine, I’ll learn calculus was that

he wanted to go take college level math courses, like in ninth grade. And so I was talking to him and his parents like, yeah, he can totally do that.

Problem is though, if he doesn’t know calculus, then not only is he going to struggle in these courses,

they’re not even going to let him into these courses if he doesn’t have the five on the AP Calc BC exam.

The big thing was getting his parents on board with it because if the parents on kid’s going to be on board one way or another.

The interesting part is once he learned all the calculus stuff, calculus became one of the things that he really enjoyed.

And now today, I still ⁓ work with him every other week.

He’s gotten through a lot of undergrad math, and so he’s actually sinking his teeth into research,

working university mathematician.

And there’s like bunch of derivatives being tossed around.

And it’s in this area that he was resisting back in seventh or eighth grade, and he’s having the time of his life right now. And if we had not pushed him through this segment of the journey that he was resistant to, he would not be doing what he’s doing right now.

Short 20 - Schools maximize for bureaucratic convenience

Jason Roberts (00:00) A lot of public schools, if you get a D or an F in a school, they will still promote you to next course. I couldn’t believe when I first heard that when I was talking to a high school math teacher in the

Pasadena school district. It’s just like, what’s the point? If they don’t know Algebra 1, why are you sending them to Algebra 2? It’s just bureaucratic convenience. Move them along. Kid can’t read in first grade, move them to second grade. Still can’t read, the third grade. They just do it.

Bureaucracies really maximize convenience. And they’re just things that work out on their their plan, or whatever. And now if you come in and you say, my fifth grader should be doing algebra or whatever.

They’re going to be, uh no. What they won’t say is that’s a massive headache for us, and we don’t want to do it. So we’re going to come up with whatever reason we can to prevent that from happening because we don’t want to deal with the headache.

They do not want to do extra work. Nobody really wants to do extra work, but I tell you, the schools really don’t want to do it. And the teachers themselves, they got like my union, I don’t have to do this. And the principal’s like, I can’t make them do because the union and the bureaucratic and it’s a nightmare. So even if you could come to them and you say,

Well, can my daughter test into algebra? But if your daughter scored like a 97%, they’re like, oh, they did miss 3%. Don’t mind the fact that we promote people with an F to the next grade. They don’t know 3%. I’ve heard that. It’s like 93%, 95%, and they would not allow them.

Every once in a while you run into one teacher, one principal, school who’s like really open to it, but that’s super rare. It’s really frustrating.

The most important thing that parents can do is encourage their kids to find what they're good at and lean into it

Jason Roberts (00:00) The great thing about life is we’re all different, and we all have our things that come easy for us. We get some things for free and then we get stuff we have to work for. And part of life is figuring out what those things are.

It’s tough when kids are not really particularly stand out in anything, and they’re trying to figure out who they are. And it’s kind of tough because I don’t have anything to build an identity on that I can be really proud of. It doesn’t always have to be a competitive thing, but it’s just something.

The most important things that parents can do is encourage their kids to do stuff, to try things, especially things they think, I think you would like this. I think you might be good at It’s kind of scattershot when they’re really young. They just do everything up through 11 or so.

And they eventually kind of will find something.

and become skilled and leveling up in something and developing a sense of pride and a sense of accomplishment and success.

Kids need adults to keep them on the rails towards their goals when the going gets tough

Jason Roberts (00:00) That’s why it’s so important for adults who can look ahead and say, look kid, I know you want to mess around this thing, but I know where you’re going, and I know what it takes to get there. And if you just guide the student to that point, they’ll be

vastly happier, vastly more successful, really realize their potential.

Justin Skycak (00:16) Let them find the things that they’re interested, don’t necessarily like push them like hardcore into things that they have no interest or no gift. Yeah, yeah, exactly.

Jason Roberts (00:24) that always backfires that always backfires

and the kids just give it up with piano or something your parents made him do it and then as soon as they’re old enough you’re like I am done I’m ever touching piano the rest of my life you know

Justin Skycak (00:33) Yeah.

But if your kid gravitates towards something that they really like doing net positive in their life.

and especially if they show a gift for it, they have some kind of like big advantage where this could really be a massive thing in the rest of their life.

Like you can’t let them quit at the slightest sign of difficulty. Kids sometimes will do that or like they love this thing until it gets a little hard and then they’re like, ah, I just want to stay in easy land. Part of supporting the kid means helping them get through those phases of the journey

that may not be the most enjoyable.

Don't let kids quit at the slightest sign of difficulty

Justin Skycak (00:00) if your kid gravitates towards something that they really like doing net positive in their life.

and especially if they show a gift for it, they have some kind of like big advantage where this could really be a massive thing in the rest of their life.

Like you can’t let them quit at the slightest sign of difficulty. Kids sometimes will do that or like they love this thing until it gets a little hard and then they’re like, ah, I just want to stay in easy land. Part of supporting the kid means helping them get through those phases of the journey

that may not be the most enjoyable.

PODCAST 4

What you cannot create, you do not understand

Speaker 2 (00:00) In software, just importing a solution from the library and saying I imported this model and I ran it. “now I’m a machine learning researcher engineer.” Like, no, no, no, you can’t just use the off the shelf. Like you can’t just use the, the theorem and wield the theorem and say, “now I am all powerful.” You actually have to go code from scratch, re-derive the result

from the bottom up to really understand the mechanics of what is it. It’s not enough to just take it off the shelf and use it. You need to know what went into building this thing.

Don't let talent become a crutch

Speaker 2 (00:00) You don’t want to let your natural gift or talent turn into a crutch. The moment you just rely on it and stop developing other parts of your game, that’s the moment when you start hitting like a really sharp asymptote, right? It reminds me of, in the original Pasadena program, oddly enough, sometimes it was the sharpest

sixth graders who could do the most work in their head, who would struggle the most a year later because they were so resistant to writing anything down. They were like, no, I don’t need pencil and paper. I just do all this in my head.

Their outsized working memory capacity became a crutch. They can only solve problems that they can immediately, like entirely fit in their working memory without usage of paper and pencil. That came back to bite them really fast.

SAT problem solving skills can be enumerated and practiced

Speaker 2 (00:00) Problem solving is like this, nebulous thing that everyone says it’s on the SAT. Like you got to get the kids to, think critically, to problem solve their way. how do we teach that? But once you, once you start drilling down into

what do the kids have to do? You realize that it all comes down to these skills that you can enumerate explicitly. And what’s more, you can arrange them in a knowledge graph. as a course. It’s a body of well-connected knowledge, hierarchical, just like anything else. so when people

refer to like, ⁓ the kid needs to learn problem solving, need to learn to think critically. just, there’s this whole body of knowledge that they just haven’t gotten, but it can be enumerated exhaustively.

The knowledge graph trades student work for instructor work

Speaker 2 (00:00) The knowledge graph,

it’s this trick that allows you to trade off student work for instructor work. Instead of having the student go through this hodgepodge of problems and hopefully infer all the structure, and just have them grind through a large enough volume that they can infer. What you do is have the system,

figure this all out behind the scenes. It doesn’t mean that there’s less work that has to happen, but there’s less work for the student. We have to do more of the work, but that’s the trick. And we can do our work at scale.

Strengthening your weak points is a force multiplier

Speaker 3 (00:00) I came in very, very little engineering background.

I had a strength in mathy coding, but until then I had just leaned into that so much that I never developed the rest of my coding abilities. And that was a severe weakness. But the silver lining of these severe weaknesses is once you shore them up, then you become so much more capable on so many more fronts.

It wasn’t fun realizing just how lacking I was on the software engineering side of things as opposed to the quant. But once I just dug in and leveled it up, suddenly there’s all sorts of things that became available to me that I was able to start working on that I wouldn’t have anticipated.

If you can't figure out a test problem in minutes, you're toast

Justin Skycak (00:00) When you’re actually talking about solving concrete problems in the format of like a math test standardized test competition test,

justin 4 1 (00:00) When you’re actually talking about solving concrete problems in the format of like a math test standardized test competition test,

Justin Skycak (00:08) you have to be familiar with this class of problems that you’re trying to solve. Cause there’s just so much contextual information about that specific kind of problem and what type of methods will unlock it. you’re not going to invent them on the spot in a minute or two minutes or five minutes

justin 4 1 (00:08) you have to be familiar with this class of problems that you’re trying to solve. Cause there’s just so much contextual information about that specific kind of problem and what type of methods will unlock it. You’re not going to invent them on the spot in a minute or two minutes or five minutes

Justin Skycak (00:28) Yeah. You don’t have the time to approach it like a, like a research problem.

justin 4 1 (00:28) Yeah. You don’t have the time to approach it like a, like a research problem

Justin Skycak (00:32) or even like a take-home If you can’t figure it out in a couple of minutes, you’re

justin 4 1 (00:32) or even like a take-home If you can’t figure it out in a couple of minutes, you’re

You can explain anything if you break it into small enough steps

Speaker 2 (00:00) You can explain anything if you break it a small step, to do too much. It’s usually they’re trying to take too big of a jump and then people just fall off. It’s like, you know, I used to rather than stairs, I used to think of it as like rocks placed across like a creek or something. And it’s like, okay, well, if they’re, three inches apart than grandma can do it. If it’s like eight inches, it’s like, well, you know, my mom and dad can do it. Okay, now it’s

a foot and a half, it’s like, well, I can do it and stuff. But now it’s good point where it’s like, well, my son can do it. Three feet, you know, it’s like, that’s what happens. It’s like, only super gifted, brilliant people can do because the pedagogy is so bad and the jumps are so big, you need to need someone who could do a standing broad jump of nine feet to make And it’s like, OK, why rather than writing textbooks that have these

six and eight and 10 feet jumps every two or three pages. And it takes them incredible mathematical aptitude to make those jumps. Otherwise they’re just like, man, I just, I don’t know. I’m like falling into the river and I’m floating down the creek and like, wait, this

Building courses at the world-class level

Speaker 2 (00:00) One thing that I always, would say, this is what I think we need out of this course is

it needs to be at the level of what you would see at Harvard or MIT or Stanford or whatever. You don’t want someone to say, well, you know, that thing at Math Academy, I mean, you know, it’s just not bad. mean, it’s just kind of And so, you know, one thing I’d ask you to do when I we first created our

university level courses, linear algebra and multivariables, like go look, what can you find? What are they doing? If you can see any published final exams or syllabi from these institutions, what are they covering?

What’s the superset of the commonality, right?

So we always say like, we can reinvent a lot, but we do not exist in a vacuum. We exist world

where things are done a certain way and people have certain understandings and expectations. And if you drift too far off that, then people just do not know how to think about what your product is or what you’re doing. And it just becomes so much ⁓ friction in crossing that, it’s just, it can cause the product to fail.

You don't learn robust code until it fails

Justin Skycak (00:00) What’s really funny, the learning curve happens, then you deploy something and it blows Yeah. All of sudden all these errors start happening down, emails are coming. And then that was a whole nother learning curve of how to write bulletproof code. Because stuff blowing up.

justin 4 2 (00:00) What’s really funny, the learning curve happens, then you deploy something and it blows Yeah. All of sudden all these errors start happening down, emails are coming. And then that was a whole nother learning curve of how to write bulletproof code. Because stuff blowing up.

Justin Skycak (00:15) It’s so stressful. It’s such a horrible experience. All these emails, nobody’s got tasks, this thing’s And so then you start going through the process of learning how to write code that won’t fall down, that’s logging everything. Cause then like what happened?

justin 4 2 (00:15) It’s so stressful. It’s such a horrible experience. All these emails, nobody’s got tasks, this thing’s wrong. And so then you start going through the process of learning how to write code that won’t fall down, that’s logging everything. Cause then like what happened?

Justin Skycak (00:28) are how to alert us.

justin 4 2 (00:28) are how to alert us.

Justin Skycak (00:30) you don’t realize things like cascading failures one thing fails, it causes another thing to fail. You don’t realize the magnitude of a screw up that can happen just from one small thing. And so you think like, well, I’ll just make things not break

justin 4 2 (00:30) You don’t realize that there’s things like cascading failures like one thing fails, it causes another thing to fail. You don’t realize the magnitude of a screw up that can happen just from one small thing. And so you think like, well, I’ll just make things not break.

Justin Skycak (00:45) And that’s the first step, but then there’s another step like, okay, what if something does break? we need to limit the breaking and its scope, cause just something’s going to break at some point, even if you’re as careful as possible.

justin 4 2 (00:45) And that’s the first step, but then there’s another step like, okay, what if something does break? we need to limit the breaking and its scope, cause just something’s going to break at some point, even if you’re as careful as possible.

Justin Skycak (00:58) or somebody is gonna run into some scenario that you just never thought would have ever happened, or somebody’s data is gonna be corrupted and then you gotta make sure that that doesn’t screw up with other people. so there’s just lots I guess more and more advanced sort of error handling and writing this kind of robust code.

justin 4 2 (00:58) or somebody is gonna run into some scenario that you just never thought would have ever happened, or somebody’s data is gonna be corrupted and then you gotta make sure that that doesn’t screw up with other people. there’s just lots I guess more and more advanced sort of error handling and writing this kind of robust code.

Justin Skycak (01:15) I mean, that’s why it’s good to start small and then like organically. Cause then the, these skills and tools and processes and everything can evolve over time so that

justin 4 2 (01:15) mean, that’s why it’s good to start small and then like organically. Cause then the, these skills and tools and processes and everything can evolve over time so that

Justin Skycak (01:28) when you do have a larger scale, it’s not like, no, what do we do now? It’s like, it’s just % bigger than it was.

justin 4 2 (01:28) when you do have a larger scale, it’s not like, no, what do we do now? It’s like, it’s just % bigger than it was.

The biggest growth happens when someone trusts you with something

Justin Skycak (00:00) the biggest growth happens when somebody says, here, you take care of it. And you’re like, oh, shit. yeah. was much right. I kind of felt like you just gave me code baby, this robot to take care of. And you’re like, don’t kill it. Like, trust you.

justin 4 2 (00:00) the biggest growth happens when somebody says, here, you take care of it. And you’re like, oh, shit. yeah. was much right. I kind of felt like you just gave me code baby, this robot to take care of. you’re like, don’t kill it. Like, trust you.

Justin Skycak (00:18) I mean, I did not study computer science in college. So I didn’t really have any sort of background in this. So was just learning the hard way, right? School of hard knocks. just, all right, crap, code blew up. Well, I guess they don’t really teach that a whole lot in computer science programs production code is where you learn.

justin 4 2 (00:18) I mean, I did not study computer science in college. So I didn’t really have any sort of background in this. So was just learning the hard way, right? School of hard knocks. just, all right, crap, code blew up. Well, I guess they don’t really teach that a whole lot in computer science programs production code is where you learn.

Skip bottom-up learning and you’re just cargo-culting machine learning

alex 4 2 (00:00) With this machine learning course, want people to do gradient descent by hand so it really gets under their skin.

Justin Skycak (00:00) with this machine learning course. want people to do gradient descent by hand. So it really gets onto their skin.

alex 4 2 (00:05) you actually come to code something like that, you know exactly what’s going on. It’s very easy to sort of look at a formula, create like a Python script, which is like 30 lines long. It kind of gives you an answer. It’s like, don’t really know what’s going on here.

Justin Skycak (00:05) you actually come to code something like that, you know exactly what’s going on. It’s very easy to sort of look at a formula, create like a Python script, which is like 30 lines long. It kind of gives you an answer. It’s like, don’t really know what’s going on here.

alex 4 2 (00:18) You know, people kind of the top down method of becoming machine learning engineers and never really did the bottom up part. So they kind of vaguely know what gradient descent is, but not really. And that’s not a good position to be in if you really want to make kind of like cutting edge or pushing the boundaries on things.

Justin Skycak (00:18) know, people that kind of went the top down method of becoming machine learning engineers and never really did the bottom up parts. They can’t vaguely know what gradient descent is, but not really. And that’s not a good position to be in. If you really want to make kind of like cutting edge technology or pushing the boundaries on things.

alex 4 2 (00:35) So yes, it is a necessary evil to kind of go through those painful calculations by

Justin Skycak (00:35) So yes, it is a necessary evil to kind of go through those painful calculations by

alex 4 2 (00:42) It’s like, dude, like now, you know, what I know you didn’t love it, but now you’ve mastered it. You know, intuition through repetition.

Justin Skycak (00:42) It’s like, dude, like now, you know, what I know you didn’t love it, but now you’ve mastered it. You know, intuition through repetition.

alex 4 2 (00:49) That’s where you learn. You develop that intuition and then you can get the abstractions stuff all fall into place. But if you try and skip the concrete examples, skip the repetitions, and go straight to the abstraction formulas, you’re like cargo cult

Justin Skycak (00:49) That’s where you learn. You develop that intuition and then you can get the abstractions stuff all fall into place. But if you try and skip the concrete examples, skip the repetitions, and go straight to the abstraction formulas, you’re just like cargo cult

math, you’re just parroting stuff. It’s like, they can’t really solve anything.

alex 4 2 (01:04) math, you’re just parroting stuff. It’s like, they can’t really solve anything.

Why you need to do gradient descent by hand

alex 4 2 (00:00) With this machine learning course, want people to do gradient descent by hand so it really gets under their skin.

you actually come to code something like that, you know exactly what’s going on. It’s very easy to sort of look at a formula, create like a Python script, which is like 30 lines long. It kind of gives you an answer. It’s like, don’t really know what’s going on here.

You know, people kind of the top down method of becoming machine learning engineers and never really did the bottom up part. So they kind of vaguely know what gradient descent is, but not really. And that’s not a good position to be in if you really want to make kind of like cutting edge or pushing the boundaries on things.

So yes, it is a necessary evil to kind of go through those painful calculations by

The knowledge graph makes sure students are ready to learn each new topic, minimizing friction

Justin Skycak (00:00) The great thing about having a knowledge graph is that it’s a very strict implementation of mastery learning. We don’t let you do topic X unless you have mastered or at least proficient in all of the prerequisites for topic X. You have to have demonstrated that. Now, as curriculum designers, especially when we’re looking at some of these test prep topics.

justin 4 1 (00:00) great thing about having a knowledge graph is that it’s a very strict implementation of mastery learning. We don’t let you do topic X unless you have mastered or at least proficient in all of the prerequisites for topic X. you have to have demonstrated that. Now, as curriculum designers, especially when we’re looking at some of these test prep topics.

alex 4 1 (00:00) great thing about having a knowledge graph is that it’s a very strict implementation of mastery learning. We don’t let you do topic X unless you have mastered or at least proficient in all of the prerequisites for topic X. you have to have demonstrated that. Now, as curriculum designers, especially when we’re looking at some of these prep topics,

Justin Skycak (00:21) where I think is really powerful is it allows you to kind of include all those really subtle prerequisites that could easily slip under the radar. But if you’re just analyzing these problems really carefully, so that’s a really important prerequisite. I’m gonna make sure that that is a prerequisite of this topic, which means you know the student is prepared with all the necessary prerequisites before they even see the topic.

justin 4 1 (00:21) what I think is really powerful is it allows you to kind of include all of those really subtle prerequisites that could easily slip under the radar. But if you’re just analyzing these problems really carefully, so that’s a really I’m gonna make sure that that is a prerequisite of this topic, which means you know the student is prepared with all the necessary prerequisites before they even see the topic.

alex 4 1 (00:21) what I think is really powerful is it allows you to kind of include all of those really subtle prerequisites that could easily slip under the radar. But if you’re just analyzing these problems really carefully, so that’s a really I’m gonna make sure that that is a prerequisite of this topic, which means you know the student is prepared with all the necessary prerequisites before they even see the topic.

justin 4 1 (00:45) Another thing, of course, just by simply enumerating all the prerequisites, you can say to yourself, well,

alex 4 1 (00:45) thing, of course, just, just by simply enumerating all the prerequisites, you can say to yourself, well,

Justin Skycak (00:45) thing, of course, just, just by simply enumerating all the prerequisites, you can kind say to yourself, well,

alex 4 1 (00:50) hang on, this topic has got 15 prerequisites. That’s way too much. That’s cognitive overload. Huge issue right there. This needs to be split up, organized in a different way. So having that information just in front of you allows you to take all the things we know about the science of learning into account, incorporate all those sort of subtle prerequisites so that the student is not struggling at any point.

Justin Skycak (00:50) hang on, this topic has got 15 prerequisites. That’s way too much. That’s cognitive overload. Huge issue right there. This needs to be split up, organized in a different way. So having that information just in front of you allows you to take all the things we know about the science of learning into account, incorporate all those sort of subtle prerequisites so that the student is not struggling at any point.

justin 4 1 (00:50) Hang on, this topic’s got 15 prerequisites. That’s way too much. That’s cognitive overload, huge issue right there. This needs to be split up, organized in a different way. So having that information just in front of you allows to take the things we know about the science of learning into incorporate all those sort of subtle prerequisites so that the student is not struggling at any point.

Superficial knowledge of a topic can get you quick wins, but you will eventually hit a wall

Speaker 1 (00:00) One sort of temptation of the top down is, is it gives almost like a perception of expertise, a perception of skill, of familiarity with the problem that you can talk a good game.

I’ve been fooled a number of times by people because like, I want to believe that they’re good. Cause I don’t want to want to spend all this time digging in and testing them. And I was like, Oh yeah. So, you know, this and they, they use all the right buzzwords and they talk about the right methodology and the right libraries and the right frameworks. And that they talk knowingly and as if they have battle scars or whatever know, how you develop complex systems and quality software. and it’s, really like talking good game.

Because you You read some thought pieces by the area, right? So you get this sort of vibe or the sort of the language of the domain, who and what’s what and what’s cool and what’s not, what are people excited about? What are the important results? And, you could talk to somebody who’s an expert for 20, 30 minutes, like, oh, this guy knows his stuff.

Yeah, yeah, we should talk to him. It turns out it was an inch deep. But it gives you a quick payoff, It impresses people around you. might get you job. might get you grant maybe or research support or whatever environment you’re But then again, you run out gas. being able to make progress.

The Cognitive Limit on Topic Prerequisites

alex 4 2 (00:00) We have direct prerequisites, which are, in the knowledge graph, they’re sort of like, you know, like it’s like one edge connecting the topic to its its prerequisites.

Justin Skycak (00:00) we have direct prerequisites, which are in the knowledge graph. They’re sort of like, you know, like it’s like one edge connecting the topic to its its prerequisites.

justin 4 2 (00:00) we have direct prerequisites, which are in the knowledge graph. They’re sort of like, you know, like it’s like one edge connecting the topic to its its prerequisites.

alex 4 2 (00:07) in terms of direct prerequisites, I usually say that three or four is probably I start getting feeling uncomfortable when it’s anything more than that.

Justin Skycak (00:07) in terms of direct prerequisites, I usually say that three or four is probably I start getting feeling uncomfortable when it’s anything more than that.

justin 4 2 (00:07) in terms of direct prerequisites, I usually say that three or four is probably I start getting feeling uncomfortable when it’s anything more than that.

alex 4 2 (00:14) It’s like, this is, this is feeling like cognitive overload.

Justin Skycak (00:14) It’s like, this is, this is feeling like cognitive overload.

justin 4 2 (00:14) It’s like, this is, this is feeling like cognitive overload.

alex 4 2 (00:18) You know what’s interesting is that you said three to four prerequisites. I mean,

Justin Skycak (00:18) You know what’s interesting is that you said three to four prerequisites. mean,

justin 4 2 (00:18) You know what’s interesting is that you said three to four prerequisites. mean,

alex 4 2 (00:23) when you look working memory literature, somebody can hold about chunks of information in working memory. about your capacity.

Justin Skycak (00:23) when you look working memory literature, somebody can hold about chunks of information in working memory. about your capacity.

justin 4 2 (00:23) when you look working memory literature, somebody can hold about chunks of information in working memory. about your capacity.

The SAT’s Vast Possibilities and Narrow Reality

Speaker 2 (00:00) Based on the SAT, based on all these foundational skills, they can be pulled together in so many different You can do the calculation. Let’s say you have like 100

subskills, 200 subskills, and you’re computing how many different combinations of three or four of those subskills, and you get an astronomically large number. Well, there’s no way that we’re gonna hit all of these combinations. But once you actually look at the exam and you see these combinations that show up over and over again, it’s a much, much smaller space.

It’s not this astronomically large number. It’s a large number, but like you can do it. If you’re willing to put in some elbow grease can take even a student who’s not particularly mathematically

gifted, can get them to fill in a lot of these gaps that a lot of genius students are just kind of inferring on the fly. You can make that explicit for them and scaffold them through that process and get them on that.

PODCAST 3

The importance of "force of will"

Jason Roberts (00:00) Anytime you wanna do something that’s outside the system, that doesn’t already have sort of a infrastructure and a thing in place, it really comes down to a force of will.

Because whatever idea you have, like, this is going to be this cool new thing, and here’s all the benefits. And that may be true, but there going to be so many obstacles. And frankly, people that are going to try and slow you down stop you. They’re going to try to derail you.

Try to pull people along with you

Jason Roberts (00:01) doesn’t mean you’re a blunt, instrument, Cause that doesn’t work. That just pisses people off. And then you get, instead of passive resistance, you get active resistance. You create direct enemies, people who hate you. You want to avoid creating enemies, cause they can do a lot of damage.

They may not come to your face and yell at you or something, but they will be sabotaging. They’ll be blowing up bridges and stuff in the middle of the night, right? You know, figured it out.

Justin Skycak (00:27) Yeah, because once somebody

hates you, then they just get pleasure out of opposing you no matter what it is. go from a state from like, okay, initially people are resistant to just like new stuff and stuff that makes more work for them. But once people hate you, like they might resist it just because they hate you, just to oppose you and get satisfaction out of it.

Jason Roberts (00:44) Right.

So you don’t want to offend people, demean people, make them feel bad. You want to try and make it as easy for them as and you want to help them get credit for it. Say, hey, let’s do this thing. This is going to be really cool. I’ll do the heavy lifting. And so you keep them in the loop. You keep them updated. Hey, just want you know, I’m this person. I think we can do this. What do you think? You kind of try and.

pull them along and make them feel as good about it as possible.

You want resisting you to be the hardest thing they're gonna have to do

Jason Roberts (00:00) You want resisting you to be the hardest thing that they’re gonna have to do. Not because you’re their enemy but just because They’re just gonna hey, I like what he’s trying to do. It sounds kind of crazy it’s work, but Okay, fine,

and you just try and make them feel good about it. You’re pulling them along on this journey, Like Lord of the Rings, they’re a hobbit. You gotta come on this journey with me, right? They don’t wanna do it. They’re in their hobbit hole and they’re like, You kinda inspire them and they’re like, all right, it’s easy to go on this journey with Gandalf than it is to resist him. You kinda have to create that dynamic.

Gifted/advanced education is necessary because not everybody wears a size-medium shirt

Jason Roberts (00:00) they did not quite realize the level of resistance that was going to come from the principals and superintendents and chief academic officers who just, they don’t believe in gifted education or advanced, they want everyone to just be at grade level, does everything at the same pace.

Also just makes our life easier too. Conveniently, right? You’ll notice that doing less, making everybody do the same thing, it not only makes them feel good in some ways, but just makes their life easier and it makes less work for them, right? Everyone wears a size medium shirt, now this is easy.

I don’t have to measure, oh, you’re extra small and you’re a large, extra large, you’re double X. Oh gee, we got all these different size shirts and how many of them we order of each and who’s wearing what size shirt? They don’t want to deal with that. Everyone wears a size medium shirt. You know, like, wait, you know, my daughter is really small. It’s kind of like a dress for her. You know, just cinch it. You know what? You know, it’s really a medium. And you’re just like, and my son, who’s actually big, it doesn’t even cover his stomach. They’re like, well, it’s like air conditioning. You know what mean? Just like, it’s bullshit, right?

What "willing something into existence" means

Jason Roberts (00:00) what does willing something into existence mean? It means giving everything you got, marshaling all the resources refusing to give up and then just putting continual pressure on something until your goal is achieved. That’s what willing into existence is. It is not lying in bed at night trying to manifest it, thinking, I just really want this to happen for me.

It’s actions all the way down. Actions, actions, actions, actions. has to be something you really want because it’s not going to come easy. that’s what it takes.

Teaching tip: communicate with parents early and often

Jason Roberts (00:00) communicate early and often. You almost cannot over communicate with parents. The error is, you talk to them and then two, three months go by and you don’t talk to them and they’re kind of out of the loop.

Justin Skycak (00:02) Mm-hmm.

Jason Roberts (00:11) kids falling off and then you’re like, this kid has spent the last six weeks doing nothing and it’s a whole problem.

Justin Skycak (00:17) Can’t let it like grow to be that big of a problem. Cause then if you do, then like everyone just digs in their heel. Like the problem is just too big to accept so everyone just wants to say like, no, you’re, you’re crazy. Your expectations are way too high. it makes it.

harder and harder to get things back on the rails because now there’s like some momentum in being off the rails. The kid is used to not doing a whole lot of work in class. Whereas if you catch that at the beginning, get it early, communicate with parent, the parent will catch the kid up that evening. If there’s only a deficit of 30 XP, 40 XP, guess what? The parent’s just going to make the kid do it that evening.

and then the kid’s going to come back, they’re fine. They’re on the rails the next day. We’re all good. Now we just maintain, maintain. But yeah, if the kid has racked up a deficit of thousands of minutes worth of learning that they didn’t do,

once things get to be that bad, it’s just like, it just becomes a bad explosive situation that you just, but the solution is just don’t let it get into that state. Cause that’s you never win.

Jason Roberts (01:27) Yeah,

The typical dynamic between parents and schools: parents act like sports agents

Jason Roberts (00:00) When I was in high school, teachers didn’t meet with parents.

ever. but now because parents lobby through a barrage of emails to the teacher to the principal to the whatever and then just make everything really painful

The parents are acting like an agent. It’s like a sports agent. what they want, a lot of them want is their have like great grades so they can go to whatever college they want to go to, But they also want their kids to learn a lot, but they don’t want their kids to do a lot of work.

Right? And kids aren’t going to learn much unless they put in a fair amount of work, unfortunately, that’s how the world works.

no, you actually gotta do work. So, but parents, they, you know, they can, they wanna do less work. want my life, I want there to be low stress at home, but I want my kid, yeah, I my kid to be really educated, but I also want them to get great grades. Okay, well, the kids, there are some kids who like work really, really hard by default. Most of them don’t. there’s a whole spectrum.

But anyway, the parents at the end of the day, once they realize that, my God, my kid’s gonna get a C, or is he gonna get some Bs when I tell them they’re not gonna get A’s and they’re like, my God, this is really, they’re not gonna get to go to Harvard or whatever dream they have. And so then they serve as this, they wanna lobby the school. And so they do, and so that has helped lower the standards, I think, especially these private schools, right?

paying 30, 40, 50, $60,000 a year and, and, and, and, know, and my kid works really hard in this. these, these costs and the, and the superintendents and the head of school and stuff, they’re like trying to keep the parents at bay. Cause that’s what pain for the pain for the schools, for the administration and the teachers are parents, parents P for pain.

The effect of extreme parental lobbying in education

Jason Roberts (00:00) so I used to coach this men’s soccer team for years and all these guys who were these ex college players and I’m a bunch of them would coach these sort of elite academy level club soccer teams, you know, under 12, under 14, under 16. And I’d be like, I asked him like, how do you like that? Like, oh, it’s great. The kids are great. He’s like, but the parents are a nightmare. I’m like, really? He’s like, oh yeah. Oh yeah. It’s like.

Why didn’t my kid play left? They should play the right side. And why isn’t he starting? And they use it, you know, and it’s just a constant lobbying for the parents. It just ruins the experience for the coach. The coach is trying to, you know, not saying every coach makes good decisions or is fair, whatever, but it’s the parents. Parents equal pain for

coaches and teachers and administration. They’re trying to everything they can to limit said pain. And a lot of the changes you see in the curriculum, well, there’s no gifted class, accelerated classes. We don’t do a lot of homework. We have makeup tests and makeups on the makeups and we have all this stuff. It’s to the complaint, the agents, parents from…

lobbying and creating all this endless cycle of pain. so then you get this situation you have now. so the teachers are just like, whatever, man. I mean, just make it really easy.

If nobody's checking the homework, how's the kid going to learn?

Justin Skycak (00:00) if nobody’s checking the homework how’s the kid gonna learn? Because the kid doesn’t know how to do these problems correctly. The parents almost certainly don’t know how to do these problems correctly. the only kids that get through are the kids who have somebody in their life or whose parents can bring in somebody into their life who can effectively do what

the teacher is supposed to be doing. You’re right, yeah.

Jason Roberts (00:19) Or they’re at the 99th percentile. They’re just super

high aptitude and they’re really into it. And so they’re going to all these extra resources because they’re a physics nerd or whatever. Cause this is what they want to do. But that’s like that, that’s the one out of a hundred kid. everyone else is like, I don’t know what’s going on. Right. And you have a couple of kids who are drafting off that one kid. He’s helping his or she’s helping her little.

Justin Skycak (00:35) Yeah.

Yeah.

Jason Roberts (00:45) a few couple friends and they’re kind of figuring it out, but everybody else is just like, what’s happening?

Justin Skycak (00:47) Yeah.

Yeah, yeah,

yeah, exactly. And very quickly, it gets just so out of hand that like, even if you had a tutor sit with the kid and help them work through the homework problems, like they wouldn’t get a whole lot out of it because the problems are now so far ahead of their ability.

Parents are not aware of how little their kids are learning

Jason Roberts (00:00) parents are not aware of actually how little their kids are learning. Whatever you think your kids are learning, in most cases, they’re learning a lot less. They know less, they’re doing less work, they have fewer skills, less understanding. You know, it’s bad. know, especially…

Things in math and physics and stuff like that that.

Justin Skycak (00:22) foundational for other skills that you might need to do. And also it’s a hierarchy

Jason Roberts (00:26) is it just takes a lot of hard, consistent work to keep making progress. It’s just that the standards have to be higher, they have to be more rigorous, they have to be quantitative. ⁓

That’s just is and if you’re not going to do it and going to make everything mushy and fluffy and we’re just going to have group discussions and projects then guess what? Everybody’s going to do less work. Kids, teachers, everybody. And then at end of the day, people come out and nobody knows anything other than like that one, two kids who were sort of were mostly self-taught and you know.

That’s it. That’s how it rolls.

Math is so poorly taught at the undergraduate level

Jason Roberts (00:00) most undergraduate math majors, We’re playing this little game of pushing symbols around, but nobody really has a great intuitive grasp of the subject itself because it’s not really taught. not really structured in that way.

pushing symbols around without concrete examples is just a really inefficient way to go about things. they just skip the concrete examples

And so you just do the theorem proof theorem and then so, so poorly done at the undergraduate level, almost without exception. There are probably a few, probably like 2 % of math professors in the undergraduate university level who take the pedagogy seriously and are really trying to teach them the rest just show up lecture, problem set, whatever.

Good luck.

Get your fundamental skills in before jumping into research

Justin Skycak (00:00) discovery learning, like, save that until you’ve got your fundamental skills. it takes so much time

to go through and make a discovery. Like, how about let’s save that for when like making the discovery actually yields like serious results.

Jason Roberts (00:12) get more out of it. If we want to get more out,

we can get more from these reps if you have the component skills in place. we’re thinking of like, what’s the most efficient way from go to A to Z?

Justin Skycak (00:18) Yeah.

Jason Roberts (00:22) the total hours put in, if you try and do stuff too early, then there’s a great inefficiencies, you brought some stuff forward that you would have learned later, but in aggregate, spent more hours getting to Z because you’re not getting as much out of it. And you could have spent that time getting these really important requisite skills in place, you know.

Justin Skycak (00:43) Yeah,

exactly. And the end result of your discovery, even if you manage to make the discovery, like, guess what? Nobody gives a shit because like, it’s not real new, it’s new knowledge to you. You did research, relative to your own knowledge base, but humanity has more knowledge. It’s not like a publishable result mathematically. when you’re going to like invest a lot of time and effort into a project,

Jason Roberts (01:01) Stuff in there, yeah.

Justin Skycak (01:07) make sure it’s something that’s like actually like impressive.

Rookies gotta make rookie mistakes

Jason Roberts (00:00) rookies got to make rookie mistakes. And part of rookie mistakes is they don’t listen to senior people, right? No, was old people. They don’t know stuff, right? I just want to do it. Okay. Rookie. See you at the end. I’ll see you at the finish line. You know, how’d that go for you? You know what I mean?

The world attempts to maintain homeostasis

Jason Roberts (00:00) The human body attempts to create homeostasis, like keeps everything the same. So whenever you want to change it, say get in better shape.

become more muscular or something. Well, guess what? Your body’s going to resist that. So you have to work hard to change your body. You have to go and lift heavy weights consistently, over a period of time and your body will change.

constantly have to be hammering away with intention.

The world is like that in a way too. There are all these sort of moving parts and these systems and these vested interests and people are just, they found a place in this world. This is my job or this is my department or this is my thing that I do and they don’t really want it to change typically.

and then somebody wants to come in and go we’re gonna change stuff they’re just like what

people don’t want to adjust. They don’t want to change. Right. Even if you can make a strong argument that this thing you want to do is going to be an improvement. It’s going to help kids or whatever. It’s easy for people to rationalize fighting against it or resisting it

The universe will bend

Jason Roberts (00:00) If you just can every day, you just keep doing what you’re doing, you just keep executing and keep improving and you just never give up and you keep pushing… the universe will bend.

Okay? The universe will bend. It cannot sustain that kind of continual pressure, the continual effort. The world will just say, all right, fine. Fine!

Math is better than video games

Justin Skycak (00:00) calculus just felt like a video game

because climbing up this math ladder, there’s infinite progression. There’s no beating the game and having to go on to a new one. It’s an infinite game. And you know, like everything that you do in this game, you can take with you later in life. This is a game that actually matters. And it’s a game that your parents are actually like really, like they’re proud of you for spending like eight hours just like holed up in your room playing this game.

Jason Roberts (00:07) The infinite game of math, yeah.

You’re

playing eight hours playing Eve online or EverQuest or World of Warcraft. What are you doing? Like, I’m in my guild! And you’re like, what? Get outside. This is terrible.

Justin Skycak (00:28) Yeah, yeah, exactly.

Yeah, exactly.

And everybody’s like super impressed when you come out like, wow, my kid taught himself calculus over the summer. And it’s so funny how similar it is to just like, my kids spent the summer like playing eight hours a day of Call of Duty and ⁓

Jason Roberts (00:51) You’re not telling

people that unless you’re just complaining like my kid only plays Call of Duty. I don’t know. I’m worried they’re going to become deranged. You know, what do we do about this? It’s the opposite.

Justin Skycak (00:57) Yeah, exactly.

Yeah,

right. It’s it’s seen as a problem. But you just change it to like calculus or math. And suddenly, the perception just does a complete 180. But the experience is the basically the same. So yeah, so I was hooked on this. I’m like, I Yeah.

Jason Roberts (01:15) you were having more fun. was more fun than playing Call of Duty or Eve Online.

Justin Skycak (01:20) Yeah, so I beat the calculus game basically, and I’m like, okay, what are we doing next? I’m like full on addicted at this point. I can’t make it through the summer without more math.

You want resisting you to be the hardest thing they're gonna have to do

Jason Roberts (00:00) You want resisting you to be the hardest thing that they’re gonna have to do. Not because you’re their enemy but just because They’re just gonna hey, I like what he’s trying to do. It sounds kind of crazy it’s work, but Okay, fine,

and you just try and make them feel good about it. You’re pulling them along on this journey, Like Lord of the Rings, they’re a hobbit. You gotta come on this journey with me, right? They don’t wanna do it. They’re in their hobbit hole and they’re like, You kinda inspire them and they’re like, all right, it’s easy to go on this journey with Gandalf than it is to resist him. You kinda have to create that dynamic.

Parents are not aware of how little their kids are learning

Jason Roberts (00:00) parents are not aware of actually how little their kids are learning. Whatever you think your kids are learning, in most cases, they’re learning a lot less. They know less, they’re doing less work, they have fewer skills, less understanding. You know, it’s bad. know, especially…

math and physics and stuff like that

if you’re going to make everything mushy and fluffy and we’re just going to have group discussions and projects then guess what? Everybody’s going to do less work. Kids, teachers, everybody. And then at end of the day, people come out and nobody knows anything other than like that one, two kids who were mostly self-taught

PODCAST 2

How Would You Teach if Your Life Depended On It?

Jason Roberts (00:00) I used to play, this game with myself. I would imagine myself as if I were the tutor for this bloodthirsty king for his kids. And he really cared a lot about the education for his kids. And he had already executed the previous two tutors.

You’re the new tutor and you have these two kids and you know there’s a decent chance that after the session’s done, he’d be like, come here, what’d you learn today? And then he starts to quiz them on.

this thing that they said they learned. And if he was unimpressed, he was gonna have your head taken off. Okay, so if you’re in that situation, how would you teach? You’re like, my God, okay, so I’m teaching them how to solve linear equations or something. You’re like, okay, am I going to just talk at them for an hour? No.

with no practice. They’ll totally, because if he asks them to do some, some linear equations and they can’t solve it, I’m dead. So I’m like, okay, here’s what I’m gonna do. Here’s what a linear equation is. This is what it represents. Here’s how you solve it. Now let’s, I’ll go through a couple examples, then I’m gonna have you guys do a couple of examples. And then I’m going to give you progressively more.

challenging ones, negative numbers or fractions or whatever, and we’re gonna kind of build on it, progressive them, we get a lot of practice, but I’m gonna keep raising the level. But it’s always going to be you going through, as a student going through the process, the procedure of performing the skill that I’m trying to get you to acquire. This would be the same situation we go on of your tennis lesson or.

learning how to play the violin or whatever, you know, you’re the coach, the instructor, the teacher is going to have you performing the skill and giving you feedback on what needs to be improved, what you got right, what needs to be adjusted to correctly execute the skill. And so that’s what I did. and, and of course, when you take that attitude and sort of, it’s a way of defining, of describing in a sort of elaborate.

in maybe somewhat bizarre deranged way of extreme accountability. If your ass is on the line, your life is on the line, how are you going to do this thing? Well, you’re probably going to be a lot more serious about it. And you’re probably going to do it a different way than if it just didn’t really matter.

Find Your North Star

Jason Roberts (00:00) when people make decisions purely for the money.

it often leads you in the wrong direction. You end up in a place like why am I so dissatisfied with my life? Why am I so frustrated? It’s like, you’re not doing what you really, really want to be doing. I mean, this stuff can be overplayed. mean, obviously you have to make a living. You have to be realistic about what you can do to actually pay for, rent or mortgage or, you know, get by. but it’s important to just, to really always be thinking,

Justin Skycak (00:03) Yeah.

Jason Roberts (00:30) you know, looking at your North Star like, what is it that I really, really want to do? Go do that.

Justin Skycak (00:35) again, it just comes down to playing the long game, right? need to be making some progress in the short-term, but your goal is not to optimize the short-term outcome of salary, of prestige or whatever.

Do the short term well enough that you can continue playing the long game. That you don’t like run out of money or like you’re on the street or whatever. the long game, that’s what you always want to be optimizing towards. Even if it feels slow in the short term or there’s some pain in the short term, it’s like, whatever, as long as you’re making progress towards the long game, ⁓ then you’re good.

Jason Roberts (01:09) Yeah, well, yeah, 100%.

The thing about it is you get too distracted with the short term and you can lose sight of the long term goal, right? And you get these short term dopamine hits, hey, I made some, I made more money or whatever. Right. And then you look up a year or two years, 10 years later, and you’re like, where am I? And it’s like, well, I mean, you’re making pretty good money and you got a pretty good situation, but you don’t, you’re not really happy with it.

Justin Skycak (01:20) It’s like shiny object syndrome, all this.

Yeah.

Getting “Inside The Trade”

Jason Roberts (00:00) you got to get inside the trade. what that means is I came from a world of high frequency trading.

What would happen is some of these companies would hire really highly educated, smart people with PhDs in physics and computer science and math they would get a group of these people and they would say, here’s our, here’s our, massive historical database of historical time series data of all the trades that happened every minute or every second or every hundred over the second for the last 10 years, go write some algorithms that can predict

what’s going to happen, where the price is going to go in the next 30 seconds or minute or 10 minutes or whatever. And they were almost doomed to fail. typically what would work better is if you’d have a professional trader who would spend years and years trading this stuff manually and understood how you made money.

with a particular kind of trade. It’s like, look, when you’re trading this, when you do this kind of trade, these are the factors to consider. These are the forces that are at play. These are the things you’re to watch out for. This is how you can lose a lot of money. that’s this hard, hard won experience from a lot, from winning and, losing on a lot of trades and learning and understanding. And an emotional instinctual reaction to, yeah, I would not buy at that point.

my experience has been, if you have been the domain expert yourself, you understand exactly how this works.

get inside the trade. Don’t, don’t just automate it. Like do the thing and to understand what the hell is going on. Really understand it. Have, emotional scars from it. Then automate it.

Justin Skycak (01:16) Yeah, and I remember the…

There is some.

Efficient Learning Techniques are Obvious if You Think About Athletics

Jason Roberts (00:00) version one is the expert just demonstrates the thing over and over and over and talks about it. And the student doesn’t get any practice on it. That’s a fail. ⁓

Or two, I’m not going to tell them how to do it. I’m to just say, go and do this stuff. and they’re just flopping around. That’s highly inefficient.

Justin Skycak (00:16) Yeah, and it becomes so obvious when you think of it ⁓ in terms of sports, Like, imagine you sign up for lessons with a tennis coach and the whole time they’re just, they’re showing you all these techniques and stuff. And then like, and then it’s done. Hour passes, like you haven’t, you’re just holding your tennis racket the whole time. You haven’t hit a ball or anything. You haven’t even swung it.

the other failure mode is they just say like, okay, you two versus you two, go play each other and I’m going to go run an errand. I’ll be back in an hour.

Jason Roberts (00:45) This is a dereliction of duty. This is not doing your job. Right? So why in a math class would we expect the situation of the teacher just talking the lecture, right? Which, know, from universities in particular, but typically most schools have done with this sort of a lecture teacher gets up and just talks. Sometimes they cold call on people.

and if they assign homework, it’s like, okay, I’m gonna give you, I’m gonna talk about tennis for an hour and then I want you to go practice by yourself.

You know, and maybe I’ll have you take a video of a couple things, I’ll give feedback. Like that, no, that is stupid. anyway, anyone who has actually tried to acquire, seriously tried to acquire skills in something, sports, music, art, anything like that, where it was important to develop these skills.

They understand that this is basically how it’s done. I mean, there’s a little variation. You can change things up a little bit and whatever, but that’s the core of what a super efficient learning process would be.

Justin Skycak (01:44) guided instruction with rapid feedback cycles of explicit direct instruction on what the student is supposed to be doing.

what’s the proper technique, followed immediately after where the student is actually going and hitting the ball, going through their reps, getting really solid on the skill. They do that on some more skills. The next session, they pull some of those skills together, compound movements.

Enjoyment is a Second-Order Optimization

Justin Skycak (00:00) Enjoyment is a second order optimization. First is performance improvement. Like, get that right. And then without lowering your rate of improvement of progress, just try to make it as enjoyable as possible.

Jason Roberts (00:12) when a parent is spending their own money, or an adult if they were doing it themselves, and you’re paying for guitar lessons or something.

Like that money matters, right? Like the accountability, if it’s not working, you’re going to be like, this is bullshit.

extreme accountability means you’re focused on a result. And, and the result, is a combination of two things. It’s getting better at the thing, but also not making

it’s so painful that the kid or the adult doesn’t want to do it anymore.

If you hire a trainer to get you in shape, the trainer’s thinking, okay, well, I need to keep you consistent, making progress, showing you that you’re making progress and closer to the goals and making it so that it’s not so painful you don’t want to come back tomorrow or next week. Right? Now, if either of those things are false, if it’s too painful, it’s a fail. They’re going to stop coming after a week or two or three or depending on how much…

pain they’re willing to take or how much suffering they’re willing to endure or how stubborn they are. But eventually it’d be like, uh, or two if they, there’s like, man, I’ve been working out with this guy for like three months and I think I’ve lost a pound.

this isn’t moving things in the right direction for I don’t know why I need to find someone to get some results because it’s got a lot of money. It’s a lot of time, you know, whatever. And even if it’s fun, he’s like, oh, he’s like funny guy. All these crazy seal stories from his time at the seals. And be like, are you getting any stronger? No, you lose your weight? No. OK, well.

Justin Skycak (01:35) So it’s like it.

you can’t have that and not have performance improvement. You can have performance improvement and not really have so much enjoyment. And it’s like, it’s a real thing. People will benefit from it.

but you’re gonna increase your surface area if you focus on the enjoyment part after. But the enjoyment part is like the icing on the cake there.

Jason Roberts (01:58) 100%.

Effective Teaching Puts Business First, Fun Second

Jason Roberts (00:00) You optimize too much for the fun and then you’ve taught the students that things are fun and they can’t really be hard.

Justin Skycak (00:00) Yeah.

Jason Roberts (00:10) you gotta be all business.

keep them focused, listening, taking their homework seriously, you know, whatever. you can’t go from being this lax teacher that the kids don’t respect,

You can’t say, right, now I’m serious. I come back from Christmas break, like maybe the principal came down and said, look, I mean, I’m looking at these tests. This stuff doesn’t look good. You got to bang, bang. then the teacher’s like, OK. You’re like, teacher, you’re like, decided you’re to be a hard ass or something.

Justin Skycak (00:37) Yeah,

there’s a directionality to it. You can’t recover from being a pushover. Once you’re a pushover at the beginning and then you become a hard-ass, like, they just hate you more. Yeah.

Jason Roberts (00:45) The kids are like, yeah, right. They just rolled around. I had

happened to me when I was in high school. I can’t remember this teacher. And she, she was like, started out like that. And she’s like, she then she realized she was a young teacher, a first year teacher or something. She just graduated from college and she was going to be our friends and didn’t take her seriously. And because she wasn’t a serious person, you know, and so we didn’t respect her. didn’t like hate her. And then when she started to be like a hard ass, then we hated her. Right.

And she was one of our biology, there were two biology teachers. The other biology teacher who was older, she was all business. There’s no messing around with Susan Radford was her name. She was all business and you’re like, pay attention, you did what you’re supposed to do. And she would lighten up a little as year went on. And then she could lighten up and then you loved her.

Justin Skycak (01:14) Yeah.

Jason Roberts (01:36) Because the most important thing for a teacher is not that they love you, is that they respect you and they do what they’re supposed to do and they take their work seriously. That’s the important thing, especially like middle school and high school, because then kids testing boundaries and they’re 15, 16, you don’t want to be in school. You want to go mess, you want to do anything but sit in your fricking biology class. Right? Even if you’re a good student, you’re like, I don’t want to be here. Right? So that model.

Justin Skycak (01:57) Yeah.

Jason Roberts (02:03) And any really good teacher understands this fundamentally. Not that you can’t do some fun things, but you have to instill respect from day one. I understand you are not to be messed with. And then you can lighten up over time. But there are boundaries, there are expectations, and there are consequences, and there are rewards.

So if you, if you, transpose that onto a learning app and you’re like, we’re fine. And this and dance and baloney and, know, and the kids are like, ⁓ you know, whatever. then you’re like, all right, now we’re going to start learning hard. The kids are like, this is stupid. I it used to be fun. No, it. You know, but.

Justin Skycak (02:43) Yeah.

Jason Roberts (02:47) We can go from the teacher that was a hard ass, and we can lighten it up a little bit. People are like, oh, think my daughter likes it a little more. It’s kind of fun. That’s how it’s going to go, I think.

Get Closer to Doing The Thing That Makes You Happy

Jason Roberts (00:00) if you really want your life to be an A, you’re like, I want my life to be awesome. And even if you don’t have an absolute specific plan, get closer to doing the thing.

that makes you happy or that you feel aligned or whatever, what terminology you want to use. Like this is the thing that I like, I enjoy doing.

Justin Skycak (00:14) Yeah.

We're trying to create the ultimate online learning system

Jason Roberts (00:00) We’re trying to ultimate.

online learning system, basically what that would do is replicate the effectiveness of the best possible human tutor that you can imagine that has almost superhuman abilities to understand exactly what you know and don’t know and what you should be working on. Okay, if that’s what you’re trying to do, then you need to continually keep that person in mind and try and mimic.

their behavior in that situation. And this is a long process and we’re getting closer all the time, but still we have a lot we can do, but you know.

You don't really learn it until you start performing it

Jason Roberts (00:00) you don’t really learn by watching somebody else. You can become familiar with something, but you don’t learn it until you actually start performing it.

Enjoyment is a second-order optimization

Justin Skycak (00:00) Enjoyment is a second order optimization. First is performance improvement. Like, get that right. And then just, without lowering your rate of improvement of progress, just try to make it as enjoyable

Maximize progress subject to constraint that pain is less than quitting threshold

Justin Skycak (00:00) Maximize progress subject to the constraint that the pain is less than the quitting threshold. And you just run that over and over. And the quitting threshold, that can vary over time too. Once you start seeing progress, you’re much more willing

endure some pain for even more progress

Don't lose your soul and become corporate

Jason Roberts (00:00) you have to be careful when you make the transition that you don’t lose soul and become corporate. you can’t sacrifice or give up the essence, which when things become about the money,

and you’re just trying to make everybody feel good, can turn into that. The money will come if you deliver value for people. It’s as simple as that. And since we don’t have any outside investors, we can make that choice, that conscious choice. So we’re not going to give in to…

easy shortcuts and things to just make it nice for, but that while sacrificing results, quality of the education.

Project-based learning can be taken way too far

Jason Roberts (00:00) The whole project-based learning can be taken way too far. Projects are built on a foundation of skills. You can’t do projects without skills and have it be efficient at all. mean, you can. It’s just going to be incredibly inefficient because the students don’t really know what they’re doing.

Kids are finger counting in 10th grade

Jason Roberts (00:00) we’ve heard from a lot of families tutors and teachers that kids

have not mastered their math facts. They don’t know the multiplication tables. They’re finger counting. Not finger counting in fourth or fifth grade. They’re finger counting in 10th grade. And if you can’t do, you know, even multiplication, if you don’t know multiplication tables, you’re gonna really struggle even with basic algebra.

The benefit of learning math with coding applications

Justin Skycak (00:00) a lot of people realize later, who go into software engineering, like they don’t really care about math in school, And then eventually they build up the foundations of coding and then they realize…

that if they just knew all their math, then they could be doing so much more. the earlier you make this happen, somebody gets interested in coding, sees how important math is to doing non-trivial coding. in life the get the motivation to skill up on both fronts. just imagine a kid graduating high school.

not only do they know pre-calculus with like coding applications but also calculus linear algebra multivariable calc and differential equations basically your core engineering math all the coding applications and they come into college and they’re just blowing the socks off of

anybody who gives them an opportunity to do some research, an internship, they already get the basics of everything. They’re ready to actually make serious impact. It’s so rare to see in an undergraduate researcher or an intern, right? you can’t count on them You just like throw them a toy problem.

whatever, but like if you can actually make a serious impact at a young age, because you have the skills to do so, then you can just compound that into a massive compression of time and figuring out what you’re interested in and everything.

You only realize growth in hindsight

Justin Skycak (00:00) I’m doing a hard thing and I’m not sure I’m able to do it at the beginning, but then we actually get through reasonably quickly and now I can do it and that’s really cool. But also do things ever stop getting hard? Like when’s it going to become easy? what they see is the short game.

And the short game is on loop, it’s hard, it’s hard, it’s hard. They don’t always see the compounding so what I would have to do sometimes is I show them like the next assignment and they’re just like, that looks hard.

then I show them do you remember back when we did logistic regression, breadth first search and you had that same reaction right there. What’s your reaction to this now? And they’re like, no, I could that’s a component of this. could, I could code that up in 10 minutes. what’s the big deal? Like, no, no, no. Remember you were saying just like three or four months ago that you were, you were groaning in the same exact way. And I guarantee you.

three or four months from now, this back propagation, this Dijkstra’s algorithm, whatever, you’re have the same reaction. gonna be muscle memory. You’re gonna have new superpowers and you’re not gonna realize that you have them until I show them to you.

It's important to tell younger people the truth

Jason Roberts (00:00) it’s important when you’re in a position where you know how things work to tell younger people, this is the situation. This is the level of talent, the level of skill you’re gonna have to have if this is the thing you want to do. Now if you don’t wanna do it, that’s fine. There’s a place for everyone, everyone has a place. Not everybody has to be a math genius. Not everybody has to learn abstract algebra.

But if you want to do the things that you say you want to do or from this list, then okay, let, let us lay out a plan to help you be on schedule to get that. So you don’t find yourself in a situation where you’re getting blown out of the water

You better know what the game is

Jason Roberts (00:00) You need to understand. look, there’s a lot of things you can do in life that are just not super competitive. It’s not that hard and just kinda go in and just do your thing and it’ll be fine. But there are other things that are just super, super hard. And there’s a lot of competition.

So you better know what the game is.

We're trying to increase optionality for kids

Jason Roberts (00:00) we’re trying to increase optionality for kids. when they’re 12, 13, 14, even 15, 16, they don’t really know who they are what they want to do.

But you want to keep those options open. As an adult, as a parent, or even as a teacher, you’re trying to help kids keep as many doors open as possible so when they get a better idea of who they are what they want to do, that they can get through the door.

Either you make decisions, or decisions get made for you

Jason Roberts (00:00) If you don’t make the decisions, the decisions get made for you. That’s just how it is. Either you go into the world and you make decisions to make things happen, or decisions be made, and they will happen to you.

Turns out if you make things happen, you make intentional decisions, make things happen, you tend to get something that’s much closer to what you really want.

Kids are finger counting in 10th grade

Jason Roberts (00:00) heard from a lot of families tutors and teachers that kids

have not mastered their math facts. They don’t know the multiplication tables. They’re finger counting. Not finger counting in fourth or fifth grade. They’re finger counting in 10th grade. And if you can’t do, you know, even multiplication, if you don’t know multiplication tables, you’re gonna really struggle even with basic algebra.

PODCAST 1

What Happens When Students Don't Know Their Math Facts

speaker-1 (00:00) You know, and I hear from teachers and chief academic officers and tutors, and they’re just apoplectic about the situation because you get kids who are in sixth, seventh, eighth, tenth grade. Finger counting.

don’t know the multiplication tables, it’s like, how the heck are gonna factor quadratic when you don’t know the multiplication tables? Can’t do it, not really. And then guess what? Now we can’t factor quadratic, now can’t do algebra.

speaker-0 (00:27) And if you do manage to grind through, ⁓ just figuring out these factors on the fly, it’s gonna take you way, way, way too long. It’s gonna take you like 10 minutes to factor.

speaker-1 (00:37) Which some kids can do in their head and be like,

10 seconds. So you’re not going to want to do any proms. It’s through, yeah, teacher gave us four factoring problems and it took me all night, you know? And you’re just like, dude.

speaker-0 (00:42) Yes.

When you get to calculus, if you’re taking 10 minutes to factor a quadratic, that’s only one component of a calculus optimization problem. You’re going to be spending half an hour on this problem that should take you two minutes.

The Secret To Success in Life is Consistent Effort

speaker-1 (00:00) The secret to success in life is consistent effort. You don’t have to, whether it’s exercise or learning math or learning a language or whatever the heck it is you’re trying to do. It’s like you don’t have to do the superhuman effort thing. Just get started and then make a consistent push every day, even if it’s only 15, 20 minutes.

Intuition is Earned Through Repetition

speaker-1 (00:00) cause people are like, ah, I don’t want to do so many problems. just want like a conceptual thing. I want an intuitive understanding. It’s like, it’s repetition. Intuition is the

of repetition. You have to do reps. That’s where the intuition comes from. Me or anybody explaining something to you that feels intuitive. You don’t have the intuition. You have to earn intuition. Intuition is earned through pain.

through failure, through suffering, through trial and error. That’s where your intuition comes from. You don’t wanna suffer, well you don’t get intuition. so, and that’s something that we, you know, obviously you wanna limit the suffering, like let’s just do the things and go through repetitions and say, okay, trial, okay, okay, right, I made a mistake there, right, okay, right, I see how this works. You do enough of the reps, whether it’s shooting free throws or…

you know, or doing math problems or whatever, it’s like you have to, you have to get the reps in, you get the intuition. But then when you get the intuition, then you can really understand how this stuff works and you can actually solve challenging problems. Because until you have the intuition, it’s hard to really see your way through innovative solutions. There’s just nothing to work with.

One of the Worst Mistakes You Can Make While Studying

speaker-0 (00:00) One thing that I have seen ⁓ students, even adult students do a lot is look back at the reference too often, like thinking that it’s free to look back at the reference. When in reality, if you are trying to recall something from your head

and you look back at the reference instead of trying your best to lift that weight off of your, long-term memory. If you look back at the reference, you’re basically just letting the spotter lift the weight.

speaker-1 (00:26) Yeah, so you’re weightlifting and you’re like, I can’t get it and the guy behind just lifts it up for you like, okay. He didn’t really lift the weight.

speaker-0 (00:31) Well, yeah.

Sometimes people will ask for the spotter’s help before they even get to the point of trying really hard. It’s like just the moment that it stops becoming super easy, they’re like, okay, look back at the example. When in reality, that is the moment when you are getting the most bang for buck out of recalling the information.

speaker-1 (00:52) The struggle during the active recall process is when you are strengthening the memory.

speaker-0 (00:58) Like, don’t look back at the reference right away. Like I want you to try to pull this from memory.

Coach the student not to actually look back at the reference unless they absolutely need to. Because that is a way that sometimes students kind of shoot themselves in the foot. ⁓

speaker-1 (01:13) They

shortchange the learning process. They shortchange it, right?

speaker-0 (01:17) Yeah. Or if they have like the, worked example up in another tab and then they’re looking at that while solving the problem. Like, is, that is the worst. Never, never do that.

Breadth-First Development

speaker-1 (00:00) It’s painful, but sometimes it’s good to do sort of a breadth first search instead of just like, hey, we’ve got some of the work, so let’s just build on that. We’re like, let’s go tackle these other things that are really, really different. That’d be a ton of work, but it forces us to generalize, to solve those concrete problems and then generalize it and pull it into the model, pull it into the UI, pull it into the user, the student experience, and so it all makes sense because.

If you go too far down the line and you have lots of users and lots of customers and your stuff, you just can’t break things again and go back. It’s it’s hardened. Like that’s where you put the road, that’s where the road is, right? Well, it’s like, well, it goes like, it’s too hard to like, you know, we’re gonna create a new highway through these, you know, through this subdivision. It’s a nightmare. So you just kind of try and figure that out early. the way you do that is you say, well,

here are all these things that we think we might want to do, or we do think we know we want to do, they’re different, it’s going to be painful, but let’s just bite the bullet and do it now. And it slows down visible progress, so perceivable velocity, product velocity, but like, why is it taking so long? like, cause we’re tackling all these really hard, massive projects that is going to pay off and we’re going to be able to release all these things. But.

speaker-0 (01:19) Playing the long game. We are just going in all directions that we think is worth pursuing.

speaker-1 (01:21) Play the long game.

You get a lot of things that are like between 60 and 90 % done. They’re just sitting there and you’re like, God, we just can’t finish with these other things because you do have emergency things going on. You do have bugs, scalability problems. have, you know, just like really important features have to roll out for a user segment. You know, like we, you know, the schools have been coming on and wanting all this stuff. It’s like, know, there’s so much stuff. You just can’t.

You can’t just blow it off. You gotta deal with it. And then it’s like, well, why isn’t this done? It’s like, you know, yeah.

How and Why to Become The Smart Kid

speaker-0 (00:00) We talk about the benefits of pre-learning the material before you go into a normal college course. Cause, it’s a roll of the dice, whether you’re going to get a decent instructor or not.

speaker-1 (00:08) There’s so much variance in the quality of instruction. You’re going to get some people,

great research mathematicians, but horrible pedagogues. it’s you and your group of study mates on these impossible problem sets. It’s more like a

framework for making you teach yourself as opposed to providing a real scaffolded learning experience.

There’s a problem set do it or not or don’t do it.

Grader might grade it, you might get it back two or three weeks later.

You can end up for some real crash and burn situations. If you already know it all, then it’s like, ah! All right,

speaker-0 (00:41) Not only does the pre-learning minimize your risk of that bad situation happening, but if you are blowing the class out of the water and interacting with the instructor, that’s setting you up. You get a, like amazing rec letter, guess who is up for whatever

opportunities that professor has in mind.

speaker-1 (00:57) you should apply for. We got a summer program. whatever. They’re like, ⁓ I got a kid. I got a

speaker-0 (01:02) You just

get a reputation for being the smart kid and it doesn’t matter if you’re being smart in real time or if you’re smart because you’ve already built up a large knowledge base. You’re just you’re a smart kid either way and you get the smart kid opportunities and that can compound into a virtuous cycle.

speaker-1 (01:18) I mean, that’s an incredible position to be in because, for any of us who’ve been like a math or physics major, and especially if you went to a place that had a lot of top-notch students,

you’re learning stuff for the first time and they are going at a breakneck pace and they are not playing games and they don’t get retakes and there are no study sessions, you know, it’s just boom, here you go. And then the average score is a 27 on the midterm. It’s like, jeez, you know, it’s brutal. And then you find out that like a bunch of the kids had actually, oh yeah, I took this at the state university when I was in high school. And you’re just like, what?

What, you guys, wait, half you guys already taken this? This is bullshit. You know, it’s like, we’re in a Spanish class and you got like a bunch of the kids who actually speak Spanish at home. You’re like, why are you in Spanish one? Your Spanish is, I don’t really write it.

speaker-0 (02:09) It’s like you get gaslit into thinking you’re dumb and everyone’s just like learning so much faster than you and then the glass shatters you realize they already came in.

speaker-1 (02:19) Oh, you guys all got the cheat codes? Oh, great. The cheat code is learn the material before you take the course.

speaker-0 (02:23) Yeah.

The point of learning ahead of time is not to sit there bored in class. It’s so that you can actually grapple with the hardest problems and actually extract learning out of those in an efficient way. You can be the go-to person for everyone needs help with the class. You’re that person. You’re getting reps on teaching this material to your friends. You’re making connections You’re the,

front running for any opportunities that the professor has in mind, whether it’s research with them, one of their buddies, internship with a company they have a relationship with a fellowship, you just you never know what it’s gonna be. But if you’re the go to person for the subject knowledge, you just get pulled into all these interesting

advantageous opportunities that just compound one thing into another. Guess what? Got a great internship? Well, your next is probably going to be even cooler because now you have this experience that nobody else has.

speaker-1 (03:21) the it’s snowball effect. The compounding effect.

speaker-0 (03:25) Exactly. And this really buys you a lot of time to figure out what want to do.

You get ahead, you get opportunities, you have time. I can afford to go like, my soul is not connecting with this job. I’m going to go try this other thing that I’ve interested in. Do that for a bit, eventually things merge together into your little niche. And it just

buys you more time to find that because if it takes you too long to find that, then you never actually do find it because you have to pick something.

speaker-1 (03:50) Gotta pick a major, gotta pick a job, you know.

The Cheat Code is Learn The Material Before You Take The Course

speaker-1 (00:00) The cheat code is learn the material before you take the course.

speaker-0 (00:05) The point of learning ahead of time is not to sit there bored in class. It’s so that you can actually grapple with the hardest problems and actually extract learning out of those in an efficient way. You can be the go-to person for everyone needs help with the class. You’re that person. You’re getting reps on teaching this material to your friends. You’re making connections You’re the,

front running for any opportunities that the professor has in mind, whether it’s research with them, one of their buddies, internship with a company they have a relationship with a fellowship, you just you never know what it’s gonna be. But if you’re the go to person for the subject knowledge, you just get pulled into all these interesting

advantageous opportunities that just compound one thing into another. Guess what? Got a great internship? Well, your next is probably going to be even cooler because now you have this experience that nobody else has.

speaker-1 (00:57) the it’s snowball effect. The compounding effect.

speaker-0 (01:00) Exactly. And this really buys you a lot of time to figure out what want to do.

You get ahead, you get opportunities, you have time. I can afford to go like, my soul is not connecting with this job. I’m going to go try this other thing that I’ve interested in. Do that for a bit, eventually things merge together into your little niche. And it just

buys you more time to find that because if it takes you too long to find that, then you never actually do find it because you have to pick something.

speaker-1 (01:26) Gotta pick a major, gotta pick a job, you know.

What Most Online CS Courses Don't Teach

speaker-0 (00:00) quite a courses leave off versus the level that you have to be at to implement the stuff. It’s not an absurdly high level, but it’s just you need to be comfortable with some of this programming logic, not just the syntax.

speaker-1 (00:14) have to have a certain automaticity with it. It’s one thing it’s like, well, I listened to a lecture and I watched a video and I did a 15 minute project with dictionaries. great start, but you’re not, you have not even come close to reaching problem solving level automaticity with these skills.

Math Facts are like Free Throws

speaker-0 (00:00) helpful more practice math facts like bring them out of the problem solving context and drill them more frequently. Kind of like if you’re in athletics, you don’t just exercise your skills by playing games

There are some skills that you just have to be really, really solid on like shooting free throws. You don’t practice like shooting free throw during a game No, you actually go to the line and practice doing that. That’s kind of like, ⁓ math facts, automaticity practice.

Lots of Kids Don't Know Their Math Facts

speaker-1 (00:00) lot of students come to us in pre-algebra and algebra and they don’t know their multiplication tables. And they’re not very good at fractions. And that is very common. Schools not doing a very good job of that.

they’ve drank some cool aid. like, we don’t have to memorize anything anymore. It’s like.

speaker-0 (00:16) Yeah, that’s totally false.

speaker-1 (00:17) That’s a whole nother discussion. It’s totally wrong. It’s like, you don’t have to practice your free throws. You’ll just like, you just know it, you know, just go play basketball. It’s like, what are you talking about? It’s so dumb.

Instructional Quality is a Roll of the Dice

speaker-1 (00:00) so variance in the quality of instruction. even if you go to an elite school, You’re going to get some people,

great research mathematicians, but horrible pedagogues. really it’s you and your group of study mates and that who kind of go, hey, let’s all work together on these impossible problem sets. you basically teach yourselves, It’s more like a

framework for making you teach yourself as opposed to a real scaffolded learning experience.

Getting Gaslit Into Thinking You're Dumb

speaker-1 (00:00) for any of us who’ve been a math or physics major, and especially if you went to a place that we had a lot of top-notch students,

you’re learning stuff for the first time and they are going at a breakneck pace and they are not playing games and they don’t get retakes and there are no study sessions, you know, it’s just boom, here you go. And then the average score is a 27 on the midterm. It’s like, jeez, you know, it’s brutal. And then you find out that like a bunch of the kids had actually, oh yeah, I took this at the state university when I was in high school. And you’re just like, what?

What, you guys, wait, half you guys already taken this? This is bullshit. You know, it’s like, we’re in a Spanish class and you got like a bunch of the kids who actually speak Spanish at home. You’re like, why are you in Spanish one? Your Spanish is, I don’t really write it.

speaker-0 (00:49) It’s like you get gaslit into thinking you’re dumb and everyone’s just like learning so much faster than you and then the glass shatters you realize they already came in.

speaker-1 (00:59) Oh, you guys all got the cheat codes? Oh, great. Okay, okay, okay. Now I get it. I get it. But anyway, the cheat code really is you’re saying, what’s the cheat code? The cheat code is learn the material before you take the course.

speaker-0 (01:02) Yeah.

The Point of Learning Ahead of Time

speaker-0 (00:00) The point of learning ahead of time is not to sit there bored in class. It’s so that you can actually like legitimately grapple with the hardest problems and actually extract learning out of those in an efficient way. You can be the go-to person for everyone needs help with the class. You’re making connections You’re

front running for any opportunities that the professor has in mind, whether it’s research with them, research with one of their buddies, whether it’s internship with a company they have a relationship with a fellowship, you just you never know ⁓ what it’s gonna be. But if you are the go to person for the subject knowledge, you just get pulled into all these interesting

advantageous opportunities that just compound one thing into another. Guess what? Got a great internship? Well, your next internship is probably going to be even cooler because now you have this experience that nobody else has.

speaker-1 (00:49) it’s snowball effect. The compounding effect.

The Secret To Success in Life is Consistent Effort

speaker-1 (00:00) The secret to success in life is consistent effort. You don’t have to, whether it’s exercise or learning math or learning a language or whatever the heck it is you’re trying to do. It’s like you don’t have to do the superhuman effort thing. Just get started and then make a consistent push every day, even if it’s only 15, 20 minutes.

Intuition is Earned Through Repetition

speaker-1 (00:00) cause people are like, ah, I don’t want to do so many problems. just want like a conceptual thing. I want an intuitive understanding. It’s like, Intuition is the

of repetition. You have to do reps. That’s where the intuition comes from. Me or anybody explaining something to you that feels intuitive. You don’t have the intuition. You have to earn intuition. Intuition is earned through pain.

through failure, through suffering, through trial and error. That’s where your intuition comes from. You don’t wanna suffer, well you don’t get intuition. you know, obviously you wanna limit the suffering, like let’s just do the things and go through repetitions and say, okay, trial, okay, okay, right, I made a mistake there, right, okay, right, I see how this works. You do enough of the reps, whether it’s shooting free throws or…

you know, or doing math problems or whatever, it’s like you have to, you have to get the reps in, you get the intuition. But then when you get the intuition, then you can really understand how this stuff works and you can actually solve challenging problems. Because until you have the intuition, it’s hard to really see your way through innovative solutions. There’s just nothing to work with.

One of the Worst Mistakes You Can Make While Studying

speaker-0 (00:00) One thing that I have seen students, even adult students do a lot is look back at the reference too often, like thinking that it’s free to look back at the reference. When in reality, if you are trying to recall something from your head

and you look back at the reference instead of trying your best to lift that weight off of your, long-term memory. If you look back at the reference, you’re basically just letting the spotter lift the weight.

speaker-1 (00:25) Yeah, so you’re weightlifting and you’re like, I can’t get it and the guy behind just lifts it up for you like, okay. He didn’t really lift the weight.

speaker-0 (00:30) Well, yeah.

Sometimes people will ask for the spotter’s help before they even get to the point of trying really hard. It’s like just the moment that it stops becoming super easy, they’re like, okay, look back at the example. When in reality, that is the moment when you are getting the most bang for buck out of recalling the information.

speaker-1 (00:51) The struggle during the active recall process is when you are strengthening the memory.

speaker-0 (00:57) Like, don’t look back at the reference right away. Like I want you to try to pull this from memory.

Coach the student not to actually look back at the reference unless they absolutely need to. Because that is a way that sometimes students kind of shoot themselves in the foot. ⁓

speaker-1 (01:12) They

shortchange the learning process. They shortchange it, right?

speaker-0 (01:15) Yeah. Or if they have like the, worked example up in another tab and then they’re looking at that while solving the problem. Like, is, that is the worst. Never, never do that.

What Happens When Students Don't Know Their Math Facts

speaker-1 (00:00) You know, and I hear from teachers and chief academic officers and tutors, and they’re just apoplectic about the situation because you get kids who are in sixth, seventh, eighth, tenth grade. Finger counting.

don’t know the multiplication tables, it’s like, how the heck are gonna factor quadratic when you don’t know the multiplication tables? Can’t do it, not really. And then guess what? Now we can’t factor quadratic, now can’t do algebra.

speaker-0 (00:27) And if you do manage to grind through, ⁓ just figuring out these factors on the fly, it’s gonna take you way, way, way too long. It’s gonna take you like 10 minutes to factor.

speaker-1 (00:37) Which some kids can do in their head and be like,

10 seconds. So you’re not going to want to do any proms. It’s through, yeah, teacher gave us four factoring problems and it took me all night, you know? And you’re just like, dude.

speaker-0 (00:41) Yes.

When you get to calculus, if you’re taking 10 minutes to factor a quadratic, that’s only one component of a calculus optimization problem. You’re going to be spending half an hour on this problem that should take you two minutes.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.