SPEAKER_00:  So, Justin, I was excited to talk to you. I'm excited to talk to you because you guys have done something which seems impossible, right? Like, if you told me that there would be a website, a system where, deeply adult learners would be excited to go and consistently solve problems in math, that probably they should have learned in high school, mostly, you know, maybe early undergrad. But that we'd be doing this voluntarily at the age of 30 or whatever, and like consistently every day and be excited about that and want to tell their friends. And also, just by the way, tell me that we should have you or someone else from the math academy team on the podcast multiple times. Thank you, everybody, for the suggestion. You told me all of this, you know, a year or whatever ago, I would have been pretty skeptical. But it seems like you guys have done this. There's something to math academy where people are excited to go online. How has that come to be? What do you think is like the quintessential thing about math academy that's made it successful in this way?
SPEAKER_01:  Yeah, well, let me start off by saying that I am also very surprised about all the adult learners that we have that are doing solving math problems consistently every day, so into it posting their progress on Twitter, turning each other on. So we, this is kind of a surprise because it was like, I mean, we started out as a school program, we're building this software to support our in-school program, and it was just kids on the system. And so we just never really had it in our head that it would take off so much with adults, too. But I guess lots of people want to learn math for lots of reasons, and there are lots of adults in that boat, too. But yeah, in terms of the question of what makes it work well for adults, I think it's well, okay, I think there's a couple things that come to mind. The thing that makes it work particularly well for adults is adults are very, very time constrained. So things having to do with efficiency, learning efficiency in particular. It's, I mean, learning, everybody likes learning efficiency. It's good for everybody, but kids generally have like a lot more time on their hands to waste than adults. And so when you're an adult, and you are, especially when you're trying to like learn more, some academic subject, it's like not only are you, are you time poor, but also you have, you have spent, you've probably spent your whole day doing cognitively taxing things. So you're just like, you, you're not going to pull the trigger on something, at least in my experience, you're not going to pull the trigger on something unless it is efficient enough, and it gives you enough bang for your buck to just get you over the hump making, making progress. And I think that's an area that has been a struggle for a lot of adult learners previously with math is just like, well, I mean, you see this in Reddit threads all the time, and like people ask like, well, I want to go learn machine learning, but I don't know the math behind it.
SPEAKER_02:  How do I learn the math? It constantly my problem. I feel like I have to go back to like arithmetic in order to properly do it. And then yeah, exactly.
SPEAKER_01:  Yeah. Right. So you ask on one of those threads, and they tell you like, well, you got to know this, that, that, that, that, that, and then you end up with this stack of like 15 textbooks, and probably only set of seven of them you actually need. There's like some other like crazy like people who tell you need like group theory and abstract algebra for machine learning. Don't listen to that. But but yeah, you'll still end up with this, the stack of a ton of textbooks. And then you're just like, you start working through them and you notice that like it's been a month and you're like 50 pages into the first one that, and that first one is 400 pages long. And then you just kind of do the math. Okay. It's going to take me like eight months to get through this first textbook, like times seven. And then you're like, well, I, there's other things that I would rather do with the next decade of my life. So then yeah, you just kind of lives it.
SPEAKER_00:  Yeah. So it's a, to fill in anyone who hasn't done math academy. And you know, that's not me anymore because I signed up for math academy last week so I could do some research.
SPEAKER_02:  I just signed up an hour ago and I got to question you on the diagnostic. I was like, I'm going to need more time.
SPEAKER_00:  Time diagnostic, Charlie. Get back to it. I know. Yeah. So just like to fill people in a little bit, the way that you can avoid the stack of 15 textbooks, even though like theoretically, many of those topics are on the, on the way is through a diagnostic test, which hopefully like narrows that set of topics a little bit. If it's granular enough, you could say, Hey, it's not the entirety of calculus. You like there are some aspects of this that you need to know that you like don't know, at least according to the diagnostic test. And then once you've narrowed that down, then like you have some measure of if I were to do some amount of work per day, then I'm at the end of this milestone in some amount of time. And so you have some clarity then in a way that you wouldn't, if it was just a Reddit thread being like, you need to go back and start from middle school and do everything in middle school. You know, you directly know some of it. And that takes as long as it takes this some like attempt to narrow it down and then give you clarity on where you're going to go from them.
SPEAKER_02:  The diagnostic plops you in on this knowledge graph that looks like it goes from fourth grade all the way through graduate school. Is that the idea? So I can take a diagnostic and it can appropriately plunk me around based on which of those individual concepts I need extra help on. Is that is that what the diagnostic is doing?
SPEAKER_01:  Yeah. So the right, so the the the diagnostic basically just places your knowledge profile on the graph. You take a diagnostic for a particular course. It's not like we suss you over like the entirety of math and we we kind of expect that you know, generally where you are, you're not confused whether like, oh, I, I am ready to study multivariable calculus, but actually like, oh, I need to learn how fractions work. At least for right now, we kind of expect that you're going to try to enroll in the appropriate course. But we do like back for prerequisite knowledge that you might be missing. So like, if you sign up for a calculus course, then we're going to be looking back like, okay, do you know your your trigonometry? Do you know your your algebra? Are there things that you're missing? If so, we'll detect them. That's okay. We'll we'll add that to your learning plan. And yeah, it's just just tracing out your knowledge profile in a section of this big graph of of mathematics that currently I think I think currently, yeah, so it starts at fourth grade and and the highest level that we have out right now is methods of proof or calculus based probability and statistics, multivariable, we have a math for machine learning course that that comprises of that contains a lot of universal level math. But yeah, we're continually expanding the curriculum. It's going to cover like all of an undergrad math major adding actual machine learning courses. Actually, we're currently developing a machine learning course that that goes through like the actual machine learning algorithms, having students learn back prop from scratch, working problems by hand. We're going to have a second machine learning course also that that that that actually gets into like transformers and stuff. So yeah, we're continually expanding this this this graph.
SPEAKER_00:  So everything in education, I feel like is a trade off, right? So it's if you go down the track of like we're making this as efficient as possible, you need to trade something else off of that. Otherwise, like every program would just be as efficient as possible. And Kate, Kate, came to that because like who doesn't want to be efficient. What do you think is the like the trade off that math academy makes or like what is the audience for whom it's just not is not going to be a good fit?
SPEAKER_01:  So yeah, so the trade off is well, the people who it's not going to be a good fit for are the people who just want to a kind of very low effort mentally easy sort of just for fun like like imagine somebody who just kind of like goes on to YouTube and is like, oh, I want to like learn some some interesting math. But what they really mean is I want to watch somebody talk about some interesting math with some pretty shapes on the on the screen and and not really have to work out a bunch of problems or anything, but just just kind of like get my brain tickled a little bit. That's that's not who we are. And also also people just want to like maybe solve like one or two math problems a day for like I'm a train to work or something for like a couple minutes. So the thing is we we're like a the way we phrase it is we work on mathematical talent development. So we we train we train our students using the same kind of techniques that that a coach would would use to train an aspiring professional athlete or musician or something. Like it's it's it's it's serious training. It's it's not just like for for or for like somebody who wants to appreciate music or like appreciate it's for it's not a math appreciation software. It's like a actual sweaty workout. Like you you have to have like some some reason I hear wanting to to have some you have to come to it with some intrinsic motivation.
SPEAKER_02:  Is that what you're saying?
SPEAKER_01:  Um, I don't know that it has to be intrinsic because there's plenty of people who want to learn math and service of other things that they might have like it's a common thing with with machine learning. People who are interested in machine learning as they might have intrinsic motivation for maybe like coding and they they find the AI part of of computer science to be really cool. But they realize like oh it has a bunch of math that they never really liked at school. I'm like, oh fine. I guess I gotta go do this math. And sometimes that softens a little bit and they're like, oh, I guess you know this this actually turns out to be like, it's not so bad. I actually don't hate it. Or maybe I like it a little bit. Or maybe maybe they do find some intrinsic motivation. They find some beauty in it. But yeah, it doesn't require intrinsic motivation. It just requires consistency and committing to it. Just like it's just like a workout at a gym. You don't you don't have to like be intrinsically motivated to go to the gym to work out. You can set up some kind of extrinsic extrinsic structure. You can be doing it for maybe not because the you can be doing it maybe to have like a beach body over the summer. Like there's a bunch of different things that'll work in terms of getting you into the gym and and working out. And as long as you're you get yourself in there and you're doing an effective workout and you're doing it consistently, then then then positive changes are going to happen.
SPEAKER_02:  It's I mean, I envisioning this like a skill tree in a video game like Diablo two or something where I'm start out as a weak warrior. But then I could be this sort of God-like Paladin at the end. And if you follow along, you can fill this thing out. Do you show the trajectory from a UX perspective? Like you are here on your larger journey against this map or is that too much info?
SPEAKER_01:  So currently we show a knowledge graph of the course. Like so you can see kind of your maybe there's like 300 topics in the course and you can see as you complete topics that kind of fill in. But I mean, that's still just a very small subset of our entire graph. And so we that's one thing we want to improve on in the future is just like really putting some some leverage behind our knowledge graph visualization. Just really getting that front and center. Not just showing the course, but like showing where you are in general, like in all of mathematics, showing a time lapse of like, hey, this is where you were one month ago. Like, now look at all the tasks you did over the past month and like, that your knowledge graph kind of fills up and everything. Every task that you do have had show like what what topic is being hit in the graph and also what topics are receiving implicit review from that task that the subscills that you're practicing everything. Also, if you if you take too much time away from the system, showing your knowledge kind of decaying kind of peeling back fading. It's yeah. So we want to really kind of show this as like a almost like a visualization of a student's mathematical brain like like a like an MRI of students math brain, but but yeah, we we we definitely want to lean into that that more and lean into that that kind of as you're alluding to the the gamification side of thing, just like level yourself up.
SPEAKER_02:  Yeah, I mean, it feels like a fingerprint. You know, everyone has your fingerprints and it's like, this is the fingerprint that led me to this particular level of mastery. And that's gonna be very
SPEAKER_01:  different from Ozzie's or yours. So that's really neat. Yeah, yeah, exactly. Exactly. It is exactly like a fingerprint. And and that's I mean, that's that's that's one of the reasons why it's so hard, like in typical classes to when you have like, I mean, in a, say just the typical school, we got like 30 kids all in the same class, and they all have this like their own fingerprint of a knowledge profile. They're all missing different pieces of prerequisite knowledge. And somehow, the teacher is supposed to give like one lesson that magically fills in calibrate to all these individual fingerprints. It just it just doesn't work so well. So yeah, that's that's, I mean, that's another reason why the why this is some kind of like is able to work so efficiently is because it's just every decision is calibrated to that that fingerprint. And I ask that question,
SPEAKER_02:  because I feel bad I haven't fully done the program yet. I'm going to. What is the, what is the core experience? Is it like solving problems that are perfect for you in this moment? I think about other interesting things. Sure. There's Khan Academy examples where like maybe the core thing was like charismatic whiteboarding videos. You know, what what is it about math academy that is the hook? Is it is it the problem based learning that with space repetition where you're always working on something that's, you know, just perfectly hard for you? Is there a
SPEAKER_01:  you? Yeah, that's exactly it. Exactly. Every moment in time that the student spends on the system is intended to be spent working on something that is just at the right level to have them moving as fast as possible too easy. And there's no point too hard. And there's no point you get it right right at the right level of difficulty where they either maybe that means they they know the prerequisites for the topic and they're ready for a new lesson on something that haven't learned or maybe it's some review some reviewing some information some some problems that they had that they had done the lesson previously, but it's been a while and now their their memories faded a bit. Whatever whatever it is, we're always trying to use the students time as effectively
SPEAKER_00:  as possible. That's yeah, that's it. Can I add something to that characterization which I think I'm coming both in favor of math academy and also like against for those who don't resonate with this. I feel like the basic structure as far as I've seen so far is you have a worked example or maybe two worked examples and then two questions and then worked examples and then and then questions like this is the this is the kind of overall structure. Yep. Now worked examples are fantastic. Like there's a lot of there's very little support for anything in educational research, but there is a lot of support for worked examples. It's like if you see somebody step through how you solve a problem, then that helps you solve approximate problems like problems that look just like that one. So like providing a worked example and then a question is a is a good like is a good overall loop. This is how you do this kind of thing and then now you do this kind of thing. This like this is very different to the Khan Academy. Well, I mean in parts of the Khan Academy structure does keep that, but in other parts it's more like well let's explore some interesting ideas in this general space and connected to other topics, whatever. The math academy is like no, we're not necessarily going to, or again from what I've seen exploring it for I think I've just heard a few hours on it. It's not so much like let's provide motivating context. Let's see how these examples connect together. Here's why you might be interested in this topic. I wanted to ask about this. I want to immediately jump to suppose you would like to take the derivative of the inverse trigonometric function here. The way that you would do that is this, now you go do that. It's like it supposes that you already have the interest in solving this kind of problem and directly without any fanfare or inefficiency shows you how to solve that kind
SPEAKER_02:  of problem and then you solve it. Okay, that's really good because I wanted to ask about this. I used to grind integrals in high school and I could crush them. I could not do that now and I was thinking back to it. I was like well maybe if I knew, and I don't even know if this is true, but integrals is about measuring volume of water in jars or something. Maybe I would have retained that in some way and I think that's probably just fantasy that having that additional context would have helped me retain it in some way. But it sounds like you're not providing like why you'd want to learn derivatives or why you'd want to learn this or giving that like context.
SPEAKER_01:  So okay, so here's the thing that the context comes but it typically comes after the baseline like mechanical skills have been laid. And more context is coming in the future that is not here currently. I'll elaborate on that. Okay, so the mechanical skills stuff. So here we have a lot of, let's take that calculus course for example. We have a lot of problems that we have topics in fact that have to do with using integration to measure volume of things. So like volume of water with an integral, the very concrete meanings of integrals. Many, many, many different real life scenarios. But it's true that we do not start with a real life scenario when we are teaching integration. We start with the kind of the most simplified mathematical idea of just like area under a curve. And then we gradually expand on that. And the reason for that is that we've been building the system in just trying to get just the raw elements of like learning efficiency in there. So building on prerequisites. And so if you are going to motivate a subject or motivate a topic, motivate some kind of operation with a scenario like that where the actual application that you're talking about, the student is not able to actually do that until they built up a lot of prior knowledge. It takes care not to kind of paint yourself into a corner where maybe you over promise like, hey, by the end of today, you're going to do something amazing with this cool math that you're going to learn. And then they like learn like a little bit of it. And it turns out not to be as amazing as you promised. Because I mean, the amazing this comes, but it comes after more practice. Or you end up having to kind of wax philosophical on like how like, the connection between measuring like the volume of water versus integration is not is not is not like self evident to students, you have to explain that. And sometimes that it can take like a lot of explanation, especially if a student is kind of like struggling to get it. And so you kind of can get students zoning out. So basically it just have to be very, very, there's a lot more areas where you can make a pedagogical mistake if you're not careful. Now expert teachers are good at this, like they can they can often like kind of introduce something in an engaging manner, while not spending the half the class like just trying to explain the connection or so like they move things long efficiently. They don't over promise, but they can use some of these cool examples to maintain interest while moving students along. And that's that's something that we we want to kind of bake into our system more in in the future. It's kind of like just, you know, it's any any just imagine like a conversation with like a really cool tutor who who makes you really excited about the subject. And maybe that involves like having some historic historical anecdotes or maybe it's like some some some applications that you're interested in. It's like maybe you're like,
SPEAKER_00:  maybe like this to me is the interesting thing where like you guys, I think in a way have really made the most of the medium. It's like if the internet is very good, or you know, if if a software system is very good at like calibrating an exercise and providing a canned worked example, and then like some dynamic, you know, feedback system of like here's the next good question for you. Like maybe that's what you should really focus on and do that very well. And if you have to say, okay, well, to an extent, we expect that you bring your own motivate motivation or motivating context. So like you shouldn't necessarily need that. This is the system where if you don't need that you do very well. It's like you kind of carved out that niche. And it's like in a way, the medium, you know, how is it that people are very excited about math academy, but people are also very excited about Grand Sanderson? It's like, well, Grand Sanderson does the the inverse of what you guys do. Whereas like no practice, no problems, but it is exciting. It does provide the motivating context. He makes the most of YouTube, like the YouTube recommendation algorithm keeps giving people who want exciting math videos, Grand Sanderson videos. It's pretty blue one round. The people who don't know that connection, all three people. The like the the fact that he can just do that and focus on that and you guys can do the like, now that you have the motivation, the problem things, maybe that's a good thing. And like you shouldn't necessarily like, like maybe you should just put a sticker on the front of it being like, this is like a gym where you come ready to train. And like evidently, there are already a lot of people who are in that mindset. Whereas like I've been waiting for someone to just give me directly the like sequence of problems
SPEAKER_02:  that I need to get into machine learning. I think that works for adult learners, but it works for the school systems where there's teachers and these other external, maybe they're externalities, but external systems where you have to like adhere to them. So I don't know, I guess math academy, you're serving a lot of different populations that I can imagine the product forking based on the sort of like adult versus like within the institutions of education type problem space where maybe you have to build more teacher tools, where it's like, how do I manage a classroom of people doing a flipped classroom blended learning type approach? And then we have to like have teachers have that bird's eye view of what people are working on in some time. And I feel like you go in a lot of different directions. I used to work at a company that was doing stuff like this 10 years ago. And that teacher insight what ended up being a
SPEAKER_01:  big area where we were spending a lot of time. Yeah, yeah, I mean, that's that is also on our on our radar. We actually taught some classes with, yeah, on the math academy system. And yeah, we had like a little elementary dashboard up monitoring the student progress and everything. And yeah, I'm sure, yeah, we've got a couple schools, a couple other schools that they use the software. And yeah, it's going to be more of a thing in the future. But I mean, so ultimately, the thing is this we want to we want to get to the point that we are just like the the ultimate math learning platform. And I think a key ingredient of that is going to be, I mean, we're layering these things on starting with learning efficiency. But we also want to like smooth out the learning process to like, we've we've we've optimized efficiency kind of, it feels like pretty close to the to the level like, it's it's hard to just to squeeze any more juice for more efficiency. The where the juice comes from now, I think is is really the keeping people getting people excited about doing the work and keeping them consistently doing the work. And so some incentives, gamification, mechanics, and some like, I mean, I think some of the like contextual, like historical anecdotes and stuff like, I think that can be very motivating to a lot of learners. But of course, like we would not want to clutter up the experience or slow down the learning. It'd be a very delicate balance. Yeah, definitely wouldn't want to go to the point of like being like a YouTube video or something. But I think there's some dose.
SPEAKER_00:  That's that's the interesting question, right? And like, what I was hinting at at the beginning, when I was like, there's always a compromise in education, how do you trade off with the efficiency? And it's like, if you were to start to cate it to that other need, can you do that in a way without compromising like what's what's working? Like, one thing that I that I came to understand better as I taught more is that the motivating context for one student is very different to the motivating context for another, where I thought that I was getting better at delivering a motivating context. When in reality, I was better at catering to a particular group of students who resonated with that, without context. So like, over time, I developed more of a kind of historical role playing approach in class, where particularly for esoteric systems concepts. So like, you know, why is the virtual memory system that we use in Intel derived architectures the way it is? It's like a weird set of things that is worth knowing. But without the historical context, like, why would you bother winning this? Like, how could you possibly understand the complexity of this thing? And so we would have this historical role playing imagine, you know, as the 1980s, and you were trying to solve this kind of problem, you want to provide this kind of memory protection on the system, how would you, what would you do that? And then people students would provide ideas and would come to like an understanding of where the trade-offs are and how you could have designed the thing that has become a legacy. And we do that five times and you end up with the current system. Now, I thought this was fantastic. Of course, you know, I rated my own teaching. And for some students, it's like, Oh, I finally understand this thing. And for other students, it's like, we just spent an hour developing the motivation that I already had. Right. Other students now are as excited to learn this thing, which I just wanted to learn because it's come up on my job previously. Or I just wanted to learn because the textbook says that that's the next thing that I should learn. And I'm a good student. And I learned the next thing in the textbook without requiring extra motivation. And so like, I thought I was getting objectively better and better at teaching based on mostly like designing the classroom experience that I would have wanted myself. The thing that may help me understand this and a little bit of feedback from the students with whom this approach resonated. But the more I listen to the feedback from the other students, the more I realize it's fine before or even maybe better before based on their kind of interpretation of. So like, I'm saying I'm giving this long story because I actually don't know whether like it is something that is resolvable necessarily. And whether the best approach is to try to find you know, the solution that is a reasonable solution for everybody. Or just like, like I said, put a sticker on the front being like this, if you are this kind of person, this is going to be the perfect product for you or perfect solution to your problem. Yeah. So here's the thing that I don't
SPEAKER_01:  think it has to be like. So what the dose of of motive of like contextual information of historical anecdotes or whatever, like, that's another thing that can be like adapted. But it's like, it's another thing that like if you're a human teacher, right, like you either have it in the lesson or you don't. And different learners have different motivational profiles, just like different knowledge profiles. And so you're you're you're you're matching or not matching to different profiles. But like if you're in an adaptive learning system that's just completely individualizing it, like maybe maybe for some students, like, yeah, we just the motivational, our approach to motivation for them is to just strip out all like historical anecdotes and everything because they don't want that because they just want to like just don't like I just want to how do you do the problems? I want to just blow through that as fast as possible. And so that would be enough to say if you guys
SPEAKER_00:  could solve that problem, that'd be very interesting in itself. It's like, could you give someone a diagnostic where at the end of it, you actually know that learn their, can we call it learning style? We know the great learning style and call it something else, but like,
SPEAKER_01:  motivational profile, maybe. Yeah, motivation style, something like that. Yeah, yeah.
SPEAKER_00:  If you can give them questions, which is which are like, oh, is it interesting to you that trigonometry came from like measuring the distance to the moon or height of mountains or whatever, you know, degree of interest in this kind of thing. Yeah, yeah, they're interest profile.
SPEAKER_01:  Yeah, I think, I mean, so that's something that we're, where I totally see on our, on our roadmap in the future as we kind of build out more more of the product. There's other things that's, that's coming first, like we're actually working on a, on a streak mechanism right now to try to help learners keep consistency and using the product. But, but yeah, where I'm excited to, to kind of try to, at least try to get our arms around that problem. Can I ask about,
SPEAKER_02:  can I ask about the map a little bit? Yeah, that's only, I feel like every example of adaptive learning that seems to be working really well is in the mathematics space. And there seems to me to be this, like the, the concepts themselves are atomic enough to have a well-defined problem around them. You can clearly map the prerequisite relationships between these atomic concepts. And then in my own experience in mind, it gets a little bit fuzzier as you move to other subjects where those lines are a little bit blurrier. And it's harder to know exactly how to tailor content at that atomic level, but then also do this sort of prerequisite tree. Have you all thought about other subjects where this could apply? Is that it's called math academy, obviously,
SPEAKER_01:  if you need a rename, but actually, yeah. So, yeah, so the, the grand plan is to expand outwards from math. But yeah, there's a limit to how far outwards, like, I don't really see us doing history courses any anytime soon or like English literature, like, not, I mean, so our focus has been math for years. And recently, we've started expanding out into math the computer science, like machine learning and we're, have started to work on just like a intro to programming course. And we're, we'll be doing like a lot of applied math stuff, like I'm sure quantitative finance and a lot of physics in general. So, yeah, we'll be definitely expanding outwards from there. I think the, the, the, the, the, the places where it starts to get a little fuzzier are, are maybe the less inherently mathy sciences, like, like biology, for example, I mean, you can make biology really mathy and maybe, maybe we'll have like a quantitative biology course at some point in the future for people who are interested in that sort of thing. But, but what we have, what we ever have like a cell biology course, I don't know, it requires
SPEAKER_00:  another cell biology course. I think it'd be great to have a cell biology. There's actually a fantastic book called cell biology by the numbers, which is like, basically saying like a lot of molecular biology is taught in a way that's too conceptual. And actually you should do a lot of, there are a lot of like back the envelope kind of calculations to build more on intuition, or what's actually happening. And so the entire course is on this basis. It's like, well, how many proteins in a cell, you know, what's what is the flow outside the cell, whatever, like, let's actually calculate this. We've got some basic numbers. I mean, the math is not complex, but it's like the use of quantitative approaches to biology that's actually like,
SPEAKER_01:  yeah, that's cool. I agree. Yeah, I mean, that'd be a core. I would probably enjoy taking that kind of course, like, yeah, almost like a, yeah, like a mathematical applications to biology sort of thing. Or maybe that just even sort of shows up as like, math, like kind of, like, applied projects that are tacked on to various math courses that way, or something like that.
SPEAKER_00:  Yeah. There's an interesting conflict there. Like, it's, I mean, I'm not necessarily a conflict. Maybe they're complementary as well. We're like, you guys done execute program,
SPEAKER_01:  or do you know, execute program? Yeah, yeah, I'm like, I'm familiar with that. I haven't
SPEAKER_02:  actually done it though. I turned out I was paying for it for the last six months, which I've recently resolved, but now I can, which now allows me to do the same thing for math
SPEAKER_00:  academy. So, execute program is like a, it's a similar approach, really, to programming. And it's very good. And it's very different to say as primer, say, or just other resources in computer science, where it's like, you have this spectrum of like, legible to illegible, or like, structured to unstructured, where the more you go on the structured side, the more you can build systems that are around like knowledge graphs and space repetition and so on, but Gary's very into knowledge graphs. You know, he puts it front and center, like, I don't know if it's on the homepage, but it's like, it's very easy to see the knowledge graph that is taking you through, and it's very into a space repetition. And like, if you really buy into that, you end up teaching the portions of the field that are easy to structure that way, which are, it's useful to know that it's important to know that. But if that was the extent of your understanding of programming, then now you're very good at recall of APIs and so on. You're very good at the things that are easy to explain, like, in that atomic form. And that is not enough, like, it's not sufficient for practice as a software engineer or understanding of the field. So like, that's not to say you shouldn't do it. Like, I think it's very good. I think execute program as a product is fantastic. And learning all these things is very useful. So it's just a question of like, again, what's the label on the front of the website? Is it like, through this, you understand all of it? Like through the quantitative biology approach, are you going to be a biologist, like, and clearly know that in that case, is it maybe missing from the typical way that somebody learns biology? Yes, like, there's a kind of gap, I think. And there should be a course like that. But it's like, I don't know, I think as an adult learner, it's easier to say, you know, I'll pay this much money per month for this thing. And there'll be a piece of my understanding. And then, like, I'll supplement it with this other thing. Like, I'll watch the Grand Sanderson video and do the math academy calculus course. And like, I will, I will mix and match what I need to learn. I think that's a, that's a kind of good thing. Like, if we're all kind of clear with the, with the student, with the user about what we're providing and how we should into the landscape,
SPEAKER_02:  that's all good stuff. Oz never watches sci-fi or anything. But I'm always reminded of the, I think 2009 Star Trek when they show Planet Vulcan. And they're like, in those learning pods. Have
SPEAKER_01:  you seen this Justin, this movie? Oh, I, I'm not sure I have, but I'm familiar with the idea.
SPEAKER_02:  It's just, it's showing Vulcan and like Spock is a young Vulcan. Oh, wait, wait, I know, I know what you're talking about. And it's like he's in a hole and there's like a robot throwing questions. It sounds like Oz is suggesting math academy stays on that trajectory, which is, I don't,
SPEAKER_00:  I don't know the answer. Yeah, I don't know the answer because it's a really interesting kind of, like, how much, how much could you go outside of the, the thing that's clearly working for math academy and have it work for more people who come with a different kind of motivation. Or does that like water down? I don't, I don't know. I think that's a really interesting space.
SPEAKER_02:  I'm curious to see how it goes. I have a slight tangent to ask about Justin. Oz and I have been noodling ideas about self publishing adventures and whatnot over the course of last year or so. And I noticed you have a bunch of textbooks that it looks like you've published. I'm just curious, you know, your thoughts on that, like, are they self published? What's your, it's quite awesome that you have a bunch of math textbooks. So I'm just, I'm curious about the story behind those
SPEAKER_01:  and what your plans are with them. Oh, yeah. Yeah. So I, it's kind of started just kind of after college where I was just, I just as a hobby. I was, right, I just thought it'd be cool to write up a math textbook. Just kind of, I don't know, I was doing a bunch of tutoring and, and, you know, once you, once you sort of leave college, you kind of feel your, your memory decaying of this stuff, especially if it's, like, you study something like math in college, then you stop using it as intensely as, as before. So I, yeah, I, it gave me kind of pleasure to, to, yeah, to just write a pet textbook as, as, as, as, as though I kind of wish that the, that were presented to me when I was first learning it. Now, my understanding of that has actually evolved a lot, so I would not consider my, my earliest math textbooks to be, like, amazing pedagogically. I mean, hopefully, they're, they're at least better than some others out there, but I would not call them the best and they'd pale in comparison to anything we have on math Academy.
SPEAKER_02:  What should you view on, like, you know, the textbook is like stamped in time. Obviously, you can have like an online version of this and you've moved to something that, you know, you can push code changes a hundred times a day. Would you still think about, I think a textbook as a,
SPEAKER_01:  an effective medium for mathematics learning? Honestly, honestly, no. I, yeah, I, I just, I mean, so the thing is like a textbook, I, I, so much of learning efficiency. I, I, I've learned is coming from adapting like every pedagogical decision to the student, what are they working on at this moment in time? And so if we talk about like breaking up a textbook into different slides and presenting those slides of content at the right time, which is, I mean, kind of starts to go in the direction of what, what math Academy is, then I would say like, okay, now that's, that's something that sounds more interesting to me, but like, yeah, even if a textbook is really easy to edit and stuff, I personally, I, I wouldn't be motivated to work on it just because I would feel like, it was constantly a lot more efficient with, with math Academy. But that, that said, that said, I think if, if I were writing a textbook on something that like, that is just very niche, not going to be covered by, by math Academy, it just like, like, like in teaching a course or, or something and, and yeah, just more niche content or something that I just wanted to do my own presentation of, outside of, of math Academy, then, then yeah, editability, easy, ease of iteration would definitely factor into that because, yeah, writing a textbook is very, is, is, yeah, it's very time consuming and, yeah, of course, lots of provisional, yeah,
SPEAKER_02:  you know, that. Well, I got a, I got a shuffle on, and get back, get back to the swing of things. But Justin, this was amazing chatting with you. I'm really excited to dive in. I have always wanted to, the movie Billy Madison, where he goes back and starts kindergarten and like goes through it each grade in a week. I've always wanted to do that with Khan Academy and it's never stuck. I think largely because I felt like compelled to watch the videos. And I feel, I feel pretty excited about math Academy to kind of go through and, you know, start from second grade or fourth grade and move on and, and win back some of those skill points that I know I had in high school that I've lost and I just, I feel bad about and I want to get them back. Yeah, yeah, well, I, I hope,
SPEAKER_01:  I hope you enjoy it. Yeah, I think, typically, once you kind of, yeah, get the wheels going, I guess it's just like working out where it's like, it might feel a little hard.
SPEAKER_02:  It's for me, it's like when during the day am I going to do this? Do I like, you know, see exactly. It's just working out. What are you going to do it? Exactly. We have to lay out the outfit before, like go to bed in your workout. I might have to do that like go to bed with a calculator in my pocket or something.
SPEAKER_00:  Yeah. Yeah. It was great that I do now is like, we have math time. It's like, I mean, I've been doing this for a while, but it's like, I sit down, you've got Beast Academy, I've got math Academy, like you do your problems, I do my problems. That's how I justify the time.
SPEAKER_02:  Nice. Okay. Actually, I need to, I need to work on that. Okay. Well, talk to you all later. Thank you. Great talking to Charlie. Okay. Bye, Justin. Bye.