Transcript - Scraping Bits Podcast #107: Proof Writing, Discovering Math, Expert Systems, Learning Math Like a Language
Why aspiring math majors need to come into university with proof-writing skills. My own journey into learning math. Math as a gigantic tree of knowledge with a trunk that is tall relative to other subjects, but short relative to the length of its branches. The experience of reaching the edge of a subfield (the end of a branch): as the branch gets thinner, the learning resources get sh*tter, and making further progress feels like trudging through tar (so you have to find an area where you just love the tar). How to fall in love with a subject. How to get started with a hard subject that you don't love: starting with small, easy things and continually compound the volume of work until you're making serious progress. How to maintain focus and avoid distractions. The characteristics of a math prodigy that I've tutored/mentored for 6 years and the extent to which these characteristics can be replicated. How Math Academy's AI system works at a high level, the story behind how/why we created it, and the stages in its evolution into what it is now. How Math Academy's AI is different from today's conventional AI approach: expert systems, not machine learning. How to "train" an expert system by observing and rectifying its shortcomings. How to think about spaced repetition in hierarchical bodies of knowledge where partial repetition credit trickles down through the hierarchy and different topics move through the spaced repetition process at different speeds based on student performance and topic difficulty. Areas for improvement in how Math Academy can help learners get back on the workout wagon after falling off. Why you need to be fully automatic on your times tables, but you don't need to know how to do three-digit by three-digit multiplication in your head. Analogy between building fluency in math and languages. #1 piece of advice for aspiring math majors.
Cross-posted from here.
Want to get notified about new posts? Join the mailing list and follow on X/Twitter.
Below is a smoothed version of the raw transcript.
DeGatchi: GMGM, everyone. My name’s Digachi, the host of Scripting Bits. For the first time ever on this podcast, we’re doing a repeat with Justin Skysack. For those that haven’t listened to the last podcast, it was Episode 102: Learning Mathematics Like an Athlete. We’re back with round two for the sprint after the marathon.
Just for a small intro for those that haven’t listened to that episode—who are you, and what do you do?
Justin: Yeah, sure, happy to be on again. Great conversation last time. My name’s Justin Skysack, as you mentioned. I’m the Chief Quant Director of Analytics at mathacademy.com. I built all of the quantitative software that drives the Math Academy learning engine for an automated online learning system.
DeGatchi: There you go. We probably won’t get too deep into it since we spoke about it on the last episode. If you haven’t listened to the last one, go check it out and come back, or you can just stay and skip the intro. Let’s start off with proof writing.
On your platform, you have proof writing, and I’ve been reading How to Prove It: A Structured Approach by Velleman, I think his name is. After you finish, let’s say, Foundations II or Foundations III—or even just starting university-level stuff—it’s super important to learn proofs.
In school, they tell you to show your work. Proof writing is that, but in the formal sense. Let’s say you make a discovery and want to solidify it as a truth—you’ve got to make a proof. It’s like the mathematical version of programming. Instead of creating a program to prove it, where there may be underlying edge cases you haven’t hit yet, a proof solidifies it. You attack it from all angles. It’s like a program that doesn’t run, in a way, and you can actually learn a lot quicker. I’m not too sure how, because I haven’t done it yet, but I have the sense that it’s like that.
Justin: Yeah, I would agree with comparing it to a program. When I taught proofs in the past, I’d introduce them as a program that runs in the mind of someone with mathematical knowledge. That makes it difficult for a novice to debug proofs sometimes.
When you write a program, the computer runs it, compiles it, and spits out errors. But when you write a proof, the “compiler” is a more advanced mathematician, like a teacher, who reads it. If you’re a novice, it can be difficult to debug your own proof because you need someone with more advanced math knowledge to parse it and be your compiler.
DeGatchi: That’s actually a good analogy. When you put it like that, if you’re using mathematics or you’re not a seasoned expert, you can look at these very elegant proofs. It’s like reading a well-edited book by a seasoned author. You pick up on their techniques, their style, and what makes it a great book. When you’re reading that book and understanding it, you can do the same with math—reading proofs from advanced experts.
For example, you can study something like the Pythagorean theorem and see how they reached their conclusion. You can read the whole process, understand how their mind worked to arrive at that discovery, and eventually pick up how they think. Then you can form your own style and personality.
Justin: Totally. We see that in coding all the time. There’s a joke that the best way to understand what’s going on in someone’s head is to read their code.
DeGatchi: There’s not always going to be documentation. You have to read the code, and that’s actually the documentation because it represents how they think and communicate their ideas. I hadn’t thought of this before, but it makes proof writing even more important to me. Reading very advanced proofs will lead to very advanced thinking. I hadn’t realized that before, but now I’m even more set on reading proofs. It’s just like reading a book, picking up their ideas, and translating that to math. It’s like reading a poem or any kind of writing to understand someone’s ideas. That’s a great way to think about this episode.
Justin: One interesting thing about proof writing is that, at first glance, some proofs don’t have many symbols in them. They might look approachable, but then you start reading, and your mind gets bent and demolished by all the logic in the proof.
If you haven’t trained yourself in proof writing and you just open a math textbook to read proofs without many symbols, it can still feel like a rollercoaster. Proofs are like a coding language—a very esoteric one. They often use language in subtle, specific ways with precise mathematical meaning.
Sometimes, these same forms of language are used more loosely in conversation. If you read a proof with conversational language in mind, it can be confusing. This is even more justification for learning proofs instead of jumping into the deep end and hoping to swim.
DeGatchi: I wouldn’t even say it’s just about reading proofs. Many math books explain the math in sentences, and you don’t want to sit there wondering, “What does this symbol mean?” Then you forget the sentence you just read and get stuck, having to reread it. It’s a waste of time, doing things twice.
Math is a language, and just like Japanese or Mandarin, the symbols represent something. They also have corresponding spoken words. It’s like two languages, and that’s kind of what math is like. I guess it’s very similar to Japanese.
Justin: Totally agree. If you’re trying to read math and keep getting interrupted, wondering, “What does this symbol mean?” or “What does this phrasing mean?” there’s no way you’ll be able to see the forest for the trees or understand the whole proof. It’s just like reading. If someone doesn’t know the definitions of various words or is struggling with sounding them out, there’s no way they can read a Shakespeare play if they’re stuck on individual words.
DeGatchi: Exactly. You have areas where this is a large component. If you don’t know symbols for intersection, union, subset, element of, etc., and you’re reading this kind of logic and reasoning where symbols are embedded within words, you won’t read fluently, and you won’t understand the whole thing.
It’s like reading a book. You’ll forget the story and fixate on something, then go down a tangent, like typing into ChatGPT, “What does this symbol mean?” You spend five minutes trying to understand one symbol, and then you have to read the whole thing again.
I think it’s super important. I’ve heard a lot of math students don’t study proof writing for a long time, and it holds them back later on.
Justin: That’s something I’ve seen over and over. People come out of high school solid on the mechanics of calculus, thinking they’re all set for college. They’re excited, maybe planning to major in math, and they get into a serious university, signing up for the hardest math courses.
They were at the top of their class in high school, did well on AP or IB calculus exams, or other college-credit calculus tests. Then they get to university and just get their ass handed to them. There’s a level of mathematical maturity often assumed for students coming into a math major, especially those pushing hard to take advanced classes.
You might find yourself in a class where proof writing ability is baked into the prerequisites, but you’ve never actually gone through it. You don’t know what a proof is, you’re not fluent with unions, intersections, or other proof mechanics, and you feel like you’re drowning.
Last summer, I talked to someone who was in this situation at the University of Chicago. They entered as an aspiring math major but got their ass handed to them because they lacked proof-writing foundations. Their classmates often came in with a wealth of knowledge, having done extra reading outside of the normal school curriculum.
These classmates not only knew calculus but also topics like groups, linear algebra, and multivariable calculus—very advanced material. University courses are often built for that kind of student. They don’t slow down for students who still need to learn what a union or intersection symbol is.
This person ended up switching majors to physics. It wasn’t because they enjoyed physics more but because the playing field felt more level. Beginning physics courses required less background knowledge. Many physics derivations are more mechanical—you start with a formula and rearrange symbols to get to the result.
In pure math, proofs often involve not just symbolic manipulations but logical rules of inference, which can get complex. Their reaction was, “I wish someone had told me in high school that this game was being played, and proof knowledge is part of it.” Just because you excel in high school math doesn’t mean you’ll be at the top of your class in college. The people sitting next to you often have additional background knowledge, largely from proof writing.
It’s something super important to learn before university. Graduates from a serious math program will have proof-writing experience—or at least they really should. Many people get filtered out of math majors simply because they didn’t know proof writing beforehand and found it hard to catch up on their own.
DeGatchi: It seems like trying to program without understanding the language—or like trying to write a novel without knowing English. That’s what proof writing feels like. It’s the punctuation, the wording, combining words to create sentences—the foundations of the language.
For some reason, nobody wants to learn the language. I guess they see it as inapplicable, but it’s relevant to every part of math. You can’t avoid it. It’s weird how people try to.
Justin: Right, those set manipulations you were talking about—they show up in pure math, applied math, and even other fields. It doesn’t matter; they’re just part of it.
DeGatchi: Any kind of query is just a set manipulation, really. That’s what it seems like. Hmm, it’s interesting. Your journey is interesting too. You went from computer scientist to math software developer at Math Academy. What was that transition like?
Justin: Actually, before computer science, I came from a math background. In high school, I was super into math. You know how I described those students in high school who are crazy about math—learning linear algebra, multivariable calculus, proof writing—going beyond calculus? That was me. I self-studied about three-quarters of an undergrad math major during high school using MIT OpenCourseWare.
When I got to college, I majored in math. At the same time, I started getting into coding more seriously and began working in data science. About a year after graduating, I found Math Academy. That’s when I got involved.
Initially, in 2018, Math Academy was experimenting with videos. Our first concept of guided lessons involved video content. I made hundreds of educational videos and wrote a lot of content. By 2019, I started working with Jason on the code side, building the automated system.
For me, the transition was easier compared to others who start in computer science and want to move into math. I came from a math background and transitioned into computer science.
DeGatchi: I see. One thing I wonder about is how to progress with math. With programming, it’s easy to know what’s next. After finishing a project or learning a topic, you expand to another area, build another project, or add new files that connect to the main one. That’s how you learn things like multi-threading or transitioning from single sequences to parallel sequences.
With math, it feels unclear. I’ve never learned a language before, and there are so many branches. How do you decide what to do next? How do you go beyond your skill level? Do you read more advanced books?
Let’s say someone finishes Math Academy—what do they do next? Do they just try to read some advanced book they might not understand and figure out how to learn from it? What’s the next step?
Justin: It’s a great question. Initially, a highly structured curriculum is vital. Math Academy covers everything from fourth grade up to university-level math. But what do you do when you reach the other end? It all comes down to what you’re trying to accomplish.
Math is a very branching subject. As you work through the foundations—set manipulation, proof writing, calculus, algebra, and so on—that core knowledge forms the trunk of the tree. Once you have a solid undergraduate math education, you face an existential question: what’s next?
At this point, the trunk has branched out so much that, even if you aim to be the world’s greatest mathematician, you can’t be the best in every single area. There are too many branches. You have to start deciding what you actually want to do.
This can be difficult for people who followed the standard curriculum without considering what interests them beyond it or why they’re learning math. You face a fork in the road—really more like 20 different forks—and have to make a decision.
I recommend building up your foundations while keeping in mind your goals. What is your goal in learning math? What are you trying to achieve? Is there a specific field in computer science you want to explore, such as building autonomous agents? Do you want to be an academic mathematician working on theorems about prime numbers? You need to decide what you want your future to look like.
Once you know your goal, you can start picking up books and working on projects in that direction.
DeGatchi: Right. So I guess when you finish the trunk of Math Academy, every field relies on that foundation. That’s why it’s like the trunk. Once you finish it, you should be able to pick up any book and understand it. Is that what happens?
Justin: I’d say loosely, yes. Loosely, that’s correct, but even with the trunk of knowledge, there are books and resources that may be too specialized. It’s still a gradual process. You often have to scaffold yourself into a subfield, starting with some introductory material.
DeGatchi: But you can understand the introduction because you have the foundations, right?
Justin: Right. Having the mathematical foundations puts you in a place where you can read introductory material in a niche field of math.
DeGatchi: And then you get to the point where you can reset again.
Justin: Exactly. The tricky part is that, at the lower levels of math—elementary school through calculus, even the university-level math covered in Math Academy—it’s scaffolded really well. You don’t have to worry about what resources to use, what order to follow, or how to review. Everything is structured for you.
You just get on your computer, do your XP, solve the problems we tell you to solve, and you’re all set. But when you finish all the math in the system and it’s time to specialize in a subfield, you have to start managing your own learning process.
That’s a skill in itself. It’s not about math knowledge anymore; it’s about keeping yourself accountable, solving problems, doing projects, and scaffolding them in difficulty. You can’t start with something too hard—you need something at the right level to build from.
It gets more challenging as you approach the edge of a subfield. The resources available to teach yourself get worse because, at the cutting edge, people are just publishing research papers. That’s what you have to learn from. It takes time for cutting-edge knowledge to become well-scaffolded and accessible.
DeGatchi: Right. The further up you go in the branch, the thinner it gets, and the resources just get worse and worse, making it harder to learn.
Justin: Exactly. Thinner, sparser, and shittier.
DeGatchi: That’s right. This is such an important skill because you have to direct your own learning to get to the cutting edge. With programming, you can keep building, optimizing, and eventually reach the edge. With math, you rely on others who are good at communicating knowledge, and then you have to build the next level on your own. At that point, you need to be a creative thinker because no one is there to help you—you have to do it yourself.
Justin: Right. In any field, once you reach the edge, it becomes about creative production. That’s related to acquiring new skills, but it’s very different because you’re creating new things.
DeGatchi: Totally different game at that point. That’s super interesting.
Justin: It’s interesting. The first time this happened to me was when I learned a lot of math and got really into a subfield of machine learning, particularly neural nets. This was back in 2014, and convolutional neural nets were pretty much the cutting edge of machine learning. Now there are tons of courses on how to do convolutional nets, and it’s very well-structured. If you want to learn about them today, it’s easy to do so.
But just ten years ago, the resources for learning that kind of stuff were so sparse. I got really interested in machine learning and neural nets and hit the edge. It felt like trudging through tar. It was so hard. After learning from MIT OpenCourseWare—calculus, differential equations, probability, statistics, and other topics, which had a lot of information available online—it felt like progress slowed to a crawl. Those resources were well-organized enough that you could make good progress if you were motivated.
But once I hit the edge, it became really hard to find textbooks or structured material. It was like walking through tar. I had an existential crisis, wondering if this was really what I was interested in. It turned out the answer was no. I got tired of trudging through that tar. I just wasn’t committed enough to the neural net stuff.
DeGatchi: When you go into a niche, you really have to be interested to explore and be creative. Otherwise, you just burn out because there’s no help. It makes you think—are you truly passionate enough to keep doing this?
Justin: Interest becomes very important because that’s what motivates you to keep going despite the tar. You have to say, “Okay, I’m trudging through tar, but I love this thing. I love the tar.” It has to be worth it. It’s not just about moving fast and acquiring knowledge—it’s about really caring about the subject.
DeGatchi: I think that’s why I started preaching that people should actually start to enjoy mathematics, really love it, and see the beauty in it. For me, learning math feels like learning a language. I’ve never really learned a language before if you don’t count programming, which I think is just modified English.
The only way I see to motivate yourself is to find the beauty in these fucking shapes and random symbols. I don’t know how people can stay “motivated” all the time without falling in love with what they’re doing. Why continue if you’re not going to be passionate about it? People can have goals, sure, like saying, “Oh, this will help me do X, Y, Z.” But for a grand goal, I think that quote—”fall in love with the journey, not the destination”—applies. If you enjoy the journey, there is no destination. You’re already there. It just continues, and eventually, you find yourself at the edge of chaos, becoming a pioneer in some sense.
That’s how I started to view math. Just do it every day. I’ve seen this happen with other things too—if you spend all your time with drug addicts, you’ll probably become a drug addict. Spend time with millionaires, and you might become a millionaire. Spend time with math all the time, and you’ll probably become a mathematician. There’s something in that, right? It’s about immersing yourself in it.
The hardest part is just starting each day. Once you start, you naturally get into a flow state, and then six hours go by, and you think, “This is the greatest.” It’s the same with anything. Starting is always the hardest part.
When people make New Year’s resolutions, they sign up for the gym, go for two weeks, and then stop. It’s like there’s this initial burst of motivation that tanks shortly after. From your experience teaching and learning, how do you stop that from happening? I know it’s simple to say “just start,” but are there structural approaches to make starting easier? How can you make opening the gate a bit smoother to ease into it?
Justin: I would totally agree that often, to really fall in love with a subject and develop passion, it’s not love at first sight. You have to spend time with it and develop a habit of engaging with it.
How do you get started with something when you don’t have that passion at the beginning? Especially when it’s hard and taxing on top of that? It’s asking a lot. How do you get yourself to start when the reward is unknown, it’s very hard, and you don’t love it at all?
The answer, I think, is to start small. To use math terminology, you do an epsilon amount of it. Find ways to reduce the barriers. If it’s very hard, make it easier. You don’t have to do it for a long time at first.
For example, with fitness, if you haven’t been exercising for months and have been sitting all day, day one isn’t about going to the gym or running five miles. It’s about going for a walk—a 20-minute walk, 10 minutes, or even five minutes. If even five minutes feels too hard, go for a one-minute walk. If that’s too much, walk to the end of the room and back. Do 10 jumping jacks if that feels manageable. Whatever the lowest bar is, set it there.
Then, each day, raise the bar slightly. For example, if walking to the end of the room and back is your starting point, do it twice the next day. Then four times the day after. Soon, you’re walking for a few minutes. After the first week, you might find yourself thinking, “This is boring. Why not walk outside?” Then, you’re walking around the block.
A week later, you’ve built up to a 10-minute walk. Eventually, you might think, “This is getting boring—why not jog instead?” Now you’re jogging a couple of minutes, then scaling it up until you’re running for 10 or 20 minutes, covering miles. If you tire of running, maybe you switch to push-ups, pull-ups (or hangs if you can’t do full pull-ups), or other calisthenics.
This approach works for anything, not just fitness. For math, for instance, if someone is trying to get started, don’t jump straight to problems at the edge of your ability. Solve a few problems in areas you already know. If it’s arithmetic, start with multiplication or times tables—even fourth-grade material.
It might feel silly at first, but you’ll get into a rhythm. Once that feels easy, you’ll naturally do more next time. Over time, this compounding effect leads to exponential growth in what you can accomplish. You trick yourself into starting with something easy, develop comfort with it, and then build up to serious work.
DeGatchi: By analogy, when you go to the gym and you’re first starting, it’s super shit. But then you get to a point where the muscle doesn’t hurt anymore, and it gets easier. You don’t want to lift the really light weights anymore. You want to be on the edge of chaos—just below your max, where you can still rep it out. You feel the pump, you’re trying hard, and then the next day you’re a bit better because your body adapts. You go a little heavier, and it’s an infinite loop as long as you keep the habit.
The key is keeping that momentum through habit and routine. When I changed my schedule to waking up at 5 AM, skipping rope first thing in the morning, taking a shower, and then doing 30 deep breaths instead of a 30-second cold shower, it became a repeatable routine. The trick is creating a routine that you don’t even have to think about.
With math, it’s the same. Start small. Do multiplication for one minute. Make it as easy as possible to start. When I begin skipping, the hardest part is literally picking up the rope. Once it’s in my hand, I want to swing it around, and I think, “Well, this is boring—let me jump.” Then I put on some music, and I’m into it.
When I’m trying to do math, I sit at the computer and look at something at the edge of my knowledge, like differentiation with multiple variables. My brain says, “I don’t want to do this right now,” and I don’t. But if I just did one minute of multiplication or five minutes of something repetitive, I’d warm up. Then my brain would want more stimulation, and I’d chase something harder.
It’s like how junk food, porn, or social media hijacks your brain’s dopamine system. They raise your dopamine requirement, making it harder to get satisfaction from important tasks. Cutting back on these bad habits helps.
People in places you aspire to be don’t necessarily have more than you—they have less. They don’t watch porn, spend hours on social media, or indulge in bad habits. The solution is doing less of the bad things.
I tried sitting in a blackout room, staring at a wall, to test what boredom would do. With a book next to me, I got so bored that I started reading it. Pain becomes preferable to no stimulation. Now, I go to bed two or three hours without blue light—no computers—and leave one book by my bed. When I get bored, I read that book.
It’s important not to have too many options. If you have ten books, it’s harder to pick one. With one book, you’ll read it. Boredom, combined with one option, forces you to do the hard thing.
The internet is the ultimate distraction. It’s easy to type in a site like x.com. Web blockers can help, but if you’re a good addict, you’ll find ways around them. What I’ve noticed is that removing bad habits makes good habits easier, and they eventually replace the bad ones.
Justin: That’s a really good point. It’s not just about how easy something is in an absolute sense—it’s also about what it’s competing against. What other actions are competing for your attention?
You can try to make the task you’re trying to do easier, but if you’re in an environment where it’s even easier to do something you know you shouldn’t, you’re sabotaging yourself. Part of it is setting up your environment to minimize your choices or manipulate yourself into taking the right action. It’s like your past self is manipulating your present self.
DeGatchi: For example, if you’re at a very formal business event where everyone’s in suits, you’re not going to scream out loud because there would be instant consequences. You’re in an environment where the social norm dictates certain behavior.
You can apply that principle to anything else. You have to create the social norm for yourself. If you’re in a math class and everyone is talking about math, you’re going to do math to compete, stay engaged, and feel part of the norm. It’s about not standing out in a negative way. That’s how I see it. The question really becomes: how do you remove the bad things so only the good things are left?
Justin: That reminds me of how social study clubs often work well for people. For instance, you join an online Discord server where they host study sessions. Everyone does math for 20 minutes, chats for five minutes, then goes back to math for another 20 minutes.
There’s a social norm in place that funnels you into productive behavior, even if it’s not something you would have done on your own.
DeGatchi: Ultimately, it’s about comparing yourself to the person you want to be. That person doesn’t even have to exist—it could be a future version of you. What does that person not do? Compare yourself to that. Theoretically, if you got rid of all the things you’re not meant to do, how much better would you be?
I think that’s a much better framework than just figuring out what you have to do. Everyone knows what they have to do. Deep down, you know you’re supposed to do the good thing instead of the bad thing. It’s really about figuring out how to stop doing the things you’re not meant to do. That’s a counterintuitive way of thinking, at least for this generation.
By the way, I read your blog. You mentioned mentoring a math prodigy, and I’m curious—what makes them stand out? What makes them a prodigy? Is this something you can learn or develop over time, like unlocking your inner prodigy?
Justin: That’s a good question. Unfortunately, the answer comes down to individual differences. The kid I’ve been mentoring has some traits that are advantageous for math. He’s loved math from an early age and is super interested in it.
The way I met him was back when I was doing more tutoring, about six years ago. His parents were looking for a tutor to keep him occupied for a few hours each week. He was constantly doing math experiments and mini-projects on the properties of numbers. He would talk non-stop to his parents about math, and they felt someone needed to engage with him mathematically.
His parents described him as being “amped up on math,” like how some kids get with sugar or dessert. When I started working with him, I’d say, “That’s a cool experiment. Did you know this relates to a property in arithmetic?” I’d give him problems to explore further, which naturally led to algebra and calculus. Over time, we covered a wide range of math fields.
He benefited from being very intrinsically interested in math from a young age. Neither he nor I know exactly why he fell in love with it. That’s the big question—why did he love math so much? I’m not sure if that process can be replicated in someone who doesn’t already love the subject. Some people fall in love with math later, but the process that led this kid to immerse himself so deeply might come down to how his brain was “initialized,” so to speak.
Another advantage he has is cognitive differences. I know some people feel uncomfortable discussing things like IQ, but differences in working memory capacity, generalization ability, and forgetting rates exist. These can also be domain-specific.
DeGatchi: People are naturally better at some things than others.
Justin: Exactly. Not everyone’s brain is the same. People have advantages, just like some have physical advantages that make them better suited for certain sports—different phenotypes. It’s the same cognitively.
This kid has very high generalization ability, particularly in mathematical and quantitative reasoning. His forgetting rate is shockingly low. Working with him feels different compared to other bright students who aren’t at the same prodigy level.
When you go through problems or introduce new material to him, it’s like reminding someone who’s just rusty on something they’ve learned before. It’s the strangest thing. Imagine you’ve forgotten some math from high school, but when you review it in a textbook, you say, “Oh, yeah, I remember this.” Now, imagine you’ve never seen that material before, yet your internalized and generalized knowledge base integrates it as if you’re just refreshing an old memory. That’s what it’s like with this kid.
This speeds up knowledge acquisition. Typically, when working with a student, you might cover material in a session, but if you wait a week or two, much of that knowledge has decayed. You have to reteach it. It goes faster the second time, but you still have to repeat the process.
With really bright students, they retain more, so less time is spent reteaching. But with this student, his memory fades so slowly that it’s almost negligible. After a week or two, you can pick up right where you left off with almost no friction.
All these factors compound, accelerating his progress through math.
DeGatchi: Really lucky genetics, basically.
Justin: It’s a lucky roll combined with leaning into his interests and a lot of hard work. None of this would happen without the hard work. But it’s a lucky roll where the hard work feels natural because he’s built the habit, he’s good at it, and he’s so in love with it that it’s part of his identity. This is who he is.
There are helpful factors that can definitely be replicated, but there are two things that are hard to replicate if you take any random person and try to get them to the same level. First, how do you instill a love of mathematics that early? Second, they likely don’t have the same advantageous cognitive differences for math.
The hard truth is that most math learners experience a lot of friction during the learning process. There are ways to reduce that friction—developing more interest in the material, doing review, mastery-based learning, spaced repetition, and developing a habit. These approaches can accelerate math abilities well beyond the status quo. But to reach that prodigy level, I think it comes down to finding an area where you have a real competitive advantage.
DeGatchi: You could develop that interest deeply if you find something you really like about it. Maybe it just comes down to problem-solving. I’ve changed careers multiple times—crypto, trading, math, AI, robotics, biology—I’ve done everything. But it’s all encompassed in one thing: problem-solving and wanting to understand something deeply that hasn’t been uncovered yet.
Maybe there’s some generalization and creative thinking involved. Like linking things together—hitting ideas against each other until they align. That’s at least how I approach it. Being persistent takes you a long way, too. Resilience is something you can learn.
You can go far with basic skills. Creativity, for example, is a skill. It’s about combining as much knowledge as possible from different fields into one thing. It’s not sectioned off; it’s open and interactive. That’s a skill—using a book, writing your ideas down, and linking everything together later. Creativity is just combining and banking ideas.
Justin: I should say that even if a person may not reach the level of prodigiousness of someone else in the field, paying attention to what fuels great people in that field—consistent, efficient practice habits, not messing around during practice, and a deep love for what they do—can still get you very far. If you replicate those things in your life, you can go much further than anyone would expect.
If you don’t have any other competitive advantages, such as cognitive differences, the question remains: how good can you get? Probably not as good as the best in the field, but you’ll still get pretty damn good.
DeGatchi: It’s not an even playing field. If everyone did the same amount of work and had the exact same lives, genetics would play a big role. But even some people who aren’t genetically superior in some ways don’t have the basic fundamentals of resilience and persistence. The real magic comes from people who have those traits, even if they aren’t as genetically inclined.
The prodigy is the combination of hard work and genetic advantages. But anyone can follow hard work and get to a place where even people with the lucky dice roll don’t get, simply because they totally commit.
Justin: Right. It’s the saying, “Hard work beats talent when talent doesn’t work hard.”
DeGatchi: But then you have those people who are insanely good, you know, six or seven, and they can just do everything. That’s interesting. It seems like it’s a really deep passion for and desire to learn, which compounds as you get better. The more you improve, the more you want to understand, and the more you realize you don’t understand because you know there’s so much. That makes you want to go even more. It’s basically just finding positive feedback loops.
Yeah, anyway, that’s exactly right. Let’s jump over to the AI side of things, as that was the focus of this episode. With Math Academy, you’re behind the AI system, which presents the reviews, the lessons, the multi-step spaced learning, and the lessons to do at specific times instead of giving you a manual curriculum. It presents it through knowledge graphs. If you don’t finish or fail a lesson or don’t get a proper review, it pushes you back to a prerequisite. If you fail that, you go back until you pass, then move up.
A few questions, but first, explain it briefly, and then we can dive into the details.
Justin: Sure. The ultimate goal of the system is to get you learning material in the most efficient order possible. Some things are obvious, like doing prerequisites before more difficult topics, but there are other things that are less obvious, like how often you should review material. That depends on how well you did on them initially, how strong your knowledge is, and how much review you’re getting from other tasks.
For example, if you’re solving a linear equation, like 2x + 3 = 4, you’re implicitly reviewing something simpler, like 2x = 4. A two-step equation encompasses a one-step equation as a sub-skill. You have to take that into account when deciding how much you need to review. If you do a two-step equation lesson, that counts as your review for one-step equations because you’re reinforcing that sub-skill.
It also comes down to tracking nuances with review, which you have to monitor through the knowledge graph and the prerequisite structure of mathematical topics. You also need to pick tasks that leverage all that information to ensure an efficient path through the material.
So, not only do you track that information, but you make active decisions. For example, this student has five topics they need to review soon because they are in their spaced repetition cycle. These reviews are coming up, and there’s a lesson they know the prerequisites for, but they haven’t done it yet. However, this lesson encompasses those five reviews they need, or maybe it covers three of them fully and the other two partially. This allows those reviews to be pushed into the future, optimizing time.
The goal of the system is to minimize the amount of work the student has to do, while ensuring they receive all the spaced reviews they need and are in a position to successfully acquire new knowledge. It’s an optimization problem—minimizing effort while adhering to the constraints of review cycles and successful learning. Essentially, it solves the math problem of optimizing the learning experience.
DeGatchi: I didn’t notice this. I don’t explicitly do old reviews either. But I think it’s just natural in math because, for example, if you’re dealing with fractions and doing differentiation, fractions are baked into the differentiation, so you’re reviewing fractions anyway. You’re also reviewing their properties.
It’s not like you have to go back to the basics and review them. Since math is hierarchical, it’s all based on the same concepts. You could be reviewing five different things with just one problem. I don’t know—something like quadratic fractions in differentiation later, it’s all part of the same thing at some point. You could optimize this quite heavily. It’s a good problem to solve.
But how did you go from scratch to formalizing this model—space repetition and hierarchical node structures? What was that process like in creating this system and making it something you could then apply?
Justin: This started back in the summer of 2019, when Jason, founder Ian, and his wife, Sandy, founded Math Academy. I started working with Jason in the summer of 2019. He pulled me into the coding side because he knew I had experience in data science, and this seemed like a data science problem or at least a quantitative programming problem.
Previously, I had been doing content-related work for Math Academy, but Jason had this idea that Math Academy needed to become an automated learning system. The goal was to emulate the behaviors of an expert tutor. It didn’t need to emulate every single behavior, like asking how the student’s day was, but there were specific decisions made during the learning process that made a tutor so much more effective than a teacher who had to teach a class of 30 students with different knowledge profiles.
How do we replicate those decisions made on an individual basis? Jason had done a lot of reading about spaced repetition, mastery learning, Bloom’s two-signal problem, and other techniques. At the time, I wasn’t as familiar with these learning techniques, but I had come across similar principles during my self-study, like knowing I needed to review things occasionally and not jumping straight to the end of the textbook. I didn’t have the specific terminology for it, but I understood the concepts.
He pulled me in that summer, and that’s when we started building the system. Initially, it was very simple—a raw spaced repetition system. There was no trickling repetition credit through the knowledge graph, and we didn’t have encompassing prerequisites. It was just a basic system: you either passed a lesson or you failed it. If you passed, it was added to your review queue for spaced repetition, and you could do higher-level lessons as long as you passed the prerequisites. There were no diagnostics, and we were starting from scratch.
We put one student on the system, an original student from Math Academy’s in-school program who had moved out of state and was sad about not being able to take the in-person classes anymore. We asked if she wanted to try out the automated system. She was an eighth grader taking Calculus BC on our system.
She was our initial test student. She worked through it, and it was rough at first. Occasionally, things would break, and she was getting more review than was necessary, so she did more work than needed. However, it was still very efficient compared to a typical in-person class with 30 other students. At least, it was doing mastery learning and spaced repetition. It wasn’t perfect with spaced repetition, but it was doing things that wouldn’t have been done well otherwise.
She came out of that year with a 5 on the exam—no teacher, just the automated system. That’s when we realized we could lean into this more. We could improve the efficiency of spaced repetition by tracking the flow of credit through the graph. We began focusing more seriously on that in the following year.
The summer of 2019 to 2020 coincided with the pandemic. Right as the student was getting ready for the AP Calc test, the pandemic hit. This forced our hand. We had already been thinking about adding more optimizations for efficient learning, but now we had to figure out how to continue teaching students remotely.
The natural answer was to scale up our system so it could support our in-person classes, which were now going remote due to the pandemic. This was a huge undertaking. I actually moved in with Jason and Sandy during the pandemic and worked with them every day, including weekends, to get the system ready for the fall. We needed it to run the in-person classes remotely.
We had to make the system more efficient and more robust. Previously, we had been storing spaced repetition records as separate database records, which worked fine until we needed to make changes to the model. Then, everything had to be recalculated, and it became unwieldy. That’s when Jason told me to create a model class that dynamically calculates the student’s knowledge profile on the spot, rather than storing everything in the database every time.
I started taking over the modeling side, and by fall 2020, the automated system was ready for the in-person classes, which were now going fully remote. What ended up happening was that the automated system worked so well that students were learning even more than before. During the fully remote instruction, students using the automated system covered more material and achieved a greater degree of mastery than they did in the in-person classes.
DeGatchi: What was the process of actually designing the architecture? Yeah, it was a ton of designing, and I’ve seen the train as well. Is that correct?
Justin: Actually, the AI we use is an expert system. It’s based on a math company where we’re domain experts in the subject of math and teaching math. Everything we want the model to do, we pull out of our heads. We think, “This is what a good tutor would do. This is what an expert would do,” and this is the information they would use during their decision-making process. Even something like the knowledge graph with prerequisites and encompassing records—this is all manually set by us.
When you do that, you don’t have to train a model to figure it out. You just give it the parameters it needs because you already have those parameters in your head. This depends on the problem being human-scale. For something like a computer vision problem, where you have to detect objects in images, you can’t manually set all the parameters. It would require immense domain expertise. But in our case, the amount of information that needs to be encoded into the model is a lot, but it can be manually encoded in this knowledge structure.
That’s how we got around the need for model training. We don’t have to train the model or have it learn information. We just set up the model’s structure and initialize it with baseline, reasonable parameters that we know from experience will work. We observe the behavior, make sure it’s going well, and calibrate those parameters over time as we get data.
So, we don’t actually do machine learning in the conventional sense. We don’t use large language models or anything like that. It’s more like doing physics—having some intuition about how parameters should be, then conducting experiments to refine that intuition. The specific type of AI we’re using, an expert system, was more popular in the 80s and 90s. Today, AI is generally associated with large language models, neural nets, and machine learning, but there are other subfields of AI, and we’re using one of those.
DeGatchi: I think I’ve done the same thing. You’re so good at something that you just made it up. I called mine heuristic-based. I didn’t even know that was the name for it. That’s super cool. Initially, I thought, “How the heck is it generating questions and telling me all this stuff?” But if it’s manual, then I get it. But yeah, that stuff takes so long. That’s why it actually got me into AI—I had to learn to improve my own model. But I had a side project that was really different.
Okay, interesting. So there’s no real training. How do you test it, find edge cases, and make sure it’s actually working as you want it to work? That’s super hard. I did it, and it’s not a fun process.
Justin: For us, it started with our one test student who went through the course and did well. Then we just scaled up with more students, paying very close attention to the tasks they were being served, how well they were doing, and using quizzes. We had the AP Calc BC exam as our external validation metric.
I also created various validations to run against the database to ensure things were logically consistent. For example, a student shouldn’t be getting a lesson again if they’ve already passed it, unless they failed it in the future. We check these kinds of logical conditions, and I have hundreds of validations to make sure everything is on track.
DeGatchi: That’s a big part of it. I imagine a basic validation would be, you know, serving a lesson, then reviewing it a week later. If they fail the review, give them the lesson again. That’s a basic condition.
Justin: It’s various things like that. To give a concrete example of how we refine the model, we were always looking at student tasks and hearing feedback from students and teachers. One thing we noticed early on, I think it was in the 2020-2021 school year, was that we had a student doing pre-calculus level work, I want to say integrated math three, or maybe algebra 1. They had a task that involved matrices, learning the elements of a matrix and how to multiply matrices.
We noticed that they kept getting reviews on identifying the elements of a matrix, even after they had learned how to multiply matrices. We were looking at this student’s task and thought, “This is dumb.” Most things were working as intended, but there were too many easy reviews. A tutor wouldn’t go through something like that. After learning how to manipulate matrices, a tutor wouldn’t ask, “Can you point to an element of this matrix?” The student already knows that. They’ve layered so much knowledge on top of it.
That was the moment when we realized we needed encompassing topics in the database. We needed to know what topics encompass others, and the model had to reason about that. The space repetition needed to trickle down the graph. This realization came from just looking at the student’s tasks. Jason pointed it out to me. He said, “This kid is wasting time on these reviews. Spending five minutes on this review is a waste of time.”
It came down to what you were saying earlier: don’t do the bad things. If you want to be good, to perform well, just don’t do the bad things. One of those bad things is giving students silly reviews that are too easy, making them ask, “Why am I doing this?” Then they ace it and just keep repeating it.
DeGatchi: That’s interesting. For reviews and lessons, would you categorize them, for example, like, “This is using reciprocals and complex numbers,” and then it moves on to differentiation? Then you could cross out all those reviews because they’re part of the same concept, and provide a different one, something similar to that?
Justin: It’s a little similar to that, but more fine-grained. The way we encode these encompassings is that an encompassing is a relationship between two topics. One particular topic encompasses another particular topic. But yeah, it’s the same idea. If there is an encompassing, you can knock out the review. We also do partial encompassing. Part of the topic is encompassed, but not the whole thing. In that case, it’s counted as maybe 25% of an encompassing. It doesn’t fully knock out the review, but it postpones it and puts you a little further in the spaced repetition process.
DeGatchi: What happens if you don’t ace the review? For example, if you get one right, fail one, and then get the other two right. Do you get the extra two XP? Or is it a module thing where specific review questions are combined to make a review?
Justin: The review questions are actually selected randomly from the pool of questions at the end of the lesson—the harder questions that are more representative of the real thing the lesson is scaffolding up to. If you pass the review but don’t get everything right, it still puts you further in the spaced repetition process. It counts as a repetition, but you don’t get as much credit as if you had aced the full review.
It’s a kind of fuzzy space repetition. Normally, space repetition is thought of as discrete intervals: review one, review two, and review three. But with trickle-down credit, partial encompassing, and partial accuracy on the task, it all feeds into this fuzzy metric of how much of a repetition the performance is worth. If you barely pass a review, you won’t get a full repetition. It might be like half a repetition or less. If you ace a review, you get more than a full repetition—maybe one and a half repetitions.
In addition to the specific performance at a given moment, we also measure the student’s overall accuracy and how they’re doing in general. This affects the speed of the spaced repetition process. The amount of credit you get for how well you performed on the topic is multiplied by your learning speed based on your overall accuracy. We also measure how difficult topics are for students in general because some topics are just intrinsically easier than others. We use that to adjust the process.
Each spaced repetition is calibrated to how well you performed on the topic, how well you’re doing overall, how hard the topic is, and your learning speed. That’s the model to keep in mind when thinking about spaced repetition.
DeGatchi: When I was doing reviews, sometimes it’s actually quite easy, but I mess up because I read the question wrong for some reason. It’s weird. I see one number, look back, and it’s actually a different number when I fail. I’m like, what the heck? It happens all the time. I think, “These questions are actually quite easy,” and then after I fail and a new one comes up, it’s actually significantly harder. What the heck? The failed review questions are much harder than the actual initial ones. It’s frustrating.
Justin: Yeah. Our question selection during a task or review is just random. It’s not that we’re intentionally giving you a harder question; it’s just the luck of the draw. Slight variations in difficulty happen. We initially looked into whether we should create a dynamic system, where if a student gets a question wrong, we select a slightly easier question. It wouldn’t be much easier since the pool of questions is similar, but we considered whether it was worth the effort. If the student gets it right, we’d use a slightly harder question.
The tricky part is that we can’t lower the bar for success. If you can only do the easiest problems in the pool of questions, we’d unintentionally lower the bar for success. That’s not ideal.
DeGatchi: It’s really about leaving a lot of room for progress.
Justin: Exactly. The benefit of random selection is that we don’t have to worry about that. We ran an experiment where we looked at students going through lessons with randomly selected questions. In some cases, the random selection emulated adaptive difficulty by chance, while in others, it did not. We analyzed the success rates in both scenarios, comparing when random selection happened to emulate more nuanced difficulty adaptation versus when it didn’t.
It turned out it didn’t really make a difference in terms of passing the lesson. The pass rates were nearly identical. This suggests that the variation in difficulty is small relative to the knowledge a student needs to acquire to complete the question. Even if they get the question wrong, they still acquire enough knowledge from the experience to be successful with future questions.
DeGatchi: You know what would be really nice for the harder ones? I think there’s no option to just look over the past and have some past ones.
Justin: Yeah, we’ve actually thought about something similar, but that’s a good point. The way we thought about it was for people who have fallen off the wagon, maybe skipped a few days of scheduled XP, and haven’t done anything. When they come back into the system, they have the same tasks as before. It’s like the trainer wants you to continue lifting the same level of weights. But if you’re a personal trainer, you’re probably going to say, “Why don’t we do an easier workout to get you back on track?”
We know how difficult various topics are because that’s one of the metrics we measure for adaptive spaced repetition and learning speed. The idea was, if someone has fallen off, we could refresh their tasks and intentionally choose the easiest lessons available. We could tell them something like, “Hey, if you do one of these, you’ll get a 50% XP boost.” That would help them get back into the flow. But, as you mentioned, a rapid-fire review system would be worth thinking about too.
DeGatchi: I think it’s a similar business model. You could say it’s the same for things that require habits, like the gym or signing up for a coworking space. A lot of these services make money from people who sign up but never actually go. But when people want to come back, it’s so hard to get back into it after taking a break. It feels much harder, and then you’re just paying for the subscription without actually going. This happens with all subscriptions.
Having something that helps people ramp back up would be super helpful. Doing something easier to build momentum, and then it snowballs from there. I remember, I think it was you or someone else who posted a rapid-fire times tables challenge.
Justin: I think that was a Mirage Cut from Learn to Be.
DeGatchi: Yeah, I did that. It was pretty fun, actually. You can notice which ones take longer, and you focus on those. It’s a great way to ramp up your day with math—doing something easy at first, then moving on to something more challenging on Math Academy, and getting into a flow state.
Justin: I like that quick five minutes of jump rope or whatever in the morning.
DeGatchi: Exactly. That’s something I started doing, and it worked pretty well. It actually came up when I was listening to the podcast with James. One thing that was really interesting was the automation of knowing the fundamentals, like your timetables or even arithmetic. Some people don’t even memorize these basics. I went down this rabbit hole of memorizing my exponents. I started noticing that I was memorizing things off by heart, like 4 squared, 6 squared, and even cubes. I was surprised because I had been doing it, but then I noticed there were times tables I didn’t memorize perfectly, like 9 times 8 or 7 times 9. There were just a few that I couldn’t quite remember, and it was annoying. I thought maybe I need to do this every day, like reading them or doing a lot of skipping.
For some reason, the exponents were much easier for me than the times tables, like 9 times 8. I don’t know why I couldn’t remember them as well, but that was interesting. It’s super important though, because as you said, if you don’t spend time thinking about the basics, it slows down your creative path. It’s like writing in English. If you’re trying to write a paragraph and you’re stuck on figuring out punctuation, like a semicolon or an asterisk, it’s just going to slow your creative flow down. It distracts you from your flow state. That was super important. I went down this whole thing and thought, “Maybe I should…” I saw that John von Neumann could do division with eight digits, or multiply eight digits, and I went down this rabbit hole of mental math, doing calculations in my head without a calculator. I started doing square roots and fractions in my head, trying not to use a calculator. That helped me understand it better, but it was super hard. Did you ever try that—memorize big times tables or work out mental math?
Justin: Yeah. I haven’t spent a lot of time on it, but I’ve definitely, like most people who are interested in that, gone down the rabbit hole of how big of numbers I can multiply. It’s interesting to look at those algorithms. But the limit is working memory. You have to come up with tricks to compress information to open up more room.
DeGatchi: Yeah. I thought people attributed John von Neumann’s success to his ability to multiply six or eight digits, and it makes sense. But do you really need to be able to do that at the end of the day? I don’t know. Then I thought, maybe creativity is really the only thing that matters, and who cares about big calculations? That’s what calculators are for. But then if you take the argument a step further, you might ask, why even do math? If calculators can do it, right?
Justin: I wouldn’t say that the ability to multiply huge numbers was causal in von Neumann’s success. I think the factors leading to his success also predisposed him to be interested in and capable of multiplying these numbers. Most professional mathematicians do not know how to multiply large numbers.
DeGatchi: It’s more about the reasoning behind everything. That’s the most important.
Justin: Exactly. There are things that come up often enough that you need to know how to do quickly. Times tables, for example, come up all the time. Basic multiplication, like within 12, or multiplying by 10, those things come up frequently. It’s important to have those automatic. But when it comes to something like 456 times 872, you never really need to do that by hand. It’s unnecessary. Even if you could do it in your head, a calculator would actually be faster. For single-digit multiplication, doing it in your head is definitely faster, but for really long computations with big numbers, even the fastest mental calculator wouldn’t stand a chance against a new electronic calculator.
DeGatchi: I wonder what the point is of trying to automate all these calculations. That’s a good point. But another important factor is how often the problem arises. Depending on that, you should automate it. For example, the square root of four—well, that’s basic. But understanding logarithms, I didn’t realize how important they were until I started doing Math Academy. Logarithms, exponents, and square roots are incredibly important. I’d say they’re just as important as multiplication and division. Same with reciprocals. I didn’t really understand that either, and I wrote an article about these basic things. I see them as being exactly the same as multiplication. Being able to do those without a calculator is really important. Honestly, I didn’t realize this until I started doing Math Academy seriously.
Justin: Totally. You don’t realize the importance of it until you start doing high-level math and have to do the computations yourself. Then you realize how important it is to know your logarithms and roots.
DeGatchi: I was explaining it in very simple terms, understanding why it’s important, why it was created, and what all the pieces mean—like the logarithm base, your input, what it means, and the use case for it. It’s just a tool at the end of the day, a symbol or character that’s used very often. What’s the underlying meaning of it, and why is it used? It’s like, why is a semicolon used, for example? When do you use it? Understanding this is key, because this is what builds up very advanced theorems. It’s just a fear of the components of simple things at the end of the day. Understanding the simple things and how they interact with each other—that’s how you get fluency in math. At least, I think that’s how I would build it, though I don’t actually have fluency in math.
Justin: I think the language analogy is very good. Especially in comparing the logarithm operation to a particular character or word that gets used often. That’s exactly the right way to think about it. I think you’re spot on with that.
DeGatchi: If you can’t understand it, you can never build these sentences or strings of them. I mean, that’s what covariance is, for example—it’s just combining things together. Same with gradient descent; it’s all the same stuff, just arranged differently to create a bigger idea. Another analogy would be biology. It’s the same concept: single-cellular logarithms to multi-cellular systems. It’s just compounding more of the same stuff to build something more advanced. But yeah, that’s my analogy. I think it works well.
Justin: The math is ultimately kind of like a language.
DeGatchi: It’s just a time thing at the end of the day—how often do you use it? It’s like living in Japan or Berlin; you’ll pick up the language faster if you’re immersed in it. But man, it’s been such a pleasure talking to you again, waiting for part three.
Justin: It was great. I had a lot of fun.
DeGatchi: Until then, do you have any advice for aspiring math students on how to get to a higher level of mathematical understanding?
Justin: For aspiring math students, I’d say, in addition to being strong on your foundations, if anyone is hoping to do a math degree and thinks all they need is AP calculus, it’s important to learn proof writing. Learn set operations and get some proof writing experience. Otherwise, it’s going to be a rough ride in a serious math program. There’s often a gap of missing knowledge that’s not filled enough by whatever onboarding resources these programs provide. Some programs have summer work to prep for courses, but it doesn’t always make up for not having a serious course in proof writing. I’d recommend that aspiring math majors focus not only on your foundations but also on getting automaticity with things like logarithms, roots, and algebra. It should feel as easy as reading a paragraph of text. Absolutely get to that level with basic proofs as well.
DeGatchi: I think math is just a language. Treat it like learning Japanese—do it every day, think about it every day. If you immerse yourself, it’ll become fluent. It’s just like when you’re a child and you’re around your parents, and they’re speaking English. You pick it up because they speak it all the time. It’s the same idea with math; you build it up over the years. That’s why you know a language fluently. You’re just surrounded by it. I find the same concepts apply to math.
Justin: That sounds good.
DeGatchi: Alright, sick. That was beautiful. Thank you so much, Justin, for sharing your experiences. We’ll definitely have you on again down the road.
Justin: I’d be happy to.
DeGatchi: Thanks for coming on again. We’ll speak soon. Goodbye.
Prompt
The following prompt was used to generate this transcript.
You are a grammar cleaner. All you do is clean grammar, remove single filler words such as “yeah” and “like” and “so”, remove any phrases that are repeated consecutively verbatim, and make short paragraphs separated by empty lines. Do not change any word choice, or leave any information out. Do not summarize or change phrasing. Please clean the attached text. It should be almost exactly verbatim. Keep all the original phrasing. Do not censor.
I manually ran this on each segment of a couple thousand characters of text from the original transcript.
Want to get notified about new posts? Join the mailing list and follow on X/Twitter.