Self-Transcript for Golden Nuggets Podcast #40 (Round 4): How Justin learns, new ML course, the magic of Twitter

Rationale, vision, and progress on Math Academy's upcoming Machine Learning I course (and after that, Machine Learning II, and possibly a Machine Learning III). Design principles behind good math explanations (it all comes down to concrete numerical examples). Unproductive learning behaviors (and all the different categories: kids vs adults, good-faith vs bad-faith). How to get the most out of your learning tasks. Why I recommend NOT to take notes on Math Academy. What to try first before making a flashcard (which should be a last resort), and how we're planning to incorporate flashcard-style practice on math facts (not just times tables but also trig identities, derivative rules, etc). Using X/Twitter like a Twitch stream.

Cross-posted from here.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

Link to Podcast

The transcript below is provided with the following caveats:

1. There may be occasional typos and light rephrasings. Typos can be introduced by process of converting audio to a raw word-for-word transcript, and light rephrasings can be introduced by the process of smoothing out natural speech patterns to be more readable via text.
2. The transcript has been filtered to include my responses only. I do not wish to infringe on another speaker's content or quote them with the possibility of occasional typos and light rephrasings.

Justin: Thanks. Great to talk to you guys again. I would have had enough of this. It’s mainly been working on the machine learning course, working on these coding projects for the course. We’ve got our normal lesson topics that you do by hand, but we’re also going to have coding problems.

It’s a multi-step problem, so we’ve got some bigger, more applied problem context that pulls a bunch of these lower level skills together. You might come into the multi-step having done some stuff with neural nets by hand, maybe working out one iteration in a very simple case of a back prop and a forward propagation and doing problems about different structures of neural nets.

But in the multi-step, that’s where all these skills get pulled together. You actually code up a neural net and we break it up for you. But the prompt is not just like, hey, build a neural net. It’s like, build this component of the neural net. Great, now build this other component. Now link those two components together and you do that for like eight or ten questions.

By the end of it, you’ve built a neural net, you’ve run it successfully, maybe you’ve even run it on some parameter setting that illustrates a kind of shortcoming of the simple vanilla thing that you built and then we have you build a more complicated, sophisticated workaround and that demonstrates the improvement.

It’s been working on just scoping out a bunch of these problems and kind of guiding the person who’s doing most of the heavy lifting and building them.

Justin: Most of the variation is going to come from the underlying lessons, the by-hand problems. That’s where we really need the repetition to ensure that you’re acquiring these component skills and are solid on them. When it comes to the project, not as much variation is needed. You just need a different project.

Maybe one project is building a simple feedforward neural net, maybe another one is building a convolutional neural net, maybe another one is using neural evolution instead of gradient descent for training it. It’s kind of like, I think the way to think about it is, if you’re in a machine learning class, you would be doing some math, hopefully math problems on the subject matter. Those math problems would be more scoped down, almost like solving equations or just working out various parts of algorithms. You’d do it a bunch of times with different numbers.

It’s kind of like you’re the computer in a simple case going through different data sets, different processes. It’s almost like just working out algebra problems or calculus problems—you can just rapid-fire those. But you wouldn’t really say, “Okay, build me a neural net today and then build me another one tomorrow in the same way.” It’d be more like, “Okay, just build me a neural net. Okay, great, you build a neural net. Now let’s make it more sophisticated.”

It blends repetition-focused learning into project-based learning. I’ve talked about project-based learning before—oh my goodness, everybody is doing project-based learning, and nobody is learning anything from it. But that failure mode is only when students don’t have the component skills in place, which unfortunately seems to be most of the time in most cases of project-based learning, at least in my experience.

But if you get that right, if you get those component skills in place, then yeah, the projects are great. That’s the reason.

Justin: That’s a good question. Honestly, I don’t really do a whole lot of that kind of learning anymore. It’s mostly just focusing on production personally. You reach the edge of known things and you’re in some particular direction and subdomain, focusing on trying to produce more knowledge, more technology, more tools, whatever. You turn into almost like a researcher who’s at the edge. Yes, it’s not very efficient, but it’s a different game.

Personally, when I was more focused on acquiring known existing knowledge bases, like learning math, physics, computer science, and that sort of stuff, I latched onto MIT OpenCourseWare and various textbooks used there. It was painful. It doesn’t have, when you piece together from these online resources, even something fairly cohesive like MIT OpenCourseWare, it doesn’t have spaced review, it doesn’t have mastery learning in the sense. It’s not broken up into things that you demonstrate knowledge of and then move on to correct because you’re left to your own devices to structure that however you want.

That was back when I was in high school and I didn’t really know a whole lot about the science of learning. I made a lot of suboptimal decisions and experienced a lot of pain from that. I always harp on: I can’t just read the textbook and take notes; you have to actually solve problems. But my first approach to learning math, I definitely fell into that failure mode where I would read some stuff, take notes, and think I understood it. I thought, oh, I can totally speed run this stuff.

Then I speed ran some of it, got to a point where nothing was really making sense, I was way out of my depth, and I asked myself, “Why can’t I do this now? What’s different from what I’m doing now versus math class?” I realized, okay, math class, we actually do homework. I need to do homework.

If I had to go about it nowadays, say somebody said, “Hey, Justin, you need to learn undergraduate biology. You’re going to have a final exam in one year on an overhaul of undergraduate biology. If you fail, we’re shipping you off to Antarctica or something,” I’d have to know if it was a life-or-death situation. How would I go about it?

I think step one would be to find a curriculum that is really good. A curriculum where experts are piecing together the content for you, reducing as much friction as possible from the learning experience. You wouldn’t want to go out and just try to discover all this knowledge on your own because you’ve got centuries of knowledge to discover. You can’t just sit down, go from philosophers thinking to biology, meditate on it, and maybe do some key experiments. You need to go faster than that, and that’s what the curriculum is meant to do.

Step one is finding a good curriculum. What’s a good curriculum for biology? I don’t know. I looked into this last year, interested in what kind of adaptive learning solutions exist for biology, but I didn’t really find anything impressive.

Justin: I was really impressed by it. Were there assessment questions in it too? Did it test you on the stuff? Or is it more like a Three Blue and Brown video where it’s passing them at a high level?

Justin: Never. I have to take a look at that. That’s really cool.

Justin: That’s a really good point. Honestly, one of the reasons I haven’t gone and learned biology is because it takes a long time. If you could compress it down to a smaller number of time without skimping on the rigor of the material, that would be nice.

What you mentioned about LLMs designing a curriculum is actually pretty interesting. I’ve seen more of that nowadays, where people will ask an LLM to set up some kind of structured learning environment for them. It’s not perfect, but it definitely comes a long way. I’ve seen some people create, for instance, an arithmetic knowledge graph using an LLM.

There were some issues with it. If you had no access to a good curriculum, and it was either that or figure things out on your own or work through a five-year curriculum, it makes sense that you could use the LLM for guidance on how to go about it. Once you have it, it can give you a map of the territory, which might help in piecing together all these materials online.

For example, if I want to learn biology, maybe Smart Biology covers some of this curriculum, and then there’s a more problem-set-focused resource that I combine with it. It’s an interesting problem to think about.

Justin: Oh, actually, not in particular. My wife is in a virology PhD program, and I always enjoy when there are people in your life who are in some nerd hole doing something. It’s nice to be able to talk to them, not just as a layperson—like, “Oh my God, Justin doesn’t know what a cell is, let me explain,”—but have an actual conversation with some of the more technical details.

I guess that’s one of my main motivations, just to be able to have higher-level technical conversations with her about things. I think it’s similar to when I have kids. They’re probably going to have interests that are different from mine, most likely, and I’d like to be able to talk to them about those interests in proper depth.

Being able to talk to my wife about biology in proper depth is like the initial version of a problem that’s going to occur later down the road, which is being able to talk to my kids in proper depth about whatever they are interested in. It got me thinking, well, if they know a lot more than me, how do I go learn a bunch without just having them turn into my personal tutor?

If you have a kid who’s really interested in some subject and knows more than you, they’re probably not even going to be great at explaining it to you because explaining is a separate skill on its own. Anyway, this isn’t a very directed answer, and this probably isn’t a strong enough motivation to really get me over the hump of seriously learning biology.

It’s sort of like I run into people who are like, “Oh, math is pretty cool, it’d be cool to know that,” but it’s not enough motivation to get you over the hump of solving problems and mentally sweating, being exhausted all the time. That’s why I haven’t been thinking about learning biology lately.

Justin: That’s really interesting. The reason it comes to mind for me is because I was in a situation as a kid where I wasn’t as lucky. I had two parents who really loved me and wanted the best for me, and that was amazing. I’m super fortunate in that respect. But they also didn’t know anything about math, physics, or science in general. I didn’t have any family members like that.

It just felt like there was nobody I could have a serious conversation with about things that interested me unless I went out and tried to find some university professor to enjoy. Which has its benefits of forcing you out of your comfort zone and meeting new people. But anyway, that’s why it’s on my mind.

Justin: I agree. That’d be interesting. I think I’m going to try that, actually—just having some conversations with an LLM about biology and see how far I can get with it, how easy it is.

One failure mode that a lot of people who try to do that fall into is they don’t actually know the principles of effective learning. They’re not asking the right questions. They’re not telling the LLM to structure the learning experience in a way that’s optimal. Some of it makes me wonder, okay, if you do have a good idea of that, and you can try to instruct the LLM how to teach you this thing you don’t know, how efficient can you make that?

Justin: That’s really interesting. I think I started experimenting with this. It’d be interesting to know. You’d clon. That’s the one to use.

Justin: I was thinking about making a lab notebook, like, “Here’s how I started trying to go about it, here’s what I tried, here’s the results, here’s what was good, here’s what was bad.”

Justin: Thanks. I appreciate you noticing the effort that goes into the output.

If I had to boil it down, what my name in a loop on this stuff is: I’m trying to just constantly stay in motion throughout the day, knocking out things I know I should be doing. Whenever I find myself getting a little bored of something, I switch to something else. I have enough things to do that I can just pick something else off the shelf that I’m really excited about. There’s always something to do that needs to be done, so I just continually go through this cycle, picking up different things to focus on.

It wasn’t always like this in my life, but since I got involved with Math Academy, which turned into a life-consuming experience, there’s just no shortage of things that actually need to be done.

On Twitter, for instance, if you approach it less as an enjoyment activity and more as a task—like, “Nobody knows about Math Academy, that’s a problem. We need to spend time talking about it, getting excitement going, engaging with the community”—it becomes just something that needs to be done. It can be fun, but it’s something that is part of the process.

I experiment with different strategies. I found that Twitter engagement can be structured in a way where I’m working on something else for half an hour, maybe get a little bored, my mind starts wandering, and whatever it wanders to can sometimes be wrapped up in a tweet within a couple of minutes. I just go post it and then switch back to something else. I can’t spend the next hour just scrolling through Twitter. I need to get back to moving the needle on another task.

I try to be very realistic about what’s moving the needle and what’s just enjoyment. I align those things as much as possible throughout the day.

Sometimes, if I’m having trouble going to sleep because I’m thinking about too much stuff, I might pick up my phone and spend 10 minutes on Twitter. That kind of mentally exhausts me enough to be ready to sleep. It has the byproduct of producing some valuable things towards my goals.

It’s all about compacting all the tasks that need to be done into the most efficient way, given my motivation and the importance of each task.

Justin: In terms of trying to maximize words written, I never just sit down and say, “Oh, I’m going to write about X, Y, Z and make myself do that.” It’s more about taking advantage of thoughts that occur naturally. For example, if somebody posts an interesting comment to a tweet, I might see it, and immediately a couple of sentences come to mind. I’ll post it as a reply and think, “Oh, that would make a good seed for another post in the future.” Then I save a link to it.

Later, when I feel a little bored with what I’m working on and want to do something quick for five minutes, I might look at the links of comments I’ve made. I’ll pull one off the shelf, read it again, and new things to say come to mind. Since it’s been several hours or even a day or more, I take advantage of that, write it down, and often that creates a flow experience that leads to more writing.

I don’t spend a lot of time thinking, “What should I write?” I try to avoid that. I just take advantage of the natural thoughts, put them down, and ship them off.

Another part is that the more you write, the easier it is to write other things. The more I write, the more automatic I become with ideas I’ve said before. I have analogies, tight quotes, and perspectives I can refer back to, and they don’t have to be the exact same words. It’s more about regenerating the memory.

Oftentimes, that refinement process is like retrieval practice, and the more practice I get, the easier it becomes. I remember when I first started doing one Twitter post a day, I viewed it as just needing traction on Twitter. Every morning, I’d find something I’d written previously and turn it into a nice little Twitter post. That worked okay.

Then, I had a conversation with Jason, the founder of Math Academy, who suggested I treat this like a Twitch stream, just putting out whatever interesting thing I’m thinking about. I stopped posting right away, and at first, it took more time and effort. But eventually, as I practiced, it became automatic, faster, and easier. The quality increased too.

Even just scrolling through Twitter for 10 minutes before, I used to find only one thing and force myself to come up with an interesting perspective. It would take longer to get it from my head to the post box. Now, after practicing more, it’s much easier to find the right words.

It goes back to a common quote about writing: “What’s the trick to becoming a good writer? Write a lot.” It just comes naturally and helps refine perspectives.

I should also mention that sometimes, when I put a lot of effort into an article or blog post, and I know it’s really good, I might copy a relevant snippet I’ve written before—something punchy or eloquent—and tweet it at the right time. I remove any extraneous details that aren’t relevant to the context, do a couple minutes of editing, and shoot it off.

In addition to having all the ideas in my brain, I also have a database of previous writing to pull from. Building up that database and doing a lot of writing makes it easier to generate more output in the future.

Justin: It is a game. You have an audience on Twitter. A Twitter audience is different from a Facebook audience or an Instagram audience or a Hacker News audience. They all have different ways.

Justin: Learn to write a Twitter post. Oh man, that’d be funny.

Justin: One expert for me. That’d be hilarious. Imagine a course on Twitter writing techniques, and one of the units is meme creation. One module is text memes, another is image memes. Another one could be…

Justin: If you don’t have the freedom to choose what you’re working on, or at least structure it—maybe you have a bunch of different things that you need to get done, but it’s up to you when you want to work on them and in what order—then not having that freedom is a major impediment to using the strategy.

Justin: It’s a good question. Honestly, there might be something about me where I’m a little more cognitively set up not to have these pulling attention problems. My failure mode is often when I get too engrossed in something. When I head down for too long, I miss things around me that are important to pay attention to. I don’t know. I wish I had some kind of tip or secret—like, “Oh, here’s how you don’t get addicted to Twitter” or “Here’s how you can make yourself continue to read books.” But personally, it’s not a problem I’ve experienced. Does that happen to you?

Justin: Oh, yeah, I see what you mean. I guess one thing I can say about that is, whenever I’m going through something, if I feel like taking a little break and going through Twitter, if I haven’t made a post within several minutes of looking through Twitter, I just get this bad feeling in my stomach that I’m wasting time. It’s like a reverse thing for me. It just makes it easy to close out the app and work on something else. But I could totally see, if you don’t get that feeling, or if there’s some element of the addictive quality that overcomes it, or maybe someone’s not paying attention to that feeling, it’s easy to get sucked into this black hole of passive consumption. I guess maybe that’s part of it. I view Twitter as a mode of production, not a mode of consumption. That’s what it is to me, and that probably makes it easier not to get sucked into the black hole of consumption.

Justin: James, what’s your experience like on Twitter?

Justin: Totally. I’ve heard that same thing from people who say, if you want a job, just post some cool stuff on Twitter and reach out to people. It’s amazing how good of a networking tool it can be. It sounds like you’re coming from a similar perspective where it’s less about consuming information on Twitter and more about using it as a tool to produce something, meet people, or have some kind of end goal in mind with it.

Justin: I think that’s the idea. Yeah, it makes sense. It’s less about pattern matching to success and more about trying to shape whatever you’re trying to say to be a little bit tastier to the Twitter audience while still retaining the authenticity of what you were trying to put out in the first place.

Justin: Yeah, I think I do try to keep it on brand, but I consider my brand to be pretty wide. Anything relating to Math Academy, math, coding, learning, productivity, skill development, it’s all fair game. There are limits, though. I wouldn’t post everything. For example, wedding pictures or stuff that I would think belongs on something like Facebook or Instagram. I try to keep my Twitter posts things that I think other people will find interesting or valuable, beyond just “look at me.” I also try to stick to things that I’m confident talking about. For instance, I haven’t posted anything about learning more biology, but if I tried to get an LLM to guide me through this and ran a lab notebook, that would be adjacent to learning and talent development. Suddenly, it feels relevant, and there’s a connection, so I can talk about it.

But if there’s no connection like that, if it’s just an isolated thought with no link to this knowledge graph that encompasses all my Twitter content, I’ll hold off on posting it.

Justin: That’d be funny. Yeah, just have some anonymous account.

Justin: Yeah, I try not to get sucked down too far. One thing I noticed is that whenever you write something and somebody nitpicks something, you’re just like, “Oh my God, seriously? Are you going to nitpick that?” I think it’s often an indication that you need to be more defensive in your writing. You need to state a caveat or some kind of clarification in the actual writing. I feel like I’ve taken a lot of those experiences and tried to pull proof by writing from it. So that’s one reason I sometimes poke the trolls, to see the trolls. But then there comes a point where it’s like, okay, now you’re just trolling. I’m not really extracting any more knowledge on how to defend against people like you in the future, and then I just have to leave it.

Justin: Let other people read it. Yeah. I guess that’s less of a defense; it’s more of an amorphous meaning. So that comes out.

Justin: The funny part is, they still share the posts. They still comment on the post. So we get more business. It all helps the algorithm.

Justin: Yeah, had a great time too. Be interested to know how that task switching turns out for you.

Prompt

The following prompt was used to generate this transcript.

You are a grammar cleaner. All you do is clean grammar, remove single filler words such as “yeah” and “like” and “so”, remove any phrases that are repeated consecutively verbatim, and make short paragraphs separated by empty lines. Do not change any word choice, or leave any information out. Do not summarize or change phrasing. Please clean the attached text. It should be almost exactly verbatim. Keep all the original phrasing. Do not censor.

I manually ran this on each segment of a couple thousand characters of text from the original transcript.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.