Watch on YouTube or Listen on Spotify

What we covered:
– As Math Academy has grown over the past year, we're getting a better sense of general do's and don'ts when scaling a startup. We've learned hard lessons about overloading the database, the task processor, and our team, requiring numerous infrastructure and process updates.
– Schools have been using the system and we've built plenty of additional features to, among other things, accommodate unique billing schemes and make it easy for teachers to manage classes on the system.
– We've intentionally grown organically and were self-funded, which forced us to do things manually at the beginning. Years ago, we taught math classes in person and Jason onboarded our first online users on hundreds of hour-long individuals and calls. These were crucial experiences to learn who our customers are, what they want from the product, and common failure modes.
– In our experience, doing things manually at the beginning ensures that you 1) build a product that customers actually get value from, and 2) you don't clutter your product with unnecessary bells and whistles that don't add value. In other words, you have to do the manual work to earn the right to scale.

0:00 - Introduction
2:18 - Building infrastructure to handle increasing load
3:41 - Bringing on AWS expertise to robustify the backend
4:22 - An overloaded database enters a new realm of physics
5:50 - Prioritizing execution over perfection in start-ups
6:33 - Paying the bill for accumulated infrastructure debt
7:53 - Improving job prioritization of the task processor
9:52 - Benefits of scaling organically
11:42 - Wisdom is the result of failures
12:18 - There is no substitute for experience
13:17 - Focusing on solving problems, not advertising
14:48 - Upgrading with surgical precision
15:35 - The pain-point compass
17:04 - Managing finite time and resources
18:27 - Development of the gravity feature
20:42 - Gravity is a suggestion, not a hard override
22:25 - Limiting gravity to avoid cognitive overload
28:29 - Balancing customization and customer confusion
31:28 - The feature sandbox
33:58 - Increasing volume of customer support emails
35:22 - Additional infrastructure requirements for schools
36:18 - Learning about the customer through direct interaction
38:14 - Step 1: Manually added schools using spreadsheets
40:22 - Step 2: Developed tools to handle specialized school requests
41:23 - Step 3: Goal is 100% self-service sign-ups for schools
42:32 - Solve the problem manually first, then automate it
43:44 - Why focus on schools?
46:15 - Math Academy goes to college
49:37 - You can’t anticipate every edge case
52:14 - Letting user behavior build the product roadmap
58:54 - Becoming successful means working harder
1:00:24 - The customer support hurdle
1:03:27 - How Justin’s expanding roles drove growth (both personal & company)
1:09:03 - Teaching as market research for Math Academy
1:10:52 - The value of having been inside the trade

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

Watch on YouTube or Listen on Spotify

The raw transcript is provided below. Please understand that there may be typos.

Justin Skycak (00:00) Welcome to the Math Academy podcast. I’m Justin Skycak, Chief Quant and Director of Analytics at Math Academy, and I’m here with our founder, Jason Roberts, to discuss lessons from scaling a startup. As Math Academy has grown over the past year, we’re getting a better sense of general do’s and don’ts when scaling a startup. We’ve learned hard lessons about overloading the database, the task processor, and our team, requiring numerous infrastructure and process updates.

Schools have been using the system and we’ve built plenty of additional features to, among other things, accommodate unique billing schemes and make it easy for teachers to manage classes on the system. We’ve intentionally grown organically and were self-funded, which forced us to do things manually at the beginning. Years ago, we taught math classes in person and Jason onboarded our first online users on hundreds of hour-long individuals and calls.

These were crucial experiences to learn who our customers are, what they want from the product, and common failure modes. In our experience, doing things manually at the beginning ensures that you, one, build a product that customers actually get value from, and two, you don’t clutter your product with unnecessary bells and whistles that don’t add value. In other words, you have to do the manual work to earn the right to scale. So what are the challenges that we faced and what lessons did we learn? Let’s get into it.

Jason Roberts (01:26) Justin, do you got some stuff in mind for your review tech wise or?

Justin Skycak (01:32) Yeah, so some things that come to mind on the tech side that we’ve done is only just kind of go in an order of what we did last year. So I think the first kind of tech that I remember us working on was back in the spring ⁓ to when we had started on the machine learning and computer science courses. We had kind of mapped out a lot of how the coding questions are going to work and these coding projects and stuff like that. that is, I mean, we’ve pretty much figured it out, right? And you got like some kind of a

a prototype and it’s just a matter of like actually like moving it into the course, putting all the real questions on it and fighting all the details that come up there. yeah, there’s that and then I’ll just go through the quick list right now and then we can revisit. There’s also a lot of infrastructure that we built, just system infrastructure to deal with ⁓ scaling challenges. ⁓ Because I mean,

Right. When you have like 10 users, a hundred users, like you don’t, it’s kind of dumb to, to, worry about like, Oh, well this worked great when we have more than that. It’s like, well, don’t worry about it. It’s like focus on getting more users and then you earn the right to solve the problems that come with that. But there was a lot of things like, well, we had to upgrade the database. We had to go kind of more systematically figure out like what queries are running slow. There was, I think it was like one or two.

kind of blow ups, right? Where we were down for like 20 minutes. I think one time it was down for like an hour or so. That sucked. Well, luckily the hour outage happened like on a Sunday night. ⁓

Jason Roberts (03:07) Yeah, that was.

Yeah,

a of people notice. night on the West Coast, which, yeah, that’s probably one of our lowest usage times.

Justin Skycak (03:23) That Asia was waking up about that time, right?

Jason Roberts (03:26) Right.

That

was I’m trying to remember database was pinning. Right. We just, had to upgrade the Ram. it, we just didn’t have enough Ram was the big problem. And so I hired this, I found an AWS expert and I was like, I need some, I was out looking for people and I found this guy and he come on, he came on and we were really kind of deciphering some of the, the, metrics on the AWS dashboard and other things. And he’s like, I think.

I think you just don’t have enough. I don’t think you have enough RAM. I think we need to upgrade the RAM on the database server. And we did and that solved it. Although there was, ⁓ wait, there was something that brought it to light that you had to.

Justin Skycak (04:11) Yeah,

so I remember this being kind of a cascading fit, where it’s like, might, listeners might think like, OK, well, isn’t this something that just kind of sneaks up on you and gradually becomes a problem? But really, it’s like, once the database gets pinned at like 99 % plus, it’s like it enters almost into a new realm of physics, where all these queries start becoming like, not just like linearly slower, but like exponentially slower. And so the big blow up happened because the

the query that was fetching the data for, for XT leaderboards suddenly became super, super slow. And that’s something that’s on everybody’s dashboard. So it’s running like all the time. And because that became super, super slow, people’s dashboards were taking forever to load. that what, what do people do when their dashboard is taking forever to load? refresh. Exactly. So now we’re just getting like, yeah, spammed by all these. Exactly.

Jason Roberts (05:03) hammered.

by our own users.

Justin Skycak (05:09) So yeah, so that got sorted out. But yeah, now it’s like anytime we see the database up, there was a time recently, right? When it was above 90%, when that we, that do track down and fixed. Yeah. But it was like anytime it kind of starts creeping up like really high, we’re just like, ⁓ no, that’s

Jason Roberts (05:28) Yeah, I mean, so the same guy that helped us fix that initial problem, which was, I don’t know, four or five months ago or whatever that was, I hired him. So he helped out with this more recent problem, but now we’re doing like a weekly call of just a series of different types of upgrades and optimizations. So it’s like, you know, what I’ve noticed has happened is that, you know, when you’re a small company, you’re just…

Just trying to do like this 80-20 solutions on everything to get stuff working. Sometimes you put a little more love into something because you know it’s really important. lot of times it’s like, okay, that’s a solid B, B plus, whatever time to move on. got like 30 other things I got to do. I can’t just sit there and, you know, tweak this thing for the next three days. But then what happens is there’s some things that you do even less than 80-20, you’re just like just good enough. Like, okay, it works. I’m moving on. And that could be like with infrastructure too.

and all these kinds of performance things. so, but they’re sort of to accumulate and, and then it can run into problems. in terms of the infrastructure, you know, this guy is going to help me like upgrade all the things that I probably should have been upgrading before. You know, it’s like, Hey, you know, like one of it is moving from an RDS instance to Aurora, which is like this infinitely scalable database, you know, cloud database is offering from AWS, which will be actually less expensive for us.

but it will be more performant and we don’t have to worry about like, my God, like we’re at 80, 90, 85, 90%. We gotta do something. It’s just, this won’t happen.

Justin Skycak (07:05) This will just expand with the. ⁓

Jason Roberts (07:07) Yeah, because I don’t like that. That was not a fun experience either time. That was extremely stressful, extremely frustrating. You’re like, my God, what are we going to do? Because then, because two things, not only can it like, if you don’t fix it, it’s a problem. But then sometimes, you know, people are, some people are experiencing slowness. Some people they’re not getting much service at all. But you just have to take the whole thing down to upgrade a server. It’s like, it’s going take at least a 20 minutes. And I’m just like, my, I can’t just take this thing down for 20 minutes or more.

Justin Skycak (07:11) Yeah.

Jason Roberts (07:36) 30 minutes without giving people heads up, people are going to get pretty upset. You know, you want to, you always want to avoid doing something like that. So anyway, that’s these things accumulate, you accumulate technical debt. think you accumulate what you might work on more infrastructure debt. know?

Justin Skycak (07:53) There was another type of ⁓ scaling challenge that we ran into that actually, I remember it didn’t exactly take the website down, but people were not getting some people. It was just the task processor got so slow because it got backed up with course re-initializations because it wasn’t prioritizing jobs correctly. So I worked on that for a good like week or two just in the kind of behind the scenes processors that are computing.

like what tasks you get next to make it so that they prioritize the jobs better. Because what happened is we came out with this really awesome new functionality that when the course gets updated, we’re going to go send an email to everybody who’s in the course with the information. And then we’re going to re-enact, like refresh their.

their tasks and their completion date and everything. It’s like, well, great. So we got this all there. It’s going to be so slick. And then what happens the first time we do it? Like, well, we reinitialize a course that has like, it’s one of our biggest courses. And now the task processor is backed up like doing all these like course or initializations and people are not getting, that’s just clogging up all the.

Yeah, a processor that is supposed to be like generating new tasks for people. So now that people are sitting there without tasks, it’s like, well, okay, what you need to do is like any, any jobs that are for people without tasks, any jobs that are like actually task generation, not just a course refresh or information refresh, you got to prioritize those and then get through the backlog afterwards once you’re, once you don’t have any high priority jobs to do. anyways, that works great now. But that was,

That was another one of those moments that was like, my God, why is this? Why, why is this not working? Like, cause we didn’t know exactly. So it took like half an hour even diagnosed what is going on and then fixing it. Again, not fun, but, I mean, there’s the things that, ⁓ you always say it’s like, we, got to, you run into those issues on a smaller level of scale. You get those solves and then it’s not a problem with the bigger level scale. As long as you’re, as you’re like growing.

incrementally running into this problem, you’re scaffolding yourself into this higher level of scale is not such a big issue. mean, yeah, it’s painful in the moment and maybe some people are frustrated, but it’s like, it’s not like we’re like growing by a factor of like a hundred X in a very short timeframe. And suddenly like you have all those problems to deal with at that scale. it’s like catastrophic.

Jason Roberts (10:24) I think there’s a lot of benefits to organic growth as opposed to artificial growth. So when you go and raise a lot of capital from a venture firm and you hire a ton of people and you just spend a lot of money in advertising and you just let go, right? It’s sort of like the similar problem. When somebody wins a lottery and they have a lot of money and they didn’t really earn it step by step, so they don’t know how to manage it, they don’t know how to appreciate it, and they kind of blow a lot of it and they…

creates all kinds problems in our life, the relationships with other people, figures it was just thrust on them as opposed to like you spent 30, 40, 50 years building your wealth. It’s like, you know how to manage it, you know how to interact with other people in regards to it, you know, and whatever. So it’s a similar thing. It’s like, life usually goes better when you go in a nice, smooth upward gradient, as opposed to just bam, you’re here. You don’t have the appreciation for not only

the benefits but also the you know you would appreciate all but the pitfalls and issues so you know.

Justin Skycak (11:24) That’s a great analogy, the lottery ticket. Yeah, that’s exactly what it is.

Jason Roberts (11:29) Yeah, you win the lottery and your life is transformed, but you haven’t developed the insights, the skills, the judgment for how to manage it, the wisdom, right? Because wisdom is a result of failures, right? And so in order to build towards something good, whatever it is, you’re going have lots of failures along the way. So you’re going to ⁓ gain wisdom over the years, and that wisdom will allow you to

operate effectively at higher levels when you get there. That’s why you take a 23 or 25 year old and you make them president of the company or something. Okay, well that’s going to be a disaster, right? You know, unless it was like a little startup that person built themselves. And even then, if it happens too fast, they didn’t develop the skills. They don’t know how to manage people. They don’t know how to, you know, do a lot of things correctly, right? So I think it’s hugely better in life. It doesn’t mean that you can’t speed run stuff to some degree, but you don’t get to, you shouldn’t skip steps.

Justin Skycak (12:28) Yeah, you can speed run by, compressing the, the work by making it more efficient stuff, you can’t speed run by, by skipping. Right. Exactly. There’s, no shortcut. There’s just running faster on the long way.

Jason Roberts (12:41) Just

do stuff more efficiently. You try it. And the way you do things efficiently is you actually do things. You run experiments like let’s try this. Some things work. Some things don’t. You learn from those. You go, that hurt. That felt good. That worked. What can we take away from this? We learn and you can do that. Okay. Now let’s run another experiment. So let’s build on this or whatever. But all this stuff is our honest lessons that you are.

incorporating into a compressed sort of wisdom. And I think that, you know, one of the reasons that I wanted to approach Math Academy the way we have, which is that we have not taken outside investment. It’s all completely self-funded. We don’t pay for advertising. simply try and do everything the hard, we put things on a hard mode. It’s like, you want more users? Create a better product. So people talk about.

Right? I’m not like I’m ever against advertising, my bias is to try and do things in a hard way, but also because the hard way means that you have to solve problems. You don’t get to postpone the problems or ignore the problems or paper over the problems. And so, you know, by doing things slowly and then things ourselves, we learn how to, you know, the conversation we have with Alex about, you know,

how to improve the pedagogy. We learn about how to create ⁓ more efficient, more robust ⁓ infrastructure, cleaner code, clean, you know, whatever, but you got to learn all those things. So I don’t know, go, don’t skip steps, do things as efficiently as possible, but gain the wisdom along the way so that you’re operating in as optimal way as possible at every stage you’re at.

Justin Skycak (14:32) So yeah, so that’s definitely, I’m sure there’s more scaling challenges that we’ll face in the future, but at least we got our head above water now. I got what, I probably just jinxed us.

Jason Roberts (14:39) Yeah, well, you know, it’s.

No, yeah, you know, it’s but. You know, so bringing on this AWS consultant has been really helpful. He’s great, really knowledgeable, and then we’re kind of going steps. He’s not like, well, I’m going to create a whole new thing for you. It’s like every week we’re going to upgrade a service. We’re going to make something more robust, going to make something more secure. We’re going make something more fault, whatever it is.

And we’re just gonna keep doing that within two or three months, it’ll be state of the art in every way. I mean, it’s already not bad, but it could be better. And I just want, you just don’t want to have any unnecessary weaknesses in your system. I mean, whether it’s things just running out, you’re running out of compute or memory or something’s insecure in some way. So I don’t want that.

Justin Skycak (15:35) Yeah, so I guess there’s a different, we always talk about like these kind of made up problems that you can envision into the future, but it’s like, well, if you have suffered at the expense of a problem in the past and you know that problem is going to come back shortly, if you don’t kind of like stay on top of it, then you can reasonably infer that it’s a good idea to avoid this problem. This is not a made up problem. Like you’ve seen it. It’s not a ghost. This is actually a concrete thing that that

punched you in the face before. if you don’t, if you shield yourself from it again, it’s going to come back at you with a baseball bat. like, let’s avoid, let’s avoid that.

Jason Roberts (16:12) Because if I had, but the thing is, and because we went through this together, if I had just told you about this stuff, well Justin, we got to make sure blah, blah, blah, you know, it would have made as much sense to you as like you suffered through it. You had to help us to detect where the inefficiencies were coming from. You had to help debug queries. had to find, you know what mean? So now you’re like, okay. Now going forward, you’d like, okay, we got to be careful with some things, right?

You know, because as we get a bigger user base, so, okay, what was one of the lessons? Okay. As you get a larger user base, you can’t just say, do something for all these people at the same time. Like that can’t happen. Things have to be throttled. Things have to be queued up and, and, and throttled. Um, you know, it’s fine when you have like a hundred users or a thousand users, but you get to a certain scale and it’s like, boy, everybody who’s an integrated math was a lot.

Justin Skycak (17:04) Yeah, she really got to prioritize jobs. Right. Cause it’s like, it’s, I mean, I’ve definitely felt this in the past year or two, or like the, used to have to-do lists that I could actually like, kind of like get through, you know, but now it’s just like, it’s, it’s a, related race problem does not check out the rate in is as much higher than the rate out. So it’s like, you need to prioritize that the workload every there’s always.

Jason Roberts (17:06) You know.

Justin Skycak (17:31) There’s no concept of the list going away, but there’s the concept of prioritizing, like new items that are added to the list, they have to be ranked appropriately and this rating has to be adjusted frequently. although, so now it’s like, now it’s not just like you and me and Alex and the team in this situation. It’s like literally the processors are in this situation. They have so much to do that often, I mean, just sometimes there are spikes when there’s so much to do all at once that you have to.

prioritize and even it out. yeah, let’s talk about some additional, some of the scaling challenges, right? That a lot of infrastructure are just tweaks and improvements that we made to that over the past year, but also gravity feature, which is currently being beta tested.

Jason Roberts (18:20) Yeah, probably, probably just made a couple of tweets to that and we can roll that out to more to probably everybody, I guess. I, yeah, so the gravity feature, the, that was imagined was, you know, people would, people would say, well, you know, we’re using math Academy, but I’d really like for my daughter to be able to do these specific topics because they’re going to be tested on this thing or as part of what’s going on in school or whatever. Right. They, and, um,

You know, up till now, there was no real way for anybody to influence and prioritize certain learning paths. You know, because it’s like, as long as it’s like, like, they’re, all we’re trying to do is optimize that they get to the course as quickly as possible. Not that they get to solving systems of linear equations as soon as possible. That happens whenever it happens. You know, it doesn’t really matter. And then most people, most parents or adults are like, I don’t, as long as I go through everything in the course, I don’t care. But there are certain situations where

Adults who are surlundering on their own care and their certain instances where parents care because they’re, don’t, they may not be living in a vacuum. They may be using math Academy to help them do something else, which has. Targets to learning certain things. Also in a school, in a classroom, if a teacher is using math Academy, so the K I got 10, 20, 30 kids all using math Academy and they’re all in say algebra one. And I’m like, you know, you’re a teacher and you’re like, well,

Okay, it’s great that everybody’s learning optimally, but I kind of would like to corral people to get to certain points so we can do things together as a class. Like we might have a project that we’re going to do, but everybody needs to know a certain, certain skills in order to be, get anything out of this project. Right. So if they could say, well, I’m going to add gravity to certain topics, it will influence, it’ll pull students to there as quickly as possible. So that’s, that’s kind of what gravity is. And that’s why people.

wanted it.

Justin Skycak (20:19) Yeah, it gives people more agency over what they’re learning on the system. All at the same time, it’s like, well, you can’t override the system to say, I want to do this, I want to do that. Because it might not be a good decision. You might be missing prerequisites and stuff. But you can at least tell the system what you want to do. And it’ll go towards that situation in a way that works.

Jason Roberts (20:42) Just as if you were talking to a tutor. let’s say you’re, let’s say that your parent, let’s say as a parent, you hired your tutor to work with your kid. And let’s say they’re taking calculus and your kid’s a little behind because whatever reason they weren’t, they were sick. They weren’t paying attention. They didn’t do really, they weren’t taking the class seriously for the first quarter or second quarter. And so now it’s like, okay, well, we’ve got to remedy this situation. Well, you know, you have a, you have a, midterm coming up in a couple of weeks and that’s going to cover.

definite intervals. And you’re like, you’re struggling. They’re struggling with basic integration techniques, much less solving definite intervals. you’re like, you know, we tell the tutor, say, okay, well, listen, next, next, in the next week, you’re gonna take this test. The tutor will be like, okay, yeah, okay, I was gonna do some related rate stuff and but we’ll push that off the side. We’re gonna go and focus on integration techniques and

you know, areas and things and definitely have roles and limits of integration. We’ll just, we’ll tackle that stuff. Fine. That’s fine. And that’s, that’s what you would do in real life, you know, and, ideally math academy should work like an expert tutor. I mean, that’s, oh, that’s really our mental model. Like whenever you and I talk about a cycle, what should the system do? It’s like, okay, well, what would you do if you were tutoring? What would you do right in this situation? Then we should do that. The system should operate as much in that. So that’s.

You can’t get there as a tutor and just say, I want them to cover this topic right now. The tutor would be like, well, they don’t know. I can’t teach them that because they don’t know all these other things and need it. But what I can do is I can get them there. And you’d be like, OK. I respect your expertise as a tutor, but please just get her there as soon as possible because she can’t afford to not do well on this midterm, you know, whatever. So yeah.

Justin Skycak (22:25) Just like a tutor, it’s like, well, if you want to cover a large swath of knowledge in like a single day or two days, like, sorry, that’s just not going to happen because there’s a limit how fast you can build upon this knowledge. It’s like, we can’t go from not knowing any calculus to knowing a topic halfway in the course. We can’t do that over like two days. Even if you were to put in the work to try to build up this, all those missing prerequisites along that path, when you…

You’ve said this before in previous podcasts, when you build too much on unstable, unconsolidated knowledge, you struggle. And so you’re not really going to get through those topics successfully. Most likely you’re just going to be, you’re to get to a point where you’re just failing stuff because you haven’t allowed the prerequisite knowledge to consolidate. You haven’t done any review on that. You’re just trying to like eat the whole elephant in one bite.

Jason Roberts (23:18) And the thing is too, is that as we know from cognitive science is that long-term memory consolidates when you’re sleeping, right? Which means there’s only so much you can do in a certain day. Why? Because, okay, the biggest cheat to overcome, okay, so you have a certain amount of working memory. The average person has four to seven slots of working memory. So if you’re solving a problem, have to think of it like the size of your desk and each part of the problem is a

paper and you can only have so many papers on the desk before you just can’t look at them all at the same time. Okay? Some people have lucky and they have like the bigger desks, I can spread stuff out. Some people have smaller desks, less working memory. Okay. So what’s the cheat to get around that is the cheat is any information that is not the elements of the problem you want to be in your long-term memory. Cause if it’s a long-term memory, it does not take a slot up in your working memory. Right? So you get it for free.

And if it’s short-term memory, so you have working memory, you have short-term memory, you long-term memory. Short-term memory would be something maybe you just learned 15 minutes ago or an hour ago or whatever. Long-term memory is something that is embedded long-term. Okay, you know, you might have learned days, weeks, months, or years ago. Okay, so if you’re, if everything, if there’s a lot of things that have to do with this problem are all short-term memory, that’s all taking slots up in working memory, you just, you can’t.

Justin Skycak (24:46) cognitive overload. you’re, when you’re working memory utilization gets pinned up against the limit of its capacity. You enter this realm of almost like that database physics, right? Or does the weird stuff start, you may start making silly mistakes.

Jason Roberts (25:03) You’re

so overwhelmed, I don’t really know what’s going on. You can’t operate effectively. Yeah, exactly. So if you say, well, I’m going to learn three topics in a row today that build on each other, each one of those topics is probably taking up at least one, if not two slots of your short-term memory. So even if you’re a more gifted high-altitude learner who has more working memory, you’re quickly taking up those slots because you’re

you know, because it’s short-term memory, not long-term. Now, if you just learned one to two things at most a day, you let this stuff consolidate to long-term memory, then you’re operating with your full efficiency on the working memory. And so what you want to do, and ideally it goes to long-term memory, it’s usually not consolidate after one, but usually after repetition or two, right? So it’s like, it’s not just due to today.

Justin Skycak (25:58) Yeah, we talk about, right now we’re talking about it as though it’s like a discrete zero to one event, which I mean, you can kind of approximate it. Like, I mean, at the beginning, like not knowing it, sort of knowing it, but yeah, it’s really a fuzzy process, but just for listeners, like, yeah, there’s more, it’s a little more complicated, but this is essentially the big picture of what matters.

Jason Roberts (26:18) So speed running your way, there is a limit. So you have to put a governor on that process. Because I remember we first did that and then I remember thinking, was like, my God, I know what people are going to do.

Justin Skycak (26:31) Yeah, I’m gonna speedrun.

Jason Roberts (26:32) I wanna learn 10 topics in a row and I just wanna do them all one day. You can’t, that’s not gonna work. And then they’re gonna get confused and frustrated and they’ll think it’s our fault because it’s confusing and they understand the material.

Justin Skycak (26:42) Yeah, and the reason that this is such a real like this is not just a made up people are going to do this. It’s like we have people who I mean, not most people have more reasonable XP basis, but there’s a portion of people who are just like serious, like hardcore, like putting up hundreds of XP per day. They’re they’re learning like 10, 15 topics per day. And if you you allow them to just like, why don’t we just

instead of taking those 10, 15 topics and spreading them out like breadth wise along their knowledge frontier, if you allow them to go depth wise, now they are gonna overload their working memory. Because when you go depth, when you go breadth wise, like that’s fine. You can learn that you’ve got the prerequisite knowledge consolidated for all these different topics you wanna do. So you can learn, like you take 10, 15 topics along the breadth of the knowledge graph of what.

or you know the prerequisite material, no big deal. Cause any one of those, it only requires material that you’ve already consolidated. But when you start going like, oh, let’s just learn a bunch on same day depth wise, let’s go 10, 15 topics stacked on top of each other. Well, every single one of those topics that you’ve learned that day, like has not had time to consolidate yet. So it’s really kind of in your working memory. It’s putting pressure on your working memory much more than if you had spread that out.

breath wise. that’s

Jason Roberts (28:09) And people, a lot of people, people typically do not have any intuition about this. Like, why can’t I? And so what happens is they immediately do that, right? And I immediately said, right after I released it or was about to, I said, Justin, I just realized this is what’s gonna happen. We need a throttle. Like, they’re gonna do this. You’re like, really? yeah. I mean, the reality is that you wanna give people enough… ⁓

freedom to operate the system in a way that really fits what they’re trying to achieve, but you gotta put limits on it so it doesn’t allow them to do something that’s gonna be detrimental, right? Otherwise, the whole effort becomes a fail, they become frustrated, you lose them as a customer and they blame you. It’s like, okay, I’m not gonna let you drive the car into the side of a wall. It’s not a reasonable thing to do, the thing’s gonna auto brake, it’s not gonna happen, you know?

And so because, and then of course it turns into, they get frustrated, upset, and then they sit, I get like a page long email of them complaining about stuff. And you’re just like, you know, it’s like the one hand you want to give them this ability to customize stuff, but now it’s a customer support headache.

Justin Skycak (29:18) Yeah, you got enough of those. And even just from the gravity beta testing rollout, I mean, there’s other things where people just used it in different ways that, well, I mean, you had predicted some of these ways. ⁓

Jason Roberts (29:34) If there’s a way to misuse it, they’re gonna misuse it. I mean, I misuse it in a way that it’s not gonna, you know, you know, it’s like, hey, you know, like, I have some screws, I’m just gonna hammer it in, I’m not gonna use a screwdriver. It’s like, do you not, this is the hammer, this is a screwdriver, you don’t do it this way. I don’t understand, I hammered it in, but the screwdriver won’t work to bring it out, and I’m like, what are you doing, you know?

Justin Skycak (29:56) Well, you gave me the hammer. Why didn’t you tell me? Why didn’t you prevent me from using it in this way? Now I have now I just like screwed up all my construction. Yeah.

Jason Roberts (30:05) Yeah. So you, so there’s a danger, there’s a downside to giving people too much ability to customize, giving them too much control because they can tie themselves up in knots and which misses up, which causes them to be, have a poor customer experience. And then of course it increases your customer support load because then they’re confused and then they’re upset at you. And it’s just this whole thing where if I just said, we’re just not going let you do it. And then where it’s like a minor, it may be only a minority of people wanted to do it.

You know, but so, you know, you kind of have to make that calculation. Is it worth giving the freedom to do this knowing that it’s going to have to, we have to create a, you know, it’s getting the interface in a way that it’s simple enough for them to do it. So they don’t tie themselves up to not prevent some from doing it and it gives them enough information to understand why they can’t do stuff. Right. Cause even when you do explain it, they still don’t necessarily read it very carefully. Cause nobody reads that message box pops up and says, Hey, don’t it. They’re like, yeah, okay. Like they don’t read it.

and then they send you an email. You’re like, I write the message box, which said, you know what mean? So it’s like, it’s, there’s no right answer. It’s just, you have to decide what, pick your poison, right? Do I have, do I have a product that just doesn’t give anybody any control? And you’re just saying, look, it doesn’t give you any control. This is the way it works. And people are like, why? It’s frustrating. I want to do this. I’m not gonna use it. You’re like, okay. Okay, you give a bunch of people control and they tie themselves in knots and you have a lot of customer support problems. So it’s like.

Justin Skycak (31:28) It’s one of those problems like, you know, at the beginning when you have 10 users or whatever, zero user, like it doesn’t matter. You don’t have to worry about customer support. There’s no drag factor from customer support. Exactly. So your problem is just to attract customers, right? So you’re just like, build this. Yeah, exactly. Yeah. But right. Once, once you get to a,

Jason Roberts (31:40) enough customers for it to be that big of

customer place. One customer would be great.

Justin Skycak (31:55) a certain level of scale. It’s like, well, you can really shoot yourself in the foot by releasing something that is half-baked or just not even full-baked, by releasing it out to just everybody. now, I mean, it’s already like, even just the baseline amount of customer support is like a serious drag factor. And it’s just like,

Goodness. Like you plan on like, I’m going to do this, this, and this today. And then it’s just like, actually, no, I’m just going to do customer support for the first few hours of the day. And then I’ll get to what I was meaning to do.

Jason Roberts (32:27) I mean customer support is, ⁓ it’s a lot, you know, because Math Academy is, I mean it’s not a complicated, it’s not like learning, it’s not like Microsoft Excel or something or Photoshop, which is a huge range of what you can do and it takes a lot to get people up to a certain level, but ⁓ it’s also not trivial. And so in a lot of these people get on websites and they just were like, you know, they just want to just do whatever, right? They don’t want to read, they just, they don’t even click stuff, they just.

They don’t try things. just immediately email you like explain stuff. so, you know, like that load could be, be, be excessive, you know? I mean, it’s mostly people who just aren’t, even trying the most basic thing. You know, they don’t, they don’t, they don’t try to find their settings by just like, let me go to my name. Oh, there’s settings, hit the settings. Like they just don’t even do that. They’re just like, how do I change stuff? You’re like, really? I mean, I get it. If you spend five minutes and you’re a little of 10, five, you get frustrated and you’re like, ah, where’s the thing?

But you don’t even, this is weird. But the people do it and then it’s like, well, you gotta respond to them, you know?

Justin Skycak (33:36) Yeah,

reality of situation, got to deal with it. Would prefer if it were a different way, but this is the way it is. That’s what we got to adapt to.

Jason Roberts (33:46) It’s not that people are like, ⁓ people are just impatient and everybody’s like continuous partial attention. They’re doing it while they’re doing something else. they’re yeah, you know, they’re just kind of, you know. So we have to constantly try to do, to think ahead and be like, is this going to create customer support load? If so, how do we get ahead of it? I don’t know. that’s the, and given that my wife is,

customer support, like I suffer when there’s customer support load because she has a bad day. She’s like, I got so many emails. I got a bunch of people being really mean or being really rude. And she’s like in a really bad mood because of that. got, know, ⁓ cause it sucks. You know, it sucks that you get someone sends you emails all calf and it’s usually cause something they’re on, you know, nine times out of 10, it’s something they messed up. And then they’re yelling at you and it’s just.

You know, even if even though you can develop a little bit of a thick skin to it, it’s still it’s not as really not enjoyable. And so you try and do everything you can to make this stuff clear. But the thing is, like, and then you send emails to explain stuff, it would get mad. Why are you sending me emails? We see three emails like that. Right. And then it’s just like it’s a no win. It’s no win situation. It’s nowhere. I don’t know. So you can chip away at different sizes and do best you can, I guess, is like. Yeah.

Justin Skycak (35:11) Yeah, yeah, right. One of those things we’re just got to kind of grow into and figure out at each level of scale how to deal with it. So another thing that I wanted to talk about on the tech side is just schools using the system now. So we’ve had individual customers and we’ve had our own school program, which…

Jason Roberts (35:21) Exactly.

Justin Skycak (35:37) you would just do stuff in the database manually, Insert records, like, and so now we actually have like way more schools on using the system, which is like, that’s great. That’s awesome. But it’s also, it’s like, well, you need to build all this infrastructure, all this functionality to manage that effectively. Cause you can’t just be inserting manually, inserting all these records all day and they need to see information. They need dashboards. need

They need all sorts of stuff to use the system effectively. we don’t give them the ability to use the system effectively, then what’s the point of ⁓ having them use the system? It’s going to fail and we’re going to get the blame for it. So it’s like, well, got to give them support. it’s like, it just reminds me of, know, back when we were just getting our first customers, you would onboard every single customer through a video call for like half an hour, like an hour, just like an hour.

Jason Roberts (36:31) I was never less than an hour, almost always an hour, sometimes more. I try to keep it to an hour. ⁓

Justin Skycak (36:40) Serious chunk of time. Yep.

Jason Roberts (36:42) 100, 200.

Justin Skycak (36:44) 200.

Jason Roberts (36:46) That was a lot of phone calls, man. That’s a of Zoom calls. mean, okay, so that was a critical thing to do because when you’re doing a startup initially, you’re really trying to do customer development, which means you’re really trying to understand who your customer is, what the problems are that need solved. Right? Because you have this idea in your head of that. This is the customer, this is what they’re using, and this is what they’re going to do. And even if it’s sort of…

Mostly correct. There’s just a lot of detail that you don’t have. You need to go from low resolution to high resolution. And you’re like, right, I need this, I need that. I need to make this more clear. I need to change this setting or make this setting something that they’re do or whatever. So, you I would get on with all these people and so Sandy would set them up. For initially, they would just go and they’d basically reach out.

from the website and Sandy would schedule a call and put on my, would show up on my calendar and she’s like, well, you got three calls today. You got one at 10, you got one at 12, and you got one at one. was like, ⁓ geez. And I mean, I always enjoy talking to people, but it was exhausting. And girls, you’re basically saying the same kind of stuff over and over again. And of course, I don’t have as much time to actually build the solutions, right? Because I spend all the time hearing about, thinking, God, I gotta build this, gotta build that. So I did that with people.

And that was great. And then we got the onboarding strong enough and then setting strong enough where you have to do that anymore. And now it’s like, and then it got a point where schools are reaching out and say, want to like, we’ve heard some great things at math Academy. We’d really like to use it with this group of students or whatever. And we’re like, okay. And saying to be like, well, there’s this charter school and wherever, and they got 10 or 20 kids or something, and they want to.

And she’s like, what do you, how do you want to set this up? Initially, I just set it up like, as if it was like a parent with a bunch of kids accounts. Right. There was no, was like, they’re 20, you know, where the UI was not really built for. was sort of, you know, and I was set it up kind of manually. They’d send us a spreadsheet, you know, with, there’s this, these two classes of these teachers and each one had these students and this is the courses they’re in. And I’d have to, and I wrote a bunch of scripts and a bunch of

stuff and I’d have to manually go set it would take me like half a day to get that all set up because there’s always special case stuff I hadn’t thought about it’s like well there’s subscription but then it’s like get an invoice and then this person and then that you know and you’re just like and the same you’re like hey well can you set up this school that I’m just like oh yeah yeah half my day right and you know because thing is that because I was doing everything myself I had to be really careful am I typing the email

Did I spell this person’s last name correctly? Right? Because you’re doing all this stuff and it’s like, I’m just transcribing it and copying and pasting. And so that’s why it’s so slow. And it’s also cognitively taxing because you’re just worried about making a dumb mistake. Right? And then you just send them an email saying, hey, I just set up your classes for you. And this is how you want to log in. And this is their passwords. this is what you get because we don’t have like onboardies. I have to write a long email to try and help them understand how everything’s set up and what to do.

And I did that for, you know, I’ve been doing that for a year and a half or whatever it is. You know, was, and it was first it was like, it was infrequent enough. There was entity. was like, well, I’m not going to build an onboarding thing. Cause I just had like one school, right? And then it’s like two schools and that’s three and that’s five. Then it’s eight and it’s 10, you know, and then you’re just like, gee, it’s happening all the time. And it’s like really.

It’s like great to have a school, but then it’s also really annoying. And then they’re constantly emailing what, you know, Jason, is there a way that we could see this as a way it could do that? We want to add some students. We want to do this and they don’t have a way to do that. So you’d have to do it. All right. I would have to do it. Sandy would get an email from them and then cheat email me. You sell stuff. I mean, is this right? But you do everything manually. And so finally I was like, okay, this is no longer sustainable. And so I need to build.

a UI where Sandy can just go and there’s a dialogue box and go up and she can just like have all the stuff that she needs to create a school and the admin and everything. And then that person gets an email and then they can go in and then we build and then I built all the screens for adding team members, instructors and admins and adding students and setting up classes and all that kind of stuff. And so now they just get their admin account and it’s like, okay, go, go, go, go, go do your thing. you know. And so that was a huge.

like step forward. Because now that, but it’s not complete because now, but schools can still, cannot go to our website and up themselves. They have to kind of email Sandys. And a lot of them don’t even know that there’s most people that come to it, they wouldn’t even think there’s a school offering. It’s sort of like you go to restaurant and have like a special menu item or something. You’d have to know. it’s, so we need to, what we need to do is we need to have a, another thing on the website. like, are you a school?

Yes, I’m an admin. This is the school that I email addresses. this. How many, what do you want to do? And then they can just go in and enter it just like they’re an adult setting up our own account or for their kids. And then they can set everything and we could just maybe at the most, if we want to confirm it, you know, cause they’re going to so many accounts, but it followed the same. I follow the same procedure. You know, I remember listening to an interview years ago with ⁓ Brian Chesky, the founder of one of the founders of Airbnb. And he was talking about how.

you know, initially they just had a big spreadsheet of photographers that lived in these different cities where they had, where people were trying to rent their places out and they would, on the spreadsheet, they would assign it, go take photos of this place, you know, so they put it up and it was all manual. It was just a spreadsheet of stuff. But that’s how you start. Everything was like manual and you just grind through it that way until it get to a point where like, okay, this, this, this can’t work anymore. We have too many photographers, too many cities, too much we can’t.

You know, and then it sort of forces you to do it now winds up happening. At least it seems to happen in my case is you do the manual thing just a little too long. Right? Cause you don’t want to have to bite the bullet because it’s like, okay, do I spend two, three, four hours setting up the school, which is really annoying. Or do I spend, you know, however long a day, day and a half or whatever it is building out this UI stuff and all the back end functionality or just set this thing. I know for the school stuff, was course it was.

more like a couple weeks of work and you’re just like, I don’t have time for this.

Justin Skycak (43:19) Yeah. And it’s like a, it’s a continual thing too, right? As like, I mean, you, you build the thing that was, that was needed for the most painful thing of the time. But then as the schools start using the system more and more, they, you realize more stuff that they need and just, yeah.

Jason Roberts (43:38) Well, exactly. that’s, that’s what your customers kind of pull you forward. You know, it’s funny, this one guy, he’s like, you know, Jason, why don’t you just focus on homeschool and stuff? And I was like, what? Cause that seems to be enough market or whatever. said, yeah. This thing is, that schools are, you know, schools are going to use it anyway. Right. They’re, they’re going, they’ll just use it the wrong way and it’ll create customer support for us.

Because if they decide they’re going to use it, so they’re myself, because what happens is that people reach out, they’re kind of like really excited about it, like, you know, I’ve used this myself or my own kid or whatever. I think it’s great. I want to use it with our micro school or charter school or public, you know, this gifted group or whatever it is. And they’re like, if I say, well, we don’t really have a school set up, what they would just do is set it up like the parent with like 30 or a hundred kids. And it’s a giant mess. And then they’re emailing with support issues. And so they’re going to use, I don’t get to decide.

You know, and so all we can do is facilitate and then as, fine, if you’re going to use it, then let’s try and make it, you know, as seamless of an experience for you as possible so that not only are you happy, obviously we want more customers. That’s great. We want people to use it benefit from the system because that’s our mission is to make math more effective, you know, effective learning available to everybody. But the other thing is, you know, I just, I like the idea of that we’re

we’re going wider than just like one little niche of ⁓ customer. But they’re going to use it anyway. So you just got to deal with it. And if you don’t make it easy on them, they’re going to just make you suffer because they’re just going to

Justin Skycak (45:19) You’re going

to pay one way or the other. ⁓ If you don’t build the functionality, you pay in support. Or you pay to build the functionality. if you build the functionality, you actually get something in return for the work that you put in. You actually continue moving forward, whereas the support is just like you’re just burning time.

Jason Roberts (45:40) You know, and it might, know, there’s someone could say an argument, say, well, Jason, if you just niche down, he just really hyper-focus us as one group and you just say no to all these other people. And I’m like, that’s one approach. I get it. Some companies might do that. I don’t like that. I don’t, I’m, I don’t want to do that. I don’t.

Justin Skycak (45:57) We don’t want to be like, just the, you know, getting known as like, this is the math system for. Home schools. that’s it. that’s. No, anyone, anyone who wants to learn math, who is serious about learning math. That’s our customer. That’s who we want to solve.

Jason Roberts (46:05) For gifted kids or where the.

That’s our question.

Yeah. And, uh, well, most recently we had our first university professor want to use it. And so that was a whole different thing. So he’s like, I had, I did a phone call with him and he’s like, so he teaches at a liberal arts college in the Northeast. And he’s like, he’s like, yeah, Jason, know, said the thing is, he’s like, if I try and get this through the math department, it’s going to take forever.

He’s like, but university students or college students in these courses, like you can say, well, you’re going to use web assign or this certain types of, there’s certain resources or digital textbooks or textbooks that they just, this is a requirement for the course. He says, if it’s like underneath a certain amount, a hundred or $150 or whatever it was per quarter or something for the student, then they can pay. If it’s more than that, the math department would be like that’s.

You know, we can’t require them to spend, you know, thousand dollars or something, right? So this is not, so it’s like, it’s within this price range, it’s reasonable. And then, then it can work. And, that created a whole different flow because he has the class, he’s setting up with the class, it’s Calc 2, right? And he’s like, but I’m not paid, he’s not paying for it. So they’re paying for it separately, but they need to immediately get funneled into this class.

They need to have this payment search. They’re not doing a subscription. They’re paying for a, you know, three or four five month. This is the term, right? You’re doing a semester long or a quarter long thing. It’s as many as from this date to this date. You’re paying for it and you can renew it, but you’re paying a one time thing for access for this period of time. And each student is paying with their own credit card. And, but yet they can only do some for one course and they can only go and they immediately get funneled to class.

Totally different thing. Because if I just said, well, just tell all the kids, all your students to sign up and then send me their emails, and then I’ll manually create a class, it’s like a nightmare.

Justin Skycak (48:14) Yeah.

Jason Roberts (48:16) And so I bit the bullet and spent like a, you know, whatever, a couple of days working on it because then the sign up flow is all different, how it interacts with SRIPE and because now you’re doing invoices, payments as opposed to subscriptions and it’s the whole thing. And, but now that it works, it’s like any college professor comes to say, hey, you know, we’d like to use Math Academy for this. It’s like, great, here’s a Deceptive class and.

Now you go, now, now what you, you go, you know, I still, had to say his instructor. had to set him up as an instructor, but once he’s an instructor and it’s for a university, he just goes, the professor just goes and hits invite students and you paste in a bunch of emails addresses like you do with any of these kinds of group oriented apps, Slack or anything else. They get emails say, Hey, you’ve been invited to count to caught by professor, blah, blah, blah. And click here to accept it. And boom, take some, they have a signup page, right? You know, purchase the whole thing. So it’s just like.

Justin Skycak (49:12) Nice. Yeah.

Jason Roberts (49:12) Nice.

So the next step, of course, just like for schools, is that when you go sign up, it would say, you a school K through 12 or a university, are you professor, are you independent? So then they could go down kind of like a wizard and then would funnel them into the sign up process. And then it would help them get everything configured. So that would be kind of the next step. That way they don’t have to email me and we have to do the whole thing.

Justin Skycak (49:37) Yeah, you know, it’s kind of a kind of interesting thing, a realization that I’ve had while watching you do all the stuff to support the schools is that like, never realized how much effort goes into setting up just, you know, sign up and payment systems. it’s like, like schools are like, it’s not like they just put in like a, it’s not like an individual user payment, right? It’s like a completely different sort of system and different schools have sort of different ways of.

doing this, right? So you got to kind of support all sorts of general conventions for that sort of thing. yeah. But yeah, so that’s another, just yet another one of those things that pop up.

Jason Roberts (50:23) It’s like you kind of have to organic, as we talk about do things organically, because you don’t really know ahead of time. So we said, like we’re going to create this system and it’s going be for schools, it’s going to be for adult students, it’s going to be for parents, you know, and then you really, you don’t understand how the configurations need to work and what people really need to know. You kind of guess, but it’s all just very confusing. And I remember thinking about it a sign up for, even just the more basic, it’s like, so,

There’s an adult account, a parent account, but they’re not actually a user, they’re a parent, but then they have students, but they themselves could be a user. And you have an adult thing. mean, was very, like, because nobody does it like that. If you go to, like, my model, my mind is you go to, like, IXL and you pay your 10 bucks a month, so your kid can do drills or something like that. You just sign up. You don’t really have a parent account. You don’t really see what they’re doing. The kid just has a login or something. And just like, we need more oversight. We need more stuff.

You know, I’m sure there are other sites that do something similar, I hadn’t had any, I hadn’t seen anything like it. So I had to kind of sit there and think through like, what is Huzzle Sneaking work?

Justin Skycak (51:28) Imagine if you had tried to build out all that payment infrastructure just at the beginning, know, before doing this manually, before going through all the reps. And he had some, some vision of like, ⁓ this is how things should work. Doesn’t this make sense in theory? And then he built all that out. It’s like, not only would that be time wasted, but you’d also have to, you’d have to choose between like kind of undoing and rebuilding it all or, or, or just trying to like contort it into some way that it should be, or trying to force customers to.

to use it the way that you think it, which is not the way that the billing actually occurs in practice. And it just would have been a bad situation either way. So just another instance of that kind of like earn the right to automate, earn the right to solve the problem. Like just do it manually until you understand what the heck you’re doing.

Jason Roberts (52:14) Yeah, that’s sort of interesting. It’s like, I think there’s a dance that happens between the entrepreneur and the customer. So you could say, well, I think that I have a hypothesis that this is a problem that needs to be solved. And then you might go talk to some potential customers before you build anything, or you might build something simple, and then you get some feedback from them, and then you…

fine tune your direction and then you get some more feedback and it goes through process of kind of a, kind of a, you know, give and take, right? And so you can’t just go off and build a ton of stuff and not get feedback from customers because you just build the wrong stuff. So even if, whether it be building an initial version of your product, you say, I’m just going to work on this for years and then release it. That tends to be pretty dangerous because you, you miss the mark. You’re, you could just wasted.

years of time and a ton of money. And there are lots of examples of that happening at a very large scale in business, as well as innumerable smaller scale examples of that that most of us never heard of. But even on a, when you’re doing a feature set or an individual feature, it’s like, you know, mean, like we’re talking about the gravity thing. Like we hear people want to do something, you know, or, you know, people on Sandy will say, you know, people keep asking, they want to do this thing.

because she’ll get tons of emails from people. Or even you and I will hear from them on Twitter, whatever people want stuff. it’s like, sometimes it’s stuff that we anticipated. You know, we said, you know, thought people might want to do that at some point. Yeah, we should do that. And so the case is you’re like, really? Hmm, OK. You know, and so you just have to, you can’t be overly, I guess, of arrogant about, you know, what the world wants or cares about. You have to say, well, I think people will like this.

But it’s usually, but I think it works a lot better when you can have a general direction, but you let the customer lead to some degree where almost like there was that exam. I can’t remember who made this up, but it’s like, I read this at one time and talking about it’s better to, if you’re like laying out like a campus, like a university campus, put the sidewalks where the paths through the grass occur and then pave over those. Cause that’s where people were going.

Because if you put them over somewhere else, people are just going to walk through the grass anyway. Right? You just wasted a bunch of money and now you got to what? Rip up the sidewalk, an or grab multiple sidewalks kind of in close proximity, which would be kind of silly. Yeah. So all this stuff, building stuff by hand, doing stuff manually, doing tons of zoom calls, talking to people. You’re just, you’re getting to know your customers’ problems and needs and you’re dealing and you’re solving their problems manually. A lot of times it’s really painful, but that sort of informs you about what the

with the shape of the problem really is because sometimes it’s more complex or nuanced than what you initially imagined. And it wasn’t until you had multiple customers all asking for certain things. if you’re just like, the first customer asked for it, then you build it. That might have been sort of an odd case or a special case. And you just kind of got unlucky because you built.

Justin Skycak (55:26) want to overfit to an equals one.

Jason Roberts (55:29) Yeah, but you say, OK, well, I’ve done this like 20 times. I think I got it pretty good. This is the problem. So that’s kind of been our approach is just trying to be trying to do whatever we can to help customers, even if it means me doing stuff manually in the database, writing scripts, whatever, doing phone calls, zoom calls, just trying to support them however I can. And then eventually going, OK, I need to build the second phase is.

you know, is I might be building some administrative UI so that Sandy can just take care of it in two minutes as opposed to me taking me three hours. Yeah. And then after that, then you move it to like a self serve kind of sign up wizard settings or whatever that they can do. And then that’s sort of the progression.

Justin Skycak (56:14) This is like the silver lining of the pain of doing this over and over again, right? It’s like if you’re feeling pain from manually doing something because it’s tedious, because you’ve done it before, it’s like, what that pain really represents, it’s like, I already know how to do this. It’s not that hard. You just do this and this, but it’s like really annoying to do. It’s like, that’s a marker of opportunity to automate.

Jason Roberts (56:40) Here’s an example just popped up yesterday. So we have this parent whose two kids were at one particular school that uses our system. And the parent has a supervisory account for the two kids. One of the kids is no longer at that school, they’re at another school. But they want to sign up an account for the student who left the school to still use it.

And I was like, what’s their insane? He’s like, well, they can’t sign up on my walk in sound because our email is already in the system. Like you can only have one email per user account, so they can’t go sign up. Right. So I’m thinking, okay, so. how often is this going to, said, well, I can maybe just. I’ve been like thinking, okay, maybe I’ll just, let me just change their email to something temporary. Tell them to sign up and then I’ll like flip the IDs and you know what I do manually. Right. Cause I’m like, how often is this going to happen?

Well, first it always starts that way. Well, how often is it going to happen? Right? Well, it’s like the first time it’s ever happened or maybe the second, and then it’ll happen again. And the bigger you get, then pretty soon it’s a thing. And then it’s like, cause I told her, I I have too much to do this stuff. High priority things to do. can’t build a workflow for this now. Can we just, can we just do it manually? And she’s like, yeah, sure. You know, like we always do, but eventually, okay, what happens? Your, your emails and you go, you try and sign up.

your email is already used, it detects it, this is used as a view only supervisory account for something else. You don’t have a credit card on file, but you got a great credit, no, it detects that, says, okay, you want to sign up, you want to put a credit card on, you know, it’s a whole, the whole thing. It’s not super complicated, but you know, it’s, could be a day or two of work to get totally right. These things are always more complicated than you think. And not, and I mean, oh, well, like,

There’s some database things that happen and some payment things have to happen. But then there’s all the UI flow that sometimes is you got to make it work. So doesn’t confuse the person that the customer is there like, wait, what is happening anyway? But that’s example, right?

Justin Skycak (58:36) Yeah.

This kind of thing plays out like over and over and different. This was just one example of a thing that is a constant flow of these sorts of situations, right?

Jason Roberts (58:47) Yeah, well, how can I do a thing and I want to do this other thing and this? That’s like, oh, geez, okay, how am I going to do this? I mean, look, I’m incredibly grateful that we’re in a position where we have these problems.

Justin Skycak (58:59) great problem to have.

Jason Roberts (59:01) Right. And we have customers. mean, I’m every day, I’m just like, this is so great. Even though it can be overwhelming at times and exhausting and frustrating when you get customer support load of, you know, I’m thinking, you know, it’s, it’s like, my God. then just like, it’s a good problem to have, but it’s, you know, it’s still like, you’re not out of it. You know, I was talking to a friend of mine, you know, day and he was almost like,

you you’re good. I’m like, dude, like what, you think I’m like retired or something just because math Academy is profitable. Like, what do you, I mean, we’re, you know, we’re in the middle of it. mean, I have just the, you know, I have a ton of stress working. I work by, I work seven days a week, you know, almost every week, even on an off day when I kind of half-assed that there might be a Sunday where I only put three or four hours in. And I’m like, wow, that was a complete day off, you know.

But that doesn’t happen very often because I couldn’t live with the guilt and the, you know, also there’s just too much and Sandy can never take it off. She’s like, if I take a day off, I wake up and I got 50 more emails, support emails to deal with than I would have had. Can never, ever, ever take a day off. know, people say, well, Jason, why don’t you just hire some customer support? Okay, well, first of all, yeah, we do need to hire.

We probably do need to hire some help with customer support. the problem with customer support is you have to have deep knowledge about your customers and how the system works. People who would want to get customer, it’s like this turnover rate is crazy. So you really want to hire someone who’s really high caliber person. So how do you get someone who’s really high caliber person who’s not going to leave and, know, cause you’re gonna take a month to get them really be able to answer questions, right. And be sort of real insider and be the kind of person that would be a net positive and then not to.

Not to leave. right now it’s, you know, it’s Sandy’s doing all the customer support still. Which is not sustainable, but that’s the, that’s, that’s the reality where we’re at right now. And, ⁓ but yeah, she could never, never take that off. So it’s like, and my friends said that I was like, dude, what are you talking about? Like you’re acting like I’m like some, you’re an easy, what? This is working my ass off. Crazy. About how much work this is.

Every freaking day.

Justin Skycak (1:01:22) It’s much preferable to the stress of uncertainty early on that kind of like weighs on you. It’s a hard day’s work where you can actually like go to sleep being like, all right, you you just think of like a lumberjack goes out, chops wood, it’s tired as hell, falls asleep versus the person who’s figuring out just how am I going to be alive next year as a.

Jason Roberts (1:01:45) Am I going to get a job? If you’re unemployed and you’re trying to get a job, the kind of stress that comes with that kind of situation. Yeah. Can’t sleep. my God. Yeah, I 100 % agree. As I said, feel incredibly grateful that things are going well and that we have control over our own destiny. That’s just, you know.

Justin Skycak (1:01:52) This is worst. Keeps you up at night.

Yeah,

it’s a crap ton of work.

Jason Roberts (1:02:10) There’s a ton of work. It’s like, you have to be signed up for it. It’s like someone who’s maybe like a player in the NFL. Yeah. And on one hand, it’s like they have to grind. are in physical pain all the time from how hard it is on their body to train their ass off year round. They have to eat incredibly well. have to, you know, it’s just ton of work. it’s hard, but they’re incredibly grateful. I’m in the National Football League, man. Like I am paid to play football.

I mean, this is awesome, but it’s also super, super hard for them.

Justin Skycak (1:02:42) Yeah. know, one of the reactions I got from somebody when I was letting them know like, like we actually, we got from not having customers to having customers being profitable. It’s great. And they’re like, wow, great. So that means he can like, take some rest time now. Right. Kind of recuperate. What are you talking about? No.

Jason Roberts (1:03:03) Oh, you got drafted. made it. Dude, mean college football is nothing to the NFL. I I’m like this is what I do all day every day. Am I going to get cut next year? My contract? mean, am I going to get injured? I’m out. Like I got to do every possible thing I can do to stay on the field. You can pay. Yeah, that’s kind of how it is. Exactly. So well, you know, it’s funny. I was thinking about like, I was just talking to Sandy the other day about how like, you know,

the role you’ve played, how that’s expanded so much from just purely quantitative and it’s just gone more and more and how you’ve just gone through this whole process. And I think it’s really, really beneficial because you have a much more, a much deeper understanding of the entire business, you know, so that it’s like.

Justin Skycak (1:03:51) Yeah, I feel like I’ve gotten reps on basically everything. Yeah, like building the quantitative infrastructure, building just general backend infrastructure, the content side of thing, tons of lessons, tons of questions, building tools for the content doing, guess, I guess I am on X now and like helping out with marketing.

Jason Roberts (1:04:12) your personality you got a fits to all that’s kind of

Justin Skycak (1:04:15) I don’t even know what I… Who even am I? I don’t even understand myself anymore.

Jason Roberts (1:04:25) Yeah, it’s like you’re thrown in the ring. It’s like the movies. You have the person just thrown into some crazy action movie and they just have to adapt. And at first they’re just kind of a normal person. And then all of a sudden, after just years of the situation, they just have to grow into, to survive. And that’s kind of what you’ve been. But the fact that you’ve had so much experience with so many things, it’s leveled you up in so many areas so quickly, which I think…

Justin Skycak (1:04:33) Yeah.

Jason Roberts (1:04:54) And why that’s important is that you have a much more holistic understanding of everything so that when you’re building things, it’s not you’re just in the silo that you don’t really understand what’s going on, you know.

Justin Skycak (1:05:02) Definitely a lot of, you know, I can think back to times when I was just working on quant stuff. That’s like, well, my concerns, my, my, how do we improve the system? How do we, whatever like, well, it’s always got this quant lens that I’m looking to, right? So it’s like, it’s like people, people see what they’re familiar with, right? So it’s, and, and, and now it’s kind of like, yeah, looking back, can see, yeah, just getting, getting the reps kind of broad, getting, skilling up and spinning up on, every area definitely is, it’s,

I have much better understanding of.

Jason Roberts (1:05:34) And then go for a business that had zero customers. Yeah. And was, I mean, was like, how long do we work on this for? Maybe two years. You worked on it two years. I had been working on it for many years before that, but you worked on it for two full years before we had our first customer, right?

Justin Skycak (1:05:53) Yeah,

yeah, started working with you on the software side or summer of 2019.

Jason Roberts (1:05:59) And it was August of 2021 when we had our first customer.

Justin Skycak (1:06:02) That’s right. Yeah. And I should, should say one thing to clarify for listener. I anyone who’s wondering like, wait, like, didn’t you say earlier not to work on something for too long before like, well, what we were doing is we had this in-person school system, right? In the Pasadena Unified School District that we were building the software, running classes with having students solve problems on the system. So it’s like, well, they weren’t our customer.

Jason Roberts (1:06:29) for

free.

Justin Skycak (1:06:30) But we got a lot of signal on like, okay, what are the actual real problems that need to be solved? That’s really the, it’s not so much that you can’t go too long without any customer. It’s like, you need to get real signal on the real world. Like what is it, the problems that people are experiencing and get wood on the ball, moving those solutions forward.

Jason Roberts (1:06:53) Yeah, so I had a very, the very initial version of the system I was using was like maybe 2016, I would say. Something like that, 2015, 2016, yeah. And so there was several years of us using, it was more of a manual system, you know, for the first three or four years where you would just, as an instructor, would create an assignment, just select a bunch of problems from different topics and the kids would do them and.

That was where we started. But then of course we implemented all the space repetition and interleaving and all those kinds of things. And doing it manually, which was just sort of a very rough way to do it. that having done that and then, and then trying to push kids through so they could, you know, finish calculus in middle school and you’re really trying to optimize outcomes and efficiency. And so all those things that we learned, you know, there was a lot of information that was gained even by the time you came on. Although you are helping, you were working as a

teaching at TA and.

Justin Skycak (1:07:52) I

to list that earlier. Yeah. In addition to the business side of like the content software, everything also like literally teaching classes with, with the software. Like I’m like working on like coding up stuff while sitting in the teacher’s chair as the kids are doing their work. I’m going to go in and like, go check around on all the kids and go back to like coding. So, oh, this kid got, uh, need some help with this thing. Like, okay, well, uh, what does that mean about this? Oh, the system should have done this, should have done that. Okay. Let’s, let’s fix that. Let’s.

You know. ⁓

Jason Roberts (1:08:23) Well, and

every day when you would come home, okay, when you are yours, would, but before, before the pandemic, cause you, cause you sheltered in place with us during the pandemic, before that you would come to our place and work with me every day after teaching. And so then we would do a full, um, like kind of a, um, a breakdown of how the day went. Every day I’d like, I said, let’s walk through how to go and do this. How’s this kid doing? How’s this kid doing? What’s going on with them? What’s going on with that, this class?

And you would go through everything. would just, but that was really helpful for us to thinking through the strengths and weaknesses of what we’re doing, what we needed to do.

Justin Skycak (1:09:03) Totally. And you know, there’s the, when it comes down to like user personas, we didn’t have to make up any user persona. Like I get to name the literal kids. I got like 10 kids in this category. can tell you the names, like exactly what happened, like all the context around it. Like, yeah. Yeah.

Jason Roberts (1:09:21) It’s like you

went to some some class and got a teaching. You’re say, well, some students this and some students like, no, no, I can tell you this is, this is what they did. This is why this is how it works. Yeah. But, but even before then, like you had still like the previous year you were like a TA and a, you would substitute for some of our high school classes. Even middle school classes, you would, you would run teaching at TA sessions.

Justin Skycak (1:09:48) Yeah. Yeah. This was back when I was just tutoring full time, like just a freelance tutor. Yeah. 2018.

Jason Roberts (1:09:53) It was 2018.

And then you had taught a summer course in AMC, like problem solving. So you had done a lot of, you were doing a lot. It wasn’t like you just jumped in and started working, you already had a lot of experience.

Justin Skycak (1:10:08) Yeah.

And that was after working as a tutor at Mathnasium for four years. I have 300 kids I worked with there that I can…

Jason Roberts (1:10:20) One that I

Justin Skycak (1:10:20) know

all the categories. Yeah.

Jason Roberts (1:10:22) Well, then all the kids you knew, well, okay, also all the kids that you tutored individually, right, for SAT and other math classes, ACT, and you taught as a full-time, as a physics and calculus teacher at that private school for a year. Yeah, that’s right. ton of experience, you know, all these different things pulling in. yeah.

Justin Skycak (1:10:37) Yep.

Jason Roberts (1:10:47) Understand how it works. Yeah, I think that’s the thing is I think when I think it’s harder to really Develop a lot of insight and how something should work until you see all the failure modes and you’d experience them personally many times and just like this is This is how this is not gonna work. This is how this is gonna

Justin Skycak (1:11:07) Yeah,

that’s funny. Yeah. This is how this is not going to work. That’s really… Do that enough times that I mean, what’s remaining is the truth, right? It’s what’s going to work. A great part of those, like you don’t have to depend on anyone else telling you like, this is how it’s going to work. You don’t have to depend on trusting somebody else like, this person knows what they’re talking about. They say this is what generally works. I guess that’s what it’s like.

Jason Roberts (1:11:19) Here’s what’s going to look

Justin Skycak (1:11:33) I can tell you, I can tell you exactly. Let me go through the derivation of this principle derivation by repeated attempt failure, attempt failure, attempt failure. Okay. I guess this is the only thing left that can work. ⁓ great. That worked great. Well, there we go.

Jason Roberts (1:11:49) Yeah, I mean, the only way it really helped, it was like, you know, when you and I would talk about it and I would give you some suggestions and it would just sort of accelerate. it’s like, I would recommend emailing the parents early and often about the, you’re like, that’s a good idea. And I mean, it’s like, just sort of, it allowed you to speed run some, some progressions, but to suffer some of the stuff. then I’d say, well, you know, I would, you might, should do this if you’re not doing that or maybe consider doing that. And so you’d be like, huh, okay. But you would, you were still suffering.

Justin Skycak (1:12:07) Yeah, exactly.

Yeah.

Yeah. Well, I think it’s also like, I mean, you can also tell it’s like somebody who actually knows what they’re talking about gives you advice. You don’t, it’s just like the life quotes thing that we talked about earlier, or it’s like, well, you, have some idea of what it means, not quite, but you, you at least have a direction. And, and when you start leveraging what you think that means, you, see some improvement in the situation. And so you’re like, yeah, there’s not like, you don’t exactly escape all of the, all of the

pain and go from zero to solved exactly. But you, at least you, you see a path and it was like, Hey, like I try, I, I, I leverage this advice to the extent that I could understand it. And it actually mitigated some bad things that could have happened. And the bad thing that did happen actually happened because, you know, in hindsight, I didn’t do this, which is probably what Jason had meant in his advice. It’s like, well, you just keep, keep doing, keep leaning into it more and more.

Jason Roberts (1:13:17) Yeah, I think that’s what ideally in life is like you need to do things yourself, but you need to have somebody who’s maybe done it before who can just sort of help you recognize and sort of frame what’s going on so then you can kind of speed run the lessons you take from it. You’re like, okay, well, yeah.

Justin Skycak (1:13:34) Exactly.

No skipping, only running faster down the long path. Well, any other tech stuff that we’ve forgotten that you want to cover or you want to call it?

Jason Roberts (1:13:45) There’s a ton of stuff we could talk about, I think we’re good for now. I’m like, you know, see, I’m always itching to get back coding. It’s fun to talk about it and stuff, but then you’re just, the more I talk about it, more I’m like, God, I gotta get to work. I got stuff to build.

Justin Skycak (1:14:00) Yep. Same. Yeah. Just going to get off and get back to the data analysis stuff that we’re talking about last night.

Jason Roberts (1:14:10) Yeah, yeah. All right, cool. All right, man. Well, that’s a wrap.

Justin Skycak (1:14:13) We’re out.

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.