Why is the EdTech Industry So Damn Soft?

If you depend on a massive base of learners, most of whom are unserious, that puts hard constraints on how you teach. You have to employ ineffective learning strategies that do not repel unserious students.

When I tell people I work in math edtech, it’s initially kind of embarrassing.

They think of edutainment videos or arithmetic games where dinosaurs dance in front of you for answering 2+2 correctly.

Now, I’m fortunate enough to work on a learning system that’s pretty hardcore. Every decision we make is based on the science of optimizing student learning. It’s like quantitative finance but instead of optimizing return in the stock market, we’re optimizing learning efficiency in students’ brains. We go all the way up from 4th grade to university-level math (like, serious math major math, well beyond calculus).

But why are other companies / learning systems so damn soft in this industry?

To some extent, I think we can trace the softness back to the expectation that online learning should be free or ridiculously cheap.

You know what happens if a company does that?

They back themself into a corner where their survival depends on a massive user base.

But the problem is, most people are not serious about learning.

Effective learning feels like a sweaty, exhausting workout with a personal trainer, and you have to come back and do that at least several hours each week.

Yeah, everybody wants to learn, but only a small fraction of people are willing put in the work.

If you depend on a massive base of learners, most of whom are unserious, that puts hard constraints on how you teach.

You have to employ ineffective learning strategies that do not repel unserious students.

Which basically means you’re a fraud.

You’ve put yourself in a position where the only game you can play is convincing your users they’ve managed to learn things despite putting in little to no work (for instance, by cherry-picking the simplest cases of each topic and letting students move on despite poor performance on prerequisite material).

Okay, let’s trace this issue back even further: why are edtech companies so afraid to charge money?

On one hand, there’s some degree of societal expectation that all learning/education should be free.

But also, maybe it’s also a matter of (again) nobody wants to do the work.

You know what happens when someone pays you a nontrivial sum of money? They hold you accountable for results.

When your existence depends on your ability to make students learn, that’s a lot of pressure!

If a student gets stuck and can’t make progress in your system, then you’re out of a job.

If the learning on your system doesn’t show up in students’ grades and test scores outside of your system, then you’re out of a job.

But there’s a silver lining.

When you hold students accountable for learning, they hold you accountable for providing material that is easy to learn from.

And more generally:

When you hold your users accountable for getting value out of your product, your users hold you accountable for creating a valuable product.

Some Clarifications in Response to Follow-Up Questions

I received some great follow-up questions about this post after it gained some traction on HackerNews (August 2024). Below are some points I’d like to clarify. Feel free to contact me if you have any additional questions that aren’t addressed here.

How can you justify subscription pricing for education (\$49/month or \$499/year for Math Academy)? Compare this with buying a book. You get a high-quality resource which you own, forever. Information should be a one-time payment.

It sounds like you’re comparing Math Academy’s pricing to that of a textbook or collection of video lectures, whereas a better comparison is 1-on-1 coaching from a personal trainer who is developing your mathematical talent.

To understand just how expensive this would be otherwise, let’s work through the cost computation. How much would it cost to develop a student’s mathematical talent to the maximum degree with a 1-on-1 private coach, assuming a reasonable amount of daily work time that is in line with the amount of time that students would be working anyway during the school year?

In an academic subject like mathematics, 1-on-1 private coaching would be obtained from a tutor. Now, we are NOT calculating how much people typically pay for supplemental tutoring. We are NOT supposing that the tutor functions as a supplemental assistant who helps a student through their class homework. INSTEAD, we are supposing that the tutor functions as a main instructor, specifically, a private coach who engages the student in 1-on-1 talent development using a personalized training program that is tailored and constantly adapting to their individual needs.

We are supposing that the tutor is hired to completely replace the student’s mathematical training from school, which, as a conservative estimate, is approximately 1 hour per day, 5 days per week. (This estimate is conservative because students typically have 50 minutes of class each day plus 30-60 minutes of homework.) A tutor typically charges at least $50/hour, and $50 × 5 days/week × 52 weeks/year = $13,000.

This ballpark lower bound is in line with Guryan et al. (2023), who describe a successful low-cost tutoring intervention (40 minutes per school day, 1 tutor per 2 students) that cost about $4,000 per student per year, with tutors being paid a yearly stipend of only $16,000 (plus benefits) while working through the entire school day (6 class periods). Under these conditions, a full hour of fully individualized tutoring (1 tutor per student) each school day would cost $12,000 per student per year (= $4,000 × 2 × 60/40). (Note that while these tutors described by Guryan et al. (2023) possessed strong math skills, they were not long-term expert coaches – rather, they were “willing to devote one year to public service – for example, recent college graduates, retirees or career-switchers – but do not necessarily have extensive prior training or experience as teachers.” Needless to say, long-term expert coaches would be far more costly and harder to find.)

All this to say: bringing that $13,000 pricetag down to $499/year (26x cheaper) makes mathematical talent development accessible to many, many more people. Sure, that’s not everyone, and there are still people who are priced out. But to me at least, providing a 26x cheaper option feels like a good starting point towards a goal of making mathematical talent development accessible to more and more people.

Of course, the following question still remains: “Why do you even need the learning experience to feel like working with a personal trainer? What is the benefit over a textbook, Khan Academy, MIT OpenCourseWare, etc.?”

The answer to that question: learning efficiency.

Let me tell you about my own experience. I self-studied a bunch of math subjects on MIT OpenCourseWare (OCW) when I was in high school. OCW is a good resource and I came a long way with it, but for the amount of effort that I put into learning on OCW, I could have gone a lot further if my time were used more efficiently. Just to name a handful of inefficiencies in OCW:

• not super scaffolded $\to$ you periodically run into situations where you bang your head on a wall thinking "how the heck did they get from here to there?" and it takes a long time to figure out what kind of logical leap is happening (if you figure it out at all)
• doesn't track your knowledge / make sure you've mastered the prerequisites for anything new you're supposed to learn $\to$ you often feel a large gap between your level of knowledge and the new material, which leads to more banging your head on a wall trying to figure out what prerequisite knowledge you're missing and how to learn it
• no spaced review $\to$ you quickly get rusty on a lot of what you learn, which not only means you come out of the course having forgotten a lot of content, but even during the course, you're constantly forgetting prerequisites
• doesn't adapt to your level of performance $\to$ you waste a lot of your time doing the wrong amount of work. Sometimes you grasp a topic quickly and end up doing way more practice problems than you need; other times you struggle with a topic and don't do enough practice problems to reach mastery
• leaves the definition of "mastery" open to interpretation by the learner $\to$ as a learner, it's hard to know when you've mastered something well enough to continue moving forward. You often think you've learned something well enough, when you actually haven't -- but you won't know unless there's an expert who is evaluating your knowledge. On the flipside, you can also take things too far being a perfectionist, spinning your wheels on the same topic for a week over some minor point that doesn't make perfect intuitive sense to you, when it would be more productive to just keep moving forward and solidify your understanding by building on top of it.

I could keep going with this list (happy to do so if you’re interested, just contact me), but by now you probably get the point:

all of these things introduce unproductive friction into the learning process, leading you to make less educational progress per unit time/effort that you put towards learning.

That’s one reason why I’ve been so motivated to help build Math Academy. We take away as much of this learning friction as possible and maximize your learning efficiency.

That’s our main value proposition: sure, it’s possible to learn math elsewhere, but it’s way more efficient with us.

Despite it being possible to learn math elsewhere, most learners don’t actually do it because there’s so much friction in the learning process.

And that’s the real kicker: efficiency is important not only because you make faster progress, but also because you’re less likely to quit.

In practice, people get off the train and stop learning math once it begins to feel too inefficient. In anything you do, once the progress-to-work ratio gets too low, you’re going to lose interest and focus on other endeavors where your progress-to-work ratio is higher.

Efficiency keeps that progress-to-work ratio as high as possible, keeping you on the math learning train as long as possible.

(I also want to emphasize that I think it's great that there are free resources available. Different students have different needs; some are willing to invest in higher learning efficiency while others prefer a free resource even if there's more friction in the learning process. And that's perfectly okay!

It's kind of like how, in the fitness industry, there's a variety of options. Some people want personal training, other people just want full gym access, other people just want a couple pieces of minimalist equipment that they can use to work out from their home.

But like I said, while (e.g.) OCW is a great resource, and I came a long way with it, and it was totally life-changing... for the amount of effort that I put into learning on OCW, I could have gone a lot further if my time were used more efficiently.)

As a follow-up to the previous question: couldn’t a student reach maximum benefit by doing one or two sessions a week with a talent development coach and having homework to do on the other days? This brings cost comparison down to 5x or 10x.

I see your point – however, in my comparison, what I’m trying to get at is this: imagine that when a student goes to school each weekday, instead of spending an hour in a traditional class that is not personalized to their needs, they spend an hour with a 1-on-1 teacher who engages the student in personalized training exercises. This is essentially what Math Academy is.

When people hire tutors for supplemental tutoring, it’s typically once or twice a week, but as I explained in the previous question, that’s NOT what we’re talking about here. What we are talking about is a setting where the teacher / tutor / coach continually sits next to the student, analyzing their performance on every single problem that they do, and decides the exact moment to move the student on to a new topic or problem type.

Additionally, I want to point out that in the Platonic ideal of education, there is no real difference between “lesson” and “homework” – instead, minimum effective doses of instruction are interspersed with minimum effective doses of active problem-solving, where every single problem is carefully selected in response to the learner’s performance on the previous problem(s). If you want to engage in optimal practice, you need your talent development coach there to provide feedback and guide you on how much practice to do on each skill, what to practice next, and to keep you pushing forward into new skills. Without your coach there, you might be able to continue making progress in an absolute sence, but the practice would be suboptimal in an absolute sense.

If you’re still skeptical about this, then just do a thought experiment: say you had a kid and they had the opportunity to work with a 1-on-1 expert math tutor for 1 hour per day, 5 days per week, instead of attending a traditional class. Wouldn’t that be kind of a no-brainer? ;)