Can You Automate a Math Teacher?

For many (but not all) students, the answer is yes. And for many of those students, automation can unlock life-changing educational outcomes.

Lots of people try to answer this question despite not having experience on all sides of it: teaching manually, developing an automated learning system, and manually working with students who are learning from an automated system.

This is precisely my background. So, I figure I have a responsibility to weigh in.

• I write all the adaptive learning algorithms for Math Academy, manage our student learning analytics, and calibrate our system to student learning data.
• I spent about a decade teaching/tutoring math human-to-human, as well as several years working hands-on with students who were using the Math Academy system.
• I spent several years pushing the limits of what's possible with manual teaching, developing what was (during its operation from 2020-23) the most advanced high school math/CS sequence in the USA.

The Spectrum of Human-vs-Machine Reliance

It’s true that as we’ve refined our content and algorithms over the years to better scaffold the instruction, personalize the learning experience, and adapt to the needs of individual students, we’ve enabled more and more students to achieve comprehensive mastery without the need for a human math teacher in any capacity. (For instance, many students get a 5 on the AP Calculus BC exam working solely on our system, no human teacher needed.)

This works for surprisingly many students. But the thing is – despite our best efforts, there are also many students who will go off the rails without a human teacher or tutor to keep them on track. For example:

• Some students have major attention issues and need a human to reel them back in when they get distracted every minute.
• Some students have severe math anxiety and will totally freeze up if they don't have a human next to them providing encouragement.
• Some students find basic math extremely hard and need a human to help them work with physical manipulatives (e.g. counting blocks) and/or explain a concept a hundred different ways until something sticks.

There are also students who are “on the edge” in the sense that most of the time, they are fine learning without a human teacher, but every once in a while, they need some human support. Some teachers whose students are like this use Math Academy as the primary resource in their classroom and walk around checking in and helping students as needed.

Additionally, there are other students who are capable of learning without a human teacher but who enjoy the environment of a classroom with a human teacher where they can have ad-hoc enrichment discussions about various aspects of math that interest them.

All this to say, when it comes to teaching math, I think humans and machines can enjoy a symbiotic existence. There is so much demand for math learning that goes unserved – so much that even if machines are capable of serving a lot (or even most) of it, there will still be more than enough demand to require a supply of human teachers.

Human-vs-machine reliance in education is on a spectrum, and while the distribution will likely drift towards less human involvement over time, I think the following things will always be true:

• Some students will learn entirely from machines.
• Some students will learn their core curriculum entirely from machines but will enjoy ad-hoc enrichment discussions with humans.
• Some students will learn primarily from machines but will need human support every once in a while.
• Some students will need humans throughout most or all of their learning.

And if you replace the word “machines” with “books” in the above, they have always been true. The distribution has always existed; new technology (books ⟶ computers ⟶ adaptive learning software) just shifts it upwards.

Passion vs Instruction/Accountability

Some people think a machine should not ever replace a teacher, even if a student is able to learn the subject matter just as effectively (or even more effectively). I never really understood this viewpoint until I was accosted by one of its proponents. They summed it up as follows:

Here are two reasons why this viewpoint is misguided.

1. Instruction/accountability is more important than passion.

Although there is a dimension of “passion” along which teachers can be very positive or negative, while AI (at least in the current conception) is more neutral – if a math teacher spends “90% of their time/effort just being an adult who likes math” at the exclusion of optimizing instruction strategies and holding students accountable for learning, then the outcome is still a disaster.

I’ve gotten a front-row seat to the dumpster fire on multiple occasions. A well-intentioned teacher focuses all their energy on class discussions about mathematical beauty and cool applications, thinks that because they’re so good at that they don’t have to optimize instruction strategies and hold students accountable, and graduates students who can’t solve even the most basic kinds of problems that they were supposed to have learned in the class. This creates catastrophic issues for students, their teachers, and their educational program as a whole as they enter more advanced classes without the prerequisite knowledge.

In my experience, when you weight instruction/accountability versus passion, you get the following outcomes:

• great outcome: HIGH instruction/accountability, HIGH passion
• good outcome: HIGH instruction/accountability, NEUTRAL passion
• bad outcome: HIGH instruction/accountability, NEGATIVE passion
• bad outcome: LOW instruction/accountability, HIGH passion
• terrible outcome: LOW instruction/accountability, NEUTRAL passion
• worst outcome: LOW instruction/accountability, NEGATIVE passion

2. There is a lot to gain from optimizing instrution/accountability.

The above outcomes suggest while there is value to be had from a passionate human teacher, even students who learn solely from an AI machine can still enjoy good educational outcomes (which is better than most students receive from most human teachers).

But that’s an understatement. In our experience, optimizing instruction/accountability has allowed us to accelerate student learning by 4x – meaning that on our system, serious students learn 4x the amount of material in the same time (or the same amount of material in a quarter of the time) as compared to traditional classrooms. And that’s being conservative, since our courses tend to be even more comprehensive than what you’d find in a traditional classroom (our courses aim to cover the superset of all content that one could reasonably expect to find in any major textbook or standard class syllabus).

This can be life-changing for students. It launches them into the greatest educational life hack: learning advanced math (and coding) rigorously at a young age and benefitting wildly from the resulting skills and opportunities. This life hack can rocket students into some of most interesting, meaningful, and lucrative careers. But it remains unknown to most students who have the potential to capitalize on it, because few people realize how quickly students can learn in an environment where instruction and accountability are optimized.

With that in mind, let’s add the missing outcomes (“life-changing” and “best”) to the top of the list:

• best outcome: MAX instruction/accountability, HIGH passion
• [optimized machine]
  life-changing outcome: MAX instruction/accountability, NEUTRAL passion
• great outcome: HIGH instruction/accountability, HIGH passion
• good outcome: HIGH instruction/accountability, NEUTRAL passion
• bad outcome: HIGH instruction/accountability, NEGATIVE passion
• bad outcome: LOW instruction/accountability, HIGH passion
• [most human teachers]
  terrible outcome: LOW instruction/accountability, NEUTRAL passion
• worst outcome: LOW instruction/accountability, NEGATIVE passion

Effective Teaching Requires Automation

Of course, it’s hard for a human teacher to achieve this level of effectiveness by optimizing instruction/accountability. Such optimizations have been known in the literature for a long time – for instance, mastery learning, spaced repetition, the testing effect, and varied practice have been researched extensively since the early to mid 1900s, with key findings being successfully reproduced over and over again since then. But the problem is that using these strategies systematically to their fullest extent requires an infeasible amount of effort for any human teacher.

For instance, consider spaced repetition. You have to keep track of a repetition schedule for every topic for every student, and each time a student learns (or reviews) an advanced topic, they’re implicitly reviewing many simpler topics, all of whose repetition schedules need to be adjusted as a result.

In fact, before building our online system, we actually did a very loose approximation of spaced repetition while teaching in a human-to-human classroom. It turned out that, teaching just two classes with only a handful of students in each class, it took more time and effort than a full-time job to implement a very loose approximation of spaced repetition (for the class as a whole – not even personalized to individual students). And that’s just one of many strategies that are necessary for effective teaching!

Compared to the scenario described above, a standard teaching load consists of about 3x as many classes and 4x as many students per class. Let’s say optimal teaching requires (conservatively) 5 different cognitive learning strategies to be implemented on a fully personalized level. Then, for a typical teacher, we can ballpark-estimate that optimal teaching would require the time and effort of about 3 x 4 x 5 = 60 full-time jobs. Which is totally infeasible.

Totally infeasible for a human, at least. But not for a machine. Our solution was to take all of these strategies (including plenty of others not mentioned above such as interleaving, layering, cognitive noninterference, cognitive load minimization) and build an adaptive automated online learning system that leverages them to their full extent. Our goal is for our system to emulate the decisions of a perfect human tutor who knows everything about their student and everything about math.

To sum it up: the most significant way to improve teaching is to offload as much as possible to an automated system so that the single human teacher is no longer a bottleneck. Using long-known learning strategies to their fullest extent requires an infeasible amount of effort for any human teacher – but just because a human can’t do that, doesn’t mean that there’s little to gain from it.