Learning Rate Differences Exist

A recent study measured a 2x learning rate difference between the 25th and 75th percentile -- likely an underestimate due to methodological choices. The authors reported it as 1.5x and called it an "astonishing regularity."

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.

The existence and magnitude of learning rate differences is a hard truth that many refuse to accept because they wish it weren’t so.

There’s a recent paper that

• not only measured the difference as 2x between the 25th and 75th percentile
• but also did this using a methodology that systematically undershoots the difference (by assuming all differences in initial performance stem entirely from differences in background knowledge unrelated to learning rate).

Unfortunately

• the authors did not acknowledge the methodological shortcoming,
• they reported the 2x factor as 1.5x in rate of raw increase in expected accuracy (which is really 2x when you measure in log-odds, a more appropriate metric that accounts for the fact that it’s harder to increase accuracy when one’s accuracy is high to begin with),
• and they concluded that 1.5x difference between 25th and 75th percentile is “An astonishing regularity in student learning rate” and named the paper as such,
• so it inspired a bunch of media articles with titles like "the myth of the quick learner" and "quick learners don't exist,"
• even though it's actually evidence to the contrary.

More info: Critique of Paper: “An astonishing regularity in student learning rate”

Want to get notified about new posts? Join the mailing list and follow on X/Twitter.