AI & Learning
Adaptive Practice and Its Limits
Adaptivity is the most over-promised word in educational technology. Two effects in the learning-science record are real and worth building on; almost everything sold above them is decoration.
Adaptivity is the most over-promised word in educational technology. It is printed on landing pages beside the kind of verbs — learns you, understands you, meets you exactly where you are — that imply a private intelligence attending to each student with a teacher's intuition. The reality underneath is almost always smaller, and the smallness is not a failure. It is the point. A handful of effects in the learning-science record are robust enough to build on, and they ask for restraint rather than ambition. Most of what is sold as adaptive learning is decoration laid over those few effects, and the decoration is where the harm begins.
This is an attempt to draw the line honestly: to say what sequenced, spaced, and adapted practice can really do, on the evidence; to name where the science is firm and where it thins into marketing; and to argue that the most useful adaptive system is the one that does less, says so plainly, and shows its reasoning. We build for four national examinations, and we have come to believe that the conservative version of this technology is not a compromise. It is the only version that keeps its promises.
What Memory Actually Wants
Begin with the oldest finding in the field. More than a century ago, Hermann Ebbinghaus measured his own forgetting and drew the curve that has carried his name ever since: knowledge, once acquired and then left alone, decays steeply at first and then more slowly, until little remains 1. The curve is not a counsel of despair. It is an instruction. It tells you precisely when a memory is most in danger, and therefore when a well-timed return will do the most good.
Two interventions bend that curve, and the evidence for both is among the sturdiest in cognitive psychology. The first is spacing: practice distributed across time produces far more durable learning than the same amount of practice massed together, an effect confirmed across hundreds of experiments and a wide range of materials and intervals 2. The second is the testing effect: the act of retrieving a fact — pulling it back out of memory, effortfully, rather than reading it again — is itself one of the most powerful ways to fix it there 34. Recall is not a measurement of learning that happens elsewhere. Recall is the learning.
Put plainly, memory wants to be asked, and it wants to be asked again later. Almost everything an adaptive practice system can legitimately do is a way of arranging those two requests well. The schedule on which a student meets a question again, and the insistence that they answer it before they are shown the solution, account for the bulk of the genuine benefit. They are not glamorous. They are merely true.
The Corridor of the Not-Yet-Known
If spacing and retrieval describe when to practise and how, a third idea describes what. Lev Vygotsky's notion of a zone between the already-mastered and the still-impossible — the region a learner can reach with support but not yet alone — gives sequencing its target 8. Practice that lands inside the already-known is comfortable and nearly useless; practice that lands in the impossible is demoralising and equally useless. The whole art is to keep a student in the narrow corridor where the work is hard enough to matter and possible enough to finish.
This is the honest core of difficulty selection, and it is far more modest than the marketing implies. A system does not need to model a mind to do it. It needs a defensible estimate of which skills a student has consolidated and which remain fragile, and a rule for serving the next item at the edge of that frontier. The estimate can be made transparent: a difficulty band, a recency, a count of recent successes. Knowledge tracing — the family of techniques that infers mastery of a skill from a sequence of right and wrong answers — formalises exactly this, and it has done so, without mystery, since the mid-1990s 9. What it produces is a probability, not a portrait. Shown to the student, it is a useful map. Hidden behind a confident interface, it becomes a small fiction.
Advance on Evidence, Not on the Calendar
The third pillar predates the computer entirely. Benjamin Bloom's mastery learning proposed something quietly radical for a schooling system organised around the calendar: that a learner should move to the next unit when they have demonstrated command of the current one, not when the timetable says so 5. Hold the standard fixed and let time vary, rather than holding time fixed and letting attainment vary. The studies that followed found gains large enough that Bloom framed the challenge of reproducing one-to-one tutoring at scale as education's defining problem 6.
The shape of an individual skill's mastery curve explains why this matters for sequencing. Gains per attempt are steepest while a skill is fragile and taper as it consolidates; the marginal value of a fourth correct repetition is far smaller than the value of the first. A system that advances on evidence spends a student's limited attention where the curve is steep and stops spending it where the curve has flattened. That is the entire economic argument for adaptivity, and it is a good one. It is also bounded. It does not require the system to be clever. It requires it to count honestly and to wait.
Honest adaptivity is narrow, legible, and conservative. It schedules, it withholds the answer until you have tried, and it advances you on evidence. Everything sold above that line should be read as decoration until proven otherwise.
The Engineering Is Mostly an Interval
What, concretely, does a defensible adaptive practice engine decide? Less than its name suggests. Strip the language away and the working part of most spaced-repetition systems is a rule for choosing one number: how long to wait before showing an item again. The Leitner system, devised in the early 1970s with index cards and a row of boxes, already contained the whole idea — answer correctly and a card moves to a box reviewed less often; lapse and it falls back to the front 7. Modern schedulers refine the spacing with response history and, more recently, with optimisation that targets a chosen recall probability rather than a fixed ladder of intervals 14. They are better tuned than a shoebox of cards. They are not different in kind.
This is worth saying because it sets a ceiling on honest claims. An interval scheduler that withholds the answer until the student has attempted recall is delivering the two effects that the evidence most strongly supports, and that is genuinely valuable. But it is not reading the student's mind, and it cannot diagnose why an answer was wrong, only that it was. The richer judgement — recognising a misconception, choosing the next explanation, knowing when to push and when to console — is tutoring, and the evidence is clear that this is where the largest effects live and where automated systems still fall well short of a skilled human 1213. A practice scheduler is a fine instrument played within its range. The trouble starts when it is sold as an orchestra.
Where the Claims Outrun the Evidence
The gap between what adaptivity can do and what it is said to do has a recognisable shape. The most durable example is the "learning styles" promise — the idea that matching instruction to a student's preferred visual, auditory, or kinaesthetic mode improves outcomes. It is intuitive, it is popular, and the controlled evidence for it is essentially absent 11. Adaptivity that claims to detect and cater to such styles is adapting to a variable that does not predict learning. It is precision aimed at nothing.
A subtler failure is the manufactured number. A confident percentage — 87% mastery, learner profile complete — invites a trust that the underlying estimate cannot bear. Knowledge tracing yields a probability with real uncertainty around it; rendering it as a crisp figure on a dial converts an honest guess into a false promise. The remedy is not better marketing but plainer disclosure: show the band, show the recency, show how many recent attempts the estimate rests on, and let the student see the seams. When a system cannot justify a number, the honest move is to withhold it, exactly as it withholds an answer until the student has tried.
This is why the conservative version of adaptivity is also the more honest one. The mechanisms that survive scrutiny — distribute the practice 2, make the student retrieve 34, advance them on demonstrated mastery 56, keep them in the reachable corridor 8 — are the ones a careful review of effective learning techniques keeps returning to 10. They are unglamorous, decades old, and reliably real. Everything stacked above them should be treated as decoration until it earns its place.
A Discipline of Restraint
We hold ourselves to a narrow definition, and we prefer it that way. Adaptive practice, done honestly, schedules retrieval against the forgetting curve, withholds the solution until a genuine attempt has been made, advances a student on evidence rather than on the clock, and keeps the work inside the corridor where effort is repaid. Each of those rests on findings that have survived repeated, adversarial testing. None of them requires us to claim a private understanding of the student we do not have.
The questions a student practises here are not invented on the fly to flatter an algorithm; they are drawn from a verified bank, some three thousand strong, each one solved and checked before it is ever served. The sequencing chooses among real, vetted items and shows its reasoning where it can. What it cannot justify, it does not display. That is the whole of it, and the smallness is deliberate.
The temptation in this field is always to promise the teacher and ship the scheduler. We would rather ship the scheduler, call it a scheduler, and be precise about the modest, real good it does. Restraint is not the absence of ambition. In a field this crowded with overclaiming, it is the most ambitious thing a learning system can offer: the discipline to do only what the evidence supports, and to say so plainly enough that a student can trust the rest.
References
- 1.Ebbinghaus, H. (1885). Über das Gedächtnis: Untersuchungen zur experimentellen Psychologie. Duncker & Humblot.
- 2.Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380.
- 3.Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.
- 4.Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966–968.
- 5.Bloom, B. S. (1968). Learning for mastery. Evaluation Comment, 1(2), 1–12.
- 6.Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16.
- 7.Leitner, S. (1972). So lernt man lernen: Der Weg zum Erfolg. Herder.
- 8.Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Harvard University Press.
- 9.Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253–278.
- 10.Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques. Psychological Science in the Public Interest, 14(1), 4–58.
- 11.Pashler, H., McDaniel, M., Rohrer, D., & Bjork, R. (2008). Learning styles: Concepts and evidence. Psychological Science in the Public Interest, 9(3), 105–119.
- 12.Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78.
- 13.VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.
- 14.Tabibian, B., Upadhyay, U., De, A., Zarezade, A., Schölkopf, B., & Gomez-Rodriguez, M. (2019). Enhancing human learning via spaced repetition optimization. Proceedings of the National Academy of Sciences, 116(10), 3988–3993.
Start preparing with EuraStudy
Join the waitlist to be first in when your curriculum opens.