The Harvard University strategic initiative HarvardX is running a massive open online course (MOOC) that features adaptive learning and assessment algorithms that tailor course material in response to student performance.

“Adaptive learning functionality,” through which a computer system can fit learning experiences to the needs of each student, had not been offered previously in a HarvardX course and is featured in a few courses across the edX online learning platform. But it is being used in “Super-Earths and Life,” taught by Harvard’s Phillips Professor of Astronomy Dimitar Sasselov. The presentation is designed to gain a preliminary assessment of the technological feasibility and implications of adaptive functionality to online course design. Depending on the results, the technology may expand to other HarvardX MOOCs.

“Adaptive learning programs are very good at speeding up information acquisition and lengthening retention, as well as individualizing learning to help learners see where they have difficulty,” said Peter K. Bol, Harvard’s vice provost for advances in learning (VPAL) and Charles H. Carswell Professor of East Asian Languages and Civilizations.

According to Robert A. Lue, HarvardX faculty director and professor of the practice of molecular and cellular biology, digital content when used properly can respond faster than a classroom instructor. “One of the big advantages of learning in the digital space is the ability to get real-time feedback,” he said. “Adaptive functionality allows users to optimize learning based on their performance, offering a personalized path to mastery.”

And, as VPAL research faculty director and Harvard Government Professor Dustin Tingley said, the possibilities of adaptive learning go beyond simple multiple-choice questions. For instance, it can figure out where a student’s strengths lie and adjust the material accordingly. “The idea is to tailor the experience so that students are more likely to get what they need,” said Tingley. “The broader mission is to make sure that students are really benefiting from the online learning experience.”

VPAL researchers and HarvardX technologists collaborated with educational technology company TutorGen to implement an adaptive learning algorithm within a subset of “Super-Earths and Life” content. VPAL researchers leveraged the existing algorithm to build out, in-house, an adaptive learning technology to use within a HarvardX course in different ways.

Grading the MOOCs

“Super-Earths” users were evenly divided into experimental and control groups. On course homework pages, control group learners received a predetermined, non-adaptive set of problems of varying difficulty (easy, medium, and hard). The experimental group received the same assessments, but its problems were served sequentially, one by one, in an order based on prior performance. Given a wrong answer, the assessment provides an easier follow-up question; given a right answer, it serves up a more difficult one.

The adaptive algorithm estimates the user’s mastery of a learning objective each time he or she answers a problem tagged with the objective (even if the problem is outside the adaptive assessments). If the problem is advanced, the engine serves the instructional system advanced materials covering the necessary learning objectives, providing students with an option to study these before attempting the problem. Students in the control group also have access to these materials in an optional part of the course.

Ideally, students will progress to the harder material at their own pace. More importantly, the technology could help prevent students from feeling daunted and dropping out before completing the course.

“Super-Earths” is a logical HarvardX course to assess the impact of adaptive technology, since its layout features a number of multiple-choice tests. The adaptive approach, Sasselov said, “works well for ‘Super-Earths’ because the class covers a very interesting topic, but also because it involves physics and biology — high-level science. Students on an advanced level are able to push further, and the adaptive course is the ultimate way to accomplish that.”

Preliminary results suggest that the adaptive algorithm is helping foster efficient learning in “Super-Earths.” In an effort to examine the effect of adaptive technology on learner performance, engagement, and completion in online courses, VPAL research analyzed “Super-Earths” course data between the course’s launch in mid-October and the first week of January. Differences in learner efficiency and performance, on average, were noted in the adaptive experimental group in a number of key areas.

The adaptive group moved faster through the course materials, showing a lower net time on task (4.37 hours) than the control group (4.80). The adaptive group also showed more persistence in course assessments (making 2.99 attempts per advanced problem, compared with the control group’s 2.41 attempts per advanced problem), while attempting fewer problems overall (3.99) than the control group (5.42). Overall, the experimental group outperformed learners in the control group by achieving a 19 percent larger knowledge gain after completing adaptive assessments, compared with non-adaptive settings in the control group.

VPAL research and HarvardX plan to expand adaptive learning functionality in future HarvardX courses and further explore how the technology can improve online learning outcomes.

“There is strong evidence, broadly, in the field that adaptive learning is one particularly exciting way to leverage the possibilities of a digital platform,” said Lue. “Our hope is that adaptive functionality will over time become a feature in a significant percentage of HarvardX classes, and provide powerful new assignment types in the Harvard classroom.”

More detailed analysis of adaptive learning and assessment algorithms deployed in Super-Earths is available on the VPAL research website.