Massive study on MOOCs
Harvard, MIT report provides new insights on an evolving space
Since “Year of the MOOC” became a catchphrase in 2012, massive open online courses have had their fans and detractors. Some have claimed that online learning is a “disruptive revolution” and a harbinger of the end of residential colleges, while others have called MOOCs at best “mere marketing” or at worst an abject failure, singling out low completion rates.
Expanded data and research about MOOC participants and evidence-based assessments of online learning trends might, however, begin to move the conversation beyond anecdotes and heated opinions.
Today, a joint Harvard and MIT research team published one of the largest investigations of MOOCs (massive open online courses) to date. Building on their prior work — a January 2014 report describing the first year of open online courses launched on edX, a nonprofit learning platform founded by the two institutions — the latest effort incorporates another year of data, bringing the total to nearly 70 courses in subjects from programming to poetry.
“We explored 68 certificate-granting courses, 1.7 million participants, 10 million participant-hours, and 1.1 billion participant-logged events,” said the study’s co-lead author, Andrew Ho, a professor at the Harvard Graduate School of Education and chair of the HarvardX research committee. The research team also used surveys to gain additional information about participants’ backgrounds and intentions.
Ho and MIT’s Isaac Chuang, professor of physics, professor of electrical engineering and computer science, and senior associate director of digital learning, led a group effort that delved into the demographics of MOOC learners, analyzed participant intent, and looked at patterns that “serial MOOCers,” or those taking more than one course, tend to pursue.
“What jumped out for me was the survey that revealed that in some cases as many as 39 percent of our learners are teachers,” said Chuang. “This finding forces us to broaden our conceptions of who MOOCs serve and how they might make a difference in improving learning.”
Key findings
The researchers conducted a trend analysis that showed a rising share of female, U.S.-based, and older participants, as well as a survey analysis of intent, revealing that almost half of registrants were not interested in or were unsure about certification. In this study, the researchers redefined their population of learners from those who simply registered for courses (and took no subsequent action) — a metric used in prior findings and often cited by MOOC providers — to those who participated (i.e., by logging into the course at least once).
Participation in HarvardX and MITx open online courses has grown steadily, while participation in repeated courses has declined and then stabilized
From July 24, 2012, through Sept. 21, 2014, the end of the study period, an average of 1,300 new participants joined a HarvardX or MITx course each day, for a total of 1 million unique participants and 1.7 million total participants. With the increase in second and third versions of courses, the researchers found that participation in second versions declined by 43 percent, while there was stable participation between versions two and three. There were outliers, such as the HarvardX course CS50x, “Introduction to Computer Science,” which doubled in size, perhaps due to increased student flexibility: Students in this course could participate over a yearlong period at their own pace and complete at any time.
A slight majority of MOOC students are seeking certification, and many participants are teachers
Among the one-third of participants who responded to a survey about their intentions, 57 percent stated their desire to earn a certificate; nearly a quarter of those respondents went on to do so. Further, among participants who were unsure or did not intend to earn a certificate, 8 percent ultimately did. These learners appear to have been inspired to finish a MOOC even after initially stating that they had no intention of doing so.
Among 200,000 participants who responded to a survey about teaching, 39 percent self-identified as a past or current teacher. Twenty-one percent of those reported teaching in the course topic area. The strong participation by teachers suggests that even participants who are uninterested in certification may still make productive use of MOOCs.
Academic areas matter when it comes to participation, certification, and course networks
Participants were drawn to computer science courses in particular, with per-course participation numbers nearly four times higher than in courses in the humanities, sciences, and social sciences. That said, certificate rates in computer science and other science- and technology-based offerings (7 percent and 6 percent, respectively) were about half of those in the humanities and social sciences.
The larger data sets also allowed the researchers to study those participating in more than one course, revealing that computer science courses serve as hubs for students, who move to and from related courses. Intentional sequencing, as was done for the 10-part HarvardX Chinese history course “ChinaX,” led to some of the highest certification rates in the study. Other courses with high certification rates were “Introduction to Computer Science” from MITx and “Justice” and “Health in Numbers” from HarvardX.
Those opting for fee-based, ID-verified certificates certify at higher rates
Across 12 courses, participants who paid for “ID-verified” certificates (with costs ranging from $50 to $250) earned certifications at a higher rate than other participants: 59 percent, on average, compared to 5 percent. Students opting for the ID-verified track appear to have stronger intentions to complete courses, and the monetary stake may add an extra form of motivation.
Questions and Implications
Based upon these findings, Chuang and Ho identified questions that might “reset and reorient expectations” around MOOCs.
First, while many MOOC creators and providers have increased access to learning opportunities, those who are accessing MOOCs are disproportionately people who already have college and graduate degrees. The researchers do not necessarily see this as a problem, as academic experience may be a requirement in advanced courses. However, to serve underrepresented and traditionally underserved groups, the data suggest that proactive strategies may be necessary.
“These free, open courses are phenomenal opportunities for millions of learners,” Ho emphasized, “but equity cannot be increased just by opening doors. We hope that our data help teachers and institutions to think about their intended audiences, and serve as a baseline for charting progress.”
Second, if improving online and on-campus learning is a priority, then “the flow of pedagogical innovations needs to be formalized,” said Chuang. For example, many of the MOOCs in the study used innovations from their campus counterparts, like physics assessments from MIT and close-reading practices from Harvard’s classics courses. Likewise, residential faculty are using MOOC content, such as videos and assessment scoring algorithms, in smaller, traditional lecture courses.
“The real potential is in the fostering of feedback loops between the two realms,” said Chuang. “In particular, the high number of teacher participants signals great potential for impact beyond Harvard and MIT, especially if deliberate steps could be taken to share best practices.”
Third, advancing research through MOOCs may require a more nuanced definition of audience. Much of the research to date has done little to differentiate among the diverse participants in these free, self-paced learning environments.
“While increasing completion has been a subject of interest, given that many participants have limited, uncertain, or zero interest in completing MOOCs, exerting research muscle to indiscriminately increase completion may not be productive,” explained Ho. “Researchers might want to focus more specifically on well-surveyed or paying subpopulations, where we have a better sense of their expectations and motivations.”
More broadly, Ho and Chuang hope to showcase the potential and diversity of MOOCs and MOOC data by developing “Top Five” lists based upon course attributes, such as scale (an MIT computer science course clocked in with 900,000 participant hours); demographics (the MOOC with the most female representation is a museum course from HarvardX called “Tangible Things,” while MITx’s computing courses attracted the largest global audience); and type and level of interaction (those in ChinaX most frequently posted in online forums, while those in an introduction to computer science course from MITx most frequently played videos).
“These courses reflect the breadth of our university curricula, and we felt the need to highlight their diverse designs, philosophies, audiences, and learning outcomes in our analyses,” said Chuang. “Which course is right for you? It depends, and these lists might help learners decide what qualities in a given MOOC are most important to them.”