Runeer via iStock
The Internet was supposed to revolutionize and democratize education. In particular, massive open online courses (MOOCs) were going to, as the New York Times wrote in 2012, “bring the best education in the world to the most remote corners of the planet.”
to your inbox.
While MOOCs have indeed opened up courses taught by faculty from renowded institutions to students all over the world, a persistent concern is low completion rates: only about five percent of people who register for the courses actually finish them. Most people, it seems, have a hard time engaging with course material in the isolation of their own homes—particuarly when they have little financial skin in the game.
“People have started to realize that MOOCs may not do what some of the early visionaries thought,” says Jan Van Mieghem, professor of managerial economics and decision sciences at the Kellogg School. “A key concern of online education is obviously the lack of collaboration among students.”
He wondered whether a collaborative environment could be cultivated in a MOOC. And, if so, if it would help students do better in the course.
So Van Mieghem and fellow Kellogg School researchers—Gad Allon, also a professor of managerial economics and decision sciences, and Dennis J. Zhang, a doctoral student—decided to test out a few techniques in their own MOOC. Specifically, they decided to see what the impact would be of encouraging interactions via the class discussion board and one-on-one digital discussions with fellow students.
“A key concern of online education is obviously the lack of collaboration among students.”—Jan Van Mieghem
They found that these small prods to interact can have a modest impact on students’ engagement and performance.
The researchers’ MOOC—a five-week-long course on scaling operations that consisted of four weekly lectures and a weekly quiz—attracted more than 24,000 registrants, some of whom paid to take the class in return for a certificate of accomplishment and others of whom took it for free. About 4,200 students submitted at least one of the quizzes.
In their first experiment, the researchers sent a survey at the beginning of the second week to all course participants, asking for feedback on the first week’s material. Those who returned the survey were then divided into two groups, one of which recieved an email reminding them to contribute to the course’s discussion board.
The encouragement worked. The simple nudge increased visits to the board by 26.5 percent and increased posts to the board by nearly 97 percent.
And visits to the board were linked to better performance. The researchers found that each additional visit to the board in the first week increased the likelihood that a student would complete the following week’s quiz by about 3.5 percent. And, on average, each student who received the nudge visited the board four additional times a week, meaning that overall, students who received the encouragement to visit the board were, on average, 13 percent more likely to complete the following week’s quiz.
The visit did not improve quiz scores, however, and after that first week, visits to the discussion board had rapidly diminishing influence on quiz-completion rates.
Still, Van Mieghem points out that the cost—a single email—was very small. “We did a very minor kind of stimulus,” he says. “We just said to students, ‘Hey, don’t forget to go to the discussion group.” That there were effects at all, he says, is a “hopeful sign.”
In their second experiment, the researchers invited some participants to take part in online one-on-one discussions about the course material. They found that students who took them up on their offer were 10 percent more likely to complete their weekly quiz—and their quiz scores increased by 2 to 10 percent in subsequent weeks.
The upshot is that the direct communication fostered by one-on-one discussions seems to be more conducive to actually learning the material. Still, only 7 percent of students who received an invitation to take part did so. One barrier is the time and effort it takes to set up—and show up for—a digital discussion. Another is the anonymity inherent in online education: “If people don’t show up, there’s no way to penalize them,” Zhang says. “And they won’t even feel bad, because there’s no shaming factor. If they don’t show up, no one knows who they are.”
The researchers note that their results may not be relevant to all online educators. For one, neither manipulation had a measurable effect on one group of students—those who actually paid to take the course. Paying students, it seems, were highly motivated to take part in the discussion board and one-on-one chats, even without the researchers’ nudges to interact.
But one immediate takeaway is that, among nonpaying students, early social engagement packed the most punch.
“You really have to think hard about the incentives that will help [students] at the beginning of the class,” Zhang says, “which is not the typical approach with discussion boards.” Much current thinking, he says, is focused on ways to highlight and recommend posts that would be relevant to students. But in the early weeks of the course, there are too few posts for that strategy to be helpful.
It is a principle that also holds true for online commerce, according to Zhang. For businesses that use online surveys to request customer feedback, for example, sooner is better than later. “If you’re trying to facilitate interactions with customers, you have to engage them early, or they will lose interest.”
Because the MOOC will be offered on an ongoing basis, the researchers will continue to experiment with ways to encourage social interaction—and hopefully, course completion. They are optimistic that the sheer scale of online education will make it possible for them to refine their techniques.
Many educators running a field experiment “would be impressed if they had 30 students,” says Van Mieghem. “But we’re talking about 10,000 or more students. So the scale, and the amount of data, and the kind of experimentation we can do—it’s just very impressive, and something that cannot be done in a typical, physical channel.”
He expects that, even if online courses are not the revolutionary force some people imagined they would be, they will continue to evolve.
“The reason I got involved is that we have no idea how this is going to work out,” he says. “And my feeling was, instead of letting it come to me, I’d rather be there early and be able to influence it. There’s no doubt that, one way or another, some of this technology will stay.”
Zhang, Dennis J., Gad Allon, and Jan A. Van Mieghem. 2015. “Does Social Interaction Improve Service Quality? Field Evidence from Massive Open Online Education.” (September 21). Available at SSRN.
Will AI Eventually Replace Doctors?Maybe not entirely. But the doctor–patient relationship is likely to change dramatically.
3 Tips for Reinventing Your Career After a LayoffIt’s crucial to reassess what you want to be doing instead of jumping at the first opportunity.
What Happens to Worker Productivity after a Minimum Wage Increase?A pay raise boosts productivity for some—but the impact on the bottom line is more complicated.
6 Takeaways on Inflation and the Economy Right NowAre we headed into a recession? Kellogg’s Sergio Rebelo breaks down the latest trends.
What Is the Purpose of a Corporation Today?Has anything changed in the three years since the Business Roundtable declared firms should prioritize more than shareholders?
How to Get the Ear of Your CEO—And What to Say When You Have ItEvery interaction with the top boss is an audition for senior leadership.
Why We Can’t All Get Away with Wearing Designer ClothesIn certain professions, luxury goods can send the wrong signal.
Why You Should Skip the Easy Wins and Tackle the Hard Task FirstNew research shows that you and your organization lose out when you procrastinate on the difficult stuff.
How Are Black–White Biracial People Perceived in Terms of Race?Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
Which Form of Government Is Best?Democracies may not outlast dictatorships, but they adapt better.
When Do Open Borders Make Economic Sense?A new study provides a window into the logic behind various immigration policies.
Why Do Some People Succeed after Failing, While Others Continue to Flounder?A new study dispels some of the mystery behind success after failure.
How Has Marketing Changed over the Past Half-Century?Phil Kotler’s groundbreaking textbook came out 55 years ago. Sixteen editions later, he and coauthor Alexander Chernev discuss how big data, social media, and purpose-driven branding are moving the field forward.
How Old Are Successful Tech Entrepreneurs?A definitive new study dispels the myth of the Silicon Valley wunderkind.
How Offering a Product for Free Can BackfireIt seems counterintuitive, but there are times customers would rather pay a small amount than get something for free.
Immigrants to the U.S. Create More Jobs than They TakeA new study finds that immigrants are far more likely to found companies—both large and small—than native-born Americans.
College Campuses Are Becoming More Diverse. But How Much Do Students from Different Backgrounds Actually Interact?Increasing diversity has been a key goal, “but far less attention is paid to what happens after we get people in the door.”
How Peer Pressure Can Lead Teens to Underachieve—Even in Schools Where It’s “Cool to Be Smart”New research offers lessons for administrators hoping to improve student performance.