A Gentle Nudge Can Increase Participation in MOOCs

While MOOCs have indeed opened up cours­es taught by fac­ul­ty from renowd­ed insti­tu­tions to stu­dents all over the world, a per­sis­tent con­cern is low com­ple­tion rates: only about five per­cent of peo­ple who reg­is­ter for the cours­es actu­al­ly fin­ish them. Most peo­ple, it seems, have a hard time engag­ing with course mate­r­i­al in the iso­la­tion of their own homes — par­tic­uar­ly when they have lit­tle finan­cial skin in the game.

“Peo­ple have start­ed to real­ize that MOOCs may not do what some of the ear­ly vision­ar­ies thought,” says Jan Van Mieghem, pro­fes­sor of man­age­r­i­al eco­nom­ics and deci­sion sci­ences at the Kel­logg School. ​“A key con­cern of online edu­ca­tion is obvi­ous­ly the lack of col­lab­o­ra­tion among students.”

He won­dered whether a col­lab­o­ra­tive envi­ron­ment could be cul­ti­vat­ed in a MOOC. And, if so, if it would help stu­dents do bet­ter in the course.

So Van Mieghem and fel­low Kel­logg School researchers — Gad Allon, also a pro­fes­sor of man­age­r­i­al eco­nom­ics and deci­sion sci­ences, and Den­nis J. Zhang, a doc­tor­al stu­dent — decid­ed to test out a few tech­niques in their own MOOC. Specif­i­cal­ly, they decid­ed to see what the impact would be of encour­ag­ing inter­ac­tions via the class dis­cus­sion board and one-on-one dig­i­tal dis­cus­sions with fel­low students.

They found that these small prods to inter­act can have a mod­est impact on stu­dents’ engage­ment and performance.

Mea­sur­ing Impact

The researchers’ MOOC — a five-week-long course on scal­ing oper­a­tions that con­sist­ed of four week­ly lec­tures and a week­ly quiz — attract­ed more than 24,000 reg­is­trants, some of whom paid to take the class in return for a cer­tifi­cate of accom­plish­ment and oth­ers of whom took it for free. About 4,200 stu­dents sub­mit­ted at least one of the quizzes.

In their first exper­i­ment, the researchers sent a sur­vey at the begin­ning of the sec­ond week to all course par­tic­i­pants, ask­ing for feed­back on the first week’s mate­r­i­al. Those who returned the sur­vey were then divid­ed into two groups, one of which recieved an email remind­ing them to con­tribute to the course’s dis­cus­sion board.

The encour­age­ment worked. The sim­ple nudge increased vis­its to the board by 26.5 per­cent and increased posts to the board by near­ly 97 percent.

And vis­its to the board were linked to bet­ter per­for­mance. The researchers found that each addi­tion­al vis­it to the board in the first week increased the like­li­hood that a stu­dent would com­plete the fol­low­ing week’s quiz by about 3.5 per­cent. And, on aver­age, each stu­dent who received the nudge vis­it­ed the board four addi­tion­al times a week, mean­ing that over­all, stu­dents who received the encour­age­ment to vis­it the board were, on aver­age, 13 per­cent more like­ly to com­plete the fol­low­ing week’s quiz.

The vis­it did not improve quiz scores, how­ev­er, and after that first week, vis­its to the dis­cus­sion board had rapid­ly dimin­ish­ing influ­ence on quiz-com­ple­tion rates.

Still, Van Mieghem points out that the cost — a sin­gle email — was very small. ​“We did a very minor kind of stim­u­lus,” he says. ​“We just said to stu­dents, ​‘Hey, don’t for­get to go to the dis­cus­sion group.” That there were effects at all, he says, is a ​“hope­ful sign.”

In their sec­ond exper­i­ment, the researchers invit­ed some par­tic­i­pants to take part in online one-on-one dis­cus­sions about the course mate­r­i­al. They found that stu­dents who took them up on their offer were 10 per­cent more like­ly to com­plete their week­ly quiz — and their quiz scores increased by 2 to 10 per­cent in sub­se­quent weeks.

The upshot is that the direct com­mu­ni­ca­tion fos­tered by one-on-one dis­cus­sions seems to be more con­ducive to actu­al­ly learn­ing the mate­r­i­al. Still, only 7 per­cent of stu­dents who received an invi­ta­tion to take part did so. One bar­ri­er is the time and effort it takes to set up — and show up for — a dig­i­tal dis­cus­sion. Anoth­er is the anonymi­ty inher­ent in online edu­ca­tion: ​“If peo­ple don’t show up, there’s no way to penal­ize them,” Zhang says. ​“And they won’t even feel bad, because there’s no sham­ing fac­tor. If they don’t show up, no one knows who they are.”

Take Aways

The researchers note that their results may not be rel­e­vant to all online edu­ca­tors. For one, nei­ther manip­u­la­tion had a mea­sur­able effect on one group of stu­dents — those who actu­al­ly paid to take the course. Pay­ing stu­dents, it seems, were high­ly moti­vat­ed to take part in the dis­cus­sion board and one-on-one chats, even with­out the researchers’ nudges to interact.

But one imme­di­ate take­away is that, among non­pay­ing stu­dents, ear­ly social engage­ment packed the most punch.

“You real­ly have to think hard about the incen­tives that will help [stu­dents] at the begin­ning of the class,” Zhang says, ​“which is not the typ­i­cal approach with dis­cus­sion boards.” Much cur­rent think­ing, he says, is focused on ways to high­light and rec­om­mend posts that would be rel­e­vant to stu­dents. But in the ear­ly weeks of the course, there are too few posts for that strat­e­gy to be helpful.

It is a prin­ci­ple that also holds true for online com­merce, accord­ing to Zhang. For busi­ness­es that use online sur­veys to request cus­tomer feed­back, for exam­ple, soon­er is bet­ter than lat­er. ​“If you’re try­ing to facil­i­tate inter­ac­tions with cus­tomers, you have to engage them ear­ly, or they will lose interest.”

Because the MOOC will be offered on an ongo­ing basis, the researchers will con­tin­ue to exper­i­ment with ways to encour­age social inter­ac­tion — and hope­ful­ly, course com­ple­tion. They are opti­mistic that the sheer scale of online edu­ca­tion will make it pos­si­ble for them to refine their techniques.

Many edu­ca­tors run­ning a field exper­i­ment ​“would be impressed if they had 30 stu­dents,” says Van Mieghem. ​“But we’re talk­ing about 10,000 or more stu­dents. So the scale, and the amount of data, and the kind of exper­i­men­ta­tion we can do — it’s just very impres­sive, and some­thing that can­not be done in a typ­i­cal, phys­i­cal channel.”

He expects that, even if online cours­es are not the rev­o­lu­tion­ary force some peo­ple imag­ined they would be, they will con­tin­ue to evolve.

“The rea­son I got involved is that we have no idea how this is going to work out,” he says. ​“And my feel­ing was, instead of let­ting it come to me, I’d rather be there ear­ly and be able to influ­ence it. There’s no doubt that, one way or anoth­er, some of this tech­nol­o­gy will stay.”