Marketing Mar 7, 2016

How Impor­tant Is User-Gen­er­at­ed Content?

An episode of cen­sor­ship in Chi­na sheds light on the pow­er of cus­tomers talk­ing about brands on social media.

Yevgenia Nayberg

Based on the research of

Stephan Seiler

Song Yao

Wenbo Wang

Months before the lat­est sea­son of Game of Thrones, fans took to Twit­ter to share their excite­ment. Tweets like I’m pret­ty sure Win­ter is com­ing in April!” and Which house will rule the Iron Throne this spring?” demon­strate their gid­di­ness about upcom­ing episodes. No doubt, HBO is thrilled with this activity.

Whether it is a new episode of a TV show or the lat­est iPhone, com­pa­nies have long sus­pect­ed that cus­tomer posts to microblog­ging sites like Twit­ter can help pro­mote their prod­ucts. The notion has led com­pa­nies to offer perks such as ear­ly access, free sam­ples, or upgrades for users they think are influ­en­tial in online communities.

But this is main­ly a hunch on the com­pa­nies’ part.

Intu­itive­ly you might expect that user-gen­er­at­ed con­tent has some effect, oth­er­wise why would firms be doing it?” says Song Yao, assis­tant pro­fes­sor of mar­ket­ing at the Kel­logg School.

These user-cre­at­ed microblogs are very dif­fi­cult to study. Although a firm can place its own tweets strate­gi­cal­ly, it is impos­si­ble to know — or more impor­tant­ly, test — the impact of cus­tomer tweets. And since most prod­ucts are pro­mot­ed in mul­ti­ple ways, attribut­ing suc­cess to tra­di­tion­al ads ver­sus user-gen­er­at­ed con­tent can be problematic.

You can’t say the high rat­ings for Game of Thrones are a result of users’ tweets — it’s just an intrin­sic pref­er­ence that’s dri­ving both the show’s rat­ing and the Twit­ter con­tent,” Yao says.

I didn’t expect the effect to be this large. In that sense, our study con­firms that this is a very impact­ful and effec­tive mar­ket­ing method.”

Add Insight
to your inbox.

We’ll send you one email a week with content you actually want to read, curated by the Insight team.

Sign up here

But then a real-world scan­dal cre­at­ed a clever nat­ur­al exper­i­ment that made it pos­si­ble to tease apart the impact of user-gen­er­at­ed con­tent on a product’s success.

Along with Stephan Seil­er at Stan­ford Uni­ver­si­ty and Wen­bo Wang at the Hong Kong Uni­ver­si­ty of Sci­ence and Tech­nol­o­gy, Yao looked at an unusu­al sit­u­a­tion: What hap­pens to a TV show when user-gen­er­at­ed com­ments dis­ap­pear because parts of a microblog­ging plat­form are tem­porar­i­ly shut down?

From Polit­i­cal Coup to Mar­ket Data

In Feb­ru­ary 2012, Wang Lijun, vice may­or of a city in south­west Chi­na, revealed details of a businessman’s mur­der and sub­se­quent cov­er-up to the Unit­ed States Con­sulate in Chi­na. The juicy scan­dal — filled with sto­ries of defec­tion, offi­cial cor­rup­tion, and polit­i­cal fall­out — fueled rumors that spread rapid­ly via Sina Wei­bo, a Chi­nese microblog­ging plat­form sim­i­lar to Twit­ter. To stem gos­sip, the gov­ern­ment placed a par­tial block on Sina Wei­bo. All user-gen­er­at­ed com­ments — whether about the scan­dal or not — were shut down for three days.

So, for exam­ple, a tele­vi­sion chan­nel could post that a show would air at a par­tic­u­lar time, and a user might retweet that, but users could not add that they would be watch­ing or that they were excit­ed about a par­tic­u­lar episode.“TV shows were total­ly irrel­e­vant to the scan­dal but were affect­ed as well,” Yao says. There was much less microblog­ging activ­i­ty dur­ing those three days.”

This pre­sent­ed a per­fect oppor­tu­ni­ty to study the impact of user-gen­er­at­ed posts on TV ratings.

The researchers tracked con­tent cre­at­ed by Sina Wei­bo users before, dur­ing, and after the block across 24 main­land cities and Hong Kong. At the same time, they observed the rat­ings of 166 TV shows and ana­lyzed how the cen­sor­ship affect­ed view­er­ship ratings.

Microblogging’s Pow­er

The Wei­bo black­out had a sig­nif­i­cant effect.

Shows with a nor­mal­ly heavy Wei­bo pres­ence took the biggest hit. Episodes of those shows had sig­nif­i­cant­ly low­er rat­ings dur­ing the three-day block com­pared with their nor­mal view­er­ship lev­els. These same shows were large­ly unaf­fect­ed before or after the block — only episodes that aired dur­ing the cen­sored Sina Wei­bo win­dow were impacted.

Cities with high­er pro­por­tions of Wei­bo users also report­ed greater drops in episode rat­ings. Unlike the main­land cities, Hong Kong, which was not sub­ject to the cen­sor­ship and has more Twit­ter than Sina Wei­bo users, saw no major impacts from the block.

What we found is that rat­ings only dropped in the cities with high Wei­bo pen­e­tra­tion and only dropped for shows with high Wei­bo activ­i­ty,” Yao says.

The researchers also want­ed to con­trol for anoth­er pos­si­ble variable.

What if people’s atten­tion was actu­al­ly divert­ed to the scan­dal, and they were no longer inter­est­ed in watch­ing TV at that point because the scan­dal was far more inter­est­ing?” Yao says.

To elim­i­nate that expla­na­tion, the team used the Baidu search index, a met­ric sim­i­lar to Google trends, to mea­sure people’s inter­est in the polit­i­cal scan­dal. Search activ­i­ty relat­ed to the scan­dal, for exam­ple for the term Wang Lijun,” spiked at three key moments: when the news broke, when his supe­ri­or, Bo Xilai, was removed from office, and when Bo was arrest­ed and charged with bribery and corruption.

But the Sina Wei­bo block did not coin­cide with any of these events, so it was unlike­ly peo­ple were sim­ply focused on the scan­dal instead of new episodes, says Yao. The evi­dence is quite convincing.”

Sur­pris­ing Impact

The researchers then quan­ti­fied the impact of user-gen­er­at­ed posts on ratings.Increasing the amount of user-gen­er­at­ed con­tent by 100 per­cent (such as going from none to restor­ing all of it, as hap­pened dur­ing and after the cen­sor­ing event) increased view­er­ship rat­ings by 2 percent.

This seems to be a fair­ly small num­ber, but it’s actu­al­ly a sig­nif­i­cant impact,” Yao says. If there’s noth­ing else going on, the fluc­tu­a­tion in rat­ings from episode to episode is about 9 per­cent, so the Wei­bo effect’ accounts for rough­ly 20 per­cent of the nor­mal fluctuations.”

These find­ings are par­tic­u­lar­ly excit­ing because it is usu­al­ly very dif­fi­cult to quan­ti­fy the suc­cess of user-gen­er­at­ed con­tent. Most mar­ket­ing firms nor­mal­ly look to clas­sic A/B tests to try out dif­fer­ent pitch­es. But that is impos­si­ble to do with user-gen­er­at­ed tweets.

With­out the evi­dence from this nat­ur­al exper­i­ment, I could nev­er make this con­clu­sion about user-gen­er­at­ed con­tent with such cer­tain­ty,” Yao says. I didn’t expect the effect to be this large. In that sense, our study con­firms that this is a very impact­ful and effec­tive mar­ket­ing method.”

Lever­ag­ing User Tweets

The results extend well beyond a sin­gle episode of a TV show, even if it is Game of Thrones.

Like tra­di­tion­al ads, user-gen­er­at­ed tweets seem to work in two ways: by pro­vid­ing infor­ma­tion and by per­suad­ing oth­ers to share the poster’s excite­ment. Firms could lever­age each dimen­sion of tweets in dif­fer­ent ways, Yao sug­gests. And because the team found that user-gen­er­at­ed con­tent before a show had a sig­nif­i­cant impact on rat­ings — more so, in fact, than tweets dur­ing the show — their find­ings can be applied to tra­di­tion­al con­sumer advertising.

For the launch of a new prod­uct, engag­ing users to share infor­ma­tion, pos­si­bly by offer­ing ear­ly access or tri­al ver­sions to more influ­en­tial con­sumers, can boost awareness.

In the case of a sta­ble prod­uct, let’s say Coke or Pep­si, the tweet itself might con­tain lit­tle infor­ma­tion,” he says. But firms can work on the oth­er dimen­sion: How can I per­suade con­sumers in the com­mu­ni­ty to con­sume this product?”

We sus­pect tweet­ing will work more effec­tive­ly in a cat­e­go­ry that’s fast mov­ing. If there’s a new prod­uct, or a movie, or a dai­ly deal like a Groupon, social media can help dis­sem­i­nate infor­ma­tion quick­ly,” he adds.

Regard­less of the prod­uct, hav­ing bet­ter mon­i­tors of user-gen­er­at­ed con­tent and sen­ti­ment can help mar­ket­ing efforts, Yao says. Firms should think about how to cre­ate an active com­mu­ni­ty and how to encour­age the cre­ation of user-gen­er­at­ed content.”

Featured Faculty

Song Yao

Member of the Department of Marketing faculty until 2017


About the Writer

Jyoti Madhusoodanan is a Bay Area-based science writer.

About the Research

Seiler, Stephan, Song Yao, and Wenbo Wang. 2015. “The Impact of Earned Media on Demand: Evidence from a Natural Experiment.” Working paper.

Read the original

Editor’s Picks


Pod­cast: How to Be a Great Mentor

Plus, some valu­able career advice that applies to just about everyone.


A New Way to Per­suade Kids to Drink More Water and Less Soda

Get­ting chil­dren to make healthy choic­es is tricky — and the wrong mes­sage can backfire.


How Can Social Sci­ence Become More Solutions-Oriented?

A con­ver­sa­tion between researchers at Kel­logg and Microsoft explores how behav­ioral sci­ence can best be applied.


Buy­ing a Com­pa­ny for Its Tal­ent? Beware of Hid­den Legal Risks.

Acquir­ing anoth­er firm’s trade secrets — even unin­ten­tion­al­ly — could prove costly.


Take 5: Tips for Widen­ing — and Improv­ing — Your Can­di­date Pool

Com­mon bias­es can cause com­pa­nies to over­look a wealth of top talent.


Every­one Wants Phar­ma­ceu­ti­cal Break­throughs. What Dri­ves Drug Com­pa­nies to Pur­sue Them?

A new study sug­gests that firms are at their most inno­v­a­tive after a finan­cial windfall.


4 Key Steps to Prepar­ing for a Busi­ness Presentation

Don’t let a lack of prep work sab­o­tage your great ideas.


Video: How Open Lines of Com­mu­ni­ca­tion Can Improve Health­care Outcomes

Train­ing physi­cians to be bet­ter com­mu­ni­ca­tors builds trust with patients and their loved ones.


Here’s a Bet­ter Way to Sched­ule Surgeries

A new tool could dri­ve sav­ings of 20 per­cent while still keep­ing sur­geons happy.

Politics & Elections

Why Eco­nom­ic Crises Trig­ger Polit­i­cal Turnover in Some Coun­tries but Not Others

The fall­out can hinge on how much a country’s peo­ple trust each other.


Build­ing Strong Brands: The Inside Scoop on Brand­ing in the Real World

Tim Calkins’s blog draws lessons from brand mis­steps and triumphs.


How the Cof­fee Indus­try Is Build­ing a Sus­tain­able Sup­ply Chain in an Unsta­ble Region

Three experts dis­cuss the chal­lenges and rewards of sourc­ing cof­fee from the Demo­c­ra­t­ic Repub­lic of Congo.