Marketing Mar 7, 2016

How Important Is User-Generated Content?

An episode of censorship in China sheds light on the power of customers talking about brands on social media.

Yevgenia Nayberg

Based on the research of

Stephan Seiler

Song Yao

Wenbo Wang

Months before the latest season of Game of Thrones, fans took to Twitter to share their excitement. Tweets like “I’m pretty sure Winter is coming in April!” and “Which house will rule the Iron Throne this spring?” demonstrate their giddiness about upcoming episodes. No doubt, HBO is thrilled with this activity.

Whether it is a new episode of a TV show or the latest iPhone, companies have long suspected that customer posts to microblogging sites like Twitter can help promote their products. The notion has led companies to offer perks such as early access, free samples, or upgrades for users they think are influential in online communities.

But this is mainly a hunch on the companies’ part.

“Intuitively you might expect that user-generated content has some effect, otherwise why would firms be doing it?” says Song Yao, assistant professor of marketing at the Kellogg School.

These user-created microblogs are very difficult to study. Although a firm can place its own tweets strategically, it is impossible to know—or more importantly, test—the impact of customer tweets. And since most products are promoted in multiple ways, attributing success to traditional ads versus user-generated content can be problematic.

“You can’t say the high ratings for Game of Thrones are a result of users’ tweets—it’s just an intrinsic preference that’s driving both the show’s rating and the Twitter content,” Yao says.

“I didn’t expect the effect to be this large. In that sense, our study confirms that this is a very impactful and effective marketing method.”

Add Insight
to your inbox.

We’ll send you one email a week with content you actually want to read, curated by the Insight team.

Sign up here

But then a real-world scandal created a clever natural experiment that made it possible to tease apart the impact of user-generated content on a product’s success.

Along with Stephan Seiler at Stanford University and Wenbo Wang at the Hong Kong University of Science and Technology, Yao looked at an unusual situation: What happens to a TV show when user-generated comments disappear because parts of a microblogging platform are temporarily shut down?

From Political Coup to Market Data

In February 2012, Wang Lijun, vice mayor of a city in southwest China, revealed details of a businessman’s murder and subsequent cover-up to the United States Consulate in China. The juicy scandal—filled with stories of defection, official corruption, and political fallout—fueled rumors that spread rapidly via Sina Weibo, a Chinese microblogging platform similar to Twitter. To stem gossip, the government placed a partial block on Sina Weibo. All user-generated comments—whether about the scandal or not—were shut down for three days.

So, for example, a television channel could post that a show would air at a particular time, and a user might retweet that, but users could not add that they would be watching or that they were excited about a particular episode.“TV shows were totally irrelevant to the scandal but were affected as well,” Yao says. “There was much less microblogging activity during those three days.”

This presented a perfect opportunity to study the impact of user-generated posts on TV ratings.

The researchers tracked content created by Sina Weibo users before, during, and after the block across 24 mainland cities and Hong Kong. At the same time, they observed the ratings of 166 TV shows and analyzed how the censorship affected viewership ratings.

Microblogging’s Power

The Weibo blackout had a significant effect.

Shows with a normally heavy Weibo presence took the biggest hit. Episodes of those shows had significantly lower ratings during the three-day block compared with their normal viewership levels. These same shows were largely unaffected before or after the block—only episodes that aired during the censored Sina Weibo window were impacted.

Cities with higher proportions of Weibo users also reported greater drops in episode ratings. Unlike the mainland cities, Hong Kong, which was not subject to the censorship and has more Twitter than Sina Weibo users, saw no major impacts from the block.

“What we found is that ratings only dropped in the cities with high Weibo penetration and only dropped for shows with high Weibo activity,” Yao says.

The researchers also wanted to control for another possible variable.

“What if people’s attention was actually diverted to the scandal, and they were no longer interested in watching TV at that point because the scandal was far more interesting?” Yao says.

To eliminate that explanation, the team used the Baidu search index, a metric similar to Google trends, to measure people’s interest in the political scandal. Search activity related to the scandal, for example for the term “Wang Lijun,” spiked at three key moments: when the news broke, when his superior, Bo Xilai, was removed from office, and when Bo was arrested and charged with bribery and corruption.

But the Sina Weibo block did not coincide with any of these events, so it was unlikely people were simply focused on the scandal instead of new episodes, says Yao. “The evidence is quite convincing.”

Surprising Impact

The researchers then quantified the impact of user-generated posts on ratings.Increasing the amount of user-generated content by 100 percent (such as going from none to restoring all of it, as happened during and after the censoring event) increased viewership ratings by 2 percent.

“This seems to be a fairly small number, but it’s actually a significant impact,” Yao says. “If there’s nothing else going on, the fluctuation in ratings from episode to episode is about 9 percent, so the ‘Weibo effect’ accounts for roughly 20 percent of the normal fluctuations.”

These findings are particularly exciting because it is usually very difficult to quantify the success of user-generated content. Most marketing firms normally look to classic A/B tests to try out different pitches. But that is impossible to do with user-generated tweets.

“Without the evidence from this natural experiment, I could never make this conclusion about user-generated content with such certainty,” Yao says. “I didn’t expect the effect to be this large. In that sense, our study confirms that this is a very impactful and effective marketing method.”

Leveraging User Tweets

The results extend well beyond a single episode of a TV show, even if it is Game of Thrones.

Like traditional ads, user-generated tweets seem to work in two ways: by providing information and by persuading others to share the poster’s excitement. Firms could leverage each dimension of tweets in different ways, Yao suggests. And because the team found that user-generated content before a show had a significant impact on ratings—more so, in fact, than tweets during the show—their findings can be applied to traditional consumer advertising.

For the launch of a new product, engaging users to share information, possibly by offering early access or trial versions to more influential consumers, can boost awareness.

“In the case of a stable product, let’s say Coke or Pepsi, the tweet itself might contain little information,” he says. “But firms can work on the other dimension: How can I persuade consumers in the community to consume this product?”

“We suspect tweeting will work more effectively in a category that’s fast moving. If there’s a new product, or a movie, or a daily deal like a Groupon, social media can help disseminate information quickly,” he adds.

Regardless of the product, having better monitors of user-generated content and sentiment can help marketing efforts, Yao says. “Firms should think about how to create an active community and how to encourage the creation of user-generated content.”

Featured Faculty

Song Yao

Member of the Department of Marketing faculty until 2017

<p>&nbsp;</p>

About the Writer

Jyoti Madhusoodanan is a Bay Area-based science writer.

About the Research

Seiler, Stephan, Song Yao, and Wenbo Wang. 2015. “The Impact of Earned Media on Demand: Evidence from a Natural Experiment.” Working paper.

Read the original

Trending

Editor’s Picks

Careers

Podcast: How to Be a Great Mentor

Plus, some valuable career advice that applies to just about everyone.

Marketing

A New Way to Persuade Kids to Drink More Water and Less Soda

Getting children to make healthy choices is tricky—and the wrong message can backfire.

Innovation

How Can Social Science Become More Solutions-Oriented?

A conversation between researchers at Kellogg and Microsoft explores how behavioral science can best be applied.

Innovation

Buying a Company for Its Talent? Beware of Hidden Legal Risks.

Acquiring another firm’s trade secrets—even unintentionally—could prove costly.

Careers

Take 5: Tips for Widening—and Improving—Your Candidate Pool

Common biases can cause companies to overlook a wealth of top talent.

Innovation

Everyone Wants Pharmaceutical Breakthroughs. What Drives Drug Companies to Pursue Them?

A new study suggests that firms are at their most innovative after a financial windfall.

Careers

4 Key Steps to Preparing for a Business Presentation

Don’t let a lack of prep work sabotage your great ideas.

Healthcare

Video: How Open Lines of Communication Can Improve Healthcare Outcomes

Training physicians to be better communicators builds trust with patients and their loved ones.

Operations

Here’s a Better Way to Schedule Surgeries

A new tool could drive savings of 20 percent while still keeping surgeons happy.

Politics & Elections

Why Economic Crises Trigger Political Turnover in Some Countries but Not Others

The fallout can hinge on how much a country’s people trust each other.

Marketing

Building Strong Brands: The Inside Scoop on Branding in the Real World

Tim Calkins’s blog draws lessons from brand missteps and triumphs.

Economics

How the Coffee Industry Is Building a Sustainable Supply Chain in an Unstable Region

Three experts discuss the challenges and rewards of sourcing coffee from the Democratic Republic of Congo.