Marketing Mar 7, 2016
How Important Is User-Generated Content?
An episode of censorship in China sheds light on the power of customers talking about brands on social media.
Months before the latest season of Game of Thrones, fans took to Twitter to share their excitement. Tweets like “I’m pretty sure Winter is coming in April!” and “Which house will rule the Iron Throne this spring?” demonstrate their giddiness about upcoming episodes. No doubt, HBO is thrilled with this activity.
Whether it is a new episode of a TV show or the latest iPhone, companies have long suspected that customer posts to microblogging sites like Twitter can help promote their products. The notion has led companies to offer perks such as early access, free samples, or upgrades for users they think are influential in online communities.
But this is mainly a hunch on the companies’ part.
“Intuitively you might expect that user-generated content has some effect, otherwise why would firms be doing it?” says Song Yao, assistant professor of marketing at the Kellogg School.
These user-created microblogs are very difficult to study. Although a firm can place its own tweets strategically, it is impossible to know—or more importantly, test—the impact of customer tweets. And since most products are promoted in multiple ways, attributing success to traditional ads versus user-generated content can be problematic.
“You can’t say the high ratings for Game of Thrones are a result of users’ tweets—it’s just an intrinsic preference that’s driving both the show’s rating and the Twitter content,” Yao says.
“I didn’t expect the effect to be this large. In that sense, our study confirms that this is a very impactful and effective marketing method.”
But then a real-world scandal created a clever natural experiment that made it possible to tease apart the impact of user-generated content on a product’s success.
Along with Stephan Seiler at Stanford University and Wenbo Wang at the Hong Kong University of Science and Technology, Yao looked at an unusual situation: What happens to a TV show when user-generated comments disappear because parts of a microblogging platform are temporarily shut down?
From Political Coup to Market Data
In February 2012, Wang Lijun, vice mayor of a city in southwest China, revealed details of a businessman’s murder and subsequent cover-up to the United States Consulate in China. The juicy scandal—filled with stories of defection, official corruption, and political fallout—fueled rumors that spread rapidly via Sina Weibo, a Chinese microblogging platform similar to Twitter. To stem gossip, the government placed a partial block on Sina Weibo. All user-generated comments—whether about the scandal or not—were shut down for three days.
So, for example, a television channel could post that a show would air at a particular time, and a user might retweet that, but users could not add that they would be watching or that they were excited about a particular episode.“TV shows were totally irrelevant to the scandal but were affected as well,” Yao says. “There was much less microblogging activity during those three days.”
This presented a perfect opportunity to study the impact of user-generated posts on TV ratings.
The researchers tracked content created by Sina Weibo users before, during, and after the block across 24 mainland cities and Hong Kong. At the same time, they observed the ratings of 166 TV shows and analyzed how the censorship affected viewership ratings.
The Weibo blackout had a significant effect.
Shows with a normally heavy Weibo presence took the biggest hit. Episodes of those shows had significantly lower ratings during the three-day block compared with their normal viewership levels. These same shows were largely unaffected before or after the block—only episodes that aired during the censored Sina Weibo window were impacted.
Cities with higher proportions of Weibo users also reported greater drops in episode ratings. Unlike the mainland cities, Hong Kong, which was not subject to the censorship and has more Twitter than Sina Weibo users, saw no major impacts from the block.
“What we found is that ratings only dropped in the cities with high Weibo penetration and only dropped for shows with high Weibo activity,” Yao says.
The researchers also wanted to control for another possible variable.
“What if people’s attention was actually diverted to the scandal, and they were no longer interested in watching TV at that point because the scandal was far more interesting?” Yao says.
To eliminate that explanation, the team used the Baidu search index, a metric similar to Google trends, to measure people’s interest in the political scandal. Search activity related to the scandal, for example for the term “Wang Lijun,” spiked at three key moments: when the news broke, when his superior, Bo Xilai, was removed from office, and when Bo was arrested and charged with bribery and corruption.
But the Sina Weibo block did not coincide with any of these events, so it was unlikely people were simply focused on the scandal instead of new episodes, says Yao. “The evidence is quite convincing.”
The researchers then quantified the impact of user-generated posts on ratings.Increasing the amount of user-generated content by 100 percent (such as going from none to restoring all of it, as happened during and after the censoring event) increased viewership ratings by 2 percent.
“This seems to be a fairly small number, but it’s actually a significant impact,” Yao says. “If there’s nothing else going on, the fluctuation in ratings from episode to episode is about 9 percent, so the ‘Weibo effect’ accounts for roughly 20 percent of the normal fluctuations.”
These findings are particularly exciting because it is usually very difficult to quantify the success of user-generated content. Most marketing firms normally look to classic A/B tests to try out different pitches. But that is impossible to do with user-generated tweets.
“Without the evidence from this natural experiment, I could never make this conclusion about user-generated content with such certainty,” Yao says. “I didn’t expect the effect to be this large. In that sense, our study confirms that this is a very impactful and effective marketing method.”
Leveraging User Tweets
The results extend well beyond a single episode of a TV show, even if it is Game of Thrones.
Like traditional ads, user-generated tweets seem to work in two ways: by providing information and by persuading others to share the poster’s excitement. Firms could leverage each dimension of tweets in different ways, Yao suggests. And because the team found that user-generated content before a show had a significant impact on ratings—more so, in fact, than tweets during the show—their findings can be applied to traditional consumer advertising.
For the launch of a new product, engaging users to share information, possibly by offering early access or trial versions to more influential consumers, can boost awareness.
“In the case of a stable product, let’s say Coke or Pepsi, the tweet itself might contain little information,” he says. “But firms can work on the other dimension: How can I persuade consumers in the community to consume this product?”
“We suspect tweeting will work more effectively in a category that’s fast moving. If there’s a new product, or a movie, or a daily deal like a Groupon, social media can help disseminate information quickly,” he adds.
Regardless of the product, having better monitors of user-generated content and sentiment can help marketing efforts, Yao says. “Firms should think about how to create an active community and how to encourage the creation of user-generated content.”
Seiler, Stephan, Song Yao, and Wenbo Wang. 2015. “The Impact of Earned Media on Demand: Evidence from a Natural Experiment.” Working paper.
One Key to a Happy Marriage? A Joint Bank Account.Merging finances helps newlyweds align their financial goals and avoid scorekeeping.
Take 5: Yikes! When Unintended Consequences StrikeGood intentions don’t always mean good results. Here’s why humility, and a lot of monitoring, are so important when making big changes.
How Are Black–White Biracial People Perceived in Terms of Race?Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
Will AI Eventually Replace Doctors?Maybe not entirely. But the doctor–patient relationship is likely to change dramatically.
Entrepreneurship Through Acquisition Is Still EntrepreneurshipETA is one of the fastest-growing paths to entrepreneurship. Here's how to think about it.
Take 5: Research-Backed Tips for Scheduling Your DayKellogg faculty offer ideas for working smarter and not harder.
How to Manage a Disengaged Employee—and Get Them Excited about Work AgainDon’t give up on checked-out team members. Try these strategies instead.
Which Form of Government Is Best?Democracies may not outlast dictatorships, but they adapt better.
What Went Wrong at AIG?Unpacking the insurance giant's collapse during the 2008 financial crisis.
The Appeal of Handmade in an Era of AutomationThis excerpt from the book “The Power of Human" explains why we continue to equate human effort with value.
2 Factors Will Determine How Much AI Transforms Our EconomyThey’ll also dictate how workers stand to fare.
When Do Open Borders Make Economic Sense?A new study provides a window into the logic behind various immigration policies.
Why Do Some People Succeed after Failing, While Others Continue to Flounder?A new study dispels some of the mystery behind success after failure.
Sitting Near a High-Performer Can Make You Better at Your Job“Spillover” from certain coworkers can boost our productivity—or jeopardize our employment.
How the Wormhole Decade (2000–2010) Changed the WorldFive implications no one can afford to ignore.
What’s at Stake in the Debt-Ceiling Standoff?Defaulting would be an unmitigated disaster, quickly felt by ordinary Americans.
What Happens to Worker Productivity after a Minimum Wage Increase?A pay raise boosts productivity for some—but the impact on the bottom line is more complicated.
Immigrants to the U.S. Create More Jobs than They TakeA new study finds that immigrants are far more likely to found companies—both large and small—than native-born Americans.
How Has Marketing Changed over the Past Half-Century?Phil Kotler’s groundbreaking textbook came out 55 years ago. Sixteen editions later, he and coauthor Alexander Chernev discuss how big data, social media, and purpose-driven branding are moving the field forward.
3 Traits of Successful Market-Creating EntrepreneursCreating a market isn’t for the faint of heart. But a dose of humility can go a long way.