Marketing Mar 7, 2016
How Important Is User-Generated Content?
An episode of censorship in China sheds light on the power of customers talking about brands on social media.
Months before the latest season of Game of Thrones, fans took to Twitter to share their excitement. Tweets like “I’m pretty sure Winter is coming in April!” and “Which house will rule the Iron Throne this spring?” demonstrate their giddiness about upcoming episodes. No doubt, HBO is thrilled with this activity.
Whether it is a new episode of a TV show or the latest iPhone, companies have long suspected that customer posts to microblogging sites like Twitter can help promote their products. The notion has led companies to offer perks such as early access, free samples, or upgrades for users they think are influential in online communities.
But this is mainly a hunch on the companies’ part.
“Intuitively you might expect that user-generated content has some effect, otherwise why would firms be doing it?” says Song Yao, assistant professor of marketing at the Kellogg School.
These user-created microblogs are very difficult to study. Although a firm can place its own tweets strategically, it is impossible to know—or more importantly, test—the impact of customer tweets. And since most products are promoted in multiple ways, attributing success to traditional ads versus user-generated content can be problematic.
“You can’t say the high ratings for Game of Thrones are a result of users’ tweets—it’s just an intrinsic preference that’s driving both the show’s rating and the Twitter content,” Yao says.
“I didn’t expect the effect to be this large. In that sense, our study confirms that this is a very impactful and effective marketing method.”
to your inbox.
But then a real-world scandal created a clever natural experiment that made it possible to tease apart the impact of user-generated content on a product’s success.
Along with Stephan Seiler at Stanford University and Wenbo Wang at the Hong Kong University of Science and Technology, Yao looked at an unusual situation: What happens to a TV show when user-generated comments disappear because parts of a microblogging platform are temporarily shut down?
From Political Coup to Market Data
In February 2012, Wang Lijun, vice mayor of a city in southwest China, revealed details of a businessman’s murder and subsequent cover-up to the United States Consulate in China. The juicy scandal—filled with stories of defection, official corruption, and political fallout—fueled rumors that spread rapidly via Sina Weibo, a Chinese microblogging platform similar to Twitter. To stem gossip, the government placed a partial block on Sina Weibo. All user-generated comments—whether about the scandal or not—were shut down for three days.
So, for example, a television channel could post that a show would air at a particular time, and a user might retweet that, but users could not add that they would be watching or that they were excited about a particular episode.“TV shows were totally irrelevant to the scandal but were affected as well,” Yao says. “There was much less microblogging activity during those three days.”
This presented a perfect opportunity to study the impact of user-generated posts on TV ratings.
The researchers tracked content created by Sina Weibo users before, during, and after the block across 24 mainland cities and Hong Kong. At the same time, they observed the ratings of 166 TV shows and analyzed how the censorship affected viewership ratings.
The Weibo blackout had a significant effect.
Shows with a normally heavy Weibo presence took the biggest hit. Episodes of those shows had significantly lower ratings during the three-day block compared with their normal viewership levels. These same shows were largely unaffected before or after the block—only episodes that aired during the censored Sina Weibo window were impacted.
Cities with higher proportions of Weibo users also reported greater drops in episode ratings. Unlike the mainland cities, Hong Kong, which was not subject to the censorship and has more Twitter than Sina Weibo users, saw no major impacts from the block.
“What we found is that ratings only dropped in the cities with high Weibo penetration and only dropped for shows with high Weibo activity,” Yao says.
The researchers also wanted to control for another possible variable.
“What if people’s attention was actually diverted to the scandal, and they were no longer interested in watching TV at that point because the scandal was far more interesting?” Yao says.
To eliminate that explanation, the team used the Baidu search index, a metric similar to Google trends, to measure people’s interest in the political scandal. Search activity related to the scandal, for example for the term “Wang Lijun,” spiked at three key moments: when the news broke, when his superior, Bo Xilai, was removed from office, and when Bo was arrested and charged with bribery and corruption.
But the Sina Weibo block did not coincide with any of these events, so it was unlikely people were simply focused on the scandal instead of new episodes, says Yao. “The evidence is quite convincing.”
The researchers then quantified the impact of user-generated posts on ratings.Increasing the amount of user-generated content by 100 percent (such as going from none to restoring all of it, as happened during and after the censoring event) increased viewership ratings by 2 percent.
“This seems to be a fairly small number, but it’s actually a significant impact,” Yao says. “If there’s nothing else going on, the fluctuation in ratings from episode to episode is about 9 percent, so the ‘Weibo effect’ accounts for roughly 20 percent of the normal fluctuations.”
These findings are particularly exciting because it is usually very difficult to quantify the success of user-generated content. Most marketing firms normally look to classic A/B tests to try out different pitches. But that is impossible to do with user-generated tweets.
“Without the evidence from this natural experiment, I could never make this conclusion about user-generated content with such certainty,” Yao says. “I didn’t expect the effect to be this large. In that sense, our study confirms that this is a very impactful and effective marketing method.”
Leveraging User Tweets
The results extend well beyond a single episode of a TV show, even if it is Game of Thrones.
Like traditional ads, user-generated tweets seem to work in two ways: by providing information and by persuading others to share the poster’s excitement. Firms could leverage each dimension of tweets in different ways, Yao suggests. And because the team found that user-generated content before a show had a significant impact on ratings—more so, in fact, than tweets during the show—their findings can be applied to traditional consumer advertising.
For the launch of a new product, engaging users to share information, possibly by offering early access or trial versions to more influential consumers, can boost awareness.
“In the case of a stable product, let’s say Coke or Pepsi, the tweet itself might contain little information,” he says. “But firms can work on the other dimension: How can I persuade consumers in the community to consume this product?”
“We suspect tweeting will work more effectively in a category that’s fast moving. If there’s a new product, or a movie, or a daily deal like a Groupon, social media can help disseminate information quickly,” he adds.
Regardless of the product, having better monitors of user-generated content and sentiment can help marketing efforts, Yao says. “Firms should think about how to create an active community and how to encourage the creation of user-generated content.”
Seiler, Stephan, Song Yao, and Wenbo Wang. 2015. “The Impact of Earned Media on Demand: Evidence from a Natural Experiment.” Working paper.
How Much Do Boycotts Affect a Company’s Bottom Line?There’s often an opposing camp pushing for a “buycott” to support the company. New research shows which group has more sway.
5 Takeaways on the State of ESG InvestingESG investing is hot. But what does it actually deliver for society and for shareholders?
Could Bringing Your "Whole Self" to Work Curb Unethical Behavior?Organizations would be wise to help employees avoid compartmentalizing their personal and professional identities.
When Do Open Borders Make Economic Sense?A new study provides a window into the logic behind various immigration policies.
Which Form of Government Is Best?Democracies may not outlast dictatorships, but they adapt better.
How Has Marketing Changed over the Past Half-Century?Phil Kotler’s groundbreaking textbook came out 55 years ago. Sixteen editions later, he and coauthor Alexander Chernev discuss how big data, social media, and purpose-driven branding are moving the field forward.
What Happens to Worker Productivity after a Minimum Wage Increase?A pay raise boosts productivity for some—but the impact on the bottom line is more complicated.
Why Do Some People Succeed after Failing, While Others Continue to Flounder?A new study dispels some of the mystery behind success after failure.
What Went Wrong at AIG?Unpacking the insurance giant's collapse during the 2008 financial crisis.
Why Well-Meaning NGOs Sometimes Do More Harm than GoodStudies of aid groups in Ghana and Uganda show why it’s so important to coordinate with local governments and institutions.
3 Tips for Reinventing Your Career After a LayoffIt’s crucial to reassess what you want to be doing instead of jumping at the first opportunity.
How Are Black–White Biracial People Perceived in Terms of Race?Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
Podcast: Does Your Life Reflect What You Value?On this episode of The Insightful Leader, a former CEO explains how to organize your life around what really matters—instead of trying to do it all.
Immigrants to the U.S. Create More Jobs than They TakeA new study finds that immigrants are far more likely to found companies—both large and small—than native-born Americans.
In a World of Widespread Video Sharing, What’s Real and What’s Not?A discussion with a video-authentication expert on what it takes to unearth “deepfakes.”
College Campuses Are Becoming More Diverse. But How Much Do Students from Different Backgrounds Actually Interact?Increasing diversity has been a key goal, “but far less attention is paid to what happens after we get people in the door.”