Product Q&A Forums Hold a Lot of Promise. Here’s How to Make Them Work.

And yet other online communities, like Stack Overflow, have thrived.

The demise—or success—of sites like these are of interest to Achal Bassamboo, Kellogg professor of operations. He seeks to understand how these sorts of online communities work, in particular, because they’re a growing trend in customer service.

“A big thing happening in customer support is that people have moved to self-help because they don’t want to wait at call centers,” Bassamboo says. “And FAQs are available, but the next level is making it more dynamic, so people can ask questions and others can help them out.”

A wide range of businesses launch and nurture online communities. Some of these communities, like Quora, host a broad spectrum of questions and answers; others, like Stack Overflow, an online community for technology developers, target a specific industry. Still others, like the communities run by Apple and Microsoft, are geared toward the products and services sold by a particular company.

These online communities can be a boon for businesses: they can unlock user knowledge, reducing customer-service labor costs and wait times, all while providing better answers to customer queries. But only if they stick around. Like Yahoo Answers, many of these communities have languished or disappeared completely, including Walmart Moms and Generation Benz.

Bassamboo and collaborators Neha Sharma, a Kellogg doctoral candidate in operations, and Gad Allon from the University of Pennsylvania wanted to understand the factors driving online-community success and failure, and what such forums can do to perform better. They define success in this context as growing the number of users and, critically, maximizing the proportion of users’ questions that get answered. After all, if a community attracts a barrage of questions, but only a tiny fraction receives answers, the site is unlikely to flourish.

Online communities “need to balance their growing user base and make sure enough people get answers to sustain that base over time,” Sharma says.

Sharma, Allon, and Bassamboo conducted a study—named a finalist in the IBM Best Service Science Student Paper competition—that modeled online-community dynamics to understand how changing the cost of asking a question would impact user participation and platform sustainability.

They found that moderately increasing the cost of asking—such as through penalties for vague questions—improved the proportion of questions answered, making the community more sustainable. “There’s a sweet spot between too low a cost and too high of one for asking questions that is best for these communities,” Bassamboo says.

Understanding Online Communities

To shed light on online-community dynamics, the researchers built a dynamic model of such systems using concepts from game theory.

Bassamboo points out key features of the model: “There are two kinds of users coming through the door: those coming to ask questions and those coming to answer; and the people providing answers can also ask questions.”

Across users, the key factor is whether participating in the community is worth their while—the payoff could be from having a question answered or from enjoying a good reputation as someone who provides useful answers. “The idea is that people aren’t just providing answers out of the goodness of their heart,” Bassamboo says. “There has to be something in it for them.”

As such, changing the platform’s rules of engagement can have a large impact on the functioning of the community and incentives for participation. Specifically, the researchers manipulated how easy it was for hypothetical users to ask a question—or “the personal cost to asking a question,” as Bassamboo puts it.

Sharma describes the different types of costs questions can incur: In Stack Overflow, for instance, “you get a violation if your question seems like a homework question or overly big or complicated. In some systems you get penalized for asking a question that was already answered. That helps users decide whether they want to ask a given question or not.” Similarly, those who answer questions consider the chance that their answer will be considered the best one, typically determined by the asker, yielding reputation points for the answerer.

Guiding the research was a key question: How would raising the cost of asking a question affect the health of the online community?

The Benefit of Question Costs

The researchers ended up with a somewhat unintuitive answer.

“The intuitive guess would be that there would be a threshold cost of asking a question above which the platform would collapse and below which it would survive,” Bassamboo says. “So, if the cost of asking a question is very low, you should have lots of users on the platform, but as it goes up, the number of users goes down and the platform would die.”

In contrast to that linear relationship, the research findings suggest a sweet spot where a moderate cost of asking questions is optimal for the online community’s functioning. “When you increase the cost of asking questions,” Sharma says, “fewer people ask, but it increases the probability a question gets an answer. When people who ask get answers, they are more satisfied and stay on the platform, and the user base becomes sustainable over time.”

Looking at it another way, if the cost of asking questions is very small, a large number of people ask questions, and that results in lots of competition for the attention of users who may be able to provide answers. “All the questions are fighting for the attention from the same finite number of people in that community,” Sharma says. “Then the probability of getting an answer to a specific question goes down, and people drop out and stop asking questions.”

Indeed, the researchers think of the quality of the online community as determined largely by the proportion of questions that are answered. Bassamboo illustrates that idea with a personal example: “If I’m doing homework with my kids and we Google a question, we get very excited when we find a place where someone asked that same question. But it’s disappointing when you see there’s no response to the question. It feels like the end of the road.”

The implication is that imposing a moderate cost on asking questions and thus raising the share of questions answered can improve the overall quality, effectiveness, and sustainability of online communities.

Getting More Questions Answered

The findings point to practical advice for those seeking to develop robust online communities.

“You want a community where there are not only good questions asked but answers to those questions,” Bassamboo says. “So you want to raise the bar of asking a question by penalizing weak or vague questions, resulting in more pinpointed, specific questions. That’s where it’s worth expending some effort and energy.”

For example, a community could go as far as blocking the IP addresses of those whose questions are repeatedly considered weak. Or a community might seek to improve the proportion of questions answered by directing questions to certain participants who are more likely to provide a good answer. “If the system knows you might have the answer, it should be structured to match questions with you,” Bassamboo says.

Sharma notes another, even more effective way high-profile communities are boosting answer rates: “On Apple’s platform, the cost of asking questions isn’t very high. So they’ve spent some money placing their own representatives on the platform to answer harder questions. That way it avoids the situation where people with the hard questions never get an answer because it was never viewed by someone knowledgeable enough to answer it.”