Lisa Röper
As data analytics becomes a more pervasive business tool, many leaders are being sold on the idea that all you need to diagnose any perplexing problem is more data. While there’s no doubt that quantitative analysis can play a powerful role in telling you what happens, even the most robust, granular data won’t tell you why something happens.
Instead, employing a combination of qualitative and quantitative methods to identify both the what and the why, according to two Kellogg School professors, is what makes an analytics strategy a useful tool for change.
“Each has something powerful to offer,” says Joel Shapiro, a clinical associate professor of data analytics at Kellogg. “Quantitative analysis helps you identify broader trends, while qualitative analysis digs into human motivation, but the insights are hard to scale.”
David Schonthal, a clinical associate professor of innovation and entrepreneurship at Kellogg, says the real value is in how these two approaches complement each other. “When you combine data analytics with a deeper understanding of a customer’s motivation and experience—that’s how you will create better products and services.”
So how exactly is this done? How should companies avoid the misconception that more data provide all the answers and instead combine “qual” and “quant” to find better solutions to important business problems?
Determine Where to Focus
When searching for new approaches to a long-standing challenge, collecting and analyzing data such as sales figures or conversions can lead to surprisingly fruitful insights.
“This is one place where quant can really help—just knowing where to focus design efforts is extraordinarily valuable,” says Schonthal. “Data can act as a source of inspiration, not just a source of validation.”
Say, for example, that a university has a retention problem with its nontraditional student population. A quantitative analysis can identify that women who live far from campus and have young children are at greatest risk of dropping out. That information is useful—to an extent—in that it identifies who is at risk and where to focus.
But knowing who drops out is not the same as knowing why they do so, which would help the school know how to solve the retention problem. At first glance, these data might suggest that offering childcare might be an appropriate strategy to enhance retention. But the numbers alone are not able to explain whether the retention problem is due to a lack of childcare, poor public transportation options, too much homework, or something else entirely.
Similarly, analyzing data can also make it easier for businesses to avoid addressing the wrong problems, chasing the wrong opportunities, or getting lost in minutia. If analysis reveals that new mothers make up a very small segment of total students, for instance, this might inform the university’s decisions about how much time and effort to invest in recruiting, childcare, or curriculum design.
“Data can act as a source of inspiration, not just a source of validation.” — David Schonthal
For example, Netflix instituted the $1 million “Netflix Prize” with the intention of improving its movie recommendation algorithm by 10 percent. Research groups around the world spent years before achieving the goal—with an algorithm so complex that Netflix never implemented it. Once the company added user profiles to customer accounts, the accuracy of recommendations increased by far more than 10 percent.
“Had Netflix thought of the user interface and algorithm holistically, instead of as distinct functions,” Schonthal says,” they would have invested in designing something intelligent rather than in squeezing the last few digits out of the recommendation algorithm.”
Capture Underlying Motivations
As companies harness the power of data analytics, however, it helps to remember that even if they find an interesting trend or relationship in the data, they may not fully understand how the variables are related or how that relationship will change over time.
“All predictions are based on past relationships,” Shapiro says. “But the environment is constantly shifting. What is true of Amazon shoppers today might be true tomorrow, but for how long? It’s hard to say. So, a business has to ask itself: ‘What are all of the reasons this might not be true tomorrow, or next year?’”
Understanding the possible reasons why a trend might exist is where more qualitative data methods can often help companies.
Say you work in the financial services industry. You know that banking has changed tremendously over the past two decades, with ATMs, online banking, and apps displacing most tellers. Yet a quantitative analysis indicates that your bank still has a hard time getting customers to sign up for “eBanking” accounts.
While the data can reveal that eBanking accounts are unpopular, it might not tell you why customers are resistant to eBanking. Is it a lack of trust? Are customers turned off by the website’s design? And just because the analytics show that app users seem happier with e-banking than desktop users, that does not mean the solution is to redesign the website; it could just be that app users are more comfortable with all types of e-commerce and e-service.
Qualitative analysis—in the form of focus groups, surveys, and customer observation—might provide some insight here by examining customers’ motivations.
What might this look like in practice? Take, for another example, IDEO—where Schonthal also works as Senior Director of business design. The company recently gathered a team of data scientists and designers to help a major travel company reinvent its customer sales and service processes.
An analysis of the travel company’s sales-team data found that although each salesperson worked at the same rate of commission, a handful were consistently outperforming their peers by a wide margin. Still unclear, however, was why that was happening—and how it might be replicated.
Through interviews and observation, IDEO learned something interesting: these high performers often ignored the tools and interaction recommendations that the company provided. Instead, they used unsanctioned methods to help build stronger personal relationships with customers—such as connecting with customers on social media and via text message. This highly personal, somewhat informal approach to their customer communications paid off in the form of materially increased sales, much higher employee satisfaction, and greater customer loyalty—often to both the company and the sales associates themselves.
Scale Your Insights
Still, insights derived from individual interviews and observations will not be useful unless a company can determine how applicable they are to most customers. The most effective tool to track how people behave on a large scale is quantitative analysis.
“You use ‘quant’ to figure out what happened,” Shapiro says. “You use ‘qual’ to figure out why. Then at some point, you need to explicitly test your hypotheses about people’s motivations—to see if they scale into cost-effective solutions.”
This is where analytics re-enter the picture. By returning to quantitative analytics, companies can measure how a potential change might impact revenue, savings, cost, or whatever its value drivers might be.
In IDEO’s work with the travel company, for instance, even after the team had learned about the unconventional approaches used by some of the most successful salespeople, they still needed to understand whether those approaches could help lower-performing members of the sales team. Can these methods help anyone improve, or was this something only the high-performers can pull off?
“It’s always a process of triangulating what you learn in the qualitative research with the factors indicated by the data,” Shapiro says. “When ‘qual’ and ‘quant’ are presented as self-contained methods of analysis, they can lead to bad assumptions. Ultimately, the two should be linked in this dynamic, ongoing process of using data to solve problems.”
