What Might Be Missing from Your Analytics Strategy
Skip to content
Data Analytics Innovation Apr 3, 2018

What Might Be Missing from Your Analytics Strategy

Quantitative data is not enough to solve your trickiest problems.

Marketers collect qualitative and quantitative data.

Lisa Röper

Based on insights from

David Schonthal

Joel K. Shapiro

As data analytics becomes a more pervasive business tool, many leaders are being sold on the idea that all you need to diagnose any perplexing problem is more data. While there’s no doubt that quantitative analysis can play a powerful role in telling you what happens, even the most robust, granular data won’t tell you why something happens.

Instead, employing a combination of qualitative and quantitative methods to identify both the what and the why, according to two Kellogg School professors, is what makes an analytics strategy a useful tool for change.

“Each has something powerful to offer,” says Joel Shapiro, a clinical associate professor of data analytics at Kellogg. “Quantitative analysis helps you identify broader trends, while qualitative analysis digs into human motivation, but the insights are hard to scale.”

David Schonthal, a clinical associate professor of innovation and entrepreneurship at Kellogg, says the real value is in how these two approaches complement each other. “When you combine data analytics with a deeper understanding of a customer’s motivation and experience—that’s how you will create better products and services.”

So how exactly is this done? How should companies avoid the misconception that more data provide all the answers and instead combine “qual” and “quant” to find better solutions to important business problems?

Determine Where to Focus

When searching for new approaches to a long-standing challenge, collecting and analyzing data such as sales figures or conversions can lead to surprisingly fruitful insights.

“This is one place where quant can really help—just knowing where to focus design efforts is extraordinarily valuable,” says Schonthal. “Data can act as a source of inspiration, not just a source of validation.”

Say, for example, that a university has a retention problem with its nontraditional student population. A quantitative analysis can identify that women who live far from campus and have young children are at greatest risk of dropping out. That information is useful—to an extent—in that it identifies who is at risk and where to focus.

But knowing who drops out is not the same as knowing why they do so, which would help the school know how to solve the retention problem. At first glance, these data might suggest that offering childcare might be an appropriate strategy to enhance retention. But the numbers alone are not able to explain whether the retention problem is due to a lack of childcare, poor public transportation options, too much homework, or something else entirely.

Similarly, analyzing data can also make it easier for businesses to avoid addressing the wrong problems, chasing the wrong opportunities, or getting lost in minutia. If analysis reveals that new mothers make up a very small segment of total students, for instance, this might inform the university’s decisions about how much time and effort to invest in recruiting, childcare, or curriculum design.

“Data can act as a source of inspiration, not just a source of validation.” — David Schonthal

For example, Netflix instituted the $1 million “Netflix Prize” with the intention of improving its movie recommendation algorithm by 10 percent. Research groups around the world spent years before achieving the goal—with an algorithm so complex that Netflix never implemented it. Once the company added user profiles to customer accounts, the accuracy of recommendations increased by far more than 10 percent.

“Had Netflix thought of the user interface and algorithm holistically, instead of as distinct functions,” Schonthal says,” they would have invested in designing something intelligent rather than in squeezing the last few digits out of the recommendation algorithm.”

Capture Underlying Motivations

As companies harness the power of data analytics, however, it helps to remember that even if they find an interesting trend or relationship in the data, they may not fully understand how the variables are related or how that relationship will change over time.

“All predictions are based on past relationships,” Shapiro says. “But the environment is constantly shifting. What is true of Amazon shoppers today might be true tomorrow, but for how long? It’s hard to say. So, a business has to ask itself: ‘What are all of the reasons this might not be true tomorrow, or next year?’”

Understanding the possible reasons why a trend might exist is where more qualitative data methods can often help companies.

Say you work in the financial services industry. You know that banking has changed tremendously over the past two decades, with ATMs, online banking, and apps displacing most tellers. Yet a quantitative analysis indicates that your bank still has a hard time getting customers to sign up for “eBanking” accounts.

While the data can reveal that eBanking accounts are unpopular, it might not tell you why customers are resistant to eBanking. Is it a lack of trust? Are customers turned off by the website’s design? And just because the analytics show that app users seem happier with e-banking than desktop users, that does not mean the solution is to redesign the website; it could just be that app users are more comfortable with all types of e-commerce and e-service.

Qualitative analysis—in the form of focus groups, surveys, and customer observation—might provide some insight here by examining customers’ motivations.

What might this look like in practice? Take, for another example, IDEO—where Schonthal also works as Senior Director of business design. The company recently gathered a team of data scientists and designers to help a major travel company reinvent its customer sales and service processes.

An analysis of the travel company’s sales-team data found that although each salesperson worked at the same rate of commission, a handful were consistently outperforming their peers by a wide margin. Still unclear, however, was why that was happening—and how it might be replicated.

Through interviews and observation, IDEO learned something interesting: these high performers often ignored the tools and interaction recommendations that the company provided. Instead, they used unsanctioned methods to help build stronger personal relationships with customers—such as connecting with customers on social media and via text message. This highly personal, somewhat informal approach to their customer communications paid off in the form of materially increased sales, much higher employee satisfaction, and greater customer loyalty—often to both the company and the sales associates themselves.

Scale Your Insights

Still, insights derived from individual interviews and observations will not be useful unless a company can determine how applicable they are to most customers. The most effective tool to track how people behave on a large scale is quantitative analysis.

“You use ‘quant’ to figure out what happened,” Shapiro says. “You use ‘qual’ to figure out why. Then at some point, you need to explicitly test your hypotheses about people’s motivations—to see if they scale into cost-effective solutions.”

This is where analytics re-enter the picture. By returning to quantitative analytics, companies can measure how a potential change might impact revenue, savings, cost, or whatever its value drivers might be.

In IDEO’s work with the travel company, for instance, even after the team had learned about the unconventional approaches used by some of the most successful salespeople, they still needed to understand whether those approaches could help lower-performing members of the sales team. Can these methods help anyone improve, or was this something only the high-performers can pull off?

“It’s always a process of triangulating what you learn in the qualitative research with the factors indicated by the data,” Shapiro says. “When ‘qual’ and ‘quant’ are presented as self-contained methods of analysis, they can lead to bad assumptions. Ultimately, the two should be linked in this dynamic, ongoing process of using data to solve problems.”

Featured Faculty

Clinical Professor of Strategy; Director of Entrepreneurship Programs at Kellogg; Faculty Director of the Zell Fellows Program; Director of the Levy Institute for Entrepreneurial Practice

Clinical Associate Professor of Managerial Economics & Decision Sciences

About the Writer
Drew Calvert is a freelance writer based in Los Angeles.
Most Popular This Week
  1. 3 Things to Keep in Mind When Delivering Negative Feedback
    First, understand the purpose of the conversation, which is trickier than it sounds.
  2. Podcast: Workers Are Stressed Out. Here’s How Leaders Can Help.
    On this episode of The Insightful Leader: You can’t always control what happens at work. But reframing setbacks, and instituting some serious calendar discipline, can go a long way toward reducing stress.
  3. What Went Wrong at Silicon Valley Bank?
    And how can it be avoided next time? A new analysis sheds light on vulnerabilities within the U.S. banking industry.
    People visit a bank
  4. How Are Black–White Biracial People Perceived in Terms of Race?
    Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
    How are biracial people perceived in terms of race
  5. Will AI Eventually Replace Doctors?
    Maybe not entirely. But the doctor–patient relationship is likely to change dramatically.
    doctors offices in small nodules
  6. Leaders, Don’t Be Afraid to Admit Your Flaws
    We prefer to work for people who can make themselves vulnerable, a new study finds. But there are limits.
    person removes mask to show less happy face
  7. Which Form of Government Is Best?
    Democracies may not outlast dictatorships, but they adapt better.
    Is democracy the best form of government?
  8. What Went Wrong at AIG?
    Unpacking the insurance giant's collapse during the 2008 financial crisis.
    What went wrong during the AIG financial crisis?
  9. What Happens to Worker Productivity after a Minimum Wage Increase?
    A pay raise boosts productivity for some—but the impact on the bottom line is more complicated.
    employees unload pallets from a truck using hand carts
  10. At Their Best, Self-Learning Algorithms Can Be a “Win-Win-Win”
    Lyft is using ”reinforcement learning” to match customers to drivers—leading to higher profits for the company, more work for drivers, and happier customers.
    person waiting for rideshare on roads paved with computing code
  11. When You’re Hot, You’re Hot: Career Successes Come in Clusters
    Bursts of brilliance happen for almost everyone. Explore the “hot streaks” of thousands of directors, artists and scientists in our graphic.
    An artist has a hot streak in her career.
  12. Why Do Some People Succeed after Failing, While Others Continue to Flounder?
    A new study dispels some of the mystery behind success after failure.
    Scientists build a staircase from paper
  13. Immigrants to the U.S. Create More Jobs than They Take
    A new study finds that immigrants are far more likely to found companies—both large and small—than native-born Americans.
    Immigrant CEO welcomes new hires
  14. Take 5: Tips for Widening—and Improving—Your Candidate Pool
    Common biases can cause companies to overlook a wealth of top talent.
  15. Why Well-Meaning NGOs Sometimes Do More Harm than Good
    Studies of aid groups in Ghana and Uganda show why it’s so important to coordinate with local governments and institutions.
    To succeed, foreign aid and health programs need buy-in and coordination with local partners.
  16. How Has Marketing Changed over the Past Half-Century?
    Phil Kotler’s groundbreaking textbook came out 55 years ago. Sixteen editions later, he and coauthor Alexander Chernev discuss how big data, social media, and purpose-driven branding are moving the field forward.
    people in 1967 and 2022 react to advertising
  17. How Peer Pressure Can Lead Teens to Underachieve—Even in Schools Where It’s “Cool to Be Smart”
    New research offers lessons for administrators hoping to improve student performance.
    Eager student raises hand while other student hesitates.
  18. How Much Do Campaign Ads Matter?
    Tone is key, according to new research, which found that a change in TV ad strategy could have altered the results of the 2000 presidential election.
    Political advertisements on television next to polling place
  19. Take 5: How Fear Influences Our Decisions
    Our anxieties about the future can have surprising implications for our health, our family lives, and our careers.
    A CEO's risk aversion encourages underperformance.
More in Data Analytics