What Might Be Missing from Your Analytics Strategy
Skip to content
Podcast | Insight Unpacked Season 1: Extraordinary Brands and How to Build Them
Data Analytics Innovation Apr 3, 2018

What Might Be Missing from Your Analytics Strategy

Quantitative data is not enough to solve your trickiest problems.

Marketers collect qualitative and quantitative data.

Lisa Röper

Based on insights from

David Schonthal

Joel K. Shapiro

As data analytics becomes a more pervasive business tool, many leaders are being sold on the idea that all you need to diagnose any perplexing problem is more data. While there’s no doubt that quantitative analysis can play a powerful role in telling you what happens, even the most robust, granular data won’t tell you why something happens.

Add Insight
to your inbox.

Instead, employing a combination of qualitative and quantitative methods to identify both the what and the why, according to two Kellogg School professors, is what makes an analytics strategy a useful tool for change.

“Each has something powerful to offer,” says Joel Shapiro, a clinical associate professor of data analytics at Kellogg. “Quantitative analysis helps you identify broader trends, while qualitative analysis digs into human motivation, but the insights are hard to scale.”

David Schonthal, a clinical associate professor of innovation and entrepreneurship at Kellogg, says the real value is in how these two approaches complement each other. “When you combine data analytics with a deeper understanding of a customer’s motivation and experience—that’s how you will create better products and services.”

So how exactly is this done? How should companies avoid the misconception that more data provide all the answers and instead combine “qual” and “quant” to find better solutions to important business problems?

Determine Where to Focus

When searching for new approaches to a long-standing challenge, collecting and analyzing data such as sales figures or conversions can lead to surprisingly fruitful insights.

“This is one place where quant can really help—just knowing where to focus design efforts is extraordinarily valuable,” says Schonthal. “Data can act as a source of inspiration, not just a source of validation.”

Say, for example, that a university has a retention problem with its nontraditional student population. A quantitative analysis can identify that women who live far from campus and have young children are at greatest risk of dropping out. That information is useful—to an extent—in that it identifies who is at risk and where to focus.

But knowing who drops out is not the same as knowing why they do so, which would help the school know how to solve the retention problem. At first glance, these data might suggest that offering childcare might be an appropriate strategy to enhance retention. But the numbers alone are not able to explain whether the retention problem is due to a lack of childcare, poor public transportation options, too much homework, or something else entirely.

Similarly, analyzing data can also make it easier for businesses to avoid addressing the wrong problems, chasing the wrong opportunities, or getting lost in minutia. If analysis reveals that new mothers make up a very small segment of total students, for instance, this might inform the university’s decisions about how much time and effort to invest in recruiting, childcare, or curriculum design.

“Data can act as a source of inspiration, not just a source of validation.” — David Schonthal

For example, Netflix instituted the $1 million “Netflix Prize” with the intention of improving its movie recommendation algorithm by 10 percent. Research groups around the world spent years before achieving the goal—with an algorithm so complex that Netflix never implemented it. Once the company added user profiles to customer accounts, the accuracy of recommendations increased by far more than 10 percent.

“Had Netflix thought of the user interface and algorithm holistically, instead of as distinct functions,” Schonthal says,” they would have invested in designing something intelligent rather than in squeezing the last few digits out of the recommendation algorithm.”

Capture Underlying Motivations

As companies harness the power of data analytics, however, it helps to remember that even if they find an interesting trend or relationship in the data, they may not fully understand how the variables are related or how that relationship will change over time.

“All predictions are based on past relationships,” Shapiro says. “But the environment is constantly shifting. What is true of Amazon shoppers today might be true tomorrow, but for how long? It’s hard to say. So, a business has to ask itself: ‘What are all of the reasons this might not be true tomorrow, or next year?’”

Understanding the possible reasons why a trend might exist is where more qualitative data methods can often help companies.

Say you work in the financial services industry. You know that banking has changed tremendously over the past two decades, with ATMs, online banking, and apps displacing most tellers. Yet a quantitative analysis indicates that your bank still has a hard time getting customers to sign up for “eBanking” accounts.

While the data can reveal that eBanking accounts are unpopular, it might not tell you why customers are resistant to eBanking. Is it a lack of trust? Are customers turned off by the website’s design? And just because the analytics show that app users seem happier with e-banking than desktop users, that does not mean the solution is to redesign the website; it could just be that app users are more comfortable with all types of e-commerce and e-service.

Qualitative analysis—in the form of focus groups, surveys, and customer observation—might provide some insight here by examining customers’ motivations.

What might this look like in practice? Take, for another example, IDEO—where Schonthal also works as Senior Director of business design. The company recently gathered a team of data scientists and designers to help a major travel company reinvent its customer sales and service processes.

An analysis of the travel company’s sales-team data found that although each salesperson worked at the same rate of commission, a handful were consistently outperforming their peers by a wide margin. Still unclear, however, was why that was happening—and how it might be replicated.

Through interviews and observation, IDEO learned something interesting: these high performers often ignored the tools and interaction recommendations that the company provided. Instead, they used unsanctioned methods to help build stronger personal relationships with customers—such as connecting with customers on social media and via text message. This highly personal, somewhat informal approach to their customer communications paid off in the form of materially increased sales, much higher employee satisfaction, and greater customer loyalty—often to both the company and the sales associates themselves.

Scale Your Insights

Still, insights derived from individual interviews and observations will not be useful unless a company can determine how applicable they are to most customers. The most effective tool to track how people behave on a large scale is quantitative analysis.

“You use ‘quant’ to figure out what happened,” Shapiro says. “You use ‘qual’ to figure out why. Then at some point, you need to explicitly test your hypotheses about people’s motivations—to see if they scale into cost-effective solutions.”

This is where analytics re-enter the picture. By returning to quantitative analytics, companies can measure how a potential change might impact revenue, savings, cost, or whatever its value drivers might be.

In IDEO’s work with the travel company, for instance, even after the team had learned about the unconventional approaches used by some of the most successful salespeople, they still needed to understand whether those approaches could help lower-performing members of the sales team. Can these methods help anyone improve, or was this something only the high-performers can pull off?

“It’s always a process of triangulating what you learn in the qualitative research with the factors indicated by the data,” Shapiro says. “When ‘qual’ and ‘quant’ are presented as self-contained methods of analysis, they can lead to bad assumptions. Ultimately, the two should be linked in this dynamic, ongoing process of using data to solve problems.”

About the Writer
Drew Calvert is a freelance writer based in Los Angeles.
Most Popular This Week
  1. Your Team Doesn’t Need You to Be the Hero
    Too many leaders instinctively try to fix a crisis themselves. A U.S. Army colonel explains how to curb this tendency in yourself and allow your teams to flourish.
    person with red cape trying to put out fire while firefighters stand by.
  2. What Triggers a Career Hot Streak?
    New research reveals a recipe for success.
    Collage of sculptor's work culminating in Artist of the Year recognition
  3. What’s the Secret to Successful Innovation?
    Hint: it’s not the product itself.
    standing woman speaking with man seated on stool
  4. Which Form of Government Is Best?
    Democracies may not outlast dictatorships, but they adapt better.
    Is democracy the best form of government?
  5. How Much Do Campaign Ads Matter?
    Tone is key, according to new research, which found that a change in TV ad strategy could have altered the results of the 2000 presidential election.
    Political advertisements on television next to polling place
  6. What Went Wrong with FTX—and What’s Next for Crypto?
    One key issue will be introducing regulation without strangling innovation, a fintech expert explains.
    stock trader surrounded by computer monitors
  7. How Are Black–White Biracial People Perceived in Terms of Race?
    Understanding the answer—and why black and white Americans may percieve biracial people differently—is increasingly important in a multiracial society.
    How are biracial people perceived in terms of race
  8. Immigrants to the U.S. Create More Jobs than They Take
    A new study finds that immigrants are far more likely to found companies—both large and small—than native-born Americans.
    Immigrant CEO welcomes new hires
  9. How Experts Make Complex Decisions
    By studying 200 million chess moves, researchers shed light on what gives players an advantage—and what trips them up.
    two people playing chess
  10. Yes, Consumers Care if Your Product Is Ethical
    New research shows that morality matters—but it’s in the eye of the beholder.
    woman chooses organic lettuce in grocery
  11. Why Well-Meaning NGOs Sometimes Do More Harm than Good
    Studies of aid groups in Ghana and Uganda show why it’s so important to coordinate with local governments and institutions.
    To succeed, foreign aid and health programs need buy-in and coordination with local partners.
  12. Product Q&A Forums Hold a Lot of Promise. Here’s How to Make Them Work.
    The key to these online communities, where users can ask and answer questions, is how many questions get useful answers.
    man sits at computer reading Q&A forum
  13. What Went Wrong at AIG?
    Unpacking the insurance giant's collapse during the 2008 financial crisis.
    What went wrong during the AIG financial crisis?
  14. When Do Open Borders Make Economic Sense?
    A new study provides a window into the logic behind various immigration policies.
    How immigration affects the economy depends on taxation and worker skills.
  15. What the New Climate Bill Means for the U.S.—and the World
    The Inflation Reduction Act won’t reverse inflation or halt climate change, but it's still a big deal.
    energy bill with solar panels wind turbines and pipelines
  16. Post-War Reconstruction Is a Good Investment
    Ukraine’s European neighbors will need to make a major financial commitment to help rebuild its economy after the war. Fortunately, as the legacy of the post–World War II Marshall Plan shows, investing in Ukraine's future will also serve Europe's own long-term interests.
    two people look out over a city
  17. How Has Marketing Changed over the Past Half-Century?
    Phil Kotler’s groundbreaking textbook came out 55 years ago. Sixteen editions later, he and coauthor Alexander Chernev discuss how big data, social media, and purpose-driven branding are moving the field forward.
    people in 1967 and 2022 react to advertising
  18. The Political Divide in America Goes Beyond Polarization and Tribalism
    These days, political identity functions a lot like religious identity.
    people engage in conflict with swords
More in Data Analytics