Podcast: How to Prepare for Your New Algorithmic Coworker

What is clear: humans will be working alongside this technology in some capacity, and leaders need to understand its strengths and weaknesses to make the most of it.

“These models get their juice on the information that’s fed into them, and one of the best competitive advantages that all companies have is internal data that no one else has,” says Brian Uzzi, a professor of management and organizations at Kellogg.

On this episode of The Insightful Leader, we talk to Uzzi and Dave Ferrucci, an adjunct professor of management and organizations at Kellogg—as well as the founder, CEO, and Chief Scientist of the hybrid AI company Elemental Cognition—about how these large-scale language models work and what leaders should make of them.

Editor’s note: To take our listener survey, visit kell.gg/podsurvey.

PODCAST TRANSCRIPT

Laura PAVIN: At this point I don’t have to tell you that artificial intelligence is having a moment. Much like crypto a year and a half ago, AI is everywhere. But the big difference between the two? A lot of people are actually using AI.

[clips of the different types of AI]

PAVIN: With the increasing popularity of tools like ChatGPT and DALL-E, AI is hitting the mainstream, and pundits, academics, and honestly everybody is wondering: How is this going to change our economy, our society, and really just our entire lives? The rise has come on so quickly, in fact, that some are saying we need to hit the pause button.

[news clip of petition]

David FERRUCCI: Giving us a chance to take a breather and think about the implications is probably wise. This petition may ultimately be largely symbolic in getting people to wake up and say, “This is something serious going on. We need to think about its implications.” And I think that’s right on.

PAVIN: That’s David Ferrucci. He’s the founder, CEO, and Chief Scientist at Elemental Cognition, which builds AIs. He’s also an adjunct professor at Kellogg. And he recently gave a talk along with professor of management and organizations Brian Uzzi about what the AI future could look like and what it could mean for businesses. Uzzi agreed that taking a moment to think critically about AI is important.

Brian UZZI: I remember 15 years ago, when social media started. There was a similar petition among scientists. We didn’t know what was gonna happen. Yes, it looks like there’s a Biblical gift from God. All this knowledge. And then it became monetized, and once it became monetized, then, you know, it had a completely different trajectory, and it turned into a world of, you know, incivility, disinformation, and we still can’t decide what kind of regulation we want for that.

PAVIN: This is The Insightful Leader. I’m Laura Pavin. Today we take a step back to reflect on one of the most popular AI tools, ChatGPT. We’ll look at how it is like, and unlike, human intelligence, how it might shape the workforce, and how business leaders can prepare for what’s to come. That’s next.

…

PAVIN: Alright, so before we can get to what all this AI stuff means for business leaders, let’s talk about how this technology actually works. Like I said before, we’re going to be focused on ChatGPT this episode. If for some reason you’ve resisted the urge or just haven’t had the time to try ChatGPT yourself, essentially it looks like a standard text box where you can type anything in, like, say, “give me 5 vegetarian recipes that I can cook for dinner this week.”

PAVIN: And it’ll say something like….

Jessica AI LOVE: Certainly! Here are five delicious vegetarian recipes for dinner that you can try this week: Lentil and Vegetable Curry—a comforting and flavorful meal that’s easy to make. Heat some oil in a pot, add chopped onions, garlic, and ginger, and sauté until soft. Add curry powder....

FERRUCCI: They’re incredibly good at predicting the sort of language that humans might write because they’re trained on massive amounts of human-written content. And then, when you give it, you know a prompt, it quickly narrows in on very similar contexts that it has seen before and stored these distributions and starts to predict the next likely word.

PAVIN: It makes these predictions over and over again in a matter of seconds until what you have in front of you seems like a totally coherent response. It really does feel like magic when you use it.

AI LOVE: Preheat your oven to 375°F and line a baking sheet with parchment paper. Remove the stems from the Portobello mushrooms and clean the caps. In a large skillet, heat the olive oil over medium heat. Add the minced garlic and sauté for about 1 minute until fragrant....

PAVIN: It’s able to make these predictions, because of all that text data that Ferrucci mentioned, ChatGPT-4—OpenAI’s latest model—was fed around 570 gigabytes of data from the internet. That’s roughly 300 billion words. Ferrucci says, in some ways, ChatGPT mirrors a form of human intelligence.

FERRUCCI: One way we learn is we look at patterns, we look at syntactic or structural patterns. So you can imagine someone reading a text, and then they get a question, and they look at that question, and they syntactically look back at the text and say, where does it match best?

PAVIN: A human example of this would be when we’re asked something we don’t actually know anything about. Even without a lot of knowledge, we can pretty much give a response that makes sense, even if it’s inaccurate. ChatGPT is doing exactly that on a massive scale. It’s just predicting what word would be most likely to occur next. It’s pattern-matching.

But this sort of ability is really different from other forms of human knowledge, Ferucci says. For example, we humans build structures in our brains to understand how things work. But language models? They’re just looking at statistical co-occurrences.

It’s the difference between knowing how to cook and knowing how to sound like you know how to cook. ChatGPT doesn’t actually know anything about vegetarian food. It’s just very good at synthesizing a bunch of information to show something that seems convincing.

AI LOVE: Creamy tomato and basil pasta….

PAVIN: Eerily convincing.

AI LOVE: A classic Italian pasta dish that’s easy to make and bursting with flavor.

FERRUCCI: The interesting thing about them is, there’s so much language out there that a lot of the way we think is embedded, is projected, by the language we use. So when you project, if you could process enough language when you project out of that language, that language starts to mimic the way we think.

…

PAVIN: All of this has a lot of people very excited about the potential power of ChatGPT and other language models. Imagine having a tool that could write drafts of emails for you, help you think through a big presentation you’re about to give, or even plan an entire trip abroad for you. It’s pretty cool.

But it’s also got people freaked out. If AI can write emails and design presentations and plan trips, what can’t it do—and how long until it comes for our jobs?

Uzzi thinks a forthcoming study from the University of Pennsylvania gives some insight.

UZZI: Basically they went into all the job data that we have collected in terms of skills and tests within jobs across all types of jobs. And the basic approach was to say, what could AI do 50 percent faster than a human could do?

PAVIN: What they discovered was pretty incredible. They found that 19 percent of the workforce could see at least half of their tasks involve or be largely done by language-model AIs. And that 80 percent would see at least 10 percent of their tasks involve AI. Meaning that workers in practically every industry are going to see tools like ChatGPT play some sort of role in their work.

UZZI: Now one of the things that’s exciting and also a little scary about this is when you look at 80 percent of the jobs, a lot of them are really high-income jobs—accountants, attorneys, mathematicians interpreters, journalists, public relations people, court reporters—so the implications are really far-reaching, in what they would be able to do.

PAVIN: The reason these white-collar jobs in particular would be impacted by ChatGPT is that all of these careers heavily involve language, synthesizing information and making arguments from different perspectives. Often, AI doesn’t actually need to interpret or understand what it’s communicating—it just needs to use the right words in the right sequence, and humans will be satisfied with that.

So should we be freaking out because our jobs are going to be replaced by a chatbot next year? No. Or at least not in the immediate future, according to Ferrucci and Uzzi. But what does seem clear is that we can’t ignore this technology, because it is going to play a role in our work. So, lesson one is this: Get your hands dirty and play around with this thing. Try to understand it. See what things it’s good and bad at. See if it can do some parts of your job for you or make a task easier. The more you experiment, the better you’ll get a grasp on what this tool is actually capable of.

…

PAVIN: Experimenting with these tools will also show you that chatbots have some very serious problems. Like we said earlier, ChatGPT is really good at predicting and generating convincing answers to questions and prompts. The issue is that, what it’s able to present is highly skewed by the data it’s trained on.

Going back to our vegetarian-dishes example, if the data used to train an AI didn’t include many examples of mushrooms being used in cooking, those aren’t going to show up very frequently in the recipes. And that’s a shame, because mushrooms are tasty! So we as humans have to act kind of like an editor, interrogating what biases are present in the answers you’re receiving.

But AI isn’t just influenced by the data it’s trained on. The answers you get from ChatGPT are also highly influenced by the questions themselves. Meaning that if you ask the same question two different ways, you can get two different answers.

As just a quick benign example. If I type in here, “What are the most important things to see if I’m visiting Rome?” Here’s what I get:

AI LOVE: Rome is a city with a rich history and culture, and there are many important things to see and do. Here are some of the most popular attractions and sights that you might consider visiting: The Colosseum—the iconic amphitheater is a symbol of Ancient Rome and a must-see for visitors of the city. The Vatican museums—home of some of the world’s most famous art and artifacts, including the Sistine Chapel and St. Peter’s Basilica.

PAVIN: And on and on it goes with a listicle of specific places. However, if I just type, “What should I do in Rome?” I get this:

AI LOVE: There are plenty of things to do while you’re in Rome. Here are some suggestions: Explore the city on foot—Rome is a great city for walking and exploring on foot. Take a stroll through the winding streets and alleys to discover hidden gems and charming neighborhoods. Take a guided tour:here are many guided tours available in Rome that can take you to the most important sights and provide you with historical context and interesting anecdotes.

PAVIN: … With the prompt going on to give general ideas for ways to experience Rome. Obviously, the differences here are trivial—it’s just vacation advice. But you can imagine when it comes to political, moral, or social topics that these differences might not be trivial.

So while these tools are powerful and sometimes feel really magical, we should always be asking why particular questions lead to certain results.

FERRUCCI: While intelligence might do a lot of the grunt work for us, we’re going to have to make those value judgments. And that comes from learning how to be a critical thinker. Learning how to be objective, open up our minds, how to actually validate, check, understand, build rational models for what we’re hearing, understanding what’s an assumption and what’s a logical inference: that’s critical thinking.

PAVIN: So lesson 2, while you’re experimenting with these tools, make sure that you’re also asking questions. Why did you get this result? I asked it the same question but in this different way and got another answer. Why is that? What biases might I be bringing to this tool? And what things might the tool not be telling me?

…

PAVIN: OK. So what does all this mean for you as a business leader? What impact will these have on your company or the team you lead? For Uzzi, it comes down to data.

UZZI: These models get their juice on the information that’s fed into them, and one of the best competitive advantages that all companies have is internal data that no one else has. But that data tends not to be managed well. It tends to be, you know, in whole different parts of the organization. It can’t be combined together. But information is in fact the thing that makes the whole system go. And so organizations should pay a lot more attention to managing, curating, combining, cleaning the data they have, because that is their secret sauce relative to their competitors.

PAVIN: Remember, AI is only as powerful as the data it’s fed. So you’ll need to think about whether your company’s data is AI-ready.

And there’s another thing companies should consider. Here’s Ferrucci again.

FERRUCCI: A company should understand the technology, what its strengths and weaknesses are, and look at their current processes and their current functional roles, and see where can this help. Where can this make us more efficient? But at the same time be cautious of again issues around transparency and accountability, bias and veracity. But given an awareness of that, you know, where can we inject it to help the company be more efficient. I think that certainly all companies should be thinking about that.

PAVIN: Now, this might be challenging, given that running your actual company is already taking up a lot of the bandwidth of your employees. Because, honestly, who has time to learn yet another piece of technology?

One solution, Uzzi says, is for companies to create a task force that can dedicate some time to understanding this technology and help provide guidance on an approach at the organization.

UZZI: Part of what’s going to happen is they’re going to have to create some specialized groups that can take a lead on this, that’d be like an internal consulting group that can help other parts of the organization learn fastest.

PAVIN: Giving yourself and your employees space and time to learn and play with these tools may ultimately be what leads to a breakthrough in how you do your work.

...

PAVIN: So, what should you think about AI? It’s hard to say. But even for all his concerns, Ferrucci is overall really excited about its potential.

FERRUCCI: As I said, we have to learn. We have to be cautious, but I am fully optimistic that there is no future of the human species without AI. It is the main accelerator. So anyway, I couldn’t be more optimistic.

[CREDITS]

PAVIN: This episode of The Insightful Leader was written and mixed by Andrew Meriwether. It was produced by Laura Pavin, Jessica Love, Emily Stone, Fred Schmalz, Maja Kos, and Blake Goble. Special thanks to David Ferrucci and Brian Uzzi. Want more The Insightful Leader episodes? You can find us on iTunes, Spotify, or our website: insight.kellogg.northwestern.edu. And again, we’re taking a break, but we’ll see you real soon.