What Makes Oil Prices So Volatile?

Take 1973, when oil prices skyrocketed unexpectedly. Within two years, US unemployment had doubled, leaving the nation deep in the throes of a recession. (The rest of the world soon followed.) Similar tailspins occurred in 1979 and 1991, both caused by sharp increases in oil prices.

“The few recessions where we can name the culprit are oil-driven recessions,” explains Sergio Rebelo, a finance professor at the Kellogg School.

But what makes oil prices so volatile? Despite the high stakes, economists have not had enough data to answer this question. Do shocks to suppliers yank prices upward, or is shifting demand from developing countries part of the problem? And will this volatility continue?

Rebelo set out to answer these questions with fellow economists Gideon Bornstein, a PhD student at Northwestern, and Per Krusell of Stockholm University. The researchers had one critical advantage: a massive oil industry dataset containing historical information on every oil field in the world.

“Those data allowed us to take a close look at the inner workings of the oil industry for the first time,” Rebelo says.

With the data in hand, they have teased out how supply and demand each contribute to the volatility of oil. Counterintuitively, they also predict that fracking—the controversial technique of fracturing underground rock to tap its oil reserves—will eventually smooth out the market’s jagged roller-coaster ride.

“It’ll take out the peaks and troughs of oil prices,” Rebelo says.

A Data Treasure Trove

Three years ago, Rebelo and his coauthors procured a proprietary dataset covering some 14,000 oil fields operated by 3,200 companies since 1900. (For the purposes of their analysis, the researchers zeroed in on the period from 1970 to 2015.)

The data set, which is meant to be used by energy companies, provides unprecedented amounts of information, including not only the amount of oil produced annually, but which countries are producing it, and—crucially—the dollar amount that companies invest in their oil fields each year.

With these data in hand, the researchers set out to draw some basic facts about the oil industry over the last half century. “For instance,” Rebelo says, “we didn’t know much about the investment behavior of this industry.”

When they charted year-to-year changes, the authors found that the amount of resources energy companies invest in oil exploration and extraction was more volatile than oil prices. And when oil prices increased, investment in the oil industry increased along with them, as energy companies sought to capitalize on the higher profit margins.

Curiously, though, those investments did not seem to immediately affect the amount of oil extracted. “We show it’s very difficult to produce more oil in the short run,” Rebelo explains.

Suppliers can boost oil production in one of two ways: by extracting more quickly from existing oil fields, or by finding and drilling new fields. But there are upper limits to how fast oil can be extracted—and developing new wells takes time. The upshot: companies cannot boost production quickly in response to high oil prices.

“In our data, there’s an average lag of 12 years between investment and production,” says Rebelo.

What Explains Oil-Price Fluctuations?

The researchers also set out to understand what drove the volatility in oil prices.

Ask your average person, and you will likely get an answer that revolves around suppliers, Rebelo says. “When you read the Wall Street Journal, it sounds like OPEC—Saudi Arabia in particular—by producing more oil or less, can really manipulate the market.”

Indeed, a quick look at the data revealed that at certain moments, suppliers had effectively cut production—like in 1973, when the oil-producing Arab member nations of OPEC placed an embargo on oil exports to the US. During this period, “we see oil fields getting shut down,” Rebelo says.

But were these moments telling the full story? To find out, the researchers created a mathematical model to estimate the fraction of all oil-price fluctuations that could be attributed to shifts in supply.

What they found is that abrupt changes in supply explain only about half of the jumps in prices.

Rebelo notes that turning oil production on or off is a serious decision that costs millions of dollars. “It’s not like you have a tap, like you’re in the bathtub and you can put in more warm water or less,” he says.

That difficulty in adjusting supply in the short run explains why a full half of oil-price fluctuations were due to changes in consumer demand, which moved up and down according to the changing needs of the world economy.

Shifts in oil investment, on the other hand, appear to be driven primarily by fluctuations in demand. Rebelo says that this is due to the lag between investment and production, combined with the fact that the kinds of supply-side interruptions that ratchet up oil prices tend to be short-lived.

When the first Gulf War started, for example, oil production stalled in Iraq and Kuwait, leading prices to soar. But Rebelo says it did not make sense for companies to try to capitalize on those high prices by finding new oil sources. “By the time that new oil comes online, the shock will be gone, and the prices will be back to normal.”

Demand, on the other hand, tends to be more stable, and therefore provides a better incentive for investment, says Rebelo. “If China is going to grow more than expected for another decade”—and thus experience a steadily growing thirst for oil—“then it makes a lot of sense to invest.”

Fracking and the Future

Where does fracking fit into this picture?

“Fracking started at a time when there was a lot of volatility in oil prices. So fracking and volatility tend to be linked,” Rebelo says.

But his new findings defy the consensus that fracking exacerbated volatility. Instead, the study suggests that fracking could eventually help stabilize oil markets. Fracking, which is short for hydraulic fracturing, involves the high-pressure injection of water into the ground to extract oil and natural gas. It is an environmentally controversial practice, in part, because of concerns over contamination of nearby groundwater.

Because it takes an average of 12 years to produce oil from an investment, companies have not historically been able to quickly capitalize on high prices. Today, however, that is no longer the case. The researchers found that fracking firms can produce oil much more nimbly than traditional firms.

“It takes on average about one year between investment and production in a fracking field,” says Rebelo.

As such, when oil prices rise, fracking firms will be able to take advantage of many booms before they end. And by producing more oil, they will put downward pressure on oil prices. Conversely, when oil prices are low, they are able to more easily halt production, driving prices upward.

In total, the authors calculate that fracking will reduce oil-price volatility by an astounding 65 percent.

Rebelo warns that we are still in the midst of the fracking transition, so it might be a while before the full effect can be seen. “We don’t have a situation where the fracking industry is yet mature,” he says. (The paper did not look at the potential environmental costs of fracking.)

Rebelo and his coauthors are interested to see if their prediction is borne out. Until then, they are glad to have helped shed light on such an important sector of the economy.

“We had this unusual opportunity to study the mechanics of the oil industry,” he says.