What accounts for the tendency of housing prices to steadily increase for several years—and then steadily decline?That deceptively simple question may have a new answer, based on the research of Charles Nathanson, an assistant professor of finance at the Kellogg School.

Few economic stories get more consistent press coverage than the housing cycle. The media regularly report on “housing starts,” or the number of new home constructions, along with the average cost of new homes and swings in supply and demand. Nearly everyone agrees that the health of the housing market is critical to the economy’s prospects.

But for all that, there are few certainties about precisely what drives the housing cycle, especially the kind of extreme boom and bust that happened in the early 2000s. Substantial research into the question has not yielded a clear picture of the housing crash. Researchers are also struggling to come up with ways of identifying—or preventing—bubbles.

The challenge is that so many variables—like fluctuating interest rates and construction costs—affect the workings of supply and demand. One thing is certain though: sellers eventually overshoot, buyers will not pay the asking prices, and the boom becomes a bust.

“How do buyers decide what they are willing to pay for what will likely become their most valuable asset?”

Nathanson and his coauthor, Edward Glaeser of Harvard University, suggest that an error buyers make in assessing the future value of a house helps to explain the ups and downs of the cycle. The error, which involves forecasting future home prices based on past and current ones, is small but widespread and contributes to momentum toward higher housing prices. When prices reach an unsustainable level, the momentum swings in the other direction, and prices return to the previous level.

One Decision, Two Types of Buyers

For most people, buying a house is unlike any other purchase, and one with which they have minimal experience. “We usually think of people as better at things they do repeatedly,” Nathanson said. Buying a home is not one of those things for most people. Moreover, a home purchase typically requires the buyer to take on debt that she expects to pay back over the span of decades. Many buyers expect to sell their house at a profit before the loan is paid off. A house is an investment.

But how do buyers decide what they are willing to pay for what will likely become their most valuable asset?

“When you buy a house, you’re probably going to sell it at some point,” Nathanson said. “How much you pay for the house depends on how much you think you can sell it for in 7 to 10 years. That’s a difficult problem. To figure that out, you’re going to look at past house prices and say, ‘Are house prices in this area going up?’”

To be fair, predicting future prices based on past prices is “a very reasonable thing to do,” Nathanson said. But the problem is that it is hard to do this correctly. In fact, buyers fall into two distinct groups when it comes to forecasting house prices: those who consider all the relevant information, who behave in a way that maximizes profit, or whom economists label “rational” actors; and those who make assumptions based on limited information, or “naïve” buyers.

How do these groups behave when it comes to gauging home prices?

Rational buyers know that prices are not a precise reflection of supply and demand. They recognize that beliefs and expectations are baked into the numbers, and they filter out the psychological factors that inflate home prices, focusing instead on items tied more closely to demand. According to Nathanson, they look at things like “how strong the local economy is, how many people want to live in that neighborhood, and how good the local schools might be in the future.”

Naïve buyers, on the other hand, believe their information is better than it actually is. Consciously or not, they use a rule of thumb that past and current prices accurately reflect demand. “They see prices went up five percent and assume demand to live in that area went up five percent,” Nathanson said. This kind of buyer may look at a house listed for $200,000, see that other homes in the area have similar prices, and assume the prices accurately reflect demand. That makes them willing to pay the list price or even a little bit more if competing with others. “Then you get these things that look like housing bubbles,” Nathanson said.

Small Error, Big Effects

The error naïve buyers make in forecasting future prices based on the past may seem small, but it can snowball. When the results of a model simulating rational behavior are set against a model simulating naïve behavior over several years, the effects are striking.

Nathanson and Glaeser found that over time the error leads naïve buyers to overestimate the actual level of housing demand—and thus overvalue homes. Future buyers, in turn, make the same error. This becomes a sequential and self-reinforcing process, and the gap between prices and demand grows ever-wider.

Interestingly, though naïve buyers expect prices to rise, they actually underestimate the near-term rise in the value of homes during a boom. This is true for the same reason that they err when looking to past prices for guidance. They do not consider the fact that future buyers will update their beliefs.

“Naïve buyers don’t realize that other buyers are learning from prices, too,” Nathanson said. “So if I’m thinking about buyers in the future, I know prices are going up today, but I think that buyers in the future aren’t going to learn anything from that. But in reality, they will look at the rising prices, and that will increase their optimism.” Thus, when current buyers fail to take the perceptions of future buyers into account, they underestimate the value of the house in question.

This error is key to understanding the housing market’s momentum—or the tendency for prices to continue on their current path within a given swing of a boom and bust cycle. The gap between naïve buyers’ expectations (what they think the house is worth) and reality (that it is worth more than they think it is) surprises them. It provides a psychological push, realized as momentum, that keeps prices ascending over the short run.

Past the five-year time horizon, however, buyers’ expectations outstrip the actual growth in prices. The same process, working in reverse, then pulls prices back down. Naïve buyers now overestimate the true level of demand and are surprised when homes are worth less than they think they are, creating momentum fueled by sequential undervaluing. And that contributes to a bust.

Curb Your Exuberance

As Glaeser and Nathanson note, this is “only one possible model of semi-rationality in housing markets. Many other forms of irrationality may exist.”

But even if they do not provide the final word on the subject, the authors do offer an explanation for something that is typically assumed but not proven: the psychology, or “irrational exuberance,” that shapes the housing cycle.

“One of the stories [about the boom in the housing market in the early 2000s] is that people just got overly optimistic somehow,” Nathanson said. “And that the bust was predictable,” once buyers started to believe that houses were overvalued.

“We actually have a theory [presented above] for why people would consistently be too optimistic during booms and too pessimistic during busts,” Nathanson said. “In several other papers about housing cycles, including my own work, this optimism and pessimism is just assumed.”

Ultimately, the aim of understanding the ups and downs of the housing cycle is to help legislators and institutions create laws and policies that prevent the kind of dramatic swing in prices that happened from 2000 to 2006. The bust that followed inflicted devastation across the economy and played a critical role in the broader financial crisis that began in 2008.

As Nathanson and Glaeser write, “the economics of real estate bubbles is still in its infancy,” and it is too early to make policy prescriptions with any confidence.

But if their thesis holds up, “the lesson is that people are learning from prices the wrong way,” Nathanson said. “So the question is: Is there a better way for people to forecast future prices from past prices—one that people can use, and that’s not terribly complicated?” That is something Nathanson is exploring in current research.

As he pointed out, the practical implications of such follow-on research could be valuable: “There might be some very simple rules that people can use, that won’t lead them astray. If we can actually discover some simple rule people could use to avoid booms and busts, we could educate people about what that is.”