Computer Models at the Fed

From this modest start, the science — and, some would say, the art — of computer modeling of the economy has become indispensable to policymakers and businesses seeking to forecast economic variables such as GDP and employment or to analyze the likely effects of policy changes. The Fed's main computer model since the mid-1990s, known as FRB/US (commonly pronounced "ferbus"), has about 380 equations covering the behavior of households, firms, inflation, relative prices, numerous interest rates, and government taxes and spending (at the federal, state, and local levels), among other phenomena.

Yet even as large-scale macroeconomic models such as FRB/US have attained a role probably undreamed of by Irma and Frank Adelman, their usefulness is debated within economics circles — a reflection of a rift, starting in the 1970s, between many research economists in academia and their counterparts in policymaking institutions and businesses.

The Road to FRB/US

Modern econometric models are descendants of work done by researchers at the Cowles Commission (later the Cowles Foundation) at the University of Chicago from 1939 to 1955. (The organization then moved to Yale University, where it has been since.) The Cowles researchers had the benefit of already-existing theories of the business cycle, efforts by Simon Kuznets and others to collect macroeconomic data, and pioneering attempts by Jan Tinbergen to create models of the economies of the United States and his native Netherlands.

From this starting point, the Cowles group established an approach in which they represented the economy as a set of simultaneous equations — that is, equations that had to be solved together, not one by one. Each equation specified how some economic variable (such as aggregate personal consumption) on the left side of the equals sign depended on some other variables, which reflected what economic theory or the researcher's judgment suggested about the determination of that variable. The model could then be estimated using statistical methods. This "esti­mated" model could then, in theory, be used to forecast the path of the economy or analyze policy changes.

Lawrence Klein, who joined the Cowles Commission after finishing graduate school at MIT, continued the Cowles approach to model building at the University of Michigan, Oxford University, and the University of Pennsylvania, eventually receiving a Nobel Prize for his work. Writing in 1950, before the computer age had reached econometrics, he noted that an "annoying problem" in such research was "the laboriousness and complexity of computation" — the problem that Irma and Frank Adelman would address on the night shift later in the decade using a model he had co-created.

At the Fed's Board of Governors, work on an econometric model of the U.S. economy began in 1966 as a collaboration between Fed economists and academics. The resulting model, which was used by Fed staff starting in 1970, was known as "MPS" for the institutions involved (MIT, the University of Pennsylvania, and the Social Science Research Council). The staff started work on a global model in 1975, which led to MCM, for "multi-country model," coming into use in 1979.

As it turned out, the collaboration on MPS in the mid-to-late 1960s would be the high-water mark of joint work between policymakers and academic economists on macroeconomic models. Interest among academics in such projects declined afterward — the result, in large part, of a single article by Robert Lucas of the University of Chicago that did not initially attract much attention. In the article, published in 1976, Lucas presented what is now universally called the "Lucas critique": In simple terms, he argued that Cowles Commission-style large structural models were all but useless in analyzing the future effects of policy changes because they failed to account for people's and firms' expectations, especially the possibility that their expectations would anticipate possible policy changes. In his view, to the extent that economic actors were able to anticipate policy changes, and thus adapt to them, models that could take into account only the prior behavior of individuals and firms would generate "invalid" results.

FRB/US at the FOMC

In reaction to the Lucas critique, as well as various limitations that the Fed encountered in using the MPS and MCM models, Fed economists began work on successors to them in 1991 and 1993, respectively. The resulting models, FRB/US and its international counterpart, FRB/ MCM, replaced the earlier ones in 1996.

FRB/US, which the Fed's Board of Governors released to the public on its website in 2014, added extensive and complex mechanisms for factoring in expectations. When using the model, Fed staff can determine the assumptions they want it to make about how different players in the economy — for example, financial-market participants, nonfinancial firms, and households — form their expectations of the economy and policy and how accurate their expectations are.

Todd Clark, a senior vice president in the Cleveland Fed's research department and head of its macroeconomics group, says that FRB/US "was a product of trying to build in a lot of the things that had been learned about macroeconomics since the old MPS model was put in place."

The results of FRB/US simulations make their way into monetary policymaking at the Fed in several ways. First, they are used directly by Fed economists and Federal Open Market Committee (FOMC) members to analyze the outcomes of possible policies. For example, then-Vice Chair Janet Yellen noted in speeches in 2012 that she had used FRB/US to obtain projections of how long inflation would remain in abeyance if the Fed continued its policy of low interest rates. Second, forecasts from FRB/US are included in the Tealbook, the set of materials that the research staff prepares for the FOMC in advance of committee meetings. Finally, and probably most importantly, FRB/US forecasts are one input into the staff's own forecasts, which are a central part of the Tealbook.

The staff forecasts are "judgmental," meaning the staff makes its own subjective decisions about how much weight to give various pieces of quantitative and nonquantitative information. Christopher Sims of Princeton University reported in a 2002 article that these judgmental forecasts have been "historically slightly better" than the FRB/US forecasts; in interviews he conducted with Board of Governors staff members, they told him that the superiority of the judgmental forecasts came, not from better foresight on the humans' part, but instead from superior knowledge of the current state of the economy. All other things equal, a more accurate starting point means better forecasts.

In assessing the current state of the economy, according to Sims, one area of advantage for the staff over FRB/US and other current computer models — beyond the staff's ability to assimilate unstructured quantitative and nonquantitative information — is a better ability to assess how unusual shocks to the economy are likely to play out. Events that have not been defined within a model, or are outside the statistical experience of the model, such as an oil-price shock, a major terrorist attack, or a large-scale financial crisis, are beyond the model's ken. "Analysis of such historically unusual disturbances — including the determination of whether they really are historically unusual — will inevitably involve an element of subjective judgment," Sims noted.

The Rivals

Outside the Fed, FRB/US has been criticized from a number of directions. For some economists, such as Ray Fair of Yale University, its way of handling expectations disconnected it from the statistical theory underlying the original Cowles Commission-style large models. For others, FRB/US does not go far enough in addressing the issues raised by the Lucas critique.

Two other families of macroeconomic models have swept macroeconomic research in academia, largely because they sidestep Lucas' objections to traditional models. One of these, known as DSGE models, for "dynamic stochastic general equilibrium" models, emerged in the 2000s. DSGE models generally embody a world in which individuals and firms know a lot about the future: While they don't know specifically what will happen, they do know all of the possible shocks to the economy and the chances of each of those shocks actually occurring. Richmond Fed research director Kartik Athreya, in his 2013 book Big Ideas in Macroeconomics, explained, "DSGE, taken literally, just means a model in which decision makers think about the future, where that future is uncertain, and where the outcomes do not surprise people beyond what the realization of uncertainty itself does."

Use of DSGE models within the Fed has been growing. Economists at the Fed's Board of Governors have developed two, known as EDO (a model of the U.S. economy) and SIGMA (a multi-country model). The research departments of several Reserve Banks — the Chicago Fed, the New York Fed, and the Philadelphia Fed — have also developed and used DSGE models.

The answer to the question of whether FRB/US or DSGE models give better forecasts and policy analyses is not yet clear. Economists at the Board of Governors fed economic data from mid-1996 to late 2004 into EDO and found that its forecasts were "as good as, and in many cases better than, that of the forecasts of the Federal Reserve staff and the FRB/US model." But they noted that EDO, having been developed after the period in question, benefited from previous research, including the Board's own research, "on what types of models are likely to explain the data well."

Although DSGE models avoid the limitations of traditional models with regard to expectations, they do have limitations of their own. Current DSGEs assume a "representative" household — that is, they generally assume all households behave identically.

Yale's Ray Fair, a rare academic proponent of traditional large-scale macroeconometric models, contends that the level of knowledge of the future assumed by DSGEs is unrealistic. "That's a highly restrictive assumption," he says. "Sometimes stock markets and bond markets are pretty good, but to say that the average person or the average firm has that kind of sophistication seems highly unrealistic. And it makes a big difference: Properties of the model are very sensitive to whether you generally assume that or not."

Apart from the trade-offs made by builders of DSGEs, Fair argues, the significance of the Lucas critique as a practical matter has itself been overstated. "There's nothing wrong with the logic of it," Fair says of the critique. "The question is how empirically relevant it is. It may be that the things Bob [Lucas] was worried about may be small quantitatively relative to other things."

The other major family of macroeconomic models that has emerged in reaction to Lucas' 1976 article is VARs, or vector auto-regressions, first proposed by Princeton's Sims in 1980. In this approach, the researcher simply makes a list of the variables that he or she believes are relevant to whatever issue is being looked at. Beyond that list, there's no need for economic theory: The researcher doesn't need to specify how the variables are related to one another. Loosely speaking, the variables and some prior values of the variables are all regressed on past values of each other.

Clark of the Cleveland Fed says all three families of models have something to offer. "You see in modern central banking the use of a range of models within the Federal Reserve System," he says. "There's an old quote from a statistician, George Box. ‘All models are wrong, but some are useful.' "

Of DSGE models and models like FRB/US, Clark says, "They are useful for helping us understand fundamental issues with monetary policy and other policies. They're also helpful for telling a story around a forecast and giving us insight into the structural forces that might be driving the outlook."

At the Richmond Fed, a type of VAR known as a time-varying parameter VAR, built by Thomas Lubik and Christian Matthes, is used to forecast the U.S. economy and to analyze policy questions. An advantage of this type of model, Lubik says, is that it can deal with nonlinear behavior in the way some variables influence the economy, such as the effects of interest-rate changes when interest rates are near zero. To work on diagnostic questions about the economy — what caused X to happen? — Richmond Fed researchers use a variety of other models, including a DSGE model.

One of the drawbacks of DSGEs and VARs, according to Lubik, is that they are difficult to analyze and adapt to the needs of the policymakers when they are implemented on a large scale. While they enjoy academic respectability, sometimes the utility of the theoretically imperfect model makes it the better choice. "This has been the tension for the last 10 to 20 years between academics and policymakers," he says.

On the policymakers' side, the theoretical limitations of traditional models, and of hybrids like FRB/US, are well understood. "But at some point, you need answers fast," Lubik says. "FRB/US in general tends to perform quite well for forecasting and policy analysis."

Whether quick and dirty or slow and theoretically clean, computer models are essential to monetary policymaking at the Fed. But when the next major negative shock to the economy occurs, it may well be one that model-makers didn't envision — putting human judgment at a premium over computer chips more than ever.

Readings

Board of Governors of the Federal Reserve System. "FRB/US Model."

Brayton, Flint, Andrew Levin, Ralph Tryon, and John C. Williams. "The Evolution of Macro Models at the Federal Reserve Board." Carnegie-Rochester Conference Series on Public Policy, December 1997, vol. 47, pp. 43-81. (Article available with subscription.)

Fair, Ray C. "Has Macro Progressed?" Journal of Macroeconomics, March 2012, vol. 34, no. 1, pp. 2-10. (Article available with subscription.)

Sims, Christopher A. "The Role of Models and Probabilities in the Monetary Policy Process." Brookings Papers on Economic Activity, 2002, no. 2, pp. 1-62.