Consensus or Individual Forecasts?
Let me start with this: if you haven’t read Nate Silver’s The signal and the noise then I suggest you go pick it up right away. For anybody who is interested in the business of forecasting, Silver’s book is a must read. Silver, a baseball analyst-turned political analyst-turned forecaster extraordinaire, gained notoriety when he correctly predicted the 2008 U.S. presidential election almost flawlessly. In his book, he discusses the intricacies of forecasting and the enormous challenges presented when one attempts to anticipate the future, applying rigorous analysis combined with large amounts of rationality and common sense.
Silver goes over the theoretical and practical aspects of forecasting in a wide range of fields, from metereology or earthquake prediction to baseball and – you guessed it – economics. Be warned: economists don’t come out looking very flattered in this book. In spite of increasingly sophisticated models, for all our efforts, economic forecasting has not improved greatly in the past years. In the book, Jan Hatzius, Chief Economist at Goldman Sachs and a very respected professional in the field puts it bluntly – if perhaps a little dramatically: “nobody has a clue”.
The truth of the matter is that, as Hatzius explains and as any economist worth his salt knows, macroeconomic forecasting is very complicated. Anyone claiming differently is either a fool or a liar. Predicting the behaviour of something as complex and enormous as an entire economy is a daunting task for a number of reasons; some are rather evident like the fact that economies are in perpetual change, therefore one cannot always extrapolate behaviours and relationships from past business cycles into the next one. Some are not. For instance, the sheer size of raw economic data available is something that most would consider an advantage, a sort of commodity that would allow us to test models massively in order to reach increasingly accurate results. Instead, the thousands of economic indicators available produce a vast amount of statistical noise, making the establishment of meaningful relations of causation between variables a serious challenge.
So is this to say there is no hope to obtain success when trying to forecast economic variables? Not exactly. Individual forecasts can and will achieve a certain level of success, but they can be improved upon. How? This is when consensus forecasts kick in.
My friends at FocusEconomics have been publishing consensus forecasts reports for over 15 years, so they have a massive database of economic projections that comes in very handy in order to illustrate the point.
The charts above show forecasts for GDP growth for the four largest economies in the Euro area. These forecasts were collected during March 2012 (and published at the start of April, hence the chart notes), exactly one year before the publication of the final 2012 GDP figures for those economies. This period is an arbitrary choice made by me, but a one-year horizon can generally be considered a reasonable timeframe to evaluate the accuracy of an economic forecast.
The blue dots represent the individual forecasts made by banks, financial institutions and research institutes across the world, while the red dot at the right end represents the consensus forecast for each country. The green line marks the final GDP result published one year later: -1.4% GDP growth in Spain, +0.7% in Germany, 0.0% in France and -2.4% in Italy.
These charts allow us to evaluate the effectiveness of the consensus approach quickly and intuitively. Spain and Germany come up as nothing short of a smashing success, with the consensus forecast accurately predicting GDP growth in both countries. The case of Spain is particularly compelling, as not a single panelist correctly predicted the final outcome, yet the consensus forecast was right on the money.
France shows a more typical scenario, where the consensus forecast is beat by some of the panelists but manages to be more accurate than most. Finally, Italy presents us with a case of a deeper-than-expected downturn, where the final GDP result overshoots panelists expectations, including the consensus forecast (arithmetically, it couldn’t be any other way).
Thus, a quick overlook already shows that the consensus forecast approach appears to be more reliable than a large number of individual projections, with a few exceptions in the overall sample. However, it is when we apply a standard statistical procedure to evaluate the accuracy of those forecasts that the results are extraordinarily revealing. Calculating the root mean square error (RMSE) – a frequently used system to calculate prediction errors in statistics – we obtain that the aggregate forecast is a whopping 98% more accurate than the average individual forecast in the case of Germany and 92% more accurate in the case of Spain. These extraordinary results, however, are the product of the consensus forecast predicting GDP growth with pinpoint accuracy in both those countries, a result that unfortunately is not all that frequent, therefore it was to be expected that calculations would yield an enormous difference against the individual forecasts.
For France, the consensus forecast is 52% more accurate than the average individual one, still an excellent result. However, it is the Italian case that provides possibly the most revealing result of all. As we saw before, Italy suffered a much larger contraction in GDP than anticipated by the market, therefore the consensus forecast fell quite far from the actual outcome and was actually beat by more than half of the panelists. You could therefore think that individual forecasts beat the aggregate one in this case. You would be wrong. When applying RMSE, the consensus forecast still was 7% more accurate than the average individual one. And this proves the central point of this thesis: the consensus forecast will not correctly predict the final outcome each time; the consensus forecast will not beat all individual panelists each time – in fact a certain number of individual forecasts will beat the aggregate one most of the time – BUT the consensus forecast will be more accurate than the average individual projection. Perhaps the consensus forecast won’t be able to predict the next global crisis, but the point is, it will still be able to give you a better projection than your average individual forecast.
Admittedly, my sample is limited to only four countries, one economic variable and one time frame, so those specific error levels cannot be extrapolated to a wide scale (for what it’s worth, I later run the same test for 2012 GDP projections in another 10 Euro area countries and found the consensus forecast to be on average 29% more accurate than the individual ones). Still, it’s a small sample for projections limited to one year, but whether these specific error numbers are significant enough or not, the key issue is that empirical studies show that the phenomenon of aggregate forecasts beating individual ones appears to happen in nearly every field under study.
As I mentioned before, a certain number of individual forecasts will likely beat the consensus any given time, so one may be tempted to try to choose one among those instead in order to find guidance for future results. The problem with this approach is that it has been widely documented that the past accuracy of a forecaster bears almost no weight regarding its future performance. In other words, if you decide to trust someone else’s view based on getting it right once, you’ll do so at your own risk.
Facing an uncertain, unknown future, the best you can do is try to minimize the risk and reduce the margin for error, and that’s where the consensus forecast is a reliable tool. Some of the differences between forecasts and the actual outcome that we saw before may seem trivial to the untrained eye, but, for instance, in a one trillion euro economy like Spain, a 1% error in predicting GDP growth translates into a sizable 10 billon euro difference. Whether you are a policy maker planning the government’s budget or a financial officer attempting to anticipate future demand for a product, wouldn’t you rather be on the right end of that projection?
Ángel Talavera is an economist based in London who specializes in macroeconomic analysis and country risk. Any views or opinions in this entry are solely those of the author and do not necessarily represent those of the company.
5-year economic forecasts on 30+ economic indicators for 127 countries & 33 commodities.
Date: July 16, 2014
TagsVenezuela Eastern Europe World Bank Africa Canada South Africa Brexit Argentina Asia Precious Metals Commodities Company News India Exchange Rate Fed Iran G7 Germany Gold Inflation Russia Sub-Saharan Africa Industrial Metals Commodities Infographic Consensus Forecast Oil Australia precious metals Brazil Mexico Euro Area Economic Growth (GDP) China Major Economies Investment UK Portugal United States Energy Commodities European Union Colombia Agricultural Commodities Latin America Trade Emerging Markets Ukraine Nordic Economies Greece Panelists OPEC Japan Financial Sector oil prices Spain Tunisia Turkey Commodities MENA Vietnam Italy Base Metals Commodities Forex USA IMF Housing Market Banking Sector Unemployment rate France
12 minutes ago
12 minutes ago
34 minutes ago
18 hours ago
18 hours ago
- Spain in a global context: developed economy with some challenges
- How much is crime costing Latin America?
- Predictions & Estimates from Economist Daniel Lacalle
- What economy will the new Dutch government inherit?
- “The data is not a true reflection of reality in India” Interview with Société Générale India Economist
- 2017 & 2018 Economic Outlook for the Top Oil Producing Countries
- Which countries will have the highest and lowest inflation in 2017?
- What are the prospects for Emerging Economies in 2017?
- What to expect in Asia for 2017
- Top Economics & Finance Blogs of 2017
- Latam to Resume Moderate Growth in 2017 but Important Risks Plague Outlook
- 4 Key European Elections That Will Impact the Economy in 2017
- How are security concerns and political chaos affecting Turkey’s economy?
- Global growth to edge up in 2017
- Set to breach targets again? Debt and deficit outlooks for Southern European Eurozone countries in 2016 & 2017
- What does Donald Trump mean for the U.S. economy?
- How will emerging markets perform in 2017?
- The economic impact of a break in U.S.-Philippines ties
- Trump election: Base metals surge due to infrastructure plan
- 5 updates on the Venezuelan economic crisis
- Canada: When your neighbor’s house is on fire…
- Short-term pain before long-term gain? A look at French labor reform and economic growth
- Asia: Unremarkable growth & unfulfilled promises?
- How India's latest monsoon is affecting the economy
- Russian economy update in wake of OPEC deal announcement
- Innovation in Latin America: Potential Goes Untapped Due to Weak Economic Conditions
- The Wisdom of the Crowds and the Consensus Forecast
- There's no end in sight to the Venezuela crisis
- Can the peso predict the U.S. election results?
- A Look at the European Union Political Calendar
- Survey of international economists shows uncertainty surrounding elections damaging U.S. growth prospects
- FocusEconomics partners with leading online statistics provider Statista
- China: Recent postive economic data may be papering over the cracks
- Sub-Saharan Africa's 2016 & 2017 growth rates
- The Italian Dilemma: Weak banks pose risk to already faltering domestic demand
- How much money do migrants from Latin America send home?
- The U.S.' (Not So) Mysterious Case of the Missing Men
- What to expect from the G20 economies by 2020
- The Pain in Spain: Robust GDP growth cannot mask the persistent structural deficit