Consensus or Individual Forecasts?
Let me start with this: if you haven’t read Nate Silver’s The signal and the noise then I suggest you go pick it up right away. For anybody who is interested in the business of forecasting, Silver’s book is a must read. Silver, a baseball analyst-turned political analyst-turned forecaster extraordinaire, gained notoriety when he correctly predicted the 2008 U.S. presidential election almost flawlessly. In his book, he discusses the intricacies of forecasting and the enormous challenges presented when one attempts to anticipate the future, applying rigorous analysis combined with large amounts of rationality and common sense.
Silver goes over the theoretical and practical aspects of forecasting in a wide range of fields, from metereology or earthquake prediction to baseball and – you guessed it – economics. Be warned: economists don’t come out looking very flattered in this book. In spite of increasingly sophisticated models, for all our efforts, economic forecasting has not improved greatly in the past years. In the book, Jan Hatzius, Chief Economist at Goldman Sachs and a very respected professional in the field puts it bluntly – if perhaps a little dramatically: “nobody has a clue”.
The truth of the matter is that, as Hatzius explains and as any economist worth his salt knows, macroeconomic forecasting is very complicated. Anyone claiming differently is either a fool or a liar. Predicting the behaviour of something as complex and enormous as an entire economy is a daunting task for a number of reasons; some are rather evident like the fact that economies are in perpetual change, therefore one cannot always extrapolate behaviours and relationships from past business cycles into the next one. Some are not. For instance, the sheer size of raw economic data available is something that most would consider an advantage, a sort of commodity that would allow us to test models massively in order to reach increasingly accurate results. Instead, the thousands of economic indicators available produce a vast amount of statistical noise, making the establishment of meaningful relations of causation between variables a serious challenge.
So is this to say there is no hope to obtain success when trying to forecast economic variables? Not exactly. Individual forecasts can and will achieve a certain level of success, but they can be improved upon. How? This is when consensus forecasts kick in.
My friends at FocusEconomics have been publishing consensus forecasts reports for over 15 years, so they have a massive database of economic projections that comes in very handy in order to illustrate the point.
The charts above show forecasts for GDP growth for the four largest economies in the Euro area. These forecasts were collected during March 2012 (and published at the start of April, hence the chart notes), exactly one year before the publication of the final 2012 GDP figures for those economies. This period is an arbitrary choice made by me, but a one-year horizon can generally be considered a reasonable timeframe to evaluate the accuracy of an economic forecast.
The blue dots represent the individual forecasts made by banks, financial institutions and research institutes across the world, while the red dot at the right end represents the consensus forecast for each country. The green line marks the final GDP result published one year later: -1.4% GDP growth in Spain, +0.7% in Germany, 0.0% in France and -2.4% in Italy.
These charts allow us to evaluate the effectiveness of the consensus approach quickly and intuitively. Spain and Germany come up as nothing short of a smashing success, with the consensus forecast accurately predicting GDP growth in both countries. The case of Spain is particularly compelling, as not a single panelist correctly predicted the final outcome, yet the consensus forecast was right on the money.
France shows a more typical scenario, where the consensus forecast is beat by some of the panelists but manages to be more accurate than most. Finally, Italy presents us with a case of a deeper-than-expected downturn, where the final GDP result overshoots panelists expectations, including the consensus forecast (arithmetically, it couldn’t be any other way).
Thus, a quick overlook already shows that the consensus forecast approach appears to be more reliable than a large number of individual projections, with a few exceptions in the overall sample. However, it is when we apply a standard statistical procedure to evaluate the accuracy of those forecasts that the results are extraordinarily revealing. Calculating the root mean square error (RMSE) – a frequently used system to calculate prediction errors in statistics – we obtain that the aggregate forecast is a whopping 98% more accurate than the average individual forecast in the case of Germany and 92% more accurate in the case of Spain. These extraordinary results, however, are the product of the consensus forecast predicting GDP growth with pinpoint accuracy in both those countries, a result that unfortunately is not all that frequent, therefore it was to be expected that calculations would yield an enormous difference against the individual forecasts.
For France, the consensus forecast is 52% more accurate than the average individual one, still an excellent result. However, it is the Italian case that provides possibly the most revealing result of all. As we saw before, Italy suffered a much larger contraction in GDP than anticipated by the market, therefore the consensus forecast fell quite far from the actual outcome and was actually beat by more than half of the panelists. You could therefore think that individual forecasts beat the aggregate one in this case. You would be wrong. When applying RMSE, the consensus forecast still was 7% more accurate than the average individual one. And this proves the central point of this thesis: the consensus forecast will not correctly predict the final outcome each time; the consensus forecast will not beat all individual panelists each time – in fact a certain number of individual forecasts will beat the aggregate one most of the time – BUT the consensus forecast will be more accurate than the average individual projection. Perhaps the consensus forecast won’t be able to predict the next global crisis, but the point is, it will still be able to give you a better projection than your average individual forecast.
Admittedly, my sample is limited to only four countries, one economic variable and one time frame, so those specific error levels cannot be extrapolated to a wide scale (for what it’s worth, I later run the same test for 2012 GDP projections in another 10 Euro area countries and found the consensus forecast to be on average 29% more accurate than the individual ones). Still, it’s a small sample for projections limited to one year, but whether these specific error numbers are significant enough or not, the key issue is that empirical studies show that the phenomenon of aggregate forecasts beating individual ones appears to happen in nearly every field under study.
As I mentioned before, a certain number of individual forecasts will likely beat the consensus any given time, so one may be tempted to try to choose one among those instead in order to find guidance for future results. The problem with this approach is that it has been widely documented that the past accuracy of a forecaster bears almost no weight regarding its future performance. In other words, if you decide to trust someone else’s view based on getting it right once, you’ll do so at your own risk.
Facing an uncertain, unknown future, the best you can do is try to minimize the risk and reduce the margin for error, and that’s where the consensus forecast is a reliable tool. Some of the differences between forecasts and the actual outcome that we saw before may seem trivial to the untrained eye, but, for instance, in a one trillion euro economy like Spain, a 1% error in predicting GDP growth translates into a sizable 10 billon euro difference. Whether you are a policy maker planning the government’s budget or a financial officer attempting to anticipate future demand for a product, wouldn’t you rather be on the right end of that projection?
Ángel Talavera is an economist based in London who specializes in macroeconomic analysis and country risk. Any views or opinions in this entry are solely those of the author and do not necessarily represent those of the company.
5-year economic forecasts on 30+ economic indicators for 127 countries & 33 commodities.
Date: July 16, 2014
TagsChina Commodities MENA Canada Ukraine Economic Growth (GDP) Forex Sub-Saharan Africa Base Metals Commodities Nordic Economies Agricultural Commodities South Africa France Japan Unemployment rate Company News Euro Area Precious Metals Commodities Consensus Forecast USA Russia Banking Sector Spain Venezuela Inflation India Italy Australia Iran OPEC UK Panelists European Union Fed Energy Commodities Colombia United States Germany Infographic IMF Industrial Metals Commodities Eastern Europe Vietnam Mexico Gold Greece Exchange Rate World Bank Trade Africa Major Economies Investment oil prices Financial Sector Brexit Argentina Emerging Markets precious metals Asia Tunisia Housing Market Oil Latin America Brazil Portugal G7 Turkey
Do Chinese authorities now have the space to push on with structural reforms? What’ll this mean for the eonomy? https://t.co/G22UhegEfy
1 hour ago
China continues to pull its weight while India’s outlook cut https://t.co/ZMSCiS0G4A
4 hours ago
5 hours ago
5 hours ago
We trimmed their forecasts for Sub-Saharan Africa for a third consecutive month in July. Find out why:… https://t.co/bpUsVzN5Rr
6 hours ago
- Are uranium prices making a comeback?
- Taxing the Economy: Achieving a Delicate Balance
- Is the UK really "shackled to a corpse"?
- How will Latin America’s upcoming lengthy election cycle affect the reform agenda and credit ratings?
- How will emerging market economies perform in 2017?
- Chilean Economy in Focus: Interview with Senior Economist of the Chamber of Commerce of Santiago
- CEOs Rank Top Economies for Growth Opportunities
- The Mobile Ecosystem & Latin America's Economy
- Prospects and Challenges for the Global Economy: Interview with Tim Cooper from BMI Research
- How will the Fed reduce its balance sheet & and how will the ECB end QE? - 19 economic experts weigh in
- Thoughts on "unwinding" QE from Frances Coppola
- Gold: The Most Precious of Metals (Part 3)
- The Fed and ECB at a crossroads: Unwinding QE
- Spain: The economy that continues to silence the critics
- Latin America: The Most Unequal Region in the World
- The History of OPEC: Has it been a Success?
- FocusEconomics Announces 2017 Analyst Forecast Awards Winners
- Latin America’s rising unemployment bucks nearly decade long trend
- Escape from the Central Bank Trap by Daniel Lacalle
- China's economic rebalancing act: What to look out for in 2017
- Driving Growth in Latin America: Challenges & Priorities
- Is the Global Economy Rebalancing?
- Commodity exporters face challenging times
- Recent Global Events Facilitate Mercosur-Pacific Alliance
- 23 economic experts weigh in: Why is productivity growth so low?
- Mexico's outlook as Trump nears 100-day mark
- Interview with Oxford Economics Senior Economist on implications of the possible outcomes of the French Presidential Election
- The anxiety of the small saver in a world of negative interest rates
- Brexit negotiations. Between Uncertainty and Urgency
- An Economic History of the EU from El Blog Salmón
- Baby Boomin': Implications of high population growth in Latin America
- Survey of International Economists Predicts a Le Pen Defeat in French Elections, Says Macron has Best Economic Plan
- Spain in a global context: developed economy with some challenges
- How much is crime costing Latin America?
- Predictions & Estimates from Economist Daniel Lacalle
- What economy will the new Dutch government inherit?
- “The data is not a true reflection of reality in India” Interview with Société Générale India Economist
- 2017 & 2018 Economic Outlook for the Top Oil Producing Countries
- Which countries will have the highest and lowest inflation in 2017?
- What are the prospects for Emerging Economies in 2017?
- What to expect in Asia for 2017
- Top Economics & Finance Blogs of 2017
- Latam to Resume Moderate Growth in 2017 but Important Risks Plague Outlook
- 4 Key European Elections That Will Impact the Economy in 2017
- How are security concerns and political chaos affecting Turkey’s economy?
- Global growth to edge up in 2017
- Set to breach targets again? Debt and deficit outlooks for Southern European Eurozone countries in 2016 & 2017
- What does Donald Trump mean for the U.S. economy?
- How will emerging markets perform in 2017?
- The economic impact of a break in U.S.-Philippines ties
- Trump election: Base metals surge due to infrastructure plan
- 5 updates on the Venezuelan economic crisis
- Canada: When your neighbor’s house is on fire…
- Short-term pain before long-term gain? A look at French labor reform and economic growth
- Asia: Unremarkable growth & unfulfilled promises?
- How India's latest monsoon is affecting the economy
- Innovation in Latin America: Potential Goes Untapped Due to Weak Economic Conditions
- Russian economy update in wake of OPEC deal announcement
- The Wisdom of the Crowds and the Consensus Forecast
- Can the peso predict the U.S. election results?
- There's no end in sight to the Venezuela crisis
- A Look at the European Union Political Calendar
- Survey of international economists shows uncertainty surrounding elections damaging U.S. growth prospects
- FocusEconomics partners with leading online statistics provider Statista
- China: Recent postive economic data may be papering over the cracks
- Sub-Saharan Africa's 2016 & 2017 growth rates
- The Italian Dilemma: Weak banks pose risk to already faltering domestic demand
- How much money do migrants from Latin America send home?
- The U.S.' (Not So) Mysterious Case of the Missing Men
- What to expect from the G20 economies by 2020
- The Pain in Spain: Robust GDP growth cannot mask the persistent structural deficit