Businesses Need to Have the Right Balance of Soft Data and Hard Data in Their Forecasting Models

By Andrew Duguay, Chief Economist

Any well-balanced economic forecasting program incorporates both hard and soft data. Although an unprecedented economic shutdown has brought soft data to the forefront, hard data remains a bedrock of forecasting. For companies to build accurate forecasts, it’s essential to understand how and when each category of data should be weighted within their financial models. In a COVID environment, soft data has proven to be particularly useful in gaining insight into the economy’s current state.

It’s important to understand how we define “hard” and “soft” data:

Hard data – In terms of economic indicators, hard data is made up of concrete results within a specific area of the economy that shows an output. Examples can include the unemployment rate, monthly retail sales, etc. By nature, these data sets are retrospective as they show real results over a period of time. 

Soft data – On the other hand, soft data sets are developed based on sentiments, such as the consumer confidence index or industry surveys. These data sets are future-focused, but they can often be considered less reliable when the sentiment doesn’t match the group’s eventual actions. 

Hard data is typically more reliable during times of economic stability, since it provides real results, and is still essential for medium- and long-term forecasting. But it paints an incomplete picture of the current economic environment since it relies on historical data, which was significantly interrupted by the shutdown. 

As an example, consider the airline industry. Ticket sales here constitute hard data, while the psychological willingness to travel indicates soft data. The former is not as relevant for short-term forecasting because the industry, like many others, has been frozen and provides fewer insights into when consumers might start flying again, at what rates, and on what timeline. Ticket purchases require long periods of time to be able to tease out trends and extrapolate projections. A recent gap in data means that consumer activity will need to build up for the next few months to identify large consumer data set patterns.

However, soft data is especially relevant now because it gives insight into consumer intent, which is useful when sales are too low to produce significant enough data sets to be mined for forecasting. Hard data will remain instrumental for forecasting, especially in the 4th quarter, 2021 and beyond. Still, it would behoove airliners to pay special attention to relevant search engine queries and website searches, even if they haven’t yet translated to new sales, to build short-term forecasting models.

Indicators that have long correlated have recently exhibited discrepancies. Consider the disparity between the Automatic Data Processing Total Nonfarm Private Payroll Employment figure and the U.S. Bureau of Labor Statistics (BLS) jobs. Usually, the former is a good indicator of what the latter will be, but the ADP May report indicated 2 million in job losses, a sharp swing away from the 2.5 million gained that the BLS reported. This wasn’t the only discrepancy: In April, the gap between the BLS number and the ADP number was 2.2 percentage points, and in March, the difference was 2.24 percentage points. In other words, the discrepancies started when the pandemic began in earnest in the United States. 

That’s not to say that hard data is not reliable in these current circumstances. It just means that when an economy shuts down in an unprecedented event like this one, some indicators simply don’t have the conditions to behave “normally.” Under stable conditions, including more predictable, depressed circumstances, hard data is critical to building forecasts. It remains vital, but, due to the inconsistent nature of this economy, soft data should be increasingly incorporated into your data mix when planning and forecasting over the next few months. 

About Andrew Duguay
Mr. Duguay is a Chief Economist for Prevedere, a predictive analytics company that helps provides business leaders a real-time insight into their company’s future performance. Prior to his role at Prevedere, Andrew was a Senior Economist at ITR Economics. Andrew’s commentary and expertise have been featured in NPR, Reuters, and other publications. Andrew has an MBA and a degree in Economics. He has received a Certificate in Professional Forecasting from the Institute for Business Forecasting and Certificates in Economic Measurement, Applied Econometrics, and Time-Series Analysis and Forecasting from the National Association for Business Economics.