Choosing the Right Forecasting Technique
Monday January 1, 2018, by Prasenjit Debnath
Forecasting has become an integral part of our lives. From critical business decisions to decisions about our personal lives, we consciously or subconsciously use forecasting to get the best out of a situation. For the business, though, forecasting is make or break. Forecasting helps businesses see what’s lying ahead of them and align their actions based on that. A simple example is what volume of a product to store. Based on the historical purchase behavior, businesses can foresee the volume of products that will be purchased and stock their products accordingly. That way, customers are not denied a product and no product gets wasted, thereby optimizing revenue.
Forecasting is the technique of using the historical data to predict the future. As simple as it may sound, businesses find this difficult to do. This is in great part because there are way too many forecasting techniques out there and business leaders get confused about which to use. We will look into some of these techniques used across the world and how to choose them for a specific business problem.
Forecasting is broadly divided into two categories: Qualitative and Quantitative.
Qualitative techniques are the ones which apply knowledge of the business, market, product and customer to make a judgment call on the forecast. There are many qualitative techniques used in forecasting. These techniques are primarily based on opinion, like the Delphi Method, Market Research, Panel consensus etc.
The Delphi method is very commonly used in forecasting. A panel of experts is questioned about a situation, and based on their written opinions, analysis is done to come up with a forecast.
The Market Research method is a more systematic and formal way to estimate market sentiment and come up with a forecast based on various hypotheses.
Panel Consensus techniques assume that a group of experts brought together will result in better predictions. Here, there is no moderation and the panelists themselves come to a conclusion with regards to the forecast.
Preferred Time-Period: 0-3 months
Qualitative techniques work best for a short-term forecast. In cases of long-term forecasting, the market research method may give better results as compared to the other techniques.
Qualitative techniques are usually used in the forecast of new product sales. Since the new products don’t have any historical data, these techniques form the basis on which the forecasts are developed. It is also used to forecast sales for a new market.
Most of the methods are based on an elaborate questionnaire that is passed to the experts or survey respondents. Based on the responses and opinions, analysis is done to come up with an optimal forecast.
Cost of Forecasting:
Qualitative forecasting is usually very high as compared to quantitative methods.
Time required to develop such forecasts is also high and can range anywhere from 2-3 months or more.
Quantitative Techniques use the data gathered over time and use statistical techniques to come up with a forecast. There are two types of quantitative techniques – Time Series and Causal.
Time Series Forecasting:
For time series forecasting, the historical data is a set of chronologically ordered raw data points. One way it is different from Causal forecasting is the natural ordering of the data points. One assumption made for a time series forecast is that components like trends, seasonality, cycles etc. will repeat themselves. Line charts are often used to understand time series forecasts. Time series forecasting is used across most business domains like Finance, Sales, Operations etc. Time series can help businesses identify cyclical patterns, trends, growth rates and any irregularity or variation in the series of data.
Some of the commonly used time series forecasting techniques are:
Moving Average (MA): Moving average or simple moving average is the simplest way to forecast by calculating an average of last ‘n’ periods. The average value is considered to be the forecasted value for the next period.
Exponential Smoothing (EA): EA is one of the commonly used techniques where we produce a smoothed time series by assigning variable weights to the observed data point, depending on how old the data is. A special case of Exponential Smoothing is the Box Jenkins method where the model is applied to find the best fit of a time-series model to past values of a time series. EA is suitable for datasets with no trend and having varied levels. Some advancements of EA are Holt’s method and Winter’s method which can be applied for datasets having varying trends.
ARIMA (Autoregressive integrated moving average): ARIMA is a statistical technique that makes use of time series data to predict the future. An ARIMA model has three components: autoregressive, integrated and the moving parts of the dataset. ARIMA essentially auto-correlates its own prior deviations from mean thereby placing importance on the time series part of the data. It takes care of trends, seasonality, cycles, errors and non-stationary aspects of a data set when making forecasts. One important consideration for ARIMA is that the dataset should have at least 36-40 historical data points with minimum outliers.
X11 Forecasting: X11 is a forecasting technique which was adapted from the US Bureau of Census X-11 Seasonal adjustment program. Essentially, the program was used to seasonally adjust monthly or quarterly time series data. What X11 does is, it applies additive or multiplicative adjustments for the seasonality factor in a dataset and creates an output dataset with the adjustments in place. The adjustment of seasonality assumes that the seasonal fluctuations can be measured on the time series and can be differentiated from trend cycles, regular trades, holiday effects and irregular fluctuations. X11 is one of the most complex ways of time series forecasting and it has the ability to integrate the ARIMA model into its existing model.
Forecast Period: Less than a year
Time series forecasting techniques work the best for a short- to medium-term forecast for up to a year.
Time series forecasting is usually used in the forecast of sales, inventory or margin.
For any forecasting where seasonality is present, a minimum of two years of data is required to effectively forecast using time series techniques. In other scenarios, less than two of years of data will suffice. ARIMA works best with a minimum of three years of data whereas for X11 techniques, a minimum of 5 years of data should be available.
Cost of Forecasting:
ARIMA and X11 have higher costs of implementing and model re-training as compared to other time series techniques, as they need multiple iterations to come up with the final forecast. Time series techniques have very low cost as compared to Qualitative techniques.
Time required to develop such forecasts can range from a day to a month depending on the complexity of data.
Causal forecasting is the technique that assumes that the variable to be forecast has a cause-effect relationship with one or more other independent variables. Causal techniques usually take into consideration all possible factors that can impact the dependent variable. Hence, the data required for such forecasting can range from internal sales data to external data like surveys, macroeconomics indicators, product features, social chatter, etc. Usually causal models are continuously revised to make sure the latest information is incorporated into the model.
Some of the most commonly used Causal models are:
Regression Model: Regression is one of the most common techniques used to understand a variable relationship in a dataset. In this method, a function is estimated using the least square technique between the dependent and independent variables which defines the interaction among them. A simple example would be forecasting the margin of a business (dependent variable) based on factors like cost of goods sold, inventory holding etc. (independent variables).
Econometric Model: The econometric modeling technique uses economic variables to forecast future developments. It relies on the interaction between the economic variables and the internal sales data. Some of the economic variables are CPI, Exchange rates, inflation, employment rate etc. Econometric models are a system of interdependent regression equations and it is this nature of the model that gives better results in explaining causalities as compared to ordinary regression.
Leading Indicator Models: The leading indicator technique uses a combination of regression models and willingness to buy survey results to identify causation between movement of two time-series variables. One of the variables here is an economic activity and the other is the dependent variable. A good example of Lead Indicator would be to find if the time series of an economic activity (say CPI) precedes the movement of times series of the dependent variable (say Sales of a company) in the same direction.
Forecast Period: Medium- to Long-term
Most causal forecasting models work best for medium-term forecasing (up to a year).
Causal forecasting can be used to forecast at a granular level. For sales, it can be used to forecast by product, product category, subclass etc. It can also be used for any forecast where there are multiple forces at play which impact the dependent variable
The Regression and Econometric Models usually need at least 2 years of data to work with. The Leading indicator method however needs a combination of sales data for 5 years and willingness to purchase survey data to come up with a meaningful forecast.
Cost of Forecasting:
Regression can be done without burning your pockets. Econometric and Lead Indicator models however use economic activity and survey data which makes it costlier to execute such forecasts.
Time required to implement a regression model can range from a week to a month depending on the nature of data, complexity etc. Econometric and Lead Indicator models take a minimum of 1 month to execute.
The above factors give you a brief snapshot of the nuances involved when considering any forecasting technique. However, analysts need to consider other factors such as Business understanding, Stage of Business (new, growth or steady) and Market understanding to identify the right technique. For example, it’s critical to understand the stage of business as different forecasting techniques get applied at different stages. For a new business where there is a lack of historical data, it’s imperative to use surveys or panel discussions to come up with an estimate, whereas growth and steady state businesses can use a combination of either time series or causal forecasting techniques to come up with an accurate forecast.
There are many other modern day forecasting techniques or variations of the traditional ones which have come up to solve different problems. However, I have tried to highlight those which are most commonly used to carry out any forecasting exercise. Businesses need to be careful in choosing the right technique, and thorough knowledge of the technique is as important as the understanding of the business or the problem at hand. With growing demand for data driven forecasting, businesses should also consider making forecasting an enterprise-level focus. This will ensure that businesses make correct use of forecasting and keep themselves updated on the latest forecasting techniques.