Do judgmental adjustments of statistical forecasts of demand improve accuracy? A large empirical study showed that on average they do, but only when something important has happened, is planned, or is expected that has not been included in the statistical model.
While management adjustments can incorporate useful information to improve forecast accuracy, they are expensive and they may introduce bias. Because the managers' adjustments in the study were made in an unstructured manner, it seems likely that further improvements in the use of managerial judgments are possible.
Demand forecasting is a crucial aspect of the planning process in supply-chain companies. The most common approach to forecasting demand in these companies involves the use of a computerized forecasting system to produce initial forecasts and the subsequent judgmental adjustment of these forecasts by the company's demand planners, ostensibly to take into account exceptional circumstances expected over the planning horizon.
Making these adjustments can involve considerable management effort and time, but do they improve accuracy, and are some types of adjustment more effective than others? To investigate this, we collected data on more than 60,000 forecasts and outcomes from four supply-chain companies. In three of the companies, on average, judgmental adjustments increased accuracy. However, a detailed analysis revealed that, while the relatively larger adjustments tended to lead to greater average improvements in accuracy, the smaller adjustments often damaged accuracy.
In addition, positive adjustments, which involved adjusting the forecast upwards, were much less likely to improve accuracy than negative adjustments. They were also made in the wrong direction more frequently, suggesting a general bias towards optimism. Models were then developed to eradicate such biases. Based on both this statistical analysis and organisational observation, the paper goes on to analyse strategies designed to enhance the effectiveness of judgmental adjustments directly.