In the second quarter 2018 edition of MHI Solutions, Pat Davison, MHI Director of Standards, weighed in on the correlation between harnessing analytics and accurate prognostication. The article is reprinted in full, here.
It’s spring and the start of the 2018 Major League Baseball season. With the start of the new season comes the pundits’ predictions of who will win the World Series in October. Even though the season hasn’t begun yet, the so-called experts are predicting the Dodgers as five-to-one favorites to win it all in 2018, followed by the Indians and Astros at six-to-one. But how confident can we be in their predictions? How confident are the experts in their predictions? And what information did they use to make their predictions, and would they be able to make better predictions with more data?
More importantly, what does this have to do with Material Handling Logistics Roadmap 2.0?
According to the Roadmap, “Big Data,” is predicted to shape everything from final-mile routing to long-term infrastructure planning to freight forwarding to ultimate decision-making to network optimization, all of which is anticipated to improve forecasting. GE’s Jeff Immelt was reported to postulate, “the marriage of Big Data analysis and Industrial engineering promised a nearly unimaginable range of improvements.”
The promise of Big Data will almost certainly help transform our industry and help bring about new technologies, such as autonomous vehicles. But how much of those “improvements” will actually help to predict future activities? Will more data computation capabilities help us make better forecasts, or simply provide us with more confidence in the forecasts we would have otherwise made with existing data?
For that matter, the Material Handling Logistics Roadmap 2.0 is a prediction of the state of material handling and logistics in the year 2030. How accurate should we think our predictions are, and would additional data have resulted in better predictions?
The question of the use of information among experts was researched by Paul Slovic in the early 1970s and the findings were presented in “Behavioral Problems of Adhering to a Decision Policy.” Slovic’s research involved eight experienced horse handicappers being shown a list of 88 variables found on a typical past performance chart. The handicappers were asked to handicap ten-horse races based only on five of the 88 data points for a series of races. When they did this, the handicappers all used different sets of information to make their selections. Their pick success rate was 17%, which was 70% better than the 10% probability of picking a single winner out of ten horses at random. Moreover, when asked of their confidence in their prediction, they reported a 19% confidence, which pairs almost exactly with the 17% success rate. At this point, the handicappers proved their worth.
Next, the handicappers were asked to select the 10, 20 and 40 most important pieces of information, and were asked to predict race results and well as report their confidence in their prediction, based on the information gleaned from the additional data. The results showed that the handicappers were no better at predicting the results with the additional information, but their confidence in their prediction rose steadily with more information. In fact, with 40 pieces of information, the handicappers’ confidence rose to 34%–nearly double the confidence level as compared to five pieces of information—but their race predictions remained at 17%. Simply put, the inclusion of more information in the experts’ decision-making process only affected their confidence in their predictions, not the predictions themselves, and with more information, their confidence became more and more separated from reality.
How can we correlate these findings to the promise of Big Data made in the Material Handling Logistics Roadmap 2.0? In other words, when does “Big Data” simply become “Data,” can it be used to improve forecasting, and at what point do Mr. Immelt’s promises bear fruit? Since the inception of Moore’s Law in the mid-1960s, our computing abilities have increased by a factor of millions, if not billions. But even with computing power in our pockets that would have been unimaginable 40 or 50 years ago, are we any better at predicting future activities? Why can’t we put one-to-odds on the Dodgers, the Indians, or any other team? Perhaps the handful of tried-and-true data points our industry business leaders use, such as the commodities prices, consumer confidence, and the strength of the dollar are sufficient at helping material handling business leaders forecast at a level that provides an advantage over the status quo while keeping the confidence in their forecasts in check.
Don’t get me wrong—Artificial Intelligence and Big Data are coming (if they’re not here already), and they will be instrumental in transforming the material handling industry by providing increased visibility, granularity, and applications in real time. But when it comes to predicting the future, the promise of more sensors producing more data feeding more algorithms might not yield better results.
The Material Handling Logistics Roadmap 2.0 starts with an admission of things the first Roadmap got wrong, just three years prior. When it comes to baseball, the best predictor of future success might be a bullpen full of strong arms and a lineup of home run hitters. For many of our industries, the price of steel and oil will remain a primary factor in our business leaders’ forecasts. And for those members who rely heavily on the importing or exporting of goods, the relative strength of the dollar to foreign currencies will set the stage for future performance. Additional information might only help us to believe we are making better predictions, instead of actually making better predictions. In the end, maybe that’s just the way it’s supposed to be.