Originally published November 12, 2015
Business is the process of managing under conditions of uncertainty. To ensure optimal outcome business leaders must base their decisions on information of varying reliability. This is not new, humans always had a need for making decisions with uncertain outcomes. “Is he a friend or foe?” “Should I trade my food for a hammer?” “Can I reach safety if I turn and run, or should I fight the lion?”
The common sense paradox
Such decisions rely on instincts and common sense. However, we have gradually moved away from a society where decision makers could act upon limited information or gut feeling to an information society driven by data where common sense is often not applicable. The problem is that while society has evolved, we have not. The corporations of today are governed by people whose brains are better suited for hunting mammoths than taking financial decisions.
Luckily, computer algorithms are already better than humans at diagnosing cancer, predicting supplier reliability, general election results and many other areas abundant with data. In other words, if you are still basing decisions on experience, area expertise and common sense, you are probably a worse decision maker than you think.
Can data be trusted?
Realizing that human judgement in a business context is poor, organisations are increasingly basing decisions on data driven facts. Meanwhile, a recent study by KPMG found that 58% of organisations have difficulties evaluating the quality of the data and its reliability. Once people start questioning the validity of the data, they are much less likely to have an appetite to take on more data driven projects in the future.
Unless there is a way to determine the trustworthiness of the information extracted from the data, the information cannot be transformed into insights. Business leaders are overwhelmed with information, yet only 19% are “very satisfied” with the insights their D&A tools provide. Insights with poor uncertainty must be filtered out to allow decision makers to focus on insights which really matter.
The good news is that we have scientifically proven methods to evaluate the uncertainties of the answers data & analytics provide. Uncertainties can be separated into two categories: statistical uncertainties and systematic uncertainties.
- Statistical uncertainty originates from the limited size of the data sample. “How many units will I sell tomorrow?” If you have only yesterdays sales figures you can make a rough estimate, but with years of data you can estimate an average with a much better precision, and you know by how much your sales vary from the average day-to-day. To improve the precision you need more data of the same type you already have.
- Systematic uncertainty comes from bias in your method or the input data. Are you counting units transiting through your store room? If you do not account for discarding of unsold units you systematically overestimate your sales. To improve the accuracy either (a) improve your data quality or (b) get more data of a different type. In the example above daily cash flow could do the trick.
From insights to value
While a proper uncertainty estimation allows information to be refined into insights, and thereby significantly reducing the information overload of business leaders, it is not by itself enough to deliver value to the organization. Value is generated when insights are put into action. Which insights are actionable depends on expectations. For example, if your sales are down by a few percent, it can either be
- Within statistical uncertainty – it is just random, tomorrow could be different by chance
- Within systematic uncertainty – cannot draw firm conclusion due to poor data quality or other issues
- Outside the uncertainty range – sales really are going down, decision makers should be prompted for corrective action
It is tempting to say that only the last outcome is an actionable insight, but it depends on what business question the analysis is trying to answer. For example, if predictions and actual outcomes are within uncertainty, your financial forecasting is working as indented.
A business issue
When uncertainties are underestimated very bad things can happen. As uncertainty increases, so does risk. A wrong D&A answer can have disastrous long term effects. Could a better understanding of the uncertainty of borrower’s ability to pay back mortgages have avoided the 2008 financial crisis? Also overestimating uncertainties are bad for business, since conclusions are not drawn when they should resulting in that decisions are not taken when action is most needed.
Uncertainty is not an academic issue. It is an important business issue. Next time, ask how well you know the numbers.
