Home The C-SuiteChief Compliance Officer (CCO) Can Excessive Reliance on Statistics Undermine Managerial Decision-making and Policy Formation?

Can Excessive Reliance on Statistics Undermine Managerial Decision-making and Policy Formation?

by internationaldirector

By: David Winter, Columnist, International Director

Albert Einstein once said in a famous quote: “If the facts don’t fit the theory, change the facts”. One of the reasons that this quote is sensible even today is that often facts fall short of the big picture. What’s worse, those facts can often be manipulated and misrepresented, which severely undermines their authenticity. Our superior computational power can help us process terabytes of data (facts) in a considerably short time. However, the reliability of such data remains an issue to be addressed. Statistics are the cornerstone of institutional and organizational work; but when those statistics are inadequate or inaccurate, the results can often be impaired decision-making and poor policy formation—and disastrous consequences.

The reliability of statistics is not the only issue that needs to be addressed to improve decision-making. The other issue is related to whether facts (or data) are all we need to make good decisions. If that were the case, then machines could make decisions for us, and humans would be redundant.

Reliability of data and findings

Statistics are exactly what they are called: statistics. They are raw data that is intended to support us in exploring the world and devising proper responses to different phenomena. But while statistics are effective in telling us about what something actually is, they are less effective in telling us why it is the way it is. Relationships are harder to pinpoint than raw data. For that reason, the academic world is now dealing with the issue of false science.

The problem of irreproducible research

There are many ways to verify your statistics and conclusions. 

One way is to do a retest and see if you get the same results. Another is done by using two different instruments to measure the same thing (skill, knowledge or any other variable). Yet researchers and scholars are still struggling with the problem of irreproducible research (which means research that does not yield the same results when repeated). Such research has no significance, wastes time and money, and it is accruing at alarming rates. For example, pharma and tech companies can only reproduce between 11 and 25 percent of high-impact research papers in the field of cancer research. In other words, at least 75 percent of the progress we think we have made in that area has not been actually made.

This shows a serious deficiency in the way research is conductedand indicates that pinpointing the rootcauses of problems in different fields by using statistical evidence is not always a flawless process.

Statistics in policymaking

In policymaking, statistics are essential tools, not only for policy formation but also for measurement of results and accountability. But the flaws begin to appear when there is a strong focus on one indicator and neglect for others. For example, gross domestic product (GDP) is the main indicator used for measuring economic growth and conducting comparisons. It is also used by politicians to commend their own achievements. Yet, when scrutinized, this indicator falls short in many areas. It often fails to take into account inflationary pressures, considers government spending as a component of it (without considering how efficient this spending is or the extent of corruption in the government) and also does not say anything about the level of social equality in the country. Those shortcomings alone can render the GDP measure inferior to other measures, such as the Human Development Index or Equality Index. Yet, it is still the number one measure used by almost all politicians and central bankers when forming their policies.

Human judgement and initiative can often trump data.

We live in an age in which we excessively rely on data. Data is certainly needed for research and academic purposes. Textbooks are written and taught to students after deriving their contents from data-driven research. MBA (Master of Business Administration) graduates often have extensive knowledge about all types of models and frameworks but little real-world experience. On the other hand, many successful businessmen do not even have graduate degrees.

Entrepreneurs such as Spotify’s co-founder Daniel Ek, who left his prestigious school—Royal Institute of Technology in Sweden—after only eight weeks, and Mark Zuckerberg, who dropped out of Harvard to work fulltime on Facebook, prove that academic education, which is largely founded on data, is not all that you need (that is, if you need it at all). Henry Ford, Bill Gates and Steve Jobs were all forces of change, and none of them had a collegeeducation. Each of them enabled a revolutionary change in society and/or their industries.

A strong sense of initiative, intuition and real-world experience often trump MBA knowledge. Those leaders managed to build their large businesses worth billions of dollars not with the use of data or statistics, but mostly with their passion.

Statistics overly focus on macro phenomena.

Statistics can help us see trends and gauge the size of different phenomena. They help us count the number of people with a certain disease, for example, but give us little or indecisive information about the causes. They help us measure unemployment to devise social-security and benefits programs, but do not tell us about individual cases. Statistics look at things from a macro leveland often fail to study the micro cases in depth. The solutions that statistics recommend are also macro solutions and are mostly based on broad generalizations that can and often do lead to misguided stereotyping. Statistics, by their very nature, can help devise effective solutions for most but not all cases for one key reason: they are not decisive about the causes of different phenomena.

Final thoughts

Statistics are crucial, but they are not the Holy Grail for our problems. They enlighten us about how things are in reality but present us with a set of challenges when we want to examine the causes in depth. Owing to that, they are not enough on their own for problem-solving, and they are not a substitute for human judgement (although they complement it). For this reason, the potential of artificial intelligence (AI) to replace human workers is limited to low-skill jobs. Numbers do matter, but eventually our efforts should be focused on benefiting humanity, not bragging about numbers. It is worth remembering here another Einstein quote: “Not everything that counts can be counted, and not everything that is countable counts”.


Related Articles

Leave a Comment