What Covid-19 has taught me about Analytics
Last night South Africa's president, Mr Cyril Ramaphosa, made some announcements on the opening up of the economy after a month long lockdown. The government decided to take a risk based approach, which according to Ramaphosa, will be guided by science and data.
This made me think about how data and analytics have become entrenched in even the most information-averse households. People are talking about infection rates, deaths per million population and some even quote statistics from data aggregation websites such as Worldometer. Although you cannot dispute the contribution data and analytics have made to fighting this disease, I couldn't help but reflect on some of the areas we as Analytics Practitioners still need focus on.
Integration of data
It is very clear that integration of data from several sources (health care facilities, medical labs etc.) is a stumbling block for Covid-19 decision-makers. Governments and other organisations have been in a race against time to get adequate data from these sources, and integrating that in a central repository seems to be quite a challenge.
Talk to any analytics person, and they will confirm this to be the elephant in the room. We have not yet been able to adequately sort out integration across systems, and as such getting a big picture view is still a struggle. Although we very often seem content with the status quo, a situation that requires regular information flow will highlight the inadequacies we may have with data integration. It is therefore imperative that business critical data be prioritised from an integration point of view. If not, fast decision making in a critical time will be compromised.
AI still stifled by data dictatorship
Artificial Intelligence is quite mature in certain industries such as finance and insurance, however there has been very slow adoption in others. This pandemic has highlighted the lack of AI within the health industry, and that should be a red flag for the rest of us as well.
One of the core principals of AI development is adequate and relevant data sets, and as such data democratisation is crucial. Obviously data privacy played a massive role in the lack of data democratisation for the current crisis, however very few valid reasons exist for the rest of us to still have a silo approach to data. The value that can be unlocked through data democratisation far outweighs the need to keep certain information within functional departments confidential from those outside the said department. This is fundamentally a shift in the thinking of data and its purpose within an organisation, and provides the basis for data being a shared asset to all. No manager should have complete control over the data in their department and keep data scientists from exploring and mining it for valuable insights.
Once this stumbling block has been cleared, data scientists and analytics practitioners can focus on using data for the development of advanced analytics models that will add value to the organisation as a whole, therefore removing local optimisation.
Cloud migration needs to be fast tracked
The cloud has instilled fear in some decision makers, mostly due to a lack of understanding. The major reason for this fear is that data privacy may be compromised, which in all fairness is valid. That being said, restriction on physical movement has highlighted the inefficiency of on-premise solutions to a modern workforce.
A large portion of the world's working population was forced into remote work over the last few weeks, and those with little cloud based infrastructure were left with frustrations beyond believe. As the benefit realisation of remote work is taking shape, decision makers will have to invest in cloud migration in order to provide a more collaborative working environment from remote locations. This is especially true for those working with data sets pushing stock standard hardware to the limit.
The Noah-rule will be key to success with analytics
My boss recently introduced me to the Noah-rule which is attributed to Warren Buffet. The rule states:
“Predicting rain doesn’t count, building Arks does”
The post-Covid world will be one with very little cash to throw around, and as such the sexy programmes will be shut down. Fortunately for many years, we in the analytics world have been riding the wave associated with our profession. Lots of investment has come our way, and much has been written about analytics being like the proverbial holy grail to business success. A lot of the hype holds true, but too many analytics projects just never add tangible value, or simply remain in POC mode ad infinitum.
We therefore need to push beyond predictions, and really focus on getting use cases across the line. But even more important, we will have to work with stakeholders to ensure the value of analytics is unlocked in business. That typically means a much closer working relationship with execution functions within a business to ensure that business processes, planning, budgeting and operations are considered along with any fancy data science model. If not, analytics will slowly morph back into a functional capability with a slow death.
Data literacy is a bigger problem than we think
Many reputable media sources have made claims about infection rates, and whether or not it is increasing or decreasing. I have also observed quite a few people commenting on forums such as LinkedIn about the success of lock-downs. One thing that became very clear to me is that there is still a massive data literacy problem. The data we are confronted with on a daily basis is not a reflection of infection rates, but rather of confirmed infections. You have to consider the number of tests conducted in combination with the infection rates in order to make any conclusions. The same logic applies when comparing one country to another. Clearly absolute numbers are not always telling the story, case in point being Belgium who has recorded the largest number of Covid deaths per million population.
Not only do we as data practitioners need to be more critical of the data we collect for analyses, but also spend a lot more effort in educating consumers of the data on its limitations. We also need to be more informed about data bias, especially when it comes to developing models that rely on it. Organisations that truly want to become insights driven will have to invest in education programmes for the decision-makers, more specifically on simple data and statistical foundations. This will not only empower them to ask the right questions, but also ensure the right insights are derived from analytical outputs.