By: Nick Evans, Marketing Practice Director, Jaywing
Purists will tell you that data science is a unique combination of advanced mathematical, analytical and programming skills which enable practitioners to manipulate, manage and analyse huge volumes of data, possibly generated in real time and measuring everything from electricity consumption by the second through to the behavioural complexities of millions of users across a web platform.
The use of the term ‘data science’ or ‘data scientist’ has become over used in recent years. Businesses have re-branded their analytical teams to ride the wave of interest that has characterised the apparently ‘sexy’ role of the data scientist in recent years. But more of that later.
In this article, we explore the biggest breakthroughs that have transformed advanced data analytics – from the rise of Big Data to the introduction of Artificial Intelligence.
Here are the biggest breakthroughs that have firmly placed data science on the map:
- The introduction of Big Data
If in its purest form, data science is the ability to manage and make sense of massive data volumes, then its origin in a business sense lies with the big digital players such as Amazon, Google and Facebook. Given their business models, they had to employ people who could gain control of the vast quantities of data they generate each second, and who could design and develop supporting technical infrastructures in order to extract value and manage their businesses effectively.
The Big Data environments based on Hadoop were developed within companies like Google. There were huge infrastructures of parallel processing capability that could scale easily as data volumes grew and were designed to rapidly process large volumes of data. With this technology came the need to develop new data access, manipulation and analysis tools hence the largely open source capabilities like Spark, Hive and Python.
- Normalisation of digital/online to customers
These huge volumes of data have been generated by consumers, who have transferred their lives online in increasing numbers, making transactions, carrying out research and sharing their experiences every second of every day. Moving away from high-streets, roughly 80% of consumers now shop online and over 39 million (66%) use social media.
- Customer empowerment
The key driver for data science in the last 10 years has been the continued growth of online and digital, and the corresponding growth of marketing channels and devices. These have increased the opportunity for engagement between brands and customers, leading to a kind of empowerment that puts consumers in control of their choices. They decide when and what they want to buy, and when and how they chose to communicate or be communicated with – rather than brands dictating.
These consumers are complex, they interact with businesses across different platforms in different ways and leave digital data clues about their behaviour as they go.
- Brands adapt to meet expectations
These clues provided a new opportunity for brands to better personalise customer experiences and stand out amongst competitors. But it’s one that many businesses still struggle with because of the challenge it presents of how to view an individual’s entire customer journey across online and offline channels.
In such a competitive world, where in the last ten years, the customer has become both more in control, and also more expectant of getting what they want, using these data to understand the customer and communicate responsibly with them has become crucial. This complexity demands expertise in data analysis, data science and insight – something the best analytical teams have prioritised in recent years.
With full sight of in-store visits, social media interactions and email engagements, as well as purchase data, data scientists can build a more accurate view of business performance, where to focus marketing spend and tailor customer journeys.
- Focus on accountability and ROI in marketing
With the power of data and the added responsibility it brings, we are now seeing more data-literate senior marketers sit at an Exec-level, enabling businesses to leverage data as a commercial asset through an intimate understanding of marketing and customers, expertise in profitable revenue and experience in strengthening brands.
One of the ways marketers are influencing at board level is by proving the impact of their campaigns. With the data boom bringing an abundance of channels, attribution has become a complex challenge in recent years. The task of using data and analysis to effectively understand how marketing channels work together through the entire path to purchase, and to identify which ones contribute to the sale and which don’t, is huge. This has been met with a renewed focus on return on investment, something that was a novelty among marketing departments 10 years ago but has now become key to their vocabulary.
- Technological advancements
Until recently, there were a few well-established players that monopolised the data and analysis technology sector, but things have changed. Originating in university research departments across the globe, we’ve seen a rise in excellent open source analytical and programming software such as R, Python, and Spark, which have become the de facto starting point for large scale data analysis/data science projects given their cost (or lack of it). But most importantly, they have been developed specifically to work with these new, large, and in many cases, unstructured data sources that characterise the new environment.
- Resource shortages
One thing that hasn’t changed is resource shortages – data was rarely a ‘cool’ subject, other than for those who could see its potential to be business transformational. It’s cooler now, possibly due to the influence of Silicon Valley and the appearance of Millennials making billions in new technologies and changing the way we view the world.
But, while there has been an increase in the number of universities offering degrees in Data Science, it is still the case that academia, from high school through to university, hasn’t quite caught up with the current, never mind future, demand for these skills.
The now widely quoted McKinsey report of 2011 suggested a huge demand for ‘deep analytical talent’ – between 140,000 and 190,000 by 2020 in the US alone, with 1.5 million more data savvy managers required to take advantage of data. While not all businesses will need extensive teams of pure ‘data scientists’, the rise in prominence of ‘data’ as a value driver has amplified the need for more ‘deep analytical talent’ across all industries and sectors.
A more recent survey by McKinsey confirmed this picture, showing that 48% of businesses found it more difficult to attract talent with analytics skills compared to other talent required by the organisation, while only 13% found it easier.
- New regulations – The General Data Protection
The growth in data, and its use and misuse, has led to high profile issues around intrusion and consumer privacy. The single most important piece of data for many marketers is ‘consent’, which is difficult to retrieve once it’s lost. It needs to be treated respectfully and considerately, but many businesses just can’t resist bombarding customers at every opportunity with a message that might, just might, this time stick. The recent European directive GDPR seeks to enshrine consumer protection in legislation, ensuring businesses seek consent responsibly and make data access easy for consumers. This is backed up with heavy fines for any business that happens to transgress (up to 4% of global turnover or £20m).
The requirement to pay greater respect to consumer privacy requires greater understanding of their needs and motivations, which returns us to better insight, driven by better analysis of better (and larger) data volumes. Moreover, it helps to build trust and rapport with a brand. Those that fail to meet these standards won’t just face the fine, they risk losing their reputation and customers. As such, the need for more and better data science has never been more important.
- The emergence of Artificial Intelligence (AI)?
This leads us on to our biggest development – Artificial Intelligence and machine learning. Marketers appear more hesitant about the immediate benefits of AI compared to other sectors, such as medicine and manufacturing, where it’s been deployed to make highly significant and consistent detections of cancers or engineering defects. Likewise, financial services see benefits in improving the effectiveness of commercial and personal lending decisions where incremental improvements in modelling can add millions to the bottom line. But marketing will benefit. AI opens up the opportunity for more marketers to undertake data analysis and predictive modelling efficiently and effectively without the need for large data science teams.
This will allow more businesses to be more competitive, with the ability to predict conversion, re-purchasing, lapsing, basket abandonment and more. Brands will act on this to tailor communications and deliver individually personalised messages to delight customers and stand out amongst competition.
Furthermore, AI’s abilities around image recognition open up huge opportunities for understanding visual clues left by consumers in their social media posts, or mining sentiment in text to get quickly to the issues of customer service.
Far from replacing jobs with computers, AI is more likely to free up employees to do their jobs better, overlaying the emotional intelligence and nuance that no machine could match, and which is fundamental to good commercial decision making.
How is the future of data science looking?
With the world of data ever-growing and technology advancements showing no signs of slowing down, we are only just at the tip of the iceberg in terms of this role’s value for decades to come.
A cultural shift has already seen growing respect for data scientists, with increasing numbers of senior management asking how to derive value from data strategically. Eventually, data scientists will be part of every brand’s infrastructure as the norm. Through significant cultural changes, the role of a data scientist will be seen as fundamental as more and more brands use sophisticated analytics to compete. Those that don’t will get left behind and be overshowed by competitors who meet customer needs time after time.