Rather than focusing on the elusive ‘data scientist’, organisations do well instead to bring broader skillsets into contextualising Big Data for their projects
We live in a world that is increasingly digital and there are few transactions or interactions in modern life that don’t leave some form of digital footprint. Our activity in business, personal and social transactions is not only changing how we live – it also yields significant amounts of data, which is fuelling the digital transformation of business.
According to a recent US study, The State of Big Data Infrastructure: Benchmarking Global Big Data Users to Drive Future Performance (April 2015, Vanson Bourne), the amount of data organisations have has increased by an average of 16 percent in the last two years and is predicted to rise by a further 24 percent in the next two years. But data in and of itself tells us very little – it is ultimately the context that yields information.
Data and Context Must Work Together
A great way to illustrate the importance of context for data is to think of footprints on a beach. Each footprint tells us that people were on the beach which, in essence, is just data.
To give the data meaning, we need to look at the context of how that data came about, or how the footprints were made. The volume of people on the beach that day and the individual characteristics about those people (height, weight, size of the footprint, how deep they are etc.) are factors that provide context for how those footprints were made.
The illustration shows us that the footprints by themselves only tell us that people were on the beach; however, it’s the contextual information that will help us derive understanding and meaning about the behaviour of people.
To gain information, you need to add context and value to the data. The relationship between Big Data and Digital, it is the platform (i.e. the beach) allows us to gather a wide array of data (i.e. footprints), but it’s the context that gives it meaning.
So what is the lure of Big Data and why do our footprints matter?
The Relationship Between Big Data and Digital
Let’s get a baseline on what Big Data and Digital are.
Big Data is an aggregation of our online history, including how we navigate, shop, search and behave online. It is a rich collection of information about how we interact with organisations, with each other and that provides a deep reservoir of data that has implications, opportunities and – in some cases – dilemmas for the marriage of Big Data and Digital.
Digital, in this context, is the method by which our footprint is recorded. “Digital” is the smartphone, the tablet, the Internet, social media and many other forms of engagement that have become – for many of us – part of the daily norm.
Our behaviour online is trackable and that is a contract many of us either unwittingly make or accept knowing that there are benefits to having a better or more convenient user experience.
The Consumer Perspective
A consumer’s perspective may be ‘where I go is my business’ yet we live in the paradox of wanting anonymity online while at the same time wanting the convenience or perceived benefits of a personalised experience that delivers the information we want or need.
The perceived benefit is a significantly improved user experience, a reduction in time spent filtering non-relevant information and a reduced advertisement-to-sale timescale. Where it gets uncomfortable is the knowledge that our individual footprints become an information asset for the collector of that data; intelligence which can be sold on to third parties that may or may not have our best interests at heart.
For business, it can present a conundrum – how to meet the privacy and personal security expectations of customers while at the same time delivering a better service?
The Business of Big Data
In the study mentioned above, many organisations see Big Data as an important facet to digital transformation and perceive Big Data to be critical for strategic business goals, such as increasing revenue, new market development and improving the customer journey.
For the organisations surveyed, most were already experiencing or anticipating that they could deliver more effective, targeted marketing and sales campaigns, increased revenue and a better understanding of how to engage with their customers.
The study identified five top issues for organisations to overcome to successfully deliver Big Data project implementation:
- Insufficient infrastructure
- Organisational Complexity
- Security, compliance and governance concerns
- Insufficient budgets and resources
- A lack of visibility into information and processes due to absent or limited capability.
The ever-increasing volume and complexity of real-time data sets are proving to be a challenge for business on many levels, however, the study showed that some 84 percent of businesses surveyed saw benefits outweighing the challenges.
One of the challenges of Big Data projects is too often the focus is on the platform for the analysis, machine learning and systematising process, however the missing ingredient is also that it’s about solving problems, discovering patterns within the business (and for its customers) and finding the right solutions for those problems – and upskilling your teams is just as vital.
Data Science – the Missing Link or a Big Data Unicorn?
Despite having plenty of data at its disposal, a business may struggle to extract the valuable nuggets from the mountain of information it has about its customers and operations due to the lack of skills within the team to do so. The platforms exist to collect and silo data, however, there need to be the skill sets in the business to interpret it, distil into insights and deploy actionable strategy.
A recent study from Accenture flagged the ability to understand statistics, machine learning, sound visualisation skills and being able to design experiments to test the quality and integrity of the data as a major challenge for most businesses.
It is not insurmountable however and it’s possible to utilise existing skills augmented with outside expertise. Enlisting both business and IT teams into the project will help provide a balanced skill set to contextualise and analyse data while an outside expertise can fill the skills gap.
Too often, the conversation turns to the shortage of the elusive data scientists, when businesses can still succeed in delivering Big Data projects well by ensuring that the right team is pulled together from within the organisations existing skills sets.
We believe that quality thought leadership is worth sharing and encourage you to share with your colleagues. If you’re interested in republishing our content, here’s what’s okay and what’s not okay.
To speak to our team about how we can help your business deliver better projects, please contact us.