Businesses have access to more data than ever before, as well as the data science to translate that into focused business value.
Paradoxically, all this data can be awfully distracting. Even organisations who aren’t consciously trying to extract and cultivate their business data are suffocating under the sheer weight of it.
According to analysts at IDC, the volume of digital data is doubling every two years. In fact there are about as many ‘bits’ of information in the digital universe as there are stars in the physical universe. Businesses are both creating a large proportion of this, and collecting/retaining it over the long-term to meet compliance requirements and achieve valuable insights through data analysis.
Taming this colossal beast and bringing order to the chaos starts by implementing a coherent data management strategy. Here are five steps to getting there:
1. Set your objectives
Data is so complex that the chances of success from a ‘second-hand’ data management strategy, used at another organisation, are far lower than you might expect. Follow a process that creates a strategy for your organisation’s specific needs. Do this by creating a wish-list of answers you want from data, that chime with your overall corporate strategy. Things like:
- Knowing when it’s cost-optimum to discontinue old products, so to develop a reputation for innovation
- Why certain groups of customers report lower satisfaction levels from the same service
- Identifying opportunities for incremental efficiency improvement in manufacturing and logistics processes
The trick is to prioritise the must-haves from the nice-to-haves by thrashing this out with other key stakeholders.
illumo digital regularly conducts these data management fact-finding sessions with customers, understanding their high-level objectives, examining available data and providing guidance on which data to use to create business intelligence.
2. Map and choose data sources
Knowing where your optimum data comes from is vital, and you need to map it in its entirety before you select the sources to manage. Don’t be surprised if you’ve assumed (or inferred) you have data on a given metric, only to find it doesn’t actually exist. Consider the areas below, or have a data expert do this for you. Only then can you create a leaner, more logical set of data sources.
- Accuracy – how reliable is the data source as a credible basis for business decision making?
- Format – is the source producing information in the form of unstructured data (like images or videos) or structured data (such as spreadsheets and databases)? The latter category is typically easier to manipulate.
- Extractability – what challenges are presented by tapping or exporting data from the source? These might include impenetrable software protocols, legal restrictions on the use of data, or the fact that some data is produced by disparate systems.
- Duplication – do data sources produce the same data, leaving you with multiple crossovers? If so, which is the best source/s, and is it feasible to ‘turn off’ any you don’t need?
- Cost – do you pay third parties for any data sources, and is this necessary? What are the hidden operational costs associated with your own data recording and processing activities?
- Value – how ‘rich’ is the data? In other words, how detailed, contemporaneous, mission-critical, or just plain ‘hard to get hold of’ from anywhere else?
3. Manipulate data into a common model
Adopting a Common Data Model (CDM) across your chosen data sources promotes compatibility and therefore easier analysis. As well as delivering coherence out of an otherwise disjointed and fragmentary picture, the composition of the CDM also takes its cue from the objective-setting in step 1, giving these the best possible chance of success.
Needless to say, manipulating data sources and models is a technical business. We use tools such as SQL Server Integration Services, to perform data Extraction, Transformation and Loading (ETL), which is essentially the mechanism for combining multiple sources into a single data source. This source may then be used as the references source for information within the whole organisation, often referred to as the Master Data Model, or MDM.
4. Enrich data
By this point, you’ve invested a lot of effort in a data management framework. But what if some of the data isn’t quite working to its full potential in its present form? This is where data enrichment comes in.
In its simplest form, data enrichment can be employed to increase the accuracy of data records via algorithms and database lookups. Spelling and keying errors from the data capture process can be caught and corrected in this way, for example.
At the other end of the spectrum are data enrichment activities concerned with applying assumptions and logic to data extrapolation. This discipline is called ‘Hypothesis Generation’. Rather than look at lots of data and try to make sense of it, hypothesis generation sets out to solve highly focused problems with the data variables that impact it. This significantly enriches the value of poorly regarded raw data.
5. Implement a secure, compliant data governance framework
A persistent challenge in the battle to combat data chaos is the requirement to demonstrate visibility and governance over information and data processes. Any inability to address this increases the risks of compliance failure and cyber breach.
Effective corporate data governance dictates that corporate data architecture must be structured in such a way that it is easy to comply with standards and regulations – as well as good cybersecurity practice. Compliance requirements differ according to the organisation in question, but achieving these aims broadly encompasses:
- Where data is stored and who can access it
- Which data is sensitive and to what degree
- Which data is identifiable to an individual
- What data sources and other gateways for data entry exist, both human and machine
- And numerous other considerations
Organisations increasingly recognise the need to be data-driven; making decisions based upon logical, actionable business insight. The way to avoid ‘data’ getting in the way of ‘intelligence’ is to pursue a data management strategy that focuses resources on the data that matters, makes it fit for purpose, and puts it to work for the goals of your business.
