< Back to all posts

Overcoming ‘data embarrassment’ in digital transformations

Businesses which decide to tackle digital transformation will be presented with a long list of questions about their existing data. These include; where their data is stored; what sort of data is available; and is that data consistent? Many organisations are unable to answer these questions, leading to a sense of “data embarrassment”, around the perception of the quality and completeness of their data.

Data embarrassment usually stems from an organisation’s history. Often the existing business model is one that has never required the business to think about the collection and organisation of its data. As a result, the idea of spending money on data collection is a completely alien concept.

In fact, until about five to 10 years ago, investing in the rigorous collection of data was unlikely to have been considered a major value proposition for most businesses. Realistically, the only organisations with a long-term understanding of the kind of value data analytics offer their business are those for which the rigorous collection of data has been mandated by law. For example, education, (especially universities), medicine and financial services.

This lack of formal mandating in the general project management space has led to a lack of rigorous data collection, and an abundance of poor quality and very divergent data.

Start small…

A major challenge of data embarrassment is that business leaders tend to respond by trying to rectify their “messy” data using processes which make sense to humans. They take a sweeping approach, to try to overhaul the problem and arrange all their data in a neat and tidy manner. This is often a very large and expensive exercise however it is rarely the best way to address the issue.

Overcoming data embarrassment calls for the smallest possible adjustment to a business’s data structure, which yields the greatest impact. To do that, it is essential to understand how the data will be used by new technologies, rather than thinking about it in terms of traditional processes and a human perspective.

Once this small first step has been taken, organisations do not need to focus too heavily on the rest of their data. As time goes by, it is expected that new technology will become better at dealing with unstructured data. So, a sizeable portion of the data that already exists and may be poorly organised, messy, and incomplete, could actually be useable five years from now.

Machine learning experts – either hired in-house or sourced externally – can design a road map for what is likely the most useful data across an organisation.  Once that is identified, it can be organised first. If a small amount of data can be organised and fed into AI or machine learning processes, that agile approach will yield immediate benefits. This is a much smarter and cheaper option than cleaning up the entire house, as part of a year-long program before an organisation starts to do anything with the data.

The risk of inaction

Doing nothing to tackle data embarrassment and digitise means organisations are potentially jeopardising their future, and at the very least compromising their competitive edge.

Those companies that have begun to report on their data for machine use, based on what machines can do now, will gain a competitive advantage from that process. A small amount of clever investment now, which considers the abilities of machines and artificial intelligence to use data as it stands today, offers an immediate commercial benefit, and mitigates the risk of falling behind if nothing is done.

Organisations can start by thinking about the low-hanging fruit, rather than feeling that they need to get their entire house in order. Incremental approaches to cleaning up data and then applying that data to AI and machine learning is the right head-start to have. Making small changes as early as possible can help businesses take advantage of digitisation.

Insights