This post follows a recent Red Olive round table, at which technology leaders from the insurance sector discussed data governance. Here, we examine their thoughts about the unique challenges of hosting and using data in the cloud.
Most business processes are built on the back of data. That means many could be improved if they were hosted in the cloud. With almost limitless resources, short contracts and logical, quantifiable billing, cloud offers an effective – and cost effective – way for organisations to take their processing to the next level.
It also introduces new challenges that highlight the need for a comprehensive and flexible data governance policy.
Cloud potential
Many organisations already grade the quality of their data on premise, often using machine learning and artificial intelligence. This is just one area in which the scalability of cloud can deliver more effective results quicker, effectively cutting processing times and impacting revenue.
The resources of cloud-based machine learning can be used to identify patterns and outliers within massive data sets, even if they’re more subtle than a human could detect manually. Doing so gives the data more nuance and context and enables those who rely on it to have greater confidence in the decision making it underpins. This is particularly important in insurance. With sufficient data, an insurer or underwriter can more accurately gauge a level of risk when setting premiums for individual customers. Doing so allows them to deliver a bespoke quote, while simultaneously minimising their own exposure, rather than trying to fit a diverse customer base into a series of less specific boxes.
Similarly, automated processing of massive data sets has the potential to detect and combat claims fraud, forecast fraudulent claims before they are made, and to reduce human involvement in claims processing. Reducing losses this way means the insurer can pass on savings to the customer, with the potential to attract new business and reduce churn.
Transition, heritage, and migration
For this reason, among others, any organisation that handles data, particularly where it relates to individuals, has a responsibility to keep it safe. This responsibility is heightened when its processing or storage takes place in the cloud, since, in some cloud architectures, it’s all too easy to leave a database or API open, exposing the data to the world at large. This could result in the data being compromised, the processes on which it relies being questioned, and even the business itself facing legal action as a result.
An effective data governance policy helps avoid such issues, starting at the point where the data is first migrated off site.
It’s essential that any data object’s lineage is fully understood, and that it’s preserved through the transition – especially if the process has elements of ELT or ETL. Years ago, it would have been necessary to manually deconstruct the data prior to any migration, and ensure it was in a suitable state for hosting on the remote system. That’s no longer the case. Now, data can be carved into domains, and we can look at the source systems themselves.
So long as there’s an effective data governance policy in place, administrators should be able to derive everything they need to know at this level, and the metrics that quantify the data should follow it to its new home.
Once it’s there, additional elements of a governance policy will apply.
Cloud and overreach
Cloud enables mesh architectures and data fabrics, which could only have been replicated on premise with an enormous and expensive appliance. These would be out of reach of all but the very largest organisations, and this lack of integration has resulted for most in a level of built-in control that doesn’t arise in cloud.
For this reason, when considering mesh architectures and data fabrics, the organisation’s governance protocols must be particularly strong, in large part due to the elasticity of cloud architecture, and any subsequent temptation to use it to its fullest.
Development teams’ ambitions used to be constricted by the size and configuration of their on-site infrastructure. Some processes had to be run in a particular way, and resources might have had to be booked and scheduled. In effect, code had to be designed to work within a specific performance box, which naturally limited what users could do.
Today, however, multiple teams can be working with the same data simultaneously since they’re no longer restricted to using on-site resources. The first the data’s custodians might hear of this is when they receive higher than expected bills from their cloud provider. By then, their teams may have been doing whatever they chose for a month, without oversight, increasing the potential for duplicated effort and wasted resources.
Keeping a close watch on how cloud is employed, therefore, is a key aspect of effective online data governance.
Data delivering benefits
If the output of these activities – whether authorised or not – is going to be of use to the business, it must also be catalogued, and the code behind it must be productised.
And, at the same time, it’s important not to authorise data use simply because it is innovative, cloud focused, or has the potential to deliver a healthy return. The principles of data governance must remain front and centre throughout, alongside more familiar factors like engineering and systems maintenance.
This has undeniably become more complicated and difficult to control in cloud, where engineers may be one step removed from the hardware they’re overseeing, but it’s by no means impossible. Any additional effort required is commensurate with the potential return.
Cloud may be familiar, but it’s also a relatively recent invention. We’re just starting to understand its full potential, which means we need to find a balance between innovation and its rational, justifiable use.
It’s often said that with great power comes great responsibility. That has never been truer than here, where data governance and the cloud are concerned.
Don’t put your data, your customers, or your reputation at risk. The team at Red Olive can help you devise a data governance policy that considers the unique challenges – and opportunities – of working with cloud infrastructure. Call the team today.
