Part 2 - In this post I want to review the 3 top data re-architecture principles that will make your business lean and efficient with respect to its business insights and analytics. These principles will help transform your current application centric data architecture to a user centric approach.
Principal One: Visibility = Lineage
Organization executives need to be able to answer the following three questions:
Who is the owner of a particular data set?
Who is using the data down to the attribute level and for what purpose?
What is the data lineage in the workflows we are using?
A CIO/CDO that cannot answer these questions has a lot of work in front of them. Achieving data lineage visibility is not easy. How many times have you de-commissioned a database WITHOUT actually knowing who is using it? That’s a shot in the dark. If someone complains, then you know who is connected to the database. Is this really the best way to manage data lineage visibility?
Principal Two: The Decentralization Factor
If you are going through your digital transformation process, then you know how complicated the process of normalizing and changing the architecture of data in your legacy platforms can be. Every decade, there is another push for a new kind of data lake #EDM system. The migration to the cloud has introduced us to great offerings by the likes of #Snowflake, #Redshift, #Databricks, #Dremio and more. Sounds cool, let’s just transfer everything and our problems are solved, right? Wrong!
First, you may have already put all of your data in an on-premise #datalake such as #Hadoop (or any other #dataswamp). Now you need to move it again? Who is going to pay for that?
Decentralization is a major trend, and our digital economy is not one of a centralized nature. Data is everywhere and will remain everywhere so the notion that all data would magically be organized in these great cloud-based data lakes is a dangerous one. Decentralized applications, distributed ledgers, #crypto and #digitalassets don’t lend themselves well to a centralized application centric architecture. The future is user centric! So, how do you maintain control of an ecosystem of data that is not in one location?
Principal Three: Automatic Consent
A crucial factor in preparing for our decentralized digital era and a user centric data architecture is the concept of #automatic #consent management (ACM). Today we look at data usage as a one-way road; I give you permission to see my database and control what you can see with a role-based mechanism. Good but not enough!
In the new digital era, requests to view your data are very common and might be requested by new and unknown vectors. For example, a new retailer loyalty program is asking for the frequency of their product usage to enhance their sales and usage analytics. Would the user or organization need to sign paperwork, open data ports, and set up whitelisting just so the retailer can take the data from the user? No, that would be too time consuming and that isn’t how it should work in the New Digital Economy. A mechanism (in our case we use #smartcontracts) should be put in place to provide speedy, and more importantly, automatic and dynamic rules enforcement and consent for data usage. Take the case of the #openbanking initiative in the UK. It’s a noble idea that requires banks to allow a user to move his account registration information from one bank to another. I expect this regulation to expand to include other PII and maybe even behavioral data in the #mobility scope. So how would a bank ask the user for permission to use a particular data attribute with no automation?
By now you can see the challenge enterprises are facing in preparation for this new era. New enterprises have the benefits of hindsight and lack of legacy and therefore can adopt faster, while older, larger organizations face the risk of latterly being disintermediated by a decentralized digital era.
How would you prepare for this challenge? Take a serious look at our DREAM Fabric.