Data Governance is one of those most misunderstood concepts. A lot of BS fills up around this word in many companies. A host of companies suck at this. In fact, Gartner had made some really drastic predictions in this area in 2012, such as – by 2016, 20 percent of CIOs in regulated industries would lose their jobs for failing to implement the discipline of information governance successfully. 2016 is here – not sure if this doomsday prophecy has come true, but there won’t be a lot of disagreement that not too many companies perform data governance well.
It is Trust and Confidence, Stupid…
When all the BS is pared down, Data Governance is all about having high trust and confidence in data and analytics. Trust and Confidence in context of data are pretty much synonymous to words like Quality and Reliability in context of cars.
Everyone would, of course, agree that car companies and modern manufacturing methods in general, have largely solved quality and reliability problem (never mind an occasional disaster story). The credit for this largely goes to Taiichi Ohno and Toyota Manufacturing System he pioneered. This system had precious little to do with manufacturing per se – it was a philosophy, known as Lean, that they happened to apply to manufacturing. This philosophy has now been successfully used in as diverse fields as software development (think – Agile/Scrum).
Lean Philosophy 101
The book, Lean Thinking, is an excellent primer on Lean philosophy. The book makes quite a fascinating reading and I would highly recommend to anyone who wants to change their outlook to life in many areas.
According to Lean Thinking, the biggest enemy in any process is “Muda” – the Japanese word for “Waste”. According to the book, muda is specifically any activity which absorbs resources but creates no value – mistakes which require rectification, processing steps that aren’t actually needed, movement of people or goods (or data?) without any purpose.
Any improvement activity, therefore has to start with ruthless attack on and effort to eliminate muda.
To accomplish this, the Lean Philosophy follows the continuous improvement framework by –
- Clearly establishing Value for the ultimate customer
- Clearly defining Value Stream that goes into accomplishing the value
- Establishing Flow in the Value Stream by ruthlessly eliminating non-value added steps and activities
- Establish Demand Pull so that flow is triggered only when there is demand for it
- Achieve Perfection – which of course is philosophical and never ending journey of continuous improvement
To quote the book – “We are all born into a mental world of “functions” and “departments”, a commonsense conviction that activities ought to be grouped by type so that they can be performed more efficiently and managed more easily.” This approach keeps members of the department busy, all the equipment running hard, but adds long waits and quality issues for the product from customer standpoint.
Results of Lean Philosophy are indisputable at this stage. Dramatic improvement in quality, drastic reduction in inventory, lead times and any other negative indicator are now beyond any contention for anyone who has done this.
So What This Has Anything To Do With Data Governance
A lot actually. Here are some examples –
Think about how many times you might have heard – “just ingest the data”. This is a great example of creating activity without establishing value. If you establish the value first, you would avoid a whole lot of subsequent futile conversations and wasteful engineering effort.
A company made significant change to “just ingest the data” thinking by defining business goals (“we need metrics to track product launch success”), making them specific, timely and measurable, putting some upfront thought about what those goals should be in terms of definitions, and then think about what would it take to build them technically.
The result of this was – the product launch was engineered for measuring success through data out of the gate. Data and analytics were not an after thought and a fire drill – it was part of minimum viable product for the release itself.
Think of how many hops data makes in your company before it moves from Click to Analytic. Each hop introduces latency and chance for error without adding any value. Eliminate these hops and you would have taken a step to establish “Flow”.
Lean Toolset
Lean toolsets are fairly established and time tested. Some of them are –
Poka Yoke (mistake proofing) – For example, adding record counts on both sides of ETL process and aborting process if they do not match.
Kaizen (continuous improvement) – Measuring data quality and having process in place to improve it through standard TQM techniques
Visual Cues – We established a big TV monitor in our break room to provide visual cues to the team so that they can spot abnormalities in data trends and proactively initiate correction.
Culture Change
Many organizations lament lack of “data culture”. Lean practices have well established culture transformation methodologies that have helped many organizations. Rather than it being an abstract concept, Lean roadmap has potential to affect real culture change that can make lasting change to having continuous improvement culture.