Data quality – polluter pays?

Ken O’Connor has written a great blog where he uses the analogy of upstream factories and river pollution for data quality.

I can see myself using this analogy in the future (with appropriate attribs to Ken) with others such as data quality debt to help articulate issues around the whole data management/quality space.

Do you want to continually keep paying to clean up the effects of the pollution or don’t pollute in first the place?

With the environment there is the ‘polluter pays principle’.

Would an analogous principle of – ‘poor data quality producer pays principle’ – be useful in an enterprise?

Data architecture – so what?

Read an interesting post How to Present Data Quality Dimensions For Maximum Impact .

The blog is about data quality but it applies to all aspects of data management.

It offers good advice – when presenting to business users:

• don’t talk about the technicalities
• put it in terms that they will understand
• tell them how you are able to solve their problems.

The author sums it up quite succintly – “ask yourself if your presentation answers two simple words – so what?”.

Last year I attended a DAMA UK event – So how do we remain relevant? – on a related theme – the need to better articulate the value of data management.

Someone mentioned the importance of the 3 R’s:

• Revenue – increase it
• Reduce costs
• Regulatory – ensure meeting risk and compliance standards

Whatever we are working on – should always be meeting one or more the 3 R’s.

And as importantly – we should be ready to articulate them in terms that will resonate with the particular business audience.

The question was asked – what would your ‘elevator pitch’ be for the benefits of data architecture (insert any aspect of data management) to the CEO, CIO, business manager, IT project manager, developer etc?

I tend to use the generic – data architecture is about ensuring – “the right data, to the right people, at the right time”.

I realise I need to do a lot of refinement on this to ‘effectively sell’ data architecture to the rest of the business!

Technical debt – what about data quality debt?

In this article Martin Fowler discusses technical debt.

He says: “In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt. Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development because of the quick and dirty design choice. We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future.”

He also explains the use of the ‘Technical debt quadrant’. 

Is the debt Inadvertant or deliberate? 

Is it a reckless or prudent debt? Can you afford to pay the debt back in the long run? 

Is it worth having a similar metaphor – ‘data quality debt’? 

It might be useful when discussing data quality and data management issues within project teams and with the wider business. 

So what constitutes data quality debt? 

Often the sort of decisions taken in projects that lead to: 

• the quality of data degrading overtime.
• having to continually support the system to mitigate against data issues.
• having to carry out expensive data cleansing exercises.

Some examples of data quality debt: 

• Not identitying who in the business is responsible for the data.
  • If no one  is responsible  for keeping it up to date it will degrade overtime.
• Not having clear definitions of the data and it’s purpose.
  • If you don’t - semantic integration with other systems will be an issue and the data is likely to be misused.
• Setting a column in table to null - even though business requirements say it should be compulsory.
  • Some rows in the initial data import didn’t have this data – it was easier to set it to null rather than go back to the business.
• Not having any data quality profiling/metrics - to ensure data quality is maintained over time. 
  • ‘If you cant measure it you cant manage it’.

These are only few examples I am sure there are many more!