Business Intelligence, Big Data and Analytics


I’ve just attended a two day seminar – Business Intelligence, Big Data and Analytics given by Rick F. van der Lans.

He combines business awareness with a deep technical background – whilst presenting in a relaxed, approachable manner.

Perhaps confirmation bias on my part but his views on the need for business agility leading to the technical requirement for data visualisation really resonated with me.

If you get the chance I would definitely recommend attending any event where he is presenting or reading the articles on his website.


Power of Information 2011


I attended this Blueprint BI event last week.

The keynote included presentations from both SAP and Microsoft, in which they outlined their new offerings in the BI space e.g. SAP BusinessObjects 4.0 and PowerPivot respectively.

Two interesting concepts were mentioned in a number of sessions given by Blueprint.

  • Disposal BI
  • BI competency centre

Disposal BI
From what I understand this encompasses two key ideas:
1) The need for BI projects to start providing and showing value right from the ‘get go’.
2) That in certain scenarios there is the need to provide immediate BI solutions that are just good enough to meet specific requirements. But, there is recognition, that whatever is produced might need to ‘thrown away’ in the near future.

There was an interesting debate about this topic.

What if creating something quickly now has a negative impact in the longer term?

Are many organisations really willing to dispose of something once they have even made relatively minor investments in terms of time and resources?

BI competency centre (BICC)
Where appropriate a new business unit should be created that is responsible for BI across the enterprise. Effectively consolidating the people, tools and processes that are normally spread across an organisation. Whilst I can certainly see the potential advantages of such an approach. But, one of my concerns (and they didn’t really cover this area) is how such a group would interact with other existing groups in an organisation that traditionally have (some) responsibility in the BI space – IT, data architecture and more broadly data management groups. Also, the success of such a group would be dependent on how willing individual business units were willing to support a central group.

It would be interesting to hear the experiences of anyone who has worked in an organisation that has set up a BICC.

Bringing it together
Used appropriately disposal BI could offer real advantages to an organisation. The key questions being centred on whom and how are people going to decide when it is appropriate. To a large degree it relates back to the standard question of ‘tactical vs. strategic’. This is where PEAF’s enterprise debt concept is really useful for framing this type of discussion.

A group within an organisation – be it a BICC or another existing group – has to have a strategic overview for the overall enterprise.

Without such a group, the key enablers for effective BI and definitely for ‘disposal BI’ – such as strategic data architecture, quality and integration work is unlikely to be carried out – and ‘disposal BI’ could become another excuse to continue carrying out tactical work, building up enterprise debt in the process.

SAP Business Objects BI/EIM 4.0 release event


I attended the SAP Business Objects BI/EIM 4.0 release event in London yesterday.

What really impressed me was the emphasis nearly all of the presenters placed on having a sound data management strategy within an organisation (I’ve been to many vendor events over the years where the technology is sold as the silver bullet for an organisation’s data issues).

A quote used in one of the presentations – on the need for business definitions – particularly resonated with me:

“Lack of consensus on what data means keeps business analytics peripheral to decision making.”

On the technology side two products/areas were of particular interest:

Information steward. First impressions from a 30 minute demo? This looks good and is definitely something I will do further research into.

It seems to have 3 core areas*:

  • Data integration, profiling and quality*. Lots of functionality/’dashboarding’ around setting up data rules and profiling data. The data quality KPI dashboard for key business entities eg customer, products etc. was impressive.This particularly resonated with me as providing data quality ‘statistics’ is something I have been working on recently – though not to the level this tool seems to have.
  • Metadata management. This includes the abiltity to carry out impact analysis on changing an information asset eg what upstreams systems source the data and linkages to downstream systems/reports that consume the data. All displayed in a nice UI.
  • Business definitions glossary. This area was not included in the demo. But, from what I could gather, this can be used as part of overall enterprise definition management process eg making it easier for subject matter experts to keep them up todate – with the ability to link definitions to physical data sources.

* The notes I took have this all as one area but I suspect that it is actually split into 2.

SAP High-performance Analytic Appliance – HANA. Current business information/analytics infrastructure might be typified as:

OLTP – ODS – data mart(s) – data warehouse(s)

Lots of infrastructure, ‘plumbing’ and copying/moving of vast quantitites of data – often required for for performance reasons.

HANA is SAP’s move to exploit the advantages of in-memory computing. Rather than having the data on disk – it is now possible to process massive quantities of data in real time, in-memory, in a much more performant way. This will remove the need to have lots of different environments for performance reasons. One environment for both transactional and all reporting/BI/analytical requirements.

It seems like a generational shift in physical database ‘storage’ technology and would seem to offer benefits in terms of having real ‘real time’ BI/BA and the reductions in infrastructure..

It could also have an impact in other areas of data architecture.

No more excuses such as ‘denormalise for performance’?

With this ‘generational shift’ might there also be a move to using a true relational database management system eg Rel, with finally, a true separation of physical and logical concerns?

Microsoft Business Intelligence Seminar March 2011


I attended this event a few days ago.

Its purpose was – “to give you a good understanding of Microsoft’s BI strategy and its platform consisting of Microsoft Office 2010, SharePoint 2010, and SQL Server 2008 R2.”

There definitely seems to be a lot going on in the MS reporting/BI space with – and the guest speaker eluded to better things to come in the next release of SQL Server – Denali later this year.

Key take aways include:

  • Users increasingly want to perform more self-service analysis and to ‘build their own’ BI solutions with minimal dependence on IT support. The ‘traditional’ approach of IT taking a long time to build these reporting/BI solutions is no longer sustainable – as often by the time the project is ‘finished’ business needs have changed so the solution no longer meets requirements. MS’s core message was the need for organisations to get the correct level of balance between – end user agility vs control.
  • MS see the current set of vendors split between two camps. The traditional ‘control’ world with big vendors eg IBM, Oracle and SAP and the newer more end user focused – ‘agile’ – vendors eg MicroStrategy, QlikView and Tableau. They see their position (perhaps not unsurprisingly) as being unique in that they have the product stack that can stradle both camps. End users can have more ‘agile self service BI’ using tools such as Excel 2010/PowerPivot – whilst products such as Sharepoint and SQL Server can support the more the ‘control’ side of things.
  • End users usage – how many people were accessing Excel/PowerPivot and how often they are refreshing their datasets from source systems – can be monitored via Sharepoint. This is potentially one of the core enablers for getting the balance right between agility and control. End users can start with self-service BI via PowerPivot and if it reaches a critical mass of usage across the organisation then it can be migrated to a corporate solution.
  • A new Business Intelligence Semantic Model (BISM) that will power Microsoft BI front end experiences such as Excel, Reporting Services and SharePoint Insights. I wasn’t aware of this before this event – the following blog from the SQL server team gives a good overview about it.

An issue that I have with the self service BI visualisation tools space is the tendency to gloss over data management/quality issues that most organisations face. Better end user data access with dashboards with ‘slicing and dicing’ is a really good idea. But, if the underlying data structures, enterprise definitions and data quality are poor – how accurate a business insight will these tools really provide?

Therefore it was good to see that MS seem to be taking steps into the wider data management space. They already have Master Data Services and they mentioned that Denali would include “Data Quality Services for knowledge-driven data cleansing and Impact Analysis and Lineage”. Definitely an area to keep an eye on.

Pervasive business intelligence


I attended the IM 2010 – Information as an asset conference last week.

It was a thought provoking day with a really good mix of topics – ranging from people’s experience of running an effective data governance programme,  looking at PowerPivot, to a great presentation by Frank Buytendijk – ‘Dealing with dilemmas: where business analytics fall short’ – which looked at wider cultural issues related to organisations.

The main themes that I took away were:

  • Information technology is becoming more pervasive – with information being the key to tying it all together.
  • In this ‘joined up world’ – effective data management and governance become even more critical for providing true business intelligence.
  • Thinking about the meaning/definitions of data in your organisation is a good place to start on this journey.

The slides for the presentations can be downloaded here.