Wednesday 28 January 2015

Measuring the measures

So, how should we measure the measures?  How do we define success in a data viz implementation - using Qlik or otherwise?

Is success a pass through the UAT process?  If the client has paid the bill, does that imply success for the contrator?  Do we ever go back and find out?  What kind of post-implementation evaluation do we do?  I know that I can tell you anecdotes of several enormously successful implementations where tthe client has achieved great ROI - but I can't tell you that I learned of these by any great process of measuring that success.  Nor can I tell you about the implementations that were less successful,  or were enormously successful but we never recorded it - these have not been measured and have faded from memory.

At a basic level, a data visualization can be considered successful if the values that are being encoded can be decoded by the user. No matter how slick and technologically advanced the implementation, if the values are not decodable, then that is a fail.  However, that decoding process may be intuitive and easy or it may be complicated and hard - just ticking the box on the ability to decode can't define success.  Do we need to take the success measure up some levels and consider the user experience?

So, our measures can be decoded correctly and the users are happy with the experience.  Now, how do we confirm that the measures that we encoded were the right measures in the first place?  How well were the project goals met?  Are the users even using our wonderfully crafted displays at all?  Or do they simply go straight to the pivot table every time and spend five minutes every morning getting the measure that they actually needed?

If we have been through the process of scoping the project correctly then we will have created a list of measures and key indicators that are correctly linked to the strategic aims of the organization.  But we should go further.  We should create a list of measures of success of the implementation.  Some may not be 100% accurately measureable, but the questions should be there.

You might have a measure like, "sales margin increases by 5%".  This is easily measured.  Or you might have a success measure of "users are happier running reports" - not so easily measured but something that can be estimated.

Of course, these measures of measures are not just used as a success criteria, they become part of a feedback loop of change and improvement where there are always new goal to achieve.

We should always keep measuring the measures.


Stephen Redmond is author of Mastering QlikView, QlikView Server and Publisher and the QlikView for Developer's Cookbook
He is CTO of CapricornVentis a Qlik Elite Partner.
Follow me on Twitter   LinkedIn

2 comments:

  1. Thanks for sharing your post. Most programming languages you can go out and learn, but it occurs to me that a QlikView developer is something you have to be. By that I mean that you have to have skills in a large number of areas – and a failing in one can negate brilliance in another. https://intellipaat.com

    ReplyDelete
  2. Hey Keylabs online trainings, stop being a spammer. Putting your spam on my blog is very bad practice and leads me to believe that you are not a good training organization.

    ReplyDelete

Note: only a member of this blog may post a comment.