Tuesday, 23 April 2013

Performance using Labels or Column() in charts

If I have a chart that calculates Total Sales: Sum(Sales), and Total Costs: Sum(CostPrice*Quantity), then I have 3 options to calculate the Total Margin:

Option 1:  I can just use the same expressions in full:
Sum(Sales) - Sum(CostPrice*Quantity)

Option 2:  I can reference the column values using the Column() function:
Column(1)-Column(2)

Option 3: I can reference the labels of the first expressions:
[Total Sales] - [Total Costs]

Now, someone once said to me that the last option, although it might appear to be the best because you are re-using existing values, actually calculates slower than the others.  I have often wondered if that was true.  Someone else suggested to me recently that the 2nd option actually performs quicker.  I decided that I should find out!

Buried deep in the Document Properties of a QVW, on the Sheets tab, there is a list of all the objects on the sheet along with their last calculation time (CalcTime) in milliseconds.  If you looked at this for one of your charts, you might find that it says "0".  This is because the last calculation might have been from cached data so the only accurate CalcTime is when the chart is first initialized in QlikView or when the first of a particular set of selections are made in a recently opened document - where the cache has not been established yet.

So, what I did was to take a document with about 5,000,000 sales transaction rows and build a simple Straight Table of Sales / Costs / Margin by Country.  I made 3 copies of this document, each copy making the Margin calculation in one of the above ways.  The only object in each document is the straight table.

I then completely closed QlikView.  I opened one of the documents, recorded the "Initialize" CalcTime for the chart, made a selection of 2 countries (UK and USA) and recorded the "Selection" CalcTime for the chart.  Then I shut down QlikView again and repeated.  I did this 8 times for each of the documents, opening them in random order.

Here are the results:

Option 1Option 2Option 3
InitializeSelectInitializeSelectInitializeSelect
219125219125219125
250140250125202125
234125265156203124
218125219125218125
250125218141218140
219125281156250156
234141218141219140
219125218125297141
Mean230.4128.9236.0136.8228.3134.5
Mode219.0125.0218.0125.0219.0125.0
Median226.5125.0219.0133.0218.5132.5

There is some slight differences.  But, statistically, I would have to say that there is no real difference between the 3 options!

One thing that I would say is that I don't like using the Column() function in a Straight Table.  This is because the user may have the option to swap columns around and that would invalidate the expression.  Other than that, it appears that you can just use whatever option you like and it won't make any difference to QlikView performance.


Stephen Redmond is CTO of CapricornVentis a QlikView Elite Partner. We are always looking for the right people to join our team.
Follow me on Twitter: @stephencredmond

8 comments:

  1. Hi Stephen,

    I did a similar test some time ago (http://www.qlikfix.com/2011/06/21/testing-the-performance-implications-of-variables-and-label-referencing-versus-direct-expressions/), comparing expressions, variables and column references. Came to the same conclusion, the differences are negligible.

    I had actually expected column referencing to use some sort of caching, but maybe QlikView is so smart that it recognizes that you're reusing the same expression ;)

    Cheers,
    Barry

    ReplyDelete
    Replies
    1. Whoops! Didn't mean to look like I was copying some existing work.

      Interestingly, my results show a very slight advantage for using label references. However, as I said, I don't believe that it is statistically significant.

      Delete
    2. No worries, wasn't meant like that. I get this question quite often, so I assume others do as well, turns out to be the case :)

      Delete
  2. Stephen and Barry: I got different results when using more resource intensive functions in the chart. http://blog.axc.net/?p=1042

    ReplyDelete
    Replies
    1. Very interesting, seems like results are being cached. Do you have any data for using variables as well?

      Delete
  3. I do not. I would expect the same results as explicit expressions, because the variables are really just doing text replacement.

    ReplyDelete
  4. Hi Barry/Mike, were you guys able to do more research on why label referencing is 2X faster than explicit expressions when complex aggr expression is used ?

    ReplyDelete
  5. concur with Mike - with very complex expressions, labels were two times faster in my case. My table was over 90 million rows

    ReplyDelete