Wednesday, 20 April 2011

Improve load performance by writing better script!

I came across a document recently where a date was being loaded from a text field in YYYYMMDD format.  The developer had used left, right and mid functions along with MakeDate to parse the date.

This is something that I might have done myself several years ago - before I knew better.  I wondered to myself if there would be much of a load speed differential between this method and Date# - which is the method that I would use today.  So I did some testing.

This particular machine was running v9 so it could have used some load speed tweaking.  I took a data set of 3 million records and compared both using the two methods.  The left, right, mid method took about 2 minutes (not a fast machine).  The Date# method was under 1:30 - 3/4 the time for the other method.

I wonder if you went back and reviewed some of your old code, if you might speed it up a little by just doing things better, just because you now know better.


Stephen Redmond is CTO of CapricornVentis a QlikView Elite Partner

1 comment:

  1. Ha! Definitely. I keep thinking about the "Eat This, Not That" books, and wish there were a "Code This, Not That" book, to show the evolution of methods for building more standard processes.

    Your example would be perfect for an "I used to do this like A, but now I do it like B" instance.

    ReplyDelete