More on Dual - Problem in Sets

I have discussed using Dual a couple of times in the past. It can be enormously useful. Sometimes it can throw you a bit - especially if you do not realise that you are using a Dual.

An example comes up in my response to the post of jhoffmann on QlikCommunity.

He was asking about doing calculations in a Set. This is, of course, eminently achievable using the dollar-expansion syntax. You do have to watch it when using a Dual value.

He was using a field called "RollingMonth". I assume, because it comes from the (old) training manual, that this was created in the Script using:

...
Date(MonthStart(date_field), 'MM/YY') As RollingMonth,
...


This syntax creates a Dual value - with the month's date (as an integer) and the text as per the format string.

This is quite fine in a Set if you use a simple calculation such as Max:

Sum({<RollingMonth = {"$(=Max(RollingMonth))"}>} Amount)

You see, Max knows about Dual and will return the correct value so that the comparison will work - it returns the Text value. It is the text value that is required in the Set comparison. If the max month is January 2010, Max will correctly return (in this case) "01/10" - even though it is alphabetically less than the "12/09" before it. Max does the Dual sort on the number and returns the text. Simple.

The problem arises with some other functions that are not so "Dual aware". For example, if we wanted to see the values from the same month last year, we might try this:

Sum({<RollingMonth = {"$(=AddMonths(Max(RollingMonth),-12))"}>} Amount)

The problem is that AddMonths takes a date value as a parameter and returns a date. It doesn't care about the text value. It uses the Dual's numeric value to do the calculation and then returns a date/numeric value. For example, it might be "01/01/2009". This will not work in our Set above (RollingMonth={"01/01/2009"}) because the text doesn't match and the Sum will return 0.

To fix this one, we just need to re-establish the Dual value:

Sum({<RollingMonth = {"$(=Date(AddMonths(Max(RollingMonth),-12),'MM/YY'))"}>} Amount)

Now the calculations will work perfectly.

More on "Sets =- or -=???"

A while ago I blogged about using =- or -= in a set.

I answered a post in QlikCommunity today (http://community.qlikview.com/forums/p/24894/95176.aspx#95176) that demonstrates the use very well.

The question was from sebagiar. He asked how to get an average out of values while ignoring 0s. For example if I have 3 values: 100, 0, 200 then he wants a result of 150, not the 100 that the straight Avg function will give.

My response is to use the following set on the field:

=Avg({< F1 -= {0}>} F1)

So, why "-=" instead of "=-"? Well, the later just tells the field to use all values except 0 - it will exclude any selections on that field. The first one is "additive" (well, I guess I should say "subtractive") and will include and selections on that field and then drop out the zeros.

The right negative

I recently had a situation where a customer was not happy with the values returned in a budget variance column.

Simple calculation for variance - (Spend - Budget)/Budget

However, in some of their profit centers there was no budget but there was spend! I.e.: (Spend)/0.

Rather than just display a "-" or 0, they wanted this to display as -100%, to really flag it up.

Following on from my previous post about the RangeMax function in QlikView, I thought about it but then, deciding that wouldn't work for me here, I decided on the Alt function.

The Alt function is quite useful because, like RangeMax, you give it a series of Expressions but what it does is returns you the first one that results in a valid number. So, if I use:

Alt((Spend-Budget)/Budget, -1)

When the variance percentage is Null (when Budget is 0), then the value returned will be -1 (-100%). Exactly what I needed.

Alt can also be useful where you have mixed date formats in a date stream - using multiple Date# functions in an Alt and one of them should give you a number.

Keeping it positive

I had a request that when the result of a particular expression was negative, then it should just display as zero (it was a measure of how far out of compliance the result was - a negative result meant that they were in compliance so zero was appropriate).

So I could do a simple:

if(myExpression > 0, myExpression, 0)

That is OK if the expression is simple, but if the expression is more complex then it becomes unwieldy and long and hence difficult to maintain.

A much more elegant way is to use RangeMax like this:

RangeMax(0, myExpression)

Very simple, if my expression dips into the negative then the max value in the range becomes 0. Exactly what I want to achieve and much easier to maintain.

Lies, damned lies...and then there is statistics

I spotted this in a Microstrategy publication which showed the "TCO" for Microstrategy over other operators:



This is a fairly typical use of a flawed statistic to prove a point.

You see, if you have a 10 user QlikView system, you have 1 IT person looking after it - the same guy that is looking after Windows, Email, buying coffee, etc. The same guy can probably look after QlikView as the company grows. So that is a 1 IT person for 10. Push that out to 1000 users and it become a "statistic" of 100 IT people per 1000.

Given that, until recently, QlikView has been playing more in the SME sector, then these numbers are going to be more than less typical. Therefore it probably says something that the "statistic" is only 21.9/1000.

At the other end, of course Microstrategy are best because of only 4.9 IT people per 1000. There is no real qualatitive data behind this. Microstrategy play in the Enterprise sector where they will regularly sell large CAL deals as well as open ended deals - we have no idea how many, exactly, users there are.

Its probably good to know that while your 21.9 QlikView IT people are busy looking after the Email server and making the coffee, your 4.9 Microstrategy guys are busy doing Microstrategy and nothing else.

Using Alerts in QlikView

Alerts are quite easy to set up. With a few tricks, they can become quite powerful.

Let me start with a very simple data set:

CountrySales:
LOAD * INLINE [
Country, Sales
England, 400
Ireland, 300
Scotland, 200
Wales, 100
];

Now, I want to create an Alert when there are any countries that have Sales > 200.

To create an alert, I need to be able to create an expression that will return a true value. I could use a simple aggr expression like:

=Sum(Aggr(If(Sum(Sales)>1, 1), Country)) > 0

This will work. However, it means that we are going to be limited to what information we can include in the Alert message. We can include the number of Countries, but we couldn't, for example, say which ones they were.

A more powerful way is to combine Advanced Search, Bookmarks and Alerts.

If I create an Advanced Search against the Country field such as:

=Sum(Sales)>200

Then I save this search as a Bookmark. Now, I can create an Alert with this Bookmark used where the Alert is simply:

=Count(Distinct Country) > 0

And the Alert message could be something like:

='Number of countries exceeding 200 = ' & Count(Distinct Country) & chr(13) &
'Countries: ' & Concat(Country, ', ')

Of course, I can bring any other information into this message that is related to the countries that match the Bookmark.

Now, if I set-up the "Mail" options under User Preferences, I can have this document email an alert to someone.

If I turn on the alert to use "Batch Mode", this can happen on the server when the document is reloaded.

AND Mode in list boxes

By default, multiple selections in list boxes are "OR" mode. For most applications this is exactly what we need.

However, there are situations, such as that requested by Wolfgang Praschnig on QlikCommunity this week:
http://community.qlikview.com/forums/p/22224/84898.aspx.

Wolfgang wants to be able to see customers who have bought one product AND bought another product. The default action of QlikView would be to show customers who had bought one product or the other.

To enable "AND mode" in a list box, there are a couple of rules that we need to follow:

1. The field in the list box must be contained in a table that has only 2 fields.

2. The other field in the table is the association (or ID) field. This field is only contained in this table.

3. The table must have been loaded with the "DISTINCT" keyword.

This can sometimes be tricky to achieve, so hopefully an example will help demonstrate.

Lets say that we have 3 tables:

Orders
OrderID, CustomerID, OrderDate

OrderDetails
OrderID, LineNo, ProductID, Quantity, Price

Product
ProductID, ProductName

What we want to do is connect the product name to the Orders table in a way that we can employ AND mode.

This means that we will need to create a table that has the OrderID (to associate to Orders) and a ProductName field (which we will call "ProductNameAND") that we can put in our List Box with AND mode on.

So, first we need to create a table with the OrderID and ProductID from the OrderDetails table:

ProductAND:
LOAD DISTINCT
OrderID, ProductID As PID
RESIDENT
OrderDetails;

Note that I have aliased the ProductID field as PID - this is because I am going to drop it in a minute. Also note the DISTINCT keyword.

Now, bring in the Product name:

LEFT JOIN (ProductAND)
LOAD DISTINCT
ProductID As PID, ProductName As ProductNameAND
RESIDENT
Product;

Finally, drop the PID field so we are left with only the 2 fields as required.

DROP FIELD PID;

Now, we can add the ProductNameAND field in a List Box and the AND mode will be available.

Interesting feature here. When you click on an item in an AND mode box, an "&" will appear to the left of it. Pressing CTRL and clicking on additional "possible" (in white) items will add further "&"s beside those items. However, if you click on an "excluded" item or you click and hold the mouse down for a second on "possible" items, it turns to a "!" instead - i.e. "NOT". This can be really powerful for queries.