tag:blogger.com,1999:blog-22681591937481718662024-03-13T23:09:35.477+00:00Qlik TipsUsed to be just Qlik stuff but now pretty much anything Data is in scope.Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comBlogger177125tag:blogger.com,1999:blog-2268159193748171866.post-7751366055167953582021-02-11T23:58:00.000+00:002021-02-11T23:58:15.922+00:00Time to say goodbye ... to the Qlik Luminary Program<p>It has been a blast!</p><p>Back in 2013, Qlik launched something called The Qlik Luminary Program with <a href="https://web.archive.org/web/20150202183219/http://www.qlik.com/us/company/luminary-program/luminary-profiles" target="_blank">10 "Charter" members</a> drawn from some of their larger customers.</p><p>At the time, to be honest, I didn't really pay a lot of attention to it. I didn't know any of the charter members, and I didn't really know what the program was for, or even how to get involved with it. I was also pretty busy with other stuff! As well as spending my days pretty much working all the time with QlikView, and leading a team of QlikView developers, I had just published the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developers Cookbook</a> that June and was working on <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>.</p><p>What I didn't know was that Qlik were planning to anoint a set of people from their ecosystem to be Qlik Luminaries, and had polled their own people for candidates. It came as a pleasant surprise to me when I heard from <a href="https://www.linkedin.com/in/mabaeyens/" target="_blank">Miguel Angel Baeyens de Arce</a>, then an Enterprise Architect at Qlik, that he had nominated me and that I had made the grade!</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-rvGwgkst42g/YCW-7Ce0k2I/AAAAAAAADx4/c3WMGnFPXlQ-uY4dokc_8mfHGO8LC1D5QCLcBGAsYHQ/s800/2014%2BQlik%2BLuminary.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="800" data-original-width="600" height="320" src="https://1.bp.blogspot.com/-rvGwgkst42g/YCW-7Ce0k2I/AAAAAAAADx4/c3WMGnFPXlQ-uY4dokc_8mfHGO8LC1D5QCLcBGAsYHQ/w240-h320/2014%2BQlik%2BLuminary.jpg" width="240" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">First Luminary certificates signed by former Qlik CEO Lars Bjork</td></tr></tbody></table><br /><p>At Qonnections 2014 the new class of Luminaries were well feted. It was great to make a lot of new friends there, especially <a href="https://twitter.com/seebach" target="_blank">Torben Seebach</a>, with whom I pounded the Orlando sidewalks at ungodly hours before the sun came up to make it too hot to run. It was a great week. </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-o2UqsS2F6yA/YCW_ExPxQbI/AAAAAAAADx8/rmU_6mh-vW0TeZERLXpySis5EvenY1jQQCLcBGAsYHQ/s600/Luminary%2BSelfie.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="450" data-original-width="600" src="https://1.bp.blogspot.com/-o2UqsS2F6yA/YCW_ExPxQbI/AAAAAAAADx8/rmU_6mh-vW0TeZERLXpySis5EvenY1jQQCLcBGAsYHQ/s320/Luminary%2BSelfie.jpg" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Luminary Selfie 2014</td></tr></tbody></table><br /><p>We had an equally good time later that year when attending the Qlik Sense launch in Barcelona - where our Luminary status so overawed <a href="https://www.linkedin.com/in/qlikluminary/" target="_blank">Brian Booden</a> that he decided to spill his drink on me!</p><p>That was sadly the last big Qlik event that I got to attend for a while. Not long after publishing <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, I moved to work for Deutsche Bank - one of Qlik's large customers. It is fair to say that I had some good supporters within Qlik because, despite a drop off in social media activity and blogging, I managed to do enough to maintain my Luminary status through my period there.</p><p>My next job move moved my back to the Partner ecosystem, joining one of Qlik's biggest global partners, Accenture. I was luckily able to get back to Qonnections in 2018, and it was great to meet all my Luminary friends again. </p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-TUC-hvIlK54/YCXAQarS_8I/AAAAAAAADyM/rgoeK-BE73gKeU75eu3Ed9fgSoGYPr8dwCLcBGAsYHQ/s2048/Luminary%2BSelfie%2B2.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1204" data-original-width="2048" height="188" src="https://1.bp.blogspot.com/-TUC-hvIlK54/YCXAQarS_8I/AAAAAAAADyM/rgoeK-BE73gKeU75eu3Ed9fgSoGYPr8dwCLcBGAsYHQ/w320-h188/Luminary%2BSelfie%2B2.jpg" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Luminary Selfie 2018</td></tr></tbody></table><br /><p>A big highlight of that year was the couple of days in Lund at the Luminary meetup at Qlik's original headquarters.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-PII8l44d7JE/YCXAkn4q4xI/AAAAAAAADyU/fZlQmzAGN-k__LsaqZ8WNtd1cS8EI0YPwCLcBGAsYHQ/s1518/Qlik%2BHeadquarters.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1268" data-original-width="1518" src="https://1.bp.blogspot.com/-PII8l44d7JE/YCXAkn4q4xI/AAAAAAAADyU/fZlQmzAGN-k__LsaqZ8WNtd1cS8EI0YPwCLcBGAsYHQ/s320/Qlik%2BHeadquarters.jpg" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Qlik HQ in Lund</td></tr></tbody></table><br /><p>A change of role within Accenture in 2019 saw me actually move back to being a Qlik customer again as I am now part of Applied Intelligence team within Accenture's Global IT group, who own and run a very large Qlik infrastructure. We have a great team, especially my colleague <a href="https://twitter.com/c_afungchwi" target="_blank">Christine Afungchwi</a>, who I was delighted to see joining the Qlik Luminary Program in 2020.</p><p>2020 saw a lot of changes for everyone, not least the move of Qlik World (formerly Qonnections) to an online format. Our Luminary meetup in the fall had to also go online, but the Program Team really knocked it out of the park with the setup and made it an event that I will never forget.</p><p>For me, 2020 also saw a big Qlik highlight where the Luminary Program team helped enable the Qlik Meetup community to go online using their WebEx connections. The very first one was the <a href="https://www.qlikmeetupscotland.co.uk/2020/04/10/qvms-episode-1/" target="_blank">Qlik Virtual Meetup Scotland (QVMS)</a>, and I was extremely privileged that <a href="https://www.linkedin.com/in/qlikluminary/" target="_blank">Brian</a>, <a href="https://www.linkedin.com/in/georgebeaton/" target="_blank">George </a>and <a href="https://www.linkedin.com/in/andrew-mcilwrick-a341738/" target="_blank">Andrew</a> agreed to allow me to present at that meetup alongside Qlik's <a href="https://twitter.com/elif_tutuk" target="_blank">Elif Tutuk</a>.</p><p>So now we are in 2021 and the Qlik Luminary Program is changing. They are renaming it to <a href="https://www.qlik.com/us/solutions/customers/qlik-luminaries" target="_blank">Qlik Luminary 50</a>, reducing the number of members, and no longer accepting folks from the Qlik Partner ecosystem, restricting it to customer "advocates and counsellors". I think that after this length of time that it is probably right that there is a change, and it will be interesting to see what happens. But I will no longer be there.</p><p>I am in a strange position because I work for a Partner organization - Accenture regularly win the Partner of the Year accolade - but actually, because of my role in our Global IT group, I am a Qlik customer so probably could still apply. However, I think that it is time for me to retire.</p><p>While I am still passionate about Qlik, and still have some recognition as an expert across the ecosystem, it is no longer my full-time job as it was back in 2014. Although I am still an active user of Qlik tools - I have QlikView open most days and often help other teams with troubleshooting - most of my work these days is in PowerPoint and most of my team are working in Python. I never use any of the Qlik Data Integration tools - I have never needed to.</p><p>I haven't been active on Qlik Community for some time, though I remain reasonably active on social media, and have somewhat rejuvenated this Qlik Tips blog recently. But it is really just an occasional hobby as I am not working with the tools enough to generate good content for more regular updates.</p><p>Over the last year or so, as well as the QVMS, I have also presented at global industry and academic conferences, and local Qlik Meetups, where I have identified myself as a Qlik Luminary and shared my ideas and knowledge around data. I will continue to do that - except the Luminary part - as it is part of my DNA to share and educate. I suspect that most of that in future will not be Qlik related, but I will continue to apply to Qlik World to speak and share my ideas. I will continue to attend where I can. I will continue to blog here ad-hocly.</p><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-vEToQOdaNTM/YCXBE8rBqPI/AAAAAAAADyc/BU6d56vdUn4En8aAG6nUuHLNv4uaE3vKwCLcBGAsYHQ/s934/Qlik%2BMeetup%2BPresenting.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="474" data-original-width="934" height="162" src="https://1.bp.blogspot.com/-vEToQOdaNTM/YCXBE8rBqPI/AAAAAAAADyc/BU6d56vdUn4En8aAG6nUuHLNv4uaE3vKwCLcBGAsYHQ/w320-h162/Qlik%2BMeetup%2BPresenting.jpg" width="320" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Presenting at the Qlik Meetup Dublin</td></tr></tbody></table><br /><p>I am committed to providing feedback that is constructive and actionable to Qlik, but that forum will be via our regular connects from within Accenture, where I will continue to advocate for Qlik use - not because I am a Luminary but because I believe it is a great product and we get great value out of using it.</p><p>I probably have a lot of the aspects that Qlik are looking for in their Luminary 50, but I don't think that I am going to be the Luminary that Qlik need at this stage of my career. However, I would still encourage other Qlik customers to think about <a href="https://www.qlik.com/blog/2021-qlik-luminary-applications-are-now-open" target="_blank">applying to the new Program</a> - I would love to see a diverse pool of people in the new club. Even if you don't think you match all the exact requirements, if you are a customer and you are passionate about Qlik, then send in an application!</p><p>I couldn't write a goodbye to the Luminary Program without mentioning three very special people: <a href="https://www.linkedin.com/in/viktorialindback/" target="_blank">Viktoria Lindback</a>, <a href="https://www.linkedin.com/in/carolynjanedavis/" target="_blank">Carolyn Davis</a> and <a href="https://www.linkedin.com/in/gillianjcf/" target="_blank">Gillian Farquhar</a>. You three have been nothing but helpful, generous and absolutely brilliant over the years and I hope that I can always call you my friends. I will always be available to you if you need help or advice.</p><p>In the words of the late, great, Douglas Adams, so long, and thanks for all the fish!</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-_eqhQNvfVgQ/YCW8xnu1yEI/AAAAAAAADxs/GJ3NUYP1QZ4boZBUxSjtrSUqVMY4oz8iwCLcBGAsYHQ/s407/Swedish%2BFish.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="322" data-original-width="407" src="https://1.bp.blogspot.com/-_eqhQNvfVgQ/YCW8xnu1yEI/AAAAAAAADxs/GJ3NUYP1QZ4boZBUxSjtrSUqVMY4oz8iwCLcBGAsYHQ/s320/Swedish%2BFish.jpg" width="320" /></a></div><br /><p><br /></p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-65814505429843449202021-01-07T21:45:00.001+00:002021-01-07T21:45:33.920+00:00Revisiting the QlikView Cookbook - Containers as Multiboxes<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p>
<p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p>
<p>In this post, I'll look at the recipe on <b>Using containers as an alternative to multiboxes</b>.</p><p>For those who are unfamiliar with the QlikView Multibox, it was a way for us to add multiple filters into a relatively small area:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-MZz957R-uRI/X_d83r8f4aI/AAAAAAAADwI/2x0LU2S27PoYdKIGGKZnH6KuyHAaDpoAACLcBGAsYHQ/s241/Multibox%2B-%2BQV%2BVersion.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="142" data-original-width="241" height="236" src="https://1.bp.blogspot.com/-MZz957R-uRI/X_d83r8f4aI/AAAAAAAADwI/2x0LU2S27PoYdKIGGKZnH6KuyHAaDpoAACLcBGAsYHQ/w400-h236/Multibox%2B-%2BQV%2BVersion.png" width="400" /></a></div><br /><p>When a user clicked on the Dimension name, the relevant List box would pop out. This is not dissimilar to how the Filter pane works in Qlik Sense when there are several Dimensions added and not enough room to display values:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-bLDHeP250dY/X_d99gK4upI/AAAAAAAADwQ/fFf1S-qtqysLcUKRNeT9JeDLQ_Un0ZJgACLcBGAsYHQ/s388/Multibox%2B-%2BFilter%2BPane.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="388" data-original-width="318" height="400" src="https://1.bp.blogspot.com/-bLDHeP250dY/X_d99gK4upI/AAAAAAAADwQ/fFf1S-qtqysLcUKRNeT9JeDLQ_Un0ZJgACLcBGAsYHQ/w328-h400/Multibox%2B-%2BFilter%2BPane.png" width="328" /></a></div><br /><p>The Cookbook presented an alternate option whereby a Container is used to contain multiple List Boxes:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-3v4nXTnLKpk/X_d-37Zta1I/AAAAAAAADwY/YfyWWoaOF2UkuwWOZdPgjEHs5duTF0FCgCLcBGAsYHQ/s244/Multibox%2B-%2BQV%2BContainer.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="172" data-original-width="244" height="282" src="https://1.bp.blogspot.com/-3v4nXTnLKpk/X_d-37Zta1I/AAAAAAAADwY/YfyWWoaOF2UkuwWOZdPgjEHs5duTF0FCgCLcBGAsYHQ/w400-h282/Multibox%2B-%2BQV%2BContainer.png" width="400" /></a></div><p>It worked quite well (in my humble opinion!)</p><p>Now, there is also a Container object in Qlik Sense, so we can achieve something similar. This is the load script from the Cookbook:</p>
<p></p>
<blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><div style="text-align: left;"><span style="font-family: consolas;">Data:<br /></span><span style="font-family: consolas;">LOAD * INLINE [<br /></span><span style="font-family: consolas;"> Country, Product, SalesPerson, Category, Sales<br /></span><span style="font-family: consolas;"> USA, Widget, Joe, Widgets, 1234<br /></span><span style="font-family: consolas;"> USA, Woggle, Joe, Widgets, 1983<br /></span><span style="font-family: consolas;"> USA, Brogue, Jane, Footwear, 1175<br /></span><span style="font-family: consolas;"> USA, Clog, Jane, Footwear, 1376<br /></span><span style="font-family: consolas;"> UK, Widget, Tom, Widgets, 954<br /></span><span style="font-family: consolas;"> UK, Woggle, Tom, Widgets, 953<br /></span><span style="font-family: consolas;"> UK, Brogue, Tom, Footwear, 989<br /></span><span style="font-family: consolas;"> UK, Clog, Tom, Footwear, 875<br /></span><span style="font-family: consolas;"> Japan, Widget, Mike, Widgets, 1265<br /></span><span style="font-family: consolas;"> Japan, Woggle, Mike, Widgets, 1345<br /></span><span style="font-family: consolas;"> Japan, Brogue, Jane, Footwear, 1425<br /></span><span style="font-family: consolas;"> Japan, Clog, Jane, Footwear, 1324<br /></span><span style="font-family: consolas;">];</span></div></blockquote>
<p></p><div>Loading this in Qlik Sense Business, we can then add a Container and 4 separate Filter Pane objects for the 4 Dimensions:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-JNGhRhlJaes/X_d_zIjf-MI/AAAAAAAADwg/V1BvdAlfqvYzu3Gg4Phil_GY5XAmJDXrQCLcBGAsYHQ/s632/Multibox%2B-%2BQS%2BContainer%2BWide.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="262" data-original-width="632" height="166" src="https://1.bp.blogspot.com/-JNGhRhlJaes/X_d_zIjf-MI/AAAAAAAADwg/V1BvdAlfqvYzu3Gg4Phil_GY5XAmJDXrQCLcBGAsYHQ/w400-h166/Multibox%2B-%2BQS%2BContainer%2BWide.png" width="400" /></a></div><br /><div>If you don't have too much real estate to devote to it, then it will automatically collapse to a version with a menu to select the Dimension:</div><div><br /></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/--ER6k8OFfO8/X_eAT5yNI6I/AAAAAAAADwo/IBKlPluVSV4auyIYjLdIlASLMQkJ58m3wCLcBGAsYHQ/s318/Multibox%2B-%2BQS%2BContainer%2BNarrow.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="264" data-original-width="318" height="333" src="https://1.bp.blogspot.com/--ER6k8OFfO8/X_eAT5yNI6I/AAAAAAAADwo/IBKlPluVSV4auyIYjLdIlASLMQkJ58m3wCLcBGAsYHQ/w400-h333/Multibox%2B-%2BQS%2BContainer%2BNarrow.png" width="400" /></a></div><br /><div>I prefer this to the default Filter Pane method.</div><div><br /></div><div>Worth noting that the default would be to have "Sales Person" twice - as a Title for the Container and for the Filter Pane - you can blank out the Filter Pane's title to make it go away.</div><div><br /></div>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-1990443681283375782021-01-04T20:28:00.004+00:002021-01-07T21:45:49.407+00:00Revisiting the QlikView Cookbook - Section Access Gotchas<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p>
<p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p>
<p>In this post, I'll look at the recipe on <b>Section Access Gotchas</b>. In Qlik, Section Access is the area of script where we connect users to the data to allow row-level security. It is very powerful and relatively easy to implement for many typical use cases.</p><p>Before diving in, it is worth pointing out that Qlik Sense Business has a great feature, missing from QlikView when I was writing the Cookbook, that means if a <b>Section Access</b> script would result in the locking out of the developer, then tags the reload as unsuccessful and won't save the faulty setup. This will help with a lot of <b>Section Access</b> issues!</p><p>In the recipe I showed a number of different issues. Starting off with this load script:</p>
<p></p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><div><span style="font-family: consolas;">Section Access;<br /></span><span style="font-family: consolas;">Access:<br /></span><span style="font-family: consolas;">LOAD * INLINE [<br /></span><span style="font-family: consolas;"> ACCESS, USER.EMAIL, Link<br /></span><span style="font-family: consolas;"> ADMIN, stephen.redmond@notmymail.fake, *<br /></span><span style="font-family: consolas;"> USER, user1@notmymail.fake, US<br /></span><span style="font-family: consolas;"> USER, user2@notmymail.fake, UK<br /></span><span style="font-family: consolas;"> USER, user3@notmymail.fake, Fr<br /></span><span style="font-family: consolas;">];</span><span style="font-family: consolas;"><br /></span><span style="font-family: consolas;">Section Application;</span><span style="font-family: consolas;"><br /></span><span style="font-family: consolas;">Sales:<br /></span><span style="font-family: consolas;">Load * Inline [<br /></span><span style="font-family: consolas;"> Link, Country, Sales<br /></span><span style="font-family: consolas;"> US, USA, 1000<br /></span><span style="font-family: consolas;"> UK, United Kingdom, 800<br /></span><span style="font-family: consolas;"> Fr, France, 750<br /></span><span style="font-family: consolas;"> De, Germany, 940<br /></span><span style="font-family: consolas;">];</span></div></blockquote>
<p style="text-align: left;">Now, when I login as the Admin, I get to see all the data as expected. However, when I login as user1, I still see all the data when I was expecting to only see US data! What is going on?</p>
<p>Well, a simple rule on the Section Access tables is that all the fields must be in capital letters. Simple fix so:</p>
<p></p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><p style="font-family: consolas; text-align: left;"><span style="font-family: consolas;">Section Access;<br /></span><span style="font-family: consolas;">Access:<br /></span><span style="font-family: consolas;">LOAD * INLINE [<br /></span><span style="font-family: consolas;"> ACCESS, USER.EMAIL, <b>LINK</b><br /></span><span style="font-family: consolas;"> ADMIN, stephen.redmond@notmymail.fake, *<br /></span><span style="font-family: consolas;"> USER, user1@notmymail.fake, US<br /></span><span style="font-family: consolas;"> USER, user2@notmymail.fake, UK<br /></span><span style="font-family: consolas;"> USER, user3@notmymail.fake, Fr<br /></span><span style="font-family: consolas;">];</span><span style="font-family: consolas;"><br /></span><span style="font-family: consolas;">Section Application;</span><span style="font-family: consolas;"><br /></span><span style="font-family: consolas;">Sales:<br /></span><span style="font-family: consolas;">Load * Inline [<br /></span><span style="font-family: consolas;"> <b>LINK</b>, Country, Sales<br /></span><span style="font-family: consolas;"> US, USA, 1000<br /></span><span style="font-family: consolas;"> UK, United Kingdom, 800<br /></span><span style="font-family: consolas;"> Fr, France, 750<br /></span><span style="font-family: consolas;"> De, Germany, 940<br /></span><span style="font-family: consolas;">];</span></p></blockquote>
<p>Updating the Link to LINK in both tables and now User1 only sees US data. What of user3? If I log in as User3 I get Access Denied! This is because of a second Gotcha - any data field that links users to data and that contains letters, must only contain capital letters! So, "Fr" doesn't work. Simple fix:</p>
<p></p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><p style="font-family: consolas; text-align: left;"><span style="font-family: consolas;">Section Access;<br /></span><span style="font-family: consolas;">Access:<br /></span><span style="font-family: consolas;">LOAD * INLINE [<br /></span><span style="font-family: consolas;"> ACCESS, USER.EMAIL, LINK<br /></span><span style="font-family: consolas;"> ADMIN, stephen.redmond@notmymail.fake, *<br /></span><span style="font-family: consolas;"> USER, user1@notmymail.fake, US<br /></span><span style="font-family: consolas;"> USER, user2@notmymail.fake, UK<br /></span><span style="font-family: consolas;"> USER, user3@notmymail.fake, <b>FR</b><br /></span><span style="font-family: consolas;">];</span><span style="font-family: consolas;"><br /></span><span style="font-family: consolas;">Section Application;</span><span style="font-family: consolas;"><br /></span><span style="font-family: consolas;">Sales:<br /></span><span style="font-family: consolas;">Load * Inline [<br /></span><span style="font-family: consolas;"> LINK, Country, Sales<br /></span><span style="font-family: consolas;"> US, USA, 1000<br /></span><span style="font-family: consolas;"> UK, United Kingdom, 800<br /></span><span style="font-family: consolas;"> <b>FR</b>, France, 750<br /></span><span style="font-family: consolas;"> <b>DE</b>, Germany, 940<br /></span><span style="font-family: consolas;">];</span></p></blockquote>
<p>Now, User3 can login and see the France data. All is great. Or is it?</p><p>But let us just quickly log back in as the Admin - we can only see US, UK and France! Where is Germany? This is a Gotcha whereby all of the users must be linked to all of the data or the security will remove access to the unlinked data. Interestingly, also having an admin user who is linked to none of the data will also work. It does look like the Admin user is linked to everything via the "*", but that isn't actually everything - it is only everything that is linked! Confused yet?</p><p>We can fix it like this:</p>
<p></p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px;"><p style="font-family: consolas; text-align: left;"><span style="font-family: consolas;">Section Access;<br /></span><span style="font-family: consolas;">Access:<br /></span><span style="font-family: consolas;">LOAD * INLINE [<br /></span><span style="font-family: consolas;"> ACCESS, USER.EMAIL, LINK<br /></span><span style="font-family: consolas;"> ADMIN, stephen.redmond@notmymail.fake, <b>NONE</b><br /></span><span style="font-family: consolas;"> USER, user1@notmymail.fake, US<br /></span><span style="font-family: consolas;"> USER, user2@notmymail.fake, UK<br /></span><span style="font-family: consolas;"> USER, user3@notmymail.fake, FR<br /></span><span style="font-family: consolas;">];</span><span style="font-family: consolas;"><br /></span><span style="font-family: consolas;">Section Application;</span><span style="font-family: consolas;"><br /></span><span style="font-family: consolas;">Sales:<br /></span><span style="font-family: consolas;">Load * Inline [<br /></span><span style="font-family: consolas;"> LINK, Country, Sales<br /></span><span style="font-family: consolas;"> US, USA, 1000<br /></span><span style="font-family: consolas;"> UK, United Kingdom, 800<br /></span><span style="font-family: consolas;"> FR, France, 750<br /></span><span style="font-family: consolas;"> DE, Germany, 940<br /></span><span style="font-family: consolas;">];</span></p></blockquote>
<div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-sga1WoPFMMo/X_N6UTWHwCI/AAAAAAAADv8/eI-hZNYVHkYOyvd7HFEfr1uY_J2sjwf6ACLcBGAsYHQ/s428/CountryBarChart.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="351" data-original-width="428" height="328" src="https://1.bp.blogspot.com/-sga1WoPFMMo/X_N6UTWHwCI/AAAAAAAADv8/eI-hZNYVHkYOyvd7HFEfr1uY_J2sjwf6ACLcBGAsYHQ/w400-h328/CountryBarChart.png" width="400" /></a></div><br /><p>Now, the Admin user can see everything. The "NONE" is arbitrary and can be any value as long as it doesn't exist in the data. Actually, I prefer to script things so that at least one user is linked to all of the data values. That way I don't have to rely on the "*".</p><p><br /></p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-83202280569375079802020-12-22T12:57:00.002+00:002020-12-22T12:57:41.523+00:00Revisiting the QlikView Cookbook - Using OMIT<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p>
<p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p>
<p>In this post, I'll look at the recipe on <b>Blocking user access to a field using OMIT</b>.</p><p>Picking this one was spurred by a message last week from a good friend who reminded me of a <a href="https://www.qliktips.com/2011/04/blocking-user-access-to-field-using.html" target="_blank">blog post that I wrote on the topic back in 2011</a>!</p><p>Privacy of fields have been around longer than GDPR or and other legislation. In the Cookbook, I imagined a scenario of a HR application that contained salary and absence information that we might want to restrict access to. I am sure that we can all imagine many other fields that we might want to restrict - including names!</p><p>Anyway, here is the code setup to replicate the recipe in Qlik Sense Business:</p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><div style="text-align: left;"><span style="font-family: consolas;">Section Access;</span></div><div style="text-align: left;"><span style="font-family: consolas;"><br /></span></div><div style="text-align: left;"><span style="font-family: consolas;">Access:</span></div><div style="text-align: left;"><span style="font-family: consolas;">LOAD * INLINE [</span></div><div style="text-align: left;"><span style="font-family: consolas;"> ACCESS, USER.EMAIL, OMITGROUP</span></div><div style="text-align: left;"><span style="font-family: consolas;"> ADMIN, stephen.redmond@notmymail.fake, </span></div><div style="text-align: left;"><span style="font-family: consolas;"> USER, user1@notmymail.fake,</span></div><div style="text-align: left;"><span style="font-family: consolas;"> USER, user2@notmymail.fake, SALARYONLY</span></div><div style="text-align: left;"><span style="font-family: consolas;"> USER, user3@notmymail.fake, SALARYANDABSENCE</span></div><div style="text-align: left;"><span style="font-family: consolas;"> USER, user4@notmymail.fake, ABSENCEONLY</span></div><div style="text-align: left;"><span style="font-family: consolas;">];</span></div><div style="text-align: left;"><span style="font-family: consolas;"><br /></span></div><div style="text-align: left;"><span style="font-family: consolas;">OmitGroups:</span></div><div style="text-align: left;"><span style="font-family: consolas;">LOAD * INLINE [</span></div><div style="text-align: left;"><span style="font-family: consolas;"> OMITGROUP, OMIT</span></div><div style="text-align: left;"><span style="font-family: consolas;"> SALARYONLY, Salary</span></div><div style="text-align: left;"><span style="font-family: consolas;"> SALARYANDABSENCE, Salary</span></div><div style="text-align: left;"><span style="font-family: consolas;"> SALARYANDABSENCE, Absence</span></div><div style="text-align: left;"><span style="font-family: consolas;"> ABSENCEONLY, Absence</span></div><div style="text-align: left;"><span style="font-family: consolas;">];</span></div><div style="text-align: left;"><span style="font-family: consolas;"><br /></span></div><div style="text-align: left;"><span style="font-family: consolas;">Section Application;</span></div><div style="text-align: left;"><span style="font-family: consolas;"><br /></span></div><div style="text-align: left;"><span style="font-family: consolas;">Employees:</span></div><div style="text-align: left;"><span style="font-family: consolas;">LOAD * INLINE [</span></div><div style="text-align: left;"><span style="font-family: consolas;"> EmpID, Name, Salary, Absence</span></div><div style="text-align: left;"><span style="font-family: consolas;"> 1, Joe Bloggs, 50000, 3</span></div><div style="text-align: left;"><span style="font-family: consolas;"> 2, Jane Doe, 45000, 5</span></div><div style="text-align: left;"><span style="font-family: consolas;"> 3, Fred Frank, 30000, 0</span></div><div style="text-align: left;"><span style="font-family: consolas;"> 4, Jeri Jublek, 19000, 12</span></div><div style="text-align: left;"><span style="font-family: consolas;">];</span></div></blockquote><p style="text-align: left;">The blank values for the ADMIN and user1 here are deliberate. They could really be anything - blank, "NONE", "MISSING", etc. - anything that would not connect these users to the OmitGroups. The effect here is that these users will not have any OMIT entries so will see all the fields - as in the image below:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-DfKTFtZFaFc/X-Hryndv0aI/AAAAAAAADvI/oMhrEDsysLAXV9PjOjPz9MgaTynrywhugCLcBGAsYHQ/s635/OMIT-AllFields.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="166" data-original-width="635" height="105" src="https://1.bp.blogspot.com/-DfKTFtZFaFc/X-Hryndv0aI/AAAAAAAADvI/oMhrEDsysLAXV9PjOjPz9MgaTynrywhugCLcBGAsYHQ/w400-h105/OMIT-AllFields.png" width="400" /></a></div><br /><p style="text-align: left;">For the users with the <b>SALARYONLY </b>restriction, they cannot even see the Salary field. It will appear as if the field does not even exist in the data model:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-7wBD4tbRWVM/X-Hr6h40ItI/AAAAAAAADvM/Ei_x8c6vUd4lOwxwegZQ4WyC_y8YGIbHACLcBGAsYHQ/s636/OMIT-SalaryOnly.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="167" data-original-width="636" height="105" src="https://1.bp.blogspot.com/-7wBD4tbRWVM/X-Hr6h40ItI/AAAAAAAADvM/Ei_x8c6vUd4lOwxwegZQ4WyC_y8YGIbHACLcBGAsYHQ/w400-h105/OMIT-SalaryOnly.png" width="400" /></a></div><br /><p style="text-align: left;">For those with <b>SALARYANDABSENCE</b>, they will be missing both fields:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-2d4f7EkbpEI/X-HsCTlVh9I/AAAAAAAADvU/IYQLl0XVdhoVDOxJVEDnTrd0mKY5xBfQQCLcBGAsYHQ/s636/OMIT-BothOmit.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="173" data-original-width="636" height="109" src="https://1.bp.blogspot.com/-2d4f7EkbpEI/X-HsCTlVh9I/AAAAAAAADvU/IYQLl0XVdhoVDOxJVEDnTrd0mKY5xBfQQCLcBGAsYHQ/w400-h109/OMIT-BothOmit.png" width="400" /></a></div><br /><p style="text-align: left;">In our post-GDPR world, OMIT is is feature of Qlik that you should definitely consider when you have PII or sensitive fields involved. Make sure to be explicit about the users who can and can't see these data.</p><p style="text-align: left;"><br /></p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com0tag:blogger.com,1999:blog-2268159193748171866.post-2562341800437417572020-12-14T22:06:00.001+00:002020-12-14T22:06:13.773+00:00Revisiting the QlikView Cookbook - Rounding Time to Quarter Hour segments<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p>
<p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p>
<p>In this post, I'll look at the recipe on <b>Consolidating a date-time value into quarter hourly segments</b>.</p><p>The premise behind this recipe is one that is relatively straightforward, we would like to round down Time Stamp data to a less unique value. The best reason for doing this is to simply remove the uniqueness of the data - Time Stamps could represent millions of values in a day, because they can be sub-second, when there is really only 86400 seconds in one day, 1440 minutes and 96 quarters. The less unique your data, the better that Qlik can store it. Of course, you need to bear in mind the business need for the presentation of the values - if they don't need minutes and seconds, don't store them!</p><p>The simple load statement for this recipe is:</p><div style="text-align: left;"><span style="font-family: Courier;"><b>Load<br /></b></span><span style="font-family: Courier;"><b> TimeCounter,<br /></b></span><span style="font-family: Courier;"><b> TimeStamp(TimeStamp) as Date,<br /></b></span><span style="font-family: Courier;"><b> WeekDay(TimeStamp) As WeekDay,<br /></b></span><span style="font-family: Courier;"><b> TimeStamp(Round(TimeStamp, (1/(24))), 'M/D/YY HH:mm') As Hour,<br /></b></span><span style="font-family: Courier;"><b> TimeStamp(Round(TimeStamp, (1/(24*4))), 'M/D/YY HH:mm')<br /></b></span><span style="font-family: Courier;"><b> As QuarterHour;<br /></b></span><span style="font-family: Courier;"><b>Load<br /></b></span><span style="font-family: Courier;"><b> Today() + (Rand()) As TimeStamp,<br /></b></span><span style="font-family: Courier;"><b> 1 as TimeCounter<br /></b></span><span style="font-family: Courier;"><b>AutoGenerate(1000);</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b><br /></b></span></div><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-gAvAMe_hb3Y/X9fd0ZoOPxI/AAAAAAAADuE/PqUzJgYEC2Qf4jLoKKy_HvlgLwc_Bdf5wCLcBGAsYHQ/s957/RoundedQuarters.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="404" data-original-width="957" height="169" src="https://1.bp.blogspot.com/-gAvAMe_hb3Y/X9fd0ZoOPxI/AAAAAAAADuE/PqUzJgYEC2Qf4jLoKKy_HvlgLwc_Bdf5wCLcBGAsYHQ/w400-h169/RoundedQuarters.png" width="400" /></a></div><br /><p>This does give us a result which may be good for our purposes. However, we need to bear in mind that this is a mathematical rounding, and that may not match business rules that we have to abide by. In this example, we can see that the 06:00:00 hour covers time slots from 05:30:00 to 06:29:59. The 06:15:00 quarter covers times slots from 06:07:31 to 06:22:29. These ranges may not be what a business user would expect - would you expect 5:30am to appear in the 6am group?</p><p>If you want your 06:00 to be 06:00:00 to 06:59:59, then this might work: </p>
<p></p>
<p></p>
<p style="font-family: Courier; font-weight: bold;">
TimeStamp#(Text(TimeStamp(TimeStamp, 'YYYYMMDD HH'))&':00','YYYYMMDD HH:mm') As Hour</p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-7150703413446349982020-12-09T21:23:00.000+00:002020-12-09T21:23:04.186+00:00Revisiting the QlikView Cookbook - Dynamic part-to-whole Pie Chart<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p>
<p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p>
<p>In this post, I'll look at the recipe on <b>Creating a dynamic part-to-whole pie chart</b>.</p><p>Now, I know that pie charts are not everyone's first choice, but for part-to-whole comparisons they work just as well, if not better, than other alternatives. (If interested in more on this, read my IEEE VIS 2019 paper, <a href="https://arxiv.org/abs/1908.00630" target="_blank">Visual cues in estimation of part-to-whole comparison</a> here).</p><p>Probably most will agree that this style is not the best:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-OHYANdB3MEc/X9E-RbCqo8I/AAAAAAAADtk/3oKbDHIiN8w3egl6vYDGstgbq4DAj-8kQCLcBGAsYHQ/s556/Pie-NotPartToWhole.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="365" data-original-width="556" height="263" src="https://1.bp.blogspot.com/-OHYANdB3MEc/X9E-RbCqo8I/AAAAAAAADtk/3oKbDHIiN8w3egl6vYDGstgbq4DAj-8kQCLcBGAsYHQ/w400-h263/Pie-NotPartToWhole.png" width="400" /></a></div><br /><p>The large number of segments here tempts the user away from part-to-whole into part-to-part comparisons. There are argument that say this would be equally as useful as a bar equivalent, but I won't make them here.</p><p>To setup an example in Qlik Sense Business, I loaded the following data:</p><div style="text-align: left;"><span style="font-family: Courier;"><b>LOAD * INLINE [<br /></b></span><span style="font-family: Courier;"><b>Country, Sales<br /></b></span><span style="font-family: Courier;"><b>USA, 53453<br /></b></span><span style="font-family: Courier;"><b>Mexico, 21317<br /></b></span><span style="font-family: Courier;"><b>Canada, 14545<br /></b></span><span style="font-family: Courier;"><b>UK, 12333<br /></b></span><span style="font-family: Courier;"><b>France, 21333<br /></b></span><span style="font-family: Courier;"><b>Germany, 23123<br /></b></span><span style="font-family: Courier;"><b>Japan, 44342<br /></b></span><span style="font-family: Courier;"><b>China, 34234<br /></b></span><span style="font-family: Courier;"><b>Australia, 12345<br /></b></span><span style="font-family: Courier;"><b>];</b></span></div><p>The setup of the dimension is quite important. Instead of using the normal field as a dimension, I use a calculation:</p><div style="text-align: left;"><span style="font-family: Courier;"><b>=if(<br /></b></span><span style="font-family: Courier;"><b> Aggr(Sum(1), Country)>0,<br /></b></span><span style="font-family: Courier;"><b> 'Selected: $(=Concat(DISTINCT Country, ', '))',<br /></b></span><span style="font-family: Courier;"><b> 'Others: $(=Concat({<Country=E({$})>} DISTINCT Country, ',<br /></b></span><span style="font-family: Courier;"><b> '))'<br /></b></span><span style="font-family: Courier;"><b>)</b></span></div><p>It is using Aggr to group together Countries that are selected separately from non-selected. We then need to setup the Measure expression to ignore selections on Country:</p><div style="text-align: left;"><span style="font-family: Courier;"><b>Sum({<Country=>} Sales)</b></span></div><p>Now, when we make a selection on Country, the pie chart won't drill to that selection, it will show us that selection versus the other Countries:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-QP0rgEhDPsw/X9E_yJvEuII/AAAAAAAADtw/-u7BNTwgsq0looZOn8Rs3AbTbKWxwXdXQCLcBGAsYHQ/s722/Pie-PartToWhole.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="314" data-original-width="722" height="174" src="https://1.bp.blogspot.com/-QP0rgEhDPsw/X9E_yJvEuII/AAAAAAAADtw/-u7BNTwgsq0looZOn8Rs3AbTbKWxwXdXQCLcBGAsYHQ/w400-h174/Pie-PartToWhole.png" width="400" /></a></div><br /><p>It is true part-to-whole!</p><p><br /></p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-3400697759274160952020-12-06T21:15:00.001+00:002020-12-06T21:15:57.329+00:00Revisiting the QlikView Cookbook - Using TOTAL<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p>
<p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p>
<p>In this post, I'll look at the recipe on <b>using TOTAL to calculate the percentage of total and subtotal</b>.</p><p>I like this recipe because it really helps to understand the power of the <b>TOTAL </b>function, which has been a core part of Qlik for a long time, pre-dating Set Analysis for use in vertical calculations.</p><p>We start off with a simple set of Sales data:</p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><div style="text-align: left;"><span style="font-family: Courier;"><b>LOAD * INLINE [</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>Country, City, Sales</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>USA, San Diego, 23712</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>USA, Dallas, 49888</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>USA, New York, 71621</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>USA, Boston, 55033</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>UK, London, 13444</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>UK, Birmingham, 31212</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>UK, Manchester, 39712</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>Germany, Berlin, 49100</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>Germany, Frankfurt, 59102</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>Germany, Munich, 41568</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>Japan, Tokyo, 41412</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>Japan, Yokohama, 55554</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>Japan, Osaka, 12345</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>];</b></span></div></blockquote><p>Then we need to create a Pivot Table with <b>Country </b>and <b>City </b>as Row Dimensions and the following 3 measures:</p>
<table>
<tbody><tr><th width="30%">Label</th><th width="70%">Expression</th></tr>
<tr><td>Sales $</td><td style="font-family: Courier;">Sum(Sales)</td></tr>
<tr><td>Total Sales %</td><td style="font-family: Courier;">Sum(Sales)/Sum(TOTAL Sales)</td></tr>
<tr><td>% Sales by Country</td><td style="font-family: Courier;">Sum(Sales)/Sum(TOTAL<country> Sales)</country></td></tr>
</tbody></table>
<p>Set the display format of the two percentage measures to an appropriate % format. Turn <b>Show totals</b> on for the <b>Country </b>dimension.</p><p>When the pivot table first displays, the <b>Country </b>dimension will need to be expanded to show the cities.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-DKocmiuwOS8/X81HzGR2yCI/AAAAAAAADtU/X1qMMlHBMq4oQPaja9sOkZLtCWsiZb-lQCLcBGAsYHQ/s696/PivotWithPercentTotals.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="696" data-original-width="626" height="400" src="https://1.bp.blogspot.com/-DKocmiuwOS8/X81HzGR2yCI/AAAAAAAADtU/X1qMMlHBMq4oQPaja9sOkZLtCWsiZb-lQCLcBGAsYHQ/w360-h400/PivotWithPercentTotals.png" width="360" /></a></div><br /><p>Hopefully it is straightforward to see what has happened here. Total by itself tells the function to ignore the dimensions in the chart. Therefore the <b>Total Sales %</b> calculation takes each individual sales total and divides it by the overall total. So, for Berlin, that is 49100 / 543703 = 9%.</p><p>When we include one, or several, dimension within the angle bracket after the TOTAL, we tell Qlik to ignore all dimensions in the chart, except for the ones lists. So when we include <b>Country </b>in the angle brackets, it will give us the Sub-Total for that country. Again, for Berlin, that is 49100 / 149770 = 32.8%.</p>
<p><br /></p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-14030475075785761022020-11-29T19:42:00.001+00:002020-11-29T19:42:55.326+00:00Revisiting the QlikView Cookbook - Labels on a Line Chart<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p>
<p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p>
<p>In this post, I'll look at the recipe on <b>Replacing the legend in a line chart with labels on each line</b>.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-oSL4kmw0gwE/X8P1Qgv3JgI/AAAAAAAADsk/bB_3YZ6_37s88BZ7a4ceifoXOpFvn0KGQCLcBGAsYHQ/s1330/Sense%2BLine%2BChart%2Bwith%2BLegend.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="500" data-original-width="1330" height="150" src="https://1.bp.blogspot.com/-oSL4kmw0gwE/X8P1Qgv3JgI/AAAAAAAADsk/bB_3YZ6_37s88BZ7a4ceifoXOpFvn0KGQCLcBGAsYHQ/w400-h150/Sense%2BLine%2BChart%2Bwith%2BLegend.png" width="400" /></a></div><br /><p>The idea here is that the legend on a chart can be somewhat removed from the data and not always easy to match the colours to the lines, especially for those with a visual impairment such as colour blindness.</p><p>The trick here is to use a small hack with the Dual function to only make labels printable for the last value of the line. Here is a quick run-through of the method - which isn't a million miles away from the QlikView method presented in the book.</p><p>Start with some data:</p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><div style="text-align: left;"><span style="font-family: Courier;"><b>CrossTable(Country,Sales)</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>LOAD * INLINE [</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>Date, USA, UK, Japan, Germany</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>2013-01-01, 123, 100, 80, 40</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>2013-02-01, 134, 111, 75, 50</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>2013-03-01, 155, 95, 70, 60</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>2013-04-01, 165, 85, 88, 50</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>2013-05-01, 154, 125, 90, 70</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>2013-06-01, 133, 110, 75, 99</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>];</b></span></div></blockquote><p>Now, we can quickly create a simple line chart by <b>Date </b>and <b>Country </b>as in the image above, with a measure of <b>Sum(Sales)</b>.</p><p>Now, if we edit the measure to this:</p><blockquote style="border: none; margin: 0px 0px 0px 40px; padding: 0px; text-align: left;"><div style="text-align: left;"><span style="font-family: Courier;"><b>Dual(</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b> If(Date=Max(total Date), Country, ''),</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b> Sum(Sales)</b></span></div><div style="text-align: left;"><span style="font-family: Courier;"><b>)</b></span></div></blockquote><p>So here we are checking if the current date matches the maximum date. If it does, return the name of the Country. If not, return a blank string.</p><p>Before we change anything else, we should change the measure's <b>Number formatting</b> option to be <b>Measure expression</b>, otherwise Qlik Sense will override our Dual and apply a default numeric formatting.</p><p>Now, down in the <b>Appearance </b>section, under <b>Presentation</b>, we turn on the option to <b>Show data points</b>. Set the <b>Value labels</b> option to <b>Auto</b>.</p><p>Just below this, in <b>Colors and legend</b>, turn off the <b>Show legend</b> option - we don't need it!</p><p>If all has gone to plan, we should have a line chart that is something like this:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Y91KhEzhdcI/X8P5DnILDaI/AAAAAAAADsw/BvtTjoX1KLA2f0hpR3RKEL3T8-JJ3CjZwCLcBGAsYHQ/s1331/Sense%2BLabels%2Bas%2BLegend.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="501" data-original-width="1331" height="150" src="https://1.bp.blogspot.com/-Y91KhEzhdcI/X8P5DnILDaI/AAAAAAAADsw/BvtTjoX1KLA2f0hpR3RKEL3T8-JJ3CjZwCLcBGAsYHQ/w400-h150/Sense%2BLabels%2Bas%2BLegend.png" width="400" /></a></div><br /><p>This will not be the perfect solution in every circumstance. If you have very many lines then it just won't look great. But it is something worth playing with, and certainly a hack worth knowing about.</p><p><br /></p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-57638346949000821942020-11-29T18:48:00.000+00:002020-11-29T18:48:16.797+00:00Making Sense of Themes<p>Episode 14 of the <a href="https://www.linkedin.com/events/qvmsep14-barryharmsen-iancrosla6737711105138794496/" target="_blank">Qlik Virtual Meetup Scotland (QVMS)</a> had some very interesting speakers. Old friend and fellow Luminary, Barry Harmsen, from Bitmetric, was talking about his <a href="https://sensetheme.com/">sensetheme.com</a> site, so I thought that I might delve a little into the idea of themes and explain why I think it might be a good idea for you to explore them.</p><p>For some historical context, the idea of Themes is not new to Qlik. Anyone who has used QlikView will know to what level of detail you can specify colours, various fonts, borders, shadows, etc. on various objects and then use the <b>Theme Maker</b> function to export those settings into an XLM file for use in other documents.</p><p>When Qlik Sense was first released in 2014, it didn't have too much in the way of ability to change things. You got the plain white background and the default set of fonts. You could change the odd colour, but not much else.</p>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-epCxqVaQH6E/X8Pa1YawdBI/AAAAAAAADr8/ZRKYqr4niusxkqRp2fsG5eIYxfWdIl39gCLcBGAsYHQ/s1036/Sense%2BClassic.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="510" data-original-width="1036" height="198" src="https://1.bp.blogspot.com/-epCxqVaQH6E/X8Pa1YawdBI/AAAAAAAADr8/ZRKYqr4niusxkqRp2fsG5eIYxfWdIl39gCLcBGAsYHQ/w400-h198/Sense%2BClassic.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The Qlik Sense "Classic" theme<br /></td></tr></tbody>
</table><br />
<p>Many folks grumbled at this and Qlik listened. Starting from Qlik Sense (on-premise) February 2018 release, custom themes could be used to style an app. This feature has subsequently been rolled out on SaaS offerings.</p><p>Out of box on the current version of Qlik Sense Business, there are four themes available. <b>Sense Focus</b> is the default, but you can also choose <b>Sense Breeze</b>, <b>Sense Horizon</b>, or the "Classic".</p>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody>
<tr><td style="text-align: center;">
<a href="https://1.bp.blogspot.com/-q4b1pogAD9Q/X8PdO6SRjVI/AAAAAAAADsQ/tj36jFhUwoMJ474gYRE1nCVHfgTorH9awCLcBGAsYHQ/s1036/Sense%2BFocus.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="510" data-original-width="1036" height="198" src="https://1.bp.blogspot.com/-q4b1pogAD9Q/X8PdO6SRjVI/AAAAAAAADsQ/tj36jFhUwoMJ474gYRE1nCVHfgTorH9awCLcBGAsYHQ/w400-h198/Sense%2BFocus.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sense Focus - the default theme<br /></td></tr></tbody>
</table>
<p></p>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody>
<tr><td style="text-align: center;">
<a href="https://1.bp.blogspot.com/-Vmx5YyGykT8/X8PdOvMF1QI/AAAAAAAADsI/G5E0AG35cCYwRdARaI23G5E-jVBba2YiQCLcBGAsYHQ/s1036/Sense%2BBreeze.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="510" data-original-width="1036" height="198" src="https://1.bp.blogspot.com/-Vmx5YyGykT8/X8PdOvMF1QI/AAAAAAAADsI/G5E0AG35cCYwRdARaI23G5E-jVBba2YiQCLcBGAsYHQ/w400-h198/Sense%2BBreeze.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sense Breeze theme<br /></td></tr></tbody>
</table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody>
<tr><td style="text-align: center;">
<a href="https://1.bp.blogspot.com/-iT13ZoOFers/X8PdO__JflI/AAAAAAAADsM/tq8Hepyr08MPxvF0Vm3LxiaVNYdNXBe0QCLcBGAsYHQ/s1036/Sense%2BHorizon.png" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="510" data-original-width="1036" height="198" src="https://1.bp.blogspot.com/-iT13ZoOFers/X8PdO__JflI/AAAAAAAADsM/tq8Hepyr08MPxvF0Vm3LxiaVNYdNXBe0QCLcBGAsYHQ/w400-h198/Sense%2BHorizon.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Sense Horizon theme<br /></td></tr></tbody>
</table>
<br />
<p>A theme is simply a JSON file with a collection of settings. If you are comfortable working in JSON and with CSS properties, then a good place to start is the <a href="https://help.qlik.com/en-US/sense-developer/September2020/Subsystems/Extensions/Content/Sense_Extensions/CustomThemes/custom-themes-getting-started.htm" target="_blank">Getting started building custom themes</a> page in the Qlik documentation. If you are less comfortable, then <a href="https://sensetheme.com/" target="_blank">Bitmetric's Sense Theme</a> tool is another excellent choice.</p><p>For free, you can download a load of different examples, from Classic QlikView to Tableau! Also for free, with some limitations, you can create your own theme file and use their friendly UI to help you setup the colours.</p><p>For a small monthly fee, you can unlock more features and get rid of those pesky pop-up ads. I can see many businesses taking this option as they will want to apply individual theme elements to their Qlik Sense applications, and this will become a valuable tool for their designers to work with.</p><p>Time to dive into themes?<br /></p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-55135780475943676172020-11-23T22:47:00.000+00:002020-11-23T22:47:13.667+00:00Revisiting the QlikView Cookbook - Boxplot<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p>
<p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p>
<p>In this post, I'll look at the recipe on <b>Creating a box plot chart for a simple data set</b>.</p>
<p>The Boxplot in Qlik Sense is a little more advanced than the one that I originally wrote about in QlikView. It has an element of sophistication around it which may actually initially confuse a user, especially one who has been used to other statistics tools. When I use a Boxplot in QlikView, I generally expect to add one dimension, and then expect that the values will be presented for each value in that dimension. I can, of course, get quite sophisticated with my calculations, and there is even a Wizard to help me get started with creating the necessary <b>AGGR </b>expressions I may need.</p><p>The Qlik Sense Boxplot will actually assume that I want to do the calculations using similar <b>AGGR </b>expressions and, if I don't, I need to make some manual edits. It is probably easier if you let me show you what I mean with an example. First, some sample data - only slightly more complex than in the original recipe:</p><div style="text-align: left;"><span style="font-family: Courier;"><b>Load <br /></b></span><span style="font-family: Courier;"><b><span> </span>'USA' as Country,<br /></b></span><span style="font-family: Courier;"><b> Ceil(Rand()*12) as Month,<br /></b></span><span style="font-family: Courier;"><b> Round(Rand()*50, 0.01) as Value<br /></b></span><span style="font-family: Courier;"><b>AutoGenerate(100);<br /></b></span><span style="font-family: Courier;"><b>Load <br /></b></span><span style="font-family: Courier;"><b><span> </span>'UK' as Country,<br /></b></span><span style="font-family: Courier;"><b> Ceil(Rand()*12) as Month,<br /></b></span><span style="font-family: Courier;"><b> Round(Rand()*20, 0.01) as Value<br /></b></span><span style="font-family: Courier;"><b>AutoGenerate(100);</b></span></div><div><br /></div>
<p>After loading, if I drop a Boxplot onto my sheet, it offers me the usual <b>Add dimension</b> and <b>Add measure </b>options. If I try to recreate the original QlikView Cookbook chart, I might try with Country and Value - actually you will note that, as usual, it forces you to add an aggregation.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-LsnjOtwwrV0/X7wvmZ3z-wI/AAAAAAAADp0/M1awvcJR3b85Eqpz4EoZHhH3aukga-cOgCLcBGAsYHQ/s508/Boxplot1.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="304" data-original-width="508" height="239" src="https://1.bp.blogspot.com/-LsnjOtwwrV0/X7wvmZ3z-wI/AAAAAAAADp0/M1awvcJR3b85Eqpz4EoZHhH3aukga-cOgCLcBGAsYHQ/w400-h239/Boxplot1.png" width="400" /></a></div><br /><p>Ok, so what happens next? Well, you might be surprised to see only 1 skinny box plotted!</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-lSMiuGSA5Nw/X7wwQTamNQI/AAAAAAAADp8/53aETi8c4ecHg82sfcDlNTgtOp84j11BwCLcBGAsYHQ/s325/Boxplot2.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="325" data-original-width="214" height="320" src="https://1.bp.blogspot.com/-lSMiuGSA5Nw/X7wwQTamNQI/AAAAAAAADp8/53aETi8c4ecHg82sfcDlNTgtOp84j11BwCLcBGAsYHQ/s320/Boxplot2.png" /></a></div><br /><p>So what happened? Why did it not split by Country as a QlikView user might have expected? Turning to the help file, you might find this description of adding the first dimension:</p><p><span> </span><i>This is the inner dimension, which defines a box.</i></p><p>Eh? What box does it define? Well, we have added a dimension and an expression, so this is the box it defines:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-ohEQsLb7DXo/X7wxw7nS5wI/AAAAAAAADqI/3LGGxCZBA9kvrKabcWPgTYhkTXFJ7EVYQCLcBGAsYHQ/s290/Boxplot3.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="117" data-original-width="290" src="https://1.bp.blogspot.com/-ohEQsLb7DXo/X7wxw7nS5wI/AAAAAAAADqI/3LGGxCZBA9kvrKabcWPgTYhkTXFJ7EVYQCLcBGAsYHQ/s0/Boxplot3.png" /></a></div><br /><p>So, actually, there are only 2 values being represented by the Boxplot. The plot isn't looking at all the individual values (which can't be bigger than 50 because of the way we have created them), it is representing the statistics of all the calculated values - in this case only 970.77 and 2529.77.</p><p>Let's rejig slightly. I am going to replace the <b>Box </b>dimension with Month and set the <b>X-axis</b> dimension to be Country. The result looks something like this:</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-TMyCYzFB7vE/X7w0c3GsWPI/AAAAAAAADqY/mlFUOjsaatED2t1i8gERK3lPLY1LeBTygCLcBGAsYHQ/s426/Boxplot4.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="374" data-original-width="426" height="351" src="https://1.bp.blogspot.com/-TMyCYzFB7vE/X7w0c3GsWPI/AAAAAAAADqY/mlFUOjsaatED2t1i8gERK3lPLY1LeBTygCLcBGAsYHQ/w400-h351/Boxplot4.png" width="400" /></a></div><div class="separator" style="clear: both; text-align: center;"><span style="text-align: left;"><br /></span></div>In this case, the "box" is now a table of the Sum of Value by Month, and there is a different "box" for each Country. For example, the UK "box" may look like this:<br /><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-Kx9e7Kv8Uvs/X7w0c3KTbNI/AAAAAAAADqU/WKqYRCBOHDoRBV9GhwJauJxivtnxecb4gCLcBGAsYHQ/s387/Boxplot5.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="387" data-original-width="273" height="320" src="https://1.bp.blogspot.com/-Kx9e7Kv8Uvs/X7w0c3KTbNI/AAAAAAAADqU/WKqYRCBOHDoRBV9GhwJauJxivtnxecb4gCLcBGAsYHQ/s320/Boxplot5.png" /></a></div><br /><p>The Boxplot calculations are then calculated across the 12 values in each "box". Simples!</p><p>But what are the calculations? Well, if we look into the <b>Box plot element</b> properties, we can see that it defaults to some presets from which we can choose Standard (Tukey), Percentile-based or Standard deviation. If we turn off the <b>Use-presets</b> option, we can now see what set of calculations are being used for each element in the Boxplot. In fact, this also allows us to put in our own calculations! We could, for example, use this option to go back to a version of the original QlikView Boxplot.</p><p>If add a new Boxplot on the screen and then go straight to the properties and add Country for both the <b>Box </b>and <b>X-axis</b> dimension. For the Y-axis measure, I am just going to add the value <b>0</b> - it doesn't matter too much what value you use, though it does need to return a valid calculation or the chart won't work.</p><p>Now, when I go down to the <b>Box plot element</b> section and deselect the <b>Use-presets</b> option, I can specify new expression values:</p>
<table>
<tbody><tr><th><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Property</blockquote></blockquote></th><th><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Expression</blockquote></blockquote></th></tr>
<tr><td><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">First whisker</blockquote></blockquote></td><td style="font-family: Courier;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Min(Value)</blockquote></blockquote></td></tr>
<tr><td><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Box start</blockquote></blockquote></td><td style="font-family: Courier;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Fractile(Value, 0.25)</blockquote></blockquote></td></tr>
<tr><td><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Center line</blockquote></blockquote></td><td style="font-family: Courier;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Fractile(Value, 0.5)</blockquote></blockquote></td></tr>
<tr><td><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Box end</blockquote></blockquote></td><td style="font-family: Courier;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Fractile(Value, 0.75)</blockquote></blockquote></td></tr>
<tr><td><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Last whisker</blockquote></blockquote></td><td style="font-family: Courier;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;"><blockquote style="border: none; margin: 0 0 0 40px; padding: 0px;">Max(Value)</blockquote></blockquote></td></tr>
</tbody></table>
<p><br /></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-aTL34VjlNdI/X7w4tLz5x4I/AAAAAAAADqo/cts-WtJd1RIT1BBMcS4WvSixEq5VGvXrQCLcBGAsYHQ/s416/Boxplot6.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="377" data-original-width="416" height="363" src="https://1.bp.blogspot.com/-aTL34VjlNdI/X7w4tLz5x4I/AAAAAAAADqo/cts-WtJd1RIT1BBMcS4WvSixEq5VGvXrQCLcBGAsYHQ/w400-h363/Boxplot6.png" width="400" /></a></div><div><br /></div>Now we have something that is similar to the original!<div><br />
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
</div>Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-15603716425786573992020-11-21T22:00:00.001+00:002020-11-23T22:21:18.087+00:00Revisiting the QlikView Cookbook - Tooltips
<p>It has been 7 years since the <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> was published! </p><p>The book, which was very much targeted at QlikView developers, is still available. Just for fun, I thought that it might be interesting to write a series of posts looking back at some of those recipes and seeing how they could be recreated in <a href="https://help.qlik.com/en-US/cloud-services/Content/Sense_Helpsites/about-qs-business.htm" target="_blank">Qlik Sense Business</a>.</p><p>In this first post, I'll look at the very first recipe - <b>Creating custom pop-up labels on a bar chart</b>.</p><p>In QlikView, it was necessary to manually put together a text expression to join multiple values, with their functions to ensure correct formatting, together and ensure values appear on separate rows using line feeds (using the chr function) - something like this:</p><div style="text-align: left;"><span style="font-family: Courier;"><b>= Country <br /></b></span><span style="font-family: Courier;"><b>& chr(10) <br /></b></span><span style="font-family: Courier;"><b>& 'Sales : ' & Num(Sum(Sales), '$(MoneyFormat)') <br /></b></span><span style="font-family: Courier;"><b>& chr(10) <br /></b></span><span style="font-family: Courier;"><b>& 'Costs : ' & Num(Sum(Costs), '$(MoneyFormat)') <br /></b></span><span style="font-family: Courier;"><b>& chr(10) <br /></b></span><span style="font-family: Courier;"><b>& 'Margin : ' & Num(Sum(Sales)-Sum(Costs), '$(MoneyFormat)') <br /></b></span><span style="font-family: Courier;"><b>& chr(10) <br /></b></span><span style="font-family: Courier;"><b>& 'Margin % : ' & Num(1-(Sum(Costs)/Sum(Sales)), '0.0%')</b></span></div><p>In Qlik Sense, it is a little easier. 😊</p><p>In most of the major chart types there is a sub-property in the <b>Appearance </b>property called <b>Tooltip</b>.</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-6Ud_sBL8FAY/X7mE_1k8azI/AAAAAAAADpY/8dlVgVodXNARYZG_vS-jvgHYvjKNdN_KgCLcBGAsYHQ/s452/QSCustomTooltip.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="452" data-original-width="234" height="400" src="https://1.bp.blogspot.com/-6Ud_sBL8FAY/X7mE_1k8azI/AAAAAAAADpY/8dlVgVodXNARYZG_vS-jvgHYvjKNdN_KgCLcBGAsYHQ/w208-h400/QSCustomTooltip.png" width="208" /></a></div><div><br /></div><div><br /></div>In here we can see the following options:<div><p><br /></p>
<table>
<tbody><tr><th width="30%">Option</th><th>Description</th></tr>
<tr><td>Hide basic rows</td><td>This either shows or hides the default Tooltip values. Useful to turn off if you are doing something custom.</td></tr>
<tr><td>Description</td><td>An expression that defines a piece of text to present within the tooltip. Could be used to present useful information for users such as how these metrics compare with others.</td></tr>
<tr><td>Measures</td><td>One or several measures to display. Each measure has typical sub-properties for measures such as label and numeric format.</td></tr>
<tr><td>Images</td><td>An image to display in the Tooltip - either a static one from the Media libaray, or from a Url (which could be dynamic!)</td></tr>
</tbody></table>
<p>
So, to recreate the manually constructed text from QlikView, we just need to turn off the Hide basic rows option, leave the Description blank, and add four Measures with appropriate formats. We might end up with a result like this:
</p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-GNPVB410ZKo/X7mLVYpg0iI/AAAAAAAADpk/-2joXMs_YW8e55B3Tr12lhRrAkVAIrY1ACLcBGAsYHQ/s231/QSCustomTooltipResult.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="231" data-original-width="217" src="https://1.bp.blogspot.com/-GNPVB410ZKo/X7mLVYpg0iI/AAAAAAAADpk/-2joXMs_YW8e55B3Tr12lhRrAkVAIrY1ACLcBGAsYHQ/s0/QSCustomTooltipResult.png" /></a></div><br /><p><br /></p>
<hr /> As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="https://www.qliktips.com/p/mastering-qlikview.html" target="_blank">Mastering QlikView</a>, <a href="https://www.qliktips.com/p/cookbook.html" target="_blank">QlikView for Developer's Cookbook</a> and <a href="https://www.qliktips.com/p/qlikview-server-and-publisher-book.html" target="_blank">QlikView Server and Publisher</a>. <br />
<a class="twitter-follow-button" data-show-count="false" href="https://twitter.com/stephencredmond?ref_src=twsrc%5Etfw" target="_blank">Follow @stephencredmond</a><script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="16" src="https://content.linkedin.com/content/dam/me/business/en-us/amp/brand-site/v2/bg/Chinese-LI-Logo.svg.original.svg" title="LinkedIn" width="94" />
</a>
</div>Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-81760850561798320602020-04-29T22:22:00.001+01:002020-04-29T22:24:52.341+01:00The most important form of workIf you haven't been following the new series of <a href="https://www.meetup.com/pro/qlik" target="_blank">Qlik Virtual Meetups</a>, they are worth tuning into and there is probably one in your region. Being virtual, it doesn't matter if they are in your region or not!<br />
<br />
The recent edition of the <a href="https://qlik.webex.com/recordingservice/sites/qlik/recording/play/d1cc5cb124b14f6aa36bbeee61138dce" target="_blank">Qlik Virtual Meetup Scotland</a> featured an old friend in Treehive Strategy's <a href="https://www.linkedin.com/in/donalddotfarmer/" target="_blank">Donald Farmer</a> (it also featured the ever excellent <a href="https://www.linkedin.com/in/michaeltarallo/" target="_blank">Michael Tarallo</a> talking about Qlik Sense April 2020!). In an excellent presentation, Donald mentioned two particular quotes that resonated with me and some of my own ideas around data visualisation and data literacy.<br />
<br />
The first was from (the excellent!) Michael Lewis's book, <a href="https://smile.amazon.com/Losers-Road-Everyplace-White-House/dp/0679768092" target="_blank">Losers: The Road to Everyplace but the Whitehouse</a>. The quote was:<br />
<br />
<i>"an explanation is where the mind comes to rest"</i><br />
<br />
Donald uses the quote in the context of an analyst looking at data to what they feel is a conclusion. He feels, and I agree, that it is not enough to say that you have just followed the data. You stop being critical. Donald also paraphrases Deming: "with data, your just another person with data, and an opinion". This is because we are all fundamentally human beings with many, many biases. Where our mind comes to rest we feel comfortable.<br />
<br />
For me, an interesting example of this occurred just yesterday with a chart published in this article: <a href="https://theconversation.com/three-charts-that-show-where-the-coronavirus-death-rate-is-heading-137103" target="_blank">Three charts that show where the coronavirus death rate is heading</a>. The chart, which at initial look looks like a spiral mess, with time, becomes clear and quite interesting. It is a great example of a chart that you need to spend some time with to appreciate, but the effort is rewarded.<br />
<br />
I shared my <a href="https://twitter.com/stephencredmond/status/1255263904986542083" target="_blank">opinion on Twitter</a>, and some of the other discussion is interesting. However most of the discussion stopped at, "ugh!" - people were not prepared to give the chart the benefit of the doubt and invest any time. "Ugh!", was their explanation. That is where their minds rested.<br />
<br />
The second quote was from a Harvard Business Review article titled <a href="https://hbr.org/1993/01/whats-so-new-about-the-new-economy" target="_blank">What's So New About the New Economy</a> written by Alan M. Webber:<br />
<br />
<i>"In the new economy, conversations are the most important form of work."</i><br />
<br />
Donald expressed that Data literacy is about communicating with data. It is not just about a few people understanding data, it is about raising the level in society.<br />
<br />
Conversations are enormously important in raising all the boats. Some data visualizations are merely about getting to rest - getting users to a position where they have the explanation that they want. Others are designed to try to get users beyond that point. But it is hard to get a body-at-rest moving. Conversations can help us get over that inertia. Conversations about data. Conversations over data.<br />
<br />
Will you join the conversation?<br />
<br />
<hr />
As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<br />Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com0tag:blogger.com,1999:blog-2268159193748171866.post-86390718842730348402020-04-17T16:26:00.002+01:002020-04-17T16:26:34.228+01:00How Many Segments? And other storiesFollowing on from my recent presentation at the first ever <a href="https://qlik.webex.com/recordingservice/sites/qlik/recording/d329c5e2850b47dd9b30fa213842b3f8" target="_blank">Qlik Virtual Meetup Scotland (link opens the meeting recording)</a>, there were a few questions arising. I thought it would be useful to answer them in a blog post.<br />
<br />
<table border="1" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="border-collapse: collapse; border: none; mso-border-alt: solid windowtext .5pt; mso-padding-alt: 0cm 5.4pt 0cm 5.4pt; mso-yfti-tbllook: 1184;">
<tbody>
<tr>
<td style="border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 150.25pt;" width="200">
<span style="font-family: Arial, sans-serif;">Scott</span><span style="font-family: "Times New Roman",serif; font-size: 12.0pt;"><o:p></o:p></span><br />
</td>
<td style="border-left: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 118.85pt;" width="158">
<span style="font-family: Arial, sans-serif;">Pies</span><span style="font-family: "Times New Roman",serif; font-size: 12.0pt;"><o:p></o:p></span><br />
</td>
<td style="border-left: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 181.7pt;" width="242">
<span style="font-family: Arial, sans-serif;">I like a pie
chart but how many segments before it becomes difficult to judge a value?</span><span style="font-family: "Times New Roman",serif; font-size: 12.0pt;"><o:p></o:p></span><br />
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 150.25pt;" width="200">
<span style="font-family: Arial, sans-serif;">Paul</span><br />
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 118.85pt;" width="158">
<span style="font-family: Arial, sans-serif;">Pies</span><span style="font-family: "Times New Roman",serif; font-size: 12.0pt;"><o:p></o:p></span><br />
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 181.7pt;" width="242">
<span style="font-family: Arial, sans-serif;">Would you not
just group smaller segments into OTHER?</span><span style="font-family: "Times New Roman",serif; font-size: 12.0pt;"><o:p></o:p></span><br />
</td>
</tr>
<tr>
<td style="border-top: none; border: solid windowtext 1.0pt; mso-border-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 150.25pt;" width="200">
<span style="font-family: Arial, sans-serif;">Steve</span><span style="font-family: "Times New Roman",serif; font-size: 12.0pt;"><o:p></o:p></span><br />
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 118.85pt;" width="158">
<span style="font-family: Arial, sans-serif;">Pies / Bars</span><span style="font-family: "Times New Roman",serif; font-size: 12.0pt;"><o:p></o:p></span><br />
</td>
<td style="border-bottom: solid windowtext 1.0pt; border-left: none; border-right: solid windowtext 1.0pt; border-top: none; mso-border-alt: solid windowtext .5pt; mso-border-left-alt: solid windowtext .5pt; mso-border-top-alt: solid windowtext .5pt; padding: 0cm 5.4pt 0cm 5.4pt; width: 181.7pt;" width="242">
<span style="font-family: Arial, sans-serif;">I think the long
tail on a bar chart, with the Sense minichart, is very obvious - where lots
of tiny pie wedges do not</span><span style="font-family: "Times New Roman",serif; font-size: 12.0pt;"><o:p></o:p></span><br />
</td>
</tr>
</tbody></table>
<br />
Let's start with the "how many segments" question and hopefully the thread will move through the answering of the others. First, consider this typical pie chart, produced in Qlik Sense:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-g6Qy1aw7rII/XpmUpJHBzCI/AAAAAAAADgE/EYbl_yiyzdkyv-AWcCyNzcAitzStJ5SEwCLcBGAsYHQ/s1600/Pie%2B1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="570" data-original-width="941" height="241" src="https://1.bp.blogspot.com/-g6Qy1aw7rII/XpmUpJHBzCI/AAAAAAAADgE/EYbl_yiyzdkyv-AWcCyNzcAitzStJ5SEwCLcBGAsYHQ/s400/Pie%2B1.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A typical pie chart with many segments</td></tr>
</tbody></table>
<br />
As with the majority of visualization tools, Qlik Sense has defaulted to the sensible option of sorting the segments in size order. This immediately allows us to see that the USA is bigger than Germany which is bigger than Austria. I can roughly estimate the values of each (<a href="https://arxiv.org/abs/1908.00630" target="_blank">as shown in my research</a>) and I can quickly tell that the three markets make up just under half of the whole.<br />
<br />
The very simple answer to the question of "how many segments" is, how many makes sense to meet the business requirements.<br />
<br />
On the question of grouping smaller segments, I can modify the original chart as below, and still answer those same business questions:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-LjfZrLB3tek/XpmUp4m4tUI/AAAAAAAADgk/i0ncQMKU1Z4om9NQoyhFyd5FlpE49YzKACEwYBhgLKssDAMBZVoD5cMXX6ieixTOgkooe-cBY-0Q2AQEFl5A30A__xbXV1kuPXNXRqsd6nD95puegduQ3m5_1DPHyEvyycxr-__tWWnZ9-Ar7EaGRT415lwkVzb5cGX9iueTlPDdDsyK8lBhNTn_923Acm0Ctq3h5uVctdibpG2a_7pu8wkXkcwGbVXPcDNwoKLUdZMZt5_VJfU75oet1pBm6Wk-QXUfiEUa2xSPEk5TmOgQK8kXwK21qfpWrdQODGmAzEZUltcyN_6zsEY4DXhpsz6DpXH8QyMZl1Oaft0GgYKyhcOSrCE4L8XysBX4hAVTXV6KuhXNlT2GsMbIME-wea2ZVGTm6zdcNJWHpXb6FF9WGhKyMH0QxxObUiV6IXyixH6qC22xkf3XOlh2oKV5qwvYsnMQ5tYcXflHX65Yxank-MUBEZS0ku1Aah1iESJskv2tEoM2D3IKbdcZwIPmon0xmio5MEJ92lGIyPdVcEEXv58F1hfDtGDibXk4N8LjJk9-kibC57yBGpC9w0Ac2Ug79LOPMpPJnlZYHuRVMtgKTovzfun6yFoZylX8FKS9wu2cEopKT7JANwx7g13qvAAAJZQxcWGI2qMxSnWD5MKDC5vQF/s1600/Pie%2B2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="549" data-original-width="934" height="188" src="https://1.bp.blogspot.com/-LjfZrLB3tek/XpmUp4m4tUI/AAAAAAAADgk/i0ncQMKU1Z4om9NQoyhFyd5FlpE49YzKACEwYBhgLKssDAMBZVoD5cMXX6ieixTOgkooe-cBY-0Q2AQEFl5A30A__xbXV1kuPXNXRqsd6nD95puegduQ3m5_1DPHyEvyycxr-__tWWnZ9-Ar7EaGRT415lwkVzb5cGX9iueTlPDdDsyK8lBhNTn_923Acm0Ctq3h5uVctdibpG2a_7pu8wkXkcwGbVXPcDNwoKLUdZMZt5_VJfU75oet1pBm6Wk-QXUfiEUa2xSPEk5TmOgQK8kXwK21qfpWrdQODGmAzEZUltcyN_6zsEY4DXhpsz6DpXH8QyMZl1Oaft0GgYKyhcOSrCE4L8XysBX4hAVTXV6KuhXNlT2GsMbIME-wea2ZVGTm6zdcNJWHpXb6FF9WGhKyMH0QxxObUiV6IXyixH6qC22xkf3XOlh2oKV5qwvYsnMQ5tYcXflHX65Yxank-MUBEZS0ku1Aah1iESJskv2tEoM2D3IKbdcZwIPmon0xmio5MEJ92lGIyPdVcEEXv58F1hfDtGDibXk4N8LjJk9-kibC57yBGpC9w0Ac2Ug79LOPMpPJnlZYHuRVMtgKTovzfun6yFoZylX8FKS9wu2cEopKT7JANwx7g13qvAAAJZQxcWGI2qMxSnWD5MKDC5vQF/s320/Pie%2B2.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Pie chart showing the top 3 segments versus all others</td></tr>
</tbody></table>
<br />
In this situation, I can still see answer that business question and have reduced the number of segments on display. I would argue that I can answer that business question equally well with either chart, although the first may actually deliver me additional insights, and that is the critical thing about either chart - that they answer that business question.<br />
<br />
What can sometimes be a problem, however, is that with interactive tools such as Qlik Sense the user could drill down to just those 3 countries:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-gQ6ptiKQ0JM/XpmhCi-REvI/AAAAAAAADgw/7km5SFT_ZTwykBzQmgiTHVAoDJfcCRWjACLcBGAsYHQ/s1600/Pie%2B3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="535" data-original-width="836" height="204" src="https://1.bp.blogspot.com/-gQ6ptiKQ0JM/XpmhCi-REvI/AAAAAAAADgw/7km5SFT_ZTwykBzQmgiTHVAoDJfcCRWjACLcBGAsYHQ/s320/Pie%2B3.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Pie chart after user has drilled to 3 countries </td></tr>
</tbody></table>
<br />
Now the user is no longer able to see the part-to-whole of these countries' market share. Instead, they are looking at the part-to-whole of just these countries. This may be OK! It depends on the business question that the user wants to answer. If it is a problem, you can use Set Analysis in Qlik Sense to do something about it - similar to one of my <a href="https://www.qliktips.com/2011/04/defending-pie-charts.html" target="_blank">previous pie posts</a>.<br />
<br />
Let's consider some alternatives (alt-pie charts!), starting with the simple bar chart.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-RdEzxkboKjg/XpmUpOmKTJI/AAAAAAAADgk/C9Pf6OqoaocXCNdAONAI73H4BasjuUYGgCEwYBhgLKssDAMBZVoBIijdV_8Bz6YlOoH9ezwe9cmn0AMCBI-U-2uzD-LnuTQx3bdwya6NVQz6eZdNrYXIP4RQqz9GZVPyNNczprbJo_qdvnG6xU-uQg0CUXgq101MTCzOaXcMGbYtbAjAph1eZzaseOazRsykuTOIYB6r9oNEiDD1vkcq0MX8DpZjfGMqH_6bHuyHPSX6LYW-xILe7IvmL5Gx6_YApuHo8jYepmXyZbWNUxJ5S7g2v0C-vhwN9SfxyGSZt9QLKNOu0faK3JjKaiHJZhs6j7K8s5Wqd8RY1q9F_rjFLAG8KZaAKnGs-kcQP4eRpvMqEkGNswv75htW_GYYXepQCogt7rXqaal-TP8xFREDdvc1LBkm5GyTx5gufwTaZzP5DKxAujx7Pt_tMts9i8YrhdjSNi5aoNnabE_1i9nbEX56HHbeISNPa7WQyCWbK-pzzWH47BqUYO74-0oTIUrFS3dA7Q0otdNkhrrwNXUHbitsdDF6ySybaCX7E_E5DGEuAA7ywTFaP0oWKF5yLqiZm7BsvJbAMLfp6GoGoptUkWczAu1K_YRDuhd_GeOIn8IGMElUoQLCsfMnH1z-asBkR9DXZtXXtwqzjj-YgMLyO5_QF/s1600/Bar%2B1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="587" data-original-width="1055" height="178" src="https://1.bp.blogspot.com/-RdEzxkboKjg/XpmUpOmKTJI/AAAAAAAADgk/C9Pf6OqoaocXCNdAONAI73H4BasjuUYGgCEwYBhgLKssDAMBZVoBIijdV_8Bz6YlOoH9ezwe9cmn0AMCBI-U-2uzD-LnuTQx3bdwya6NVQz6eZdNrYXIP4RQqz9GZVPyNNczprbJo_qdvnG6xU-uQg0CUXgq101MTCzOaXcMGbYtbAjAph1eZzaseOazRsykuTOIYB6r9oNEiDD1vkcq0MX8DpZjfGMqH_6bHuyHPSX6LYW-xILe7IvmL5Gx6_YApuHo8jYepmXyZbWNUxJ5S7g2v0C-vhwN9SfxyGSZt9QLKNOu0faK3JjKaiHJZhs6j7K8s5Wqd8RY1q9F_rjFLAG8KZaAKnGs-kcQP4eRpvMqEkGNswv75htW_GYYXepQCogt7rXqaal-TP8xFREDdvc1LBkm5GyTx5gufwTaZzP5DKxAujx7Pt_tMts9i8YrhdjSNi5aoNnabE_1i9nbEX56HHbeISNPa7WQyCWbK-pzzWH47BqUYO74-0oTIUrFS3dA7Q0otdNkhrrwNXUHbitsdDF6ySybaCX7E_E5DGEuAA7ywTFaP0oWKF5yLqiZm7BsvJbAMLfp6GoGoptUkWczAu1K_YRDuhd_GeOIn8IGMElUoQLCsfMnH1z-asBkR9DXZtXXtwqzjj-YgMLyO5_QF/s320/Bar%2B1.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Typical bar chart showing a measure versus a categorical value</td></tr>
</tbody></table>
<br />
Again, we typically order these charts by value, so we can still quickly see that the US is larger than Germany and both are larger than Austria. If our purpose here is to compare one country versus others, then this is the perfect chart to use. Even given that, it is not really so easy to compare the US vs. Poland or even Belgium, but interactivity can help with this. It is, however, a quite a lot more difficult to see how much of the total market is made up by the top 3 countries. That would be especially more difficult if there were a longer tail of smaller values that you might have to scroll. That is why I prefer the pie chart if the business question is a part-to-whole one. We can, of course, clump the other countries into "Others" in the bar chart, but it is still not easy to see the part-to-whole:<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-EPiumHavIbw/XpmUpImT0_I/AAAAAAAADgc/uV6bhWbTK_kfg7Y-TQOWtN1UOawG8q8FACEwYBhgLKssDAMBZVoAbHK1g72y6DopWQL6JSrqjTpjNyyDYujF_up_CfU-DpVqb9Aj_3dGeNb_EsgMlvrEQRpGVX52NJNMRUjBEsGZ0pfTj2GTioTd6PArOTUQd4u1g__JmFmjBp13qKPoN3U5X6hDNrKS1WGDISfQWLmyqID_Ytaa6e2s3nC_S0WuqkxScDp_yn4admwNi3tFL-Fn7pFiL3Mfj8HdLDEJYMOypT9XZ-K5hZsYyyqZEua4lOXSZiwPLZn4C_306hC-ZexQohS5ddxjD2dT2ifdQbMDOoP1a-8f6zvTI8g7EedQkPBQuPaEmSn7ZUYh0aSbJKaBFPCwTkoDHXhL1ER1sHuJd-0-APSvOzJjOhCtmGygONX8KKK5hx8w68ehQ_gdXifB8FhlTT00tpiOww0XwFeVUChvZ68PS5YqcwEwAxtuGRzDhNqc9EHzSn7T8jgfDzzbAD1G00OoUG8JLfsPm5RZJKyLmkGCLxylau7bBZgB4sUleVOC9SKLIBKMVoG8XIn9jrvU2i5LxFQ_c7Z8XjBmpl4SswO1XRAhWxng4MEkob_gVjQCLoPcwhl31-1RNe4QNEVX6_pBrwYZFPgxTktApDMJ7_L_vMN6S5_QF/s1600/Bar%2B2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="615" data-original-width="677" height="290" src="https://1.bp.blogspot.com/-EPiumHavIbw/XpmUpImT0_I/AAAAAAAADgc/uV6bhWbTK_kfg7Y-TQOWtN1UOawG8q8FACEwYBhgLKssDAMBZVoAbHK1g72y6DopWQL6JSrqjTpjNyyDYujF_up_CfU-DpVqb9Aj_3dGeNb_EsgMlvrEQRpGVX52NJNMRUjBEsGZ0pfTj2GTioTd6PArOTUQd4u1g__JmFmjBp13qKPoN3U5X6hDNrKS1WGDISfQWLmyqID_Ytaa6e2s3nC_S0WuqkxScDp_yn4admwNi3tFL-Fn7pFiL3Mfj8HdLDEJYMOypT9XZ-K5hZsYyyqZEua4lOXSZiwPLZn4C_306hC-ZexQohS5ddxjD2dT2ifdQbMDOoP1a-8f6zvTI8g7EedQkPBQuPaEmSn7ZUYh0aSbJKaBFPCwTkoDHXhL1ER1sHuJd-0-APSvOzJjOhCtmGygONX8KKK5hx8w68ehQ_gdXifB8FhlTT00tpiOww0XwFeVUChvZ68PS5YqcwEwAxtuGRzDhNqc9EHzSn7T8jgfDzzbAD1G00OoUG8JLfsPm5RZJKyLmkGCLxylau7bBZgB4sUleVOC9SKLIBKMVoG8XIn9jrvU2i5LxFQ_c7Z8XjBmpl4SswO1XRAhWxng4MEkob_gVjQCLoPcwhl31-1RNe4QNEVX6_pBrwYZFPgxTktApDMJ7_L_vMN6S5_QF/s320/Bar%2B2.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Bar chart with "Others" bar</td></tr>
</tbody></table>
<br />
The recommended choice for part-to-whole coming from anti-pie advocates, is the horizontal bar:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-9hwrXW6bOeg/XpmUqM0u3zI/AAAAAAAADgg/FrPRVeLlB9wJkTFUz_VJC6wlyVwftNCuACEwYBhgLKssDAMBZVoBXGFYyQleYJBi7K5Juv_csfm6nn0nV15twipeSb2xpD8rqZ-uGiVaXIqx0-eb3DqYmMnuVtEQpPT1G7AXTzUQWG-8HgLeDP_8oDypJCmStF1zJuMLh0YpkLMOlFnvAAdk6_D9I8MDyRkz078O8cWlfofFT1g2XUbbFXE_VK_oK4OUjZTBxNqBOh8tu_-geEFAdYnSEif1yLgOBl53Wd1yrqe0bju98axBQjSjavDdc-qbcA94EFthbvpgy1gD6_QoaK-cTTo9R3LLlsCyle70GsJ6ohOqAflNPbF04cE_SLyz3VxbRRhnX4Qq7IYlOCl--3cIvxbE9uRA_J6BCXkVaNyXG4bsF0hmsOBLU8hV5wPjSDoOA4cUQj6EyPUcA0lD4T2Mg7xy4fINk4oW5Mtj3vTojX_jxGa1lOMN0q_Px0SWaev8-UkTdZ2LI_f2GooAe8jfUfnXKmenfehxKPyhjtevPhSSdduHhyuqJxY1rzt7JwIW99V2_U-pHpn6EBBdP6dG6WmyNgshXFkiJzLGCKgpayvDANWcNScZAB8rYDEeOFB1ydOHLwMIIV76HQJ4KNb8gGc5qodLBBT6PXmmIzb5v5e6yMMWT5_QF/s1600/Stacked%2B1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="604" data-original-width="1063" height="181" src="https://1.bp.blogspot.com/-9hwrXW6bOeg/XpmUqM0u3zI/AAAAAAAADgg/FrPRVeLlB9wJkTFUz_VJC6wlyVwftNCuACEwYBhgLKssDAMBZVoBXGFYyQleYJBi7K5Juv_csfm6nn0nV15twipeSb2xpD8rqZ-uGiVaXIqx0-eb3DqYmMnuVtEQpPT1G7AXTzUQWG-8HgLeDP_8oDypJCmStF1zJuMLh0YpkLMOlFnvAAdk6_D9I8MDyRkz078O8cWlfofFT1g2XUbbFXE_VK_oK4OUjZTBxNqBOh8tu_-geEFAdYnSEif1yLgOBl53Wd1yrqe0bju98axBQjSjavDdc-qbcA94EFthbvpgy1gD6_QoaK-cTTo9R3LLlsCyle70GsJ6ohOqAflNPbF04cE_SLyz3VxbRRhnX4Qq7IYlOCl--3cIvxbE9uRA_J6BCXkVaNyXG4bsF0hmsOBLU8hV5wPjSDoOA4cUQj6EyPUcA0lD4T2Mg7xy4fINk4oW5Mtj3vTojX_jxGa1lOMN0q_Px0SWaev8-UkTdZ2LI_f2GooAe8jfUfnXKmenfehxKPyhjtevPhSSdduHhyuqJxY1rzt7JwIW99V2_U-pHpn6EBBdP6dG6WmyNgshXFkiJzLGCKgpayvDANWcNScZAB8rYDEeOFB1ydOHLwMIIV76HQJ4KNb8gGc5qodLBBT6PXmmIzb5v5e6yMMWT5_QF/s320/Stacked%2B1.png" width="320" /></a></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-6J2EjbBDBpM/XpmUqAOT_TI/AAAAAAAADgo/P0vYPAhnHoc66ZjEgIB9IM1rkUDicrbIwCEwYBhgLKssDAMBZVoBXGFYyQleYJBi7K5Juv_csfm6nn0nV15twipeSb2xpD8rqZ-uGiVaXIqx0-eb3DqYmMnuVtEQpPT1G7AXTzUQWG-8HgLeDP_8oDypJCmStF1zJuMLh0YpkLMOlFnvAAdk6_D9I8MDyRkz078O8cWlfofFT1g2XUbbFXE_VK_oK4OUjZTBxNqBOh8tu_-geEFAdYnSEif1yLgOBl53Wd1yrqe0bju98axBQjSjavDdc-qbcA94EFthbvpgy1gD6_QoaK-cTTo9R3LLlsCyle70GsJ6ohOqAflNPbF04cE_SLyz3VxbRRhnX4Qq7IYlOCl--3cIvxbE9uRA_J6BCXkVaNyXG4bsF0hmsOBLU8hV5wPjSDoOA4cUQj6EyPUcA0lD4T2Mg7xy4fINk4oW5Mtj3vTojX_jxGa1lOMN0q_Px0SWaev8-UkTdZ2LI_f2GooAe8jfUfnXKmenfehxKPyhjtevPhSSdduHhyuqJxY1rzt7JwIW99V2_U-pHpn6EBBdP6dG6WmyNgshXFkiJzLGCKgpayvDANWcNScZAB8rYDEeOFB1ydOHLwMIIV76HQJ4KNb8gGc5qodLBBT6PXmmIzb5v5e6yMMWT5_QF/s1600/Stacked%2B2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="603" data-original-width="1066" height="181" src="https://1.bp.blogspot.com/-6J2EjbBDBpM/XpmUqAOT_TI/AAAAAAAADgo/P0vYPAhnHoc66ZjEgIB9IM1rkUDicrbIwCEwYBhgLKssDAMBZVoBXGFYyQleYJBi7K5Juv_csfm6nn0nV15twipeSb2xpD8rqZ-uGiVaXIqx0-eb3DqYmMnuVtEQpPT1G7AXTzUQWG-8HgLeDP_8oDypJCmStF1zJuMLh0YpkLMOlFnvAAdk6_D9I8MDyRkz078O8cWlfofFT1g2XUbbFXE_VK_oK4OUjZTBxNqBOh8tu_-geEFAdYnSEif1yLgOBl53Wd1yrqe0bju98axBQjSjavDdc-qbcA94EFthbvpgy1gD6_QoaK-cTTo9R3LLlsCyle70GsJ6ohOqAflNPbF04cE_SLyz3VxbRRhnX4Qq7IYlOCl--3cIvxbE9uRA_J6BCXkVaNyXG4bsF0hmsOBLU8hV5wPjSDoOA4cUQj6EyPUcA0lD4T2Mg7xy4fINk4oW5Mtj3vTojX_jxGa1lOMN0q_Px0SWaev8-UkTdZ2LI_f2GooAe8jfUfnXKmenfehxKPyhjtevPhSSdduHhyuqJxY1rzt7JwIW99V2_U-pHpn6EBBdP6dG6WmyNgshXFkiJzLGCKgpayvDANWcNScZAB8rYDEeOFB1ydOHLwMIIV76HQJ4KNb8gGc5qodLBBT6PXmmIzb5v5e6yMMWT5_QF/s320/Stacked%2B2.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Examples of horizontal bars as an alternative to pie charts</td></tr>
</tbody></table>
<br />
As my <a href="https://arxiv.org/abs/1908.00630" target="_blank">research has shown</a>, the horizontal bar does not always work as well as a pie chart. It is a valid option, and one that you could consider, but it should definitely not be the default.<br />
<br />
To summarize, if you are asking the question "how many segments", then you may be asking the wrong question. Always remember Redmond's Rules:<br />
<br />
<ul>
<li>Use the right visual encodings (and pie charts are a valid choice!)</li>
<li>Add labels and annotations to provide context to the user</li>
<li>SFW! Make sure that you are answering the business question</li>
</ul>
<br />
The last rule can be hard, because sometimes you don't actually know the questions that the users want to answer! In those circumstances, following a <a href="https://www.nngroup.com/articles/design-thinking/" target="_blank">Design Thinking methodology</a> will probably get you where you want to be.<br />
<br />
<hr />
As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<br />Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com0tag:blogger.com,1999:blog-2268159193748171866.post-31510719950315580402020-04-01T16:38:00.001+01:002020-04-01T16:38:46.947+01:00Exponential data and logarithmic scales<br />
I have to admit that when I first saw some of the recent data visualisations from the likes of the <a href="https://www.ft.com/coronavirus-latest" target="_blank">Financial Times</a> and the <a href="https://www.nytimes.com/interactive/2020/03/21/upshot/coronavirus-deaths-by-country.html" target="_blank">New York Times</a>, I wasn't an immediate fan. That is because they were using a logarithmic scale which distorts the data. My feeling was that they should be using a population based metric to compare different territories (XX per 100,000 is common).<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-NcJy52V2Z3M/XoSkocKiYrI/AAAAAAAADfM/bV5Rd_FU9VIeBceoMeKJ9dVs9jwMaOmcACLcBGAsYHQ/s1600/Exponential%2Bdata%2Band%2BLog%2BScale.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="749" data-original-width="1223" height="243" src="https://1.bp.blogspot.com/-NcJy52V2Z3M/XoSkocKiYrI/AAAAAAAADfM/bV5Rd_FU9VIeBceoMeKJ9dVs9jwMaOmcACLcBGAsYHQ/s400/Exponential%2Bdata%2Band%2BLog%2BScale.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Comparison of exponential data shown on a normal scale and on a logarithmic scale</td></tr>
</tbody></table>
A regular scale will have regular increments in the "Y" axis so if one point is twice as high as another, you can tell that it is twice the value. A logarithmic scale grows exponentially - generally log 2, so doubling on each equal size of increase (though the presentation usually rounds grid lines to 10s). If a point is twice as high as another, the higher value is the original value squared (e.g. 4 -> 16, 8 -> 64). It can be difficult for people to interpret, especially if they are not mathematical. In fact, I would suggest that it is almost impossible for most users to quickly tell the accurate difference in magnitude between different points - merely that one point is greater or lesser than another.<br />
<br />
There is a general situation where it is useful to use a log scale, and that is where there is some skew in the data. For example, where there is a mix of some very high and many lower values - such as with exponentially growing data. In that situation, the scale of the higher values can obscure the lower values.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-zgtJWEkrq5s/XoSkoeqaCkI/AAAAAAAADfQ/-E3xX5YT2Pci9uHSNpnDciXZNSLipimkQCEwYBhgLKssDAMBZVoByfmW54lNS1bnXhw2xEXEWsaIC1Dpv_EdRAxsszG7IGOQQBQoX1ek2jVywpIfe7Gy6OGVx1UhbOEJ0FFfrCykHMEr5Ti25ywDJgca9frdYgLL24zDBKStAHSJHzaPIMDB16MTCPRb_n7hC5WLZTM-Q2c9-mMCCERGAblCI3LI1XThBZlMPg-kqYesExzAUKBbqpfYlGsDPnlgPWkltIoC4Oi-XysN9ePdjXBnqMC96wW5q1tIvO-bBrMDarvyUwhr46jfcyFTAbZQYFF204y8mck6Ix0BUwyo-cWTHxtvGQkg3PZIRbwcQ2moPv1AeA9wiLvPV4K3fd6heAESP61tu1gdUodVzUQ5JwaK85-b5exKAg0YIz7FO1AY04Y6smGisweomQ_q0sydL9vE_owu69A2zm4AKR2Nt3la5XPViQlwU2O_BCOqNVGl7pl0ulgrkLjOHrni_qm8bn8SU8sc1aQ98IylisSrBwmZpQZWmCfp6id5CvhXgH-i0Hi-nDYDfYEIpksmwo6WhcYiY0AmdshhwpUYM51u370LID3XbeecqUBctGEVe8RxLu__AEN7X4IMgSxk9a2MZ0AalD6QD8KtoqQRFMJTXkvQF/s1600/20200330%2BUS%2BStates%2BNormal%2BScale.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="733" data-original-width="1341" height="217" src="https://1.bp.blogspot.com/-zgtJWEkrq5s/XoSkoeqaCkI/AAAAAAAADfQ/-E3xX5YT2Pci9uHSNpnDciXZNSLipimkQCEwYBhgLKssDAMBZVoByfmW54lNS1bnXhw2xEXEWsaIC1Dpv_EdRAxsszG7IGOQQBQoX1ek2jVywpIfe7Gy6OGVx1UhbOEJ0FFfrCykHMEr5Ti25ywDJgca9frdYgLL24zDBKStAHSJHzaPIMDB16MTCPRb_n7hC5WLZTM-Q2c9-mMCCERGAblCI3LI1XThBZlMPg-kqYesExzAUKBbqpfYlGsDPnlgPWkltIoC4Oi-XysN9ePdjXBnqMC96wW5q1tIvO-bBrMDarvyUwhr46jfcyFTAbZQYFF204y8mck6Ix0BUwyo-cWTHxtvGQkg3PZIRbwcQ2moPv1AeA9wiLvPV4K3fd6heAESP61tu1gdUodVzUQ5JwaK85-b5exKAg0YIz7FO1AY04Y6smGisweomQ_q0sydL9vE_owu69A2zm4AKR2Nt3la5XPViQlwU2O_BCOqNVGl7pl0ulgrkLjOHrni_qm8bn8SU8sc1aQ98IylisSrBwmZpQZWmCfp6id5CvhXgH-i0Hi-nDYDfYEIpksmwo6WhcYiY0AmdshhwpUYM51u370LID3XbeecqUBctGEVe8RxLu__AEN7X4IMgSxk9a2MZ0AalD6QD8KtoqQRFMJTXkvQF/s400/20200330%2BUS%2BStates%2BNormal%2BScale.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Ten US States growth shown on a normal scale. The higher value in one state hides detail in the other states. The dashed grey lines show example exponential growth patterns.</td></tr>
</tbody></table>
As an example, consider the chart above which shows growth patterns in several US states. All have a exponential type growth, but the higher values in New York makes it difficult to see the direction in detail of the smaller values. The scale needs to accommodate the high New York values, but most of the "action" in this chart it at the smaller values.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-u5Tp_jBuJLg/XoSkoYp4-5I/AAAAAAAADfU/5uKBLqItri0n48qhC9XVD3VUDyMYHgHoQCEwYBhgLKssDAMBZVoCTy9ZakHf18E21k6bVKDgq3VuMfr81U11Wlv--pTPctKnxJCVN2LzWVyOsvp1HMJWEntFlzGNbNd-qr4S9VOfNQLC5z_XtouqHowuzDjVEPCKRyo1bGiexkjkgoeuP5SRnuEwx9sQvdBMKsHBMdd4_PqVkGoLcDOTWyfUlcmonkrPA1uMrQiQTN0w34wlmrV50VxqhDfh1ANJ-OA9j4ZwmdHRuXPBXF5W5x992gpLlEGiQ9WyOZiZWRzS1Z081kJ9s41yfuVTcDer3pCgOE45tvNiXMx3l03d1KPU3s3Yevv-EBbs-HyeNi83UfltE6RMFk5LugpJsmOkt4Jcs7PP4EmDATM-1lUrz6n9vzb1sCCtq5JIUgB-xPxQOC3IKTI-GVyEn-GSC89QTu_eJC7uT6Vo4lzgYPjR_DAYaQ8KUq-SEUJZhE0JyVbg4G77UWqvmHH55B6TRE3vT1ZP1zLXhzuIFAGGDEJOqPysc8qV6CFHowgTZ4Loa2L5Eq7-yVs2Cp8Qw7AZ_nA9UiXp9zejPmUzRUu3BlQsRoil2LCLgTUfClp9-PNsxaMPZRwAt4Xp6ENsAw8XQoPm7djDlVAEI0Tlgnn9IMNXZkvQF/s1600/20200330%2BUS%2BStates%2BLog%2BScale.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="733" data-original-width="1341" height="217" src="https://1.bp.blogspot.com/-u5Tp_jBuJLg/XoSkoYp4-5I/AAAAAAAADfU/5uKBLqItri0n48qhC9XVD3VUDyMYHgHoQCEwYBhgLKssDAMBZVoCTy9ZakHf18E21k6bVKDgq3VuMfr81U11Wlv--pTPctKnxJCVN2LzWVyOsvp1HMJWEntFlzGNbNd-qr4S9VOfNQLC5z_XtouqHowuzDjVEPCKRyo1bGiexkjkgoeuP5SRnuEwx9sQvdBMKsHBMdd4_PqVkGoLcDOTWyfUlcmonkrPA1uMrQiQTN0w34wlmrV50VxqhDfh1ANJ-OA9j4ZwmdHRuXPBXF5W5x992gpLlEGiQ9WyOZiZWRzS1Z081kJ9s41yfuVTcDer3pCgOE45tvNiXMx3l03d1KPU3s3Yevv-EBbs-HyeNi83UfltE6RMFk5LugpJsmOkt4Jcs7PP4EmDATM-1lUrz6n9vzb1sCCtq5JIUgB-xPxQOC3IKTI-GVyEn-GSC89QTu_eJC7uT6Vo4lzgYPjR_DAYaQ8KUq-SEUJZhE0JyVbg4G77UWqvmHH55B6TRE3vT1ZP1zLXhzuIFAGGDEJOqPysc8qV6CFHowgTZ4Loa2L5Eq7-yVs2Cp8Qw7AZ_nA9UiXp9zejPmUzRUu3BlQsRoil2LCLgTUfClp9-PNsxaMPZRwAt4Xp6ENsAw8XQoPm7djDlVAEI0Tlgnn9IMNXZkvQF/s400/20200330%2BUS%2BStates%2BLog%2BScale.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Comparison of ten US states using a logarithmic scale. The trajectory lines are straightened and it is easier to see the trajectory of the states with lower values.</td></tr>
</tbody></table>
When the same data is presented on a logarithmic chart, all of the lines are straightened and we get a much better view of the trajectory of each state.<br />
I can now clearly see that Michigan's trajectory appears to be heading in a slightly worse direction than New York's. I am not concerning myself with how much farther ahead on the trajectory New York is, only the direction that they are both travelling and hence making mental forecasts about Michigan's future.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://1.bp.blogspot.com/-gP8vGh0okqk/XoSzz812GnI/AAAAAAAADfg/1YP53YyFfEcCcR66KkZX0nvGE3jed2ZiwCLcBGAsYHQ/s1600/Bar%2BChart%2Bwith%2BLog%2BScale.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="591" data-original-width="1248" height="188" src="https://1.bp.blogspot.com/-gP8vGh0okqk/XoSzz812GnI/AAAAAAAADfg/1YP53YyFfEcCcR66KkZX0nvGE3jed2ZiwCLcBGAsYHQ/s400/Bar%2BChart%2Bwith%2BLog%2BScale.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Bar chart with a logarithmic scale - don't do this kids! The log scale removes the comparative power of the bar chart.</td></tr>
</tbody></table>
BTW, I am good with using log scales like this for lines, but don't do it for bar charts! The effect of the logarithmic scale is to remove the power the the bar chart has of aiding our understanding of the difference in magnitudes. These differences are encoded by the length of the bar, a log scale will distort it. Don't do it!<br />
<br />
<hr />
Qlik Luminary, Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<br />Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com0tag:blogger.com,1999:blog-2268159193748171866.post-27498671049356420552019-08-05T22:01:00.000+01:002019-08-05T22:03:34.747+01:00Pie charts ain't such a bad guy!About 3 years ago I did some research using Amazon Mechanical Turk into how well people judge segments in a part-to-whole chart. <a href="https://www.qliktips.com/2016/05/pie-charts-and-perceptual-anchors.html" target="_blank">Mentioned it in a blog post</a> back then, but didn't go into too much detail.<br />
<br />
There had been some excellent work done by Robert Kosara (from Tableau Research) along with Dean Skau into Pie Charts, Doughnut Charts and various differences. Robert has continued this with a number of papers since.<br />
<br />
At the heart of my research was this nagging thinking that pervaded the data visualization ecosystem - pie charts are bad, never use pie charts. This was kinda, fine because I could do other stuff - especially segmented bar charts or tree maps, but I always had business users asking for pie charts and not really getting me when I tried to explain that they weren't the best way.<br />
<br />
In 2015 I had started into studying for a Masters Degree in Data Analytics so was starting to get back into the academic way of thinking and looking at stuff. When I had a break from studying during the summer of 2016 I started to look around and found that there was no real basis for anyone rejecting a pie chart for part-to-whole comparison, other than they didn't really like them! Because when it came to actually testing pie charts versus other types of charts, then the pies seemed to do as well or better than the alternatives.<br />
<br />
There was some suggestion in a number of papers that pie charts actually do better as they have a number of natural visual cues - at 0%, 25%, 50% and 75% - whereas the bar chart has definitive visual cues at 0% and 100% and a less well defined cue at 50%.<br />
<br />
So, being the curious person that I am, I decided to test things for myself. I put a little money into it and spun up an Amazon Mechanical Turk account. I created a number of images (using QlikView of course!), and had the "workers" judge the size of a segment in a chart. I used a set of "baseline" pie and bar (just standard pie and bar chart) and then a set of bar charts that had additional visual cues added.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-2P7wKlXSmCE/XUiRTDM5LBI/AAAAAAAADaE/O2vV13M7yaAuX8lNFp1qzh-Xsk6mWJi9ACLcBGAs/s1600/All_Mean_CI.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="311" data-original-width="1600" height="76" src="https://1.bp.blogspot.com/-2P7wKlXSmCE/XUiRTDM5LBI/AAAAAAAADaE/O2vV13M7yaAuX8lNFp1qzh-Xsk6mWJi9ACLcBGAs/s400/All_Mean_CI.png" width="400" /></a></div>
<br />
The chart above shows the comparison of mean absolute error recorded by participants (basically, how far off the mark were they with their estimations) and the 95% confidence intervals of those results.<br />
<br />
The baseline pie chart performed better than the baseline bar, even considering the confidence intervals. This was not a surprise as it confirmed the results of an experiment from 1915!<br />
The bar chart with a numerical scale as a visual cue performs the very best - and this aligns to what Stephen Few says in his famous paper, <a href="https://www.perceptualedge.com/articles/visual_business_intelligence/save_the_pies_for_dessert.pdf" target="_blank">Save the Pies for Dessert</a>.<br />
<br />
Of course, it is not always practical to have a numerical scale on a bar chart, and I have shown that adding a perceptible visual cue at the decile positions (every 10%) performs almost as well as the scale. Much better than the baseline bar as well as a bar with visual cues at the quartile positions (every 25%).<br />
<br />
Interestingly, adding a visual cue at the quartile positions for the pie chart did not improve its performance significantly over the baseline. With quartile cues not improving the performance over the baselines, it may indicate that we do indeed pick up on those cues automatically. More research needed here.<br />
<br />
The upshot here is to not feel bad about using a pie chart for part-to-whole comparison. No need to feel embarrassed at the next visualization meetup or to share it on an online forum! Be bold!!!<br />
The reality is back to my <a href="https://www.qliktips.com/2016/02/fundamental-rules-of-data-visualization.html" target="_blank">Fundamentals Rules of Visualization</a> <span style="background-color: white; color: #666666; font-family: "trebuchet ms" , "trebuchet" , "verdana" , sans-serif; font-size: 13.2px;">(or "Redmond's Rules") which are, in summary:</span><br />
<br style="background-color: white; color: #666666; font-family: "Trebuchet MS", Trebuchet, Verdana, sans-serif; font-size: 13.2px;" />
<span style="background-color: white; color: #666666; font-family: "trebuchet ms" , "trebuchet" , "verdana" , sans-serif; font-size: 13.2px;">- Use the right visual variables</span><br />
<span style="background-color: white; color: #666666; font-family: "trebuchet ms" , "trebuchet" , "verdana" , sans-serif; font-size: 13.2px;">- Provide context with annotations</span><br />
<span style="background-color: white; color: #666666; font-family: "trebuchet ms" , "trebuchet" , "verdana" , sans-serif; font-size: 13.2px;">- SFW - make sure that the results are relevant to the viewer</span><br />
<br />
I'm not the only one who is leaning in this direction and, as I blogged about previously, <a href="https://www.qliktips.com/2018/01/data-visualization-can-be-that-simple.html" target="_blank">visualization can be that simple</a>.<br />
<br />
I finally got round to writing up an academic paper on my research and the good news (for me!) is that is has been accepted into the Short Papers section of IEEE Vis 2019 in Vancouver. If you are interested, a <a href="https://arxiv.org/abs/1908.00630" target="_blank">pre-print is available on arXiv</a>.<br />
<br />
<hr />
Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="https://twitter.com/stephencredmond" target="_blank"><img alt=" @stephencredmond" border="0" data-original-height="28" data-original-width="28" height-="" src="https://1.bp.blogspot.com/-9tyALUCigb8/XUiYRmLZeUI/AAAAAAAADaY/acpVkEiJ3dUNLE845-6GYj8N0B8g1w2AACLcBGAs/s1600/Twitter_Small.png" width="28" /></a>
<a href="https://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt=" " height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-17823001556752193172018-01-25T21:15:00.002+00:002020-11-23T22:53:44.213+00:00Data Visualization can be that simpleTwo years ago I published my <a href="http://www.qliktips.com/2016/02/fundamental-rules-of-data-visualization.html" target="_blank">Fundamental Rules of Data Visualization</a> (or "Redmond's Rules") which are, in summary:<br />
<br />
- Use the right visual variables<br />
- Provide context with annotations<br />
- SFW - make sure that the results are relevant to the viewer<br />
<br />
It was interesting for me to listen to the <a href="http://datastori.es/113-what-makes-a-visualization-memorable-with-michelle-borkin" target="_blank">latest edition of Data Stories with interviewee Michelle Borkin</a>. From the opening snippet, Michelle is confirming the fundamentals:<br />
<br />
<i>"Put a title on your graph, annotate the important things, label your axes, pick appropriate visual encodings, ..., people will understand your visualizations"</i><br />
<br />
The basis of my "rules" come from own meandering experience as a practitioner, colored by the many subjective opinions that I have encountered. Michelle's advice comes from good old fashioned scientific experimentation - it is good to see that there is some convergence between the two!<br />
<br />
<a href="https://www.ccis.northeastern.edu/people/michelle-borkin/" target="_blank">Michelle is a Assistant Professor at Northeastern University</a>, where she works on visualization (among other things). Papers include <a href="http://ieeexplore.ieee.org/document/6634103/?arnumber=6634103" target="_blank">What Makes a Visualization Memorable?</a> and <a href="https://vcg.seas.harvard.edu/content/3-publications/20150101-beyond-memorability-visualization-recognition-and-recall/2015_beyond.pdf" target="_blank">Beyond Memorability: Visualization Recognition and Recall</a>, both worth reading for anyone interested in the area.<br />
<br />
Data visualization can be simple. Learn a few basics and work with your audience to design what suits them best.<br />
<br />
<hr />
As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-15224974592440346362017-10-26T23:53:00.000+01:002017-10-26T23:56:08.740+01:00Technical Debt in AnalyticsI was lucky enough to attend the <a href="https://spark-summit.org/eu-2017/" target="_blank">Spark Summit Europe</a> this week, held in the <a href="http://www.theccd.ie/" target="_blank">Convention Centre, Dublin</a> - a really good venue.<br />
<br />
One of the concepts that appeared in several presentations (to which, of course, Spark based solutions are a natural solution!) was the idea of <b>Technical Debt</b>. The image that accompanied all the presentations was taken from a paper entitled <a href="https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems" target="_blank">Hidden Technical Debt in Machine Learning Systems</a>, a paper authored by several Google employees:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-A6DO2sNfokg/WfJiFPHdlAI/AAAAAAAACtU/vdRNjIzJGk8XbRCVKuy-iSsUmaVE2m5cACLcBGAs/s1600/hidden-debt-machine-learning-systems-605.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="278" data-original-width="605" height="183" src="https://1.bp.blogspot.com/-A6DO2sNfokg/WfJiFPHdlAI/AAAAAAAACtU/vdRNjIzJGk8XbRCVKuy-iSsUmaVE2m5cACLcBGAs/s400/hidden-debt-machine-learning-systems-605.jpg" width="400" /></a></div>
<br />
The concept is very familiar to me from many years of selling QlikView. The debt there arises from the famous SIB - Seeing Is Believing (or just plain-old Proof-of-Concept) where we would go into a prospect company, take some of their data, hack together an impressive dashboard, and wow them with how quickly we could work our magic with this wonderful tool.<br />
<br />
The debt, of course, arose when the prospect turned into a customer and wanted the POC put into production!<br />
<br />
Eh, er, em, perhaps, oh... - that difficult conversation where we have to explain exactly how much work is needed to make this wonderful dashboard actually production ready.<br />
<br />
<b>Technical Debt</b> is not a new concept. It was described as far back as 1992 by Ward Cunningham, (founder of the famous Hillside Group, developer of Wiki, and one of the original signatories to the <b>Agile Manifesto</b>). It is unsurprising to find it described in Machine Learning. The extent of it may be a bit of a surprise.<br />
<br />
Taking on debt is something that a business may accept as it may lead to growth opportunity. However, the business needs to understand the terms of the debt before they agree to it. This Google paper is worth reading and understanding.<br />
<br />
Businesses need to understand that implementing "AI" and "Machine Learning" may lead to gold, but the debts will need to be paid. You wouldn't jump into a finance agreement without consulting an adviser, don't jump into analytics without talking to someone who know what they are talking about.<br />
<br />
<br />
<hr />
As well as holding a Master's Degree in Data Analytics, Stephen Redmond is a practicing Data Professional of over 20 years experience. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com0tag:blogger.com,1999:blog-2268159193748171866.post-68715619283823826752017-10-22T16:11:00.000+01:002017-10-22T16:11:05.363+01:00Using bipartite graphs projected onto two dimensions for text classificationIt has been an interesting last couple of years - which means that I have been very quiet on this blog! But things will start to get busier here again. Busier, but different. As I have expanded my interests in wider data analytics technologies, I have been building experiences that I will start to share.<br />
<br />
For much of the last two years I have been studying for a Master's degree in Data Analytics. This ended successfully and I am looking forward to being conferred with a H1 degree next month. My final project involved creating a new text classification method based on the bipartite relationship between words and documents but with, of course, a visual element, in that I have mapped the nodes of the bipartite graph onto two dimensions and then used QlikView to allow users to explore the model.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-mAzkkLggyik/WeytxcEdbFI/AAAAAAAACsU/7ZFeYdxvu2EYh8mknaAL5HFDRRVPKyWOACLcBGAs/s1600/Classifier%2B-%2BFinal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="559" data-original-width="572" height="312" src="https://2.bp.blogspot.com/-mAzkkLggyik/WeytxcEdbFI/AAAAAAAACsU/7ZFeYdxvu2EYh8mknaAL5HFDRRVPKyWOACLcBGAs/s320/Classifier%2B-%2BFinal.png" width="320" /></a></div>
<br />
There is a <a href="https://www.researchgate.net/publication/319940273_Using_bipartite_graphs_projected_onto_two_dimensions_for_text_classification" target="_blank">published paper</a> on the method that was presented at a recent conference in Zurich.<br />
<br />
The important thing to note here is that this wasn't just a QlikView project. The model was built using Python and Spark, making use of the <a href="https://databricks.com/" target="_blank">Databricks </a>platform. As such, it is reflective of the direction of my interests over the last while - I still like using QlikView and Qlik Sense, but I have been working more and more on Big Data analytics, and Spark has been an important component of that.<br />
<br />
I really like the the Big Data landscape right now - there are so many interesting things happening. I look forward especially to what is happening in visual analytics on Big Data. Companies such as <a href="https://www.arcadiadata.com/" target="_blank">Arcadia Data</a> and <a href="https://www.datameer.com/" target="_blank">Datameer </a>are doing interesting thinks there. Qlik are, of course, working on a <a href="https://community.qlik.com/blogs/qlikproductinnovation/2017/05/24/leave-data-where-it-is" target="_blank">Big Data index</a>, and that will be interesting to see when it comes out.<br />
<br />
In the data science area, there are so many good desktop tools, but less options for working with the likes of Hadoop. I really like the new <a href="https://www.cloudera.com/products/data-science-and-engineering/data-science-workbench.html.html" target="_blank">Cloudera Data Science Workbench</a> in this regard, to allow teams of data professionals to work on code projects in a secure and governed way. I think that we will see other products making moves in this direction. For more 4GL type data processing, <a href="https://rapidminer.com/" target="_blank">RapidMiner</a> and <a href="https://www.dataiku.com/" target="_blank">Dataiku</a> already work quite well with Hadoop. SAS has Hadoop connectivity, and some accuse them of having missed the Big Data boat, but they do have a forthcoming product called <a href="https://www.sas.com/en_in/software/viya.html" target="_blank">Viya</a> that also promises to run directly on Big Data and the Cloud.<br />
<br />
When I first started working with data, it was pretty much just SQL. Access was actually a pretty advanced data analysis tool, but was crippled with larger data sizes. When I look across the landscape now, it is hard not to be excited to see what will happen.<br />
<br />
<hr />
Stephen Redmond is a Data professional. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com0tag:blogger.com,1999:blog-2268159193748171866.post-40438313635108979872017-06-27T22:45:00.000+01:002017-06-27T22:45:31.970+01:00The visual paradigm of ETL tools<br />
<table border="1"><tbody>
<tr><td><b>Paradigm </b>(from <a href="http://www.dictionary.com/browse/paradigm" target="_blank">http://dictionary.com</a>):<br />
- a framework containing the basic assumptions, ways of thinking, and methodology that are commonly accepted by members of a scientific community.<br />
- such a cognitive framework shared by members of any discipline or group.</td></tr>
</tbody></table>
<br />
Following a recent demo of quite a well know data preparation tool, I was left thinking to myself, "well, that was confusing". The workflow itself was quite straightforward, in that it was an extraction of a reasonably straightforward dataset and then creating and evaluating a machine learning process. But there was just so much visual information on the screen, with so many icons, sub-processes and connections going all over the place, that it was just difficult to understand what was going on.<br />
<br />
So, took to <a href="https://www.linkedin.com/feed/update/urn:li:activity:6281813379828969472" target="_blank">LinkedIn</a> and <a href="https://twitter.com/stephencredmond/status/876047698981474304" target="_blank">Twitter </a>on the subject and asked:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-GOAKv5EgH_E/WVLECtp81oI/AAAAAAAACog/-0I7niDOo6o9tloxwuk2ENrXxNVaFE_jQCLcBGAs/s1600/Feed.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="230" data-original-width="557" height="165" src="https://3.bp.blogspot.com/-GOAKv5EgH_E/WVLECtp81oI/AAAAAAAACog/-0I7niDOo6o9tloxwuk2ENrXxNVaFE_jQCLcBGAs/s400/Feed.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
Quite a lot of comments were forthcoming, some of them quite interesting. I especially liked the one that suggested that the visual approach of one tool was essentially self-documenting.<br />
<br />
It isn't.<br />
<br />
The problem is that there is no shared paradigm about it. Well, there is a certain amount - for example, we tend to go left-to-right (until we don't) - but there is enough different options available to users to make one user's outputs very different to another's.<br />
<br />
Let's have a look at a very simple example from Pentaho Data Integration (you might recall that I wrote an eBook some time ago on using <a href="http://www.qliktips.com/2015/04/data-preparation-for-qlik-sense.html" target="_blank">Pentaho to prepare data for Qlik Sense</a>):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-kCVuh1S2L50/WVLECg01TNI/AAAAAAAACo0/Hz4S-O7MjLgglvHSxSFD4Ngxtzq0m6RcwCEwYBhgL/s1600/PDI1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="572" data-original-width="939" height="242" src="https://2.bp.blogspot.com/-kCVuh1S2L50/WVLECg01TNI/AAAAAAAACo0/Hz4S-O7MjLgglvHSxSFD4Ngxtzq0m6RcwCEwYBhgL/s400/PDI1.png" width="400" /></a></div>
<br />
Pentaho affords the user the option to have their flows going in whatever direction they want - up, down, left, right, diagonal - and flows can cross over. I can make as messy an interface as you want - although, hey, I can understand it and that is all that matters, right?<br />
<br />
Even on a system that enforces a left-to-right paradigm, for example RapidMiner, still allows the user a lot of freedom. For example, this data flow:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-xzD6OowMDGQ/WVLECigsdpI/AAAAAAAACo0/aSUIhm6wAUoUcV9JwatohMp5QtWhBPNmQCEwYBhgL/s1600/RM1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="388" data-original-width="961" height="161" src="https://4.bp.blogspot.com/-xzD6OowMDGQ/WVLECigsdpI/AAAAAAAACo0/aSUIhm6wAUoUcV9JwatohMp5QtWhBPNmQCEwYBhgL/s400/RM1.png" width="400" /></a></div>
<br />
This is nice and simple, flows from left-to-right. Looks great, right? But what about now:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-fPmt0XdGzAo/WVLEC3W8k9I/AAAAAAAACo0/rIN16a5FPRwxnjuCjlQztCcB-fuFvaFIQCEwYBhgL/s1600/RM2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="376" data-original-width="962" height="156" src="https://2.bp.blogspot.com/-fPmt0XdGzAo/WVLEC3W8k9I/AAAAAAAACo0/rIN16a5FPRwxnjuCjlQztCcB-fuFvaFIQCEwYBhgL/s400/RM2.png" width="400" /></a></div>
<br />
Functionally, it's the exact same flow, but visually different enough from the first so as to look like a different flow to different users. How about now:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-IdtFJR4t52E/WVLEC0Wgb7I/AAAAAAAACo0/aGlB2ZJeT-kS00UmBXlo8CMWj34ULA_JACEwYBhgL/s1600/RM3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="376" data-original-width="961" height="156" src="https://3.bp.blogspot.com/-IdtFJR4t52E/WVLEC0Wgb7I/AAAAAAAACo0/aGlB2ZJeT-kS00UmBXlo8CMWj34ULA_JACEwYBhgL/s400/RM3.png" width="400" /></a></div>
<br />
Again, it is the same flow, just with processes grouped. Most of the ETL tools will allow us to "tidy" the display by grouping multiple icons and flows into a sub-process. Different users may group in different ways.<br />
<br />
Of course, when you write scripts, then you are even more free to do what you will. We can name variables whatever way we want. We can create sub routines and functions, classes and methods (depending on the language!), whatever we want. However, it does seem, and maybe this is just me, to be somewhat more controllable.<br />
<br />
Script has a top-to-bottom flow. Even when using lots of functions, within those functions the code always flows from top-to-bottom. The syntax of the language is itself a constraint that enforces something that is more readable. Because the code is essentially structured text, we can even automate an enforced coding standard - including commenting.<br />
<br />
This ability to automate the coding standard is actually a strength derived from many years of paradigm building. Scripting, in whatever language, has paradigms that developers quickly learn to follow.<br />
<br />
Over time, the visual tools may develop those paradigms, but I am not sure that they can.<br />
<br />
<br />
<hr />
Stephen Redmond is a Data professional. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-83165790119558123752017-01-02T15:36:00.001+00:002017-01-02T15:36:09.354+00:00Hue, Saturation and LuminosityColour is an important variable to consider when designing a visualisation. A lot of Qlik developers, if we think of it at all, will think of colour in terms of a mix of red, green and blue - each of them usually as a numeric value between 0 and 255. A lot of web developers will think hexadecimally - #00 to #ff, with the R/G/B being expressed as a hex number like #00df87.<br />
<br />
There is another way to think of colour, especially when thinking about how to represent scales, and that is to consider hue, saturation and luminosity.<br />
<br />
Hue is what a lot of people will think of when they think of colour - almost the pure wavelength of the light spectrum, running from red to green to blue:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-Fq36Ay_jpk4/WGpmgp9ASJI/AAAAAAAACeg/iW8Vmkv5yT017hKVtWw6bxntGURmLUmqgCEw/s1600/HSL3.ColourBar.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="30" src="https://2.bp.blogspot.com/-Fq36Ay_jpk4/WGpmgp9ASJI/AAAAAAAACeg/iW8Vmkv5yT017hKVtWw6bxntGURmLUmqgCEw/s400/HSL3.ColourBar.png" width="400" /></a></div>
<br />
But is is actually a loop, because the blue runs back through to red again. Perhaps it is easier to represent as a circle (indeed, the CSS hsl function uses a value between 0-359, representing degrees on the colour wheel):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-cpnZQHHxbgk/WGpmghWIaGI/AAAAAAAACec/Np0oYQpxDcI439htvs4W1xs-hMYMqInbgCLcB/s1600/HSL2.ColourWheel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="199" src="https://3.bp.blogspot.com/-cpnZQHHxbgk/WGpmghWIaGI/AAAAAAAACec/Np0oYQpxDcI439htvs4W1xs-hMYMqInbgCLcB/s200/HSL2.ColourWheel.png" width="200" /></a></div>
<br />
In QlikView, the HSL function takes a value between 0 and 1 for the hue. 0 is pure red, 0.33 is pure green and 0.67 is pure blue.<br />
<br />
A changing hue is used by some designers to represent a scale - the so-called "rainbow scale". However, this is wrong on a number of levels. Not least of these is that there is no well accepted norm to say that red is low while blue is high and green is in the middle. Of course, we also have to remember the we need to design visualisations that may be used by people with colour blindness. Therefore, if you are representing a single climbing or falling scale, you should really just stick to a single hue value. If you are creating a diverging scale, then two hue values can be used.<br />
<br />
Saturation means the level of saturation of the hue relative to grey - how much colour is there. This can be seen in the standard Microsoft colour picker:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-srQ6GzIUcuM/WGpmghagjCI/AAAAAAAACeo/YOl2nCqr5IInmxd51ro-6G1Byp8Laf-2QCEw/s1600/HSL1.ColourPicker.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="349" src="https://3.bp.blogspot.com/-srQ6GzIUcuM/WGpmghagjCI/AAAAAAAACeo/YOl2nCqr5IInmxd51ro-6G1Byp8Laf-2QCEw/s400/HSL1.ColourPicker.png" width="400" /></a></div>
<br />
So, we can see that, for each hue, the less saturated then the more grey. Very low saturation for any hue will effectively mean just grey. So, saturation is potentially useful to represent a scale - with a single hue (for example, green):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-1wN4Xgmj1bU/WGpsIzn36lI/AAAAAAAACe0/yfyLyTUDG-cMo2dcNBUyn87LA7ai8PLBwCLcB/s1600/HSL3.GreenSaturation.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="30" src="https://4.bp.blogspot.com/-1wN4Xgmj1bU/WGpsIzn36lI/AAAAAAAACe0/yfyLyTUDG-cMo2dcNBUyn87LA7ai8PLBwCLcB/s400/HSL3.GreenSaturation.png" width="400" /></a></div>
<br />
One thing that we should be aware of is that it is not possible for us to see subtle differences in the saturation, so it is always better to have a stepped scale, with 10 steps being an absolute maximum (<a href="http://colorbrewer2.org/">colorbrewer2.org</a> uses 9 as a maximum for this!):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-o8JYJcqGiNw/WGpsIwbvVJI/AAAAAAAACe4/HzdVxPDkeG4PXGNoFpT2Q7Mwq0xDSeVlQCEw/s1600/HSL3.GreenSaturation_Step.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="30" src="https://4.bp.blogspot.com/-o8JYJcqGiNw/WGpsIwbvVJI/AAAAAAAACe4/HzdVxPDkeG4PXGNoFpT2Q7Mwq0xDSeVlQCEw/s400/HSL3.GreenSaturation_Step.png" width="400" /></a></div>
<br />
Luminosity defines the levels of light that are emitted. We need to be careful here because this is often confused with brightness. However, luminosity is something that can be objectively measured but, like saturation, brightness is a subjective human measure. We can use luminosity as a scale:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-B2OYU-aa1So/WGpuSCOxMEI/AAAAAAAACfE/bKReWcTaKEszEmf1eUuUfuM5TiUbnxpAwCLcB/s1600/HSL4.Luminosity.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="30" src="https://3.bp.blogspot.com/-B2OYU-aa1So/WGpuSCOxMEI/AAAAAAAACfE/bKReWcTaKEszEmf1eUuUfuM5TiUbnxpAwCLcB/s400/HSL4.Luminosity.png" width="400" /></a></div>
<br />
As with saturation, we should consider using a stepped scale:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/-yXqNXRGc8uw/WGpuSNMI6JI/AAAAAAAACfI/pgr64UpYLWENzuGMDI8bhN4obZOEj87eACLcB/s1600/HSL4.Luminosity_Step.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="30" src="https://2.bp.blogspot.com/-yXqNXRGc8uw/WGpuSNMI6JI/AAAAAAAACfI/pgr64UpYLWENzuGMDI8bhN4obZOEj87eACLcB/s400/HSL4.Luminosity_Step.png" width="400" /></a></div>
<div>
<br /></div>
So, why would we worry about HSL? Because they are easily programmable! In both Qlik (all of the images here are built in QlikView using the HSL() function) and web/css technologies, there is a HSL colour function that will accept a hue, saturation and luminosity value. Even better, in both cases, the saturation and luminosity values are represented by percentages - which are ideal for calculating scales.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://4.bp.blogspot.com/-3deFVkMaPkI/WGpx7uUUJjI/AAAAAAAACfU/AzgJZeee3SgqgSnd4tc38iSaDzhbf3lFwCLcB/s1600/HSL5.Diverging.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="23" src="https://4.bp.blogspot.com/-3deFVkMaPkI/WGpx7uUUJjI/AAAAAAAACfU/AzgJZeee3SgqgSnd4tc38iSaDzhbf3lFwCLcB/s400/HSL5.Diverging.png" width="400" /></a></div>
<br />
<br />
<hr />
Stephen Redmond is a Data Visualization professional. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com2tag:blogger.com,1999:blog-2268159193748171866.post-81080115766166675852016-05-31T00:55:00.004+01:002016-07-28T22:20:51.751+01:00Pie charts and perceptual anchorsThere have been some information on a couple of really excellent research papers on pie charts released today from Robert Kosara (<a href="https://twitter.com/eagereyes" target="_blank">@eagereyes</a>) and Drew Skau (<a href="https://twitter.com/SeeingStructure" target="_blank">@SeeingStructure</a>) :<br />
<br />
<a href="https://eagereyes.org/papers/a-pair-of-pie-chart-papers">https://eagereyes.org/papers/a-pair-of-pie-chart-papers</a><br />
<br />
I had, coincidentally, been doing some research of my own on perceptual anchors and how that relates to performance of pie charts versus stacked bar charts in part-to-whole comparisons. It would suggest that pies are not the terrible bad-guy after all. Who knew!<br />
<br />
<br />
<hr />
Stephen Redmond is a Data Visualization professional. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com1tag:blogger.com,1999:blog-2268159193748171866.post-87148890611217522652016-02-21T18:38:00.000+00:002016-02-21T18:39:44.903+00:00Fundamental rules of data visualizationThere are many "rules" of data visualization that we read in many publications. Some contradict others and some just don't make any sense. Some are accompanied by extensive amounts of <a href="http://www.amazon.com/Proofiness-Youre-Being-Fooled-Numbers/dp/0143120077" target="_blank">proofiness</a>, but often is missing appreciation of the fundamentals. I can use algebra to prove to you that 1+1=1, using perfectly legitimate algebraic transformations, but it is invalid because it breaks a fundamental rule (for those who are interested, I will add it at the end of the post).<br />
<br />
I like to preach three fundamental rules of data visualization to those who will listen:<br />
<br />
<b><span style="font-size: large;">1. Data visualization is all about ratios</span></b><br />
This is so fundamental as almost ridiculous to have to mention, but we need to mention it. Any visualization that seeks to juxtapose several values for interpretation must do so using some kind of visual ratio.<br />
<br />
There are many kinds of visual ratios and some are more effective than others. Cleveland and McGill (1984) gave us the order of effectiveness of interpretation for these ratios:<br />
<br />
<ul>
<li>Position on a common scale</li>
<li>Position on non-aligned scales</li>
<li>Length</li>
<li>Direction</li>
<li>Angle</li>
<li>Area</li>
<li>Volume</li>
<li>Curvature</li>
<li>Shading</li>
<li>Color saturation</li>
</ul>
<br />
To try and create a data visualization that is not based on some kind of visual ratio is a fundamentally flawed approach. Every ratio is not always appropriate for every visualization either, so we need to learn about what works where.<br />
<br />
<span style="font-size: large;"><b>2. Data visualization is all about context</b></span><br />
We can create the most wonderfully beautiful bar charts and present them on a large screen in Times Square or print them on the most opulent paper in the most vivid colors, but without context they are just rectangles.<br />
Context devices will include such simple elements as titles and axes - enough annotation so as to allow the reader to understand exactly what they are looking at.<br />
As Amanda Cox, Graphics Editor at the New York Times, said in her Eyeo Festival talk:<br />
<br />
<div style="text-align: center;">
<i>The annotation layer is the most important thing we do... otherwise it's a case of here it is, you go figure it out.</i></div>
<br />
<b><span style="font-size: large;">3. Data visualization is about SFW</span></b><br />
This is the most important thing from a business point of view - and good data visualization is about creating a good solution for the business. SFW stands for <i><b>So What</b></i>.<br />
I will always remember the day when I had spent hours on a great dashboard to present to a board-level executive at one of our most important clients. It was technically awesome! Really pushing the boundaries of what the tool could do.<br />
I proudly showed it off at the executive presentation. My client sat patiently through it until, finally, he looked me straight in the eye and said:<br />
<br />
<div style="text-align: center;">
<i>So f***ing what?</i></div>
<br />
He was right of course. My technically advanced dashboard had a huge fundamental flaw - I had failed to connect it correctly to the business problem. It wasn't a good solution at all - except in my head.<br />
Fundamentally, we need to make sure that our data visualizations connect with the audience that they are intended for. The first two rules give us the correct technical result, the last gives us the brilliant business solution.<br />
<br />
We can create some great business solutions by following these three rules. They may not look great, they may have garish colors, but if the CEO is able to use them to track his business then that is a very good dashboard.<br />
<br />
To achieve glory among your peers, you need to start going beyond the fundamentals. Learn what works and what doesn't in most situations. Know when you should use a pie chart and when you shouldn't. Learn how to lay things out. Learn the best colors to use. This does lead to a fourth rule that could be considered fundamental:<br />
<br />
<span style="font-size: large;"><b>4. Get out of the way and show the numbers</b></span><br />
We don't talk about all the color and layout stuff for the good of our health. There are good reasons for doing things in the ways that you will read about in the books. Learn about the reasons for good consistent layout, easy on the eye colors and clean presentation.<br />
Above all, learn that if we don't follow the fundamentals then we start to potentially obscure the data, and this is a flaw that is important to correct.<br />
Get out of the way and show the numbers.<br />
<br />
<br />
For those that are interested, 1 + 1 = 1:<br />
<div style="font-family: courier; font-weight: bold; text-align: center;">
<br /></div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
a = b = 1</div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
<br /></div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
a = b</div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
<br /></div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
a^2 = ab</div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
<br /></div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
a^2 - b^2 = ab - b^2</div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
<br /></div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
(a + b)(a - b) = b(a - b)</div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
<br /></div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
a + b = b</div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
<br /></div>
<div style="font-family: courier; font-weight: bold; text-align: center;">
1 + 1 = 1</div>
<div style="font-family: Courier; font-weight: bold;">
<br /></div>
<div style="font-family: Courier; font-weight: bold;">
<br /></div>
<hr />
Stephen Redmond is a Data Visualization professional. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-37615479518857695362016-02-02T22:52:00.000+00:002016-02-02T22:54:28.951+00:00How to lie with charts - crude oil versus retail gasoline pricesAfter watching a news item this morning, I posted the following question to social media:<br />
<br />
<span style="background-color: white; color: #292f33; font-family: "arial" , sans-serif; font-size: 26px; letter-spacing: 0.26px; line-height: 32px; white-space: pre-wrap;">If oil has dropped from > $100 / barrel to < $30, why are consumers still paying > €1 / litre?</span><br />
<br />
There were some interesting responses. There was in my mind a suspicion that the retail prices were not coming down as quickly as the crude prices - but I had nothing to back that up with. I decided to investigate.<br />
<br />
Taking crude oil prices from <a href="http://www.eia.gov/dnav/pet/pet_pri_spt_s1_d.htm" target="_blank">US Energy Information Administration</a> and monthly retail price data from <a href="http://www.theaa.ie/AA/Motoring-advice/Petrol-Prices.aspx" target="_blank">AA Ireland</a>, I put the two together quickly in QlikView. I decided to fix the time period to January 2010 to January 2016, as the last time the Irish government added an additional excise duty to fuel was in December 2009, so I knew that wouldn't interfere with the figures.<br />
<br />
I plotted the data on a time series and, Aha!:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-yj-5kMUJyGU/VrEtrBDzeWI/AAAAAAAACNE/p2ZaDJ6eHII/s1600/OilPrice1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="156" src="http://4.bp.blogspot.com/-yj-5kMUJyGU/VrEtrBDzeWI/AAAAAAAACNE/p2ZaDJ6eHII/s400/OilPrice1.png" width="400" /></a></div>
<br />
"Black and white!", I thought to myself. How obvious. While the crude price has been dropping like a stone, the retail price has had a much gentler decent. I better get straight onto the press to reveal the petrol companies evil intent towards the good people of Ireland.<br />
<br />
But wait! There is a real problem here. The problem is that we have started both axes at zero - which is usually a sacrosanct rule. However, in this case, because we are not comparing the same value ranges, it is actually a mistake. By forcing both ranges into one area, I am actually distorting both of them.<br />
<br />
In QlikView, the fix is simple, we just take off the force zero option for both expressions, revealing a much different state of affairs:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-e8mgm0-kVus/VrEtrKW5ANI/AAAAAAAACNM/0MtnGcET5Lc/s1600/OilPrice2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="156" src="http://4.bp.blogspot.com/-e8mgm0-kVus/VrEtrKW5ANI/AAAAAAAACNM/0MtnGcET5Lc/s400/OilPrice2.png" width="400" /></a></div>
<br />
The crude and retail prices have actually been varying in a very similar way over the period. If I calculate the Pearson's correlation coefficient for these two series, it comes out at aproximately .77 - which is generally considered a high correlation for this type of data. In fact, if I drill into the last couple of years, the correlation is even tighter:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-a_xeDhAIaSg/VrEtrDpMQ3I/AAAAAAAACNM/em5B8X5VdkY/s1600/OilPrice3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="156" src="http://2.bp.blogspot.com/-a_xeDhAIaSg/VrEtrDpMQ3I/AAAAAAAACNM/em5B8X5VdkY/s400/OilPrice3.png" width="400" /></a></div>
<br />
The correlation coefficient for the last 25 months data calculates at approximately .95!<br />
<br />
Any data scientists in the room might be tempted to normalize the data (calculating the z-scores) so that we can plot them on the same axis. When we do, we get a similar view to the one above:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-fBcBnBV9x_U/VrEx5Vp8xTI/AAAAAAAACNU/4Wey4ss1B_U/s1600/OilPrice4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="156" src="http://1.bp.blogspot.com/-fBcBnBV9x_U/VrEx5Vp8xTI/AAAAAAAACNU/4Wey4ss1B_U/s400/OilPrice4.png" width="400" /></a></div>
<br />
And here is an example in Qlik Sense Cloud:<br />
<br />
<iframe frameborder="0" height="515" src="https://charts.qlikcloud.com/56b13189a8b55c110086392a/chart.html" width="620"></iframe>
<br />
So, perhaps the oil companies are playing a straight bat on this one. There are many different variations into what goes into the retail price of a litre of fuel. The crude oil price is one of those, but quite significant. If we can see a good correlation between the two, then we can have some sense of confidence that all is operating fairly.<br />
<br />
The main point here though is that it is quite easy in a lot of visualisation tools to accidentally tell the wrong story. You may have best intentions, but you may end up telling visual lies.<br />
<br />
Be careful out there!<br />
<br />
<hr />
Stephen Redmond is a Data Visualization professional. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.comtag:blogger.com,1999:blog-2268159193748171866.post-73602253710295613682016-01-29T22:49:00.001+00:002016-01-29T22:54:20.104+00:00CRISP-DM for Data Viz projectsDo you have a methodology for implementing Data Visualization projects?<br />
<br />
How do you go about working with your stakeholders to deliver value?<br />
<br />
The conception of CRISP-DM is 20 years old this year. It was conceived of as a process to formalize data mining (Cross Industry Standard Process for Data Mining) but if we have a look at the diagram below, it really fits for data visualization too!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://upload.wikimedia.org/wikipedia/commons/b/b9/CRISP-DM_Process_Diagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://upload.wikimedia.org/wikipedia/commons/b/b9/CRISP-DM_Process_Diagram.png" width="319" /></a></div>
<br />
When do we not do all of these steps in a data visualization project? If you are not doing them, why not? I'm OK if you don't, as long as you know why you are not.<br />
<br />
It is definitely worth a data visualization practitioners while to review the documentation - much of it freely available online (start with <a href="ftp://ftp.software.ibm.com/software/analytics/spss/support/Modeler/Documentation/14/UserManual/CRISP-DM.pdf" target="_blank">CRISP-DM 1.0 Step-by-step data mining guides</a>)<br />
<br />
<hr />
Stephen Redmond is a Data Visualization professional. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com0tag:blogger.com,1999:blog-2268159193748171866.post-13347873505698135372015-08-10T21:35:00.000+01:002015-08-10T21:35:59.771+01:00Data ExperienceDo people believe in the data more if they are holding something in their hand? Do they, literally, give it more weight? According to recent Dutch research, it just might.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-85QgzQ7NMx8/Vcj4GjMTh7I/AAAAAAAACIs/It2WT9L_ex8/s1600/iPad_QlikSense.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="228" src="http://4.bp.blogspot.com/-85QgzQ7NMx8/Vcj4GjMTh7I/AAAAAAAACIs/It2WT9L_ex8/s400/iPad_QlikSense.jpg" width="400" /></a></div>
<br />
My interest was piqued by the recent episode of the <a href="http://datastori.es/data-stories-58-domestic-data-streamers/" target="_blank">Data Stories podcast</a> (interestingly, they are back to being sponsored by Qlik again) where they interviewed <a href="https://www.linkedin.com/pub/dani-pearson/87/803/b3" target="_blank">Dani Llugany Pearson</a> from <a href="http://domesticstreamers.com/" target="_blank">Domestic Data Streamers</a>.<br />
<br />
Domestic Data Streamers create some wonderful installations, transforming data into art. These are installations that people can interact with and influence by adding to the data. It is a really marvelous concept. People can see, walk around, touch, and engage with data. Dani described it as an <b>Info Experience</b> as opposed to a static info graphic.<br />
<br />
During the discussion, some recent research from the Netherlands was mentioned and they kindly shared the link to the research in the show notes. This research, performed by Nils B Jostmann, Daniël Lakens and Thomas Schubert, shows that people holding a heavier weight will effect cognition and lead people to assign more importance. It is an intriguing idea and you can read the research here:<br />
<br />
<a href="http://www.researchgate.net/publication/26747401_Weight_as_an_embodiment_of_importance" target="_blank">Weight as an embodiment of importance</a><br />
<br />
So, if we attach more importance to more weight, do we see data visualizations on an iPad as being more important than the same data on an iPhone? Intriguing! How does that data seem on a desktop computer, where the only weight is in the mouse that we slide it across the desk?<br />
<br />
Weight, of course, is only one facet of our data experience. The visuals must be important too, just like the research that shows that people eat less when they can't see food, they just don't enjoy it as much:<br />
<br />
<a href="http://www.ncbi.nlm.nih.gov/pubmed/11836454" target="_blank">Vision and eating behavior. (2002. Linné Y, Barkeling B, Rössner S, Rooth P.)</a><br />
<br />
How about being able to touch and interact with the data? How does that make me feel about it?<br />
<br />
In their 2011 paper, David Spiegelhalter et al discuss the ethical imperative to provide transparent information. This is because when we build dashboards for other people, "the desired outcome must be considered from the start" - we have to think about what we are trying to present before we design the dashboard. We are more persuading than informing where we should be more informing:<br />
<br />
<a href="http://www.sciencemag.org/content/333/6048/1393" target="_blank">Visualizing Uncertainty About the Future (2011. Spiegelhalter, Pearson, Short)</a><br />
<br />
So, a visualization tool that I can hold in my hand and feel the importance of the data, that looks good enough to eat, and allows me to inform myself rather than being persuaded by someone else would be the ideal <b>Data Experience</b>. I wonder where I could get one of those?<br />
<br />
<hr />
Stephen Redmond is a Data Visualization professional. He is author of <a href="http://bit.ly/1F14EPY" target="_blank">Mastering QlikView</a>, <a href="http://bit.ly/1hpNwcU" target="_blank">QlikView Server and Publisher</a> and the <a href="http://bit.ly/13AFWcO" target="_blank">QlikView for Developer's Cookbook</a>
<br />
<a href="http://twitter.com/stephencredmond" target="_blank">
<img alt="Follow me on Twitter" height="26" label="Follow me on twitter" src="https://g.twimg.com/Twitter_logo_blue.png" width="27" /></a>
<a href="http://ie.linkedin.com/in/stephenredmond/" target="_blank">
<img alt="LinkedIn" height="26" src="https://static.licdn.com/scds/common/u/images/logos/linkedin/logo_in_nav_44x36.png" title="LinkedIn" width="28" />
</a>
Stephen Redmondhttp://www.blogger.com/profile/10815476951939159307noreply@blogger.com1