A quick performance comparison with Qlik Sense – AWS EC2 vs Azure Virtual Machines

Previously, I tested the performance of a load script while using RecNo() and RowNo() functions. This conveniently gave me a script which consumes up to 25GB of RAM, along with considerable CPU power.

So, what about testing it on two cloud boxes? I’ve chosen a machine from both AWS and Azure, loaded them with Qlik Sense September 2018 and run the load script.

Total Test Duration by Host

The summary: The AWS box was approx 8% faster than the Azure box.

Continue reading “A quick performance comparison with Qlik Sense – AWS EC2 vs Azure Virtual Machines”

Comparing Autonumber, Autonumberhash128, Autonumberhash256, Hash128, Hash160 and Hash256 outputs in Qlik Sense and QlikView

There’s often a discussion about what each of these autonumber/hash functions does in Qlik. We commonly see these functions used for creating integer key fields, anonymising data (for example, names in demo apps), and maintaining long string fields for later comparison (as the hashes are shorter than the strings they replace).

Sample outputs from the random number generator, with all the functions present

To do this, I’m using the script below. I’m also keen to show outputs from QlikView vs Qlik Sense, and results of running the same script on another machine.

My observations are the following:
AutoNumber/AutoNumberHash128/256 – different output per load as the value is typically based on the load order of the source data
Hash128/160/256 – the same output, across every load. Stays the same between Qlik Sense and QlikView, and also between different machines

Continue reading “Comparing Autonumber, Autonumberhash128, Autonumberhash256, Hash128, Hash160 and Hash256 outputs in Qlik Sense and QlikView”

Qlik load performance with RecNo() and RowNo()

Using RecNo() or RowNo() will impart a performance impact on your load script. I discussed these functions in a previous post where I looked at the output of RecNo vs RowNo.

I recently spotted an unexpected slow-down in a load script, which was caused by using one of these functions. In summary:
– Using RowNo() in a simple load script is considerably slower than RecNo()
– If you must use RecNo(), it may be faster to do this in a direct load
– If you must use RowNo(), it may be faster to do this in a resident load

Example script for one of the tests – load data from disk and add the RowNo

 

Continue reading “Qlik load performance with RecNo() and RowNo()”

VirtualBox on Windows 10: This 64-bit application couldn’t load because your PC doesn’t have a 64-bit processor

Sometimes VirtualBox doesn’t behave when it imports Virtual Machines (appliances).

I exported a Windows 10 VM from one machine to another, imported and ran it to receive a recovery message on launch.

My W10 VM isn’t happy

Continue reading “VirtualBox on Windows 10: This 64-bit application couldn’t load because your PC doesn’t have a 64-bit processor”

Compacting Windows 10 VMs and shrinking their VDI/VMDK disk images on VirtualBox host

If you’ve got a couple of VMs, then trimming them occasionally will help with storage management on the host. Disk files (like VDI and VMDK images) will grow over time if the VM was configured to expand the disk as needed – but they will never shrink on their own.

This was tested on a Windows 10 VM managed by VirtualBox, the starting file size was over 40GB, as it had been used for various roles and grown over time.

Our test VM - starting size
Our test VM – starting size

Continue reading “Compacting Windows 10 VMs and shrinking their VDI/VMDK disk images on VirtualBox host”

Qlik Counter Functions and their outputs – RecNo() and RowNo()

In this post I explore the outputs of RecNo() and RowNo() to demonstrate the difference visually.

These two fields are often used interchangeably, but they provide different output. In summary:
– RecNo() is a record number based on source table(s)
– RowNo() is a record number based on the resulting table

As a result, RowNo will always provide a unique integer per row in an output table, while RecNo is only guaranteed to be unique when a single source is loaded (RecNo is based on each single source table, interpreted individually rather than collectively).

A snapshot of the test output

Continue reading “Qlik Counter Functions and their outputs – RecNo() and RowNo()”

Passing variables to applications via the QlikView QMC with Script Parameters and Publisher

Following this post which talks about methods for configuring apps, this post looks at how to achieve this via the QMC in QlikView and set variables in reloaded QVWs.

You’ll need to have Publisher enabled to access the features used for this example.

In a QlikView deployment with Publisher you’ll have access to the Reload tab for a task. This includes the option to add Script Parameters, which essentially leads to duplicate apps based on the provided field or variable name.

However – provide just one variable value, and you’ll pass this into one copy of the app.

Continue reading “Passing variables to applications via the QlikView QMC with Script Parameters and Publisher”

Doing your Qlik App Configuration in the load script, include files, or QMC (Publisher or the Qlik Repository Service APIs)?

For me, the bare minimum when it comes to app configuration in Qlik Sense or QlikView is:
– Setting paths and libs for use in data load script
– Setting “boilerplate” variables – e.g. the number/date/currency/etc formats
– Setting HidePrefix, SearchIndex and other app-specific behaviours
– Setting Product or Customer specific settings for consumption in the UI

This post adds some detail on config methods for Qlik Sense and QlikView, which fills the second tab from an earlier post about versioning.

 

Script for setting a base library based on config variables
Script for setting a base library based on config variables

 

In a multi-product/customer setup (see this previous post about deployment frameworks for context) where we follow common standards for names and folder structures we can streamline this configuration to reduce the maintenance and deployment burden.

Continue reading “Doing your Qlik App Configuration in the load script, include files, or QMC (Publisher or the Qlik Repository Service APIs)?”

Version sheets and tabs in the load script for Qlik Sense and QlikView

In the absence of a source control system like SVN or GIT, a quick and easy way of capturing changes in a app is to update a version control tab before passing the app back to other developers, or onto testers.

This is a very low-tech solution, but the format below works well in shared environments. The first two tabs of your application should be:
– Version (explain the app and list changes you’ve made)
– Config (set the configuration values like paths, dates, etc used in your application’s data load and UI)

Version and Config sheets will become a familiar sight
Version and Config sheets should become a familiar sight

Continue reading “Version sheets and tabs in the load script for Qlik Sense and QlikView”