The results (when assessing ONLY PostHistory – the largest input file), with the exception of tests 6, 8 and 10 (all store operations on data originally loaded from a QVD), show that enabling encryption for QVDs increases load time, and enabling both QVD and QVF encryption increases this further.
No surprises there.
I’ll look into this in more depth in a follow up post.
Observation on QVD file size
There was no noticeable increase in QVD file size following encryption – see screenshots of before and after below.
Considerations for next time
Instead of using a burstable instance (B4ms), I should have used a general instance such as a DS3 to ensure a baseline level of performance
The server size was likely too large for the smallest data set I used, meaning that operations completed too quickly for any variation to be meaningful, while Posts and PostHistory were more suitable
This time, I used Azure files for the primary read/write location. While we should assume performance remains consistent over time, testing with a provisioned disk attached to the VM would be a better test to remove any potential variability
Services were not restarted between every test, only between test modes (i.e. encrypted, unencrypted) – it would be a better control to begin all tests following a restart of at least the engine service
The initial test created the QVD files which were then overwritten by all following tests – ideally these would have been deleted between tests (incidentally, no obvious variation appeared between tests 1 and 2)
There was no system monitoring set up – this would provide insights as to CPU and IO utilisation throughout and would be a useful addition to the time statistics
From Qlik Sense Enterprise on Windows September 2019 (the option became configurable via the QMC in November 2019), users have the option of enabling one or both of QVD and QVF encryption – adding at-rest protection from people who shouldn’t have access to your valuable files. This is enabled server-wide and applies to all content on the environment.
The big issue that I see is that any security office is going to require that any and all encryption is enabled – without understanding the issues this could cause.
Application icons are prominently displayed throughout the Qlik Sense hub, and they are usually either the default blue, or some horrendously stretched icon that has no business being on an enterprise system.
This simple tool (packaged as an extension and accessible as a mashup) helps users generate consistent, appropriate app icons for use in their apps.
If you’re doing anything but vanilla Qlik Sense development, it’s likely you’ll need to get to the object IDs at some point. You can do this by appending /options/developer to the end of the page URL and clicking on each object in turn, or using dev tools – but that’s slow.
This bookmarklet makes getting the object IDs fast in Chrome (and, begrudgingly, IE11).
I’m often asked how to create mashups with Qlik Sense, and I strongly believe that it’s both easy and intuitive to leverage Qlik Sense APIs to build mashups…when you understand the options available to you.
There’s often a discussion about what each of these autonumber/hash functions does in Qlik. We commonly see these functions used for creating integer key fields, anonymising data (for example, names in demo apps), and maintaining long string fields for later comparison (as the hashes are shorter than the strings they replace).
To do this, I’m using the script below. I’m also keen to show outputs from QlikView vs Qlik Sense, and results of running the same script on another machine.
My observations are the following:
– AutoNumber/AutoNumberHash128/256 – different output per load as the value is typically based on the load order of the source data
– Hash128/160/256 – the same output, across every load. Stays the same between Qlik Sense and QlikView, and also between different machines
I recently spotted an unexpected slow-down in a load script, which was caused by using one of these functions. In summary:
– Using RowNo() in a simple load script is considerably slower than RecNo()
– If you must use RecNo(), it may be faster to do this in a direct load
– If you must use RowNo(), it may be faster to do this in a resident load
In this post I explore the outputs of RecNo() and RowNo() to demonstrate the difference visually.
These two fields are often used interchangeably, but they provide different output. In summary:
– RecNo() is a record number based on source table(s)
– RowNo() is a record number based on the resulting table
As a result, RowNo will always provide a unique integer per row in an output table, while RecNo is only guaranteed to be unique when a single source is loaded (RecNo is based on each single source table, interpreted individually rather than collectively).
In the absence of a source control system like SVN or GIT, a quick and easy way of capturing changes in a app is to update a version control tab before passing the app back to other developers, or onto testers.
This is a very low-tech solution, but the format below works well in shared environments. The first two tabs of your application should be:
– Version (explain the app and list changes you’ve made)
– Config (set the configuration values like paths, dates, etc used in your application’s data load and UI)