Task Chaining in Qlik Sense Enterprise SaaS

Today, QSE SaaS doesn’t do task chaining. Instead you have two options:

1. Use a scheduled reload task for every app, and try to make sure the start time for task 2 will always be later than task 1
2. Use Qlik SaaS REST APIs to trigger the reload

This post covers a simple approach (i.e. Qlik script only) for triggering reloads of downstream apps on completion of an app reload in a way that doesn’t require much maintenance or any external systems or integrations.

For those who haven’t looked into the QCS APIs in the past, there’s a lot of information on the Qlik.dev site; and specifically this video on the topic of task chaining with Qlik-CLI.

The theory

Using the reloads API, we need just one thing to trigger a reload – an app ID. So, all we need to do after a reload is make a call to this API to trigger the next app. This means we only need to put a scheduled reload task on the first app in the chain, with every following app triggered by the proceeding application.

To make this work, we need:

1. A REST connection to the /reloads endpoint
2. A data file connection to store the logs in
3. A parent application (app 1 – this will be triggered by QSE SaaS and a scheduled task)
4. A child application (app 2 – this will be triggered by the load script of app 1)

The reason for doing this in the load script of a normal app (and not as a separate “task manager” app) is because scheduling and running a series of task manager apps is tedious, and setting up a large number of these apps may not be supported by Qlik in the QCS environment.

Note: Fair warning, the reloads API is experimental and may change.

The REST connection

We are going to be using the /api/v1/reloads endpoint and hitting it with a POST request. To set this connection up, you’ll need the following details:

• URL: https://<tenant>.<region>.qlikcloud.com/api/v1/reloads
• Type: POST
• Body: {“appId”:”<app GUID>”}
• Header: Authorization: Bearer <API token>

The subroutine

The inputs and outputs of this sub are listed in the script. You should put this as the last script to execute in your load script.

Sub sTriggerReload(sub_appID,sub_connAPI,sub_connLog)

	/* 
    
    This subroutine triggers the reload of a QCS application (directly, not using scheduled tasks)
    
    INPUTS:
    * sub_appID = the GUID for the app to be reloaded
    * sub_connAPI = a REST data connection that can access the tenant APIs with appropriate privileges
    * sub_connLog = a folder data connection for storing reload trigger log files (these will be stored as "ReloadLog_<AppID>_<ReloadID>.qvd")
    
    OUTPUTS:
    * Send a POST message the task API to trigger the relevant app reload
    * Store a log file to record the reload trigger to assist with finding this event in audit logs if needed
    
    REST CONNECTION CONFIG
    * URL: https://<tenantname>.<region>.qlikcloud.com/api/v1/reloads
    * Type: POST
    * Body: {"appId":"<valid app GUID>"}
    * Header: Authorization: Bearer <API key>
    
    */
    
    // Connect to the REST connection
    LIB CONNECT TO '$(sub_connAPI)';
    
    LET sub_QueryBody = '{""appId"":""$(sub_appID)""}';

    // Collect data from the response for logging
    // Configure app ID for reload
    RestConnectorMasterTable:
    SQL SELECT 
        "id",
        "appId",
        "tenantId",
        "userId",
        "type",
        "status",
        "creationTime",
        "__KEY_root"
    FROM JSON (wrap on) "root" PK "__KEY_root"
    WITH CONNECTION (BODY "$(sub_QueryBody)");

    ReloadLog:
    LOAD DISTINCT	
    	[id] 			AS [Reload ID],
        [appId] 		AS [Reload App ID],
        [tenantId] 		AS [Reload Tenant ID],
        [userId] 		AS [Reload User ID],
        [type] 			AS [Reload Type],
        [status] 		AS [Reload Status],
        [creationTime] 		AS [Reload Creation Time],
        DocumentName()		AS [Reload Trigger App ID],
        DocumentTitle()		AS [Reload Trigger App Name]
    RESIDENT RestConnectorMasterTable
    WHERE NOT IsNull([__KEY_root]);
    
    // Set variables to produce log filenames
    LET sub_ReloadTime 	= Timestamp(Peek('Reload Creation Time',0),'YYYYMMDDhhmmss');
    LET sub_ReloadID 	= Peek('Reload ID',0);
    
    // Check to see if the reload request returned rows, and the variables carry data. If not, fail this reload
    If (NoOfRows('ReloadLog') <> 1) OR ('$(sub_ReloadTime)' = '') OR ('$(sub_ReloadID)' = '') THEN
    	// Fail with an error for the log
        Call Error('An unexpected number of rows was returned by the reloads API, or invalid data was found.');
    END IF;
    
    TRACE >>> Returned reload $(sub_ReloadID) at $(sub_ReloadTime);
    
    // Store logs and clear model
    STORE ReloadLog INTO [lib://$(sub_connLog)/ReloadLog_$(sub_appID)_$(sub_ReloadID)_$(sub_ReloadTime).qvd] (qvd);
    DROP TABLE ReloadLog;
    DROP TABLE RestConnectorMasterTable;
    
End Sub;

// Call - pass in the app ID, the REST connection name, the folder connection name
Call sTriggerReload('ab77b40d-4a30-46d9-9d2b-2943c6b82902','<rest connection name>','DataFiles');

The latest version of this script can be found in this GitHub repo.

What’s missing from this?

If you want to take this forward, you’ll want a few things:

• Error handling – for when you get a weird response from the API
• An app to blend the log files this produces with the reload event logs to track duration and errors
• To store this script as a text file and load it as an include so it doesn’t exist in a million different apps
• To consider whether you want to trigger reloads by app ID or by app and space name