What is Big Data? Thinking about a definition suitable for me

I recently worked on an app that was loading from several hundred gigabytes of CSVs, and attempting to perform expensive transformations on these files in Qlik Sense. Normally this isn’t a problem, but due to the way the transformation was written, the result was a saturated server…and I found myself reflecting what “Big Data” means to different people (and to myself).

A recruiter’s post on LinkedIn also made me chuckle, as it highlights the disparity between definitions well. From my view of big data, I doubt that someone working in that field is likely to be interested in a role where one of the three core job skills is Excel…

 

Strong in Data Manipulation/Excel/Big Data – so which is it?

 

My observation is that there are two camps – one side that classifies using the “V’s”, and another using an altogether simpler definition! Continue reading “What is Big Data? Thinking about a definition suitable for me”

CentOS 7 Linux Minimal Install – ifconfig: command not found (let’s use ip instead)

As someone who doesn’t closely follow what’s happening with Linux tools, I recently tried a minimal CentOS 7 install, and found out by chance that a command I use a lot on Windows has been deprecated in a number of Linux releases.

Not having run into this before, my first thought was to see if it was in the core repos.

yum provides ifconfig

ifconfig is part of the net-tools package

Continue reading “CentOS 7 Linux Minimal Install – ifconfig: command not found (let’s use ip instead)”

SEATT updated to 1.4.0

I’ve updated Simple Event Attendance (SEATT WordPress plugin) to version 1.4.0. This brings bug fixes, and updates in line with the release of WordPress 4.5.3, as well as the upcoming 4.6.0 release.

The plugin can now also be found on GitHub: https://github.com/withdave/SEATT to help you track changes if you’ve made any modifications yourself.

This covered some of the feature requests at https://withdave.com/2017/05/seatt-feature-request-may-2017-update/, as well as fixes for get_currentuserinfo in WP 4.5:

* Replaced get_currentuserinfo with wp_get_current_user to fix deprecation issue
* Fixed bug where users couldn’t register to an event with limit set to 0 (unlimited)
* Fixed bug where server time was used rather than blog local time on front end form
* Updated admin and template to allow use of TinyMCE for WYSIWYG editor. Can now also add images, colours etc.

Qlik Sense and general port troubleshooting on Windows Server 2012 R2

One of the beauties of Qlik Sense Enterprise is that it can be installed on nearly any Windows machine, with very few pre-requisites.

One of the drawbacks is that you may then be tempted to load it onto a multi-purpose box, or at the very least, one with a number of core Windows roles installed (especially if you’re just testing).

Uh oh, it's a 404
Uh oh, it’s a 404

This post is going to offer some quick ways of resolving port clashes, and working out what’s binding where without changing the core Qlik Sense Enterprise installation.

Continue reading “Qlik Sense and general port troubleshooting on Windows Server 2012 R2”

SEATT updated to 1.3.0

I’ve updated Simple Event Attendance (SEATT WordPress plugin) to version 1.3.0. This brings bug fixes, and updates in line with the release of WordPress 4.3.1.

Firstly, an apology for taking so long to release another version – feel welcome to get in touch if you want to discuss changes you’ve made to your code in the interim. These are largely security changes, so it’s recommended that you update when you can. The database structure hasn’t changed, so you should experience no loss of data.

Continue reading “SEATT updated to 1.3.0”

Google Maps Distance (DistanceMatrix) API for UK in JSON

The postcode code has been updated to use Google’s distancematrix api, which provides a very different set of data from the old “as the bird flies” calculation (it calculates road distance, and provides transport options etc).

The following code is merely a demonstration, which can be seen here.

<?php
// Specify Postcodes to Geocode
$postcode1 = 'BH151DA';
$postcode2 = 'BH213AP';

// Set and retrieve the query URL
$request = "http://maps.googleapis.com/maps/api/distancematrix/json?origins=" . $postcode1 . "&destinations=" . $postcode2 . "&mode=driving&language=en-GB&sensor=false&units=imperial";
$distdata = file_get_contents($request);

// Put the data into an array for easy access
$distances = json_decode($distdata, true);

// Do some error checking, first for if the response is ok
$status = $distances["status"];
$row_status = $distances["rows"][0]["elements"][0]["status"];

if ($status == "OK" && $row_status == "OK") {

// Calculate the distance in miles
$distance = $distances["rows"][0]["elements"][0]["distance"]["value"];
$distance_miles = round($distance * 0.621371192/1000, 2);

echo 'The distance between '.$postcode1.' and '.$postcode2.' is '.($distance/1000).'Km ('.$distance_miles.' miles).';

} else {
    echo "Calculating the distance between your locations caused an error.";
}

?>

Having better error checking would also be a good idea if you plan to use the above code. Using &unit=imperial is optional, as Google always returns metres – so the code runs a basic calculation on these to convert to miles.

SEATT updated to 1.2.7

I’ve updated Simple Event Attendance (SEATT WordPress plugin) to version 1.2.7. This brings bug fixes, and updates in line with the release of WordPress 3.5.

  • Fixed problems with apostrophes being escaped with numerous backspaces in admin panel and in the comment box
  • Removed first+last name from page template as this is rarely used, with list users no longer in a table format, but now in an ordered list
  • Admins can now sign up registered users simply by supplying a username in the admin panel
  • Fixed problems with wp_prepare() causing errors in wordpress 3.5
  • Deleting the plugin now removes all database tables as well as files to provide clean removal. This means uninstalling will now remove all events

 

Live at http://wordpress.org/extend/plugins/simple-event-attendance/