What is Big Data? Thinking about a definition suitable for me

I recently worked on an app that was loading from several hundred gigabytes of CSVs, and attempting to perform expensive transformations on these files in Qlik Sense. Normally this isn’t a problem, but due to the way the transformation was written, the result was a saturated server…and I found myself reflecting what “Big Data” means to different people (and to myself).

A recruiter’s post on LinkedIn also made me chuckle, as it highlights the disparity between definitions well. From my view of big data, I doubt that someone working in that field is likely to be interested in a role where one of the three core job skills is Excel…

 

Strong in Data Manipulation/Excel/Big Data – so which is it?

 

My observation is that there are two camps – one side that classifies using the “V’s”, and another using an altogether simpler definition! Continue reading “What is Big Data? Thinking about a definition suitable for me”