AWS Lightsail snapshots – now automatic for Linux

As an update to my previous post on using lambda to automate snaphots, I’ve finally logged back into the Lightsail console and seen they have added a new feature – automatic snapshots.

There’s now an automatic snapshot section – note that the manual snapshots above are the 14 days dutifully maintained by the lambda script

It looks like the feature was added in October, based on the help site.

You’re able to change the time the snapshot is taken, but not the amount of history maintained

I won’t be switching from the lambda function as I require 14 days on Lightsail boxes, but if you’re happy with the following constraints, then it’s a great option:

  • Pre-set to 7 days of snapshot retention (no more, no less)
  • Must be set to run at a specific time (no triggers)
  • Available on Linux boxes only (no Windows as yet)

AWS S3 Redirects with Cloudflare Page Rules – Fast and Free

A few weeks ago I used AWS S3 for 301 redirects, with Cloudflare on top to cache these requests, few as they were. This effectively maintains a handful of links which redirect somewhere useful, while directing all other traffic to this domain.

Stats from February for the site show 65% of requests were cached in the last week

The results are great! The switch to S3 was made on the 9th, and a Cloudflare page rule added on the 10th – at which point over 60% of requests became cached.

Continue reading “AWS S3 Redirects with Cloudflare Page Rules – Fast and Free”

Using your own domain with AWS S3 and Cloudflare

If you’re using a CNAME on your root domain, you’re gonna have problems. That’s just a DNS thing – and if you want to host a root domain on S3, you won’t be provided with an IP address by AWS. You can solve this if you use Route53, but what about if you want to keep your domain in Cloudflare?

You’ll also have problems if you want to use Cloudflare Full SSL on an S3 bucket configured for static website hosting – resulting in nothing but Cloudflare error 522 (Connection timed out) pages.

My use case is a set of simple redirects, following on from a post about 301 redirects in S3.

The easy solution to both problems is to use CloudFront to serve https requests for your bucket; but I’m going to assume that you want this solution to be as cheap as possible – and use only S3 from within the AWS ecosystem.

Continue reading “Using your own domain with AWS S3 and Cloudflare”

301 Redirects using AWS S3 with Static Hosting and an empty Bucket

S3 buckets allow you to host static content with a pay-per-get model. No monthly fees and no servers – so I considered how I could use this to redirect a limited number of URLs from an old website to a new site.

It couldn’t be a straight forward as the URLs aren’t the same (so using a CNAME, domain forward, or the S3 Redirect requests options were out), but I wanted to preserve the links, and was previously using a .htaccess file to do this. Enter static hosting, on an empty bucket.

We’re setting up the example with “my-redirect-test”
Continue reading “301 Redirects using AWS S3 with Static Hosting and an empty Bucket”

AWS S3 Cross-Region Replication – migrating existing files using AWS CLI

I’m setting up CRR on two buckets, one new, and one existing – which already contains files

When you enable cross-region replication on an existing bucket, it doesn’t copy existing files from the source to the target bucket – it only copies those objects created or updated after the replication was enabled. We need to copy the original files manually using the AWS CLI.

Continue reading “AWS S3 Cross-Region Replication – migrating existing files using AWS CLI”

Automating AWS Lightsail backups using snapshots and Lambda

Update 11/11/19 – limited automatic snapshots now available.

Some of the most glaring omissions from Lightsail are scheduled tasks or triggers – which would provide the ability to automate backups. Competitors in this space like DigitalOcean are all set, as they offer a backup option, whereas for AWS I’m assuming they hope you’ll shift over to EC2 as fast as possible to get the extra bells and whistles.

Of course you can manually create snapshots – just log in and hit the button. It’s just the scheduling that’s missing.

I have one Lightsail server that’s been running for 6 months now, and it’s all been rosy. Except – I had been using a combination of first AWS-CLI automated backups (which wasn’t ideal as it needed a machine to run them), and then some GUI automation via Skeddly. However – while Skeddly works just fine, I’d rather DIY this problem using Lambda and keep everything in cloud native functions.

Continue reading “Automating AWS Lightsail backups using snapshots and Lambda”

Deleting AWS Glacier Vaults via AWS CLI using a Lightsail Instance

Amazon Web Services (AWS) offers some very affordable archive storage via it’s S3 Glacier service. I’ve used this on a backup account in the past to store archives, and have decided it’s time to clear down this account (oh, and save $0.32 a month in doing so).

The main challenge with doing this, is that unlike S3, S3 Glacier (objects stored directly there rather than using the Glacier storage tier within S3) objects can only be deleted via the AWS CLI. And to delete a Glacier Vault, you’ve got to delete all of the objects.

This account has some wild spending. $4.90 a month!

In this post I’ll spin up a Lightsail box and wipe out the pesky Glacier objects through the AWS CLI. This doesn’t require any changes on your local PC, but will require some patience.

Continue reading “Deleting AWS Glacier Vaults via AWS CLI using a Lightsail Instance”

A quick performance comparison with Qlik Sense – AWS EC2 vs Azure Virtual Machines

Previously, I tested the performance of a load script while using RecNo() and RowNo() functions. This conveniently gave me a script which consumes up to 25GB of RAM, along with considerable CPU power.

So, what about testing it on two cloud boxes? I’ve chosen a machine from both AWS and Azure, loaded them with Qlik Sense September 2018 and run the load script.

Total Test Duration by Host

The summary: The AWS box was approx 8% faster than the Azure box.

Continue reading “A quick performance comparison with Qlik Sense – AWS EC2 vs Azure Virtual Machines”