Fixing (one case of) AWS EFS timeouts/stalls

AWS Elastic File System (EFS) is an NFS compatible network-accessible shared storage system.  It allows you to outsource the problem of HA network storage, which is highly attractive in some circumstances.  But, there are some sharp edges, which we discovered at work.

The problem

We're using it for the shared file storage of our gitlab HA cluster.  For quite a while it worked fine, then a month or two back it started occasionally stalling/timing out.  In kern.log we'd see:

Fun with SCSI tapes on Linux

I inherited an LTO-4 SCSI-attached tape drive recently (yes, old-school, I know, it's mostly just for fun), and have been fiddling with it to do some tape-based offsite backups (to complement my other offsite backups).  In doing so, I learned a bunch of things about the SCSI protocol (including its many many tendrils and offshoots), and about handy tools for interacting with SCSI devices.  Some of these things may be useful to others, so I'm putting them here.  Enjoy!

Pages

Subscribe to stroppykitten.com RSS