linux

amber: writing bash scripts in amber instead. pt. 1: commands and error handling

writing shell scripts is zero fun. the bash syntax is a mess, error handling is difficult, and any script longer than a hundred lines is basically unreadable. but we keep writing bash scripts because they’re the right tool for the job and the job must be done.

amber aims to fix this pain by being a language that gives us a sane, readable syntax that transpiles into messy bash so we don’t have to write messy bash ourselves.

this post is a three-parter that will go over the basic features of amber from the perspective of those of us who actually want to use it. we’ll start with calling shell commands and handling errors, then look at loops and if statements, and finally investigate functions.

the elegant syntax of amber is pulled away to reveal the messy bash underneath
Continue reading →
Posted by grant horwood in linux, 0 comments

bash: splitting tarballs the ‘easy’ way

there are times when we tar and gzip a directory and the final tarball is just too damn big. maybe it doesn’t fit on any of those old thumbdrives we got at the 7-eleven, or maybe we’re trying to upload it to s3 and aws is complaining it’s too large.

let’s take a look at a quick and dirty way to split our tarballs into a bunch of smaller files of a set size.

segments of a 32gbh tarball everywhere
Continue reading →
Posted by grant horwood in linux, 0 comments

managing disk space in bash

a week ago, i ran this db insert-heavy script on a server and forgot to turn off mysql‘s binlog feature first. the result, of course, is that i filled up the disk in about three minutes and brought the whole server down. not great for a tuesday. fortunately, finding and fixing the problem was straightforward. the downtime was only a couple of minutes.

in this post we’re going to go over inspecting our disk space; figuring out how much we have left and finding out what we spent all those blocks on. we’ll look at three basic tools:

df for inspecting disk space
du for getting directory sizes
find for finding files to delete these all come pre-packaged with your linux or linux-like operating system, so put that apt back in your pocket.

Continue reading →
Posted by grant horwood in linux, 0 comments

bash: uploading to s3 using curl

s3 is handy and useful, but it would be a lot more handy and useful if we had a shell script for pushing things to buckets; a script that didn’t require a special cli tool. something portable that we could add to existing scripts or throw in a cron job or distribute to our friends. a script that we could tell “put this file in this bucket” and it would just do that. that would be a good thing. let’s build that.

Continue reading →
Posted by grant horwood in aws, linux, 0 comments