grant horwood

co-founder of fruitbat studios. cli-first linux snob, metric evangelist, unrepentant longhair. all the music i like is objectively horrible. he/him.



co-founder of fruitbat studios. cli-first linux snob, metric evangelist, unrepentant longhair. all the music i like is objectively horrible. he/him.
28
May
2024

mysql: using json data and not hating it

it’s a universal truth that one of the primary jobs of backend developers is drinking from the firehose of json the frontends are constantly spraying at us. normally, we pick apart all those key/value pairs and slot them into the neatly-arranged columns of our db, keeping our database structure nice and normal. but sometimes there are good reasons to just store a whole json object or array as an undifferentiated blob.

in the dark, old days, this json would go in a text column, which was fine if all we needed to do was store it and return it. but if we needed to do something more complex like, extract certain values from that json or, god forbid, use one in a WHERE clause, things could get out of hand pretty quickly.

fortunately, modern mysql supports JSON as a native data type and offers a whole host of useful json functions, allowing us to work with json data in a way that’s almost pleasurable. in this post we’re going to go over extracting values from json and using json data in WHERE clauses.

this column can fit so much schema-less data in it
Continue reading →
21
May
2024

php: write php 8.4’s array_find from scratch

there’s an rfc vote currently underway for a number of new array functions in php8.4 (thanks to symfony station for pointing this out!). the proposal is for four new functions for finding and evaluating arrays using callables. the functions are:

  • array_find()
  • array_find_key()
  • array_any()
  • array_all()

the full details can be read here

if we’re impatient, though, we can skip waiting for php8.4 and homeroll these ourselves.

Continue reading →
21
May
2024

bash: splitting tarballs the ‘easy’ way

there are times when we tar and gzip a directory and the final tarball is just too damn big. maybe it doesn’t fit on any of those old thumbdrives we got at the 7-eleven, or maybe we’re trying to upload it to s3 and aws is complaining it’s too large.

let’s take a look at a quick and dirty way to split our tarballs into a bunch of smaller files of a set size.

segments of a 32gbh tarball everywhere
Continue reading →
12
May
2024

nginx: serving private files with X-Accel-Redirect

the problem is this: we have a bunch of files, pdfs say, on our webserver that we want people to download, but only if they’re registered users. everyone else gets 404s.

there’s no shortage of ways to homeroll a solution to this issue (i often use private s3 buckets), but perhaps the most elegant way is to configure nginx to do it for us. no vendor lock in with aws, no controller methods struggling under the weight of 50mb pdfs; just nginx serving files.

in this post, we’re going to go over how to use the nginx‘s X-Accel-Redirect header with a light sprinking of php to serve files from a restricted directory.

the "one does not simply walk into mordor" meme image
one does not simply download mordor.pdf from the server
Continue reading →
06
May
2024

mysql: using the slow query log

the cold, hard truth is that the reason your web app or api is running slow is because of your database calls. you can migrate everything to frakenphp or put compression in your http server if you want, i’m not going to stop you, but if you want to shave whole seconds off your response times, you should take a hard look at all that janky sql and yolo orm queries you wrote.

the good news is that mysql has a tool to find and log your slow queries. it’s called the ‘slow query log’, basically the perfect name, and it comes pre-installed. let’s get it turned on, set up, and look at the output.

a developer discovering they’re really bad at writing sql
Continue reading →
29
Apr
2024

managing disk space in bash

a week ago, i ran this db insert-heavy script on a server and forgot to turn off mysql‘s binlog feature first. the result, of course, is that i filled up the disk in about three minutes and brought the whole server down. not great for a tuesday. fortunately, finding and fixing the problem was straightforward. the downtime was only a couple of minutes.

in this post we’re going to go over inspecting our disk space; figuring out how much we have left and finding out what we spent all those blocks on. we’ll look at three basic tools:

df for inspecting disk space
du for getting directory sizes
find for finding files to delete these all come pre-packaged with your linux or linux-like operating system, so put that apt back in your pocket.

Continue reading →
22
Apr
2024

bash: uploading to s3 using curl

s3 is handy and useful, but it would be a lot more handy and useful if we had a shell script for pushing things to buckets; a script that didn’t require a special cli tool. something portable that we could add to existing scripts or throw in a cron job or distribute to our friends. a script that we could tell “put this file in this bucket” and it would just do that. that would be a good thing. let’s build that.

Continue reading →
18
Apr
2024

making mysql STDOUT less (or more) of a mess

you’re doing some command line mysql work, something like piping in a query to mysql and dumping the output to STDOUT. what you want, of course, is some nicely-formatted output. what you get, instead, is hot trash like this:

$ echo "SELECT * FROM albums" | mysql record_collection_db
id  artist  tite
1   Bratmobile  Pottymouth
2   Coltrane, John  Giant Steps

fortunately, mysql has some built-in formatters to help the world make sense again.

Continue reading →