8 Jul 2002 (updated 9 Jul 2002 at 13:42 UTC)
»
This diary
While I keep a LiveJournal diary, I don't want to fill it with geek content since it doesn't mostly have geek readers. So I thought I'd move that sort of thing to here.
Backups
I've been trying to find a way to back up my system onto multiple CD-Rs. Several people have basically told me that every solution out there is woefully inadequate, no-one has suggested anything good, and my searches haven't found anything good. I got quite interested in DAR for a while, but in practice its multi-archive handling is too far from the Good Thing to be worth the bother.
I've been thinking about what the Right Thing should look like. One thing it should be able to do is write each "slice" to a pipe rather than a file. This is more general than calling a hook after a slice is complete; you catch the "beginning-of-slice" as well as "end-of-slice" events, you can do arbitrary processing on the content, and you get to stop and start the archive writing process with simple flow control.
Reading the "afio" manual page, many of the options could be removed if this were possible. I sent some mail about this to the DAR maintainer; here's my discussion about how this feature could help you do backups if you've only the disk space to store a little over one slice at a time
Isn't using flow control much simpler? It would be very easy to write
another general tool that takes a filename on the command line, and
writes stdin to that file, and when a write fails because the disk is
full, it waits a little while and tries again. dar (or dar_xform)
would write to this process. If the disk filled up, dar would simply
block during a write to the pipe, because pipes are flow controlled.
While dar is blocked, another process might be busy taking a completed
slice file and writing it to CD, deleting it once it's written and
verified.
Once the old slice has been deleted, space will appear, and the
writing of the new slice will start to work again. dar won't even
notice that there was a pause, and no signals-related complexity is
needed.
And all of this happens outside dar itself. In accord with the UNIX
philosophy, a whole lot of tools that do one thing well come together
to perform a more complex task in a very configurable way. All of
this is only possible with the "pipes" approach.