Older blog entries for Stevey (starting at number 551)

The final updates of 2011

I've been informed by a couple of people that the Debian Administration site is down. Sadly it is; at the moment the host isn't showing anything on the serial console and remotely power-cycling it isn't showing any signs of life.

At this time of year I don't want to drag anybody in to take care of it, so ETA on recovery/replacement hardware is Monday/Tuesday.

In other news I've made it to year five of the KVM hosting sub-project/thing. Originally started as a Xen host its been running happily for quite some time. I suspect next year, or the year after that the price/specification ratio will end up losing out and we'll cancel the whole thing - but there are no immediate reasons to make any change.

Finally I knocked up a simple tool to validate my TinyDNS records prior to uploading them. It is simplistic, but adequate to catch the kind of mistakes I make:

• http://tinydns-lint.repository.steve.org.uk/file

Honestly it probably wants to be rationalised a little more - and check records more carefully. e.g. Ensure that the host a CNAME refers to itself exists, and making sure that the nameservers specified are valid.

I just wanted to make something quick after accidentally uploading a zonefile where I'd managed to fat-finger several important records. le sigh.

Oddly enough asking on serverfault.com showed no real suggestions - other than actually running tinydns locally and doing a zone-xfer to validate records. Overkill and harder than I'd like.

Happy New year if you care about such things..

"I finished growing up, Léon. I just get older. " - Leon

Syndicated 2011-12-31 20:29:45 from Steve Kemp's Blog

So I'm a peddler of smut nowadays.

Over the Christmas period I've not been doing too much.

December was a month that started out pretty well, in the first 10 days I had five models/friends/random-folk come to pose for me. (I have a few folk lined up for mid-January, but things tailed off quickly this month due to people having little free time).

I spent a while mulling over what I was doing with images, as I've recently been doing a fair number of more NSFW images - In December this and this were my two favourite shots.

I've posted NSFW images in various places over the past year (with permission; some volunteers/victims/friends don't want me to share anything, so I don't. They just hang on my walls.) but I was never consistent.

Anyway I realised that .xxx domains are now available, so I figured I'd snarf one up and use that. That lead to tasteful.xxx - which is mostly full of images I didn't actually take, but that will change. (Sadly "artistic" was gone!)

In more on-topic news I reported #651896 - a trivial security issue in another setgid(games) binary. I've got a couple more of those to tidy up and report in the near future.

ObQuote: "Bright light. Bright light." - Gremlins

Syndicated 2011-12-26 18:08:05 from Steve Kemp's Blog

So I removed some more software from my host

Today I was idly performing some maintainence upon one of my hosts, and it crossed my mind to look beneath /etc in there I found:

/etc/python
/etc/python2.4
/etc/python2.5
/etc/python2.6

That made me look more closely at the contents of /etc - the following command output was surprising:

steve@steve:~$ ls /etc | wc -l
187

Is that average? Heavy? Light? I have no idea, but I purged a hell of a lot of software today. Now I have only python v2.6 although for some reason I still have:

python
python-apt
python-apt-common
python-central
python-minimal
python-support
python2.6
python2.6-minimal

I suspect I could drop the pything2.6-minimal package, but for the moment I'm done. I have to make pretty people look exceptional with my magical camera.

Anyway as part of this cleanup I ran a quick sanity-check on which processes are running and I think, short of kernel processes, I'm as minimal as I can be. I understand the purpose and reason for every running service:

UID        PID  CMD
root         1  init [2]
pdnsd    14091  /usr/sbin/pdnsd --daemon -p /var/run/pdnsd.pid
root     14199  /usr/sbin/monit -c /etc/monit/monitrc -s /var/lib/monit/monit.state
root     14206  /usr/sbin/syslog-ng -p /var/run/syslog-ng.pid
root     14234  /usr/sbin/cron
102      14595  /usr/sbin/exim4 -bd -q30m
redis    14627  /usr/bin/redis-server /etc/redis/redis.conf
root     14637  /usr/sbin/sshd

These are basic services; I use monit to ensure those essential daemons keep running. The only oddity there is probably the local DNS cache, but it is useful if you run any kind of DNS blacklist-using service, for example.

root     14794  /sbin/getty -L ttyS0 9600 vt100

I need a serial console login for emergencies.

root     14796  runsv node-reverse-proxy
root     14797  /bin/sh ./run
root     14799  /opt/node/bin/node node-reverse-proxy.js --config ./rewrites.js

These three processes combine to run my reverse proxy which routes incoming HTTP requests to a number of local thttpd instances.

qpsmtpd  27309    /usr/bin/perl -Tw /usr/bin/qpsmtpd-prefork --port 25 --user qpsmtpd --pid-file /var/run/qpsmtpd/qpsmtpd.pid --detach
..

The perl SMTP daemon which runs my incoming mail, passing it to exim4 which listens upon 127.0.0.1:2525. You can read about my setup in the out-of-date writeup Chris & I put together.

 /usr/bin/memcached -m 64 -p 11211 -u root -l 127.0.0.1

Memory cache for transient items.

s-blog    thttpd -C /etc/thttpd/sites.enabled/blog.steve.org.uk
1030      thttpd -C /etc/thttpd/sites.enabled/edinburgh-portraits.com
s-hg      thttpd -C /etc/thttpd/sites.enabled/hg.steve.org.uk
s-ipv4    thttpd -C /etc/thttpd/sites.enabled/ipv4.steve.org.uk
s-ipv6    thttpd -C /etc/thttpd/sites.enabled/ipv6.steve.org.uk
s-kvm     thttpd -C /etc/thttpd/sites.enabled/kvm-hosting.org
...

One thttpd instance is launched for each distinct HTTP site my server runs. Each site runs under its own UID, with its own chrooted directory tree. This is important for security.

Each local instance listens upon 127.0.0.1 - and the reverse proxy previously mentioned rewrites connections to the appropriate one.

1016     28812     /usr/bin/perl -I./lib/ -I./ /usr/local/bin/blogspam

My anti-spam filter for blog comments.

Here is my christmas challenge. Can you identify each service upon your host? Do you know why you're running what you're running?

Me? I had no idea I had a dbus deamon running. Now I've purged it. Ha!

ObQuote - "I owe everything to George Bailey. Help him, dear Father." - It's a wonderful life.

Syndicated 2011-12-03 17:21:49 from Steve Kemp's Blog

Slaughter now tested upon Microsoft Windows hosts.

Recently I said that my perl-based sysadmin tool, Slaughter, was at the cross-roads. I wasn't sure if I should leave it alone, or update it somehow.

As I'm generally lazy and busy (yes it is possible to be both simultaneously!) I didn't do anything.

But happily earlier in the week I received a bunch of updates from Jean Baptiste which implemented support for managing Windows clients, via Strawberry Perl.

So I guess the conclusion is: Do nothing. Change nothing. Just fix any issues which are reported to me, and leave it as-is. (I did a little more than that, refactoring to avoid duplication and improve "neatness".)

As I said at the time I've had some interesting feedback, suggestions and bugfixes from people over the past year or so - so I shouldn't be surprised to learn I'm not the only person using it.

ObQuote: "Oh, yes, a big cat! My salvation depends upon it! " - Dracula (1992)

Syndicated 2011-11-26 15:21:55 from Steve Kemp's Blog

Goodbye mysql ..

Yesterday evening I updated my server to remove MySQL:

steve:~# dpkg --purge mysql-client-5.1 \
                      mysql-common     \
                      mysql-server-5.1 \
                      mysql-server-core-5.1 \
                      python-mysqldb        \
                      libdbd-mysql-perl     \
                      libdatetime-format-mysql-perl

Until last month I had two database in use, one each for a pair of web-applications. As of now one is using redis - which I'm already using for my image hosting - and the other application is using SQLite.

Until recently I had a high opinion of SQLite, although that has now been downgraded a little, it is still a thoroughly excellent piece of software. I was just surprised at little things it was missing, to the extent I had to rewrite my applications SQL.

Still one less service is a good thing, and the migration wasn't so painful..

In more productive news I recently acquired a nice external flash - the Yongnuo YN-460 II is (very) cheap and cheerful, it can be fired remotely with my triggers so I've had a lot of fun with opportunistically taking pictures and experimenting with lighting.

Most of the results are NSFW, but there are some other examples lurking around including the first time I managed to successfully capture a falling water-drop. (Not the best picture, not the most explicit effect, but fun regardless. I both can and will do better next time!)

Somebody recently asked me to write about "camera stuff under linux" and happily I declined.

Why decline? Because there are so many good tools, applications, and utilities. (I use local tools for organisation and duplicate detection, rawtherapee for RAW conversion and GIMP for touchups). Having many available options is fantastic though, and something hard to appreciate for "newcomers" to Linux.

(Yeah I waited 90 seconds - if I remembered to add -nojava - for Netscape Navigator to start, under X10, with 8Mb of RAM. Happier days are here. Sure DRM is bad, secure boot .. an open question, but damn we have it good compared to almost any previous point in time!)

ObQuote: "Yeah, obviously it is only a tactical party. I'm only having a party to eventually get sex." - Peep Show

Syndicated 2011-11-22 19:59:48 from Steve Kemp's Blog

Slaughter is at the cross-roads

There are many system administration and configuration management tools available, I've mentioned them in the past and we're probably all familiar with our pet favourites.

The "biggies" include CFEngine, Puppet, Chef, BFG2. The "minis" are largely in-house tools, or abuses of existing software such as fabric.

My own personal solution manages my home network, and three dedicated servers I pay for in various ways.

Currently I've been setting up some configuration "stuff" for a friend and I've elected to manage some of the setup with this system of my own, and I guess I need to decide what I'm going to do going forward.

slaughter is well maintained, largely by virtue of not doing too much. The things it does are genuinely useful and entirely sufficient to handle a lot of the common tasks - and because the server-side requirement is a HTTP server, and the only client-side requirement is CRON it is trivial to deploy.

In the past I've thought of three alternatives that would make it more complex:

• Stop using HTTP and have a mini-daemon to both serve and schedule.
• Stop using HTTP and use rsync instead.
• Rewrite it in Javascript. (Yes, really).

Each approaches have their appeal. I like the idea of only executing GPG-signed policies, and that would be trivial if there was a real server in place. It could also use SSL because that's all you need for security (ha!).

On the other hand using rsync allows me to trivially implement the only missing primitive I actually miss at times - the ability to recursively download and install a remote directory tree. (I solve this problem by downloading a .tar file and unpacking it. Not good. Doesn't cope with template expansion and is fiddlier than I like).

In the lifetime of the project I think I've had 20-50 feature requests or comments, which suggests it might actually be used by 50-100 people. (Ha! Optimism)

In the meantime I'll keep a careful eye on the number of people who download the tarball & the binary packages...

ObQuote: "I have vermin to kill. " - Kill Bill

Syndicated 2011-11-01 21:21:26 from Steve Kemp's Blog

So I've been creating more things

I realise it has been nearly two months since I last posted anything here. The good news is I'm still alive!

Mostly the past couple of months has been full of cute victims to take pictures of, which has helped me setup a simplified portfolio site;

• Edinburgh Portraits

I still continue to prefer images of people and I was recently pleased with the delivery of my first "photobook". Over the past couple of years I've slowly decorated my flat with prints (4"x6" - A2) of my pictures, but seeing the pictures in a nicely bound book makes them feel so much more real.

I've also been doing a little more software development, mostly relating to the archiving of images and the workflow of taking RAW images, converting them, and finally uploading via rsync. I suspect the tools I've put together are Steve-specific, but I did have some fun with duplicate image detection and eilimination - something I've written about in the past.

ObQuote: "Better to write for yourself and have no public than to write for the public and have no self" - Cyril Connolly.

Syndicated 2011-10-24 15:25:37 from Steve Kemp's Blog

Scriptable email clients

This is just a quick post to remind myself in the morning, as soon as I've made it I intend to turn my computer off and leave it off until I can re-organize my office.

I've been using mutt for my email for the past few years. Nothing compares to the flexibility of procmail/sieve for organizing server-side mail, and then mutt is ideal for reading them.

With the addition of the mutt-patched sidebar mode you can even go for a few days before realizing you're not in a graphical environment. But one thing I do long for is the ability to execute scripts at various times.

Thus far I've not actually planned what I'd like to do, but as a starting point imagine being able to execute a hook when new mail arrives? Or when you send a message matching a pattern in some fashion?

There are some things out there, such as the various hacks which are designed to abort sending a message if you mention "See attachment" in a message body but fail to add one before sendign the message. These hacks generally abuse the sendmail configuration such that they're extremely ad-hoc and hard to chain/nest.

I've mellowed out over the years and I have no interest in attempting to write a mail-client (though at the same time how hard can it be? Just restrict yourself to using inotify on ~/Maildir and offload delivery to exim and you're almost done? I guess the hard part is the UI, though I do like the mutt + sidebar layout. Write the whole thing in some scripty language?)

I'll re-examine notmuch and gnus over the next week or two, but I suspect both will continue to disappoint in various ways.

Anyway, for the moment I'm just pondering. But threading is an obvious concern. Most current mutt hooks relate to the local folder, or the local message. If I were viewing a message in one directory and a new mail notification fired for a delivery to both ~/Maildir and ~/Maildir/.people.foo I'd need to either serialise them or thread them.

Ponder ponder.

In other news I've been doing more photography recently. Nothing cohesive except for my recent experiment with shooting a "street-girl" outdoors in falling light, but that was an interesting challenge and the results were sufficient to make me want to try shooting outdoors in an organized fashion again. (Some random images have been linked to from my wee twitter page.)

ObFilm: "She doesn't get eaten by the eels at this time " - The Princess Bride

Syndicated 2011-09-05 18:40:38 from Steve Kemp's Blog

There is a reason why I test sites

Recently I was at a pub and there was an advert for pub tokens displayed on the window. Seemed like a cute idea:

• Buy & donate tokens which can be spent (only) on beer.

Perfect for friends, family, remote hackers/developers & similar.

When I got home I checked out their site. Seemed simple and nice enough, with good coverage (in terms of local drinking establishments that would accept their tokens).

I decided to sign up, with the intention of gifting my sister with some delicious beer. Unfortunately that's where it all went wrong.

I tend to act the same on all new sites. Partly to amuse myself, partly to get a feel for how safe/secure/good the site is, I'll try to login with a few different values.

You know you're in trouble when you see responses like this:

SELECT * FROM cms_module_pubtokens_users where email = '"'' AND
   password =''"' LIMIT 1

Fatal error: Call to a member function FetchRow() on a non-object in
  /home/pubtokens/U79P18WQ/htdocs/includes/functions.php on line 291

Suffice it to say I sent them an email, then poked them on twitter, but to no avail.

In conclusion they don't get my money, and I couldn't recommend them to anybody else at this point either. As I'm not a customer at least I can rest easy knowing my details haven't been compromised at any point over the past few months.

ObQuote: "It can't rain all the time" - The Crow

Syndicated 2011-07-16 10:29:47 (Updated 2011-07-16 23:06:17) from Steve Kemp's Blog

Steve, in brief

In brief:

• I made the firefox-wrapper package public. Although it seems that mozilla.debian.net is liable to be preferred by most.
• I wrote a script to archive a reddit inbox as a maildir folder, reddit2maildir. Perfect for archiving those interesting discussions that aren't public.
• I posted some random animal photographs, mostly to remind myself to take more.

Finally having recently bought the Canon 70-200mm f/2.8 lens for a King's ransom I've agreed to buy the 24-105mm f/4.0 lens from a friend - that will be my new portrait lens of choice, and I'll sell my existing 85mm f/1.8.

ObQuote: "I could help you cross your yard." - Up

Syndicated 2011-06-26 11:05:50 from Steve Kemp's Blog