Older blog entries for Stevey (starting at number 706)

The perils of the cloud..

Recently two companies have suffed problems due to compromised AWS credentials:

  • Code Spaces
    • The company has effectively folded. Thier AWS account was compromised, and all their data and backups were deleted.
  • Bonsai
    • Within two minutes all their instances were terminated.
    • This is still live - watch updates of the recovery process.

I'm just about to commit to using Amazon for hosting DNS for paying customers, so this is the kind of thing that makes me paranoid.

I'll be storing DNS-data in Git, and if the zones were nuked on the Amazon-side I could re-upload them, but users would be dead regardless - because they'd need to update the nameservers in whois before the re-uploaded data would be useful.

I suspect I need to upload to two DNS providers, to get more redundency.

Currently I have a working system which allows me to push DNS records to a Git repository, and that seamlessly triggers a DNS update (i.e. A webhook trigged by github/bitbucket/whatever).

Before I publish anything I need to write more code, more documentation, and agree on pricing details. Then I'll setup a landing-page at http://dns-api.com/.

I've been challenged to find paying customers before launching, and thus far have two, which is positive.

The DHCP.io site has now been freed. I'm no longer going to try to commercialize it, instead I will only offer the Git-based product as a commercial service. On that basis I upped the service so users could manage up to five names per account, more if you mail me privately and beg ;)

(ObRandom: Google does hosted DNS with an API. They're expensive. I'm surprised I'd not heard of them doing this.)

Syndicated 2014-06-20 12:18:46 from Steve Kemp's Blog

DNS is now resolved

I used to work for Bytemark, being a sysadmin and sometimes handling support requests from end-users, along with their clients.

One thing that never got old was marking DNS-related tickets as "resolved", or managing to slip that word into replies.

Similarly being married to a Finnish woman you'd be amazed how often Finnish and Finished become interchangable.

Anyway that's enough pun-discussion.

Over the past few days I've, obviously, been playing with DNS. There are two public results:

DHCP.io

This is my simple Dynamic-DNS host, which has now picked up a few users.

I posted a token on previous entry, and I've had fun seeing how people keep changing the IP address of the host skx.dhcp.io.. I should revoke the token and actually claim the name - but to be honest it is more fun seeing it update.

What is most interesting is that I can see it being used for real - I see from the access logs some people have actually scheduled curl to run on an hourly basis. Neat.

DNS-API.org

This is a simple lookup utility, allowing queries to be made, such as:

Of the two sites this is perhaps the most useful, but again I expect it isn't unique.

That about wraps things up for the moment. It may well be the case that in the future there is some Git + DNS + Amazon integration for DNS-hosting, but I'm going to leave it alone for the moment.

Despite writing about DNS several times in the past the only reason this flurry of activity arose is that I'm hacking some Amazon & CPanel integration at the moment - and I wanted to experiment with Amazon's API some more.

So, we'll mark this activity as resolved, and I shall go make some coffee now this entry is Finnish.

ObRandomUpdate: At least there was a productive side-effect here - I created/uploaded to CPAN CGI::Application::Plugin::Throttle.

Syndicated 2014-06-17 09:13:21 from Steve Kemp's Blog

So here's a proof of concept

The simplest possible DNS-based service which I could write to explore Amazon's DNS offering has to be dynamic DNS, so I set one up..

The record skx.dhcp.io can be updated to point to your current IP by running:

curl http://dhcp.io/set/efa6961c-f3dd-11e3-955b-00163e0816a2

Or to a fixed IP:

curl http://dhcp.io/set/efa6961c-f3dd-11e3-955b-00163e0816a2/1.2.3.4

The code is modular and pretty nice, and the Amazon integration is simple.

(Although I need to write code to allow users to sign-up. I'll do that if it seems useful, I suspect there are already enough free ddns providers out there - though I might be the first to support IPv6 when I commit my next chunk of work!)

Syndicated 2014-06-14 16:35:55 from Steve Kemp's Blog

Amazon's Route53 API is nice.

It is unfortunate that some of the client libraries are inefficient, but I'm enjoying my exposure to Amazon's Route53 API.

(This is unrelated to the previous post(s) about operating a DNS service..)

For an idea of scale I host just over 170 zones at the moment.

For the first 25 zones Amazon would charge $0.50 a month, then $0.10 after that. Which would mean:

25  * $0.50  +
150 * $0.10
             = $12.50

That seems reasonably .. reasonable.

Syndicated 2014-06-13 15:03:55 from Steve Kemp's Blog

I did get a job

In my previous blog-post I mentioned, briefly, that I'd posted a couple of adverts on Reddit looking for work.

To give more detail I did three things:

  • I made a brief blog-post on the Debian-Administration website, highlighting what I thought were interesting/useful/expected skills and experience I have.
  • I updated the site to give that link a little prominance, because .. I can.
  • I paid Reddit $10 to advertise links to that blog-post. ($5 being the minimum you could spend on any targetted advert.)

The advertisement was set to be shown in /r/edinburgh (where I live), and /r/sysadmin (where I thought some people might look if they were struggling for help).

The advertising on Reddit was painless to setup, and the traffic stats were interesting, but even though this worked out well I'm a little loathe to repeat the process - since the "non-sterling transaction fee" from my bank effectively doubled my budget.

I received a few (private) emails and comments, along with the expected grammar corrections. The end result was that I received contact from an American company founder who seemed interested.

He allowed me to write some code to solve a fun problem, appeared to enjoy the code I sent (Ruby code for dealing with (exim) email spam, that's as specific as I will be). The end result was a three month contract, which we obviously hope will lead to more permanent work.

Anyway I thought this was an atypical route to find a work, and was about a million times nicer than working with recruiters, so .. consider this documentation!

In other news it is now 10pm and I need to go to the gym and pub, in that order.

Syndicated 2014-06-10 20:59:53 from Steve Kemp's Blog

10 Jun 2014 (updated 10 Jun 2014 at 21:16 UTC) »

I'm still not a developer, but ..

Some coding updates:

My templer static site generator has now been uploaded to CPAN, and is available as App::Templer.

I've converted most of my Dockerfiles to work with docker 1.0.0, which is nice.

I also hacked up a fun DNS-server for sharing JSON-encoded data, within a LAN or other environment:

Finally I updated the blogspam-detecting site a little, on the back-end. The code is now running inside Docker containers which means I can redeploy more easily in the future.

My blog post about looking for a job received some attention via a Reddit advert I posted to /r/edinburgh + /r/sysadmin, but thus far has mostly resulted in people wanting me to write code for them .. which is frustrating.

For the moment I'm working on a fun challenge involving (email) spam-detection. That takes me back.

Syndicated 2014-06-10 13:24:35 (Updated 2014-06-10 21:16:14) from Steve Kemp's Blog

setuid/setgid binaries in Debian's Wheezy release?

If anybody has access to a complete mirror of the Debian Wheezy release, and was willing to share a list of all setuid/setgid binaries that would be greatly appreciated.

It doesn't seem to be something you can find online, so you need to manually unpack each .deb file and look at the permissions.

I don't have access to a (complete) local mirror, and so I cannot easily build such a thing, unless I go to ebay and buy a random DVD-archive.

This list would be useful for folk wanting to direct their audits ..

Syndicated 2014-06-07 09:47:50 from Steve Kemp's Blog

Reflections on Lua-based email clients

Until recently I was very happy with my console mail client, Lumail, thinking I'd written it in a flexible manner, with lots of Lua-callable primitives.

Now I'm beginning to suspect that I might have gone down the wrong path at some point.

The user interface, at startup consists of a list of mailboxes. The intention being that you select a mailbox, and open it. That then takes you to a list of messages. (There is a cool and simple to use option to open the union of multiple mailboxes, which is something I use daily.)

Now the list of mailboxes is sorted alphabetically, so the user interface looks something like this:

UI

Now the issue that triggered my rethink:

  • Can it be possible for Lua to sort the maildir list? So I could arbitrarily have the Maildir .people.katy at the top of the list, always?

Sure you think. It's just a list of strings. You could pass an array to a lua on_sort_maildirs function, and then use the returned array/table as teh display order. Simple.

Simple until you realize the user might want to do more than operate solely on the list of strings. Perhaps they wish to put folders with unread messages at the top. At which point you need a "count_unread( maildir )" function. (Which does exist.)

Anyway the realisation I had is that the CMaildir object, on the C++ side, isn't exposed to the Lua-side. So the (useful) member functions which should be exported/visible are not.

Really what I'm trying to say is that I think I've implemented and exported useful Lua primitives, but actually many more parts of the implementation could be usefully exported - but are not, and that comes from the choice I made to expose "things" not "objects". If I'd exposed objects right from the start I'd have been in a better place.

Oh well.

I continued to toy with a basic GUI mail-client last week, but I've pretty much written that off as a useful way to spend my time. For the moment I'll leave email alone, I've done enough and despite niggles what I have is absolutely the best mail client for me.

(It is a shame that Mutt is so heavyweight and hard to deal with, and that notmuch never really took off.)

Syndicated 2014-06-02 09:23:51 from Steve Kemp's Blog

Sometimes reading code makes you scream.

So I've recently been looking at proxy-server source code, for obvious reasons. The starting point was a simple search of the available options:

~$ apt-cache search proxy filter
...
trafficserver - fast, scalable and extensible HTTP/1.1 compliant caching proxy server
ssh-agent-filter - filtering proxy for ssh-agent

Hrm? trafficserver? That sounds like fun. Lets look at the source.

cd /tmp
apt-get source trafficserver

Lots of code, but scanning it quickly with my favourite tool, grep, we find this "gem":

$ rgrep /tmp .
./mgmt/tools/SysAPI.cc:  tmp = fopen("/tmp/shadow", "w");
./mgmt/tools/SysAPI.cc:    system("/bin/mv -f /tmp/shadow /etc/shadow");

Is that really what it looks like? Really? Sadly yes.

There's lots of abuse of /tmpfiles in the code in mgmt/tools/, and although the modular structure took a while to understand the code that is compiled here ultimately ends up being included in /usr/bin/traffic_shell. That means it is a "real" security issue, allowing race-tastic local-attackers to do bad things.

Bug reported as #749846.

In happier news, the desk I was building is now complete. Pretty.

 

I feel like I should write about auditing software, but equally I feel unqualified - better people than me have already done so, e.g. David Wheeler.

Also I've done it before, and nobody paid attention. (Or rather the poeple that should consider security frequently fail to do so, which is .. frustrating.)

Syndicated 2014-05-30 19:25:38 from Steve Kemp's Blog

23 May 2014 (updated 23 May 2014 at 19:13 UTC) »

Using a cubox as a media platform.

Somebody recent got in touch offering to mail me a Cubox, in exchange for me experimenting with it and writing about it. In the past I've written book reviews in exchange for receiving free copies, and while I don't want to make a habit of it I don't see a problem providing I'm up-front and honest.

So, what is the cubox-i? It's another one of those "small computers", roughly similar to the Raspberry Pi, but with slightly different hardware, and a really neat little case design, as the name suggests it just looks like a tiny two inch cube, only spoiled by the mass of cabling attached to the back.

Me? I was cheeky and said I'd have no use for one, unless it was the fancy-model. The hardware comes in 4 different versions, which you can read about on the Cubox-i product page.

Ignoring the smaller/cheaper models the fancy version is the CuBox-i4Pro, and this differentiates itself from the Rasberry Pi:

  • It has built in WiFi support.
  • It has two USB ports, and a SATA port too.
  • It has a built in infrared receiver/transmitter.
  • The onboard NIC is 1Gb - though limited to 400Mb or so due to bus-constraints, certainly faster than the Pi.
  • The on-board storage is micro SD.
  • It looks lovely.

I had two uses for this toy; the first was to be a random NAS-box hosting local backups, the second was to be a media-center. In the past I used a Rasberry PI as a media-box, but unfortunately performance was appalling, largely because of the low-spead of the USB WiFi dongle I bought.

The video playback would stall at times, even though the hardware could display full HD-output, the network constraints seemed to be a limiting factor. In the end I abandoned it and these days use it sporadically for emulation, and little else. I've been meaning to do something more interesting with it, but never quite got round to it.

By contrast the Cubox-i is wonderful at being a media-box. I've exported some shares of MP4/AVI files from my desktop host, via NFS, then downloaded a binary image of the geexbox (XBMC) distribution which I installed onto the MicroSD card via dd.

The box boots in about seven seconds, was configured to use WiFi (via "Programs | Settings"), and was streaming media in less than two minutes.

There is a Debian disitribution available for download from the cubox-i wiki, but sadly it is an ancient snapshot of Jessie from December last year. It did install, but there was no WiFi out of the box. Gunnar Wolf wrote about bootstrapping an image from sources, rather than using a binary snapshot. He's kindly shared the resulting image he built, but again sadly no WiFi support, so for the moment I'm just enjoying the media-suport.

In the future I need to decide what to do:

  • Keep the Cubox-i as a media box, using the PI for backup-hosting.
  • Avoid having two devices and lose media-streaming.

I also need to look at running Pure Debian, for obvious reasons, but if I can't use WiFi the machine is no good to me. (The TV is in a different room to the office which contains our Linux hosts.)

Either way I've not been excited about new hardware for a while, not since I bought a Logitech Squeezebox, and we're both enjoying watching media on the TV.

Syndicated 2014-05-23 17:36:13 (Updated 2014-05-23 19:13:27) from Steve Kemp's Blog

697 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!