IPC::System::Simple released - Cross-platform, multi-arg backticks
One of my greatest itches with Perl is the difficultly in doing things correctly with the system() command, which allows one to run a process and wait for it to finish, and backticks (aka qx()), which allows one to run a process and capture its output.

It's not that these commands are hard to use. system("mount /dev/backup") and I've got my backup media mounted. my $config = qx(ipconfig) and I've got my Windows IP configuration. They're dead easy to use, they're just a pain to tell if they worked.

After running, these commands set a special variable named $?, or $CHILD_ERROR if you've told perl to use English. This variable is designed to be "friendly" to "C programmers", who are familiar with the return value of the wait() system call. But who on earth wants to be familiar with the return value of wait()? After teaching thousands of people Perl over the last seven years, I've only had two people admit to know what the wait() return value looks like, and both of them needed therapy afterwards. It's a packed bit-string, and is grossly unfriendly for anyone to use in any language.

So, the usual state of affairs for many developers is they go through their day using system() and backticks and completely ignoring $?, because it's too hard. Then one day a disaster happens where they discover that all their backups are empty, or all their files are corrupted, or all their furniture has been repossessed, because their calls to system() have been failing for years, but nobody noticed because $? was too ugly to contemplate.

Those developers who have been through this process, or seen other people's lives ruined usually try to take a little more care. They recall that $? will be exactly zero if the process ran to completion and returned a zero exit status, which usually indicates success. Their code becomes littered with statements like $? and die $? or system(...) and die $?. That makes them feel warm and fuzzy until they discover it doesn't work. Their command may legitimately return a range of statuses to indicate success, and a whole bunch of things to indicate failure. Worse still, printing the contents of $? as an error message is worse than useless. Nobody understands what the hell it means; if you did, you wouldn't be printing it.

The end result of all this is that Perl sucks at something it's supposed to be good at; namely firing off system commands and being awesome for system administrators. It is this problem that IPC::System::Simple solves.

Put simply, if you're using IPC::System::Simple, you can replace all your calls to system() with run(), and all your calls to backticks with capture(), and it will all work the same way, except that you'll get incredible useful messages on error. Got a funny exit status? It'll tell you what it is, and the command that caused it. Killed by a signal? You'll get not just the number, but its name, as well a whether it dumped core. Passed in tainted data? You'll get told what was tainted. And it gets better.

Let's say that you're using rsync to backup your files from an active filesystem. It exits with 0 for good, and 24 for "files went missing". On an active filesystem, files disappearing can be considered normal, so we'd like both of these to be considered 'fine'. Anything else, we'd like to get told. Can we do this with IPC::System::Simple? We sure can! Just provide the list of acceptable exit statuses as the first argument:

run( [0,24], "rsync ....")

IPC::System::Simple's run command also works the way that the Perl security pages say that system() should run. That is, when called with multiple arguments, it bypasses the shell. Perl's regular system() when called with multiple arguments will go and use the shell anyway when on a Windows system, which is bad if you were trying to avoid shell meta-characters.

You can get the same shell-bypassing behaviour with capture(); just pass it in multiple arguments and you're done. This even works under Windows, which normally doesn't even support shell-bypassing pipes, let alone checking your command's return values and formatting your errors afterwards. You even get the full 16-bit Windows exit value made available to you, which is something $? and its dark terrors will never give you.

Best of all, installing IPC::System::Simple is a breeze. It's pure perl, with no dependencies on non-core modules. Even if you have no other way of installing modules, you can just copy the .pm file into an appropriate directory and you're done.

Don't put up with the tragedies and uncertain futures that system() and backticks can bring. Use IPC::System::Simple and make handling your system interactions correctly a painless experience.

Beating up banks - A tale of success
Note: This is not financial or legal advice. It does not take into account your personal circumstances. I am not a lawyer nor an financial professional. Doing anything described in this blog may cause your teeth to fall out, the Large Hadron Collider to destroy the Earth, and Uwe Boll to release another movie.

A little bit of history. We purchased our house (with a mortgage) years ago, before the housing price boom in Australia. We also paid off the house years ago as well, with a low mortgage and high savings it isn't that hard. However due to the way our mortgage was structured, we discovered we could keep it as a line of credit with no interest charges, and no ongoing fees, and so we did; it's an easy line of credit if we should ever need it.

Late last year, our mortgage provider (RAMS) sold their branding and distribution lists to one of Australia's largest banks (Westpac). The remaining company (RHG) continued to hold the existing loans, but now lacked all the instruments needed to gain new customers. So, how does a mortgage provider increase their shareholder value without the ability to gain new customers?

In mid-February this year, we received a slip with our statement, saying that our mortgage discharge fee had increased from $295 to $695 as of the end of January 2008; that is, as of about two weeks ago. This is the only fee that everyone with an RHG mortgage would have to pay, regardless of what they did with their loan. Being able to net a profit of $400 for every mortgage in RHG's extensive portfolio is a very wise business decision, and given they just sold their name and distribution network, it didn't really matter how much bad press they got about it.

But wait, I hear you cry! How could they increase retroactively increase the fees? Surely people must be given appropriate notice, so they could discharge their loan early if unsatisfied with the change? Well, financial institutions do have to provide notice of fee increases, but the Australian Consumer Credit Code allows this notice to be given by means of a notice published in a newspaper that's in circulation in the appropriate region. RHG claims to have published such a notice in one of our national papers at the start of January.

I'm of the personal opinion that the Consumer Credit Code (CCC) is a pile of steaming excrement, since it's completely unreasonable for consumers to read every newspaper every day to see if their fees would be increasing. What's more, while there are provisions for appeal against an unfair rise in interest rates, or an unfair rise in early termination fees, there's no specific provision for appeals against general fee increases, which makes fighting them more complicated.

My views on the CCC aside, we were left with the problem of a fee increase we didn't want to pay. If we'd been given even a single day of real notice we would have discharged the loan, but by using the sneaky newspaper announcement we simply didn't have this chance. So, what does one do?

Start by keeping a log. Just a piece of paper where you can note down when you received the notice, when you made phone calls, who you spoke to, what was agreed, and any letters you've sent or received. Hopefully you won't need this, but it really comes in handy later when you need to intimidate or escalate.

Now call the organisation with which you're having the dispute. Be polite and friendly when doing this, it's not the fault of the person on the phone that you've been screwed. Indeed, the people you're talking to are your negotiation partners, they're going to assist you in putting things right. Normally the first person you talk to won't be able to help, but they can put you in touch with someone who can. From our first point of contact we discovered that we weren't the only people who were upset about the increase, that our customer service representative agreed the situation sucked, and we were given the details for RHG's dispute resolution centre.

Now, it's worth noting that the role of a customer complaints or dispute resolution centre isn't to listen to your complaint. In any large business there are going to be people who are angry, and the complaints department exists simply to try and absorb or deflect this anger as cheaply as possible. You can expect to get canned responses, form letters, and general unhelpfulness. All of these are cheaper for the organisation than giving up $400 worth of additional fees, and many people will give up after the first couple of events. In any telephone conversations, make sure you record the full name of who you spoke to in your log. Make sure you record any details they provided, and ask for clarification if you need to. Keep being friendly and polite, even if you need to be firm in your requests. It's easier to help someone who's likeable.

Once you've collected enough information in your log to demonstrate that you're not making progress, write a letter, preferably addressed to the most senior person you've spoken to so far. Explain your case in full, provide details including dates and names with the conversations you've had with the institution. Explain that you feel that you're not getting anywhere with the dispute, that you're very unhappy with this, and that you plan to escalate this matter to $CHANNEL if it's not resolved within 10 business days. The purpose of this letter is to demonstrate that you're sick of being screwed around, and that you've got sufficient evidence of their unhelpfulness that you can escalate the matter successfully. The 10 business days is so they've got time to react (big organisations are slow). Send the letter via registered post, make sure you keep the paperwork, and give them a call a few days later to confirm they have the letter.

Your $CHANNEL will depend upon the organisation. In Australia, we have the Banking and Financial Industry Ombudsman (BFSO). The BFSO is a joyful and wondrous thing. When a dispute is taken to the BFSO, the financial institution pays for it. Any determination made by the BFSO can be rejected by the consumer, but if accepted is binding upon the financial institution. If the issue is a systemic one (eg, everyone's had a fee incorrectly charged) then the BFSO can pull out a big can of whoop-ass and unleash it on the bank. Put very simply, any dispute that reaches the BFSO is a guaranteed loss for your bank; in time and BFSO fees if nothing else. There are similar ombudsmen for other industries, the BFSO isn't unique.

Not every institution is under the BFSO's jurisdiction, and in our case that included RHG. If you're in Victoria, the appropriate body to go to is the Victorian Civil and Administrative Tribunal (VCAT). VCAT handle a whole range of small disputes (under $10,000), including credit disputes, payment disputes, and trade practices disputes. At the time of writing, lodgement fees for most consumer disputes are $34.20, although some are higher.

As it happens, I've never had a case go to either the BFSO or VCAT, and that included our dispute with RHG. Why? Because it's much cheaper for the dispute department to roll over on a $400 dispute than it is to go through all the time and expense of a legal process. Even if they win, the time taken to bring a dispute through VCAT will end up costing them more than $400. This sort of process tends to be a very reliable way, albeit a somewhat time consuming one, to resolve a dispute with most large organisations.

To end our tale with RHG, I received a phone-call from them the day I was filling in the lodgement paperwork for VCAT. We were given a long spiel about how RHG does not admit any fault, that this is a once-off decision made as a gesture of good will, and that RHG believes it has complied with a long list of sections in the CCC. When you get this phone-call, be polite, say thank-you, and don't get smart. Whoever is calling you probably doesn't want to be doing what they're doing right now, so make it easy for them. Do make sure you get their name and contact details if you're not getting the resolution in writing, you may need them later.

As it happened, everyone worked like clockwork after the phone-call. A few days later we completed all the required paperwork under the old fee structure, and our land title arrived in the mail. Huzzah!

IPC::System::Simple 0.08 released
Hot on the heels of v0.07, I've released IPC::System::Simple v0.08. The most important changes are documentation fixes thanks to Matt Kraai, a mistake fixed in one of my tests, and better support for taint under Perl 5.6.x.

You can grab the new release from a CPAN mirror near you once it finishes indexing.

IPC::System::Simple 0.07 released</b>
I've released version 0.07 of IPC::System::Simple, a module that makes it easy to do the right thing when calling Perl's system() command, including providing comprehensive diagnostics when things go wrong.

The new version provides better diagnostics when used with tainted data, better testing under some more exotic systems, and a few documentation tweaks. You can grab the new version from a CPAN mirror near you (once it finishes indexing), and if you like the module, you can even vote for it using CPAN ratings.

Holy web 2.0 Batman! I've been dugg!
Woke up today to discover an unusually large number of comments on my talk on fixing the web with greasemonkey posted to youtube. I figured that the video had been indexed somewhere, similar to the rush of views it saw when it was submitted to stumble-upon.

I discovered I'd been dugg.

Quite amusingly rather than my blog entry on the talk being dugg, or my posting on youtube, it was a copy of the video on 5min.com that was getting all the attention. I didn't even know 5min.com existed, or that my material had been posted there. Regardless, I was happy for the attention.

With maybe 500 people in the audience at linux.conf.au, 8,500 hits on youtube, and 21,000 hits(!) on 5mins, I'm guessing I've managed to absorb more than two person-months of audience time. For a three-and-a-half minute talk, I'm pretty pleased.

I've got a bunch of other talks I've been meaning to put on-line for a while now, but given the amazing response to this one, I think I'll be starting with my other lightning talks.