0800 Friday morning, machine is slow... why?
Random disk I/O, evolution doing a table scan again, and popularity contest fighting with it by reading in many many inodes.
9729 be/6 nobody 183.44 K/s 0.00 B/s 0.00 % 0.00 % perl -w /usr/sbin/popularity-contest
Time for pop-con to go - while I like giving statistics about use, this isn't the first time its chosen to get in my way.
Success! Last weekend I started working on glue to help drizzle integrate closer with hudson using subunit and pyjunitxml.
This is now up and running - you can see test runs with details of the test output. The details are being read by hudson from the xml output which created by subunit2junitxml. But the drizzle test runner only needs to output subunit, which is somewhat simpler, as well as streaming.
Dear evolution, Why are you taking 180 seconds (+- 20) to open mails ?
Dear user, I'm reading message metadata for every message in that folder, then for all the new mail that arrived while I was doing that.
Dear evolution, sqlite is good, not reading the entire maildirs in to filter mail, or open a folder, would be better.
iotop says: TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 14797 be/4 robertc 4.30 M/s 0.00 B/s 0.00 % 0.00 % evolution
Dear user, I know, but I'm still the program I was before sqlite... what to do? (bugs have been filed, tuits are currently lacking).
Dear evolution, I see you have a large db, but perhaps its just your cache size?
$ ls -l /home/robertc/.evolution/mail/imap/robertc@localhost/folders.db -rw-r--r-- 1 robertc robertc 953131008 2009-08-06 12:23 /home/robertc/.evolution/mail/imap/robertc@localhost/folders.db $ sqlite3 /home/robertc/.evolution/mail/imap/robertc@localhost/folders.db SQLite version 3.6.14.2 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> pragma default_cache_size; 2000 sqlite> pragma default_cache_size=50000; $ sqlite3 /home/robertc/.evolution/mail/imap/robertc@localhost/folders.db SQLite version 3.6.14.2 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> pragma default_cache_size; 50000
Dear user, oooo, that feels good.
2000 - the default, is 2MB of cache. evolution was using 535m of actual memory before, this gives it 25 times the cache size. My theory is that the still present read-all-message-info limitations in evolution were causing cache thrashing. (A DB doesn't help at all unless one actually is looking at less data :).
Subunit and JUnit
I've written a JUnit XML reporter for python. This allows subunit to provide glue between anything that speaks subunit (say the samba test suite) and JUnit based tools like Hudson.
I haven't run up a demo yet, but I've got a backlog of things that need decent background testing, and it seems like a good time to try some new tools out.
Faster bzr commits, and subunit.
I've just put up for a review two patches that when combined make 'bzr commit FILENAME' substantially faster. For hackers on launchpad (which I used as test data) the time drops from 1.2 seconds to 0.3 seconds. \o/. One less blocker for 2.0 (this was a regression due to the changed characteristics of the 2a repository format).
This friday, at SLUG I'm giving a talk about subunit - a test streaming and activity protocol I've had kicking around for some years now. This allows all sorts of neat coupling and introspection that adhoc formats don't. If you're familiar with TAP, subunit has similarity, but is in my (biased opinion) substantially more powerful - in ways that matter. If you're in sydney and run automated tests of [nearly] any sort, come along, I'm sure you'll enjoy it.
Re: Michael Rooney: Simple timing of Python code
This blog post talks about timing python code...
There is a great little component in the standard library: timeit.
python -m timeit "code to time"
[edit: a home has been found]
Free to good home, one Intel 510T 22 port switch. (There are a couple of dead ports but its otherwise fine - no glitches or anything).
I've just replaced my home ethernet switch - I was using 5 ports, and most of my wired machines have gigabit now. Yay.
So, if you need a 22 port network switch, let me know. If I don't hear from anyone, it'll go into next months local recycling pickup.
This will be a 'come pick it up' arrangement, unless you're going to be at the same place I am for some other event - in Sydney, if thats not obvious.
What is it with pc vendors that they make selecting machines by fairly useless things like model number (e.g. shuttle), or a hacky fugly hierarchy (e.g. auspcmarket) easy, but by attributes (e.g. '2 ethernet ports, no fans, small chassis') terribly hard.
They should take a leaf out of the way people ask each other for recommendations. auspcmarket /kindof/ does that with the systems hierarchy - but only for a few categories.
What I want I guess, is a menu of tags/attributes I can search for on a vendors site (and/or review sites like CNet's and, google shopping and so on). Doing a google shopping search for my interesting attributes above is a pretty epic fail.
Latin (As in church latin, not latin character encoding) in Ubuntu Karmic will work better than it did in Hardy/Intrepid. I had missed one of the packages that want hard coded locale data when I did the initial enablement patch.
For reference, when adding a locale:
New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.
Keep up with the latest Advogato features by reading the Advogato status blog.
If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!
