Advogato blog for badger

11 Jul 2008

Fri, 11 Jul 2008 01:08:46 GMT

Flatten a list in a python one-liner

I sometimes need to flatten a list in a list comprehension or generator expression (usually because I'm doing work inside of a template where keeping temporary variables is somewhat painful). This has stumped me for a while but today I found itertools.chain() which worked wonderfully. Here's how I found out I could make use of it:


>>> import itertools
>>> data = [(1,2), (3,4,5), (6,7)]
>>> [i for i in itertools.chain(*data)]
[1, 2, 3, 4, 5, 6, 7]

13 Jun 2008

Fri, 13 Jun 2008 20:20:15 GMT

New Election Software

As Fedora folk know, the Fedora Board Election is currently taking place. What people might not all be aware of is that the election software has been completely rewritten by one of our all-star contributors, Nigel Jones. Since I wrote the former election software, I'm well aware of its shortcomings: a manual process for setting up an election, no confirmation pages, limited styles of voting, only a single election at a time, and it just plain looked ugly. Nigel looked at all the problems and wrote a new app in TurboGears that fixed them. Not only does it solve all the problems we had before but it gives us a much more flexible framework for further enhancements. If we want new styles of voting or a new Fedora theme, it will be much easier to implement that with the new code.

So if you're on one of the Fedora IRC channels, give a warm thank-you to "G" for the excellent work he's done!

8 May 2008

Thu, 8 May 2008 08:34:31 GMT

Why I Love Open Source

I've had a bunch of items on my TODO queue for the pkgdb that would be major enhancements to its usability but haven't been able to work on them due to higher priority things constantly coming up (FAS, figuring out why common operations are so slow, eliminating unicode crashes, mass branching, etc) This doesn't mean that I don't know there are usability issues with the pkgdb, though. And every time I see those issues I have to shudder and promise myself that I'll have time to fix them soon.

Well, for at least a few of those most requested features that's no longer necessary! Christopher Aillon (caillon) submitted a patch to add UI to the user package page to filter packages according to whether you own them or not. This is great as it means I'll be able to set the default filter to owner, approveacls, and commit and people will be able to easily change the settings themselves.

An even bigger enhancment is being worked on by Ionuț Arțăriși (maploin) who has recently started hacking on the PackageDB. He's adding search capability so that we can find packages by keyword a la yum search. Because of the infrastructure change freeze that we're enforcing until Fedora 9 is out the door, I don't have a test server running his code yet. If you want to try it out, though, it's available by checking out the bzr repository:


bzr branch
bzr://bzr.fedorahosted.org/bzr/packagedb/mapleoin-devel

Instructions for configuring a test server are in the README file.

Here's a few screenshots to whet your whistles:

Front Page with Search Box

Results!

27 Feb 2008

Wed, 27 Feb 2008 07:24:08 GMT

Mugshot to the Rescue!

While doing other things after dinner I happened to see this entry go by in my mugshot stacker and thought, hmmm... I should look at that. One click later I was looking at the suspect changeset on fedorahosted's trac instance and confirmed that it would break bugzilla sync. A few minutes of editing and a fix was committed.

Moral of the story #1: When set up to watch the RSS feed for your project's commits mugshot can help get the right eyes on a problem quickly.

Moral of the story #2: If you have any doubts about a change make sure to mention them in the first line of your commit message.

8 Feb 2008

Fri, 8 Feb 2008 19:04:51 GMT

Does it say good things about inkscape or bad things about the other programs we have available that it's what I use to model objects and relationships when I program?

15 Jan 2008

Tue, 15 Jan 2008 12:28:55 GMT

Mmm... Matzo ball soup and roasted tomatoes.

12 Dec 2007

Wed, 12 Dec 2007 22:43:33 GMT

Thanks to Mike McGrath and a host of other Fedora Infrastructure people, the new Fedora Hosted server has better support for Bazaar Repositories than the old one! Under the old setup we had sftp for authenticated checkout and http for anonymous checkout. In the new scheme we have smart servers replacing both of those.

Here's a bit of preliminary timing to show you the difference. Note that these times have several inaccuracies and should only be used as an extremely general indication of speed. A few caveats that I know of:

These times don't test push, merge, and pull speed, which are the networked operations that you'll be using most frequently. They only show the inital branch operation (which, however, is the first impression that most people get of an SCM's speed.)
rsync doesn't understand the bzr repository structure so I have to download the entire repository with rsync instead of just the branch I'm interested in. In this case, the difference is 4876K for rsync vs 4556K for the bzr protocol.
Although I have a bazaar repository setup on the server, I don't have one setup on the client. In normal operation, I'd already have most of the data checked out in a repository in a different branch so a new branch would have to transfer a lot less data.
This test is performed with bzr's dirstate-tags branch format. The latest bzr has a new "pack" format that is supposed to be faster. We currently have 1.0rc3 on the server and I'm running 0.92 on the client. I'll likely be experimenting with the new format after I get 1.0 installed on both sides.

Baseline rsync This runs rsync and then does a lightweight checkout in the directory (leaving us with a branch and working tree just like a bzr branch operation.)

rsync -a rsync://bzr.fedorahosted.org/bzr/packagedb/ . cd fedora-packagedb-devel bzr checkout --lightweight

real 0m16.548s

bzr over http This is equivalent to what we have had to use on the old server:


bzr branch
http://bzr.fedorahosted.org/bzr/packagedb/fedora-packagedb-devel
Branched 258 revision(s).                                  
                  
real    1m2.844s

Ouch!

A fresh branch using the smart server

bzr branch bzr://bzr.fedorahosted.org/bzr/packagedb/fedora-packagedb-devel Copying repository content as tarball... Branched 258 revision(s).

real 0m23.192s

Much better! This is only 7 seconds slower than using rsync.

bzr+ssh smart server for rw access:

bzr branch bzr+ssh://bzr.fedorahosted.org/bzr/packagedb/fedora-packagedb-devel Copying repository content as tarball... Branched 258 revision(s).

real 0m25.501s

The ssh protocol adds a few seconds to the total time but it's also quite speedy.

One interesting point is that the smart server times are much better than the ones I posted in October. This could be because we're dealing with a different server on a different network or it could be that we've gone from using bazaar-0.18 to bazaar-1.0rc3 on the server. I'll have to do more timings after we get settled into the new machine to tell more.

12 Dec 2007

Wed, 12 Dec 2007 18:32:33 GMT

My last week was largely spent working on Fedora Infrastructure tasks: fixing a lot of annoyances, creating scripts to automate tasks, and helping to get the new hosted infrastructure up and running. This week has started off the same but I'm hoping to get some work done on some new authentication code for python-fedora before next week starts. J5, Ricky Zhou, and I have been talking about what it would take to get single sign on, certificate authentication, and openid working and I want to start modifying code to make that happen.

Note: I did get to cross one of my PackageDB tasks off without doing much of anything -- porting to RHEL5 turned out to only be a matter of getting a few packages branched for EPEL. No coding required.

25 Nov 2007

Sun, 25 Nov 2007 17:53:22 GMT

Appalachian Trail Memories

I dozed off after dinner last night and dreamt of a hiker named Raindancer (after some googling, I'm pretty sure I never hiked with a Raindancer but it seems to have been a suitable mental prod). On waking, I felt compelled to scour the Internet for people from my 1999 hike of the Appalachian Trail. It was somewhat depressing to do. There's a lot of memories of the 1999 hikers on the web but not as much that can be used to reconnect to them. So, for Lank, Butterfly, Hummingbird, Shortcut, Flame, Irie, Moose of the New FBI, Llama, Wadi, Lu the Wanderer, Walking Pine, Hobbit, Algae, Beaner, Wild Turkey, Mouse's Motor, Rockfish, Buzzard, Jojo Smiley, T-Bone, Captain, Monifa, Wide Brim, Dingo, Dragonfly, Cassiopia and Freak Dog, Goat, Tenderfoot, and all the hikers that may try to reconnect to me at some unspecified future date, Anonymous Badger is still thinking of you -- a.badger+gmail.

12 Nov 2007

Mon, 12 Nov 2007 18:18:49 GMT

Giflib 4.1.6

The story so far...

I posted a new giflib-4.1.6 release over the weekend since it was brought to my attention that several seg faults were present when corrupted gif images were fed into the utilities. At the same time, I posted a notice that I was no longer going to be updating the libungif package due to the Unisys LZW patent expiring.

Today I was wondering if many people still used giflib so I had a quick look at the sourceforge statistics:

Downloads for project giflib


Date            Downloads Bandwidth
12 Nov 2007 *  	1,693  	  819.3 MB
11 Nov 2007 	2,568     1.2 GB
10 Nov 2007 	187 	  80.6 MB
09 Nov 2007 	147 	  74.1 MB

Definite bump from the release. Cool! But wait, there's more:

Downloads for Package giflib


12 Nov 2007 *  	57  	30.3 MB
11 Nov 2007 	150 	75.2 MB
10 Nov 2007 	60 	22.9 MB
09 Nov 2007 	37 	19.0 MB

Only a hundred extra people are downloading giflib now? Where are the extra two thousand downloads coming from?

Downloads for Package libungif


12 Nov 2007 *  	1,784  	862.5 MB
11 Nov 2007 	2,418 	1.1 GB
10 Nov 2007 	127 	57.7 MB
09 Nov 2007 	110 	55.0 MB

Hmm....

So despite the fact that I sent a note that libungif is now unmaintained and the fact that I did not make a new release for it this time around, everyone is rushing to download the old libungif package instead of the new giflib.

This is a little worrisome as the segfaults fixed in giflib are still present in the libungif package everyone's downloading. So here's my intarweb plea for the day:

If you actually find libungif useful, please contact me about becoming a maintainer for it. Otherwise, please switch to giflib!