Older blog entries for baruch (starting at number 47)

I'm using quilt to maintain my kernel patches, and it's nice and dandy, but sometimes you need to validate that all your patches are applying and the kernel compiles at all stages, Enter quilt-compile-all script:

#!/bin/bash
set +e
function die {
	echo "$1"
	exit 1
}

[ -d patches ] || die "Are you in a quilt managed directory?"

quilt pop -a [ $? -ne 2 ] && die "Quilt pop -a failed." make clean [ $? -ne 0 ] && die "Make clean failed." make -j2 [ $? -ne 0 ] && "Initial make failed."

while [ "$(quilt unapplied)" != "" ]; do quilt push [ $? -ne 0 ] && die "Quilt failed." make -j2 [ $? -ne 0 ] && "Make failed." done

echo "Compilation succeeded." exit 0

I have to patch sets for the kernel folks to review, one is the HTCP congestion control algorithm and the other is performance improvements to the kernel TCP transmit code.

The first is also tied in licensing issues and there seems to be a deadlock, the kernel folks are not interested in reviewing it before the licensing is taken care of (and I understand them!) and the officials are in no hurry to solve the licensing until they'll know it will go in. I've context switched to other tasks to see who will break first.

The second set is a set of performance improvement patches initially done by my supervisor Doug Leith, and I'm improving on that now. Part of my work is to port it from 2.6.6 to 2.6.11+ which is what I'm doing now, as well as refactoring the code and throwing away bits I don't think should be there.

I've switched from the original use of web100 to use relayfs and a script to add logging code, loosely based on ideas from Aspect-Oriented-Programming.

I've passed on my journey through using kprobes (cool stuff!), but the overhead blows my code so I'd prefer to avoid it. Kprobes had a bug when placed on a ret instruction which my testing helped uncover. Relayfs had gove some improvements based on my input, so my work is helping others.

And all of that in the name of having clean code so that it will be easier to submit to lkml when I'm done. No measurement code is embedded in my production code at any time.

Thanks to Tom Zanussi for his help with relayfs and the systemtap folks for their help with kprobes.

I got bored waiting for the test machines to come back up so I can run my tests on them, so I wrote two small utilities, one is a multicast sender, I set it to run when the test machine boots up and it sends a message with the machine name in it. There is also a receiver which waits for one or more machines to notify they are up.

Now I can set my kernel to compile and load on the test machines (a script for that already exists :-), I set the receiver to run and when it exits the test is automatically run through ssh with public-key authentication.

All this means is that the usual cycle of 10 minutes waiting for a proper compile, boot and test, is now done with no human interaction, and I have time to get some more coffee (or write a blog entry).

I've been working to improve the performance of the Linux Kernel TCP stack. Apparently it's not very efficient and there are several things that can be done. The measurements I'm doing and the improvements I've done so far will be the topic of my OLS 2005 paper (should it be accepted...).

The problems with the current implementation are not evident until you actually go to long links of 200Mbit/s, they are obviously painful when you get to 1Gb/s.

The other factor in not seeing the problems currently is the use of TCP, the plain TCP congestion control algorithm (NewReno), is not effective and does not utilize the link when talking about 1Gb/s links. That's where the HTCP algorithm which was developed in the Hamilton Institute comes.

BIC sucks

A colleague has been comparing different new congestion control algorithms, and BIC is so unfair that it's the algorithm of choice if you want to starve other users from bandwidth.

In addition there is a bug in the current implementation of the Linux BIC congestion control algorithm, which makes it so much more aggressive.

Full report and patches will be provided soon.

I'm getting distracted too easily from my projects. I've shifted focus from my FlexiFW, to a program I need to track my investments, I've got that mostly working (unreleased) and now wrote a small script to automatically download torrents from RSS feeds since I wanted to get some programs with cron and the current rss downloaders are GUI stuff. I've dubbed it RSS2dl, a boring name, but then, that's just a quick hack.

In other news, I've submitted (with my supervisor) a proposal for a presentation in OLS 2005. I hope to get that accepted, I'd be happy to go to OLS, and it just never works out. This time it can actually be work :-) and not a "waste of time and money", as my SO would put it.

Why is it called SO? "SO you want to go there without me?"
:-)

I've been distracted from AutoFW and libnl by an old idea of mine for firewall implementation, a project which is currently nameless is now working on my machine. Thanks to <whomever> in the netfilter team that thought to create the QUEUE target and libipq, it now works in user mode under the temporary name of umfw (user-mode firewall).

The first stage was to create a simple interpreter firewall, the second stage was to create a compiled firewall (compile the firewall to C and then compile to object code), the next stage is to make it work in the kernel as a target for a netfilter rule and a shared object for netfilter to be able to add this rule. Third stage is now in progress.

I've also started to learn Chinese thanks to a friend from Singapore, I've learned a few sentences, wrote them in Abiword and printed it so I can learn it offline.

I've learned to write my name in Chinese, it is written 巴湖 which is pronounced Ba-Hu, which is close but not exact, but then most languages won't be able to say my name correctly anyway.

It was a lot of pain to get chinese working, I installed tons of packages, I tried uim first but couldn't get it to work so I switched to scim. It now works pretty well, but sometimes my system gets confused in what input state it is. I can now write English, Hebrew and Chinese on my Linux machine.

I now need to find a program to teach me the alphabet, I found several programs for Gnome but they either don't work, crash or don't compile. I will turn my attention to KDE and otherwise now.

Life is hard in Ireland, it's far more expensive than it was in Israel, we get by but we are cutting a lot of corners and are way down in our life standards (smaller apartment, far less restaurants) and it rains here almost everyday. In Israel it rained "almost" every day, every day (in the winter) it would almost rain, but wouldn't.

At least I'm working on the Linux Kernel and am allowed and expected to release my work to the wide public, it is available at http://hamilton.ie/net/, I'm currently working to improve the patches, after that I need to port it to 2.6.9 (or whatever it will be at the time) and submit it to the LKML for the criticism.

I'm also trying to gather my will and do some free software work, I've got a good word from a user of AutoFW so I decided to look at it again and rewrite it in C, but now I need to access netlink directly (in Python I just run 'ip monitor link addr' and parse the output), I started to use libnl but I found that it doesn't support address types and doesn't support firewall netlink so I first need to get it to know these. Lots of work.

There was also a sign of life on the websec mailing lists which reminds me that there are some overdue issues with it.

I left my job (and my wife left hers), and we moved to Ireland, I'm doing an MSc in CS working on H-TCP (a better congestion control for TCP to make it work on Gb/sec networks).

I've ported the existing code to the 2.6 kernels (2.6.6 for now) and am now readying it for release to the kernel folks to consider it for the vanilla kernel. I've fixed a few bugs doing this and learned the code better.

I'm still lacking motivation to work on my free software projects, I do the odd job for my Debian packages, finally fixing the fribidi bugs I had, but can't say I achieve what I want to do.

Moving to Ireland

I'll be landing in Ireland on the 2nd of August, a place to sleep and transport is already arranged, We're now preparing all our belongings to the move.

The hard part is sorting and throwing away, since we are not just moving all our stuff to Ireland, but rather storing most of it we need to weed the needed from the chaffe and throw away all the amassed junk from the last six or so years.

We are now on the last day of packing and it's the hardest part, we are now in a race against time and against all the small things that still need to be done.

Tomorrow we will have the movers coming early in the morning to do the move. The day after that we meet the landlord and return him the keys and close all our utilities bills.

38 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!