Older blog entries for rml (starting at number 22)

64-bit dev_t
Finally got a 64-bit dev_t in 2.5-mm. It looks like 64-bits (and not 32-bits with a 20:12 or 16:16 split) is going to be the way we go. With 64-bits, we will do a 32:32 major:minor split. Plenty of room and I think people will be satisfied. It is good someone (aeb) is putting in the work and others are happy with the progress. Linus seems to agree with Andries's code and no one else is raising a fight. Andrew ought to push it to Linus some time in the near future.

mknod64(2)

To accompany the 64-bit dev_t, Andrew also merged a 64-bit version of mknod(2), the appropriately named mknod64(2). Until glibc is updated, the mknod(1) tool in coreutils will continue using the old mknod(2) and the old dev_t. To facilitate testing, I wrote a 64-bit version of the tool, mknod64. Source tarball, source RPM, and i386 RPM are available. With this tool and a patched kernel, 64-bit device numbers are your's to play with.

Red Hat 9

Brilliant. Good job, guys.

More gtk2 applications
Red Hat Rawhide has a gtk2-enabled gvim. Beautiful! I also finally moved to Evolution 1.3. I still cannot get spelling working, but hopefully soon. And it is beautiful. I still have a bunch of dependencies on gtk+ 1.2.0, but nothing of importance is actually using it. My entire desktop is now gtk2.
life

Trigger Happy TV makes me want to live again

procps

Finally merged the procps performance enhancements Red Hat (primarily alex) wrote:
  • do not qsort the process list if we are not sorting
  • read tgid from /proc/pid/status if it exists
  • PROC_SKIPTHREADS flag for ps_readproc() to force only reading of (tgid != pid) to avoid lots of syscalls
  • Look at flags in ps_readproc() to avoid reading /proc files files that are not needed.
  • Support FILLMEM, FILLCMD, FILLENV, FILLWCHAN for above.
No new release yet with these goodies, but they are in CVS. CVS information and packages of today's CVS snapshot are available at: http://tech9.net/rml/procps/

O(1) scheduler

Turns out Linus's keen interactivity backboost hack is not all so keen -- it caused a severe starvation issue in some odd loads, primarily with setiathome. It was backed out in 2.5.67. Ah well.

Camera

The Canon 10D is turning out to be one of the best purchases I have made in some time. I truly enjoy it. Now if only I can find a reason to still use my beloved 35mm...

I applaud Zaitcev's 22-Mar-2003 journal and his 28-Mar-2003 update.

I forgot to mention in my last posting that procps 2.0.12 was released last Friday. Tarballs and RPM packages are available. Nothing too critical... a few cleanups and the removal of oldps(1). Oh, and a major bug fixed by axboe in vmstat(8) causing a fpe under 2.5.

The next release will hopefully have a whole slew of performance optimizations contributed by alex and Red Hat. Many of them are NTPL-related (the new threading package in glibc 2.3 and the 2.5 kernel - also back-ported to 2.4 for Red Hat's next release) but some are general enhancements, too. Looking very nice.

On Kernels: Misc. scheduler hacking and testing. The interactivity complaints in 2.5 are pretty much all squashed and everyone is happy. Except a potential regression/remaining issue in Andrew's latest 2.5-mm. We will see what that turns up. Some patches have surfaced on a full 32-bit dev_t. Better late than never. Maybe.

On Code Freezes: I promise to stop wishing for a code freeze out loud.

On Work: Working on some interesting projects (and fixing some not-so-interesting bugs). Pretty fun, although I long for 2.6.

On Cameras: I got my Canon EOS 10D today. Wow. Nice. There have been some complaints over sharpness and saturation vs. the Canon D60. I never owned a D60, so I cannot compare, but this thing looks amazing to me. I think I want to get a wider lens... my widest, at 28mm, is a fairly standard 44.8mm on the D10, with its cropping factor. I would really like the Canon 16-35mm f/2.8L but I think I will have to settle for the Canon 17-40mm f/4.0L when it comes out. One nice thing about the 1.6x crop (otherwise not my favorite feature) is that my favorite lens, the Canon EF 50mm f/1.4, is a nice 80mm.

On Evolution: Still playing with the Evolution 1.3 snapshots. They are really quite usable, but I still cannot get spell checking working. Without the spell check, I cannot switch from Evolution 1.2. The problem seems to be that my dictionary is not even detected. But it is installed; everything else detects it. Speaking of Evolution...

On books: Bought a couple books on Emerging Technology and Swarm Intelligence. Chaos/Complexity theory is a huge interest and fascination of mine. These similar topics will hopefully spark my interest in some new fields.
On interactivity: Oh yah baby. I am always impressed by Linus's ability to jump into a problem and solve it. Ingo and I have wondered what to do about the problems with the interactivity estimator in the O(1) scheduler for some time. It works great and it is generally correct. The problem is not even that it is sometimes incorrect, but that when it is you feel it hard. Linus has a patch that reverse propogates the interactivity bonus to tasks that help interactive tasks, even if they themselves are not interactive. This helps X, which felt the brunt of the damage due to bad decisions. Then we better tuned the scheduler knobs, most notably dropping the maximum timeslice to 200ms (from 300ms) and the average timeslice to 100ms (from 150ms). A combination of Linus's changes plus the tuned parameters are now in 2.5 and things are good. Very good.

On a code freeze for 2.5: So I am really hoping we code freeze soon. I have been saying this for awhile, because I do not want to get into the death spiral we all know too well where we start a second round of development right when we should be finishing up. It happened with 2.1 and it happened with 2.3. I do not want it to happen with 2.5. I believe we can really see a 2.6 soon. And it is going to be great.

On Cameras: I pre-ordered the Canon 10D from Adorama. Let us hope the 10D gets to the public faster than the D60 did. I think it will - plus, I got on Adorama's waitlist the moment it went up. I hope the rumor's of late-March are true. Now I just need to get a CF reader that works in Linux. And learn the Gimp better. (I am, until now, purely a 35mm guy. The 10D is attractive because it will work with my existing EOS system. And it rocks.)

On GNOME: Galeon 1.3.3 is nice. So is Gaim pre-0.60. I am to the point where my only gtk 1.x program is Evolution. I have tried the gtk2 snapshots, but spell checking does not work. I can deal with the crashes. I cannot live without spell checking. I feel naked. Soon...

My OLS paper was accepted and the abstract is now up. I am going to try to summarize the improvements to interactivity in 2.5 (to which, some may be surprised, I give a very large credit to the new I/O scheduler) and then discuss where we still have to go. Linux desktop and real-time performance is a good step above 2.4. And its not that 2.4 was bad but 2.5 is just so damn good. It is much more consistent. No more out-to-lunch-see-you-later yelled to your read I/O requests as your write I/O requests hog the day (e.g. evolution exiting and expunging a large mail folder). No more huge latencies due to VM teardown or large unlinks. Very nice.

fejj: I agree, gnome-terminal with vte is too slow. I have not turned off anti-aliasing, because I like it, but its just slow. Good thing I always do large compiles like:

make > ../makes
Or else the output speed would cut my kernel compiles in half. I really recommend the above, btw. It lets you see just the errors and warnings (which gcc kindly emits to stderr) and not the usual cruft. If you really need to see the junk, it is saved.

My home workstation has been Red Hat Rawhide for awhile now, so its GNOME 2.2 and XFree86 4.3-pre. With a couple gtk2 replacements for key apps like Galeon, I am very happy. It is very sexy.

Spent the weekend at Sebring International Raceway with my roommates, watching my father in an auto race. It was quite fun. Plenty of Porsches and beer. And he took third in class at the enduro.

Finally got the glibc folks to merge my latest sched_{set|get}affinity patch. The kernel and existing applications prototype the system calls like:

int sched_setaffinity(pid_t, unsigned int, unsigned long *)
But glibc 2.3 prototypes them as:
int sched_setaffinity(pid_t, unsigned long, unsigned long *)
On 32-bit architectures this is a trivial error, since sizeof(unsigned long) == sizeof(unsigned int), but it results in a compile failure. Ugh. Thankfully now fixed in CVS. Hopefully Red Hat 8.1 will ship with it.

Red Hat (via alex and ingo) is contributing some general (and NPTL-related) enhancements to procps which look rather promising. One might not think performance matters one lick with any of the procps utilities, but ps(1) and especially top(1) can be lethal on large machines with many many processes. Lots of file activity on /proc. Looks like that should improve a bit. wli also wrote an O(1) algorithm for vmstat(8), but neither of us have gotten around to making it backward-compatible (it only works on 2.5). Need to do that.

Still deciding what to do with the next schedutils release. The package is just taskset(1) and chrt(1) now. Setting scheduling policy, priority, and processor affinity is important (indeed, crucial for some) but its sort-of silly to keep them in their own package. Maybe util-linux?

Kernel: I really want to wrap this up sooner rather than later

A second falconer a couple weeks back! But, alas, this week there was no falconer. In fact, SNL sucked.

Eh, what is new in life... new schedutils, 1.1.0, released back on 11 Dec 2002. Nothing too monumental there but some cleanups and such. Oh, and chbatch(1) was removed, since SCHED_BATCH is unlikely to go in any kernel anytime soon. The next release will include the perhaps upsetting removal of both lsrt(1) and irqset(1), too. The rationale for the former is that procps now does everything and more that lsrt does. The rationale for the later is that, uh, it has nothing to do with process scheduling. I think, with just two utilities remaining, I should look to integrate them into an existing package, like util-linux. I also plan to expand taskset a bit: more options and behavior mirroring sgi's cpuset utility.

Kernel development is progressing nicely. The opinion on when we will be stable and thus release seems to be pushing further and further back, however. I was really hoping for a summer 2003 release. We are really solid now, but there are some issues - primarily IDE. Plus, the Grand Penguin has not yet declared a code freeze. Soon, hopefully.

Busy doing kernel work for MontaVista. It is painful to work in 2.4 these days. The limitations of the 2.4 I/O scheduler (aka the "linus elevator") are pretty apparent. Plus, out-of-the-box (with kernel preemption enabled) sub-1ms maximum latencies are possible on 2.5 today.

So the news is out over the Desktop Linux Summit mess. This desktoplinux.com news item pretty much sums up the public information. I had a talk scheduled and suffice to say, I withdrew on Thursday as this was all going down. Too bad, cause I wanted to see San Diego and I am passionate about Linux on the desktop.

Dog-cat-hybrid.

21 Dec 2002 (updated 21 Dec 2002 at 00:45 UTC) »

Been debugging the 2.5 scheduler's interactivity flaws. Andrew has found setting max_timeslice=10 and prio_bonus_ratio=10 is the sweet spot. I am not so happy we need to set the timeslice so low to make things work, but this is a start. I very much so want to keep the interactivity estimator in - but only if we can make it work. This tuning is done via the sched-tuning patch, for the curious.

Did a nice round table discussion with Andrew, Dave Jones, and others on the 2.5 kernel for Umeet '02. The transcripts should be available soon.

Learned how cascading style sheets work so redid my boring and simple webpage. Now slightly less boring and simple.

13 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!