Advogato blog for axboe

24 Jun 2000

Sat, 24 Jun 2000 02:59:43 GMT

Started rewriting the pktcdvd driver to be more modular. It needs to be split a bit, since CD-R and CD-RW writing are very different. At the same time I want to clean up the request handling, so it actually uses the exported function and doesn't mess too much with the block level internals. The latter was particularly bad, since it meant that there was lots of stuff I needed to do myself that I really had no business messing with.

Linus integrated my block per-queue freelist and modular elevator patch, after a discussion spawned about his unhappiness with the current setup. I implemented a new elevator based on his ideas (elevator_linus_*) which is the default now. Of course a couple of bugs showed up after integration, but I guess that's the way it always goes -- I have fixed these now.The only thing I need to do now is audit all block drivers and make sure they do the right thing wrt blk_init_queue and blk_cleanup_queue. If not, we could end you leaking requests as time passes (requests are not located in a static table anymore, instead we allocated them from a dedicated slab cache). I've already fixed IDE, SCSI is next. After that, I don't think there are any left. The patch for loop I pushed sometime ago, just in case the merge happened before 2.4.

20 Jun 2000

Tue, 20 Jun 2000 21:59:30 GMT

A couple of stupid errors snuck into packet-0.0.2a release. First of all, the io_request_lock booboo mentioned below. And then I rediscovered (yes, I've done this before) that SCSI uses the request queue queuedata field to store the pointer to the SCSI device. So now I have to copy that and hack around it. Ben created a CDRW branch of the UDF cvs tree, which has the block_empty function to query blocks for data. I'll release a packet-0.0.2b later today with these fixes, plus include the code to query for empty blocks. This last part alone should give us a _huge_ speed increase! I'll be testing just how much before releasing.

Battling with IDE locking, it's a big mess... I've taken the task of rewriting the internal locking upon me, let's see how this goes.

20 Jun 2000

Tue, 20 Jun 2000 02:32:46 GMT

Hey, it's been a while since my last update! I'm so bad at keeping diaries... But here goes ->

Released a new packet-writing package. This time it is fairly solid for me, not even untarring a complete kernel source tree to disc made it fall to its knees. The SGI kdb kernel debugger is included in the patch, I hope this will make it easier for folks to submit bugs. It's easier for me to trace them, which was the main reason for the inclusion. You can find the 0.0.2a release in here or at other kernel.org mirrors. Be sure to apply the -inc patch on top after patching, there were two stupid typos in the release.

Sent Andre an updated block queuing / elevator patch, and apparently it solves the weird problems he (and doc) has been seeing when really pushing IDE. I don't see quite how yet, but it points to a bug in the current elevator. kdb back traces all point to the stuck processes waiting in __wait_on_buffer indefinately - a sure sign that we are missing a buffer of a request here of there. I need to look at this some more, though.

Was sent an interesting paper on optimizing elevators for newer disk drives. When I've actually finished reading this, it'll be fun to hack this into Linux and see what kind of improvements it gives. I am an I/O nut, you know...

5 Jun 2000

Mon, 5 Jun 2000 00:16:25 GMT

People are seeing very nice improvements with my block + elevator patch, that is good news. I suppose I should split it up a bit and get around to shipping it to Linus... I added more parts to it today, a much better version of the tq_disk I did earlier. Now we don't use task queues to handle plugging, but rather tasklets (or softirq's). This made it easier and more clean to implement firing of individual devices when in need of a buffer/page/kio. This means that now firing all devices is not the main use, it can still do that however. We need that for memory pressure reponse, etc.

I'm seeing an odd XFS crash when really pushing it. I'm hoping the XFS guys will fix this soon.

2 Jun 2000

Fri, 2 Jun 2000 01:29:04 GMT

Oops, haven't updated in a couple of days...

Wrote the code to support KIOBUF_IO that SGI uses in their XFS file system pagebuf. Unfortunately there's something up with the XFS cvs right now (unable to obtain lock bla bla), so I can't diff it and send it to Chaitanya, hopefully it'll be back up tomorrow. Gave me a chance to test the new Maxtor drives I have received, they are working great. SCSI still has quite a performance edge, though.

Also finished the last updates to 2.2.16-pre and sent them to Alan. I think I made it just before he left ;-).

My CX drive is stuck in Hannover according to the UPS package tracker. I hope they can get it here by tomorrow so that I can start hacking on pktcdvd again this weekend. My fingers are really itching to get back to that... Preparing for the worst case scenario (that I still can't get my data out), I checked with Ibas how much a data recovery operation would cost. An analysis is $475, recovery estimated between $975 and $1800. Auch.

27 May 2000

Sat, 27 May 2000 00:27:11 GMT

For the uninformed - a CX drive will not work with EX nor ST Fireball electronics. It does make for interesting light and sound work, though.

Working with SGI folks to boost kio performance for XFS. Mainly just a natural extension of the original wait_on_buffer/page work I did when hacking ll_rw_blk.

Ported the recent CD + DVD-RAM fixes to 2.2.16-pre4. Now I just need to test a bit more, diff up, and send it off to Alan.

26 May 2000

Fri, 26 May 2000 00:07:53 GMT

Ripped all the BLIST_GHOST SCSI hacks out. They were there to make DVD-RAM drives appear as a SCSI harddrive and CD-ROM at the same time. Besides from being a gross hack from day one, it also relied on a static list and new drives had to be added to that list to work. Ugh. Now DVD-RAM capabilities are determined at probe time and run entirely through the SCSI CD-ROM driver. We loose partioning for now, making that work would require changes to /dev/sr* and /dev/scd* and now is really not the time to do that.

Sent some stuff to Alan to include in 2.4.0-test1-ac1 (can the numbering get any worse?). The DVD-RAM stuff mentioned above, the VCD Plextor SCSI fixes, and other minors. Every time a new kernel comes out I have a 80kb diff, I want to unload some of that! I desperately need to split up the block queueing changes as well and start feeding some of that...

Later update: I found a 6.4GB CX harddrive at last! Many thanks to Brian Vincent for pointing me to a site that sold CX drives. Ok, I did end up paying $142.50 for it - I guess that is the price for not doing frequent backups.

24 May 2000

Wed, 24 May 2000 14:00:20 GMT

Back in action and wading through heaps of e-mail. I will get that done eventually...

Went the rasmus' talk at DKUUG, mainly just to say hi to Rasmus again. I'm not much of a php/web geek!

Still looking for a Fireball 6.4GB CX drive, I can't fathom why they are so hard to find. Thanks to the people who mailed me with URL's for site that sell hardware, unfortunately nobody still has the CX as it is no longer in production. But I just know that there are a lot of them out there! I'm going to try the electronics from an EX drive in the next couple of days, maybe it'll work.

14 May 2000

Sun, 14 May 2000 00:41:06 GMT

Going on vacation today. Back in a week.

11 May 2000

Thu, 11 May 2000 01:26:19 GMT

Thanks to all the folks that wrote to me with suggestions regarding the Quantum hard drive. Unfortunately pricewatch does not have this model, but I then discovered that Quantum's web site still lists it as a current model so it can't be too bad. Tomorrow I think I'll walk from store to store until I find one.

Picked up tickets to my trip - I'll be away from 14/5 to 21/5, which I'm much looking forward to!

Put a 'CD-ROM patch FAQ' on kernel.dk. Hopefully that will save me from a lot of questions. Find it here.