Older blog entries for fuzzynerd (starting at number 8)

7 Mar 2002 (updated 7 Mar 2002 at 04:53 UTC) »

I'm creating a piece of software that greatly simplifies the process of creating bootable FreeBSD CDROMs. This is an extension of some work I did last semester for the math department. This time I'm getting independent study credit, from the Computer Science department. The goal is to have a generalized system of Makefiles that, when handed a special data file, will generate a customized bootable CD. Progress is good, thus far. I have a magic datafile that creates a bootable webserver. Check out FreeBSD To Go!

Multicast TFTP

I've started a sourceforge project, called nTFTP. The code I've already written will appear shortly, and will probably be rewritten in C++ for fun. The code will be released under the GPL. Check out nTFTP!

Ponderous thoughts:

I'm debating about trying to achieve the Advogato certification of Journeyer. I'm spending way too much time working on Free Software(TM).

It's about 2 in the morning, on a random day of my winter break. I haven't been sleeping much. Too many cool ideas I want to work on. I'm tired, distracted, and busy--I'm not going to proofread this entry as carefully as I should--but it's long past time to make an advogato entry.

My projects:

Multicast TFTP

I worked on an independent study, this semester, with Dave Hollinger, my Network Programming teacher from a few semesters ago. I decided to work on a multicast TFTP client and server, since I have a use for one in the Links Lab, on campus. Copying a 3gig hard drive image to multiple computers by unicast is slow, even at 100mb/s.

I did my best to follow the RFCs. I had some fun with it. I wrote 1300 lines of code. It even partially worked, when I was done with the semester. I still need to add more code for the multicast support, but lots of other stuff works just fine, such as unicast transfers, option negotiation, blocksize option support, retransmits, and inactivity timeouts.

I plan on releasing the code under a BSD license, when it's a bit cleaner. It's not quite ready, yet...and I keep having this horrible itch to rewrite it from scratch, perhaps in C++. :(


I wrote a message to the FreeBSD -stable mailing list about a USB Compact Flash Card Reader that I purchased, recently. It's a Simpletech reader, using a DataFab chip, with a USB product ID of 0xb000. It "almost" works with FreeBSD's usb mass-storage device support (umass). It apparently "almost" works with Linux, too, according to the Linux DataFab driver writer's page.

I went through various efforts to get the device working, but to no avail. I was able to use this as a learning experience, however. I successfully added an entry to the FreeBSD's USB device table, and I believe I successfully added an entry to the SCSI 'quirks' table, too. I learned a whole bunch about SCSI, USB, and umass. It just wasn't enough to make my flash reader work, when I was done.

This is the first time that OS source code didn't scare the hell out of me, as soon as I looked at it. It's also the first time I've modified OS code, and the result actually compiled, and did something sensible. It's corny...but it feels like a big step for me.

Project Links:

XML is wonderful. I'm the system administrator for Project Links. I'm also a programmer for them. And I'm also the release/build engineer. I make it all work. My boss gives me a lot of freedom, and I make the most of it.

Project Links has a big website. The project has existed for many years, and has accumulated a fair amount of cruft. At the beginning of last summer, for example, the web site consisted of 30,000 files, huge amounts of hand-written HTML, filled with bad-links, inconsistencies, and missing files.

For the 18 months I've worked there, I have tried to clean things up. At this point, I think we even have all our soure code, again! (The previous sysadmin lost many files, due to an incorrect backup procedure). I played some nasty tricks with a java decompiler, to fix this. The site has been trimmed down to 20,000 files, just by deleting junk. The site will even fit on a CD-ROM! I moved the project to CVS, a while ago, so developers would stop overwriting eachother's changes. I setup a reliable backup solution, so we won't lose files any more.

My next goal--I want to move Links to XML. Currently, the "modules" are hand-written HTML, with lots of templates and javascript crap. I would like to use XML and XSL to write a language that can be used to express the content of the modules in a purely presentation-free format. Ideally, the people writing the educational content would be able to write in this new language directly, instead of asking HTML programmers to design pages fitting their requests. At site "build time", a set of templates would be applied to the module XML data, and the website would be formed.

There are numerous nifty side effects of this, such as the ability to use a non-HTML presentation language. We could write an XML->LaTeX translation, and print a Project Links teacher's guide, without ever changing the module data. If I succeed in the move to XML, the website will shrink down by many thousands of files. It might even be manageable again. I would love to be able to validate the module content, and use the XML data to do searches for missing files (broken links), as well as orphaned files.

The idea of having content separate from presentation is such a simple and obvious idea, that I wonder why it is so often ignored. With XML and XSL, we can let HTML become a purely presentation-based language. Perhaps it will stop sucking, then.

So, I'm stuck in Troy while Jon Chen and Louis Gerbard are at BSDCon in California. Not fair at all. From what I've heard, Jon now has commit access. Louis is still working on it. Jon's Cardbus code is coming along nicely, and Louis is working on SoftUpdates for Darwin. It's cool knowing these people personally...but I'm going to have to work my ass off if I expect to ever catch up with them.

Louis brought up the idea of having RPI-ACM's SIG-BSD try to start up a BSD equivalent to Linux's "kernel traffic" archive. He claims that LinuxCare is willing to sponsor it. Not a bad idea, but I don't know who would be willing to sit and read mailing lists for that amount of time. Garrett might.... Funny thing is, with the group of people we have, this sounds like far too simple a task.

Last ACM meeting, a graduate student gave a presentation on his thesis topic: performance analysis in parallel systems. The meeting went really well. Everyone seemed to be paying attention, despite the fact that the majority of the presentation was probably well beyond the comprehension of most of the audience. "Dr." Jeff Nesheiwat kicks ass.

I've been busy, as usual. Last night, I got together with some friends for a SIG-BSD meeting. I was impressed-- this group seems to have the right people in it to make some big accomplishments. We decided to start work on two short-term projects: The first is to help Jon Chen finish his Cardbus code for FreeBSD. The second is to put together a FreeBSD 'bootable business card', similar to the LinuxCare 'bootable business card'. I opted for the Cardbus code, since I've been looking for an excuse to do some system programming.

I ordered a few books. Any day now, I should receive "Unix Network Programming", volumes 1 and 2, "Advanced Programming in the Unix Environment", and "Lex and Yacc". My bookshelf is already warped from the weight of my O'reilly collection. It's pretty funny.

Off to accomplish things!

One month ago, when I contacted CIS about getting unix on laptops at RPI, my goal was to have some version of unix available and functional for the freshman laptops on laptop-distribution-day. This goal was achieved Sunday, at the largest RPI-ACM event in recent history. We had great membership turnout, and we estimate that between 30 and 50 students came by for installs. We were not able to get an accurate count, due to the level of (controlled) chaos during the event. I was able to get rid of the FreeBSD T- shirts, and many of the CDs. Fliers were handed out, and I think people have an idea who the RPI-ACM is. Pictures from the install-fest can be found here on the RPI-ACM servers. I'm the guy in the black 'foo' shirt.

Only one person came by with a Thinkpad 600e. My customized install for this system failed miserably, but I have reason to believe that the problem was simply an isolated hardware-related incident. Most students were Freshmen, and brought their Thinkpad T20 models. We had a customized FreeBSD install on hand, thanks to Dave Cross, Jon Chen, and Louis Gerbarg. Jon and Dave finished the FreeBSD driver for the ethernet card in the T20, and put together a custom FreeBSD install, thus saving our event. Louis put together a server with two quad-ethernet cards to perform network installs for the T20s. Everything worked relatively well, in spite of the chaos.

Being the chairman of the RPI-ACM makes me feel like I'm becoming a manager and a politician. It makes my skin crawl. I want to be a programmer.... I don't want to leave the technical details to other people.... My biggest fear is that I'll do a good job of this, people will realize it, and I'll never actually get the hands-on work that I want. I hate to feel like all I can do is have other people do the technical work for me.... On the other hand, I really care about this club. I think I can make a positive impact as chairman, and there's no possible way that I can do all the work myself. I still cringe when people joke about me being good at this leadership thing, though....

I got a new Thinkpad T20 today. I wasn't supposed to get it for some time yet, but I managed to talk to the right people in CIS to arrange it. The justification for getting the machine early is that it will end up in the right hands to test the ethernet drivers for Linux and FreeBSD, and thus save CIS some trouble. I feel slimy for pulling favors like this, but I think it will help speed things up. I wonder how much of my motivation was the fact that I want the T20 to play with on my own.... I guess it really doesn't matter, as long as I do my best to make things happen.

Haven't slept much lately. Need to get back to some schoolwork, among other things.

I hate beaurocracy. I tried to make progress on the laptop today. This involved getting the original software reloaded on the laptop, so I could test installation methods. I really got the run-around. No one seems to know how to reimage this thing, since it's so new. The only person who appears to have the original image is in a meeting. Now I'm sitting in a computer lab, hoping for an e-mail. Oh, well. Maybe I'll still finish something of worth tonight.

I decided to stay at work for an extra four hours tonight. While no one was around I performed the hard drive upgrade to the Links Lab file server. My plan was to setup a new hard drive with the same partitions as the old one, and just dd the partitions from one drive to the other. This proved to be a little more difficult than it should have been.

Apparently 'df' does not report a partition's actual number of blocks. I created a partition based on the size of the old partition in blocks, according to df. The partition I was copying ended up being many blocks larger than the value reported by df. My guess is that this is how extra space gets reserved for the superuser. It makes sense now, but it didn't this afternoon. I ended up typing "dd if=/dev/ad0s1f of=/dev/null" just to get a block count.

When I finished this whole endeavor, and the IBM deskstar was purring like a kitten, I recounted my experience to -bsd on 'lily', an RPI-centric chat program. I was quickly told that "You should have used 'dump'", followed by "This is all in the handbook". I read about dump in the FreeBSD handbook and found it less than enlightening. I've pored over the 'dump' man page, too. The references to the 'Towers of Hanoi' algorithm seem to be completely unnecessary, but maybe that's just me. One of these days I'll try to make some additions to the handbook. It suffers from a few important omissions.

CIS contacted me about the laptop and the progress on a Linux driver. I told them that I'd check with Louis, who is currently attempting to shoehorn Jon's FreeBSD driver patches into Linux. If he succeeds, we have about one week to put together a distribution of software for the freshmen. If he fails, at least I can take a week off before school starts.

I asked CIS to setup a Redhat mirror, and some ftp space for me to upload my updates and custom packages. Hopefully it won't take the CIS beaurocracy more than a week to set this up.... We'll see.

<random> One of these days, if I ever find some spare time, I want to setup a telnet BBS. I miss being a sysop. I miss 'LORD' and 'Tradewars'. They just don't make games like that anymore...probably for good reason. :) I liked the 'community' that you could develop on a good BBS. Some of the forum sites do a fair job of creating virtual communities, but generally the number of people involved makes such sites impersonal, and inaccessible. I want a nice small system with a few hundred active users, at most. I wonder if anyone would use it, if I set it up? </random>

Someone needs to come to my apartment, and confiscate my Diablo 2 CDs, or I'm never going to get this laptop project done. I wasted a few hours playing, but it helped me relax for a while.

Jon still has the loaner T20. It looks like the problem with the ethernet card was some random data written to the EPROM during some of his testing. He fixed it, and now I don't have to buy RPI a new ethernet card. I'm happy.

My former employer sent me an e-mail, asking when I'm going to "visit". He left the Albany Stratton VA Medical Center in New York State to go work for a VA Med Center in South Carolina. "Visit" means setup his new computer network in SC. I have no great motivation to fly down to do his system administration. The problem seems to be that he can't find anyone to replace me. I setup an OSX server, and a Linux server (later moved to FreeBSD) to provide file and print services for his heterogenous network of MacOS and Win98 computers. After working for him for around two years, everything finally worked the way it was supposed to. I was able to reduce my visits to his lab to 1-3 days per month, to rotate tapes, and solve some cosmetic problems. Now everything has been moved, and the physical layout of the network is completely different. This is much more than a weekend job, but he doesn't know that.

If anyone in the area of the University of South Carolina wants a job doing system administration for a network of ~14 machines, half Win98, half MacOS, plus one OSX and one FreeBSD box, send me e- mail. I started this job during senior year of high school, and continued through freshman year of college. Someone of a similar age/experience level will probably be welcomed warmly.

I wrote some Perl last night for the Thinkpad 600e installer. Nothing fancy, but now it handles network problems better than it would have. I need to add a few more packages to the install, and I think I might be done. Some things are never going to be truly up to par, but I have to learn to live with "good enough" on some things.

I can't believe that when CIS was presented the choice of "put FreeBSD on the laptops now with full ethernet support" and "wait a few months and try it with Linux", they chose Linux. It's not definite yet, but it's crazy to hand these laptops to the freshman with Windows installed, let them play around with partitions, hardware, etc., and then try to make a uniform installer for Linux two months later. If they did things correctly now, it could spare so many headaches.

Work was...mildly productive. I spent two hours fighting with 'kdm' and 'xdm' because I wanted to provide a graphical login to the FreeBSD workstations that I'm responsible for. I ended up deriving an Xsession file from redhat's KDE install, to make it all work. The two hours were spent trying to figure out why no window managers were getting loaded on login. The problem ended up being an error in the shebang line of Xsession. /bin/bash should have been /usr/local/bin/bash. I wanted to kick myself. Hard.

I made a $100 order from Copyleft today. The news about the DeCSS trial was enough to convince me that I better buy a DeCSS shirt while I still can. Since I was buying one shirt, I decided to buy a whole bunch. I even picked up a Debian frisbee. Although my last experiences with Debian were less than amusing (I wasn't quite ready for it), the frisbee will come in handy when it's time to play the next 3am game of Frisbee Golf on RPI's campus.

A friend of mine, an extremely intelligent CS grad student, is talking about making major modifications to perl, in order to write a game. He has some cool ideas about designing the engine for a MUD that could be fully graphical, and span multiple servers. If he ever gets to the coding stage, I'll try to provide some help. Currently he's talking about making some changes to perl's 'magic': the way perl can decide what type a variable is being used as, based on the context of that variable. Way over my head. Game sounds fun, though. :)

It's been so long since I've read a book, cover-to-cover. I love to read, but I read so painfully slowly, compared to most people I know. There is so little time, there days.

Louis has secured air-fare for FreeBSD-Con. I'm horribly jealous. Falls right in the middle of midterms. I'm probably not going to be able to go. Maybe next year.

Louis convinced me to try Advogato. I really like the idea behind the site, but I have doubts about my ability to stay up to date with a diary. I guess there's a first for everything, though.

I'm currently the Chairman of the RPI-ACM. I'm hoping to make the most of this position by encouraging the use of FreeBSD and Linux at RPI.

RPI instituted a manditory laptop program last year. The school issued ~1400 IBM Thinkpad 600e model laptops to the incoming freshmen. Many of these students decided to install Linux, but the process was rather agonizing, due to lack of hardware support under Linux distributions at that time. An IBM 10/100 etherjet cardbus PCMCIA ethernet card was the worst offender. Toward the end of the year, most of the hardware was finally supported, but the most functional installs were still 'messy', and far too complicated for the average user.

This year, RPI has decided to roll out ~1500 IBM Thinkpad T20 models to the incoming freshmen. I fear that history may repeat itself. A large portion of the hardware is very difficult to get working under Linux and FreeBSD. The sound card requires ALSA, the video card requires a special X server, lilo requires special arguments to boot linux above the 1024 cylinder mark, and last but not least, the fully supported miniPCI EtherExpress Pro that ships with most T20 model Thinkpads was replaced with a rather unsupported 3com miniPCI card. Getting this straightened out is going to be some work.

My goal is to lead the RPI-ACM into providing customized installs of Linux and FreeBSD for the Thinkpad 600e and the Thinkpad T20 laptops distributed at RPI. I've been working on the 600e for a few weeks now, and I have a customized redhat distribution almost ready to roll. FreeBSD is going to have to wait on the 600e, due to some problems with the cardbus ethernet. Jon Chen, an RPI upperclassman, is working on the FreeBSD cardbus driver, but it still needs some work.

I just got my hands on the RPI model of the T20 two days ago. Jon visited my apartment yesterday, and managed to patch the FreeBSD 3com 575 driver so that it works with the miniPCI card. I haven't had any luck with Linux support so far, but it looks like there might be something written already. Looks like it could be a little messy, though.

Here I am, a week and a half before the start of the next semester, and I'm going to try to get two or three distributions of software configured and tested for several hundred people. I'm not sleeping much. Jon, who has borrowed the loaner T20, tells me that the ethernet card in the T20 has ceased to function, and it might be a hardware problem. I have a life I have to worry about, one of these days.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!