Older blog entries for rupert (starting at number 36)

ALICE, the mobile computing app server that I've been working on, is very close to a first beta release. To demonstrate the speech recognition and speech synth capabilities that it provides, it'll include an MP3 player module that's fully voice controlled.

The software is very stable, seems to be lightweight in terms of overall CPU/memory usage, and much easier to configure, since all of the modules use a central config file now. There are a couple of minor, mostly cosmetic bugs that I'm trying to squash, but good documentation has been a much greater focus. So far I've got an installation & 'quick start' guide hammered out, and I'm working on finishing a document that describes how to accomplish development tasks like creating new modules, changing the language model, and using the functions that the LCD display and speech I/O modules provide.

Good docs are something that a good majority of otherwise well written free software seems to lack. Documentation is at least as important as the actual executable, but constantly gets treated as the sort of bastard cousin of software engineering tasks. I particularly like Knuth's Literate Programming approach towards this end, although I think that adopting a middle-ground is probably a great deal more practical in most cases, since the philosophy expounded here is time intensive in the short term.

It's really cool that the app has finally gotten to this stage:

Me: What is the name of this song?
Computer: This song is Bookends, by Simon and Garfunkel.
Me: Thanks, Alice.
Computer: You're welcome, Rupert.

--- Rupert

Entirely unrelated to open-sourcery, but an interesting anecdote to boot...

Last night I was driving home from a friend's house around midnight. As I was driving up Van Ness St, a major street that cuts through San Francisco, I stopped at a traffic light. In the lane next to me was a black Chevy with a picture of a lit Olympic torch silkscreened onto the door, and the words '2002 Olympic Torch Relay' on the back. I gave the driver a friendly smile and thumbs up, and he waved back. The light changed to green, and I drove on. As I was driving along at about 30mph, the car caught up with me, and the woman in the passenger seat rolled her window down, and yelled "Hey! I have something for you!", holding out her hand. I reached out my window, grabbed the small item, and yelled back a thank you. The item turned out to be a little gold 'Salt Lake City - 2002 Olympic Torch Relay' pin. A really cool souvenir to have!

Since the torch arrives in San Francisco today, I wonder if this is the actual vehicle that carries the torch from place to place?

Phoon: Interesting to see that you got a JamCam camera. They actually work pretty decently with Linux, via the USB interface. I wrote a short doc on getting the cam set up (using gphoto and libusb). The doc can be found at http://www.yak.net/fq a/279.html.

Cheers,

--- Rupert

Hacking

The four day weekend was very productive. I got tons of new work done on alice, my mobile computing application server. Since finishing the core event distribution server, I've moved on and have re-written lcd_module, which handles output to the CrystalFontz LCD that I'm using, and speechio_module, which handles speech input from the CMU Sphinx speech recognition server. Now I'm in the middle of migrating mp3_module, which handles MP3 playing, over from the old, prototype version to the new one. Doing an almost complete re-architecture has produced several nice enhancements, including --

• Cleaner, better commented, more modular code
• Vastly improved speed and CPU usage (thanks to use of select.poll() over threads)
• More flexible channels of communication between client modules attached to the event distribution server. Data sent from clients to the server takes the form </i>destination_module: data_for_module</i>, where destination_module represents the name of the client module that the data is intended to be used by, and data_for_module is the module-specific data. The EDS uses a lookup table of module names to route incoming data to the appropriate client endpoint. Everything was 'hardwired', previously.
• Use of a central configuration file (parseable via ConfigParser), with individual sections for each module.
• Cookie-cutter event loops in clients, which speeds up client development time enormously.

Writing

More for my own amusement than anything else, I've been doing some traffic analysis of search requests on the cluster of P2P networks known collectively as Gnutella. I've been collecting a large number of (timestamp, source IP, search term) tuples, and looking for patterns within the data. Some of the metrics I'm looking at are:

• Per-IP / Per-network frequency of search terms over different time periods.
• 'Top 100' most popular search terms on a per month basis
• Correlation of search terms by approximate geographic location of source IP.
• Using data to build a weighted network that'll act as a predictive data model for testing the probability of a user (or network as a whole) searching for certain terms, given an input set of terms from that IP/network.

Depending on how things go, I may write an essay on the subject if the data proves interesting enough. Comments on this, and references to similar work would be appreciated.

Post-Yule

At least I survived seeing family. It's never as bad as I expect it'll be, and the food is always decent (my mom makes amazing desserts!). And I got some decent gifts, which included a large reference volume on military aircraft, courtesy of my brother, and some much needed home appliances from my Mom. However, I will never understand the strange phenomenon of Holiday Ties, those ties which include things like Snowmen, Reindeer, Large Bearded Fat Men, etc. Is this a uniquely US phenomenon, or are the Europeans also subject to this scourge?

Happy $holidays to all;

--- Rupert

update: 2001-12-27: fix CMU Sphinx URL.

Hacking

Lots of work on alice, my speech controlled MP3 player for the car. More than one application can use the speech service simultaneously, and this functionality is handled by a small server-app that clients (like the MP3 player), connect to. I'd originally written the server to use a threaded model, with threads exchanging data via a globally scoped hash table, but performance was horrible, and the aquisition and release of locks was a nightmare to manage.

So I started over, and rewrote the server from scratch using the select module. It was actually trickier than I thought it'd be, but paid off in terms of performance. The server has gone from taking up 99% of the CPU time (Intel Celeron 466) when idle, to using 0.5% of it on the same CPU when serving a substantial load. I'm impressed!

Yule

My brother returns from the Army tomorrow, for a couple of weeks. It'll be very interesting to hear his news, and see how much he's changed. I need to go and pick up Yule decorations at some point. It seems as if interesting things could be done with light-ropes, especially in conjunction with some X-10 hardware for control. It's a pity that there doesn't seem to be a way to do per-light addressing in the rope. If that was possible, you could take a column of closely packed light ropes, and potentially use it as a crude display device!

Aircraft

My brother expressed an interest in military aviation, so a week or two ago, I took him over to the Western Aerospace Museum in Oakland. We spent a couple of enjoyable hours wandering among their collection of Navy jets. He was really excited to be able to get up close with the museum's F-14A Tomcat, and see how things worked on it. I spent some time doing museum restoration on a variety of USAF and Navy jets in Southern California a year or so ago, so it's neat to see that he shares the same interest.

family

I'm thinking of taking my parents up to the St. Francis for high tea this year. It seems like a pleasant way to end on a good note a year that has had more downs than ups for everyone.

--- Rupert

rain

An awful, grey day. AA Flight 587 would be a tragedy in any case, but is doubly so because it happened so shortly after 9/11, and crashed in a neighborhood that's already lost so many firefighters and police officers. Many condolences to all of the families who lost someone today, and all of the folks in Queens/Rockaway and at American Airlines.

decorating

... is almost done. Carpet is evil stuff to cut, and bathroom caulk is horrifying stuff to work with, since I'd never used a caulking gun before, and ended up getting covered in it. On the upside, the place is starting to really come together, and feel like home. I can't wait to have a big housewarming party.

dshell

For some reason, the SBC/PacBell DSL modem arrived, and I didn't touch it for two weeks. I finally took it out of the box, and spent a couple of hours making it work, which roughly divided into an hour of setup, using my trusty PowerMac 9500 with 2 ether cards for fw/dhcp/nat/ip masq via the very cool IPNetRouter program made by Sustainable Softworks. The second hour was spent sifting through my drive, and extracting all of the crap that the impolite SBC/PacBell DSL install software had placed on my system. Can anyone enlighten me as to why DSL providers must use the horrid PPPoE protocol, instead of simply provisioning the modem for n number of MAC addresses and using a DHCP server cluster? In the end, it all worked though, and a benchmark revealed that I was getting 1.2 - 1.3 Mbps down, and 500- 600 kbps up, which was nice.

I want this day to be over.

--- Rupert

This greatly amused me. From a CNN news story today:

http://www.cnn.com/2001/TECH/ptech/10/25/xp. london.launch/index.html

October 25, 2001 Posted: 1:19 PM EDT (1719 GMT)
By Graham Jones CNN

(CNN) -- Microsoft has launched its new Windows XP operating system.
The system promises fewer computer crashes and will allow users to delete data from their hard drive.

Data deletion, wow! The great new feature in operating systems that the world has waited thirty years for with bated breath! Go Bill, go!

Birthdays
It seems that October is a particularly popular month for birthdays. Happy birthday to all of my fellow Advogatoers - -- criswell,TheCorruptor, deekayen,dto,ion who share the month with me :-) I turned 22 yesterday. Time really flies. It was only yesterday that I was in 7th grade! My folks had me over for an unofficial birthday dinner, and cooked up some barbequeued salmon, which was great. The big official dinner is Friday, and there should be a lot more friends and family showing up. I just wish Oliver (my brother) could make it, but he's stationed down with the Army in Texas, and won't be back until December. My dad gave me some ties for my birthday, and a free visit to a financial planner.

Hacking
At a bit of a standstill right now. I'm still in the process of unpacking everything in my new place. I've been working more on the configuration manager (PrefMgr) on my alice car-computing application, however. And trying to work out how to make the Sphinx speech recognition module go through esound happily, rather than directly grabbing /dev/dsp as O_RDONLY. I also made some additions to mwsynth, my Python app that uses the pronounciation feature of the Merriam Webster dictionary to synthesize speech. It's now got an interactive shell interface, and includes the option to use either male or female speech (which mostly works, but it seems that MW hasn't gotten every word spoken by both a male and a female, so occasionally it deviates).

My neighbour up the street has a working 802.11 node. If I set up a second node, I think we can get fairly good coverage for the whole area, since both houses are on a hill, and have excellent LOS in all directions. Thinking about using an Orinco OR-1000 that a fellow on the BAWUG list is offering for sale.

Mechanica
My car is having coolant system problems. There's a leak somewhere, since both the radiator and coolant well seem to lose fluid over a 24 hour period. The temp gauge keeps on spiking, and the water in the coolant well seems to actually get hot and boil. Time to have it looked at professionally, I think. Possibly a gasket or a water pump, both which I'm told are expensive.

I went to Mountain View and saw fair and Zack this weekend. fair's house gets my vote as one of the finest places in the Bay Area to get quality hacking done. Fast DSL, comfortable couch, nice folks, and unlimited quantities of strong Earl Grey tea! I made lots of new additions to alice, the car computing software that I've been working on. The biggest addition is a new module called PrefMgr, that acts as the central source for retrieving and saving configuration data throughout the system. Right now there's a file called alice.config, which has UNIX style line separated name/value pairs in the form VARNAME=value. When the PrefMgr client starts, it reads the config data into a hash table in memory, then opens a socket connection back to the event distribution server (EDS), which is the central core of the application. When other clients connect to the EDS (like the speech i/o module, mp3 player, etc) they register with it by sending a service name. The EDS then passes this to the PrefMgr, which sends back a list of current configuration settings for the service. These are passed back to the client, which then continues loading, using the options given.

Ideally I'd like to hack up a little web-based interface that would allow the user to easily set and change available options, rather than having to manually edit the config file each time. I considered initially using the speech rec. system to do this, but rejected it as impractical, because most of the options are set-and-forget (IP addresses, log file locations..), and setting the options in this manner would be a bit tedious. 'Set the Ee Dee Ess Aye Pee to one two seven dot zero dot zero dot one..', and so on.

Every time I add to alice, it gets closer and closer to being an application server for voice controlled programs. I think I like that.

I found an interesting article on the social aspects of talking computer systems last week -- http: //www.tlc.utexas.edu/articles/barchas.html

It talks a little about the interesting stories of how some people came to be the very recognizable voice within automated systems. UTexas' Tex class scheduling system, AOL, and the Wildfire personal assistant are mentioned. This passage amused me:

...Now, the fan groups and fascination spawned by the Wildfire voice are an indication that Harnett and company have been successful in their task of making the computerized assistant seem real.

Some men have become preoccupied with Wildfire, calling the company to find out information about her, sending her Christmas cards and starting Web sites to speculate about what she looks like and how she acts in person. For this reason, Wildfire's identity is still a secret. She lives in the Boston area. She has a day job. She thinks the foofaraw over her voice is interesting but tries to keep it in perspective. "She's not the girlfriend of the lead engineer anymore," Harnett said, "but they're still on very good terms."...

I found the article in a serendipitous fashion. I was searching for "You've got Leprosy!" in Google (recent Simpsons quote that I wanted the sound-byte for), and it was the third result returned.

My birthday is on Wednesday. I'll be 22. Ack! I feel really old, now. On my wishlist --- Lego Mindstorms, flat panels, a Dremel tool, one of those little blue Photon Lights from ThinkGeek, and some grab bags of toggle switches, LEDs, DC motors. And maybe a Cisco AiroNet 802.11b card too :-). And a BGSystems CerealBox (multi chan A/D converter w/RS-232 interface).

--- Rupert (rupe@arrow.yak.net)

I got on a plane for my Australia vacation 5 hours before the WTC attacks. Consequently, it was an strange, although still enjoyable vacation. Shortly after getting back to SF, I moved house.