Older blog entries for Xorian (starting at number 8)

On my last day visitng Microsoft Research, I wore my O'Really "Writing Word Macro Viruses" t-shirt. I was highly amused, and nobody at MS seemed to notice.

I feel I should start this entry by saying: I am not making this up.

I've spent the last two days, and I will also spend tomorrow, having what I consider to be a unique and somewhat strange experience. I've been visiting Microsoft research. What's remarkable about this is that I was invited there by employees of Microsoft research to work with them on the free software project I work on. Yes, that's right, Microsoft invited me to visit them so that they could help improve a piece of free software.

But wait, it gets better. This piece of free software is, at least in its current form, UNIX only. (chroot plays a key role in the implementation, and AFAIK Windows has no analogous function.) In other words, Microsoft invited me to visit them so that they could help improve a piece of free software which doesn't even run on Windows.

Now don't worry, the world has not gone mad, there is a logical explanation. The software in question was developed as a research project at the (now defunct) Digital/Compaq Systems Research Center. Some of the original developers (including the team leader), now work at Microsoft research. So really they have just offered to spend a few days helping me understand and improve a few tricky parts of the implementation of something they built before they worked for Microsoft, which just happens to have become free software.

But still, when I actually think about what I've been doing the past two days, it seems rather improbable.

I'm flying from snowy Massachusetts to sunny California tomorrow. After working through the middle of the week, I'll be going to this year's CodeCon, which I'm looking forward to.

Before heading out, I decided to write the code needed to get the Vesta repository mounting on FreeBSD. Luckily, it was far easier than what I had to do for Solaris.

I saw a presentation on Pin yesterday. It's pretty neat.

I've been working on instrumenting multi-threaded server appliactions to do performance analysis for the project I work on, but it's a bit of a pain. It would be nice to be able to dynamically attach the instrumentation to a running server without rebuilding or restarting it. It looks like Pin would be able to do that, though its multi-threading support is still in beta.

Earlier this year at CodeCon 2004, it became evident to me that the lack of a source-only distribution for Vesta was a significant impediment to its acceptance.  Though Vesta is free software, only binary kits have been available because Vesta is a build system that one uses instead of make, and is built using itself (which makes for a classic chicken-and-egg problem).  We simply didn't have any other way to build it.

I decided at that time that it was important to put together make build instructions for Vesta.  I would have liked to get it done six months ago, but as is the way of things, other work constrained me from having enough time to devote to it.  However, there is finally a make-based source kit for Vesta available for download.

I had a great time last weekend at CodeCon 2004. I learned many new and interesting things from the presentations. I enjoyed meeting Bram, raph, Zooko and many others. And, of course, I had fun giving my own little talk.

Random cool things from CodeCon:

I've got to stop reading Advogato diary entries late at night. They keep me up following links and thinking when I should be sleeping!

Last night I came home and read dyork's entry pointing to a discussion about running a personal Wiki on a USB Disk. That got me thinking about the possibility of running a Wiki on my Linux handheld. It could be a very powerful way of both recording notes and keeping them readily accessible. My brain's been spinning on this all day, when I really should be hacking on my main project.

For pretty much my entire professional career, I've had a habit of writing notes as I work. I just use Emacs with a few personal macros. For a while I've wanted to do something that allows me to record a little more semantic content in my notes. (For example, tagging areas of text as source code, program output, etc.) I've hesitated for a couple of reasons:

  • Using a markup system for formatting could be trouble long-term if whatever engine reads it dies off. ASCII text is pretty safe.
  • I've developed my own way of doing this. I'm reluctant to try to adapt my way of working to the paradigm of an existing piece of software. At the same time, I'm not sure I want to invest the time in writing something new that works the way I want.

But maybe I'm underestimating the software that's out there. SnipSnap looks pretty close to what I'd want, but it also looks too hefty to run on my PDA. Maybe I could adapt one of the Python Wiki engines, perhaps even give it a PyQt front end, since I'm running a Qt-based GUI on my PDA.

I guess the bigger problem would be finding a system that would allow me to automatically merge changes made to multiple copies of my journal/knowledge base. (Or extending an existing one to allow this.) I would want to be able to make changes on my PDA or my workstation or my laptop. Ideally, changes made on any one would merge into a "main" data store without any manual effort.

Maybe this is more trouble than it's worth. But on the other hand, all my notes for the last 5+ years take up less than 4M in their current text format. I could easily fit that on a CF card in my PDA, and it would be awfully handy to have them all in my pocket.

There's nothing like burning half a day on something stupid.

Last night I tried to test some changes to a program I hadn't re-built in a while, and when I ran it I got the following unhelpful message:

no such file or directory: /tmp/Verify_Cache

I scratched my head for a bit, then worked my way around to the idea that it could have to do with shared libraries, so I ran ldd on it and got:

/usr/bin/ldd: /tmp/Verify_Cache: No such file or directory

Some time later, a co-worker suggested using the LD_DEBUG environment variable. I tried that, and got the same error message as when I tried to run it the first time.

Much head scratching later, I started to look at the contents of the binaries with various other inspection tools. That's when I noticed that while "readelf --program-headers" on working binaries showed:

[Requesting program interpreter: /lib/ld-linux.so.2]

On my broken one it showed:

[Requesting program interpreter: /usr/lib/libc.so.1]

Which doesn't exist on any of my reasonably modern Linux systems. I made that a symlink to /lib/ld-linux.so.2 and my broken binary suddenly worked.

It turns out that the build instructions had gotten a little stale, and this binary had "-static" on its link line but also had shared objects on its link line. Garbage in garbage out, I guess. Still, it seems like there should be a big "attention jackass: you told me to do something stupid so I did" warning in this case.

P.S. I'm wearing my "It must be user error" t-shirt today. I obviously picked the right one, I just didn't know it was referring to me.

16 Oct 2003 (updated 16 Oct 2003 at 14:09 UTC) »

Maintaining free software for pay can be tough.

I guess I'm lucky that my employer is willing to subsidize my pet project by paying me to work on it. Then again, they depend on it for a project that they seem to think is pretty important.

On the one hand, there's all the stuff my fellow team members at work need. Bug fixes and performance enhancements mostly. The code is plenty mature, but we're in that last 10% that takes 90% of the time. It has pretty much all the features they need, though there's a trickle of new ones for them.

On the other hand, there's a boat load of stuff I see that would make Vesta more attractive to others: installable packages for popular distributions, secure authentication, ports to new platforms, a make-based source distribution, tools to help with merging branches, notification/triggers/process control, the list is pretty long.

I'm basically fully occupied with the first category. It's difficult for me to find much time to spend on features that won't directly affect the effectiveness of my co-workers (not to mention justifying spending time on such things). Then again, the more people use Vesta, the better the chance is that people outside my company contribute to its maintenance (bringing down long-term workload). It's sort of a catch-22: working on the first category is higher priority, but the second category is what will get more people using and contributing to the project, reducing the amount of stuff in both categories.

I'm thinking about this a lot becuase last month a user outside my company advanced an idea on something in the second category, and I sort of said "Sounds great, but don't expect me to do it". Rationally, I know I have limited time and I have to avoid committing to implementing significant extensions. Emotionally, I really wish I could devote a month or two to implementing the features this user obviously needs. But I just can't see myself having the time to do that.

So I said what I said and went back to the umpteen different issues I'm up to my elbows in. I can't help feeling like I made a big mistake there. I'm desperate for more users (and, more to the point, contributors), but at the end of the day I have so little energy left to do what it takes to go get them. We've got the better mousetrap, and, contrary to the saying but as is so often the case, the world cares not.

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!