Older blog entries for tromey (starting at number 23)


Tadpole. I loved this movie and laughed out loud quite a bit. (I'm a sucker for movies shot on DV.)

Build Tool

Last night I looked into making the build tool impervious to comment and indentation changes in C source. This turns out to be quite easy conceptually: you simply base the file's signature on a tokenized form of the source file. This is feasible if your tokenizer is relatively fast. I'm putting off writing this code, since it isn't entirely trivial and there are more important things that must be done first. I imagine it would be most useful to enable this by default for .h files in the project, and use a (presumably) more efficient text-based scanner for other sources.

I've also figured out how to make "config.h" files work well. The idea here is to change cpp to output more dependency information. Then config.h can compute its signature based on the defines actually inspected by a given compilation. This means that, say, adding a test to config.h will only cause a rebuild of those files which actually use the new define. Something like this would be nice for large projects.

Both these ideas are easy to implement in the current framework.

I've been re-reading the Odin paper again. I always forget how different this approach really is. Odin doesn't put intermediate files like a .o into the build tree. They exist only in the cache. I doubt I'll be going that route. For one thing, I've found that I occasionally look at the .o files to debug some problem or another. Also it feels "too different".

Tonight I'm just working on general cleanup.

Build Tool

A couple nights ago I added derived object caching (about 20 lines of Python code) and command file dependency tracking (if /usr/bin/gcc changes, your program is recompiled). Last night I changed the gcc tool to know how to do automatic dependency tracking.

Derived object caches are a bit funny to work with. For instance, rm *.o doesn't cause a rebuild. It just pulls things in from the cache. This "feels weird", perhaps due to my long history with make. Signatures also are a bit odd. For instance, the build tool is impervious to touch. Another example is if I add whitespace to a C source file, the resulting executable isn't relinked (since the object file doesn't change). (This stuff all works right now with my prototype.)


Recently I've been trying to build Eclipse with gcj. This is an adventure, since Eclipse has 75 jar files (for now I'm compiling from bytecode) and uses all kinds of packages, including things it shouldn't like com.sun.tools (hiss). I've also tried getting Eclipse to run with gij; this has shown me that working out the library issues is only the start of my problems.


Austin Powers 3. Better than I thought it would be. I found the opening sequence particularly interesting, and inventive. Parts lagged, though, and by the end I was ready to leave.

Build Tool

raph brings up some interesting points regarding language choices in a new build tool. There are a few points here that I'd like to address.

First, in automake we always separated the implementation language of the tool from the language the user used to make things happen. There have been many requests over the years to blur this barrier, but I never regretted being firm here. Blurring this line basically means you must be very confident in your particular implementation, since giving access to it means it will be much, much harder to change.

My earlier questions were really ones about the implementation language of the tool. Is it ok to have a dependency on some other, larger project (Python or Perl or ...)? Of course that isn't a question with a fixed answer. It depends on what you want to accomplish. One of my goals is wholesale replacement of the auto* tools. This makes the question particularly hard.

The user's language is of course extremely important. We've all seen how autoconf's quoting and mix of m4 and sh is confusing and limited. We've all seen make's many syntax problems.

The tension comes because in some situations the user language has to have some power, and it is tempting to just use the same language for both. For instance, expressing autoconf-style configuration requires expressive power.

This is all "front end" stuff. I still haven't really started the front end design beyond kicking a few ideas around.

raph, I'd be interested in seeing rebar. How about sending me a tar file?

Today I finally put Advo on my personal toolbar in Mozilla. I find I really enjoy reading the Advo diary entries, and I look forward to them. I think it is because some people put real thought into their entries, so they can be good reading. Also it is interesting to see what other people are working on or thinking about.

Today I did something I ordinarily dislike, which was rewiring so that I could boot the alpha box I have here. Honestly I think Red Hat forgot they ever sent it to me. It's nice for testing gcc; today I'm using it to test a gcj patch for a bug that occurs only on 64-bit architectures. What I really need to do is move that box plus the other old one to the basement and run wiring down there. Then the noise won't bug me so much. Given my general level of hardware procrastination, I think it is safe to say that will never happen.

Build Tool Rambling

Last night I did some hacking on a build tool prototype. Basically I'm trying to test some of my plans against reality; earlier hacking in this vein led me to revise how the database is laid out. Sometimes I find it easier to follow logic in the code, while I'm writing it, than I do if I try to plan everything about ahead of time. In situations like this I just consider the prototype code as part of the design process.

Anyway, the prototype tool is smart enough to avoid re-running a rule when the output won't be different. And since it is using signatures and not timestamps it is impervious to touch.

It is awfully daunting to have 500 LOC in a project and know you need 10,000 LOC.

There's a fire in Steamboat today and here in Boulder the sky has been yellow and the air smells of smoke. Ugh. It's almost enough to make me look forward to winter.

Today was pretty slow. I need to make some kind of change here.


Saw XXX today. This movie somehow managed to fall below my already low expectations. Amazing.

I'm finally back from my vacation. It was wonderfully relaxing and made me want to hack again. Nice.

Build Tool Vapors

I've done a lot more thinking about a replacement build tool. True to form, I thought about it a lot on vacation. I've written some design docs for the back end, and started thinking about the front and "middle" ends, but so far nothing suitable for public viewing.

One implementation issue I struggle with is what language to use. I wonder whether people will be willing to give up the portability of the current auto* tools in order to have a somewhat cleaner build environment. My current preference would be to write a new tool in one of the common scripting languages, probably Python (I don't know it, therefore I can't dislike it :-). Luckily implementation is a long way off, potentially infinitely long.


My Big Fat Greek Wedding. I finally saw this because several friends reported back enthusiastically. I thought it was quite boring. Also it espoused a very stereotypical view of Greek-Americans, on the borderline of offensively so. I guess I didn't see the humor in it that other people did.

A few weeks ago, before vacation, I saw The Sting at the outdoor theatre. Even though I had seen it just a couple months before, I enjoyed it again. Some movies I can see over and over.

Today I read through mjw's diary entries. I also have a difficult relationship with free Java efforts.

On the one hand, I'm proud of gcj. I think we've done some great work. On the other hand, we're still missing pieces that people seem to consider fundamental (AWT...). We still haven't found our killer app, that gets gcj widely used.

Also there's the issue that people still consider kaffe the premiere free Java. Being first really does have its benefits. Lately this hasn't bothered me much. I just work on gcj (less frequently right now), trying to make it as good as I can.

Mark has a very good point when he says that people in the community seem willing to build their projects on non-free infrastructure. From one point of view, the one I most commonly take, this is a mistake -- that work helps to enhance the hegemony of Sun, against the free community. On the other hand, I suppose one might consider it as a pragmatic approach similar to that taken by the GNU tools in 1990: we're doing this now, but when a free solution comes along, we'll migrate. Though so far the migration hasn't happened. And with the rate of change in the Java specification, it may never happen.

So, this is a concern. My own preference has been to use only free tools, to the extent possible, as soon as possible. I usually prefer to suffer a bit with less-than-perfect free tools than to support the non-free ones that may work better. I don't mind finding and reporting some number of bugs. The counter argument is "I have to get work done". Ok, fine, but let's keep an eye on the long term, the important thing: the free software.

gcj still has some legs. If you look at the web page you'll see a steady stream of news items as people add things to it. And more things are coming: Andrew has written a new inliner; we'll probably (eventually) get some nice optimizations out of the tree-ssa work; Bryce is working on the AWT merge with Classpath (when this is done I'll probably dig up some cycles to work on the peers again).

There are plenty of cool projects for interested hackers, too...

Lately I've been thinking quite a bit about the next generation build tool. I've been writing a design for the back end. Don't know if or when it will be available.

I was in Toronto last week, so I dropped out of my normal routine: little email, no Advo, etc. It's nice to do that occasionally, and another blackout starts next week...

thomasvs: I've never read Written on the Body, but I've read and enjoyed several of Winterson's books. I read The Passion about once a year; I've probably given it as a gift more often than any other book (or object).

While in Toronto I talked to Ben Elliston (who I think isn't on Advo and thus, in a sense, not a <person> :-) about requirements and thoughts for a new build tool.

Some other important (though perhaps more minor) considerations that I didn't mention before:

  • You need to hack the tools to record untaken dependencies (see the automake dependency-tracking paper), and also to tell the build tool about the other programs they exec. This might seem like overkill, but it is required at least for correctly building gcc.

  • Targets and rules require separate namespaces. That way you can still easily write a program named install or info, both examples which have actually occurred.

  • Well, there was more stuff, but I've forgotten it once again. I think I have it written down somewhere though.

Lately I've looked at ant a bit, since Eclipse uses it. I'm a bit discouraged by the use of XML. I find it hard to figure out what is going on, there is way too much text overhead. I haven't really followed the current mania for XML, in that respect I'm either lazy, stupid, or hopelessly backwards (take your pick). I haven't come to a definite conclusion regarding ant yet, given my relative lack of experience with it. I'd be interested to hear opinions.

14 Jul 2002 (updated 14 Jul 2002 at 18:05 UTC) »

automake: Today I back-ported the versioned install code to the 1.4 branch. The Gnome hackers, particularly hp, convinced me it would help them transition to 1.6 more easily. So, despite my wish for 1.4 to simply die, it lives on... expect the next "final" release soon.

Really this problem comes from some deeper problems with the automake process dating back several years. Basically, I (at the time there weren't other active automake hackers) didn't do a release for 2 years, and when I finally did it was too divergent, had too many new features, and too many bugs. So now, even though 1.6 is really quite good, people are still stuck. This is made worse by the massive changes autoconf has gone through in the meantime.

Lessons continue to be learned here. (Which, incidentally, is unpleasant since it means we are still making mistakes.) We haven't provided smooth upgrade paths from release to release, nor have we even documented the way to upgrade. That hurts the users.

These probably seem like simple things to get right. And they are. Unfortunately for everybody, I still seem to approach automake as though it is the same project it was when I started: a small script, used by a few friends, which I can randomly hack to pieces without consequence. But that world view is really out of sync with reality now. I don't know what excuse the other developers have :-).

As you can see the changes must start on the inside, with the mindset of the developers. Over the years automake has gone through some process upgrades. For instance, we have a documented release process. However, things like this help only in limited ways: when misused they can be an attempt to address an internal problem (my carelessness in making certain changes) via an external method (increased process enforcement). If this works at all, it can only be in limited ways.

sab39: Your code in Classpath didn't suck; it was fine. During the libgcj/Classpath merge, we tried to preserve the best qualities of both implementations, and as a result everybody "lost" a little bit of their work. Though I have to admit, I dislike seeing my code rewritten, especially code I put real thought into writing.

Uche: Your poem impressed me. I've never understood how people can write verse. I don't know if you remember it, but we met at the Boulder Creek Festival a few years ago. I still have your card; it floats on my desk, and occasionally bobs to the surface with the current.

I'm on a lot of mailing lists. I wish Gnus had a mailing list wizard that would let me just enter the name of the list (and maybe the type of list manager). Then Gnus could subscribe me and set up a new group for it. And it would remember my password so I could just give one command to, for example, change my email address on all the lists. Or unsubscribe without remembering anything (other than the right command). Or, better yet, automatically detect whether Reply-To munging is in effect and then set the mode in the new group appropriately.

Tonight I wrote a patch for glib. I don't remember ever doing that before, but the glib ChangeLog does :-)

14 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!