Older blog entries for gstein (starting at number 101)

We released Apache 2.0.35 as "gold" last Friday. Woot!

2 Apr 2002 (updated 2 Apr 2002 at 11:56 UTC) »
jcv: thanks for the pointer. (and thanks to cmiller) this entry was posted using advodiary.

Shifted a milestone for Subversion today, and spent a bunch o' time rebuilding everything (httpd, apr, svn, etc).

Woof... spent even more time catching up on Apache and Python email. So much email... so little time...

Heh. Standard delay here...

Subversion has now been self-hosting for over six months and we're approaching Alpha (woo!). I started working for CollabNet, and get to spend much more time with it. Besides coding, I also have the opportunity to get its milestones set properly.

Since my last diary entry, ViewCVS is now completely using EZT and is fully internationalizable. Neat stuff

mod_dav was released, but Expat and edna weren't. And I never really found the time to get the HTTP stuff for Python completed. It may go into Python 2.3 instead.

Dang. A day ago or so, I saw a little Python script to use $EDITOR to craft a diary entry and post to Advogato via XML-RPC, but now I can't recall who wrote it. Grr...

Not a lot coming up dev-wise. More work on Subversion and some new hiliting/cross-ref stuff in ViewCVS. And maybe call edna 1.0 finally and release a new version (way low priority). Bigger priority is Expat. It has been a long time since a release, and people keep running into some of the same problems... Need to talk to Fred.

All right. Been four months since my previous diary entry.

I got married in July. Went on the honeymoon. In August, helped Subversion go self-hosting. Usual hang out stuff. In September, travelled to Seattle and NYC. Cancelled a trip to Vegas after the WTC tragedy. And now we're in October.

Details are left as an exercise to the reader :-)

Currently, I've got a number of things on my plate which I'm trying to deal with in the short term:

I think that is about it. Way too much stuff, and way too little time. Sigh...

6 Jun 2001 (updated 6 Jun 2001 at 07:22 UTC) »

Woah. Just read about geocaching. Very cool stuff. Sent off an email to some friends to think about some camping/geocaching. Even without that, there are a number of caches nearby that I can go track down. Of course, there is a small matter of buying a GPS receiver... :-)

Did some SWIG work for SVN over the past couple days. Gotta get the bindings up to snuff to be able to do the cvs2svn tool.

Been integrating some patches from Mo DeJong to fix SVN's vpath builds. Seems like that has been fixed.

Not much else going on right now...

More work on the SVN build. I think that everything is licked except for vpath builds, plus a tiny smidgin of work for building the docs (from texinfo files).

After that, I'll be going back to work on the cvs2svn tool. The precursor for that will be the Python bindings to the FS library. That should be fun, and a learning experience for SWIG.

And yes, more PS2 action. A friend was in town Wednesday. We went and rented Crazy Taxi (CT) and Zone of Enders (ZoE). CT was fun, but only for a short while. There isn't really any advancement, opening of new levels, new cars to open up, etc. It is pretty much "here is the game. keep playing until you're better and can get a higher score." Same thing over and over. Ah well. Good thing that I rented it :-)

ZoE was actually quite cool. On the "easy" setting, I went through the whole thing in seven hours. I actually felt a bit short-changed. I had thought that I'd be going to Mars (the action takes place in an orbital station around Jupiter). But it actually ends before leaving. Too bad. The game was actually quite decent, but some of the fighting is actually a bit too easy. A Burst sword attack takes down the enemy in just a few hits. And being so close, they usually can't use their heavy weapons on you. I think it would have been harder if there was a meter on the use of the Burst. The graphics and the motion and the screens were quite awesome. Some of the gameplay (go here, get that, go there, use it) was a bit slow, but I actually don't mind games where there is a lot of mindless hack and slash (on the way to recovering things).

And in an embarassing moment, I trashed the edna repository at SF by doing a partial change to the CVSROOT to add checkin emails. I forgot to "cvs add" some things first, so the new files didn't go into CVSROOT, but my changes required them to be there. Bam! I couldn't check anything more in, so I couldn't even fix the darned problem. Sigh. Should be cleared up Monday when the SF admins reset the CVSROOT.

Released ViewCVS 0.7 tonite. Lots of little bug fixes (mostly dealing with oddities of CVS branches and other edge cases). Some performance improvements. Some new features, such as persistent URLs to the "head" revision and tarball generation.

Also spent some time cleaning up odds and ends with the new Subversion build system.

Weekend was pretty much a no-op. Spent a lot of time playing Summoner on my PS2. Excellent game. Although, I didn't get my summoning ring until I was level 19. To throw skill points into my Summon skill, I've got to gain levels. Feh... up to 19, it isn't too difficult. But you pretty much have to double your total experience to get to 20, and double again for 21. etc. Given that I've been playing for something like 17 hours of real time, that means that 21 is going to be pretty much impossible to reach in any respectable time. Hopefully, I'll find some badass monsters worth 10k experience or something. At the moment, the monsters that I'm fighting are all less than 1k. I chew threw them like a rabid dog with a sack full of kittens, but it will still take a long time at this rate. Not a big deal at this point, though, as my guy is a tough mo-fo; who needs to summon creatures? :-)

Ah well... it just means that I'm now concentrating more on completing the quests and the game rather than busting heads. Summoner is also kind of neat in that you have multiple party members. Their levels are jumping as we teach those monsters who's their daddy.

It is kind of funny actually... I was zooming through this crypt. "Oh, there are three monsters." I wontonly run my guy into the center of them and call down a Meteor Shower. Anybody that survives takes maybe a hit or two before they're dust. Of course, with two other people and a summoned elemental, we made quick work of the monsters. Then I get to the end of the crypt and meet the "boss", named Carados. I start bitchslapping him, and "oh. somebody is casting spells at me. gee, that tickles. hey! elemental! go smack up that Bone Mage." Carados went down with nary a scratch to the party. Heh.

Me thinks I might be screwed towards the end of the game, though, where I'd truly need that Summon skill way up. Just have to see. For now, I'll just keep crushing things. :-)

Completed and installed the new build system for Subversion. It is quite sweet. Fast, simple, and flexible. Sure beats automake for our particular use-scenario.

I could see that automake is good for FSF/GNU projects where they have a particular set of requirements, but it isn't very flexible when you have different policies. It also creates some performance issues (as I mentioned in my previous diary entry).

Shelved the rcs parser for a while. I cranked out as much performance as it can possibly get (without making the code look *really* horrible). Overall, it is somewhere between 10 and 12 times faster than when I started. For small RCS files, it is comparable for forking off rlog and parsing the result. For large files, though, rlog/parse is faster. I think the next step is to use something like mxTextTools or to use a custom RCS file tokenizer. The internal architecture is set up using a "token stream" plus the parser. That should make it easy to swap in different stream implementations.

I tried using mmap, but it was no faster than just reading the darned thing into memory (in 100k chunks). It is simply that the algorithm is not I/O bound, so using mmap to optimize the I/O doesn't help at all.

Over the weekend, I've been working on revamping Subversion's build system. We currently use automake. It is a total dog and some parts of automake are actually a bit hard to deal with. I've tossed out automake and recursive makes, with just a single top-level makefile. The inputs to the makefile are generated by a Python script. Net result is that ./configure will produce a Makefile from Makefile.in, and then the build-outputs.mk will be included by that. build-outputs.mk is generated by the Python script when we create the distribution tarballs (so end users don't need Python just to build; this is similar to how automake uses Perl, but the outputs are portable).

The resulting build process is much faster. ./configure is also going to be speedy since we only need to process one Makefile.in. In addition, automake creates a billion "sed" replacements within configure, then applies all of those to all the files. We'll be reducing the replacements to just a couple dozen. With the reduced file count, it should scream. We also don't have automake's time consuming process (producing Makefile.in from Makefile.am); my Python script executes in just 2 seconds of wall clock time. That includes examining all the directories to find .c files to include into the build.

I've got make all, install, and clean working. I still need to do distclean, extraclean, debug the "make check" target, and then do dependency generation. On the latter, the Python script will just open the files and look for #includes. This will be much more portable than automake's reliance on gcc-specific features. Oh, and we also get rid of automake's reliance on gmake.

Nice all around...

Been working on optimizing the RCS file parsing module (within the ViewCVS package). Having Python fork/exec with a pipe to "rlog" is still a lot faster than having Python directly parse the file. But it is getting closer. I'm now going to try memory-mapping the file and parse tokens that way. Could be much faster.

I want this to be really fast because it would be nice to use manual parsing rather than rlog output since there is a small amount of data loss. In particular: it is hard to reconstruct the actual RCS revision tree from just the rlog output. (hmm; maybe "hard" rather than "impossible")

The second reason is that this module will be used by Subversion's cvs2svn tool. To convert SourceForge's 49 gigabytes of CVS repository, I want this to be as fast as possible :-)

92 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!