Older blog entries for technik (starting at number 12)

24 Apr 2001 (updated 25 Apr 2001 at 02:58 UTC) »
Frustration. Demotivation. Ennui. I guess it's business as usual. Weeks of HTML, JavaScript, kludgy Java Servlets, and the hangover of years-old decisions. Wondering if I'm cut out to be a programmer.

My build system has been distributed to the development teams and comments are trickling back. I'm averaging two calls a day regarding it. So far, every problem is the result of not following all four (4) steps of the installation instructions, using a buggy JVM (pre-1.1.7B, that was released in 1997/98 !!!), or having a "unique" local directory hierarchy and not adapting the paths held in the external properties file. RTFM.

Completed teaching a section of Intro. to Unix using Linux as the OS. Despite my enthusiasm and that of much of the class, the answer to the unasked question, "How does one fail a continuing education course?" is, to my disappointment, "Don't show up and don't turn in any work." Ended up failing about two-thirds of the 26 adult students because they neglected to hand in three or more of the five assignments which constituted 50% of the grade. Were they expecting a visit from the homework fairy? Teaching another section and hopefully it will go better.

Digging into the Camel, a few RFCs, the Stevens book and an example posted to Usenet, I put together a nice ssl-tunnel in Perl which supports Basic and Digest (MD5) authentication to the proxy. Works under Win32 and Unix and under 'use strict'. Played around with Socket programming under Python for comparison and will see how this looks as classes and methods. Python's urllib2 module might make it trivial. Also scribbled an introductory article on bypassing firewalls via a HTTP proxy since it's fresh in my mind.

Organizing my ideas/notes for an article on build automation using Ant, Jython and (ack!) PVCS. While I was at it, I put up my Egoless Admin article on how to handle "difficult" user situations.

For reasons I don't even know, I'm looking at Smalltalk. Downloaded Squeak. Started experimenting and reading.

Wrote a very brief article on adapting the BSF (Bean Scripting Framework) to use Jython which I put up on my site under the articles page. It's trivial but it might save someone some time.

Go, Webmonkey! Go! A corporate update is underway and my team is retrofitting the new look-and-feel onto existing products. The old stuff was ugly and the new design is beautiful and easier to use but rehabilitating the stuff built by partly WYSIWYG editors is hell. The stuff violates the HTML 4 spec., renders differently in each browser, and is peppered with unnecessary tags and attributes.
Facing fifty-odd static and dynamic pages myself, and knowing that there are a few hundred in total distributed across the team, I reached for Perl (ask for it by name!) and Sean Burke's updated HTML::Tree module which I recalled reading about in The Perl Journal a few months ago (Aside, does anyone at liberty to say know with certainy what is going with TPJ?). A little preprocessing, a little post processing, a run through the parser and I had reduced by more than half the amount of manual editing. Compare the below code snippets to running search and replace in your favorite editor. Compare it for 50 files, or 100.

use HTML::TreeBuilder;
my $tree = HTML::TreeBuilder->new();
# deleting font elements
  if (@tags) {
    foreach $tag (@tags) {
    if (@tags) {
        foreach $tag (@tags) {
            if ($tag->is_inside('table')) {
                # apply style to <td> that holds this element
# More madness... illegal WIDTH="%" attribute must be removed
   if (@tags) {
     foreach $tag (@tags) {
I'm going to have to buy him a beer sometime.

Attended the New York Perl Monger's meeting last week. Met a few of the local mongers and Randal Schwartz, who was visiting. Nice time.

Gave an older computer to my 92-year-old grandmother. She had expressed interest in getting online and I had the pile of parts my wife had used for running MS-Word. She is a remarkably quick learner and was enthusiatic about it. In a little over an hour, she had gone from never having used a PC to managing quite well. She was satisfied for the time being with Solitaire and FreeCell (yep, you know what's coming), but hopefully will get comfortable and go on-line. I seriously thought about putting Debian GNU/Linux on the box and Mozilla but my wife pointed out that:
  1. She probably needs as much help as possible.
  2. We live two hours away and don't have a "normal" schedule.
  3. I'm the only person in the family who can support Unix.
So I knuckled under and gave her a system running Win98, IE5, and AOL. The horror.

A new advogato member recently certified me as Master. That's damn funny. Not only am I- in my opinion, even if not in the opinion of the people who employ me- merely an adequate self-taught programmer but I've made as yet no meaningful contribution to Open Source projects.

The certification on Advogato is based upon what you have done, what can be downloaded and seen. Reread the Certification Overview. At this point, I'm still in the burgeoning category of those who directly benefit from OS, but have not returned the favor. With the very thin exceptions of evangelism- demonstrating the value at work, using OS as teaching material, a couple of articles that are unpublished, being a newsgroup and listserv hanger-on - and a lot of good intentions, I haven't given anything back.

That's not Master material, that's barely Apprentice.

I'm not flaming, but the fact that someone mistakenly certified me stirs up a few internal conflicts. I "get" it in the bigger post-60's era sense, I just haven't done anything with it and that gives me a niggling guilt since I am (or, at least , may be) in a position to do something and return the favor to the community from which I benefit.

Back in December I cobbled together a new build system using Apache-Jakarta ANT and JPython. It has been simmering on my manager's desk- he still codes despite the meetings- as he worked through it and made changes in between other work. Yesterday, during a scheduled build for the primary development server he substituted my scheme for the original method and, Lo and behold!, it worked. It was entirely transparent. I met with some of the SCM people while this was going on and detailed the new system warning that at best it was a couple of months off. I'm going to revise that... it's a few days off and I have permission to give the new build method to the rest of the team. If all goes well, and it should, a total of sixty-odd developers will be using it and all of our QA and Production builds will be made using it. Building any product or even a full platform and suite for each of the six environments (don't ask) is now a simple two or three word statement.
I'm a little proud and a little worried. I already have visions of someone six months from now carping about the idiot design and bad decisions. I console myself that it is a step forward even if no one will ever see it outside these walls.

Done fighting the DB. My stored procs work and my kludge in Java to periodically extract the data and send it on its way is ready for testing. Seems I can't really test it in development because the data I need doesn't exist and the target host is on the other side of a intranet firewall (the result of intra-division politics). We slipped into a QA system and manually ran the queries which returned what we expected (the QA tables were missing a few records so we cleaned up a little while we were there).

Lately, I've become the contact for all things regarding PKI and digital certs. I have OpenSSL on a couple of covert boxes running OpenBSD and Progeny Linux for my own testing. Word got out that I'm running a CA and at first just members of our team used my server certs. Now other teams are hunting me down for certs so that they can do their development without buying certs from Verisign (no link, they don't need my help) and idling until they arrive. It's not a big deal except that every product seems to want its certs in differing formats. Maybe it's my own ignorance but their respective documentation isn't explicit so it's a merry-go-round until we find the right options. For my own sanity I'm going to clap together a few shell scripts to do this for iPlanet, WebLogic, Apache mod-SSL, WebSphere, Netscape Navigator and IE. The OpenSource PKI Book and the archives of the OpenBSD mail lists are real blessings when working this stuff out.

Wrestling with extracting and updating user account information needed by a system in another division from the viciously evil, unnormalized database upon which all our web-based products rely. This is done by hand now and needs nightly automation before some products go live. I'm past the swearing and blaspheming stage and on to working it out.

Spent time thinking about the Java-based file transfer scheme. Ignoring the details of file transfer, and setting aside half-baked thoughts about java beans, the idea of an generic threaded scheduler for tasks in Java is a good idea (and one that I've seen implemented a couple of times, badly).
The scheduler should have the following characteristics:

  • multiple task threads.
  • optional multiple tasks per thread (timed queue).
  • on-the-fly addition or deletion of tasks.
  • automatic destruction of threads without tasks.
  • per task and per thread logging.
  • simple to configure.
  • save/read configuration to/from file(s).
Ideas for it that I'm not looking at now:
  • conditional execution of tasks.
  • execution of native system executables or scripts.
    Is there any portable way to handle signals or return codes from external processes under Java?
I've hacked together a very rough prototype for the threading and tasking, something I've entirely scrapped twice. More reading and examining other people's threading code is clearly in order. I'm also thinking that embedding a http server/servlet engine like Jetty would be a good way to provide a frontend. Jetty would be great if there were adequate documentation... the javadoc and the demo are not.

Finally settled into the apartment and it feels like home. Six small boxes remain to be unpacked, the framed pictures need to be hung, a bunch of paint needs touch up, and a hundred little maintainence tasks are waiting. This weekend I'm installing closet organizers and shelving and hopefully will have time to set up the three remaining computers.

Back in the USA but still on GMT+1.

Got home to find the wall outlet powering the fridge is mysteriously dead even though the breaker is okay and other parts of the circuit are okay. Not so nice things happen to frozen foods in a dark, warm, unventilated, insulated box. I spent a few hours cleaning up the vile mess of foodstuffs that went from solid to liquid (and some to solid again).

Powered up my main system after two weeks of downtime and the video card is toast. Since the box was plugged into a UPS but was off at the switch the whole time, I'm wondering what the hell happened. At least it registers so that boot can complete and I can ssh into the box. Now is a good time to finally put in a AGP card, I suppose. I have some 1700+ messages from the various lists to which I subscribe waiting, all nicely sorted by procmail but I'm too lagged and lazy to try reading them now.

I have some fifty-odd photos of the trip to get developed.

My manager is out for a few days, my VP is on the other coast this week, and I'm not assigned to anything, so I have some time to fritter. Installed Debian potato then bumped to Progeny. Easy enough. I'll have to see how stable it is.

Online, but it looks like the trouble I took to run fresh copper from the termination block under the floor to my modem wasn't worth the trouble. These apartment buildings have old phone lines- cloth-wrapped, solid copper oxidized black- which assuming no breaks will do nicely but they run next to who-knows-what. Probably through the same channel as the electricity judging from the 60Hz hum on the phone.

At least I have 28kbps. Hopefully my current ISP will resolve their issues with mail before I make the jump to broadband of some kind. Either way, I'll be away for a few weeks so it hardly matters.

12 Feb 2001 (updated 12 Feb 2001 at 18:12 UTC) »

Spent four days renovating new apartment. All the schedules were fubar. Not a single delivery arrived on time, not a single event occurred during its window. These are lessons in project management- never have more than one task at a time on the critical path and avoid scheduling your tasks based upon someone else's coordination.
Moved in while the subcontractors were working on the floors. Everything has a fine layer of sawdust. Boxes everywhere, can't find my work ID. We're camping in the living room as we set up the rest of the space. I couldn't find the alarm clock last night but it didn't matter because without blinds on the windows, I get a 6:30am wake up from the sun. Oh, yeah, that's what daylight looks like!
Yesterday I wrapped my potted tree in a couple of black vinyl contractor bags and duct taped it tight and shut, put the top down on the Cabrio, and drove up I-95 through 34F temperatures with a 35Mph cross-wind to the new apartment. I got a lot of gawking stares. I've had it since it was a eight-inch tall desk plant and I'm not giving it up just because the movers won't transport plants and it won't fit in a SUV. It should really thrive in the bright light of the new place.

Last Thursday, the morning we started fixing up the new place, I dropped my three-year-old Palm Pilot in a fit of caffeine-deprivation so in addition to being offline, I'm also out of luck. Year 2000 was the first year I didn't keep double-entries in a paper planner and the pilot, part of weaning myself off the physical book.
Got a new pilot, the 3XE, and found that I can't restore the 400 or so entries in my Address book. Or the 2700+ entries in Datebook4.
At least my memos and the account data I keep in Strip successfully restored. I hope to sync the old one and restore it to the new one. I use JPilot, a GTK wrapper for pilot-link, but neglected to install the better backup conduit so this might be a no go.

Amusing Observation:
You can tell real geek by the order in which he sets up his apartment. This time it goes: bathroom (1st), kitchen (2nd), and computer (3rd). Other rooms follow. Had my significant other not insisted, the above order would be reversed and I would be back online. The only reason I have a system running is because she agreed (more shrugged assent) I should sync up the new Pilot.

Unsurprisingly, I've made little contribution to any software. Not at work- where I've turned up three days in the past two weeks- or on my own stuff. Next two weeks are vacation, so I'm not going to get anything done then either. One thing I probably will do is finally spend some time on Scheme. LispMe for the Pilot and a few documents will accompany me on my trip.

3 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!