Older blog entries for taj (starting at number 5)

15 Jan 2001 (updated 15 Jan 2001 at 12:13 UTC) »

After fiddling around writing applettes (not applets!) with PyQt, I've taken the plunge and started writing a small database-backed (using gadfly at this point, what a cool python hack!) personal expenditure program, somewhat like cbb at this point. This serves me in a few ways:

  • Find the strengths and limitations of PyQt
  • Doing some real KDE hacking after a long interval
  • If it turns out to be a killer app (heh heh) it may lead to PyQt and PyKDE being accepted as a viable alternative to writing serious applications for KDE.
This last one is of particular interest to me, because I feel many of the smaller applications written for KDE could be written with PyQt in a fraction of the time (I've empirically tested this theory so feel free to take my word for it). However like most language bindings it will take larger, compelling applications for users to accept the need to install these extra libraries, and for python-based applications to be accepted as a standard part of a KDE install. Hence my little financial programming distraction.

Off to Sydney tomorrow morning! whew if there is a better excuse to take 5 days off work than heading to Australia's biggest geek party yet, I've yet to hear of it. I might even get some time to play with PyQt when I'm not basking in Linux Gods' light.

And when I get back, it'll be almost time for my big day out! Hmm let's see... Bizkit, Rammstein, PJ Harvey, Superheist, Powderfinger etc... etc .. Carl Cox and the good old Melbourne doof-doof music mainstay, Phil K in the hothouse. Choices, choices... if anyone reading this is going, remember to TAKE LOTS OF BOTTLED WATER - the punters were paying 4 bucks for a bottle of water last year. I'm sure there's some sort of law against that.

A long and somewhat self-indulgent story, but perfect as a diary entry.

About 4 years ago I replaced the old foxpro-based timetabling system for the department at which I used to work at my old university. I rewrote it using C++ CGI and mysql on linux, it took me about 8-9 months to write. A lot of time was spent coding around the weird schemas that we had to keep to maintain compatibility with the old foxpro-based administration interface.

For a good few years it ran beautifully on the only "server" they were willing to give me on their meagre budget, a tiny old (now) 486/66 that didn't sweat under the load for many years.

Last year however, about a year after I moved on to the corporate world for good, they figured it was running so well that the rest of the university decided it was time to buy into our system and soon little old "lift" was handling over 70% of the university's 30K undergrad students, allowing them to view, edit and approve their timetables over the web.

Lift was finally at the end of its useful life as our timetabling server. Load was at around 15 most of the day, mostly because of the CGI and the size of the database. So I took a few days off to go and help the guys out and moved the entire system to a P2/350 they had sitting around. Everything was ok again, and load came down to ~ 1.5 at the heaviest periods of usage.

I was left with a nagging fear that even though I had put a lot of effort into writing the application well, the cruft from the old admin schemas and the fact that the whole thing was C++ didn't bode well for whoever took up maintainence of the system and it was likely that they'd come running to me for help whenever they needed a new feature. So last week I redesigned the schemas rewrote the whole thing in zope and a little python, including the admin system, in about 10 days of hacking whenever I felt like it.

MY god how far web development has come in the last few years! I can only partially attribute this hack to an increase in my own skill and focus as my experience grows, but a few years ago people were still writing non-persistent CGIs in C++ for anything more than toy apps. I'm almost mortified.

I've finally got a really good excuse for my friends at the School of Physics to never bother me with timetabling problems again. I'm happy about this, because they're really cool guys and I'd have hated to leave them hanging on a project that is at once invaluable to the entire university and paradoxically (common at this university), heavily heavily underfunded. They're smart, and I know they'll pick up enough zope to keep the system up to date in about half an hour.

A happy ending for all, a great weight off my shoulders, and an eye opener at the speed at which the ease of developing complex applications is growing.

My giving up gvim for a full-blown IDE, however, seems quite far away at this point. Especially now that I have the brilliant Konqy to complement gvim and konsole.

4 Jan 2001 (updated 4 Jan 2001 at 13:02 UTC) »

Hmmm turns out P::R is pretty slow. What a surprise. I guess it has something to do with not being able to parse directly from a stream, a perl limitation. It works a lot better if I split the stream into statements and parse a statement at a time, but then there's the annoyance of having to break it into statements again. grr

Good party at Emi and Mick's on NYE. They're going to be great together. Too much shiraz cab, painful monday.

TestGridTESTgridtestGRiDgot a week to get this project wrapped up so in can get to linux.conf.au on the 16th without a guilty conscience. I swear I'm too nice a guy.

One of the worst things about dictating that a company standardize on a development platform is that it makes it harder to use the right tool for the job. A few of the projects we are doing could be done in a fraction of the time using zope, but would be "easier" in an infrastructure and management sense, to maintain if we stuck with weblogic for all our stuff. How does one approach this problem? One can't dictate that the company always have an expert in each deployed playform at hand at any time. Otherwise we'd still have a guy just looking after our crappy old filemaker sites.

Additionally more and more of our larger clients are entrusting their internet presence to server-side Java, so our larger projects, technical issues aside, cannot really be done in anything but J2EE.

Zope is really a wonderful thing, already there are rumblings amongst the self-proclaimed "digerati" that python and zope in particular are a new wave in web development. I'm inclined to agree (particularly because I've grown quite fond of python), but both the language and its killer application have some growing up to do. Sun have really put a lot of effort into defining APIs and standards to write the kinds of applications big companies want.

Even though J2EE is large and cumbersome, I do not believe that an interface that addresses such a large number of enterprise software issues has yet been created on any platform. IMHO in the matter of Java vs Everyone Else, Everyone Else has some catching up to do. Most of these technologies have existed for decades in the form of disparate products, but nobody has so far tried to - or had enough influence to - pull it all together like J2EE.

God I can't wait to get to the conference. CALU was so much fun, and this looks like it's going to be bigger, better, faster, MORE!!!!! etc

Filling out and sending yet another timesheet to the pimp reminds me of my life goal to remove all drudgery from my life through computers. I'm sure there's a business plan in there somewhere...

What a month. Low motivation at work for managerial reasons (isn't that always the way?), I'm the only person missing the huge family reunion in Chandigarh, our family town in India. And alone at home, which isn't always a bad thing. I really enjoy my personal space, more so than most people.

I found out the hard way that the regexp handling in perl 5.6 is superior to previous perls, and that means my painstakingly created regexp grammar is almost useless in any other perl - kdoc just hangs in various parts of kdelibs while the regexp engine goes to la la land. So I picked up Parse::RecDescent and must say I'm quite impressed. I wrote up a simplified yacc-like grammar for the original regexp-based parser, and it turns out I can reuse most of it for P::R. I'm really impressed with this package despite its limitations (limited backtracking for example), but if the performance holds up I can just ship the one-file module (75k) as part of KDOC. Then it's back to the generator backends which is what I really want to be working on! And of course it'll give me some time to actually work on KDE again - it's only been bugfixes and small patches for the past six months.

Looking forward to being rid of all the junk I've collected over the past several years, but not looking forward to actually doing it. So much stuff to sell and give away! I'm going to make a new year's resolution to never let this happen to me again, atleast not until I'm certain I'm not moving for a good 15-20 years. let' see - half a dozen (working) computers, a cupboard full of electronic bits and pieces, three guitars and an amp, mounds of stupid furniture... grr what am I going to do? Hopefully selling all this stuff will finance a nice new laptop to carry with me on my travels.

Getting to grips with weblogic server and process integrator, now that our week of slee..er, training is done. It's amazing how quickly my snooze button gets toggled as soon as I sit down in anything remotely resembling a classroom.

Weblogic seems like THE implementation for J2EE. Normally a lot of this "enterprise" software is designed to look good to the people who fork out the cash, ie managers and executives. Weblogic OTOH seems to be designed for the guys in the trenches, like us programmers. I guess I'll only find out what it's really like once I finish a few projects with it, but for the time I'm enjoying learning and using WLS and Process Integrator (another fantastic bit of software, especially if you've tried to implement your own workflow engine in the past).

Good job BEA, and congratulations to whichever company/companies you bought out to acquire this stuff. Dropping the word "Tuxedo" into any chat about your other products is also a great way to remind us that you stake your reputation as a big iron vendor when you sell stuff like weblogic. There's too much half-arsed software out there and I'm surprised and impressed that you put behind your software the reputation you've gained from Tuxedo.

Meanwhile, KDOC is humming along, especially now with the Netwinder set up in the living room! I've been waiting for this Divan Programming Environment for quite some time.

Now that the parser seems to be complete I'm working on the design for the new scoping and tree searching engine. The old model was never going to be enough to handle namespaces and heavily templated code.

It looks like template specializations are still going to prove to be a problem. Preliminary analysis (ie napkin math) suggest that a full-blown implementation of this stuff is going to hit performance quite a bit, apart from being hard to do without spending more time taking the types of the template args to bits. Oh well, one problem at a time...

Next step after that is a major restructuring of the HTML output. It is functional right now but it can be so much better. CSS comes to mind first.

Started swimming again, this is good. It's been too long. Body lethargic == mind slow.

Well I've been reading some diaries on here lately and was thinking of writing one for myself eventually, so this is as good a place to start as any. Let's go:

I've had a chance go get KDOC back to a working state again, one of the very few saving graces of not having Nicole (nicnacs) here with me.

There are few joys like a huge lump of code that compiles and runs the first time. KDOC compiles C++ a lot better now, and I'm amazed at the speed. It's noticeably quicker with the giant regexps I wrote and the

(?>..)
construct that perl offers. I guess this means my old parser and regexps were horrible and that perl's regexp engine is very quick indeed.

Much as I like perl, I'm having second thoughts about using it as a language for anything over a few KLOC in code size. In projects of respectable size, clarity and documentability matter more than the clever tricks you can use.

KDOC is now roughly 6000 loc, and while it tends to get smaller every now and then as I refactor bits, not having something like KDOC for KDOC doesn't allow me to see the "big picture" the way generating docs for a C++ or java does. One thing that really bothers me is my inability to find myself a coding style that doesn't go out the window whenever I learn a new perl feature (eg I just started using ($,..) in subroutine declarations).

Most one-night hacks tend to grow as time goes on, so I think I will choose python for my next castles-in-the-sky project.

A lot of people think I'm misusing perl when I generate regexps like the one below. I guess I'm just glad I don't have to do it manually.

This one matches a method/function declaration.

(?:((?:(?:(?:(?:(?:(?:(?:(?>\:\:)|(?:[_\w~]+))+)|
(?:\s*(?:<(?:(?>(?:(?>[^<>]+)|<[^<>]*>)*)|<(?:(?>[^<>]+)|
<[^<>]*>)*?>)*?>)))+)[*&]*)(?:\s+(?:[*&]+|
(?:(?:(?:(?:(?:(?>\:\:)|(?:[_\w~]+))+)|
(?:\s*(?:<(?:(?>(?:(?>[^<>]+)|<[^<>]*>)*)|<(?:(?>[^<>]+)|
<[^<>]*>)*?>)*?>)))+)[*&]*)))*)\s+[*&]*)?)
((?:(?:(?>(?:(?:(?:(?:(?>\:\:)|(?:[_\w~]+))+)|
(?:\s*(?:<(?:(?>(?:(?>[^<>]+)|<[^<>]*>)*)|
<(?:(?>[^<>]+)|<[^<>]*>)*?>)*?>)))+))\s*
(?:(?:\[[^\[\]]*\])(?:\s*(?:\[[^\[\]]*\]))*)?)|
(?:operator\s*(?:(?:\(\s*\))|(?>[^\(\)]+)))))\s*
\(((?:(?>[^\(\)]+)|\([^\(\)]*\))*?)\)\s*((?>const)?)
\s*((?>=\s*0)?)\s*(?::[^{;]+)?)

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!