Older blog entries for cwinters (starting at number 21)

Well, it only took one day to prove myself wrong: DBD::ASAny does not support the {TYPE} property of a DBI statement handle. (And I consider it a modern database with a recent driver.) Ouch. Not only that, but it's got a bug -- if you try to pass one of the DBI SQL types (e.g., SQL_INTEGER) in with a 'quote' call ( $db->quote("O'Malley", SQL_VARCHAR) ) DBD::ASAny seems to do something naughty to DBI (something with calling type_info) -- created a simple test case and sent it to the author.

Anyway, that just means something that was going to be done later is done now -- you can create a custom 'quote' method which does or does not deal with the DBI SQL type, or can do anything else. I'd love to just use bound parameters for everything, but you run into the same problem because you need to bind a type to them. Plus, some drivers (notable DBD::Sybase) cannot use bound parameters for certain types of data (text/blob stuff). Ack -- do the simple stuff first.

So, all the tests are successful now for three databases: MySQL, Sybase ASE and Sybase ASA. I'd like to get Postgres on a machine here shortly, and I've got some devel versions of Oracle and DB2 here as well, but those last two require a time investment which I simply don't have right now. Interbase is another good one to throw in the mix.

Still trying to get installation procedures working easily and well -- OpenInteract comes with its own simple packaging system, so that you can develop a package, run a command and put the necessary stuff into a tarball and send it to someone else, who can run a command on the tarball and install it fairly simply. ('it' in this case means the data object configuration, module configuration, initial data and security, page handlers, rulesets, etc.) Again: trying to start simply. The XP book notes that programmers are trained to try and to anticipate future problems and design for them rather than just design for today. Of course, it notes, many times the problems you anticipate never occur, so that design was wasted. Issues with reusability discussed recently on here seem pertinent, but my sleep-fogged brain can't quite draw the line between them.

I wonder: when people measure code in terms of lines, do they include in-file documentation (such as POD) with that?

I need to start writing software as if it will be released to lots of people tomorrow :) Interestingly, the potential (up to 90% I'd say) release of OpenInteract has coincided with my first reading of Kent Beck's book Extreme Programming Explained. I know, I'm probably a whole 18 months (meaning: eternity) behind the rest of the software development community, but it's new to me.

Anyway, XP places a lot of emphasis on testing, an area where my discipline is sorely lacking. The happy coincidence is that releasing for other people also forces me to write tests, just so they'll know everything at least nominally works. This way, I have an excuse to get in the habit of doing something I should be doing anyway. Happy day!

A nice side benefit of the SPOPS DBI tests is that it should be easy to find out whether particular DBD drivers support the necessary {NAME} and {TYPE} attributes or not. (I think drivers for most 'modern' databases do.)

Cleaning up the code (which isn't much of the work) and making it presentable (read: installable by someone other than me) also means that I get to hack around a bit in the Makefile.PL and ExtUtils::MakeMaker land. Eeek! But it's coming along -- it's done for SPOPS, I just need to figure out how to get a normal script (to install the packages, etc.) to run after 'make install'. This is a very powerful software package, and the perl way is to make easy things easy, so... where's the easy part? (Personally, I don't qualify editing a Makefile as easy, but I can be a little dense sometimes.)

The nutty thing is that you can (apparently) use Ext::MM to create PPM files (for ActivePerl users) and for this you can use the 'PPM_INSTALL_SCRIPT' key to specify a program to run after the package/module is installed. So someone recognized this as a good idea to do. Double eeek!

Excellent news -- management has initially agreed to make both OpenInteract and SPOPS open source! I'm working on the website right now -- we want to have some infrastructure available for when -- not if :) -- people get interested and want to use either one of them.

It was actually serendipitous -- I'd written up a document about why OpenInteract should be open sourced, pros and cons, links, references, etc. My team leader (Christopher) read it and sent back some suggestions, so by Sunday evening it was looking pretty decent.

Monday afternoon, management brings up the open source issue, being initially set against it. Armed with the information from the document, however, they were child's play for Christopher. A phone call or two to me and we all seemed to be on the same page.

Now we just need to get this website up and get the stuff licenced -- I recommended Artistic/GPL, just like perl -- before anyone changes their minds...

Earlier I talked about rulesets in SPOPS, and over the last couple of days have exploited them for some nifty stuff. Implemented Yet Another Full Text Search Engine (tm) -- dead simple, none of the interesting things that DBIx::FullTextSearch does, but it's database independent (rather than tied to MySQL) and easy to understand. I like simple, especially in the beginning and especially with room to grow.

What's cool, though, is the way this works. If you want an object to be included in the full text search, you simply specify 'Interact::FullText' in the 'isa' tag of the configuration file for your class and list the fields you want to be submitted to the search. (You also need to specify an english name for your object, how to extract a brief title from your object [thru a coderef] and a url fragment on how to display it. But you should already have those specified those anyway....) That's it.

Every time you create or update an object, the rule for Interact::FullText gets triggered. You don't need to call 'index' or anything like that -- the rule gets installed when the sever is started. The rule breaks down the text object into stemmed words and saves them to the database, counting up the number of occurrences and associating them with the object. Like I said: simple. But we have plenty of space to improve it, since it's basically a black box.

First public site (Transit Vote) is rolled out now -- I'm sure little bugs will be found with more usage, but I feel pretty good about it. (also need to get our 'powered by' logo onto a credits page or something...)

Getting ready to roll out a website based on OpenInteract, the first public one. It's fairly limited -- lots of static pages, news, polls and all the user/group stuff. Nothing too complicated, so it should be a good test.

Finally (!) started writing up a document that outlines some coding standards for our company. (Since I'm the "senior" programmer, it's both my responsibility and my perogative.) Not done yet, but I think I set out some good guidelines. You don't want to make people feel like big brother is watching and mandating that everything be this plain-vanilla way. But you also want things to be consistent, and there's also a little of the NIH syndrome (just a wee bit). And there's the additional baggage of perl's rep as a write-only language and the fact that most folks we hire will likely come from a C/Java/C++ background where things like StudlyCaps are the norm.

My wife's grandmother is staying with us for a few days. She's fun to hang out with, and it's nice to have someone (besides the cats) around during the day, to eat lunch with and shoot the breeze.

After working at home for so long, I wonder if I'll ever be able to (or want to) work in a "real office" again?

It's a nice feeling when you build something, think of something new to do with it, make the small change and it just works. The feeling of just works is something I associate strongly with open source programs, and by having something of mine exhibit the same behavior I feel like I'm doing something right.

How the heck do people use windows 95/98 all the time? For the few things I use it -- translating Word documents properly, operating the scanner -- it seems that if I leave a program open for any length of time without activity (>20 minutes) and then move the mouse to disengage the screen saver, there's a nice ugly GPF box there. It crashes even when it's doing nothing?

In the process of writing a document justifying making OpenInteract (and SPOPS) opensource. It's very fair, touching on some of the negative aspects as well. But I think all the work I put into developing a packaging system will definitely pay off here.

The recent hubbub about PostreSQL benchmarking made me take another look at it, and I'll probably see how well OpenInteract works on it in the very near future. It would be nice to use a database with transactions for all of our applications, not just for the customers who cough up the dough for Sybase.

Thinking about ditching cable to help get life back in balance. The only problem with that is no 'Law and Order' all the time. But who has time for TV anyway, right?

Good and bad news of late.

First, this past weekend we went camping with Barb's aunt, uncle, mom and my sister around Erie, Pa. On the way there, we realized that Niagara Falls was only (in theory) about an hour away, so we figured what the heck.

Took a little longer than that, and it was crowded as all get out, but pretty amazing. I'd like to go when the weather isn't so great (colder, which will probably happen in about three weeks or so...), there are fewer people and it's easier to get lost in the sound and fury without some screaming baby or whatnot. Fun trip overall, though.

Another good thing: got through a nice little idea barrier last night and implemented some nifty stuff in SPOPS. Kindasorta like triggers in a DBMS, we can make it really easy to apply all sorts of disparate metadata layers on the data objects. For instance, restricting certain data objects to certain groups; allowing users/groups to 'publish' the data object and other users/groups to 'subscribe' to all objects or even those that meet certain criteria. The cool thing is that you can have any number of 'rules' (term for now) applied to two phases of an action -- for instance, when you call 'save', you can create a rule for before the object is saved, and after the object is saved. Ditto with 'fetch' and 'remove'. Cascading deletes are now really simple, as is lots of other interesting stuff.

The only thing I hate about getting over a hump like this is that I look back at the hurdles I couldn't previously jump over (or go around) and think "Duh!" and get down on myself for not seeing it earlier. But I suppose that's inevitable (and almost always not prolonged).

Bad news: encountering unexpected resistance in opensourcing OpenInteract and SPOPS. Enormous downer. It's not done yet, and I think with a good argument we will be able to release SPOPS and the crucial parts of OpenInteract. I'm working on that now. I'm proud of all the work that's gone into this, and I want to be able to share it and see people doing things I never dreamed with it.

Got back from DC last night. My dad got married over the weekend, which was... interesting. The wedding went well enough logistically -- it was outdoors and it didn't rain, nobody got food poisoning, etc. And I'm happy that my dad is happy. But I miss my mom, and it's hard to see my dad with someone else. That will diminish with time, I know, but that doesn't help right now.

Went to meetings on Monday and Tuesday, almost entirely good. (One of the clients actually said, "We want to move all our databases to the web, but we just have so many to move we haven't been able to do them all. Can you help with that?" Wow.)

Internally, we discussed the licensing of OpenInteract but didn't come to a conclusion, which kinda bites. I know that once we release it (along with SPOPS) we need to have some support mechanisms already in place, but that might be fun work, and sourceforge can take care of some of that as well.

(Incidentally, Christopher made some nifty web pages describing OpenInteract from a management POV.)

So, result of all this: Tons o' work for the foreseeable future -- but who doesn't have that?

Documentation crazy today (tomorrow too). One of the unfortunate (in a sense) things about writing documentation for a system you wrote has to be that in the course of explaining something in terms that someone else can understand, you get ideas for new/improved features for what you've already developed. It's a vicious cycle!

One of the ideas spurred from the fact that OpenInteract could actually be a product, but I hadn't thought much about this in the past. But today CP mentioned that a german company that builds content management systems is putting out some feelers. So I asked myself, "How are people going to distribute new packages? How are we going to distribute upgrades?" And everything sprouts from that: distribution (needs to be in one file), file locations (need to be standardized), configuration (need to break the monolithic config file into chunks) and all sorts of other things. Very useful.

In addition, I think there's a real learning curve that you go through when developing something that will be used by any number of people, all of whom you do not know and will not have contact with except for your README file and other documentation. The assumptions that you have regarding other software you install must be made explicit so you can anticipate questions people have.

That said, people won't read what you write. Rusty has this problem with Scoop -- bunches of people email him asking a question explicitly answered in the documentation. If you send back a terse 'RTFM', people think you're an asshole. If you take the time to answer them, you'll be doing nothing else. It's not so bad if the dingdongs think you're an asshole, but if that starts floating around and taking on a life of its own, then people who might be valuable developers may simply find another project to do.

Back to work.

Scanning fiend this weekend. Got all the wedding photos online, at least from the professional, and created step one of a nifty photo browsing application. I probably have more than 400 photos scanned in (old and new) and the only thing stopping me from doing many, many more is the task of organizing them. This project should take care of that.

Faxmodem got fried in a rough electrical storm a few days ago. I am so bummed about that. I suppose I'm lucky that nothing happened to the DSL 'modem'....

Cool stuff with the framework on friday -- talking stuff, not implementing yet. But it should make navigation pretty easy. One of the things we're shooting for with OpenInteract is to make relating disparate things (documents, links, events: SPOPS objects) simple, and not require people to search. (It's weird: IME, normal people never use the search functionality of a website. Never.) Instead, the 'related items' will always be there. Nielsen's 'Designing Web Usability' is a big proponent of this, although that's not why we're doing it. :)

Anyway, the idea is to keep the act of relating objects in the hands of the people who know what they're doing, make it easy for them to do so, and let the information flow. (Kind of like how yahoo ensured consistency by having one superorganized MLS person who knew *everything*.)