Older blog entries for Waldo (starting at number 172)

Web Applications: Non-Triggered Events?
Can anybody recommend the best method of causing a non-triggered event to occur on a regular basis within or for a web-based application? I would like to send out an e-mail regularly (say, daily) to individuals based on data stored in a MySQL database. All existing interaction with this database is accomplished via PHP. I figure that I have two choices:

a. Use a cron job. b. Have a frequently-called function check the current date and time. If it is within certain parameters, call the function that will cause the e-mail to go out.

I don't like the former, because it will make it harder for people to install this application. The latter is wasteful. Am I missing a third possibility?

Text Ads Success
Wow. This text ads system that I wrote has turned out to be way more successful than I hoped that it would be. Not the software -- I haven't had the time to do anything with that today. No, the success has been in the deployment of it on nancies.org. We've taken in $100 in ad sales today -- a Sunday, at that, when traffic is by far the lowest. If you'll excuse my starry-eyed optimism, if we made that every day, we could pay our hosting bill a half dozen times over each and every month. Taking in this money has been effectively effortless; people book the ads, and I just have to click a link to approve each ad before it runs. The money just stacks up in our PayPal account. What a beautiful thing. :)

In any case, I'm excited not only for nancies.org, but for how beneficial that this could be for other websites. I've got to package this up and get it out there...it's just a question of how much of a perfectionist that I want to be before I make it available. :)

Text Ads System
I've just completed the pre 1.0 release of my text ads system, and I've put it into testing on one my sites, nancies.org. I had been looking at Text Ads (that's the name of the program), but it's much too bulky for me...plus, it's really going in the wrong direction. I wanted something very, very simple, with PayPal integration, written in PHP, with no accounts system, no method for users to track ad performance...just a basic system good for your average small site. I've shaken out some bugs in the past 12 hours, and it's proven to be a pretty popular feature. I start spring semester on Monday, so hopefully I'll have time tonight to package this up and make a 1.0 pre release out of it.
Charlottesville Wireless Freenet
I've been planning a municipal wireless freenet here in Charlottesville for the past 18 months. Unable to make any headway on my own, I sent a letter to the City Council earlier in the week, extolling the virtues of such a system and encouraging them to consider the creation of a network blanketing downtown. Well, it worked. They love it. We've started work immediately to look into the feasibility, and it looks real easy.

There's really just one major concern, and that's bandwidth. Bandwidth is cheap if it's just for one person. But you can't just go get a DSL and resdistribute that bandwidth to 100-odd people. Well, you can, but once they find out, it's over, so that's not a great option here. An ISP-class connection is pretty much required, and that takes some serious cash, and deep-sixes the "cheap" part of "cheap and easy" with this project. Fortunately, one of the city techs is way into this project (he's an OSS/Linux/[SL|FS]. geek...gotta love it), and he's going to start looking into this with the city's vendors.

I'm optimistic that we'll have this system up in a matter of weeks.

SuSE PPC Install Attempt
I just made my first attempt to install SuSE on my PowerPC. It didn't really work out. It had me copy an install file to the root of my drive and restart. I did so, completed some information in a gnarly-looking installer, and it downloaded a bunch of stuff, complaining all the while. Every now and again, it would -- without warning -- drop me back at a menu, where I would simply select "install," not knowing what else to do. Eventually, it assured me that SuSE was as installed as it was going to get, and dropped me to a login prompt. I logged in as root, and was greeted with a prompt. There was very, very little on the machine -- just a bare bones text install. I thought that perhaps a reboot was in order -- perhaps there was some auto-configuration step that required that. So I rebooted, and now I'm in Mac OS X. No boot manager ever asked me what OS that I wanted, and I have no idea of how I'd get back to SuSE.

Oh, well.

Powerbook
I've received the 40GB IBM TravelStar drive from Smalldog, and installed it in my PowerBook (Pismo) without incident. Installing OS X took a surprisingly small amount of time, and the machine booted to the desktop before I'd realized what had happened. I've got a lot of configuration to do to get this system back to where it was prior to the loss of my 10GB drive, but I kind of look forward to starting from scratch.
Yellow Dog Linux
I set aside a 10GB partition on this new drive to install another OS. I have the YDL 2.0 discs (I ordered them to upgrade my then-primary server that ran YDL 1.2, but YDL provided no upgrade path, so the discs and lovely packaging sat idle), so I thought I'd install that. Big mistake. The installer looks like ass, and failed with some exciting new Python error each time. It was difficult to use, had a logic all its own, and never did manage to install. I'll probably try Mandrake or SuSE or something, but not Yellow Dog Linux. I've decided to give up on it.
Motherboard
For the third time, I am mailing back my Asus A7V-266e motherboard. I purchased it in March. It worked for a couple of weeks, though badly, before dying. The second was DOA. The third was DOA. I called tech support about the third last week, and the tech was baffled. He could attribute its complete lack of anything approaching functionality (including trying two different power supplies, two different sticks of memory, and two different processors) to nothing but it being completely dead. So I'm mailing this one back for a fourth. I have now spent $50 just in shipping and insurance, as Asus is not willing to cover the cost of returning each of these boards. I have used this system for a period of two weeks in the seven months that I've owned it. I'm becoming extremely frustrated with Asus. I believe that I'm going to ask that they reinstate my warranty from the date that they ship motherboard number four, but I'm not sure of what other demands that I'm able to make.
RSS Setlists
In a blatant misuse of RSS, I'm designing a simple schema to transmit set lists (from concerts) via RSS. It won't be particularly exciting, but I'll release a spec.
Linux at Work
I'm back to putting work into switching to a Linux desktop at the office. I've been fiddling with WINE, with some assistance from my local LUG, as there are two proprietary insurance industry applications that I must be able to run. Fortunately, they're extremely crude, Windows 3.1-era programs, so I suspect that it's quite doable.
Dead PowerBook Hard Drive
My local Apple repair shop called me on Tuesday: the 10GB hard drive on my PowerBook G3 Firewire (Pismo) is really and truly dead, with no realistic and economically-feasible shot at data recovery. I back up pretty well (for a guy that doesn't have a backup system), so I haven't lost any great quantity of stuff. But it's sort of like having your house burn down, even if you're fully insured and took your most precious things out of the house; you still have to rebuild your house and experience that akward homeless period in the meantime. I've ordered a 40GB drive from Smalldog, and that should show up in a week or two.
Failed Advocacy
I got a response to my rather-extensive letter to an insurance company with whom I do business. The letter, in a nutshell, was attempting to convince this anonymous company of the merits of thinking beyond their self-imposed (and, to some extent, industry-imposed) boundaries and start to work digitally on a non-proprietary level. The system that they intend to migrate to is entirely copyright- and patent-laden, made by an incompetent company that will surely prove to be a dead end. The response that I got was from a middle manager (not the VP to whose attention I sent my initial letter) who just didn't get it. She was arguing the technical merits of this system, saying that it's really a fine program and that it would work nicely. She was unable to take a step back and see how much that she's handicapping both her business and the entire industry by her willingness to use this pathetic product. I'm going to drop the issue from here. I'd like to push them farther, but it will likely affect our business relationship negatively if I'm not successful. Too bad.
Dead Laptop Hard Drive
My G3 Firewire Powerbook (Pismo) is unconscious. The hard drive has ceased to acknowledge that it has any data on it, and my best hardware and software efforts have been unfruitful. It's in the shop now. Maybe they can do something.
Virus Filtering
I feel it would be best if every major Linux distribution came with Windows executable-filtering as an easily-enabled option for the mail server. I mean, like, a checkbox or a flag-toggle to prohibit the transmission (incoming, outgoing or both) of Windows executables. Another further option could be to provide the choice either dropping the executable and sending the message anyhow, or droping the message entirely. This would surely slow the spread of the Outlook viruses (Bugbear and Klez, notably), and it's not like it would be particularly difficult to interface Sendmail and MIMEDefang as a default install option.
Red Hat 8.0
I've installed Red Hat 8.0 as my office desktop. Or, rather, as a part-time desktop to see if it's feasible to switch my family business over from Windows. I'm playing stupid, and refusing to open a terminal, in hopes that I can figure out what it would be like for the average user. Here's the weirdnesses:
  • Lack of "Network Neighborhood." I can't even figure out how to go about making an equivalent happen. I eventually had to guess that, if I opened Eazel, I could open "smb://" as the URL to get a Windows-style SMB listing. I can then browse to the servers on the network, but it's not possible to make the resulting folders appear on the desktop or mount them as drives.
  • Some stuff just doesn't run via SMB. If I click on an audio file on an SMB share, I get the following error message: "'X Multimedia System' can't open 'Tom Waits - Chrismas card from a hooker in Minneapolis' because 'X Multimedia System' can't access files at 'smb' locations. No other application sare available to view this file. If you copy this file onto your computer, you may be able to open it."
  • What's with the "Extras" menu? It doesn't make any sense. If they're programs, list them where they ought to be.
  • Lack of a "My Documents" folder. No, I don't use it either, but people want it. Particularly Mac OS X's approach, subdivided into Music, Video, Documents, etc.
  • No Flash, RealAudio, MP3 support.
I really like Red Hat 8.0. I'd lost all faith that they had any concept of how to put together a desktop environment. Plainly, I was mistaken.
26 Sep 2002 (updated 26 Sep 2002 at 19:39 UTC) »
Advocacy
Back in July, I met with an insurance company about their policy-issuance software, and was quite dissatisifed with their plans. Too closed. Too proprietary. Too likely to end badly for all parties involved. So I decided to do a little advocacy. Below is the letter that I sent to Big Insurance Company today. Identifying details removed. Apologies for the length. I encourage others to crib this for their own advocacy work.



John Smith
Big Insurance Company
555 Any Street
Any City, Any State, 55555

September 25, 2002

Dear John,

Thank you for inviting me to Big Insurance Co. in July for the discussion about the Policy Upload Project. I am pleased to be included in your plans, and look forward to testing this system. It is clear that digital document handling is the future of the industry, and Big Insurance Co. stands a good chance of leading the way in this change.

However, I'm not convinced that Big Insurance Co. is looking at the big picture. The system that you are developing, while ahead of your competition, has a scope that is far smaller than perhaps will prove useful to you long-term. It appears that your staff has done a fine job of implementing a system that fits what I assume to be their assigned parameters. The problem, however, is that those parameters are far too narrow.

The fundamental problem is that Big Insurance Co.'s digital policy transfer system is based on closed standards and third-party software, InsuranceSoft's Internet Policy System. This reliance on a single product that is without viable alternatives will inevitably lead to problems. As Jane Due pointed out during our July meeting, you are doing all that you can to avoid relying on Internet Policy System, in the sense that you can continue to function as a company without it. This is true; Big Insurance Co. could continue to function to exist with the sudden loss of computer-based policy issuance software, paper-based or digital. Agents could always type up policies on a typewriter and mail them to you via USPS. But if you can imagine transferring a sizeable percentage of your agents to reliance on Internet Policy System - say, 50% - only to subsequently remove that option several years later, I expect you'd find that your business would suffer greatly as a consequence.

The biggest cause of mass exodus from a software product is the radical altering of software licensing terms. Perhaps the most widely-discussed example of is the Microsoft licensing system. Microsoft is the new IBM, in the sense that "nobody ever got fired for buying IBM," to invoke the '70s and '80s sales line used by IBM salesmen. Recognizing their defacto superiority to the competition, Microsoft introduced their Licensing 6 program earlier this year. This new product-licensing scheme, which many companies are forced to use, raises fees anywhere from 33% to 107% over the old licensing prices. A survey of 4,500 technology managers showed that two thirds of them are not willing to sign up for this new licensing program. [1] What these companies will do as their existing software ages is unknown, but a sizable percentage of them confess to looking at switching to the Apple Macintosh or the Linux operating system. No doubt many of them will eventually be forced to pay whatever fee that Microsoft demands of them, since they have become too heavily reliant on Microsoft Windows to make switching practical.

Then, of course, there is software that loses viability because the companies simply cease to exist. Enron, WorldCom, Arthur Anderson, ImClone, Adelphia and Tyco are all some of the more well-known examples of seemingly-invincible companies that have recently fallen. Companies that bet the farm on their partnership with these collapsed corporations have obviously been hit hard by their insolvency. Now more than ever, it is important to question what the longevity of crucial vendors and business partners will be. The question, in a nutshell, is this: Who will be in business longer, InsuranceSoft or Big Insurance Co.?

As software reliance goes, InsuranceSoft's offering is an unfortunate one. Their Document Automation Platform Processing System is a virtual dinosaur, having been written for the early 90s operating system Windows 3.1 and being virtually without substantial updates ever since. Computing technology has advanced significantly in the past decade, but one would never know that from using InsuranceSoft's products. It remains reliant on long-outdated protocols from a version of Windows two generations previous, protocols that are rapidly being phased out by Microsoft. Innovation is plainly not a word that could be applied to InsuranceSoft's method of operations.

Adding insult to injury, the cost of InsuranceSoft's product is grossly out of proportion to its quality. As was brought up during our July meeting, InsuranceSoft recently quoted a price of $40,000 to Big Insurance Co. for a copy of their policy-issuance software. Incidentally, this same software, bundled with a second product, was purchased by Landers Underwriting for the comparatively-low - though still extremely-expensive - $15,000.

How does InsuranceSoft get away with selling over-priced, under-featured software? Because there is no pressure to innovate. The standard-making bodies in the insurance industry have no teeth, primarily because insurance companies are unwilling to pressure vendors to stick to industry standards. Instead, companies have come to accept the offerings of software developers without question. This artificial environment has surely eliminated anything in the way of an R&D budget from industry software developers, and has consequently left agents, brokers, and insurance companies alike with the anemic offerings that we are burdened with today.

Fortunately, this is not a problem that is without solution. As InsuranceSoft is slowly dragged into the mid-90s, and begins offering Internet-based policy issuance solutions via Internet Policy System, they are forced to rely on common protocols like XML, HTML and HTTP in order to make their system function within web browsers, the client software that they have selected, presumably to avoid having to manufacture their own. This necessary gravity towards standards provides an ideal opportunity for major InsuranceSoft customers to demand that the company go beyond mere gestures towards standards-compliance to true adherence to open software standards. Use of standard Internet and data format technologies, unencumbered by patents, will enable flexibility and expansion opportunities that would otherwise be impossible. Most important, this would prevent Big Insurance Co. from becoming locked into proprietary InsuranceSoft data and data exchange formats, thus opening up vendor competition and enabling Big Insurance Co. to control all aspects of its policy management system.

Perhaps the most obvious change along these lines would be ACORD XML compliance. This is an existing, open standard that provides all data types that are necessary in the data flow model for issuing policies. InsuranceSoft's mongrelized version of ACORD XML standards will lock Big Insurance Co. into a dataflow and data storage model that is dependent on InsuranceSoft's sub-par, copyrighted format. This is a dead end. There are few rational excuses on an industry software manufacturer's part for using any standard but ACORD's. Rational excuses are likely to sound similar to those behind the incredible $40,000 policy issuance software, which is priced as such, of course, to ensure that InsuranceSoft maintains the contract to do that work on Big Insurance Co.'s behalf.

There are also some small enhancements that could be made that would open up the system entirely. The dataflow into and out of the system could be based on SOAP [2] or, better yet, XML-RPC [3]. Data could be made accessible not just as XML, but in the OPML [4] and RSS [5] formats. These changes are particularly exciting, because they would make it trivial to allow a wide variety of applications to access, create and modify policy data. Data input methods would not be limited to the rapidly-outmoded web browser, but be open to dozens of existing applications. Because all of these standards are free, open, and patent-free, any software developer could create software to enhance this process.

This sort of direct-data-access system isn't futuristic; this is how much of the Internet works now. The outmoded web browser is rapidly being bypassed by applications that communicate directly, applications like AOL Instant Messenger, e-mail applications, Internet-based multiplayer games, Napster and its clones, news readers, and hundreds of programs with auto-updating features. This is partly driven by simple innovation and partly by necessity; who wants to do word-processing or data entry in a web browser?

The open-standards based approach is in no way unusual, either. On the contrary, this is the accepted norm for software development today. The web exists because of HTTP and HTML, both open standards. E-mail is based on a series of open standards and technologies. TCP/IP, which controls data flow on the Internet, is also entirely open. Many instant messaging systems are based on Jabber, a public-domain XML-based system that was created by a team of volunteers. The majority of web sites are hosted on the Apache web server, which was created by thousands of volunteers over many years and is also completely open. None of these things are particularly unusual - this is simply how the software community, particularly the Internet software community, has functioned for decades.

Though the insurance industry has been slow in all things technology-based, other industries are way ahead, notably real estate. The National Association of Realtors - the largest trade association in the United States - recently founded the Center for Realtor Technology (CRT), which has yielded a series of interesting industry software packages and standards in its one-year existence. Using existing real estate industry open standards (their version of ACORD XML), the group has created software packages tailored to the real estate industry that are released into the public domain. This is being done not only to provide software where previously none existed, but also to encourage commercial software developers to improve the quality of their programs. Not only are they free to incorporate the CRT's code into their own programs, but the competition and raised expectations pushes them to create better software than they might otherwise. The entire industry is benefiting. [6]

John, the crux of all of this is fairly simple: the use of open standards results in better software. Open standards enable competition, which, in turn, results in higher-quality, lower-cost software. As policy issuance and management moves into the digital realm, control over the transmission and storage of this data is rapidly becoming analogous to having access to one's own policy filing cabinets. If innovation and improvement is to be possible within Big Insurance Co. over the next decade - that's roughly the timeline of your commitment to Internet Policy System, should beta testing prove fruitful - it is crucial that your infrastructure be created and maintained on your terms, not those of a vendor over whom you will have little to no control, one whose interests are plainly how they can best profit from you and not how to best serve your needs.

Getting InsuranceSoft and other vendors to comply with open standards should be no great trick, at least on an industry scale. They are in the business of writing software that people want to buy. If customers demand a system based on open standards, InsuranceSoft will provide it. Thanks to Big Insurance Co.'s early adoption of Internet Policy System, you are in a unique position to make demands of InsuranceSoft at a point where they are prepared to comply with them. As it's unlikely that Big Insurance Co. is the only company currently testing Internet Policy System, it's possible to determine what other insurance companies are trying out this product, and form an alliance for the purpose of convincing InsuranceSoft to improve their offerings. 50% of the top 200 U.S. insurance companies rely on InsuranceSoft. These are not companies that InsuranceSoft is eager to trifle with. And you're one of them.

The future of Big Insurance Co. hinges on the success of your project, John. It is essential that the transfer to a digital policy management system not shackle you to further reliance on an overpriced, underperforming system that's outside of your control. Big Insurance Co. should only be controlled by one group: Big Insurance Co. Don't be dazzled by the newness of it all. Demand something better than the already substandard status-quo of the insurance industry. Big Insurance Co. is better than that.

Please feel free to contact me at your convenience if you would like to discuss this further. Thank you for your time, John.

Sincerely, Waldo L. Jaquith

1 "Cold Shoulder for Microsoft Plan," CNET.com, Joe Wilcox, July 30, 2002. <http://news.com.com/2100-1001-947164.html>
2 http://www.w3.org/TR/SOAP/
3 http://www.xmlrpc.com/
4 http://www.opml.org/
5 http://web.resource.org/rss/1.0/
6 "Realtor group houses all kinds of Open Source projects," NewsForge, Daniel P. Dern, March 14, 2002. <http://newsforge.com/article.pl?sid=02/03/14/0258203&mode=nocomment>

163 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!