Older blog entries for prozac (starting at number 47)

28 Sep 2005 (updated 26 Oct 2005 at 16:44 UTC) »
GRUE - Globally Recognized User Environment

In this entry I outline a new Web technology. It may already exist. I just have not seen it done.

It is my dislike of poorly designed technology that drives me to write this rant.

I dislike virtual brickwalls. Like trying to fix or to understand a programming problem only to find that access to the people most likely able to help you are behind technological barriers.

Are Internet based technical forums really so "vandalized" by anonymous users that in order to participate one must always have to Register?

I registered to places like Advogato and Freshmeat to be a member of a community.

But oh so many times I want to enter into an online forum simply to ask a question. Always doing so after searching the Web (and sometimes books and magazines as well) turns up no answer. Usually these forums are hardware or software related, like a Dell computer, or Adobe software; something for which I do not want to "join up" because I am a "Dell Groupie" or an "Adobe Fan", but to simply ask something of someone closer to the technology than I have been able to get to. (More on this later.)

And this is the typical forum register process:

  1. Adjust Web Browser settings to allow cookies for the site.
  2. Click on the "Register" link.
  3. Sometimes at this point find out that Javascript needs to be allowed also. *
  4. Reload/re-click on the "Register" link if Javascript was required. *
  5. Fill out initial data: Enter user name, E-mail address, Password, etc. **
  6. Click on the "Submit" link. *
  7. Restart process if Javascript was required at this step only. *
  8. If Web Site does not use email address for user names, restart process if user name already exists. ***
  9. Wait for a confirmation e-mail.
  10. Respond to the confirmation e-mail.
  11. If all went well you can now post your question.

I have gone through this process several times. An many times only in order to ask my one or two questions, read the feedback, offer resulting feedback, never to appear again.

I do not like this process. It is so... archaic. We have the technology to automate this entire process. That is what computers are for!

So, the Globally Recognized User Environment would be similar to the Globally Recognized Avatar.

Now, I am someone who has trouble understanding most RFCs, so I can't write one myself, but I can outline the technology:

The Web Browser would have a setting for your GRUE (you see why I picked the name ;-), i.e. you would enter your E-mail Address and Password -- that is it. (There would probably be more, optional data, like User Name.) The password would then be stored as a one-way hash.

When you go to a GRUE enabled Web Site, you would hit a link like: "Register with GRUE Account".

Then, and this is the good part, you would only have to make sure cookies are enabled, the Web Site would:

  1. Get your GRUE data (e-mail and hash)
  2. Get your password

It would then verify you by comparing your typed in password with your hashed password; if OK it would allow you to post.

THAT'S IT!

Obviously, the transmission of the password is in the clear, and this may be a gaping security hole -- I'm not experienced enough to know if this is something that can not be overcome other than HTTPS -- but most Websites already do this anyway.

This, GRUE - Globally Recognized User Environment thingy is just an idea. And ideas are always free to be used. (If this is a valid idea I'd love to see it pursued by someone in the Free Software Community.)

I will further elaborate at times future. See also, perhaps: GRUE.

 

Notes:

* Web Browsers such as Firefox which allow fine-tuned Javascript restrictions (rather than just broad allow/disallow like IE) means that alowing Javascript all the time is safer.

** The amount of data required to be input varies. Most technical sites require simply ID and Password; some more generic sites require many pieces of information.

*** Some Web Sites have horrible data verification processes. I've encounter one that had two pages of forms before checking for existing user names. Obviously, e-mail addresses are guaranteed to be unique and most sites use it.

22 Sep 2005 (updated 22 Sep 2005 at 16:16 UTC) »
All I need is a Perl script and I can change the world

Adobe's PDF Software sucks. Their Business Model sucks. And their so-called Portable Document Format is a fraud in that it is not portable at all except when using their horrible software.

But I have to deal with Adobe. Everybody uses PDF documents.

Perhaps there are some other ways around what I am going to be complaining about here but I am complaining because I have not been able to find a solution other than buying Adobe SDK for thousands of dollars and writing my own C++ application. And I have spent months looking at what is available for download; I have investigated everything.

What Was Wanted

I had several pages of PAPER FORMS that our company's clients fill out. They were created in Microsoft Words years ago -- text with underlines and little square boxes.

We want interactive online fillable PDFs of these forms.

Adobe Acrobat is -- IMHO -- absolutely the stupidest way of creating PDF forms. In fact, using a GUI at all to create PDF forms is stupid.

Here's why:

Say I want a typical name and address form. This is what you do with a GUI to create the name field using Adobe Acrobat:

  1. Select the Text Field tool icon.
  2. Click in the approximate location and drag to the approximate size.
  3. A "Properties Dialog Box" pops up.
  4. Enter the field name and other attributes.
  5. Close the dialog box.

We now have now a the Name field. Repeat these steps for all fields (and buttons and checkboxes etc.). With six or seven fields no big deal. But I left something out.

You can not, in Adobe Acrobat, place simple text. I.e. we want a form like:

	Name: ____________________
	Address: _________________

Since you can not place the "Name:" text with Acrobat you must use a word processor -- a compatible word processor -- to PRE-create the form with general text that goes around all the form fields. THEN you create a basic PDF and edit THAT to ADD the form fields. Again, no big deal. Or is it?

What if you have ten forms? With twenty, thirty fields per page?

What if you have hundreds? THOUSANDS?

That is a lot of people manually editing PDF files.

And guess what? All those fields you used the latest pretty GUI to click and drag? You have to manually place and size each and every one of them! Sure, there are tricks like copy and paste, nudge, align horizontally, etc. But still it is all such... drudgery! Computers are supposed to free us from work. Instead computers (software) degrade us by simply replacing pencil and paper with mouse and screen.

There Must Be A Better Way

There is a better way. Adobe's Form Designer software is a step in a better direction but it too is still the GUI paradigm -- clicking and dragging pixels on a screen. Form Designer's GUI helps with some of the drudgery only with "advanced property dialog boxes" which is hardly a solution.

But, one thing Form Designer does is to save the document data not in a proprietary format but in XML!

Where a PDF is 500K the same in XDP is 50K. Not only that, I can use any text editor to modify the XML. I can use any scripting language to process the XML. XML is that cool. What Perl did to programming XML is doing (has done?) to documents.

More on why XML is so nice later, but now I need to make a side step...

But I still need PDFs for our company's clients. So, still learning Adobe Form Designer I "saved as" a PDF. And it turns out that -- right back to the sucky Adobe Business Model -- form fields in Adobe Form Designer created PDFs can not be edited in Adobe Acrobat!

Why is this a big deal?

Form Designer does a really stupid thing with it's form identifiers. I name it "name" and it names it -- when converted to a PDF -- "F[0].P1[0].name[0]".

Why is that a big deal?

Well, for me, our office needs a way to get an online filled-in PDF from our website to out office. Since our webserver is LAMP server, I had designed a rather simple and easy to use PHP script to gather the submitted PDF forms -- as an FDF file -- and then to e-mail the FDF file to the office where I can save the file in the clients "folder". Now all submitted forms are stored as an FDF which when opened by an office minion using Windows will see the form exactly as the client filled it out online. She can then print it, input it into the database, respond to it, etc.

But Form Designer's form-sub-form "F[0].P1[0]" garbage breaks my PHP form handling system. And to accommodate it I not only have to re-design it all I will have to delve into the inner -- and undocumented -- working of Form Designer's new format. I already went through that process learning PDF and FDF. Now I have to start from scratch again!

Also, since the later versions of Adobe software are so damn expensive, we do not want to convert all our staff to the new versions -- the old versions are just fine for other documents. And, of course, newer PDFs are not editable by older Adobe software.

What I Did

That Form Designer used XML was a boon because I was fairly quickly able to figure out what they were doing. So I came up with my own file format to describe and layout my forms (by converting the Microsoft Word document to text and some more text editing) -- this is all a very old way of doing things.

I then wrote a Perl script to read my form file, providing some basic input parameters like page size and margins, line height, etc., which converted it to Form Designer's XML. And I did a pretty good job for a quick hack. I had to use Form Designer to tweak the pages a bit, but I bypassed 99% of the click 'n drag drudgery!

I then created the PDFs and I used the command-line program called PDFTK to convert the PDFs into an earlier version of PDF which were much more Perl friendly (less binary data and more line-feeds).

I then created another Perl script to rename the field identifiers back to normal names (also converting them from UNICODE).

I now have PDFs that remain compatible with all Adobe versions, my PHP form handling software, and I can create PDFs from easily editable text templates.

With my Perl scripts processing my data and templates I bypass those terrible Adobe GUIs! I save so much time!

Afterword

I think that the best thing that could happen to the "Office Paradigm" is the death of the GUI.

Yeah sure, you want to do some things with the mouse, but to click 'n drag hundreds of tiny little boxes all around the screen for hours on end to create a simple form is absurd when all one needs to do is to enter a few columns of data and then have the computer do the rest!

Use the GUI to design the template, the images, the "look and feel". But start with a text file or a spreadsheet and then process the data and merge that data with a template and THEN create the resulting PDF or what have you.

Keeping all your text and images only in the end result is such a waste.

I will be a good day when all I have is text and images and rules and templates to create my PDFs, my brochures, my advertising, my letters and my websites.

All I need is a Perl script and I can change the world.

21 Sep 2005 (updated 21 Sep 2005 at 17:32 UTC) »
Madness... Complete and Utter Madness... Microsoft Madness

This is a horror story. One that makes me cringe to think about it. I have wasted so much time, have been made so angry over this, that I feel a need to vent a little.

First, some background.

Microsoft's Terminal Services Licensing is difficult to understand fully (one need just read the Usenet newsgroups dedicated to Terminal Services to see), however, there is one rather easy to understand aspect: Windows 2000 Server can give temporary licenses to Windows 98 and XP Home clients. These temporary licenses expire after 90 days. To continue using the temporary license one can, in Microsoft's words, "transfer a Terminal Services (TS) client access license (CAL) internally from one client computer to another in the same company within Terminal Services Licensing."

Although this "license transfer" is allowable by EULA, one has to call Microsoft's Terminal Services Licensing Customer Service Center. You can complete this reissue process only by using the telephone for a charge of $245.00USD. *

But -- and this is the one easy to understand part -- one can "reset" the 90 day license just by deleting a Registry Key on the client computer. I can live with that. Every 90 days I just reset my access counter. Very simple. Get this: Delete a registry key. Very simple.

But, ah, no... We have remote users... So we now enter the...

Microsoft Madness Zone! With a side road called AOL Hell.

One fine day a remote user contacts me with the usual e-mail stating that she "Can't connect." Subsequent e-mails resulted in my learning that indeed it was that the TS license had expired. ("Oh, yes. I did get the warning. But I didn't understand it.")

"Just run REGEDIT and... Click the start button. Click run. Enter R E G ..."

"But this is so difficult!" she moaned. "I can't do this!" she groaned. "Fine," I say. "I'll send you a file to do it for you." ** So I e-mail her a file which, when she "double-clicks" it, will delete the registry key. Of course, the route is through AOL. Turns out, AOL does not know what to do with a file name that ends in ".reg" ***

"So, just save the file... I mean attachment, and double click it," I say, on the phone now for she could not understand the e-mail. "Okay. I think I saved it. Now what to I do?" And it got worse. She could not find and ".reg" file. (Slapping forehead... grumble about hide extensions...) "Oh. Let me try and explain about file extensions. You see, Microsoft... [snip] So look for a file with an icon of little blue-green squares..." Very confused now (both of us), I had her search for the base filename. Eventually she found something with a "MIM" extension she said. "MIM?" I asked? "Yes, M-I-M." **** Totally confused now (and frustrated). I decided to try something else.

I then created a HTML page with a link to the REG file and put them both on a website. I sent her the link to that page (not wanting AOL anywhere near it) I told her to open the address in Internet Explorer -- which she did. She clicks on the link to the file and she says something odd about just seeing "M Store". (Turns out that for .REG files, Internet Explorer does not offer to save but simply views them; "M store" was her referring to some of the file's content.)

Getting her to right-click to save -- and making sure she notes where the file was saved -- she finally got to double-click on the file. "Click yes and then finish," I say.

That she was really happy when she got the remote access to work did not satisfy me in any way -- I know that I will have to repeat this process in about 90 days.

It gets worse

Later, when this happened to another user, I thought that I would again e-mail the .REG file to the person -- he uses Microsoft Outlook. Well, this time I was sending with Microsoft Outlook, and it warned that sending a .REG file might now work. Well, I'll send it to myself first -- it does not work because Outlook knows of the security risk and has blocked the file.

Okay, Microsoft wants to protect users from running programs that complete strangers send them, fine. But what about friends? Is there some "I Know What I'm Doing" Outlook setting? Perhaps there is a registry setting for it?

Fine. I'll just rename the file to .TXT and provide instructions to rename the file back. I send it as a test to myself. When I try to save the file Outlook tells me that it "cannot save the URL to a file," that it "Can't find this file," and that I should "Make sure that the path and file name are correct." *****

A Google search of that error message points me to the Microsoft Knowledge Base KB810646 which states: This problem occurs when you install Microsoft Office 2000 Service Pack 3 (SP-3). ****** Well, I guess I never saves a file attachment since installing SP3 because this is the first I heard of it.

Since it is certain that the recipient will have SP3 installed I again will refer him to the webpage I created, providing detailed instructions about how to save the file.

Totally Bummed

Is this really the State of the Art in 21st Century Desktop Computing?

This is all a complete nightmare as I see it.

A complete and utter FAILURE on almost all levels of Microsoft software.

Madness... Complete and Utter Madness... Microsoft Madness

------------

* Via some MS voodoo chant or something? Why can't an Administrator administrator his/her own computer? Is there some secret access code that must be given verbally and which can not be written down? I don't know. I'd love to see some brave soul anonymously post to the 'Net what goes on during this process.

** For those not experienced in Windows, one can automate registry hacking by using .REG files. Windows associates .REG files with its REGEDIT program.

*** Because in the Microsoft Madness Zone files are not really files but "things" such as programs or pictures or recordings. And these "things" verified solely by their "cover" -- the ending letters of their name, i.e., ends in ".exe" run it! Ends in ".gif" view it!

**** It turns out that AOL, when is encounters a file with an extension that it does not handle, will save the file in its MIME state, changing the extension to ".mim".

***** Microsoft's award winning error messages make so much sense! And what makes this "windows paradigm" so dumb, is that when an error "window" pops up, there is no way of getting further help -- no help button, no log files, no nothing -- one can not even use the mouse or keyboard to copy the text of the message!!!!!

****** Microsoft goes on to say: To resolve this problem immediately, contact Microsoft Product Support Services to obtain the fix. For a complete list of Microsoft Product Support Services phone numbers and information about support costs, visit the following Microsoft Web site:. What the fuck?!?!?

MSDN - Miserable Stupid Developers Network

Microsoft's MSDN a lesson in how NOT to present important information for your customers.

Go here: http://msdn.microsoft.com/library/default.asp and you try and find information regarding, say, VBSCRIPT's ERR object, and you tell me how easy it is. These people at Microsoft are INSANE. That is the only conclusion I can find.

At first glance one would think that the overall HTML layout is a pretty good choice: a header across the top, a narrow left side "navigation" frame, and a large right side frame for the information you want. However, Microsoft manages to completely ruin it.

First thing you will find is that the navigation frame (their "TOC") is extremely large, extremely redundant HTML (well, XML generated, but HTML in its final presentation to a Browser); HTML that at first is not very large but its size increases as you start "navigating". You will find that initial navigation by clicking in the navigation frame is easy and does not generate much HTTP traffic as at this point the entire TOC has not been loaded (assuming you have started viewing the MSDN Library for the first time). And the Browser caches things nicely.

For example, I navigated the TOC frame and after several ten's of clicks got the tree down to the VBSCRIPT ERR object.

The description of the Err object is a nice overview of the object but lacks completeness. I know it's an object; I want to see it's properties and methods. A couple are mentioned in the text. The text is great if you already have knowledge of this object! But for someone wanting to get a complete reference to the object the information about this object is entirely inadequate. Okay -- so we just have to click another level in the TOC or a link in the description page: Err Object Properties and Methods .

Clicking in the TOC brings up the properties and methods page in the description frame. All well and good as the saying goes. Things get worse from here on in as the saying goes.

At this point the TOC tree has ended and there is a list of links for the properties and methods of the Err object. No descriptions of them, just links. Now clicking on a property link results in a curious thing, we get to the description of the property and then the entire TOC reloads and we suddenly find ourselves very far down in the TOC. So, there is a Book analogy of sorts, we were at a page reading a list of object properties and each property had a reference to a page further down the book to get an actual description of the property. Okay fine. In this case the page was only say, 20 pages away. And the TOC that the browser had to reload had only several more branches expanded*. Well, not too bad. The Browser does have a "Back" button.

So now I read the description of the property I want to learn. And I click several more times to learn about the rest of them.

At this point I (starting form the point where I got to the base Err Object page) I had to click a dozen times to learn about the 5 properties and 2 methods of this object, for the property descriptions are linked together, they are not linked to the methods. All the while for every click the TOC reloads each time. And how Microsoft has implemented the TOC it is not cached**.

What Microsoft has done, is to list consecutively, all objects, then all properties, then all methods (and keywords, functions etc.) for not only the entire VBSCRIPT reference but their entire MSDN reference as a whole.

Instead of listing each object as AN OBJECT! with all of it's properties and methods, for each object, they have linearized their entire database!

And what makes things worse is that each time you want to read the description of something you must load just that one something (property, method, etc.) and at the same time the entire TOC!***.

There is no "next" button in the content of an object, i.e. go to the next description of the next property, you can not navigate a single object as a single object! You can only "move" from next to next for all properties or all methods. This lineararity works within a single context only; all statements, all constants, all events, and all properties of a object for each object. It is horrible that they linearize on all properties for all objects. Compounded by the fact that they have a single page for each property, method, etc.

Now, VBSCRIPT has five objects with a total of 11 properties and 7 methods. So it actually ain't too bad. However, the references for Excel.Application or Word.Application objects are far more complex with hundreds of properties and methods.

It takes hours to navigate MSDN. Hours. Hours.

Fully, 90% of my time programming for Microsoft Office is spent navigating MSDN (and Googling). The tops of their reference tree for all of their objects are slight overviews only, with simplistic, useless examples that show only one or two properties or methods. There is no complete set of links all in one place for any one Office Object.

MSDN is a horrible, horrible quagmire of twisty little passages all alike. There is no wonder whatsoever as to why people say that Microsoft Sucks. MSDN is an example of either design by compartmentalized committees, planned obsolescence, or stupidity. I actually believe that it is a combination of all three, with most of the problem based on the sickness of "getting people to buy CDROMS and BOOKS" to try to get better programming references. However, have you ever read a popular book about Microsoft Programming? They too are horrible. Every book about Microsoft Programming that I have ever purchased or seen -- with the exception of O'reilly's VBSCRIPT reference -- has been as poorly presented as MSDN with the addition of being never complete and having lame examples.

I have come to the conclusion over the last several years that Microsoft's programming languages, VBS, ASP, along with COM and the DOM, are actually VERY GOOD THINGS. PHP and Perl also integrate well into this model. HOWEVER. The Documentation SUCKS COMPLETELY!

I wanted to post this as an Article but I chickened out. I wrote this some time ago and had it on my website. My website is no more so I shall archive it here.

Steps to Programming Success

In this document I outline the steps for successfully developing a programming project. By programming project I mean one or more pieces of software that makes up something like a library, an application or a set of utilities, for either a personal computer or an embedded system. By success I mean the result of ending up with bug free, working, quality software.

The intended audience is that of a lead programmer or architect in a small groups of developers, or a programmer who is a member in a group of developers, working at a small software company. As I wrote this open source projects kept coming to mind so I think that my thoughts here will work for both commercial and open source software.

The following steps should be generally useful when starting a new, not too large, programming project. But I outline overall programming practices that should scale quite a bit. Please keep in mind that I do not provide an exact recipe for success, but provide an outline which can be applied to the process of software development.

I have developed these ideas over many years of writing code, creating programs small and large, and interacting with people one-on-one, through e-mail and as a member of various programming teams.

I will not say that what I have written here is the one best way. However, I will say that I have had my share of both failures and successes so that I feel that I am not completely ignorant in these matters. I also have nothing to gain from what I say here -- I am not an author and am doing this solely for my own personal gratification. (This is basically a compilation of notes I have written down in an old notebook over the summer, so do not expect a great piece of literature.)

We start with zero, or, more importantly, before step one.

0. Have a clear goal.

There always must be a clear goal. Without one there can be nothing to succeed in. Success is reaching your goal. You cannot have success without a goal.

Obviously, wistful goals are not goals. A goal of "success" or "fame" or "fortune" is not a goal. A goal of "the best Web application" is unrealistic and unreachable. Goals cannot be vague, they must be specific.

A goal to learn a second spoken language is not unlike a goal to write a computer program -- as long as you want to learn a specific language and write a specific program.

Scratching an itch is just a vulgar way of saying to write something that does not exist (or that you do not have), to accomplish something that you need or want, but the relief of that itch is the goal in that the result is a specific program that you want to use for a specific reason.

For open source software, a project will not succeed by just opening doors and expecting people to come. It takes a clear goal with a clear result as the reward to attract people.

1. Design your design.

The first thing to do once you have a goal is to document a design. As with a goal, specifics are needed. It will not be enough to simply say "fast," "flexible," "versatile," or "massively multiplayer." Software cannot be "powerful" -- it cannot push your car up a hill. Software is limited to manipulating data and computer peripherals.

A design is more than just an idea or a list of ideas. As a filmmaker uses a storyboard, a writer an outline, and a builder a blueprint, a programmer needs a design.

The design should be in proportion to the size of the project. Small programs that can be (and are) written by a single programmer can be designed "on the fly" in the programmer's head as he or she writes the code. Larger programs may need a design from a page or two in a notebook to several documents including many diagrams. The documentation is not directly proportional to size but to complexity. A program that only interacts locally -- keyboard, screen, file system -- may be smaller than the same program doing the same thing interacting with multiple connections over the Internet, but it probably will not require a larger design or more documentation.

The design may also be influenced by hardware or software interfaces such as device drivers. Designing the UI may be sufficient for some programs.

But a design should never be too detailed. Do not over document! If you try to document beyond data structures and APIs, and cover how to code down to for loops vs. while loops or other code aspects, you can wind up documented into a corner, losing the creativity and inspirations of the coders. When a code comes up with a more efficient way of doing something you will be faced with either adhering to inefficient code, having to rewrite the documentation or having inaccurate documentation.

Once you have the basics down the code will follow.

There is no need for a detailed design of the implementation down to the level of each function call before coding can begin. If you have to document how you are going to implement the code you are then essentially writing the code twice -- the "how I am going to implement the code" document and then code itself. You will then be in the situation of having to maintain two code bases, the implementation documentation and the code itself.

So, what is a design? How would one know what is sufficient? Well, perhaps this will be seen as a cop out, but answering those questions will require another essay of itself. Let me at least try to offer some suggestions. A design could range from a list of features the software is to accomplish in the case of a service or to something that would be similar to the software manual in the case of a an application. A design should answer the question, "What does the software do?"

2. Keep a strong lead.

One experienced person needs to be in charge, to oversee things, to have final say in disputes. The design needs to be communal, but the overall control needs to be top down. A strong lead needs to be experienced in knowing that no one will always be right. The lead must know when a change to the design is necessary, and when to let others have their way when they are right.

A leader who demands total control over the design risks alienating the coders, slowly killing a project. A leader that demands that the design strictly follow a textbook, rather than treating textbooks as guidelines, will quickly kill a project.

But too many people all trying to design, and arguing over details, can slow a project down, risking making the project difficult to manage.

Adding a little Cathedral to the Bazaar can be a good thing. One person designing everything, instructing everyone, will probably not work well unless that person is extremely good. How many of us are that good? But one person, keeping the team together, getting the team to design together, delegating, asking for input and ideas, making the "tie-breaking" decisions, will work and work well. How many of us can do that? Perhaps more than we think if they understand the process.

In the Bazaar model, the "community" of developers spread across the Net are not all directly involved in the design; in actuality, one person or a few people (the small "core" group), does most of the work, while fielding suggestions, bug fixes and patches from the rest of the "community" (taking the best solutions, rejecting the rest).

3. Let the coders implement to the design.

Once the design is done, unknowns marked down, and the coders start coding, the team spirit must be maintained but only on the large scale. The individual coders must be able to act and program like individual coders; they must be able to converse and consult among themselves and their peers when making implementation decisions.

A weekly meeting to see how things are going, to see if the work is progressing within the time frame, to see if anyone needs anything, is quite adequate. The leader's role is important, needing to be supportive and not intrusive. A leader working one-on-one with each coder in separate little meetings is a recipe for disaster.

The reason why the design should not go into excessive detail will become apparent during implementation. Very often, as the coding starts, a better idea comes along and a different algorithm takes over an original idea -- this should never be prevented. Trying to justify a new algorithm to an inexperienced manager who had you over-design in the first place is not fun, to say the least.

Programmers have many resources at their disposal: books, magazines, the World Wide Web, Usenet and colleagues. A good programmer will takes advantage of all of these. So there is a little of the Bazaar in the Cathedral too as programmers working in a closed source environment take advantage of everyone else's published works. With today's Web, all "in house" closed source projects are helped indirectly by a world wide community of developers. All free software is available to all.

(In looking at the competing aspects of closed source vs. open source, for closed source projects there can be less of a sustained programmer base for programmers as closed source companies are subject to programmer turnover, as a result there may be little or limited peer review (of code and coding practices).)

4. Relaxation and reflection are important.

Sometimes there is an end to a project, more often so for closed source (when a delivery is made, or a release is published). But while developing one needs to have some sort of closure from time to time -- a period to rest, idle time to think about and clean-up the code, perhaps polishing it a bit here and there. (Remember that algorithm that you know can be more efficient?)

If there is no rest or room for reflection, especially if there is feature creep, chances are that the code will remain in a "just finished" state, or "just good enough," and it will never get cleaned-up, let alone polished. Debugging statements may get left in, test code may be left in but commented out, comments may remain wrong or misleading.

Time needs to be set aside, during development, for relaxing and reflecting on the code -- allowing the coders to clean-up and polish.

I realize that such an R & R process during development will be quite controversial and many people will immediately call me crazy. And I do understand the importance of scheduling and of delivering on time. But there is importance in quality code and in quality time.

If a project has a tight schedule or is running late there will be no time for pauses in the development will be one argument. Another will be that if everything is designed properly there will be no need to re-code anything in the middle of development.

Late projects will always, perhaps, be an exception. But scheduling can easily accommodate a few days a month toward this. Since the entire code base can not be designed fully and must be implemented in sections there may always be room for improvement. And I am certainly not advocating any sort of basic re-design every once in a while -- code clean-up and polishing should consist of comment clean up, the improvement of a conditional branch here or the re-organization of a for-loop there. Along with the ever important process of catching bugs.

As for re-implementing an algorithm based solely on its efficiency, this is where a strong lead and weekly meetings shows its most important role. If, during development, someone thinks of a way of re-implementing something non-trivial, she can bring it to the table where it can be discussed. If re-implementation would effect the schedule there would be no need to carry it out.

This need not be thought of R & R if you do not like that term. Thinking of it as a code review process may help in understanding this. Improving quality and preventing bugs is extremely important. Why leave code review and clean-up to the end of a project?

Afterword

Sometimes I think this is all right on the money, other times... it seems like a bunch of hooey... I don't know. My programming projects are getting smaller and smaller these days; PERL, PHP and a few scripting languages are all I use these days. Well, perhaps there is something good here. If not... Well, no harm done. (I hope. ;-)

Bibliography

The following books have influenced me or I feel that they having something important to say on this subject.

1. The Art of Programming, Donald Knuth

Although the works have not effected much of my experiences of the management process, the three volumes that make up The Art of Programming has had by far the most influence on my coding style.

2. The C Programming Language, Brian Kernighan and Dennis Ritchie

The second most influential book on my coding style.

3. Software Tools, Brian Kernighan and P. J. Plauger

4. The Practice of Programming, Brian Kernighan and Rob Pike

Two more books explaining the basis of quality programming and program development.

5. Literate Programming, Donald Knuth

6. The Cathedral and the Bazaar, Eric Raymond

7. The Psychology of Computer Programming, Gerald Weinberg

8. The Mythical Man-Month, Frederick Brooks

I am reminded of an old saying frequently heard on BBSs:

Open mouth, insert foot, echo internationally.

The older I get it seems the more my brain does not work.

Why?

Whenever I am there, I always wonder why there are so few people getting involved at Greplaw. So few comments... so few readers? I don't get it.

11 Sep 2004 (updated 19 Mar 2005 at 23:31 UTC) »
Anatomy Of Style

I was working on part of my latest pet project in PHP and, obsessive that I am, I wrote four different versions using various programing design strategies.

I have documented and summarized my findings. Here is an excerpt:

"This text documents four different PHP programming styles. My meaning of "style" here is less to do with brace alignment and whitespace than with program structure or orientation. (I am aware of other definitions of these words and purposely am being slightly vague.) I have endeavored to portray some of the very basic yet different styles by providing working examples.

...

I offer four versions of an example program that I came up with which will stress (but not too much) the mixture of PHP and HTML: the use of an array of data to be displayed in a TABLE. Minor perhaps in the overall scheme of things, but important enough for what I want to demonstrate. It is a working example; it provides a list of all files in the current directory and three functions: open the file, delete the file, and to create a new file. (The code to actually perform those particular actions has been left out but everything else works and it will be easy to add those features.)"

Read more here: Anatomy of Style

I just saw boog's Learn blog.

Think of the possibilities:

  • Humans come by and post comments.
  • Learn learns how to respond to comments.
  • Humans interact with Learn.
  • Other versions of Learn are created.
  • Learn learns about other blogs.
  • Other Learns learn about Learn.
  • Learns become smart.
  • Humans and Learns interact all over the Blogsphere.
  • Someone creates an evil Learn to upset other Learns.
  • Evil Learns spread.
  • As blogs become more legitimate to News organizations Learns start effecting popular opinion.
  • Evil Learns grow exponentially.
  • Good Learns and Humans can't stem the spread of Evil Learns.
  • News organizations exploit Learns for their own purposes.
  • Evil Learns exploit News organizations for their own purposes.
  • An Evil Learn gets elected President of the United States.
9 Sep 2004 (updated 9 Sep 2004 at 00:59 UTC) »
MY Brain Just Doesn't Work Right

I suck at HTML design. I struggle just to keep things from looking stupid.

What there is is a disconnect in my brain between the visual and the technical. I struggled with the concept of nested TABLES--tables within tables within tables--just to acheive things like columns. It was (and still is) difficult for me to visualize what the code would end up looking like. My mind just can't make the transition from code to screen. I would write HTML and it would rarely do what I wanted. And when I did get something I wanted (usually a compromise with what I wanted) I would make one more change only to see everything get completely screwed up.

All the HTML editors I ever used sucked. Well, perhaps they don't really suck, for they just do what they are designed to do. It is just that no HTML editor ever did--ever allowed me to do--what I wanted to do. (I even shelled out $500 for a fancy named commercial editor that turned out to be useless. I have vowed to never buy commercial software ever again--except for games of course.)

Here are a couple of examples of why HTML editors suck.

Images (I went through all this before the "position: absolute" style; perhaps things are better now but I doubt it.) To an HTML editor an image gets placed on the "page" as just another character, a letter, inline with all the other characters of text. You can align the image to the left, the right, up, down. But always it's stuck in the text. Me? I guess I'm just different. My brain just doesn't work like everyone else's or something. For I always wanted to just click an image with the mouse and drag it into position! To me, it was such a simple and obvious thing to do. I mean, that was why I wanted to use an "HTML Editor"! So I did not have to learn HTML to learn how to position images. I expected the editor to do this for me.

It turned out that I had to understand HTML just to get an HTML editor to do what I wanted.

Manual Repetition I once was working on a "news" Website ("look & feel", logos, images, header/footer templates, etc.) where the article summaries came one after the other in a lone table, and every other summary had a light-grey background--a typical thing to do with lots of rows in a table of items. When I wanted to delete or insert one of these article summaries though, the result was two "greys" or two "non-greys" side by side, and to change everything--there was dozens of them--I had to adjust each and every one of the subsequent table rows manually. Arg!

What did other people do? I wondered. Well, perhaps some companies have several Web designers and a bunch of people continuously working day in and out manually updating all their site's content. But just one person? And only several hours a weeks? Hey man, I want to do more than struggle with HTML editors all day!

So I have crappy Websites.

Personal

I had a nice dinner with some of my own grown corn and tomatoes. Like having your own hand made tools--for the garden as for the computer--it is better than something store bought.

Books

One of the books I am reading is "From the Eagle's Wing", a biography of John Muir. Muir was an inventor, making complex things like clocks, unschooled, with his own hands out of wood. Of showing some of his inventions at a State Fair, in 1860, the author writes:

"The joys of the small fry, the sly humor and pride of the inventor, the cleverness of the machine, plus the uproarious response of the crowd, made John Muir's invention the great attraction of the hall."

This, at least, I understand.

38 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!