Older blog entries for Nafai77 (starting at number 16)

New Languages

I've recently been feeling the desire to learn a new programming language. The last language I learned on my own (not directly for school or work) was Python, and that was 7 years ago! In that time, I've learned Common Lisp for my Artificial Intelligence class, Torque Script -- the scripting language for the Torque Game Engine -- for my last job, and Java for my latest job. I do feel some desire to get back to Lisp, but I'm also drawn to other languages.

Looking back, I think there were a few factors that lead me to learn and stick with Python:

  • Between a well-designed language and a great standard library, I was able to produce something that was useful within my first afternoon of learning Python. It's an entry for another day, but suffice to say, in that first afternoon I wrote the first iteration of a small utility that I used every day for 3 or 4 years.

  • A great community. I had some questions and I found #python (which was on irc.efnet.org at the time) and there were a lot of very helpful people there that helped me with my silly questions.

From that, some of my main interests in learning a language or program ultimately boil down to these two things:

  1. Is there sufficient power and ability to do something that is ultimately useful to me?
  2. Is there an active community behind and supporting the language or program?

That said, it sometimes goes against something Alan Perlis (from Perlisisms) said:

"A language that doesn't affect the way you think about programming, is not worth knowing."

In congruence with this, Eric Raymond, in How To Become A Hacker, said this:

LISP is worth learning for a different reason -- the profound enlightenment experience you will have when you finally get it. That experience will make you a better programmer for the rest of your days, even if you never actually use LISP itself a lot.

I do love learning new things that make me look at problems differently. But, unfortunately, my laziness as a programmer has made me ultimately follow what Eric Raymond has said -- I learned Lisp, I got it, I loved it -- but I never use it.

So I'm conflicted. On one hand, I have projects that I want to do -- useful things I want to write for my use. On the other hand, I want to learn something new, something elegant. So there is that conflict between utility and beauty.

That said, I'm trying to start to learn something new. I'm looking at two languages now, each with an active, helpful, and intelligent community behind it: Haskell and Factor. Each has an active channel on the IRC network irc.freenode.net (#haskell and #concatenative respectively). I've started reading the tutorials and writing a little bit of code. Haskell is a pure functional language. Factor is stack-based. Neither paradigm is something that I've done anything extensively with, so there is a bit of a learning curve.

Now all I have to do is come up with something useful to write so I can meet my first criteria above. I'll probably be writing more about what I learn of each of these languages.

Syndicated 2006-12-02 18:44:00 from Travis B. Hartwell / Software Craftsman

How to host Pyblosxom as a WSGI application with Twisted

I recently switched to PyBlosxom for my blog software. I have been using Twisted to host my website since I had my own server and wished to continue to do so if possible.


  • Twisted Web 2: Twisted Web2 has built-in support for WSGI, which is used to connect to PyBlosxom. The 0.2.0 release should work fine, though I'm running off of Twisted svn trunk.

    Note: The stated requirement to download twisted_wsgi.py found in wsgi_app.py in the Pyblosxom distribution only applies to Twisted Web 1, not Twisted Web 2, as Twisted Web 2 has WSGI support built in.

  • Twisted: If you chose to download the release tarball of Twisted Web2, you'll also have to download the 2.4.0 release. If, however, you have checked it out from subversion, you'll have both Twisted Web2 and Twisted. Refer to the Twisted site for more details.

  • PyBlosxom: The latest release should be fine, but note that I'm also running off of PyBlosxom SVN Trunk.

General Setup

  1. Get all of the above modules set up. The easiest way is to refer to the INSTALL file in each of the downloaded or checked out distributions.
  2. Create a directory for all of your configuration and run-time files. For example, I use $HOME/bin.
  3. Copy web/wsgi_app.py from the PyBlosxom distribution in the directory.
  4. Copy the config.py PyBlosxom configuration file into the directory as well, making any customizations that are needed.

Twisted Configuration

I use a .tac file for my Twisted server configuration. It is simply a set of Python code required to set up the server and its parameters. Here is a minimum working configuration, with no Twisted logging enabled:

   from twisted.application import service, strports
 from twisted.web2 import server, channel, wsgi
 from twisted.web2.channel import http
 from wsgi_app import application as pyblosxom_application

 pyBlosxomResource = wsgi.WSGIResource(pyblosxom_application)

 site = server.Site(pyBlosxomResource)
 application = service.Application('web')
 s = strports.service('tcp:80', channel.HTTPFactory(site))

I could probably make it a bit cleaner, but this is a pared down version of the more complex working version that hosts my sites. You can download and save blog.tac in the directory created above. Be sure to change the port number from tcp:80 to tcp:<custom port number> if you want the server to listen on a different port.

Startup Script

I also wrote a quick script to make it easy to start the blog server. You'll have to modify the pathnames appropriately to your particular setup. Here is the script:


 cd ~/bin/
 export PYTHONPATH=/home/nafai/code/Twisted:/home/nafai/code/pyblosxom-new/pyblosxom:$PYTHONPATH
 python2.4 /home/nafai/code/Twisted/bin/twistd -y /home/nafai/bin/blog.tac --pidfile=blog.pid --logfile=blog.log

Download and save startblog.sh in the directory created above. Make it executable by doing chmod +x startblog.sh.

Run it!

Now try running the blog service (for example, if the configuration files were saved in ~/bin:


You should be able to see your blog if you browse to http://localhost/ with your web browser. If you get a 503 or other error, check blog.log in the ~/bin directory for any tracebacks or errors. Also check the pyblosxom specific log if you have one configured in your config.py.


You may run into a bug that I've seen, but haven't taken the time to track down where it is coming from. I am getting this traceback fairly often in my blog.log file, though everything else seems to run normal:

   2006/08/24 22:22 MDT [HTTPChannel,39,] Traceback (most recent call last):
 2006/08/24 22:22 MDT [HTTPChannel,39,]   File "logging/__init__.py", line 737, in emit
 2006/08/24 22:22 MDT [HTTPChannel,39,] ValueError: I/O operation on closed file

I should track this down sometime soon.


Happy TwistedPyBlogging!

Syndicated 2006-08-25 05:43:00 from Travis B. Hartwell / Software Craftsman

How to redirect stderr and stdout to a file plus display at the same time

The biggest search term that has brought people to my blog this month has been "redirecting stderr". I realize that I probably should expound more on Redirecting stderr and stdout to a file plus displaying them so that it makes more sense.

The original example script:



if [ ! -e $OUTPUT_PIPE ]; then
    mkfifo $OUTPUT_PIPE

if [ -e $OUTPUT_LOG ]; then
    rm $OUTPUT_LOG

exec 3>&1 4>&2
exec > $OUTPUT_PIPE 2>&1

echo "This is on standard out"
echo "This is on standard err" >&2

exec 1>&3 3>&- 2>&4 4>&-
wait $tpid


The Explanation


  if [ ! -e $OUTPUT_PIPE ]; then
    mkfifo $OUTPUT_PIPE

I needed to make a named pipe -- otherwise known as a FIFO (First In First Out) -- to provide means for the shell and tee to share the output. You're probably familiar with pipes, such as ls -l | grep myfile. Named pipes are merely a way for you to do the same thing via a "file" in the filesystem. That same command can actually be done this way:

  mkfifo tmpfifo && (ls -l > tmpfifo &) && grep myfile < tmpfifo

Yeah, it's a little messy for that simple command. But it can be used in cases where a simple pipe cannot be used.


  exec 3>&1 4>&2

This saves the file descriptors for stdout (file descriptor 1) and stderr (file descriptor 2) to file descriptors 3 and 4.


  tee $OUTPUT_LOG < $OUTPUT_PIPE >&3 &

tee will, to quote from the manpage tee(1): "read from standard input and write to standard output and files". So, let's break up this line into parts to describe what's going on:

  • tee $OUTPUT_LOG: tee will take from stdin and write to the file named in the variable $OUTPUT_LOG.
  • < $OUTPUT_PIPE: tee's stdin will be redirected from the named pipe named in the variable $OUTPUT_PIPE
  • >&3: redirect tee's stdout to file descriptor 3, which is the stdout of the shell script
  • &: start tee as a background process of the shell



This line saves the process id of the tee process to the variable tpid.


  exec > $OUTPUT_PIPE 2>&1

In parts:

  • exec > $OUTPUT_PIPE: redirects the stdout of the script to the named pipe named in the variable $OUTPUT_PIPE.
  • 2>&1: redirects stderr to stdout so both stderr and stdout are redirected to the named pipe.

Then comes whatever you need your script to do. I included a few echo statements to show that both stdout and stderr are shown both in the terminal and in the log file.

After the meat of the script:

  exec 1>&3 3>&- 2>&4 4>&-

  • 1>&3 and 2>&4 restore the original file descriptors for stdout and stderr from file descriptors 3 and 4.
  • 3>&~ 4>&~: now that they are not needed, close file descriptors 3 and 4


  wait $tpid

This "pauses" the script until the tee process (whose process id is saved in the variable $tpid) exits. It will exit because its stdin (the output from the named pipe) has been closed. The named pipe closed the stdin because its input (file descriptors 1) was closed when the file descriptor stored in descriptor 3 was restored back to the original stdin.



remove the named pipe, because it is not needed any more.

Addtional Information

For additional information, Chapter 19. I/O Redirection and Appendix E. A Detailed Introduction to I/O and I/O Redirection of the Advanced Bash-Scripting Guide are very useful in understanding these concepts. Also, a good introduction to named pipes can be found in this Linux Journal article.

I hope this helped make things a little bit clearer.

Syndicated 2006-08-20 04:20:00 from Travis B. Hartwell / Software Craftsman

Handy script to search for and install Debian packages

I often find myself searching for a package with apt-cache search and then installing interesting matches with apt-get install. Whenever I find myself doing something over and over again, I want to automate it, to make it easier, faster, repeatable. To this end, I have come up with search_and_install.sh.

Some may say that I should just use synaptic or a similar tool. I find doing this is much quicker and easier to use for this use case.

To Use

To use, just run search_and_install.sh or search_and_install.sh <search terms>. If you leave off the search terms, a dialog box will prompt you for the search terms. If there are matches, a dialog box will come up listing the matching packages with their name and the short description returned by apt-cache search. Each will have a checkbox beside it. Check the packages you want to install and then hit OK. You will then be prompted for your password (or your root password, if you are on a non-Ubuntu system), and synaptic will run, downloading and installing the selected packages.


Currently, packages you already have installed are shown in the list. In future versions, I'd like to filter those out.


bash, zenity, apt, synaptic, gksu

A standard Ubuntu install should have these, and these are easily installable on any other Debian-derived distro.

Download search_and_install.sh.

This and other scripts will be linked from my Random Code page. Hopefully something there will be useful.

Syndicated 2006-07-31 00:16:00 from Travis B. Hartwell / Software Craftsman

Quick pop-up of Tomboy notes

I've started using Tomboy to keep track of my tasks lists, to keep random notes, whatever. I've found it to be very useful.

I am always seeking to be more productive and find ways to reduce the amount of distraction in my life. One example: if I am in the middle of working on a project and remember I have to pick something up on my way home from work, I want to quickly jot that down so I don't forget. But I don't want to interrupt the concentration that I have on the task at hand. Tomboy is an ideal solution for this. I am trying to implement a GTD-like system, so I have a Tomboy note titled "GTD Inbox". The idea is to put all of those things that need to be processed and remembered in my "GTD Inbox" to be dealt with later.

Normally, to get to do this, I would have to click on the Tomboy icon in the notification area and select the note title. Alternately, I would hit Alt-F12 to pop up the same menu and move down with the arrow keys and hit Enter. If I am trying to minimize the interruption of the original task, that's too much. I'd rather rely on "muscle memory" -- something that is as natural as pressing Alt-Tab to switch between windows.

With the help of Alex Gravely (thanks Alex!), I was able to come up with a popup_note.sh that uses dbus-send to open up a specified note. With that, I could set a hotkey in Metacity for each of the important notes. For example, at work Windows-i opens "GTD Inbox", Windows-n opens "Next Actions" (my list of next actions for all of my projects, GTD-style), and Windows-t opens "Today" (my list of goals and tasks for the day).

So, back to my "remember to pick up the milk" example. All I'd do is hit Windows-i, type "Pick up Milk", and hit Escape. The Note goes away and I'm back at doing whatever I was doing previously. Wonderful.

To use the script, just pass the title of the note on the commandline, like so:

   $ popup_note.sh GTD Inbox

If you want to set up global hotkeys in Metacity, do this:

   $ gconftool-2 -t string -s /apps/metacity/global_keybindings/run_command_1 "<Mod4>i"
 $ gconftool-2 -t string -s /apps/metacity/keybinding_commands/command_1 "popup_note.sh GTD Inbox"

Download popup_note.sh.

This and other scripts will be linked from my Random Code page. Hopefully something there will be useful.

Syndicated 2006-07-25 04:29:00 from Travis B. Hartwell / Software Craftsman

New Blog

I'm moving my blog from LiveJournal to my own domain. I'm using PyBlosxom, and so far I'm pretty pleased.

I plan on blogging more now that I have greater flexibility and control.

Watch this space.

Syndicated 2006-07-21 15:08:00 from Travis B. Hartwell / Software Craftsman

Redirecting stderr and stdout to a file plus displaying them

It has taken me ages to figure this out. Googling has been fruitless. Finally, after some tinkering, I was able to figure out how to properly redirect stdout and stderr to a file plus display them from within a script. Here's the idiom:



if [ ! -e $OUTPUT_PIPE ]; then
    mkfifo $OUTPUT_PIPE

if [ -e $OUTPUT_LOG ]; then
    rm $OUTPUT_LOG

exec 3>&1 4>&2
exec > $OUTPUT_PIPE 2>&1

echo "This is on standard out"
echo "This is on standard err" >&2

exec 1>&3 3>&- 2>&4 4>&-
wait $tpid


It works but it might not be entirely correct, particularly the exec &> /dev/null. Improvements or suggestions are welcome.

Update: Figured it out. I had to store stderr and then restore stdout and stderr and close the fd that I stored them in.

Update update: Looked back and realized I had a typo on my first exec line.

Syndicated 2006-07-14 08:23:00 from Travis B. Hartwell / Software Craftsman

Random Ideas


Okay, maybe this is ironic that I'm posting this on my Live Journal account, but I really hate how limited LJ is. I would love to be able to track site statistics and reads and so forth. I just got a Google Analytics account for my main domain. There is currently (and probably never will be) a way for me to put in the requisite javascript code into my template at LJ to have Google Analytics track things. Yet another reason to move off of it. After the wedding, I think I'll finally have time to get my blog set up on my own hosting account. I'm going to be using Pyblosxom as it seems to be most flexible and easiest to integrate with Twisted Web. I got Twisted Web 2 working with Pyblosxom via WSGI with a minimal amount of code and some help from #twisted.web. I'll post how I did that when I get that blog up.

Random Ideas

I figure this would be a good place to throw out my random ideas of things I want to hack on, things to figure out. Feel free to tell me my ideas are dumb. Feel even freer to tell me that they've already been implemented or that you want to write them. :)

In reference to my prior entry regarding my IRC and IM setup, here are some things I've been thinking about:

  • I want to write a programmable chat notifier, so I don't have to be constantly logged onto my session locally. I want an application that will connect to my proxy for me and, when certain conditions (regular expressions or something) are met, notifiy me via libnotify and the notification area of my Gnome panel. That way I can't let IM/IRC distract me.
  • Write a wrapper around XChat-Gnome to handle my setup more cleanly:
    • If my ssh tunnel is not started, start it, prompting me for the password
    • Turn off the notification plug-ins (sound, notification, and on screen display) to start. As bip is playing back my log to the session, I don't want to be notified.
    • Find out, via d-bus or a python plug-in or something, that the playback is done and then enable those plug-ins
    • If appropriate, on exit, kill the tunnel

I've started using Evolution as my primary mail client, so I have a few things I want to do with it:

  • Update my (not-yet-released) new mail notifier:

    • Make it more robust and clean code-wise.
    • Give more information, if possible. (Number of new messages, perhaps message information)
    • Leave an icon in the notification area until I click the pop-up notification or the system tray icon.
    • When clicking either, go to the new messages in Evolution
  • I get a lot of spam -- probably 1250 or so in the last week. I need to curb that.

    • We use spam bayes on my mail server. I need some way to retrain it, when it misses or mis-classifies something.
    • I also use spamassassin locally, which works for most of what I have.
    • First filter on spam bayes, and then spam assassin classifications.
    • I want to be able to set up something -- like search folders -- that will capture messages that are in the junk folder, but missed by one or the other.
    • I want to be able to automatically re-train spambayes and spamassassin appropriately on these.

So, for Evolution, I need to figure out the following:

  • In what order are the filters applied? Junk filters and then user-defined filters? Can I specfy the order of this? Can I specify the order of my user-defined filters?
  • Can I set up a filter that is only applied when I select it? For example, I need to write some shell scripts to handle re-training Spambayes on my remote server. I would like to be able to click on a message and then tell Evolution to pipe that message to a given program. Can I do this?

I think that's all of my random ideas for now. I probably won't get to any of this until after the wedding. Of well, priorities, you know.

Syndicated 2006-05-17 19:44:00 from Travis B. Hartwell / Software Craftsman

Ten Days and Counting

On Saturday, May 27th, my sweetheart Torie and I are getting married!

Ten days!

Torie and Travis engagement picture

Syndicated 2006-05-17 19:41:00 from Travis B. Hartwell / Software Craftsman

Iswitch-window handler update

I keep meaning to announce that my iswitch-window deskbar handler as described here, has now been included in the core Gnome Deskbar Applet distribution, as seen in the 2.15.1 announcement. Any further changes or improvements will go directly into deskbar cvs.

Enjoy! Thanks Raph and team for a great project and including my contribution!

Syndicated 2006-05-01 10:43:00 from Travis B. Hartwell / Software Craftsman

7 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!