Older blog entries for connolly (starting at number 49)

• 18:98992cbff4fd 2007-01-04 work around MS Exchange attachment bug

Exchange 2000 Server requires a file name in the Content-type header. Q12: How does Exchange handle attachments? Microsoft Article 836555

I bet you can guess the rest of the story, but I hope to write it up later today.

#swig notes

appscript and office automation

My wife does office work for a local professional and whenever I see her doing work that I know the computer could do for her, I chip in. The end-of-the-month scramble is a clear case: they take client reports, print them out and then manually sort them by officer and fax them out.

Surely I could do better with faxaway, I thought. The only question was: since the reports are in MS Word and the database is in FileMaker Pro, all on a Mac laptop, how much would I have to sell my soul to Apple and Microsoft in the process?

appscript let me drive the process from python. I did quite a bit of HyperTalk programming, but somehow I'm still a bit mystified by AppleScript: which are the language keywords and which are the application vocabulary? The FileMakerAppscriptingOverview made it trivial to crib bits like:

    fm = app("FileMaker Pro")
    if not fm.databases[db].exists():
        fm.open(FSSpec(path % db))
        return fm.databases[db]

FSSPec is deprecated in the Carbon docs, but I never did figure out a replacement.

appscript's integration of AppleScript references into python with its and con is particularly cute, but I pulled a bit of hair out before I figured out how to use it:

def officerFax(db, oName, cName):
    # hmm... I'm not sure why this str() is necessary...
    officers = db.tables['officers'].records[
            its.fields['name'].cellValue == str(oName)]

If FileMaker has a way to use real SQL, I can't find it. Plus, we're running a PowerPC version on an intel MacBook with only 0.5GB of RAM. Emulating FileMaker and MS Word is using a lot of RAM, I suspect. I looked into open source alternatives and found that OpenOffice's Base looks quite capable, and I'm sure oowriter would do the job as an MS Word replacement. I hope the python-uno bridge works on OS X so that I can switch the whole operation over one of these moths.

I did pay a price for not doing it The Apple Way. Technical Q&A QA1018 Using AppleScript to send an email with an attachment shows exactly how to attach a report to a mail message and send it to faxaway. I was able to create and address a mail message from python/appscript, but making the attachment stumped me. After verifying that the AppleScript example does work as advertised, I gave up and wrote a separate mailfaxes.py program that uses python's email and smtplib modules and skips Mail.app altogether. I had to be a little careful since the laptop runs python2.3 and the email modules have been rearranged a bit in python 2.4 and 2.5, but it was reasonably straightforward.

Driving MS Word was, predicably, even klunkier:

TMP="fax_job.htm"
def asHTML(w, dirpath, fname):
    """save current doc as HTML
    """
    w.do_Visual_Basic('ActiveDocument.SaveAs FileName:="%s",'
        ' FileFormat:= wdFormatHTML,'
        ' HTMLDisplayOnlyOutput:=True' % (TMP,))

Office X has an AppleScript interface, but it's not as rich as the Visual Basic API. I got Word to save as HTML (for processing with BeautifulSoup) but I never did figure out how to tell MS Word which directory to put it in. I wrote a posix2mac() routine to convert /posix/paths to ::mac:paths as used in AppleScript but that didn't help; I ended up with a hard-coded kludge.

Switching syntaxes with do_Visual_Basic is a little bit painful, but when it goes bad the diagnostics are pretty good. "ActivePrinter is read-only on the Macintosh," it said, where w.active_printer = p had just failed silently. The modern VB.NET PrintOut documentation isn't hard to find, but it's a little more tricky to find the 2002 PrintOut docs that are more relevant. I never did get PrintToFile working, nor did I find a way to script the PDF option in Apple's print dialogs. Thank goodness for the Appscript, Word and PDF clue which pointed me to CUPS-PDF for Mac OS X. It worked as advertised, though writing code to wait for a new PDF document in ~/Desktop/cups-pdf/ was tricky; we sent a number of reports to the wrong place due to a timing bug.

The Python Bindings for Quartz 2D rock; composing fax cover pages couldn't be easier than this:

htmltxt = coverHTML(oName, fax, subject, pages)
ctx.drawHTMLTextInRect(
        CG.CGDataProviderCreateWithString(htmltxt),
        pageRect.inset(72, 72))

and concatenating several PDFs into one was similarly straightforward. It doesn't hurt that faxing is Apple's example application.

For reference:

hh-fax2$ hg log --template '#rev#:#node|short#
#date|shortdate# #desc|firstline|strip#\n'
11:7b497e5881d8 2006-12-07 fixed nasty timing bug with PDF
virtual printer
10:5fbd62cf7025 2006-12-07 fixed SMTP details
9:adbc7966d42d 2006-12-07 back to faxaway
8:89f2688b85fc 2006-12-07 smtp host arg
7:eb4eba0ed22c 2006-12-07 mailfaxes.py starting to work
6:c317d0cb9956 2006-12-07 prepares one PDF doc per officer
5:fe8a79a7ed9f 2006-12-06 faxjob.py iterates over reports
and looks up fax numbers
4:52f42112c287 2006-12-06 better diagnostics
3:862514804543 2006-12-04 officer update mostly working
2:61dfc88ab652 2006-12-04 connecting to FM from py works
1:a1f813e53e79 2006-12-02 HTML/CSS page break test
0:aa2d0cc8a7e9 2006-12-02 save as html, doc export working

Worse is better for external disk enclosures too

I got a USB 2.0 disk enclosure this week. It's much quieter than the FireWire enclosure I bought in Oct 2003. I thought surely it was slower, but it's not.

I learned assembly on a 6809E; I thought people were crazy to deal with the 8-bit shackles of the 6502 and the broken segmented architecture of the 8086, though I could see Intel beat Motorola to market with each new generation. I'm a sucker for elegant technology. I paid the SCSI premium for years. I never actually bought a NuBus Mac nor a Newton, but I wanted to. I bought 2 PowerPC Macs, the last one just before even Apple switched to Intel. But slowly, I'm learning: worse is better. As a Web guy, I should know that even a dollar per part is a lot when you're talking about hundreds of millions of parts...

However, Intel was not interested in paying the near-dollar license fee to add an IEEE 1394 subsystem to their board. The fee was reduced to a flat 25 cents, but Intel prefered to push for its own USB 2.0 standard. As a result, they were rarely provided as standard equipment on computers other than Apple Macintosh computers (Apple owns rights to the FireWire standard), and peripheral manufacturers offered many more USB devices.

I divested from SCSI a few years aog; all my disks are commodity IDE now. I'm keeping an eye on SATA but haven't jumped yet.

Yesterday I plugged in a new 22" monitor that wants 1680x1050. Gnome was only offering up to 1280x1024 or something, so I chose that. Then I logged out of X and did `dpkg-reconfigure xserver-xorg` and set up X to do 1680x1050. When I logged in, my gnome session was set up for 1280x1024, which caused the monitor to go into a "input not supported" mode (because 1280x1024 was no longer in the xorg modeline perhaps?). I had to login using failsafe terminal mode, manually start a window manager and web browser, google to discover the name of gnome-control-center, and then tell gnome the right resolution, logout, and log in again.

bullet-proof-x looks like a good idea, to me. That's an ubuntu goal, but I hope it makes it to debian soon.

Boy do I miss debian weekly news. That's where this Dunc-Tank bruhaha hits the pavement, for me.

And I haven't updated my sid install for a few weeks because I'm hoping the firefox trademark issue will blow over.

I'm trying out ubuntu today. The reasons are pretty arbitrary; I sat down to work on the family finances last night, but I wasn't really in the mood, so I revisited the ubuntu installation on the PC I built with my son back in Feb 2005.

I like the focus on common tasks such as playing music, video editing, etc. And I like the fact that they invite support requests, not just bug reports (though launchpad is Yet Another Password that I would rather manage with OpenID). So here are the tasks I tried:

Reading Mail

Evolution seems to get my mail over IMAP+SSL pretty straightforwardly, though the password prompt reminded me that I'm not sure how secure that "remember my password" option is.

Writing Mail

I found the "bcc me on everything I send" option easily enough; the last thing I want the computer to do is to record my knowledge and then lose it, after all.

I realized I'm addicated to tab-completion of addressees, and I don't know where evolution stores the addressbook. The question that remains is whether to invest in figuring out how evolution works or to invest more in my RFC822/RDF stuff and maybe set up an LDAP proxy.

Also... how do I copy my signatures from one machine to another?

I wonder if it's using gconf; that thing feels too much like the windows registry for my tastes. Centralized, hard to manage and backup.

Chatting with colleagues via IRC

At W3C, we do a lot of teleconferences with supplemental IRC channels. You can do a lot of work asynchronously via email and the web, but getting together real-time once a week seems pretty important.

I started up xchat-gnome and I was pretty happy until I realized it wasn't logging all the notes I was taking. (integrity is job one). Automatic logging of conversations is fixed as of 2006-07-27, but the latest release is 0.13 of July 19th, 2006, but I'm running 0.11 from dapper. I tried to build from source but quickly found myself in automake hell ( ***Error***: You must have automake >= 1.9 installed)

gaim is also installed (did it come out of the box with Ubuntu? I can't remember; it is in the list of supported apps, in any case). I tried it a while ago but I use normal punctuation for direct address (i.e. "joenick, what do you think?" not "joenick: what do you think?") and gaim doesn't grok an equivalent of /set completion_char , .

So with an apt-get install xchat, I'm back to what I was using under debian.

Posting to a weblog/journal

The gnome Applications menu is not reliable enough for me to look there first as a habit; I did a google search for blog clients; I found gnome-blog in the wordpress clients list. apt told me it was already installed... oh... there it is... Applications/Internet/Blog entry poster.

real-time spell-check... do I like that? I'm not sure. I don't see a way to do excerpting, and I don't see a way to edit the source directly. Making a link was pretty painless; but... ew... I can't follow it, nor select the address. bzzt. might as well switch to emacs and nxml-mode; I don't know if gnome-blog knows that integrity is job one, after all, and I think I'm not in the mood to test it just now.

I guess I'm already doing email the Ubuntu way under debian. For the other tasks, the Ubuntu way didn't meet my needs and I had to drop back to the geek/wizard tools that I'm using under debian. My one foray into development landed in automake hell, but I'm not going to fault Ubunutu for that.

I still hesitate to invest heavily in Ubuntu given things like Joey's ubuntu gripes. I heard various bits of ubuntu/debian coordination from debconf6; I wonder how many of Joey's gripes are still outstanding.

I made a sort of new year's resolution for 2006: no more undocumented, untested code. I like to write little ditties for stress relief, but without docs and tests, the code seems to add to my stress, over time. At my first job out of college, at Convex, management devoted equal resources to coding, documentation, and testing/QA. The more time goes by, the more wisdom I see in that way of working. If the code is worth writing, it's worth testing and documenting.

It's best to write the tests first, and it's good to have use cases in mind, if not documenation written down, before that.

For personal projects, I'm not disciplined enough to write the tests first every time; if the code I write works the first time, I sometimes let myself get away with it. But I'm doing pretty well about doing test-driven debugging, at least. I write tests for any code that doesn't work the first time. And I write tests when I refactor and change things. The confidence to make changes that comes from having tests in place is very freeing.

Recent coding/testing/documenting episodes include:

• Getting my Personal Finance data back with hCalendar and hCard
• bnf2turtle -- write a turtle version of an EBNF grammar

re-discovering dotfiles

I thought I understood how .profile, .bashrc, .login worked, but it has evidently faded from memory. After years of relying on sensible defaults in debian etc., I'm using mercurial/hg for source code management. A big advantage of mercurial is that it does not need to be centrally managed; I can install it in ~/bin. But then I want to use it across an ssh connection, so $PATH has to get setup somehow. I've got that working, using .bashrc.

But I have to manually . ~/.bashrc before I can use mercurial in interactive logins. That's tolerable, but I have to remember to do it before I start up emacs, or else I get the dreaded:

Traceback (most recent call last):
  File "/Users/connolly/bin/hg", line 10, in ?
    from mercurial import commands
ImportError: No module named mercurial

when I try to use hg/emacs integration. I rely heavily on emacs/cvs integration, and working this out will be critical to moving to hg, for me.

update: Well, I got it working by symlinking .bash_profile to .bashrc; I would have thought I could do it within POSIX-standard facilities, but I can't see how. I guess I can rely on bash being on all the machines I use.

crap; This emacs CUA-mode doesn't really work. If I copy from firefox and ctrl-v in emacs, it pastes the last thing I yanked from emacs, not what I copied from firefox.

I rilllyrillyirillyrilly hate that.

I saw something about a gnome clipboard daemon... what's up with that? Ah...

It was stupid of me to think that I had the power to fix the Clipboard by creating one application. I don't have that power. Discontinuation of GNOME Clipboard Manager

So is there some freedesktop.org standard brewing? Aha...

Qt 3 and GNU Emacs 21 will use interpretation 2, changing the behavior of previous versions. freedesktop.org Standards/Clipboards

But... but...

$ apt-cache policy emacs21
emacs21:
  Installed: 21.4a-3
  Candidate: 21.4a-3

So wtf?

CUA-mode, where have you been all my life?!?!?! Found in some comments on Gerv's search for an editor. Could it be true? No more impedence mismatches between emacs's sense of paste and the rest of the gnome/mac/windows world?

Much of my hacking life has been spent in wondering whether to mac or not to mac. I don't know how many times a day I curse emacs or firefox or gnome-terminal for screwing up copy-and-paste. I have so much angst every time I try... "if I close the window that I just copied from, will paste still work?" "oh crap! I don't want to paste what I just selected; I want to paste what I carefully spent 10 minutes copying to the clipboard!"

Speaking of to-Mac-or-not-to-Mac, and imporovisation, I wrote/discovered another little three chord ditty last night (did I mention I love having a piano in the house?). Part of me wants to try out the western music notation support in Garage Band 2, but the rest of me knows that recording my knowledge using proprietary stuff is a step backwards. I have spent many, many hours trying to get RoseGarden and the like working under debian linux... preempt kernel patch... OSS vs ALSA vs Jack vs ARTS... argh! I look forward to the day when Ubuntu can afford to make it a priority.

Back from XML 2005 in Altanta, looking into a dist-upgrade, I hesitate to make the switch to the x.org X server, since I use an nvidia chip. Does the open source nv driver support 3d acceleration?

tags: debian, dirk.dm93.org, hardware, graphics