Older blog entries for connolly (starting at number 81)

Setting up project tracking with trac, hg, and a little help from my friends on freenode

One of my first tasks here was setting up version control. The guy who hired me, Russ, had experience with svn and especially TracBrowser and was aware of the trend toward distributed version control. While git has a 1st-mover advantage and bzr is maturing, I have been using mercurial (hg) since before github and the like made git's user interface tolerable and before bzr's performance was acceptable. Between Joel Spolsky's excellent Hg Init tutorial and the TortoiseHg windows shell integration, it wasn't hard to get mercurial adopted as our version control system.



Of the three, bzr is the only one that does 1st-class renames (including directories); I'm betting we won't need to cross that bridge for some time.

Mercurial's limitations around big files never bothered me before; I formed my version control habits with CVS, which handles binary files about as well as my teenage boys handle vegetables: only when given very specific instructions and even then, not gracefully. But there are a lot of Microsoft Office documents around here, this being an enterprise with largely pre-web computing norms. As a web guy, my sentiments are largely with Edd:

But really, office documents make me cry.

-- Platform agnostic productivity - once bitten

But that's where the information is, so I took a few a MS Word documents and put them in our mercurial repository. Once I realized what a hassle it would be to merge diffs in MS Word documents (is it even possible?) I realized it made more sense to put them in our trac wiki as attachments.

Trac turns out to be a great fit for our group. I knew from experience that bugzilla is a hassle that's only worthwhile in communities several orders of magnitude larger than ours. My other experience was with roundup, whose web interface is so smooth that you can just about capture status at the rate  developers discuss issues around a table. I knew that I wanted an issue tracking system, but I wasn't sure whether the integrated wiki and source code browsers would help us out or just get in the way. The wiki turns out to be a great catalyst for the switch from mailing copies around at each iteration to working in a shared web space. MS Word documents can be thrown in as attachments at first and migrated to hypertext in due course.

At first the tickets were just a way for me to track my own work, but we're getting pretty good at using trac for an approximation of the XP planning game: Russ plays business customer and I represent development. Sometimes sketching milestones and breaking them down into tickets is straightforward enough, but sometimes I don't understand what he's asking for, so I ask him to tell me stories about how it works, and we capture those in the wiki.

It's working well enough that some people are considering it as an alternative to MS Sharepoint in a few cases. I wonder how the cost of setting sharepoint compares to what I've been through with trac:

• The initial stand-alone installation of trac worked fine, but when I tried to integrate it with apache via mod_wsgi, I ran into shared library conflicts. Fortunately, IssuesWithExpatLibrary are well documented and the fix was just:
• sudo zypper install libexpat-devel
  zypper? what's that? It's the SuSE equivalent to apt-get. The enterprise investment in Novell spills over into using SuSE linux rather than debian or Ubuntu.
• rebuild apache with --with-expat=/usr/lib
• Trac has a traccas plug-in for the campus single sign-on system, CAS. I ran into redirect loops when I first set it up; not to fear: it's a a known bug (ticket 4025) and the fix was just to give  repository_dir a good value in the trac.ini configuration file.
• In one of the projects here, we provide requirements to a vendor. To help track which requirements are satisfied by a recent code drop, I took the MS Word document with the current table of requirements, saved it as HTML, tidied it, shuffled the columns around a bit with a 50-line python script, and imported it with the TicketImport plug-in (which is really great because it can do updates as well as inserts).
• It seemed a little silly to use .xls rather than .csv for the import, but the data had unicode quotes and the xlwt package is unicode-happy while python's csv module isn't.
• One of the services provided by our department has an established workflow. I'm experimenting with representing it in trac. I added a custom ticket type and used the MultipleWorkflow plug-in to give it a custom workflow. The jury is still out on whether we'll adopt trac to manage this service.

The difference in license fees is easy enough to compute: it's whatever sharepoint costs minus zero, since all of the above is free software. Perhaps I spent more time setting it up than sharepoint administrators do, but I suspect that even so, there's some leftover that I should arrange to donate to freenode, which provides discussion facilities: the #trac channel (thanks coderanger!) and #mercurial and #pythonpaste and #httpd and #suse. Next time you're stuck or you just want to bounce ideas off others that are likely to be interested, give it a try.

Syndicated 2010-08-09 21:09:00 (Updated 2010-09-18 00:51:39) from Dan Connolly

My own bit of office space... come join the fun?

The cube I work in is pretty typical, I guess, but I'm going through a bit of culture shock after 13 years of working out of an office in my home. Look what my wife got me to celebrate the transition:

How thoughtful... she confirmed that this is the exact model that Milton valued more than his job in Office Space:

Milton Waddams: [talking on the phone] And I said, I don't care if they lay me off either, because I told, I told Bill that if they move my desk one more time, then, then I'm, I'm quitting, I'm going to quit. And, and I told Don too, because they've moved my desk four times already this year, and I used to be over by the window, and I could see the squirrels, and they were merry, but then, they switched from the Swingline to the Boston stapler, but I kept my Swingline stapler because it didn't bind up as much, and I kept the staples for the Swingline stapler and it's not okay because if they take my stapler then I'll set the building on fire... (imdb)

I'm not really that picky about staplers, but one of the first things that bugged me about life in a cube was the distracting sounds, and back when I spent a lot of time on airplanes, my thoughtful wife got me something that comes in handy for that: a pair of Sennheiser PXC 250 Active Noise Canceling Headphones. When I put those on and tune in to  my last.fm neighborhood, I can maintain flow state just fine.

Plus, working on campus has its benefits. At the fitness center, Jeff's boot camp class has me sucking wind so badly I couldn't possibly worry about anything else. Or sometimes I'm just up for a few laps (salt water pool!) and a soak in the hot tub.

The reason I'm in a cube is that our department is growing in people faster than it's growing in space. If you'd like to join the fun, we're hiring! Consider applying:

• Biomedical Informatics Software Engineer  
• Clinical Application Administrator

Syndicated 2010-07-26 19:55:00 (Updated 2010-09-18 00:53:11) from Dan Connolly

An invitation from the Inventor of the Web

Tim has declared a #DanFest for Dan Connolly Wed July 14th:

Assorted folks,

As you may know, recently Dan Connolly left the W3C team after very many years of invaluable service to the web community. Following an idea of Bijan's, we will have a distributed #DanFest.

• Date: Wednesday July 14  
• Time: 7pm Central, 8pm Eastern. 

The idea is that people all over the place dine in celebration of Dan's many accomplishments while at W3C. Then, half an hour after the official start of the meal,

• Time: 7:30 Central, 8:30 pm Eastern: we toast Dan simultaneously. <------ 

We may try to Videoskype Dan in to the team near MIT.


We realize people in different timezones will not be able to dine at the same time, so eating is a timezone-sepcific thing. Those in France, please set of fireworks to mark the occasion.
Tweets should be sent at the time, and of course can point to other things.

DanC is connollydwc dckc on twitter. Twitter hashtag is #danfest.

Tim

My family and I will be on the plaza for the event. Wherever you are, I look forward to hearing from you, and I'm sure Tim does too.

For a guy who played a pivotal role in a whole new era of communication among humans, Tim Berners-Lee is a relatively unknown figure. Our culture celebrates fame, fortune, and power, while Tim is all about sharing. It's been my honor to do what I could to help him out over the last decade and a half at W3C, not to mention a blast working with all the other people who contribute so much to making the web work.

If you have never heard of the guy who brought hypertext and the Internet together, please spend a few minutes watching this video:

Syndicated 2010-07-14 04:06:00 (Updated 2010-09-18 00:54:13) from Dan Connolly

Collecting in the Car with google voice

I commute about 45 minutes to work now, after over a decade of working from an office in my home.  I'm OK to listen passively to music and such sometimes, but alone time is a great time to think and mind-sweep, and nothing is more frustrating than having an idea and having no way to collect it somewhere other than in my head.

I just found a few minutes to set up speech to text using my cell phone and google voice.

What I actually said was "Let's see if I can leave voice notes to myself..." not "voicemail to myself" but it's certainly a useful transcription.

I first got the idea when I learned that the T-Mobile myTouch 3G slide has dragon dictate for transcription right there on the phone, as well as the normal androide send-it-to-the-mothership style transcription. But I checked the marketplace before buying one and found the Samsung Galaxy S on the horizon with twice the resolution and twice the CPU speed (both needed for Gingerbread, the next version of Android), so I'm holding out. I wonder how much I'd miss the keyboard. But that's another story. Back to transcription...

I was frustrated by my first attempt to set this up because calls from my mobile phone would go straight to voicemail, and at that point, there's no way to leave a message for myself! The trick is to use the "advanced settings" to tell google voice not to go straight to voicemail:

Syndicated 2010-07-13 19:12:00 (Updated 2010-07-13 19:12:56) from Dan Connolly

"Do what I say, not what I do," says the enterprise training calendar

How ironic! The registration confirmation message for the GroupWise for New Employees class doesn't have a machine-readable calendar invitation attached.

It does have an HTML description of the event that's clearly machine-generated:

Oh for hCalendar!

That is: if the registration system added just a little hCalendar markup, and if GroupWise grokk'd hCalendar, then we'd be all set. Or if the registration system added hCalendar markup and used it to generate a .ics version.

Hmm... does GroupWise actually support .ics attachments from other systems? Or is it just because I'm using Evolution as my GroupWise client that I think this would work?

Syndicated 2010-07-01 15:33:00 (Updated 2010-07-01 15:33:38) from Dan Connolly

A Return to Patronism

Everybody wonders how artists and publishers are going to survive in the age of digital distribution.

I tried running ads here like everybody else does. For about a day. I hated it.

I don't know how business models for record labels, newspaper publishers, and movie studios will evolve, but as for artists, I hope to see a return to patronism, where the customer is the fan, not the advertiser.

My favorite paper from the W3C Digital Rights Management workshop in 2001 was Mark S. Manasse's   Why Rights Management is Wrong (and What to Do Instead) :

Readers, viewers, and listeners are fans, not thieves. ... Mechanical enforcement of licenses should be lax to non-existent. Strictly enforced licenses would either be so permissive as to be useless, or they would make it difficult to loan an album to a friend, or to bring a video to a party. ... Legal enforcement of licenses is good, however ...

Then in 2002 I had an eye-opening chat with Aaron Swartz:

03:28:52I think ... preventing people from playing songs without a license is counter-productive. song-playing should be encouraged because it makes people more likely to buy the song (i.e. if they like it)

03:31:04but if I can play my song without a license, what motivates me to acquire a license... or to compensate the artist in any way?

03:31:46do we just bag the idea of compensation for recorded music? go with the services model? i.e. pay for concerts?

03:32:06I don't see that happening.

03:32:07you pay because you want the artist to live well and continue making music (the same reason people are paying lilo)

03:32:32(or K5, etc.)

03:32:52K5?

03:33:05is lilo getting more than trivial compensation?

03:33:18this seems to involves a substantial cultural shift.

03:33:29DanC: that's what we're looking for

03:33:37http://www.kuro5hin.org/ which recently raised tens of thousands of dollars for its sysadmin

03:33:50ah, $37,000

03:34:03DanC: completely different cultural activities will be promoted, when only those that inspire their audience to donate can get funding

03:34:32people might even get what they want, rather than what's advertised to them

Journalism Will Survive the Death of Its Institutions by Knight 2007 News Challenge Winner Lisa Williams had such an impact on me that I had to include it when presenting Changes in the Languages of the Web at Web Directions North in Feb 2009, exploring a big picture around digital media, free culture, and the freedom to tinker while giving my perspective on HTML 5.


Then in May I discovered an interesting model in Trelgol Publishing: each work initially has a traditional per-copy price, but once its "world price" is met after so many copies are sold that way, the work is released to the public domain.

And just recently, among the discussion surrouding Diaspora:

This is an approach adopted by some forward-thinking musicians: for example, Jill Sobule funded her last album in the same way, garnering $75,000 in pledges from fans.

I thought the fundable.com pledge/escrow system was pretty nifty; at Christmas time in 2007, I used it to organize support for a friend after a tragedy. I tried to use it again today but I see that fundable closed permanently in October 2009! Kickstart and kapipal seem to be in the same space, but they have somewhat different models. But there's always the basic tip-jar approach:

If you're inspired by what you see around here and you'd like to see more, please support Mad Mode.

Syndicated 2010-06-04 20:22:00 (Updated 2011-05-09 20:27:08) from Dan Connolly

A new firehose to drink from: bioinformatics

After 15 years working on web standards at W3C, the title of my new position is Biomedical Informatics Software Engineer. I know what the Software Engineer part means; I have been doing that since my first job out of school in 1990. But the Biomedical Informatics part I'll have to learn, real quick.


A couple years ago, I started prospecting for some supplemental consulting work; in From KC to MIT and back again , I wrote:

My family is here in KC but my work is at MIT and Silicon Valley and Vancouver and Edinburgh and Beijing... mostly over the Internet, supplemented by travel schedule of about one trip a month. After working remotely for 10 years, I'm interested to mix in a little more local collaboration.

Little did I know that a year later, the recession would catch up to W3C and my position at MIT would be reduced to half time and that supplemental consulting would become essential. The first gig I found was more HTML 5 work, with funding from Adobe. Then something pretty different came up: working with Science Commons, on data integration to support research on autoimmune diseases.

The bulk of the work was just decoding data from the protein databank and such and using semantic web techniques to normalize it; pretty tedious ETL stuff, except that I had to learn all about genetics and the human leukocyte antigen (HLA) while I was at it. I could just barely keep up with the presentations at the NAID bioinformatics summit.

In the last few weeks of the project I got to do a 3D visualization demo.


After this pilot project, we wrote a proposal for follow-on work, and it got positive technical reviews, but   somehow the funding didn't materialize.

But now I had a new keyword to use when looking for work: bioinformatics. That's how I found this new position, just a few miles from my home, in the department of biostatistics at the University of Kansas Medical center. They're putting together an informatics team to round out their qualifications for a Clinical and Translational Science Award (CTSA), and I'm excited to be part of it.

Syndicated 2010-06-02 23:28:00 (Updated 2011-05-09 20:23:40) from Dan Connolly

Spring cleaning my desktop with Ubuntu 10.4 lucid amd64

As a developer, I am careful to rely only on free software so that that my contributions fit within the free software ecosystem, but as a user, I still rely on some proprietary software:

My first approach to running quicken was to try to keep my old ~/.wine configuration in place, but after trying to debug that for a while, I discovered that the spring cleaning approach (blow it away and start over) worked just fine. This stuff all works out-of-the-box:

Syndicated 2010-05-11 19:39:00 (Updated 2010-06-02 20:51:21) from Dan Connolly

Getting over blogging tool analysis paralysis

I'm right there with Joshua when he writes:

I haven't really written anything for this blog in a while.
There are a variety of reasons for this, but I'm generally pretty sensitive to my tools, and I haven't been thrilled with either what I am currently using or what I might use in the future. Do I want to use Wordpress on a virtual machine at some hosting provider? Do I want to write something custom on AppEngine? Or one of a dozen dozen other choices? It makes me want to lie down.

In Feb 2009, I was looking at drupal vs. wordpress and such. I'm writing this in/on google's blogger because:

• I'm tired of the primitive editing support at advogato (especially the way it mangles my paragraphs)
• We can't manage comment spam in our breadcrumbs research group blog.
• I can't manage my own wordpress install (a friendly expert told me that my barn door was open w.r.t. security; fortunately before any unfriendly experts came along)
• Google lets me use my own domain name, for free (wordpress.com charges $10/year). And it encourages me to run ads. I'm still thinking that over.
• Google's data liberation front makes me confident I can get my data back out again.

I'm also using chrome. I mostly got it going for google calendar, which bogs down firefox. App-specific browsers are clearly the way to go... mozilla prism doesn't seem to have a big enough userbase to be polished, where chrome's application tabs show that it takes this idea seriously.

Oh... and years after buying this dual-core amd64 machine, I finally installed 64bit ubuntu. More on that in another episode of madmode...

Syndicated 2010-05-05 20:09:00 (Updated 2010-06-02 20:29:12) from Dan Connolly

On building a Linux box, from December 1995

I'm purging files, and I think the 9505 Beach Hardware folder can go. beach.w3.org was the box I put together for my desktop in May 1995 when I arrived at MIT. It was a PowerSpec from Micro Center. In the folder are my 18 Dec 95 Debian upgrade notes, including gems such as:

mknod 22 0
mount -t iso9660 -oro /dev/cdrom /cdrom

dsselect detected the CDROM!

Also in this folder is a printed article:

• Building the Perfect Box: How To Design Your Linux Workstation
  by Eric S. Raymond
  Linux Journal April 1997

Props to Linux Journal for keeping good archives!