Older blog entries for connolly (starting at number 94)

30 Apr 2011 (updated 1 May 2011 at 04:24 UTC) »

Trying to replace delicious, pinboard.in, and catch with diigo

I keep trying out one more cloud based task/time/knowledge management tool, hoping it will replace several of my too many others. While browsing around the Chrome store looking for tools that sync with android, I discovered diigo. The highlight feature is really slick! I've been hoping for that feature as far back as the Amaya papers and talks from 2000. Plus, it does bookmarking and note taking. But it's not as smooth as I'd like. I wonder if that's inherent in the attempt to do so many things.

A pleasant surprise from diigo: the chrome search bar

Chrome merged the address bar and the search field a while ago. The diigo chrome extension notifies you when you search for things that match items in your library, so you don't have to build a new habit.

Why diigo hasn't replaced pinboard for bookmarking, twitter archiving

The original delicous bookmarklet clearly hit the sweet spot for bookmarking:

  1. Hitting the bookmarklet brings up a little pop-up with the URL and title filled in for you
  2. add your own note... maybe a particularly interesting quote/excerpt (optional)
  3. add some tags
  4. Hit enter/save and you're back to your web page, with the pleasant feeling that your bookmark is stored safely in the cloud (and you can get it back via their export service and/or API)

There were some lightweight features that improved the experience: auto-complete of tags and auto-suggested tags from the crowd. Then the features started getting heavy, going beyond the critical response times, and on a tip from Gerald, I started migrating my delicious bookmarks to pinboard.in. (This was long before "the vice president of bad decisions at yahoo" threw in the towel.)

The diigo bookmarklet has two critical problems:

  • It takes over the whole page (and takes too much time doing so). So you can't consult the page as you add your notes.
  • When you hit save, it takes you to your library rather than back to the page you were on.

It was the speed of pinboard that convinced me to switch from delicious, not so much the "anti-social" aspects; I did enjoy the collaborative aspects of delicious, until they went overboard and made it too painful to search my own bookmarks. I was surprised to see so much of my community using twitter for link sharing: how do they ever find the bookmarks they made?! Twitter has the attention span of a gnat; it has no interest in helping you find a bookmark you made 2 years ago. Pinboard solved that problem by adding comprehensive twitter archiving to their snappy search offering.

Diigo has a twitter archive feature, but

  • It archives only favorites, not tweets I wrote, unless pay a monthly fee. (Pinboard isn't free, but the fee is one time.)
  • It loses critical context, i.e. who wrote the tweet.
  • It lumps tweets in with notes I wrote in places like their Quick Notes chrome application

That brings me to the goal of using diigo for task management.

Why Diigo hasn't replaced Catch for gtd-style collecting

Catch supports gtd-style collecting and processing really well:

  1. With their android widget or shortcut, touch to start adding a note.
  2. Type a few words to capture what's on my mind... or more often: hit the speech input button and say a few words.
  3. Hit save, knowing catch will sync with the cloud momentarily.

I do most of my processing via catch's web interface, when I have the full bandwidth of a big screen, keyboard, and fast network. But sometimes when I have some time to kill, I use the catch android app to process notes.

I hope the diigo Powernotes android app gets there. Both catch and diigo let me log in using my google apps accounts, but:

  • Early releases required manual sync, which completely defeated the purpose of getting things off my mind, since I had to think about whether I had sync'd or not. I'm glad that's fixed.
  • Catch has "pin note to homescreen," which is handy for journaling; PowerNote doesn't seem to have anything like that. "Pin list to home screen" would be handy.
  • Saving a note without a title fails silently. This is particularly painful since the speech-to-text note taking feature defaults to an empty title. Throwing away the knowledge I just entrusted to it is pretty much the unforgivable sin for a knowledge management app. The feedback feature is really simple and the developer acknowledged my feedback right away, though, so perhaps I'll give it another chance. 
  • I can't find an easy way to list all (and only) the thoughts I collected. It supports filing notes into lists, and one of the options is "Recent notes," but that's a tease: there is no "Recent notes" when I go to view my lists. Diigo bookmarking supports the "read later" bit a la pinboard, but I don't see how to set that bit on notes. It would be handy to have a unified "read later" collection of notes/bookmarks/highlights.

Diigo for shopping? What was I thinking?

I sure wish Amazon helped me record why I'm adding to my wishlist, e.g. who recommended it, which features or review comments I'm particularly interested in. I can annotate items if I switch to viewing the whole list, but the first thing Amazon does after I hit "add to wishlist" is distract me from recording what's on my mind with offers for other products. So I did a little research on home theater systems using diigo. But while shopping does involve research, there's really a lot more to it, and Amazon is a huge machine finely tuned to help with the whole process. Amazon's universal wishlist button helps some. Besides, as we learn from gtd, the most important thing to do after capturing a thought is to put it in context where you will next act on it. And for online shopping, that place is Amazon more often than not.

Diigo community and tools

The diigo community and development team appeals to the hacker, the researcher, and the closet-librarian in me. I haven't found many familiar names/faces in the diigo community yet. The business model (freemium, with a focus on the education market) seems sensible to me, but I don't have much confidence in my ability to pick viable web businesses. (I've been involved in the web pretty much since it started; I wonder if I'd be ahead or behind if I'd invested in the web businesses I liked when I learned about them...)  With a new owner for delicious, it may be time to take another look. The delicious crowd is large enough to display some wisdom in, for example, finding interesting new python programming resources. And I once discovered that a colleague subscribed to my family movie bookmarks.

Diigo says they support the same export format as delicious, but I don't see how I can get all my data back that way, since delicious has no concept of highlighting nor lists. I see a mention of annotations in the diigo API; perhaps all the structure is captured there.

Syndicated 2011-04-30 17:41:00 (Updated 2011-05-01 03:38:32) from Dan Connolly

16 Apr 2011 (updated 28 Apr 2011 at 21:17 UTC) »

Closing music sharing loops with Amazon Cloud Player and iTunes home sharing

I enjoy the music that my wife buys, but when I shop, I seem to get overwhelmed before I get as far as the "place your order" button. It's just like in college, when I used pore over the used cassette section, only to realize it was time for my next class before I decided what to buy. Once she buys the music, I'd like to have it when I commute to work or go to the gym.

I pulled my hair out for a while trying to get my Ubuntu linux box to fool her Mac into using it as a time machine server. I couldn't even get apple file sharing working read-write (for saving playlists and metadata). When Apple came out with home sharing, I pretty much threw in the towel and resigned myself to pulling out my MacBook Air whenever I wanted to deal with the world of popular music. But mostly I just didn't bother.

Then, after she had been limping along for a while on external keyboards and mice for her MacBook due to the internal ones having broken down (and the price to service them being out of reach), we swapped out her MacBook for my Macbook Air.

This gave me a fresh shot at organizing the family music collection.

My Ubuntu box has big, cheap disks. I use LVM2 to manage three volumes, which vary based on availability and confidentiality constraints:

  • commons - for creative commons and open source stuff. No constraints; I don't have to worry about who sees it or copies it, and it remains available if I delete my copy.
  • mass-media - for popular music, DVR storage, etc. I don't have license to share this stuff freely, but availability isn't a big deal: if I lose my copy, I can easily get another, though perhaps not for free.
  • family-media - for photos, records, etc. Much of it is confidential and original.

At some point since my earlier frustrations, Ubuntu and Mac OS X have decided to get along; file sharing now Just Works. So I rsync'd all her music to the mass media volume shared it. Unlike the nightmare of merging iPhoto libraries, iTunes has an option to view "only items not in my library". Yay!

The iTunes collection includes some original stuff, such as piano recital recordings and garage band compositions. I'm dealing with my long-standing angst about that by using Musicbrainz Picard to automatically re-tag everything and then move the stuff that's outside the wisdom of crowds to the family-media volume. The collection also includes stuff that my wife imported for photo montage projects; Frank Sinatra and polka music were of great sentimental (or humor) value to the client, but I don't want it in my "shuffle all" mix on the way to work. I'm not sure how to deal with that, yet.

Reducing the redundancy feels good to the closet librarian in me, but... what if the disk goes kerflewey? It's all replaceable, but even the potential of buying it or ripping it again leads me to the aforementioned paralysis/overwhelm.

Enter Amazon Cloud Drive and Amazon Cloud Player.

Not only does the android app eliminate the hassle of firing up a Mac to use doubletwist to sync iTunes to my android phone, but cloud storage provides backup of all our popular music... or at least: all the music that I cherish enough to bother uploading to Amazon.

Syndicated 2011-04-16 03:21:00 (Updated 2011-04-28 20:26:32) from Dan Connolly

Google smiles on KC

The Topeka gag turned out to be not so far off after all: Ultra high-speed broadband is coming to Kansas City, Kansas. I work in the informatics division of KU Med Center, which gets special mention:

We’ll be working closely with local organizations including the Kauffman FoundationKCNext and the University of Kansas Medical Center to help develop the gigabit applications of the future.
While I live in the KC metro area, I live in one of the 'burbs, not KCK, so this won't affect me at home much. But it should be a huge shot in the arm to the KC startup community.

Syndicated 2011-03-31 13:38:00 (Updated 2011-03-31 13:38:46) from Dan Connolly

Adding server operations, DBA to my bag of tricks

I've always been a programmer. Then I learned enough about shipping software products at Convex and Dazel, and then global collaboration at W3C. I have always respected the people who keep the servers runningthe W3C systems team rocks!and now I'm learning a bit of that stuff too.

It's certainly happier than the feedback I've been getting while trying to learn about Oracle administration. (Frank isn't the name this guy used, but he was very frank.)

if I make a cold backup on AIX (big-endian) and I try to restore it on linux (little endian), am I more likely to win or lose?
you need to use rman or [...]
but you guys really should hire a dba
you could afford oracle, and a server for aix, and now a new linux server
you can afford a dba for a few hours
Or else, you could totally screw stuff up, especially the configuration and tuning of the linux DB
most admins now are giant #$%@# idiots who dont know that sqlplus exists. 
or they are "developers"  which their company put them in charge of the database
and are giant clueless idiots

Syndicated 2011-03-15 16:54:00 (Updated 2011-03-15 16:54:13) from Dan Connolly

2 Feb 2011 (updated 23 Sep 2011 at 09:07 UTC) »

Cleaning up my contacts with batch edits in the Google Contacts Data API

The Blizzard of Oz snow day gave me a chance to get my taxes done yesterday, and today I used the time to scratch a long-standing itch to clean up my contacts, which have been a bit of a mess since I merged two sources into one.

When I got my first android phone, a G1, I used the
Google Contacts Data API to write hipg.py for migrating my sidekick/hiptop contacts to gmail contacts. Then I grew disappointed with the G1 and went back to the sidekick. And then Microsoft bought Danger and lost everybody's sidekick contacts for a week or so. They restored some of the data, but all my street addresses were gone. I had back-ups, of course; I know better than to completely trust any cloud provider. But I hadn't written code to restore from my back-ups. So I lived without street addresses in my phone for a while.

Then I got another android phone, a Samsung Galaxy S Vibrant. I migrated the sidekick contacts again, and then, to get the street addresses, I imported the contacts from the G1 days. Google's merge tools cleaned things up to a tolerable level, but they left 2 issues:

  1. Duplicate phone numbers like:
  • +1-212-555-1212
  • 212-555-1212
  • Duplicated notes like:

    like they should be

  • I fixed a few of these by hand, but knowing that I could get the computer to do them all in batch has been itching at me for a while.

    It took just a few hours and a couple hundred lines of python to scratch that itch today with contact_fix.py.

    475:c87196981537 2011-02-02 fix all_contacts (self.)
    474:80739018dbfa 2011-02-02 - make group optional; default to all
    473:61412465cfed 2011-02-02 - fix duplicate phone numbers as well as doubled notes
    472:01c2becd5cfb 2011-02-02 batch edit works/tested in one case
    471:e7840a73836d 2011-02-02 batch edit operation (tested non-destructively)
    470:e4e531aae635 2011-02-02 run fix_note() on data from the server
    469:2ec84b8d0363 2011-02-02 - print names of contacts in a group
    468:b919239097e7 2011-02-02 contact_fix.py can login and get contact data

    I'm also working on archiving my contacts using hCard. More on that soon, I hope.

    Syndicated 2011-02-02 22:19:00 (Updated 2011-09-23 08:27:00) from Dan Connolly

    13 Jan 2011 (updated 7 Jul 2011 at 19:08 UTC) »

    A new year brings a new low in media and culture

    The new year brings a reminder that we have turned our culture over to Viacomm, AOL/TW, Disney, Clear Channel, and News Corp, with disastrous results:

    Do you know what entered the public domain in the US on Jan 1 2011? Nothing.
    — Rodney Ramsey, citing Duke Law on Public Domain Day 2011
    In a 1999 piece, How to Stop Worrying and Learn to Love the Internet, Douglas Adams said:
    ... during this century we have for the first time been dominated by non-interactive forms of entertainment: cinema, radio, recorded music and television. Before they came along all entertainment was interactive: theatre, music, sport – the performers and audience were there together, and even a respectfully silent audience exerted a powerful shaping presence on the unfolding of whatever drama they were there for. We didn’t need a special word for interactivity in the same way that we don’t (yet) need a special word for people with only one head.

    I expect that history will show ‘normal’ mainstream twentieth century media to be the aberration in all this.
    Since I was born in the late '60s, that seemed perfectly reasonable to me, as he went on to explain:
    you would think we would learn the way these things work, which is this:
    1. everything that’s already in the world when you’re born is just normal;
    2. anything that gets invented between then and before you turn thirty is incredibly exciting and creative and with any luck you can make a career out of it;
    3. anything that gets invented after you’re thirty is against the natural order of things and the beginning of the end of civilisation as we know it until it’s been around for about ten years when it gradually turns out to be alright really.
    Apply this list to movies, rock music, word processors and mobile phones to work out how old you are.
    Indeed, I have spent the bulk of my career leading development of open web standards and writing a little open source software; that is: on a shift to participatory media and free culture. But I spend plenty of leisure time on movies, handing ammunition to the old guard with every ticket.

    Games are a little less passive than movies, but I wonder where that leads us. Why We Haven’t Met Any Aliens is food for thought:
    ... if extraterrestrial intelligence is common, why haven’t we met any bright aliens yet? This conundrum became known as Fermi’s Paradox.

    I suggest a different, even darker solution to the Paradox. Basically, I think the aliens don’t blow themselves up; they just get addicted to computer games. They forget to send radio signals or colonize space because they’re too busy with runaway consumerism and virtual-reality narcissism. They don’t need Sentinels to enslave them in a Matrix; they do it to themselves, just as we are doing today.
    I just finished listening to the audiobook version of Lessig's Free Culture as I commute. The tl;dr version, from his OSCON 2002 talk, is:
    • Creativity and innovation always builds on the past.
    • The past always tries to control the creativity that builds upon it.
    • Free societies enable the future by limiting this power of the past.
    • Ours is less and less a free society.
    If you haven't at least watched the slides+audio of that talk, stop reading this now and do it. Then let's hope that Jan 1 2011 is the low point in this media aberration.

    I hesitate to call it a resolution, but maybe this year I'll make some progress toward the example that Aaron Swartz set back when the Creative Commons licenses were launched in December 2002:
    when I go to a movie, I donate money in the amount I spent to the EFF.
    Almost a decade later, I see they're still hard at work, fighting the absurdity Lessig notes in chapter 12: Harms:
    The four students who were threatened by the RIAA (Jesse Jordan of chapter 3 was just one) were threatened with a $98 billion lawsuit for building search engines that permitted songs to be copied. Yet WorldCom - which defrauded investors of $11 billion, resulting in a loss to investors in market capitalization of over $200 billion - received a fine of a mere $750 million. And under legislation being pushed in Congress right now, a doctor who negligently removes the wrong leg in an operation would be liable for no more than $250,000 in damages for pain and suffering. Can common sense recognize the absurdity in a world where the maximum fine for downloading two songs off the Internet is more than the fine for a doctor's negligently butchering a patient?

    Syndicated 2011-01-13 20:39:00 (Updated 2011-07-07 19:05:46) from Dan Connolly

    21 Nov 2010 (updated 4 Jun 2011 at 16:07 UTC) »

    "Lord of War" and the land of the ...

    Lord of War (2005) came to the top of my Netflix queue this week; it has been on my various to-do lists since February 2008 when my brother-in-law recommended it to me during my trip to New York. It's not the feel-good-movie-of-the-year, but it's got some meaty food for thought:
    The film was officially endorsed by the human rights group Amnesty International for highlighting the arms trafficking by the international arms industry.
    Lord of War - Wikipedia, the free encyclopedia

    - Weapons trading, civil wars, guerilla warfare, cold wars, the end of the Cold War, redistribution of wealth, the CIA, peace, Africa, AIDS, politics, Pacifism, chaos and anarchy, greed, immigration, the Russian mafia, conflict diamonds, massacres, prostitution, war, bureaucracy, arms embargoes, drug abuse and addiction.

    - Greed and power corrupt. Violence is in our nature.
    Lord of War [2005] [R] - 7.8.10 Kids in Mind
    Nicolas Cage once again plays the amoral main character and for most of the movie, we see through his rationalizations and we can see what's coming to him. We're right there with his wife when she says that while she failed at a lot of things, she'll never go so low as he has.

    But then-zing!-we learn that we, as U.S. citizens, are party to what he does, as it's a byproduct of U.S. foreign policy.

    And it's not just the U.S.:
    A brief postscript notes that, while private arms dealers do conduct a lot of business, the five largest arms exporters – the United States, the United Kingdom, Russia, France, and China – are also the five permanent members of the United Nations Security Council.
    Lord of War - Wikipedia, the free encyclopedia
    If that's not bad enough, there's more to see. Inside Job, called An extraordinary documentary about influence and government by Lawrence Lessig, is now playing in KC.

    Syndicated 2010-11-21 04:32:00 (Updated 2011-06-04 15:33:29) from Dan Connolly

    18 Nov 2010 (updated 18 Apr 2011 at 20:12 UTC) »

    Another meeting just snuck up on me while coding in my cube

    My new job is much more on the maker's schedule than the manager's schedule. Work at W3C basically consisted of preparing for and participating in several meetings a day with people all over the planet. Now I'm mostly coding away in my cube, and the little reminder icon at the top of my screen is not enough to break me out of it. (I'm trained to be interrupted by the buzz of my mobile phone, but sync between the enterprise calendar and android unreliable. Sigh.) Having people physically grab me means I don't miss the meeting, but it doesn't give me a chance to properly context switch.

    I don't use a laptop (mostly by choice) so access to the Web depends on either pre-filling a paper cache (printing stuff out) or using the pc/projector in the meeting room. The latter is often more disruptive to the meeting than it's worth.

    Fortunately, somebody else had printed the web page that I had prepared for today's meeting. And the group leader uses a laptop, so we had two copies.

    Part of my preparation for the meeting was to question whether we needed it at all, having recently read Tantek's thoughts on meetings. I thought the technical issues were pretty straightforward and we should just let the developer get on with it and deal with problems as they came up. But it turns out that some of my understanding of the requirements was wrong, and even in the parts that were right, brainstorming led to solutions that were better than the approach I expected development to take.

    Meeting discipline is important. My approach to the risks was finely honed over a decade of remote work, but now that we can get together in the same room without getting on airplanes, I'm learning it all over again.

    Syndicated 2010-11-18 18:47:00 (Updated 2011-04-18 20:06:36) from Dan Connolly

    17 Nov 2010 (updated 17 Nov 2010 at 15:16 UTC) »

    A VoIP puzzle

    Our home phone service is provided by a VoIP provider, ViaTalk. I have a VoIP client, sipdroid, on my smart phone. When the phone rings and caller-id shows a call I want to take, I'd like to take it on my smart phone. Is this too much to ask?

    Evidently so: when I have my smart phone register to answer calls on the home line, incoming calls no longer ring the land-lines in the house. SIP is complex enough to make configuring softphones a nightmare; for all this complexity, it can't register two devices for the same phone line at the same time? Or is the limitation somewhere else?

    p.s. If anyone knows how to solve this puzzle, I'd also like to forward caller ID to my phone as instant messages; I'm happy to write a little SIP to XMPP proxy in python, given the necessary clues about the protocol.

    Syndicated 2010-11-17 05:39:00 (Updated 2010-11-17 14:30:06) from Dan Connolly

    chorded keyboard for android, anyone?

    I don't miss a physical keyboard when I want to text "Did you feed the dogs?" messages. Swype and voice input work better.

    But anything much longer than that involves editing, which is really hard to do without chorded key combinations such as shift-select with arrow keys and mod+x/c/v/z for cut/copy/paste/undo.

    The web browser and photo browser make good use of multi-touch. Why isn't there a keyboard alternative that uses it for chorded input?

    Syndicated 2010-10-08 00:33:00 (Updated 2010-10-08 00:33:51) from Dan Connolly

    85 older entries...

    New Advogato Features

    New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

    Keep up with the latest Advogato features by reading the Advogato status blog.

    If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!