I'm purging files, and I think the 9505 Beach Hardware folder can go. beach.w3.org was the box I put together for my desktop in May 1995 when I arrived at MIT. It was a PowerSpec from Micro Center. In the folder are my 18 Dec 95 Debian upgrade notes, including gems such as:

mknod 22 0
mount -t iso9660 -oro /dev/cdrom /cdrom

dsselect detected the CDROM!

Also in this folder is a printed article:

• Building the Perfect Box: How To Design Your Linux Workstation
  by Eric S. Raymond
  Linux Journal April 1997

Props to Linux Journal for keeping good archives!

In a September item, Martin Kleppmann says:

Scala in 2009 has the place which Python had in 2004.

I bookmarked Scala (the language; not the band ;-) back in June 2007, but I didn't find a good excuse to try it out until Alexandre Bertails, the new W3C webmaster, suggested adding scala to the php/perl/python/java mix that powers w3.org. He gave a great PreparedKata on scala. I have now built a couple little projects using Scala. The experience brings me back to a June 1996 Usenet posting, where I wrote:

Modula-3 was more fun to learn than I had had in years. The precision, simplicity, and discipline employed in the design of the language and libraries is refreshing and results in a system with amazing complexity management characteristics.

I have high hopes for Java. I will miss a few of Modula-3's really novel features. The way interfaces, generics, exceptions, partial revelations, structural typing + brands come together is fantastic. But Java has threads, exceptions, and garbage collection, combined with more hype than C++ ever had.

I'm afraid that the portion of the space of problems for which I might have looked to python and Modula-3 has been covered -- by perl for quick-and-dirty tasks, and by Java for more engineered stuff. And both perl and Java seem more economical than python and Modula-3.

I'm happy to say that I was wrong; python matured quickly enough that I use it for most of the spectrum. The libraries matured quickly enough to allow me to get away from perl. And I'm pretty happy that I avoided Java long enough for scala to come along and fill in the bits of Modula-3 that Java lacks.

The main reason I never did pick up Java is that the main part of my job was project management, i.e. on the manager's schedule, and an hour isn't enough to do any software engineering. It is enough time to write, test, and document some python code! I'm doing more software development these days; working on the UI part of a Science Commons project last summer finally gave me several days in a row to dig in and learn JavaScript development. And I had to interface to a Java API in JMOL, so I dipped my toe in the Java waters using Jython. I got it working, but since I largely depend on doctest mode for emacs and never got jython working there, it's only manually tested.

I can now write, test, and document scala code, though it's about equal parts fun and frustration at this point.

The first frustration was finding that there's nothing like the python tutorial on the scala web site. The tour of scala was very tasty, but didn't teach me enough to read scala code and be confident about what's going on. I tried reading the language spec, but got lost in abstractions (that's one thing Java has over scala; GJS's Java spec is a joy to read). Alexander eventually got me to read the ebook, which is quite good, though not freely available. Shortly after that I discovered the video of Martin Odersky's FODEM talk; I think that one pleasant hour could have substituted for several earlier frustrating hours on the scala web site. And I discovered the O'Reilly scala book; people say it's nowhere near as good, but I'm going to try to migrate to it for reference purposes, since I can more easily share what I find there.

The next frustration I feared was giving up emacs in favor of a modern Java IDE. But the friendly folks in the #scala channel assured me it wasn't necessary:

<DanC>	I'm an emacs addict, but I gather the way to do
scala is with Eclipse
<paulp>	DanC: don't know where you gathered that but I
would bet eclipse user a minority.
<DanC>	oh.

 
 <DanC>	what do you use?
<paulp>	textmate.
<dcsobral>	jEdit here.

I did give up make for simple build tool (sbt); I only miss it a little; sbt emacs integration is pretty raw and next-error gets out of sync about which line to go to (workaround: restart sbt-shell). Flymake looks cool, but I haven't managed to get it working.

Giving up doctest is much harder. I learned to use scalatest, but it's no it's tedious and using the 1.0 version requires using unreleased versions of sbt (which worked fine for me). ScalaCheck is even more bothersome, as it uses level 12 scala type inference magic while I'm only a level 4 apprentice, but at least it rewards you by generating zillions of test cases for you. None of the scala test frameworks are integrated with scaladoc, the documentation framework. Every time I had to fill in a test name or description I'd think "Why is this not integrated with docs? An interpreter and REPL are as much a part of the scala culture as the python culture; surely there's a doctest for scala out there" and go searching. No joy. I did find a couple starts at doctest for Java (they use JavaScript for the REPL; Java itself just doesn't work that way). I eventually got fed up enough to start my own doctest.scala, though it's not feature complete enough to use yet.

"Beautiful is better than ugly." says the Zen of Python, and scala feels pretty elegant. But the next aphorism is "Explicit is better than implicit." Java clearly takes this too far with

FileInputStream x = new FileInputStream(file);

Telling the compiler type type of x once should be enough, and with scala, it is. But scala has lots more magic that, all together, can make it hard to read. The complexity shows up in the compiler diagnostics, which I find misleading more often than not. Scala has parallel namespaces for types and values; it's kinda cute, but consider this diagnostic:

 
 [error]
/home/connolly/projects/rdfsem/src/test/scala/rdfstdtest.scala:137:
not found: value Graph
[error]     val manifest = Graph(WebData.loadRDFXML(args(0)))
[error]                    ^

I sit there pulling my hair out, saying "Graph is imported 10 lines up; are you blind?!?!?!" But what I imported was the type, not the value. The real problem in that line of code is that scala is like java in using a new keyword for instantiating (most) classes, but python habits die hard.

And that's just the beginning when it comes to mystifying compiler diagnostics. Be very afraid of "Missing closing brace `}' assumed here." The missing brace may be very, very far away. The ScalaCheck docs really need a special decoder ring due to its use of higher order magic; check this out:

[error]
/home/connolly/projects/rdfsem/src/test/scala/strquot1.scala:33:
missing parameter type for expanded function ((x0$1) =>
x0$1 match {
[error]   case (s @ (_: String)) =>
dequote(quote(quote(s))).$eq$eq(quote(s))
[error] })
[error]     Prop.forAll((genQuotEsc) {
[error]                              ^

That "case (s @ ..." stuff isn't in my code; the compiler magically conjured it up. I only know from monkey-see-monkey-do reading of the ScalaCheck docs that the right answer is:

    Prop.forAll(genQuotEsc) {

Here the compiler is being sadistically misleading:

[error]
/home/connolly/projects/rdfsem/src/test/scala/rdfstdtest.scala:103:
not enough arguments for method apply: (n:
Int)org.w3.swap.logic.Term in trait LinearSeqLike.
[error] Unspecified value parameter n.
[error] 	println(manifest.each(u, rdf_type, what).mkString())
[error] 	                                              ^

My sin in this case was to break the rules for methods without parentheses.

Many thanks to RSchulz and company in #scala for taking my side in several battles against the compiler's disinformation campaign.

Once that battle is over, life is much more fun. That is, after all, much of the value proposition of statically typed languages, though the global consistency guarantee in the language and build tools comes with a downside that when you change a type, you can't just test a few modules without getting everything in sync.

My debugging tool so far is the trusty println(). When my code hangs, I'm used to hitting ctrl-c and getting a python backtrace. The java runtime, and hence scala runtime, just quits with no backtrace when you hit ctrl-c. Ouch.

When I asked about debugging and profiling tools in #scala, the suggestions I got were about various GUI tools, many of them commercial. I managed to get IDEA with the scala plugin configured to navigate my code, but it took 20x longer than sbt to build, and before I managed to learn to use its debugger, I spotted the bug myself. For profiling, java -Xprof worked just fine for my needs, though jvisualvm is free and packaged by Ubuntu and I did get it to attach to my running code; I'm still stumped about how to get it to tell me which methods are taking the most time, though.

I like the idea that scala is now where python was a few years ago, i.e. that the frustrations that I'm running into are rough edges that will get smoothed out soonish. The cascade that started with scalatest 1.0 requiring using an unreleased version of sbt continued thru using version 2.8.0.Beta1-RC5 of the compiler and libraries. I still love python, but I'm happy to restore an elegant statically typed languge to my toolset after Modula-3 went fallow, especially one that interoperates with the java platform everywhere from android mobile devices to Google App Engine.

tags: programming

Another installment in the to-mac-or-not-to-mac series... I recently replaced my 2004 era G4 powerbook with a MacBook Air. Hulu works a lot better with a modern CPU ;-) I'm hooked on Flash Forward now. And Miro "just worked" to grab some Ted talks for watching on the plane.

The MacBook Air comes with new Apple software too: iLife '09 has face recognition and map integration.

It looks like google's cross-platform tool does face recognition and map integration too: Google Photos Blog: Announcing Picasa 3.5, now with name tags, better geotagging and more. After watching the chromium "everything lives in the cloud" OS videos, it's hardly surprising to see Google talking about photo libraries in their offer of twice the storage for a quarter of the price, i.e. 20 GB for only $5 a year.

Google says most people have less than 10GB of photos; we have the same order of magnitude (~32GB, including videos). How long would it take to upload all that content? It took hours just to copy it across our LAN (details below). I got LAN access to the iPhoto Library working, but it was annoyingly slow.

Then there's music...

A Google music search item reminds me about Lala (hi Anselm!) and Pandora. Unlike photos, the music I listen to is mostly stuff I didn't record, so it makes a lot of sense that it lives in the cloud... if only caching were a *lot* better. I want the iPod wear-it-on-your-arm-while-you-run experience.

I read about mobile phones taking over as everything from watches to media players but watch batteries last years, an ipod shuffle goes several trips on one charge, and my cellphone needs charging every day.

Also, I want the few kilobytes of precious data (playlists, star ratings, and the like) managed as *my* data, separate from the gigabytes of recorded mp3 data. Last.fm goes one way... with scrobbling from iTunes to the cloud. How much would I be willing to pay for a subscription to all my "mix tape" style playlists? Hmm.

And how long before patronage returns as the dominant business model for creative work? Will the music of my kids' formative years be as free as Ted videos?

This past weekend, I copied the family photo library from my wife's laptop (she's the shutterbug) to the linux box in the closet and then to my new macbook air. It's 32GB including videos. I didn't record the time exactly but it seemed to take around 5hours.

Using iPhoto Library Manager, I split it into two albums: the most recent 9 months and everything older. Copying the 9 month segment using rsync over wifi concluded thus:

sent 9195085734 bytes  received 190748 bytes  1379946.95
bytes/sec
total size is 9193422268  speedup is 1.00

That's 8.5GB in just under 2 hours, which suggests 5hrs is in the ballpark.

music stats...

sent 3877121671 bytes  received 36170 bytes  2818726.17
bytes/sec
total size is 3876532701  speedup is 1.00

star ratings in iTunes

I went back to the most recent (2008-03) of my calendar sync items in the DIG breadcrumbs research blog and got hipwsgi.py from palmagent fired up, only to get "Connection refused" from pimapi.prod1.dngr.net.

Uh-oh.

I thought I could write off the sidekick altogether at that point, but:

1. Organizing weekend todo lists works with the sidekick in a way that I haven't managed to duplicate: lists on paper don't sort themselves by priority and due date; Google calendar tasks don't sync with the sidekick (nor with any usable android app that I could find).

2. How do I call my brother from the car? Using a mobile phone without my contacts would be like using the Web without DNS.

3. Android's crummy appointment notification reminded me how much I rely on a gizmo to beep when I'm supposed to stop coding and go to my appointment with the Doctor. Delegating this to a gizmo goes back to ~1996 when I first got a Psion PDA. (see some python code for psion files)

I'm not sure how I'm going to muddle thru this mix of google calendar/contacts stuff and sidekick phone... maybe I can use SMS reminders for calendar stuff, but you never know how long those things are going to take to be delivered; T-Mobile seems to deliver them 13 hours later in some cases.

For now, I'm going to pickle some state...

#swig notes

old calendar notes/links, circa 1999/2000

palmagent code: r423:4a5a8b2d237c 2009-05-01)

repository of sidekick data from palmagent/hipwsgi.py: 32:31a84807d214 2009-02-26

another repository of my PIM data: 596:6faa7311f865 2009-04-2, 595:b20e1f7fa468 2008-09-10

I've been a happy sidekick user since December 2002. In fact, what really got me interested in the android/G1 was that Andy Rubin, the danger/sidekick lead designer, was working on it.

My first few minutes with the G1 were lots of fun: google street view with the accelerometer blew me away and I had downloaded a dozen apps in no time.

But while technically all these apps can do everything all at the same time, in practice, the experience sucks. When I have a thought to capture, as Nielsen's research shows, if I don't get .1 second response time from the home button, I become conscious of the mechanics, and after 1 second, I lose my train of thought.

Other critical day-to-day features such as "get my attention when I have an appointment or a text message comes in" don't work either. The G1 gives one little beep and puts an icon in the notification bar and then goes idle. If I happen to be in noisy traffic at the time, I lose. The sidekick continues to beep every 2 minutes, so that when I eventually get somewhere quiet, I'll notice.

And speaking of quiet, sound profile management on the G1 sucks. To put the phone in silent mode, you can hold down the red/end button until a menu appears. In big letters, it says "Silent Mode"; then it tiny letters under that, it says "sound is: on". Details, details, people!

Then, when you flip it to "sound is: off", it goes silent, but it doesn't vibrate. To put it in vibrate mode, you use the button on the side that controls the ringer volume, but you have to look at the screen to see when you've held it long enough. There's no one reliable gesture sequence for managing sound profiles.

Also, I'm forever forgetting to take the G1 back *out* of silent mode. I'm spoiled by the sidekick's scheduled sound profiles; every night at 11pm, it goes into "alarm clock" profile (where appointments that I set up ring loudly but incoming messages from others don't) and every morning at 8am, it goes back to normal mode. So even if I forget to take it out of silent mode, it's all set to go the next morning.

There's an award-winning 3rd party app (locale) for managing not just sound profiles but all sorts of other stuff like wifi and gps power-saving settings... and it's configurable not just by time, but also by GPS location, nearby wifi stations, and such. But... it doesn't work. That is: I couldn't recreate the simple "be quiet at night" configuration from the sidekick. Plus, it seemed to gunk up the performance of the gizmo.

As to the roach motel, no, I don't trust t-mobile/danger/Microsoft to manage my data; I keep my own copy using some homebrew software that uses their XMLRPC interface (in fact, I keep multiple copies sync'd with hg/mercurial).

The t-mobile web interface isn't nearly as nice as google's; that's probably the main thing I'll miss as I switch back to the sidekick. That and google maps (though the GPS stopped working about the 3rd time I dropped the G1).

The backlight on the screen was intermittent for a while, but power-cycling it would bring it back. Then, with the recent software update, the screen is dark all the time. So it's a choice between replacing the G1 and going back to the sidekick. (I'm keeping an eye on the palm pre and the iPhone is ubiquitous, but I extended my t-mobile contract by two years when I bought the G1 in Feb.)

Well, I just called T-Mobile customer service and asked them to switch me to the sidekick data plan. I guess we'll see how long it lasts.

see also: The Forgotten Sidekick

tags: mobile, android

p.s. older WearableGizmo notes suffer from in-progress get-out-of-Zope migration.