Older blog entries for mbrubeck (starting at number 109)

5 Mar 2009 (updated 5 Mar 2009 at 22:04 UTC) »

Async Map and Fold in JavaScript

My latest experiment is an implementation of asynchronous/parallel "map" (and other array functions) in JavaScript and Oni. Oni is a "structured concurrency language" embedded in JavaScript. For my source code and commentary, see oni-map at GitHub. You can leave comments at GitHub too.

All this time reading Real World Haskell must really have warped my brain if a simple testing framework for a JavaScript program has got me thinking about higher-order functions and concurrency semantics...

Bash Completion for the Google Android SDK

If you're developing for Android, try out the bash script I wrote to add tab-completion for the "adb" command-line tool.

MapReduce with Parallel Make

Ted Gould’s post about transcoding music files using Make got me thinking about combining Make and RecordStream into a barebones MapReduce framework. As a demonstration, I wrote a partial solution to Tim Bray’s Wide Finder challenge, to calculate hit counts from web server logs.

More details (and code).

24 Nov 2008 (updated 24 Nov 2008 at 23:08 UTC) »

The Timeless Way of Building: Diagnosis and Repair

Architect Christopher Alexander's A Pattern Language is well known among programmers as the inspiration for the Design Patterns school of thought, but for some reason his other books are less widely read. My mother and father are both architects—"real" ones, not "software architects." :) My own amateur interest in the subject led me to read several of Alexander's books over the past couple of years.

The Timeless Way of Building is the first volume of an open-ended series. (A Pattern Language is the second.) It is in this book that Alexander first lays out the idea of a pattern language. More importantly, he explains an entire theory and practice of design and construction, of which the pattern language is only one piece.

Another piece of the Timeless Way is the practice of diagnosis and repair. Rather than a detailed plan of future projects, Alexander prefers to maintain a diagnosis of current problems. Each new project must be targeted at fixing these existing problem in the system: "At every moment we use the defects of the present state as the starting point for the definition of the new state" (p. 485). Alexander's goal is to create a type of spontaneous growth and order that is possible when change is incremental and decision-making authority is decentralized.

Alexander was hired to produce a new type of master plan for the University of Oregon campus. The result was published as the third volume in the "Timeless Way" series: The Oregon Experiment. In it, he writes again about diagnosis and repair:

It is still not clear where global order will come from, without a master plan.… How is the problem solved in an organism? Essentially, the problem is solved by a process of diagnosis and local repair.… The master plan tells us what is right for the future. The diagnosis tells us what is wrong for the present. (p. 146–148)

In software, the same problems and conclusions have led to "agile" processes, which emphasize incremental change and flexible planning. But I haven't seen any software companies adopt a consistent practice of "diagnosis"—keeping track of what's wrong or unsatisfactory with the current system. This extends beyond defect tracking; it should include a constantly updated list of code that, although it works, is hard to understand, difficult to change, or otherwise likely to cause problems. (This should be easy to generate—every programmer knows certain parts of their code base that they dread working on for one reason or another.) This knowledge can then guide new development to include "repair" and address such problems before they become defects. I've applied this on a small scale to my own work, and now I wish I had used it at Amazon, where my team spent much more time maintaining and improving old code than writing new code.

Flip Video Ultra on Linux

My Flip video camera arrived yesterday, and I'm quite happy with it. It does everything it's supposed to: starts up fast, records and plays back through an astoundingly simple interface, and mounts as a USB mass storage device. The videos play back perfectly on my Ubuntu 7.10 system, and Kino can import and edit them if ffmpeg is installed.

I did have some problems using the camera with a Windows XP laptop. The Windows program stored on the camera failed to start up, saying that "E:\SYSTEM\VIEWER\PD\settings\pathsInfo.txt does not contain a valid list." The firmware updater from the Flip web site failed with a similar error. Finally I found the Flip Video Program installer, which fixed both of those problems. (It took me a while to figure this out, so I'm documenting it here in case other Flip owners have the same problem.)

26 Nov 2007 (updated 26 Nov 2007 at 16:43 UTC) »

PDF on Kindle

I've seen plenty of good commentary on the Amazon Kindle. Robert Love's hands-on review and follow-up cover both Linux-hacker and regular customer views. But there's one question that I haven't seen answered elsewhere: Why doesn't the Kindle support PDF natively?

I think the answer is simple: The Kindle's 800×600 e-ink screen has too few pixels to render a typical PDF file readably, and its refresh rate is too slow for comfortable scrolling. The only practical solution for most PDFs is to extract the text and convert it to a reflowable format like HTML (which Kindle users can do with Amazon's $0.10 wireless service or free PC-based service). The US$700 iLiad (with 64% more pixels) is much better for PDF reading, but will still run into trouble with some files. It seems that the current generation of ebook readers comes with a decision—cheap, e-ink, or PDF: choose any two.

The refresh problem is one reason I'm not sold on e-ink. I'm curious to compare the Kindle screen to my XO laptop, which seems like an interesting compromise between normal LCD and e-ink displays.

Disclaimer: I work for Amazon, but I don't know anything about the Kindle except what I learned from the user's guide and from playing with one at the office.

27 Aug 2007 (updated 28 Aug 2007 at 15:16 UTC) »

vim-addons

Joey wonders why vim addons are not enabled by default using the new Debian infrastructure.

For single-user systems, Joey is right: enabling addons by default would be best. But for multi-user systems, I think disabled-by-default is better. Otherwise, installing an add-on would change vim's behavior unexpectedly for all users. (Unlike a daemon, which runs once per system; a vim addon is run by each user, every time they start vim.) This is especially bad because vim users tend to have carefully personalized configurations.

Should the defaults cater to multi-user systems at the expense of single-user ones? Probably not, since admins of large multi-user systems are generally willing and able to figure out how to change their configurations, while many single-user machine owners need something that "just works". Perhaps the default configuration should load all installed addons, but local admins should be able to easily change this to load only addons enabled by the current user.

Problems for shared desktop computers.

I share a home desktop computer with my wife. Our computer use is probably pretty typical of a home computer shared between a small number of users, where only one user is logged in at a time, and users have some shared files and some private files.

I've found many things that don't work as well as they could in this situation. We use Gnome on Debian GNU/Linux, but most other desktops have similar problems. I know fixes or workarounds for most of the problems below, but novice users would probably have a hard time finding them.

Some of the problems below are relatively easy to fix. Most already have bugs filed in the appropriate places, and some already have patches available. A few problems may require hard work or fundamental design changes to solve. Here is the complete list of problems:

User Switching:

  • If I select my name from FUSA while I'm not already logged in, GDM appears and I have to type my username. (The username should be filled in automatically.)
  • When GDM is started by FUSA, the X server has slightly settings, causing the fonts on the login screen to be a different size than my normal settings.
  • When I log out, I am taken either to a login screen or to another user's session, depending on the order in which users originally logged in. This is arbitrary and unpredictable.
  • If I switch to another user, my screen is locked. Later, if the computer is displaying a login screen and I type my username and password, I need to type my password a second time to unlock the screen.
  • When I shut down the computer, it does not warn me if other users are still logged in.

Filesystem:

  • By default, users can read all of each other's files, and write to none of each other's files (umask 0002). This is a reasonably sensible default, but most users on shared systems will want to change this to some degree.
  • It's very difficult for a user to change which other users can read which of her files. (Some strategies include making some or all files world writeable; adding the other users to her primary group; creating a new shared group and a share setgid directory owned by that group; changing her umask; and/or manually changing permissions of individual files.)
  • There is no good location by default for shared files. Various programs complain (correctly) about insecurity if my home directory is group-writeable, so I must create a world- or group-writeable (and setgid) directory elsewhere.
  • Copying from other locations, or using programs that do not respect umask, can cause users to inadvertently put non-group-writeable files in shared setgid directories. Other users of the shared directory have no easy way to fix this.
  • Some programs modify permissions or groups of files when overwriting them. (Usually this happens if the program moves the original file to a backup location, then writes a new version using the default settings instead of the original file's settings.)

Application data:

  • Applications that maintain a database of files (e.g. photo managers, music players) must be updated for each user when one user adds new files to a shared directory. For example, when I rip a CD to our shared music directory, my wife can't see the songs in her music player until she adds the new files to her database.
  • Applications that store their data in hard-coded locations in user home directories (e.g. Tomboy, Miro) can't share files between users. There is no setting or permission that will let my wife open Tomboy and view my Tomboy notes, for example.

Installing software:

  • When I install software using the system admin tools, it changes the application menu for all other users too.
25 Jul 2007 (updated 26 Jul 2007 at 02:24 UTC) »

Retiring from Debian

In response to the inactive developer ping, I have decided to retire from the Debian project. Between my job and the new baby, my time for hacking on other projects is nonexistent, or possibly negative. I have already passed Audacity maintainership to the debian-multimedia team.

I still use and like Debian, and I hope someday I'll return from "emeritus" status to become an active developer again.

Cyclone

While browsing through some very old backup files, I rediscovered one of my first real software projects, an IRC client for BeOS. This was a neat collaborative effort, but we weren't really on the ball with project hosting or version control. This is from a file I wrote documenting what was left of the project after it stopped:

Cyclone was started in spring 1998 by a group of developers who hung out in the #bedev channel on EFNet. The original Cyclone team included Jason Gosnell (Avatar), Jeff Hamilton (Pyrus), Matt Lewinski (mattl), John Wiggins (prok), and Matt Brubeck (mgb), with contributions from several other #bedev regulars.

The Cyclone project produced a small, elegant IRC client. Unfortunately, the team dissolved as several members moved around the country to go to new schools or jobs at the end of summer 1998. Along with the loss of some important pieces of source code, this prevented Cyclone from ever seeing a public release.

I also found some poetry I wrote in middle school, but it's compressed using a version of StuffIt that does not seem to be compatible with any open source tools. This is probably a good thing.

Check out this black and white digital back for medium-format cameras. Why aren't there any dedicated black-and-white sensors available in compact digicams or DSLRs? The sensor would have much more light to work with than a color sensor with a Bayer filter. It would also be able to use colored filters without throwing away so much information.

It would be a niche product for sure, but it would be perfect in something like the Ricoh GR-D or the Sigma DP1. The people buying those cameras are already sacrificing flexibility (both have fixed prime lenses), and many of them are street photographers and shoot primarily black and white already. Wouldn't a dedicated black and white sensor improve dynamic range and sensitivity for available-light shooting?

Add this to my DMD wishlist. Not that I could actually afford it if it existed.

100 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!