Older blog entries for cananian (starting at number 80)

How does the iPad "use the iPhone's GPS"?

A few months ago, a number of stories came out covering the iPad's remarkable-seeming ability to share the GPS of a tethered iPhone. Apple's latest location database FAQ confirms the suspicions I voiced at the time: there's no actual GPS sharing involved. Instead, Apple is using the simultaneous GPS and Wifi radios on your iPhone to "crowd-source" what I'll call a "skyhook" database (after the first company to publicly use the technique). This correlates Wifi base station identifiers with their GPS locations in real time -- including (most likely) the real time location of the "base station" created by the iPhone when it is in tethering mode. All nearby Apple devices use this database to compute their location (based on all visible wifi base stations). Since the nearby device sees the iPhone's "base station" and the iPhone is busily updating the position of that "base station" in real time (along with all the other base stations the iPhone can see), the iPad (lacking a GPS of its own) gains the apparent magical ability to compute a very accurate position for itself.

The real interesting part of this story involves user consent and privacy—do iPhone users generally know that their devices are registering their location in Apple's database in real time whenever tethering is turned on? Any device which can query Apple's location database for the MAC address of your iPhone can track the position of your iPhone whenever you are tethering. That's basically what the magical ability of the iPad/iPhone pair tells us. Did you know that?

Syndicated 2011-04-27 18:30:23 (Updated 2011-04-27 18:30:50) from C. Scott Ananian

Sugar-on-Android, week one

Last week I described a four-week plan to survey key technologies for One Laptop Per Child's forthcoming XO-3 tablet computer. Here I'll describe the results of the first week of work, which dove into Google's Android operating system. Warning: technical content ahead...

Basic design of Sugar-on-Android

1. Cross-compile gobject/GTK/gobject-introspection/cairo/dbus for Android; distribute these key libraries as NDK libraries. This is what I spent most of my time on this week: I've managed so far to port libiconv, gettext, glib, pixman, freetype, fontconfig, cairo, libxml2, and pango. (Source code)
2. Use cairo or OpenGL backends of GTK3 to render legacy Sugar activities directly to Android canvas.
3. Modularize sugar; use D-Bus for inter-module communication. Interprocess communication mechanism is Android 'intents'; these can redirect to the web or the Android Market for missing dependencies. (Collabora reportedly already has a D-Bus implementation for Android.) Sugar components can also become Android Services.
4. Implement Sugar Home/Groups/Neighborhood views and Journal as four separate Android App Widgets. These could also be implemented by providing a new Android home application, but I think the finer-grained modularity afforded by splitting these functions would yield a better design and make it easier to incorporate upstream improvements to the Android launcher.(Android Live Wallpaper is also similar in function, but not as good a fit.)
5. The Sugar Journal becomes an Android "Content Provider", which stores/retrieves content for other Sugar activities. (There is special Android support for "collection-based Widgets" and Live Folders which may be helpful.)
6. Use gobject-introspection to build a multi-language environment. Use JGIR to expose Sugar APIs to "native" Dalvik apps; use something like the Android Scripting Environment to expose Android native APIs to GIR languages (Python, JavaScript, C, etc).
7. [opportunity] Use the Android port of OLSRd to implement a Neighborhood view. Alternatively, investigate AODV routing on Android and/or AllJoyn (which also requires root access, see pg 24-25 of the manual).

Key Benefits

• Sugar is integrated into Android environment; use native Android education apps, or apps like Movie Studio which have no Sugar equivalents yet.
• Android APIs and customization hooks are good, and provide a more-extensible framework for development.

Open challenges (general)

1. The web integration story is cloudy. Java and JavaScript can call each other inside a bundled WebView widget, but this isn't supported in standard Browser app. Browser plugin interface would help.
2. No good story for building 'native' Java/Dalvik or C apps on the device. Writing a simple Dalvik compiler would help. Dalvik specs are available, and people have written Dalvik compilers for toy languages.
3. "View source" requests can be implemented as an Android 'intent' message, but no good story for implementing this functionality other than on a case-by-case basis in each activity.
4. Although the Amazon Marketplace for Android indicates that it can be done, it appears that there is no "blessed" mechanism for creating .apk files on the device and installing them. (Android bug, discussion)

Current technical issues/bugs

1. Cross-compiling for Android is currently a miserable experience. The Android NDK appears to have been put together by a team which had never seen a proper cross-compiler before. Since I only had a week for this exploration, I mostly kludged things together to get past this, but any serious work with Android should start by defining and upstreaming proper autoconf "target triplets" for Android-on-{ARMv5, ARMv7, x86} and building a proper cross-compiler. Then patches to various tools and libraries could start being upstreamed. Using the bespoke Android.mk build system of the NDK is a non-starter. No serious obstacles here, just work to do.
2. Xoom hardware is ARMv7, but Android emulator is ARMv5 only. Unfortunately, gdb is broken on the Xoom. So we're building for ARMv5 at the moment, so we can debug in the (slow) emulator.
3. No good support for shared libraries may cause activity bloat. May be able to be worked around using the new Opaque Binary Blob (OBB) feature.
4. Much existing code (fontconfig, gettext, gtk, etc) expects to read configuration files from the filesystem. Currently we are using the default fall-back configurations. OBB support may help here as well. There are a number of different storage APIs in Android, but none seems quite right.
5. It would be nice to implement a ring-style XO home screen without completing replacing the android Launcher. No clear way to constrain app layout on home screen w/o completely replacing the Launcher. Is it worth hacking the Launcher source?
6. Mesh on Android using OLSRd current requires root access. In order to run on unrooted Android devices, we need (a) proper power management for Ad Hoc mode wifi, (b) APIs to enable Ad Hoc mode, and (c) APIs to manipulate kernel routes.
7. We're building libraries without thread support because Android's "Bionic" libc has an eccentric thread library. Linking with -lpthread fails because the thread functionality is bundled into -lc. Probably just providing an empty libpthread.so would help a lot.
8. Some work has been done to build GNU libc for Android. This bloats activities even further, but might help ease library porting.
9. Porting gobject-introspection will be painful because its makefiles are not set up for cross-compiling. Some steps want to run on the target hardware, which is difficult in the Android environment.

Bottom line

I can see how the whole Sugar stack can be put together on the Android platform. The hardest part is probably just setting up packaging and a good and repeatable build environment for the different components, and getting enough adoption of this that patches to support Android can be pushed upstream. Many of the important pieces can be developed in parallel (Theme, Journal, Mesh, Friends, Home, library porting, etc). A little early to tell how hard it will be to port existing Sugar activities to the new Python/pygobject/GTK3 framework.

Syndicated 2011-04-12 17:40:48 (Updated 2011-04-12 19:08:31) from C. Scott Ananian

Exploring New Technologies

Last Monday I rejoined One Laptop Per Child as Director, New Technologies. My mandate is hardware and software for the XO-3, OLPC's upcoming ARM-based tablet computer for education in the developing world. The new machine should be lower cost, lower power, more indestructible, more powerful, and potentially more expandable than ever. There are about two million machines in the XO-1 family (XO-1, XO-1.5) in the hands of kids today. The XO-3 will build upon this impressive foundation to reach further into the poorest and least-connected regions of the world.

I will kick-off my work with a series of four week-long sprints between now and eduJAM Uruguay to investigate a number of possible directions for the educational software stack on the XO-3 tablet. On the XO-1—series machines OLPC ships Sugar, an impressive collection of educational software developed by Sugar Labs. How can we best keep the best of Sugar while yanking the UI forward into a touch-friendly tablet world?

1. This week (April 4-8) I'll begin by working on a port of the GTK3 UI library to Android. The GTK3 library contains touch support missing from the GTK2 library on which Sugar is currently based. The end goal here would be a full port of the Python/GTK-based Sugar APIs, running on something like the Honeycomb Android OS. Our existing educational activities could be ported to the new APIs without too much difficulty, but we'd largely use the existing Android OS facilities instead of the parts of Sugar concerned with low-level system management. To clarify: this is a preliminary exploration—we haven't decided to base the tablet software on Android (or anything else) yet.
2. The next week brings a new direction. During the week of April 11-15 I will start porting Python/GTK3 to Chrome or ChromeOS via the Google NativeClient plugin. This path would result in activities which more fully integrate with web technologies—even in disconnected regions of the world. On desktop machines, Sugar activities could be run inside the Chrome browser, while ChromeOS (or another embedded OS running chrome/webkit) would provide the system management functions on tablet machines like the XO-3. As with the Android port, this is an exploration, not a definite software direction.
3. The week of April 18-22 I hope to focus on mesh networking. This has a somewhat checkered history in our deployments; I hope to identify the remaining roadblocks and map a way forward to make this a flagship feature of the XO-3 software.
4. The week of April 25-29 is for the existing Python-based Sugar codebase. In order to continue moving forward, it needs to migrate to GTK3, gobject-introspection, and some other key enabling technologies. I believe it would also benefit from language-independent APIs and better modularization to allow a more incremental migration path.

The following week is Conozco Uruguay and the Uruguay EduJAM where I'll present my progress on these initial exploratory projects and discuss the path ahead with the wider OLPC and Sugar communities. Clearly, a week each is not enough time to finish any of these projects! But the focused effort should help to better identify the promise, roadblocks, and challenges in each of these paths, which then in turn will help us to plan the future.

Syndicated 2011-04-04 21:48:52 (Updated 2011-04-05 04:47:48) from C. Scott Ananian

Taste, user experience, and engineering

A recent article on Nokia's internal culture contained this interesting quote:

Designers are also, by training and predilection, inclined to design for the usual, where engineers are taught a kind of rigor that compels them to account for, and overweight, low-probability events.

This does seem to me to often be a fundamental problem in not only interaction and UI design, but also internal programming APIs and interfaces. Good engineering is a clever balance; as Larry Wall has said: "Easy things should be easy, and hard things should be possible." An engineering mindset often fixates on the hard things (the "interesting part of the problem"!) and tries to make the hard things easy (or easier), at the risk of making the easy things hard. The end result is failure.

Truly elegant engineering involves finding a view of the problem where the hard parts of the problem disappear. We're not always fortunate enough to find that solution. In falling back to a practical/possible solution, we must be careful to ensure that we keep the easy things easy — it's fine if the hard things are difficult, so long as they are possible. Effort spent making the hard things easier is wasted if it makes the easy things harder. The goal is not a uniform mediocrity of design.

Syndicated 2011-02-21 03:48:25 (Updated 2011-04-04 21:19:45) from C. Scott Ananian

Improving Hunt Software/Improving Google Docs

There's been a lot of discussion about publishing and sharing the software that Mystery Hunt teams use to collaborate to solve puzzles. This is mostly misguided, IMO: teams are very different, and they organically grow solutions to fit their unique processes. On the other hand, an increasing number of teams (including my team, Codex), are building their collaboration software on top of Google services, especially Google Docs and Spreadsheets. Rather than trying to collaborate on One True Hunt Team Software, I think it would be far more useful (for all teams) to lobby for improvements to Google's services. These raise the bar for everyone, and indirectly benefit many other people with collaborative processes.

So here's my list of improvements I'd like to see in Google services:

1. Integrating Docs and Spreadsheet. If we had an initial "sheet" of the spreadsheet with editable formatted text (not spreadsheet cells) we could actually do away with the wiki we use for capturing free form thoughts and links related to a puzzle.
2. Integrating Spreadsheet chat, Google Talk, and Jabber. We could just use the chat in the spreadsheet if it were open and accessible, instead of creating our own per-puzzle chat rooms.
3. Making publish and "setAnonymous" access available via APIs that actually work. We need to use Google Doc Script to do the setAnonymousAccess call, which is not exported via the otherwise-more-complete GData APIs, and drive a headless Firefox 2 instance via Selenium to get the publish bits enabled for the spreadsheet. That's ridiculous.
4. Providing a way for Google Doc Scripts on a spreadsheet to export data more easily. We can use a Google Form to create a spreadsheet for a puzzle, but no easy way to provide a link to that spreadsheet on the page that results after form submission, or redirect from there.
5. IIRC Google Talk support for multi-user chat is still barely supported. It doesn't use the standard Jabber protocols, for one. If this were a better supported/more standard service, we wouldn't have to run our own Jabber server.
6. And, of course, the "next generation" of all this would integrate audio and video into the chat as well. Video is probably more useful, as it communicates human emotional cues. Audio isn't easily archived or searchable, and doesn't work well in crowded rooms, so it is less useful to us. But it would be great if we could actually see some/many/most of the particants in ringhunters, maybe little live video icons next to their faces.

Any further suggestions from other teams?

Syndicated 2011-01-18 06:53:12 from C. Scott Ananian

Codex wins!

We won the Mystery Hunt this year! You can read a Boston globe article which mentions us, but mostly talks about Palindrome.

That means we have to—er, "get to"—write the Mystery Hunt for 2012. My free time for the next year has just vanished.

Syndicated 2011-01-18 05:32:43 from C. Scott Ananian

What's Wikileaks up to?

A recent article in the Economist points out that Wikileaks is not unique: modern network tools have made anonymous communication ubiquitous. You can't stop "wikileaks" by attacking Julian Assange alone. The article is incorrect, however, in claiming that anonymity is easy — in some sense anonymous leafleteers in Colonial America were better off. Bradley Manning currently sits in jail. Haystack was fundamentally flawed. There continues to be a role for organizations who desire to facilitate anonymous speech to identify and provide trustworthy and user-friendly tools and procedures.

Aaron Brady achieves a more fundamental insight by examining Julian Assange's aims. Assange's goal is to hobble "conspiracies", that is, the small cliques of power and secrecy embedded in most organizations, and he seeks to do this by causing them to fear information sharing. By this metric, Wikileaks seems to be succeeding. (Read Aaron Brady's essay for the details.)

But it's worth pausing to consider: are open organizations truly better? Is openness practically achievable? This is an organizational problem which was on the front burner at OLPC while I worked there: OLPC pledged an open development and governance model, but was continually charged with being closed, insular, and secretive in practice. We reorganized previously-internal mailing lists and pledged to conduct all important business on public archived lists. Yet there was continual backsliding. Sometimes private email was used merely to prevent embarrassment or confusion—to fact-check before making a public statement. Other times it was claimed that some measure of secret/private communication was a fundamental part of business or negotiation, necessary for interacting with external entities. In order to evaluate the latest components/plans/schedules of our partners, we had to sign NDAs. The secrecy requirements of the third-party then contaminated related discussions. In the end, even an organization with a goal of openness ended up embedding pockets of secrecy, which always threatened to grow and spread unless they were occasionally beaten back. Attempting to stand for open principles was often claimed to make OLPC "uncompetitive," as in: we couldn't hope to get the best deals/access to the latest components/whatever if we insisted on being open about everything.

The quest for openness in business seems to parallel the role of Wikileaks in national affairs. As with OLPC's business negotiations, we are being told that secrecy is an essential part of the diplomatic process, and that publishing internal cables hobbles America's ability to achieve its goals. The claim is that Wikileaks threatens to make America "uncompetitive."

Is this true? Openness is an ethical position, but not a black-and-white one. Very few people argued that OLPC (or America) should have no secrets — the debate was always "how many?" In practice if the desired answer was "as few as possible", there was always a Wikileaks-like need to continually drag private content to public forums in order to combat the creep of secrecy. Perhaps the same is true of governments.

Then again, over-reaching openness threatens individual privacy — where to draw the line? Must all our personal mistakes be made in public? Must all our national mistakes be made in public?

Syndicated 2010-12-02 21:02:49 (Updated 2010-12-02 22:22:44) from C. Scott Ananian

Airport searches.

For the record, my personal bugbear is the privacy implications of the new "advanced imaging" machines appearing at airports. But then again, I've distrusted supermarket loyalty cards and freeway FastLane programs, too -- my experience is that data which is collected will eventually escape your control (if you had any to begin with), and that no one is willing to offer a believable privacy pledge for such data (it would have to have 3rd-party audits for compliance, for example).

All that said, I just read a very reasonable article discussing the health impacts of the new scanners. I'm not going to play the alarmist card — ironically, the risks of injury from passing through a scanner are most likely about the risk of injury in a terrorism-related incident, that is to say extremely small — but it's prudent to admit honestly where risks are unknown, and to call the lie when deceptive arguments are used.

So let's not get all "ooh, scary, radiation" about this — quantum-physically speaking, everything is radiation at some wavelength — but it's worth keeping the risks in mind so you can make your own decisions, especially if you'd otherwise feel peer-pressured to "just do it". I'm just glad that as a nation we've apparently decided that this is where we're going to step up and draw the line, libertarians and liberals together. Contemplating our freedoms progessively and silently eroding away one by one is a far more worrying prospect.

Syndicated 2010-11-24 21:45:38 from C. Scott Ananian

Words With Pirates

I've caught the Words With Friends bug. Worse: the Words With Pirates subtype. (It's all the tile-placement and strategy fun of Words with less of the tedious racking your brain for obscure words; a more relaxing variant for when I don't want to think so hard.)

Today I discovered that I didn't actually understand the full word-creation rules for Words With Pirates. For the benefit of other similarly unenlightened folk, here's the accepted word list as a regular expression:

^((([gh]y?|y)?ar+(g(h?))?)|(harhar(har)?))!?$

In more verbose format, these are all the words:

ar arg argh gar garg gargh gyar gyarg gyargh har harg hargh harhar harharhar hyar hyarg hyargh yar yarg yargh

In addition, an exclamation mark is always allowed at the end, and any non-zero number of r's may be substituted for any r.

Note that the words 'har', 'harhar', and 'harharhar', with optional exclamation marks at the end, are allowed. This is a little unusual, since the 15x15 grid should allow 'harharharhar' and 'harharharharhar' to also be played -- but these are not in the dictionary. Nevertheless, these oddball forms will probably prove useful to those stuck with excess a's.

For the benefit of the obsessive, there are 15 A's (worth 2 points), 9 G's (worth 3 points), 6 H's (worth 5 points), 28 R's (worth 1 point), 3 Y's (worth 10 points), and 3 !'s (worth 10 points), for a total of 64 tiles.

Enjoy!

Syndicated 2010-10-01 16:16:07 from C. Scott Ananian

CoffeeScript and TurtleScript

Via reports on the OSCON 2010 Emerging Languages Camp, I recently discovered CoffeeScript, a very interesting "dialect" of JavaScript. The original idea for CoffeeScript seems to be to clean up the syntax of JavaScript, while preserving direct correspondence as much as possible. Over time, it seems to have grown more and more "neat features" which have increasingly-hairy desugarings into JavaScript — but the JavaScript translation of a CoffeeScript program is still very readable.

This has some relationship to my TurtleScript project, which also sought to clean up JavaScript to some degree. In TurtleScript I've tried to pare down as much of the language as possible, using the smallest number of syntactic forms and JavaScript features as possible, with the smallest possible parser/compiler/runtime system, inspired by Crockford's Simplified JavaScript.

CoffeeScript has some very elegant simplifications: for example, everything is an expression, which reduces the statement/expression syntactic distinction in C-like languages. Making the body of a function into an expression removes unnecessary return statements. Making for and while into expressions is a cute means to introduce (the power of) array comprehensions without additional syntax. CoffeeScript also has a nice way to express functions both with and without lexical this binding (a JavaScript skeleton in the closet).

Unfortunately, the full CoffeeScript language includes many many syntactic forms — even more than full JavaScript. Some of these can trivially be moved into a library, at a slight cost in compactness. While having varied syntax that precisely and concisely expresses programmer intent is certainly admirable, it makes the language harder to learn and, in my opinion, less elegant. For a scalable, learner-friendly language system I like something like NewSpeak more: a language whose goal is to grow smaller, not larger.

Syndicated 2010-07-28 20:31:54 from C. Scott Ananian