Older blog entries for nbm (starting at number 113)

Be sure to wear a flower in your hair

(This is a repost of my entry "Be sure to wear a flower in your hair" to the South African Tech Leader technology group blog.  My next post, What is a geek?, has just been posted there, if you want to read it before a week or two from now when I'll repost it here.)

It’s really hard to summarise the experience of a first visit to San Francisco, assuming you’re at least somewhat a technology geek. San Francisco (and by that, one generally means the San Francisco Bay Area) is modern technology’s birthplace and still its hometown.

Xerox PARC (as in Palo Alto Research Centre) either created or popularised implementations of modern computing aspects such as the mouse, laser printers, Ethernet, GUI/WIMP interfaces, Object-Oriented Programming with the Smalltalk programming language, and the Integrated Development Environment. The Bay Area is home to the headquarters of technology giants such as Apple, Cisco, eBay, Google, Oracle, Sun Microsystems, and Yahoo!, as well as upstarts like Facebook, Mint.com, and SugarCRM. (And SynthaSite, of course.)

At times during my visit the technology industry seemed entirely pervasive — whether it was randomly walking past three people in the street arguing the merits of various memory allocation techniques (I kid you not) or hearing that one of your colleagues just moved into the apartment the CEO of a popular social media startup just moved out of. It is hard not to let your imagination loose with the idea of what can be achieved here, especially after seeing over 3000 developers, a large portion of them probably local to the area and most certainly at least as geeky as I am, at Google’s I/O conference. (I posted quite extensively about my Google I/O trip on my personal blog, if you want to check it out.)

If I sound a bit in love, it’s because I am. I challenge anyone in our industry to somehow not be a little in love with the vibe and pace and sense of belonging you will find in San Francisco. But this isn’t really about technology in San Francisco — it’s about it in South Africa.

Romance novels suggest that sometimes you need to discover (or be reminded of) what is out there to realise quite what you have, that while you find that there’s a lot of prettiness out there, you will also discover that there have been and always will be many and unassailable reasons for you being with the one you’re with.

I needed that a bit with South Africa. I’ve always wanted to be here for the long run, but it has been hard not to get worn down little by little over the past few years by the scarcity of interesting highly-skilled work and the similar scarcity of ambition in South African technology companies. Now, I have an updated and more accurate idea of what is out there, and while South Africa does fair poorly in some comparisons, there are other, more important, aspects to take into consideration. And those mean that leaving it to find some technology heaven elsewhere sounds like a bad swap.

And it’s not like you have to be in San Francisco to wear a flower in your hair — you can experience and help create your own slice of the San Franciscan vibe wherever you are. All it really takes is creating or finding a workplace you can be passionate about using technologies you’re passionate about with people who share that passion (am I saying “passion” enough?), and finding and building a community of similarly technology obsessed people who can help you, and who you can help, and to make you feel like you’re not alone (and who you can make dinner conversation with without resorting to the weather).

I lucked out on the first one — at SynthaSite I have an ambitious company that knows how to treat their employees well, great colleagues, and challenging work — and a pantry full of snacks, lunches materialising daily at my desk, games consoles, and 40-inch TVs. And there are at least a few similarly-enlightened workplaces around, and more can be created.

I already know a number of geeks who’d give a good argument on the merits of various memory allocation techniques. It takes work, but through efforts like GeekDinner and StarCamp, we come to know more, and different, people and benefit from that meeting as they introduce us to new perspectives and, hopefully, shake our preconceptions. And not only come to know people, but also come to know more about our trade through presentations and less formal conversations sparked by an interest that perhaps we didn’t know we had before others introduced the topic.

While it is easy to moan about the lacks we have here, it seems that by our attitudes and our actions we can create an ever-increasing slice of that seemingly far-away vibe. As we kick off planning for the next StarCamp in Cape Town, and a national web technology conference, I’m hoping we will find positive attitudes and actions in finding co-organisers, presenters, sponsors, and venues.

Syndicated 2008-06-28 12:21:15 from Cosmic Seriosity Balance

First Tech Leader post up

Just before I left for my San Francisco visit, I was approached by Nic on whether I'd like to write for Tech Leader, which is a South African "editorial" group blog about technology, edited and run by the Mail and Guardian Online.

My first post, Be sure to wear a flower in your hair, is on how my trip to San Francisco and the technology vibe and sense of "anything is possible" revitalised me a bit about South Africa and the potential future that could be if technology people stay and work for change (by which I mean in the industry, but it's also good to try change things outside it too).

I'm going to try write a post a week for Tech Leader on less nitty-gritty things, and try get back to a few posts a week here after my recent fortnight of silence dealing with post-travel jetlag and accumulated work responsibilities.  I'll post a pointer to Tech Leader when I post there, and post the full content here two weeks (or so) afterwards.

Syndicated 2008-06-19 13:02:27 from Cosmic Seriosity Balance

Pylons/TG2/WSGI Sprint and sight-seeing weekend

I spent much of the weekend in sunny Sebastopol at the Pylons/TG2/WSGI sprint at O'Reilly's headquarters there.  One doesn't expect the lack of fanfare that marks the O'Reilly offices - besides a modest sign on entry to the parking lot, only a Tarsier statue made of recycled metal identifies the pretty normal-looking offices.

There's not much I can say that would probably be of interest to others other than I had a lot of fun, and the sprinters were all very friendly, great to chat to, geeky, and generally just like the great group of geeks we have in Cape Town.

My awesome carpool partner, Kelvin, not only managed not to go mad stuck with me in a car for an hour and a bit each way for two days - he was quite keen to show me around San Francisco on Sunday evening.  We did some common tourist things - went to the TransAmerica Pyramid, Union Square, past the Symphony Hall, through the Golden Gate Park and China Town, up Coit Tower on Telegraph Hill, down the "Crookedest Road", and generally meandering all over the place.

Syndicated 2008-06-03 05:58:56 from Cosmic Seriosity Balance

Google I/O: Google App Engine fireside chat

There were a few questions about the choice of Python as a language, and whether and what languages would come next, comparisons to other existing containers, and so forth.  Guido van Rossum said it was partly because Python is one of the three big languages at Google, and because it was (relatively) easy to harden the VM.  Kevin Gibbs said they had to start somewhere, and that they were committed to others.  Paul McDonald said that the two most voted-for issues on the issue tracker are language-related, and that there were teams (ie, more than one) currently actively working on languages (ie, more than one).

A couple of questions around "maturity" - the team says they'll make it clear when it is no longer a preview, and that this will probably happen when they have the billing set up and offline processing.  They expect billing to be available "toward the end of the year".

Question about HTTPS/SSL and access to encryption within GAE code.  Answer is that it's something they want to do, but don't know when they'll get to it.  Data is "strictly" partitioned between apps in the store (BigTable).

A common thread in answers were that the Google App Engine team were very interested in people being able to get their data and code out of GAE, and they're working on making it easy to bulk output the data.  They hoped that a standard would emerge for BigTable-like storage (CouchDB, SimpleDB) so that people could write code and host it on GAE or elsewhere.   And people are already working on compatible APIs to make it possible to run on other storage systems (but may not be too efficient).

Syndicated 2008-05-29 23:33:05 from Cosmic Seriosity Balance

Google I/O: Five random bytes from the Thursday Keynote

  1. In a particular period of time, 50% of the projects going live were "20% time projects"
  2. Occam's Razor applied to design - the simplest design is probably best
  3. Don't let the urgent drown out the important
  4. Have a healthy disrespect for "the impossible"
  5. The imagination is a muscle (which needs regular exercise to function well)

Syndicated 2008-05-29 22:57:23 from Cosmic Seriosity Balance

Google I/O: Google Doctype

The Hitchhikers Guide To The Web is what Google Doctype would be called if it could be called that, said Mark Pilgrim.  It's an encyclopedia of the open web - definitions and compatibility of various HTML, CSS, and JavaScript functionality, articles on how best to achieve certain effects on the web (and avoid pitfalls).

The goal of Doctype is to be the #1 hit for any search term about the open web.

The "open web" is the DOM, HTML, and CSS - "just about anything that you can do on Firefox on 64bit Linux" (ie, no Flash and Silverlight) (and not XUL either).

In scope are those "open web" topics, as well as security and accessibility concerns that affect the "open web" and the closed web features as one needs to know them when using the "open web" technologies.

The text content is licensed under Creative Commons attribution-only ("the good one"), and the code under the BSD license.

For offline use, you can use subversion to check it out from Google Code Project Hosting.

Progress so far (about two weeks) - 75 articles, 2800 pages of reference docmentation (ie, "background-color", "orange"), and 10k test cases.

Syndicated 2008-05-29 03:14:34 from Cosmic Seriosity Balance

Google I/O: Underneath the covers at Google

This session deserves a much longer post, but I just wanted to put down the most interesting stuff quick.  Basically, a back-end developers guide of how Google is put together - from how a request that someone does in a browser gets a response to how those responses are put together from multiple sources and how those sources are built up.

Everyone knows Google's love of lots of commodity hardware for their servers, but it was interesting to hear some other things - reasonably low-end networking gear too.  Otherwise, that they've back where they started in terms of machines without cases shoved into in-house-designed racks.  The scale has changed dramatically, of course.

"If you have 10k servers, expect to lose 10 a day..."

GFS's masters are same server hardware as slaves - take part in master election like any other machine.  Google puts "millions" of pages together in a GFS "file", since it uses 64MB chunks.  200+ clusters, many of them 1000s of machines, pools of 1000s of clients.  4+PB filesystems, 40GB/s read/write load (even while HW is failing constantly).

MapReduce usage within Google is growing fast - 700 new applications in a recent month at peak, currently around 10k applications.  From 171k MapReduce jobs in March 2006 to 2.2 million jobs in September 2007.  MapReduce is very optimised to keep jobs near the data they need to conserve precious network speed within the datacentre.

Google still has one large shared source base(!), from low-level libraries used by anything to domain-specific libraries to applications.  Benefits are that it's easy to find examples of usage of something so you can use it correctly, and to reuse (ie, as a library).  Drawbacks being that such reuse causes some fairly tangled dependencies.

Language usage at Google: C++ for all high-performance, commonly-accessed web stuff.  Java is used for less-performance-oriented and/or lower-volume applications.  Python is used behind the scenes for things like configuration, administration, &c.

Syndicated 2008-05-29 01:39:39 from Cosmic Seriosity Balance

Announcements from the Google I/O keynote, Google App Engine opens signups

Some interesting news was delivered during the Google I/O keynote.

In terms of Google App Engine, the announcement that got the biggest applause was that it was now open to all signups - no waiting list and a few tens of thousands of developers.

Beyond that, the two new APIs were announced - the memcache API and the Image API.

Some pricing expectations for usage beyond the free chunk given to you were given:

  • CPU: 5 million "average" page views free, 10-12c per core-hour thereafter
  • Storage: 500MB free, 15-18c per GB-month thereafter.
  • Incoming traffic: 5 million "average" page views, 11-13c/GB thereafter
  • Outgoing traffic: 5 million "average" page views, 9-11c/GB thereafter

The Google Web Toolkit 1.5 release candidate was released today, which brings Java 5 language features.

In terms of OpenSocial, the 0.8 version specification was released yesterday, and that AOL has joined the OpenSocial initiative.

Syndicated 2008-05-28 22:19:54 from Cosmic Seriosity Balance

Google I/O keynote - moving the web forward

The Google I/O keynote was entitled Client, Connectivity, and the Cloud.  The message was that "Google cares about moving the web forward".

The obvious question is why, and four reasons were given:

  • Google is a company that has only existed and could only have existed because of the web.  Thus, moving the web forward is improving what they can deliver.
  • It's a virtuous cycle - richer web apps can reach more users, and this means more usage, and this means more revenue.
  • More softly, since Google is a company of the web generation, and that's how the web was made - consensus and partnership.
  • Again softly, Google feels a debt of gratitude to the open source and the web community. 

How will they move the web forward (and, perhaps, how should the web be moved forward?).  The history of challenges and benefits of the historical and current model of computing was given:

The mainframe had a lot of power (for the time), but was not easy to get your hands on it.  Deployment was easy, since you installed your software on one computer and used dumb terminals to use it.

The PC meant accessibility, but meant less power.  It lost the ease of deployment because you had to support a variety of hardware, operating system, and applications/libraries.

The web brought the return of easy deployment with deployment on servers you control, and the use of the (relatively) dumb browser.  However, supporting scale in your application means needing "cloud computing" (perhaps a stretch for most current applications, but certainly true if you're aiming high), and the "accessibility" of clouds is currently not that great.

So, how should it move forward?

  • Making the client more powerful
  • Keeping connectivity pervasive
  • Making the cloud (ie, resources/power) more accessible 

In terms of Client, Google Gears was explained and demonstrated.  Basically, Gears is about extending the current browser to enable more rich applications.  It's not just offline/cacheability - Allen Hurff from MySpace showed off how asynchronous threads, SQL database and full-text search allow their mess age view to allow sorting and (more impressively) search without leaving the client.  The speed possible with this isn't what we expect from the web today.

In terms of Connectivity, Android was brought out.  What wasn't immediately obvious to me before, Android is a full stack for mobile phones, not just OS, framework, libraries, and so forth.  The example wasn't all that impressive, since only one phone was used, but the idea of multiple phones with different abilities having a consistent set of applications and interaction is compelling.

The Cloud discussion was perhaps of most interest to me, since it discussed Google App Engine.  It was described as a way of making aspects of the Google infrastructure available to developers (ie, not just "machines").  It is supposed to take care of all the problems you have outside of your own application - ie, not having to worry about setting up machines, installing the OS, maintaining the OS and applying security updates, logs, monitoring, and so forth.

The key goals of Google App Engine is to allow it to be easy to develop, easy to scale, and free to start.

Beyond or within these three main ways forward, three other projects were given attention: Google's GData APIs, Google Web Toolkit, and OpenSocial.

Syndicated 2008-05-28 22:06:40 from Cosmic Seriosity Balance

In Sunny San Francisco, at Google I/O

I won't say anything about the trip up, but I've been in San Francisco since Sunday afternoon (local time).  Monday was Memorial Day and the members of the SynthaSite team at the time I joined in November last year decided that we should get together for brunch, and then we headed out to see some sights around the area, taking a quick trip over the Golden Gate bridge and into Sausalito.

It was an overcast day, but I did get a glimpse of how beautiful the city can be.

Tuesday I pretended I had the capacity to do work and visited the SynthaSite offices in San Francisco.  Didn't get much done, but felt the office here in San Francisco shared a similar vibe to the one in Cape Town (although open plan feels weird after being in a three/four-person room for so long).

I've made it to Google I/O, which is way bigger than I expected it to be (and, according to one of the shirted staff-members, more than they originally expected too).  Hundreds of people were registering when I arrived, and by the time I got to the front of the A-B table (which was a good 75 people long), the A-B queue was longer than when I arrived.

Will try write something after each session, assuming the wireless works better than it does now...

Syndicated 2008-05-28 21:35:42 from Cosmic Seriosity Balance

104 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!