Older blog entries for uriel (starting at number 12)

I don't think anyone will be surprised that I love the latest creation of Ken Thompson and Rob Pike: the simple, elegant and preeminently pragmatic Go programming language.

But what makes me specially happy is how fast Go is growing in popularity, not among the masses of redditors (although it has some following even there) or programming language design hobbyists and academics, but among people building real production software and solving real problems.

It almost gives me back some hope for the future of the software industry.

While profiling Atrack and after writing some code to monitory the live memcache usage stats and display some pretty graphs, I noticed all the CPU time was spent picking and unpicking stuff from memcache, this is really expensive in App Engine because cPickle is disabled, and you are forced to use the excruciatingly slow pure python pickle module.

So for now I used a bunch of calls to string.split() and string.join() to serialize my data into a string before feeding it to memcache, this seems to improve things considerably and my CPU quota can breathe for now.

Next I would like to benchmark how well various serialization methods perform in App Engine, I have seen some comparisons of protocol buffers, json, etc, but I suspect the results wont apply to the pure-python environment in App Engine, and I think none included bencode (which I have learned to love after having to use to implement the bt tracker protocol).

In related news, I built an App Engine based site that monitors open trackers, used mako for the templates, which turned out to be bearable.

Been playing lately with App Engine. My first experiment is Atrack, a (BitTorrent) tracker that relies purely on memcached, it is fast and efficient, and less than 100 lines of code.

There is demo open installation in appspot, add http://bittrk.appspot.com/announce (or if you are paranoid https://bittrk.appspot.com/announce) to any torrents you like, I could use the stress testing!

So far I'm tracking a few thousand torrents (after torrentfreak broke the news) and barely using %1 of the App Engine quota.

Of course one of the cool things about atrack is that there is no way for me to really what torrents or how many torrents exactly are being tracked! This makes me happy but it is hard to shake the curiosity to know who and what for people are using your code! Ah, the wonders of privacy :))

While reading on the Bittorrent tracking protocol I realized that the problem it solves is much more general and a subset could easily be used for decentralized online gaming, chat systems, and to help other 'grid' like applications find each other inside the cloud.

So I started to write up an spec for Ntrack, a generalized and simplified tracker scheme, that should be backwards compatible with bittorrent clients and trackers, but makes all the bt-specific functionality optional. This is pretty much the subset implemented by atrack; I'm working on a distributed chat system that uses it and I hope others come up with other apps that take advantage of this.

P.S.: I know others have built trackers on app engine in the past, but they seem to be out of quota most of the time, and by looking at the source this is not surprising given how over-engineered they are. *sigh*

Share and enjoy.

This year Plan 9 from Bell Labs is a mentoring organization at the Google Summer of Code!

We are interested in all kinds of projects related to Plan 9, Plan 9 from User Space, 9P and Inferno. We have a list of ideas but we are happy to help you come up with a project that suits your interests and skills best.

Aside from the 4500$ provided as part of GSoC, we will send all participants a fantastic Glenda T-Shirt!

Hurry up, the deadline to apply is the 24 of March!

The first Swedish wmii9con will take place the 18/19 of February in Stockholm.

The main aim is to have fun and meet other Plan 9 and wmii users. It will be more more of a party with a bunch geeks than any sort of conference, but if you are interested in learning more about Plan 9 feel free to drop by and have some drinks with us.

I also will be speaking at FOSDEM the following week and a bunch of 9fans will co me along.

In other news the new Plan 9 web site has been quietly launched which much cleaner design and the full "sources" source tree is available over the web. We also have a more friendly address: 9fans.net

The new website is generated from troff(!) source using a new tool htmlroff , it still needs a bit of polishing but works great and the html version the papers look much nicer, for example see the Security in Plan 9 paper.

I have been quite busy with various things, but I promise to try keeping this diary more updated.

Experimental Plan 9 from Bell Labs changes and patches mailing list: plan9changes

The archives have been populated with most of the changes of the last few months to have a pseudo-changelog.

It even has RSS and Atom feeds for those that like that kind of thing, not me ;)

Did some wikifs hacking and 20h got it running in 9grid.de.

caerwyn wrote a wikifs client for the Inferno version of acme, so now one can edit the Plan 9 wiki from outside Plan 9, lets hope this doesn't bring spam ;) See the lab 43 entry in the Inferno Programmer's Notebook for the details.

And today I submitted the proposal for my talk at 22C3; 20h and garbeam already have sent theirs. I hope to make it to CCC this year at last, I have missed it the last five years because I'm always too busy with work...

wmii is moving forward towards wmii-3, full 9P support is almost done... oh, and forgot to mention that v9fs will be part of the kernel 2.6.14 release!

I'm probably forgetting some other things, but work is killing me lately, I'm considering taking a couple of weeks off and go to the hackmeeting in Menorca to relax for a while.

(Oh, and I'm a Journeyer now, yay! thanks everyone that certified me)

I'm a bit annoyed that after more than a couple of months in advogato and getting certified by a quite a few people, I still can't even post comments to existing articles, which was one of my main reasons to register...

I think I wanted to post some comment to mako's article about CC(to which incidentally I made some very small contributions in the hours prior to it's publication, mostly suggesting bits that could be deleted ;)), but by now, I have forgotten what I wanted to say, except probably that mako toned it down too much ;)

Oh, and neither can I 'register' with various of the projects I'm involved(eg., wmii)... oh well...

Ok, now I should stop whining and go hack wikifs... oh, and I had an idea for a cross between wiki, blog and irc that I plan to write for Inferno. If only days had 32 hours...

Just found that the 9P/Styx implementation in Java is alive and well, the other day I was looking for it to add the link to the v9fs homepage but could not find it. It recently moved to a new site: http://jstyx.sourceforge.net/

I don't think I have any use for it(I rather don't touch Java if I can help it), but it's nice to see more 9P implementations out there. And now that I think of it, maybe I could use it to interact with some legacy Java code we have at work from our new systems in Python and Limbo.

So far we have 9P implementations in C, Limbo, Python and Java, not bad.

I had been toying with the idea of doing a Ruby implementation, mostly for fun and to learn the language better, but I don't like Ruby, so I'm thinking Erlang would be a much better idea, as caerwyn said the other day in #inferno, what other good concurrent languages are out there? aside from Limbo and Erlang? I know none, of course there is also libthread.

In related news, at last I convinced garbeam to make wmii implement proper 9P(oh, yet another implementation, and this one under a BSD license, yay!), and the conversion is almost done, this will allow to make our filesystem interfaces much saner.

I hacked the wmii configuration/init system and worked out a simplification that got rid of that wmira kludge and lets us use Plan 9 from User Space tools without polluting the PATH. Oh and of course it's also got rid of a few hundred lines of convoluted crap. As God Ken once said:

One of my most productive days was throwing away 1000 lines of code. -- Ken Thompson

When I hack wmii, usually that is the kind of work I do, fortunately garbeam can appreciate it; that can't be said of the developers of many other projects, who are obsessed with adding more and more code rather than step back and do things right.

So slashdot, always bastion of reliable news reporting, had to bring the news: "Bell Labs Unix Group Disbanded", from this others went on to deduce that Plan 9 development has stopped.

Nothing further from the truth, not only is Plan 9 development at Bell Labs ongoing but development outside the Labs is as active as ever(and expect more news on this front soon).

Sape Mullender wrote to 9fans:

1127 ceased to exist some time ago.  The department was renamed and got a
much longer number which, of course, I can no longer remember.  The current
split of what's left of 1127 is not considered to be a big deal by most
members of the department.  There have always been many interdepartmental
collaborations, so 1127 has always been more of a management entity than a
group/club/fraternity.

 [...]

 Plan 9 is alive and reasonably well as our current vehicle for wireless
systems research.  Plan 9 has gone embedded (but rest assured, all our
wireless software runs in user space, using Plan 9's real-time scheduler to
make sure it runs when it needs to.

 Using Plan 9, wireless code becomes useful in a fraction of the time that
code can be made to run on all those commercial embedded operating systems
(which I do not care to mention) and also much quicker than on Linux which I
am sorry to say is just too large and illegible.

May the research go on!

And don't let the "Fourth Edition" release date fool you, Plan 9 ISOs are built from the latest code every night, and the current codebase is very far from what Fourth Edition was, it's just that nobody has been bothered to make a new "official" release. Plan 9, as usual, is more about code and quality than about hype and buzzwords.