Older blog entries for djcb (starting at number 148)

waiting for 22

Achilles had Patroclus. Don Quixote had Sancho Panza. Michael Jackson has Bubbles. And I have emacs. On my N810. A while ago, I already wrote about it. I even showed some screenshot of emacs running in scratchbox. But, I didn't take the final step - getting it to run on an actual N810. Recently, I tried to get that to work. Well, that was frustrating... Some hackish instructions follow. They may or may not work for you -- try at your own risk :)

  • First I tried to simply rebuild the Emacs23-packages in scratchbox; that failed, because the compilation somehow crashes QEMU;
  • Then I tried Emacs 22 instead... but the problem remained;
  • So, I decided that maybe I should try to compile the package it on the N810 itself. Again, that failed. One of many problems: package building requires a real grep, and if you try to install it, it wants to remove the whole busybox environment; sigh. I fought the system - the system won...

But, all was not lost. There are prebuilt Debian packages of Emacs22 available; I took the armel-packages from there, and tried to install them. That almost worked. Almost, because the size of emacs is almost legendary. It did not fit on my root file system on the N810.

We're nearing the solution though; I copied the contents of the .debs to a directory emacs810 (with mc); then I copied this directory to the MMC-card of the N810 (/media/mmc2/). I set some symlinks, ie.

# ln -s /media/mmc2/emacs810/usr/share/emacs /usr/share/emacs
# ln -s /media/mmc2/emacs810/usr/share/emacs22 /usr/share/emacs22
# ln -s /media/mmc2/emacs810/usr/bin/emacs22-gtk /usr/bin/emacs22
# ln -s /media/mmc2/emacs810/usr/share/applications/emacs22.desktop /usr/share/applications/hildon/

Also, you'll need to install libungif4g. That should do the trick, and emacs should show up in your Extras-menu. And we can run emacs! Victory is mine!

Well, almost. In emacs, a very useful key is the Meta-key, usually mapped to the Alt-key of your keyboard. But of course, there's no alt-key on the N810-keyboard. Instead, I decided to remap the Chr-key. I'd like to remap it in my .emacs, but I haven't been able to do so.

Anyway, as a first start, I added these to /usr/share/X11/xkb/symbols/nokia_vndr/rx-44 (the xkb keyboard mapping):

key <SPCE> { [ space, space, Tab, space ] };
key <COMP> { [ Meta_L, Meta_L, Multi_key, Meta_L ] };

modifier_map Mod1 { Meta_L };

The first one will turn Fn-Spc into Tab, which is very useful for completion (for some reason, I couldn't get M-i working in the minibuffer). The second one will turn the Chr key into the M-key (obviously, you can't run emacs without that), with Fn-Chr giving the old Chr key. Not sure what it will break - it's black magic.

Ok, that's it. These steps should be cleaned-up, pre-packaged and made single-click-available. Anyway, the steps above should hopefully get you a working hand held emacs. Happy hacking!

Syndicated 2008-02-18 21:01:00 (Updated 2008-02-18 21:40:45) from djcb

be quick or be dead

Note, image has not much to do with this post, it's emacs22 running on my N810. More about that in my next post

For a while, we've been doing semi-weekly updates to modest, usually on Friday's. And every week, we're fixing many issues, big and small. Modest has grown quite a bit, and modest + tinymail/camel has about 240K lines of code. That's quite a haystack for bugs to hide, but I'm quite happy with the speed at which we've been able to squash them. The big fix of last week was adding support for maemo-launcher, due to heroic hacking efforts by dape. Maemo-launcher significantly improves startup speed. How does it do that? To answer that, let's look at what happens at application startup time.

still haven't found what i'm looking for

Few people have the time to write a UI-toolkit, or even printf(3), for every piece of software they develop. Thankfully, we can reuse libraries to do such things for us. The most common form ar dynamic libraries. With the ldd(1) utility, you case see which ones your application uses. For example, for modest:

[sbox-CHINOOK_X86: ~] > ldd /usr/local/bin/modest
linux-gate.so.1 => (0xffffe000)
libgtkhtml-3.8.so.15 => /usr/lib/libgtkhtml-3.8.so.15 (0xf7ef9000)
libtinymail-gnomevfs-1.0.so.0 => /usr/local/lib/libtinymail-gnomevfs-1.0.so.0 (0xf7ef6000)
libtinymail-maemo-1.0.so.0 => /usr/local/lib/libtinymail-maemo-1.0.so.0
.... (69 others) ....

Now, when we start modest, we must load these libraries. Suppose, somewhere in the code, we have:

magic_check = gtk_check_button_new_with_label ("Enable magic");

When starting the program, the dynamic linker will now have to figure out at what memory location gtk_check_button_new_with_label, is to be found. And not just that function... if we look at modest, we can find the number of external function (or more general, symbols), with the nm(1) utility:

[sbox-CHINOOK_X86: ~] > nm -u /usr/local/bin/modest
U gtk_check_button_new_with_label
U gtk_check_menu_item_get_active
U gtk_check_menu_item_get_type
U gtk_check_menu_item_set_active
U gtk_clipboard_get

In total, there are are almost 1300 external symbols in just modest; and this is only a fraction of the total, as GTK+ will have a lot of external symbols as well (think Glib, Pango, ...). In total, there will be many thousands. Without going into the details, it takes a significant amount of time to do the symbol lookup. Even on a fast desktop machine it can cause noticeable delays (esp. for C++), and more so on 770/N8x0.

Another factor that affects application startup significantly is initialization: in GTK+-based applications, e.g. gtk_init takes quite some time. In particular, applying the GTK-theme is slow on 770/N8x0, because the default theme file (gtkrc) is huge: the default theme on Ubuntu ("Human") contains 242 lines, but on my N810 default theme has 7046 lines, on much slower hardware. And note that the theme is very picture-heavy, and many little images must be loaded. To get an idea of how much work must be done you can use strace(1) when starting an application... scary stuff.

And finally, another slowdown is the physical loading of all these libraries into memory. This will typically only happen the first time, as Linux will keep the data around as long as there is enough memory (note, for testing, you can force a flush with echo 3 > /proc/sys/vm/drop_caches).

i remember now, i remember how it started

What can we do about all this slowness? Enter maemo-launcher.

Maemo-launcher is a daemon that loads common libraries at startup, does a gtk_init for you, and as such, the price for doing all this work (as explained above) is only paid once. Your actual application is compiled as a dynamic library. When your application is started, maemo-launcher forks, this dynamic library is loaded, and we jump to straight to its main-function. This totally gets rid of the initializations, theme loading and such mentioned above, and saves quite some startup time.

How much are the improvements you can expect? Johannes Schmid did some testing, and found improvements of about 25% for small programs. Some non-scientific testing for modest shows that the startup-time is 1-2 seconds faster. Does that matter? Well, let's look at our 10 million ;-) modest users, all of them starting modest once a day. With a 1 second improvement, every day we save almost 4 months of time! Jokes aside, application startup times are very visible to users, and really determine whether they consider your software fast and snappy, or huge and slow; it's time well spent trying to improve that.

parting thoughts

For a very practical way of how to get maemo-launcher working with your program, see the Appendix of the MaemoMM-tutorial. It also shows how to add additional libraries to the set loaded and initialized by maemo-launcher. Recommended reading.

Note; nm as shown above does not work with stripped binaries (i.e. the binaries of which the symbol got removed). And unless you compile everything yourself, most of the binaries on your system will be stripped. However, you can still get much of the information with objdump -R; however, it it's not equivalent to nm -u, and will contain some non-external symbols.

Finally, this is the kind of post that could contain embarrassing factual errors :-) please check the comments for updates.

Syndicated 2008-02-17 11:04:00 (Updated 2008-02-17 21:59:39) from djcb

battle against time

Like many people, I get a lot of e-mail every day, and I spend a very substantial part of my time processing them. In order to keep things under control, I'm following a number of practices, which I'll describe in the following. As with most good ideas, it's not my own idea -- it's inspired by a time-management method called Getting Things Done (GTD). More about that later.

inbox zero

So how to become in control of our e-mail again (instead of the other way around)?

The most important rule here is that we should always end the working day with an empty inbox. Fantastic as that may sound, it's really the key here. So, how do we do that?

  • At least once a day (and before leaving the office), review the messages in your inbox. For each message, decide what to do with it:

    1. If it requires some action:

      • if it takes less than two minutes: do it now;
      • If someone else can better handle it: delegate (forward) it;
      • otherwise, move it to a folder NextActions;

    2. If might be interesting or important later, archive it
    3. Otherwise, it's just crap - delete it.

  • After this review round, your inbox will be empty. No message stays behind - look at them, and decide where to move them.
  • Now, periodically check your NextActions, take the necessary actions, and move messages to your archive after completing them. (I've found that maybe only one-third of my incoming e-mail end up in NextActions)

That's all there's to it. Following these simple rules, it will be much easier to deal with lots of e-mail. If you have 250 e-mails in your inbox, some of which are weeks old, it's really hard not to forget something important; also, it costs a lot of time to scan those same messages again and again (I'd say handling a mailbox with n mails has at least O(n2) complexity :-).

Of course, this is just a starting point; you could add WaitingFor-folder with a copy of mails you delegated (forwarded) for tracking. You could somehow add the calendar. You can add other to-do items. You could think of some smart way to manage those items in NextActions. And so on... All of those things are discussed in GTD.


Getting Things Done (GTD) is David Allen's method of time management, which deals with much more than just e-mail. The steps above are a sort-of GTD-light that works very well for my e-mail. Anyway, I can really recommend the GTD-book for anyone interested in time-management. It's a rare gem in the sea of 'personal-productivity' books. Read an overview on the 43 Folders-website, or their excellent article on inboxzero, which has lots of additional tips.

I've been using GTD (and especially these e-mail handling practices) for quite a while now, and it has really saved me a lot of time and frustration. I'm quite sure this will work for a lot of people; but of course, the only way to really find out is to try it out yourself. On the internet, GTD enjoys an almost cult-like following on the web - don't let that scare you away. Just be skeptical, and use what works for you.

epilogue: gtd on n8x0

You could implement quite a bit of this on your N8x0 and (surprise) modest, and I have done so quite succesfully. What's still missing is some calendar integration, and an easy way to handle (search through) huge archives of messages. Anyway, being so mobile, the N810 with modest has become a great productivity tool for me.

One geeky alternative would be to use Emacs; people like Sacha Chua and others have written a lot about using Emacs for Getting Things Done. Now all we need is to port emacs to the N810 -- project maemacs to the rescue! Or maybe someone could make an N8x0 version of Chandler? (unfortunately, the book is still better than the program).

Syndicated 2008-01-26 12:34:00 (Updated 2008-01-26 14:26:14) from djcb

kill 'em all

The last few weeks, we were suffering some instability in our beloved modest e-mail client. At specific time as well as randomly, modest would decide to call it a day, and prematurely return its pid to the rightful owner. Bug reports are not always enough to pinpoint the problem (which might be outside modest) and it's easy to be misled in some direction. We spent quite some time with the various tools at our disposal. All have their specific strengths and weaknesses, so we tend to use them combined. Let's discuss some of them.


First, there is the venerable gdb . Although armchair computer scientists snuff at the use of debuggers, down here in the trenches, it's often our last hope. Using gdb effectively takes time, and various extra difficulties (scratchbox, the ARM architecture) can make it a frustrating activity. And for people used to graphical debuggers (like the one in Visual Studio), gdb might seem a bit spartan, even when using something like GdbMode in Emacs. But once you've become friends with gdb, it's an incredible powerful tool, which even works on your N8x0.

As a small tip, in OS2008/Chinook provides the maemo-debug-scripts package, which (among others) offers native-gdb. I'm not sure what's so 'native' about it, but it provides gdb 6.6, which works much better than the apparently 'non-native' gdb 6.4, especially with threaded code. It's not clear to me why the ancient 6.4 is shipped in the first place, but there's probably a good reason. Read more about it here, which has a lot of very practical tips.


Then we have valgrind (pronounced vel-grinned). Compared to gdb, which is like a brain surgeon, valgrind resembles a tax auditor (hurray!), with bytes in your RAM as the currency. Valgrind runs your program in a virtual machine, which offers replacements for the normal memory-management functions (malloc, free and friends). When running with valgrind, the application uses valgrind's replacements. And unlike the normal ones, valgrind's versions carefully checks where you get your memory from and what you do with it, and whether you free it when you're done with it.

It's an extremely useful tool for finding memory errors that occur during runtime. One weak point of valgrind is that it doesn't run on ARM; but still, it's a great way to find memory corruptions, leaks and so on, which will show up on X86 as well. Any kind of memory error found on X86 corresponds quite likely to a crash on ARM.

Note that I've only talked about the 'memcheck' tool inside valgrind; there are many more, such as cachegrind, massif and the new iogrind, which are great for profiling your code.

A relatively recent version of valgrind is available in the OS2008/Chinook repositories.

static checking

Except from these runtime tools, there are some other ones, which work at the static, source-code level. There are things like Coverity and lint, but in my (limited) experience, they only catch a small number of problems much that weren't also caught by gcc with -Wall -Werror + valgrind/gdb. Still, it's quite attractive to use any kind of bug prevention you can get your hands on. And let's not forget one of the most important tools: careful code review. Finding some critical part of code, and then simply reading it, letting the statements play out in your mind, and imagining the interactions. The human mind is the greatest debugging tool of all (and a pretty good bug-introduction tool too... )

Now, I hope you can find a bright mind somewhere yourself... Regarding gcc, version 3.4.4 is available in OS2008/Chinook, but modest can be compiled as well (outside scratchbox) with gcc 4.2, using Ubuntu Embedded, or the rudimentary gnome-frontend. Compiling there, and on a 64-bit architecture, helped to fix some issues as well. And the newer gcc is much better at detecting with -Wall.

so far, so good... so what

Now, even with all these tools, I can promise that there are still some bugs left in modest. But also that there are quite a few less. So please get your latest update at the usual place (the application manager). If you find any kind of instability, please file bugs at the usual place, and please describe as detailed as possible what you were doing -- thanks!

Syndicated 2008-01-19 12:16:00 (Updated 2008-01-26 14:26:44) from djcb

keep on rockin' in the free world

Ok, I guess it's still in time to wish people a happy new year. Apart from the obligatory sessions of eating, drinking and reconnecting with my inner-child, Christmas has been a great time to enrich the internal uranium, and I feel full of 235U again. I'm sure I'll need plenty of it in the new year. So, once more, best wishes to all, at let's make the world a better place in 2008.

One thing to get there is of course is to polish that raw diamond that we call modest. I've been quite happy with the progress we've made since our bèta less than one month ago. We've closed quite a number of bugs, and made steady improvements in performance and the handling of specific emails and mail servers. And we've been making frequent releases, roughly every week. If you're using these weekly updates, you might not necessarily see so much difference between versions, depending of course on your particular use case. But believe me when I say that we are not sitting still :)

Anyhow, there are a couple of problems we're looking into now:

  • first, performance problems with really big folders (ie., many thousands of mails). We're trying to come up with a solution, but it's not easy (as is the case with most interesting things in life). Please bear with us;
  • second, problems with specific servers. Here you can help us! We're testing modest with different POP/IMAP/SMTP servers. But, there are many more different ones in the world, with a wide variety of versions, configurations - a combinatorial explosion. If your server doesn't play nice with modest, please file bugs with all the details (server, version, configuration,...). Also, protocol traces or PCAP-files (tcpdump/wireshark) are very useful, as are test accounts. Remember, if we cannot reproduce it, we probably can't fix it. If there's information you don't want to share with bugs.maemo.org, you may also mail me directly. That does require you to trust me, though.
  • finally, we've seen some problems with rare emails not being shown correctly. Again, if you get such an email, please file a bug, and attach the email (after stripping it of any privacy-sensitive information of course).

Anyhow, for the large majority of users, modest seems to be working quite nicely; if you haven't tried it yet, I invite you to give modest a try, and tell us what you think.

Syndicated 2008-01-06 14:21:00 (Updated 2008-01-06 19:44:54) from djcb

it was a very good year

(and it's not even over yet)

The end is nigh - the shortest day of the year. Next week, I'll be spending some time in the lovely city of Amsterdam, enjoying various Christmas-related festivities. No matter how nice it is here, it will be good to leave Helsinki behind for a couple of days, and do something different.

At last, the OS2008-images have been published. A big improvement, I should say, and there should be little reason not to upgrade... One more reason to upgrade, is that you can then install modest :-) Just this afternoon, we've published updated packages. I also look forward to play with Canola2, created by those talented Brazilians. It would be even better if they could release it under the GPL, as has been suggested.

Yes¸modest... Even though our public public bèta is not even two weeks old, we have a lot to look back on. It's been an incredible year, even just looking at the modest-microcosmos. I am very proud of the the team, a group of truly amazing individuals. Nevertheless, we won't rest on our laurels, and look forward. The latest release (Friday 21.12) should be the best so far, and will also be the last of 2007. As I said before, if you find bugs, or feel something is missing: please visit bugs.maemo.org (component: Communication/Modest). We've already cleared up quite some of the issues that were reported - your voices are being heard! In 2008 we will return with our batteries fully recharged...

Merry Christmas to All!

Syndicated 2007-12-21 20:35:00 (Updated 2008-01-26 14:27:08) from djcb

feed the world

Yet another modest-post. We received a lot of feature requests and a handful of bugs after our repeated requests. We're working hard on fixing the bugs, and let's see what we can do with the feature requests. It's not easy to find the right balance between keeping things simple and, on the other hand, adding features that people ask for. Being a long-time mutt and emacs user, I have some troubles keeping things simple & easy. But hey, nobody said it was easy.

Let's discuss something that I found quite useful: reading feeds with modest.

I like reading feeds; whenever I need to wait for something (e.g. waiting for the food during a romantic dinner), I like to read something interesting - software, science, culture, whatever. Now, while there is a feedreader on my N810, obviously I'd prefer to use modest for that.

Well, that's perfectly possible using feed2imap. I am assuming a Ubuntu/Debian system here, but it should work for other systems as well, mutatis mutandis.

  • install feed2imap and libopenssl-ruby on your system;
  • create a ~/.feed2imaprc file, it should look something like this (assuming you're using Gmail/IMAP):

    - name: Planet Maemo
    url: http://maemo.org/news/planet-maemo/rss.xml
    target: imaps://someuser%40gmail.com:somepassword@imap.gmail.com/feeds/software
    - name: Ririan Project
    url: http://ririanproject.com/feed/
    target: imaps://someuser%40gmail.com:somepassword@imap.gmail.com/feeds/misc
    - name: Boing Boing
    url: http://www.boingboing.net/atom.xml
    target: imaps://someuser%40gmail.com:somepassword@imap.gmail.com/feeds/misc
    - name: ScienceDaily Headlines
    url: http://www.sciencedaily.com/newsfeed.xml
    target: imaps://someuser%40gmail.com:somepassword@imap.gmail.com/feeds/science

    Obviously, you'll have to change the username/password;
  • Note: if your are already using a different feedreader (such as Liferea), you can export your existing feedlist to an .opml-file. You can then convert this to a config file for feed2imap:
    $ feed2imap-opmlimport ~/feedlist.opml > ~/.feed2imaprc

    (you'll have to hand-edit it a bit as above)

That's it! Now you can periodically run feed2imap, and you'll always have something interesting in your mailbox.

Syndicated 2007-12-17 19:49:00 (Updated 2008-01-26 14:27:28) from djcb

take a chance on me

First, thanks for all the very positive feedback we got about our initial modest release. This was only the start, we'll be making frequent updates - the first one was already today. If you face any particular problem, please let us know.

Some questions I saw in various places:

  • can I get it to work on OS2007? Well - in principle that should be possible, as we tried to keep the code backwards-compatible with the old API. In fact, some people claim to be happy users. However, we haven't tested that recently, and there might be some missing dependencies, so YMMV. But we'll be looking into this issue.
  • I'd really like modest to do $FEATURE, but it doesn't do $FEATURE. What can I do? Well, patches/suggestions are welcome; please discuss them on our mailing list first, or put them in Maemo Bugzilla;
  • how can I replace the official client with modest? There are some adventurous people out there... Modest might replace the current client at some stage, but right now, it's not really easy to do that.
  • where can I report bugs/feature requests? Please report them in Maemo Bugzilla, and make sure you use Communication/Modest (don't use Communications/Email, that's for the stock email client). Please specificy as clearly as possible the problem, and how we could reproduce it. Some people already filed bugs there, and even better, some where already fixed!

We're working very hard on fixing bugs and making other improvements. Modest is not perfect and there are bugs. But already, I am a quite happy modest-user myself. And feel inspired by modest and Vagalume-hacker Berto, for the incredibly flattering words, especially on my birthday!

Syndicated 2007-12-14 22:40:00 (Updated 2007-12-16 14:09:53) from djcb

automatic for the people

Ok... the modest email
has been available for months, but finally today we're making downloadable packages available for OS2008/Chinook: click here, or visit our website which has instructions.

That means that you don't have to compile things yourself anymore¸ but simply click & install - hurray!

Some important notes:

  • this is an early bèta which contains bugs; this is not production quality yet; we will be making frequent updates though;
  • modest will not replace the official email client, and many things (like search and send-as-mail) use the official email client; new-mail notifications should work though;
  • in the menus, modest is called 'E-mail' and uses the same icon as the official email client does... be careful not to confuse the two;
  • modest works nicely with GMail (both IMAP and POP), just make sure you enable POP/IMAP in your GMail settings on the web, and use user@gmail.com and not just user as your user name.

Having said that, I feel very good about what we've accomplished so far. I'd like to thank the talented hackers at Igalia for all their inspiration and perspiration (read the Igalia/modest news), the inimitable Philip Van Hoof for his work on Tinymail. Also, the fine people at Openismus made important contributions, as did Vivek, Mox and many of my colleagues at Nokia in different areas.

In a previous blog entry, I already discussed some of the features modest offers. I think it's pretty cool. Maybe not as cool as Canola2, but definitely a lot better for reading/writing email :-)

Remember, modest is 100% open source software, released under a BSD-like license. It's not perfect, but we're working hard to make it the best mobile e-mail client. Patches are welcome, as are suggestions -- post them on the feature request tracker. Even bugs are welcome in Maemo Bugzilla, and make sure you use Communication/Modest (don't use Communications/Email, that's for the stock email client). Specificy as clearly as possible the problem, and how we could reproduce it, thanks.

In some future blog entries I will discuss various cool features in Modest, but try it out yourself, and let us know what you think!

Syndicated 2007-12-11 18:44:00 (Updated 2007-12-12 20:54:22) from djcb

beyond the dark sun

Friday -- another week has flown by. Winter has not really entered Finland yet, there's just ultragray autumn weather, a lots rain, few hours of daylight, not even some Finnish wintersun... But thankfully, my moods are not very dependent on the weather, and I am happily hacking away still.

  • Googler Steve Yegge is blogging about combining Javascript and Emacs. Naturally, I am interested in just about anything combined with emacs, but this sounds quite interesting; not only about fully supporting javascript in emacs, but actually being able to write extensions in javascript (instead of elisp). Apparently, he already implemented a full javascript interpreter in elisp. Very cool... but show us the code!
  • I've been playing with Alberto Garcia's LastFM-client Vagalume; it works very nice on my N810 (versions for 770, N800 are available). last.fm is pretty cool anyways, but the vagalume makes it work very smoothly. Obviously, I am hdbngr there :-)
  • We've been working on the downloadable packages for the modest email client, it's almost done, stay tuned.
  • Said goodbye to my friend Andrea, who's going back to Italy. I'll miss him as a person and as a great host for Italian food... At least he will be in good hands there. Arrivederci!

Syndicated 2007-12-07 23:46:00 (Updated 2007-12-08 21:58:11) from djcb

139 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!