Older blog entries for djcb (starting at number 155)

it's all greek to me

It's been a while since my last blog entry... I haven't done much work on modest lately, but it is in safe hands. I did start a new little hobby project though; it's called mu, and it's a collection of command line tools to index / search e-mails stored in Maildirs. It doesn't run on N8x0 (yet), but I guess it wouldn't be very hard to port it. Of course, this kind of software has been written before - but for a hobby project, that does not really matter. It's all about trying things out.

I am taking notes about the things I learn as I go along... there's a lot of optimization stuff to discuss but unfortunately, it's too much to fit into this blog entry... will write about that later. I am off to Greece now -- to corrupt the youth of Athens. I hope I can understand the people; I taught myself a little bit, but rumours have it that the language has changed quite a bit in the last 2500 years...

And not to forget: happy birthday, GNU. 25 years... I may not always agree with RMS, but he deserves the greatest respect for his accomplishments. A George Bernard Shaw quote comes to mind:


"Reasonable people adapt themselves to the world. Unreasonable people attempt to adapt the world to themselves. All progress, therefore, depends on unreasonable people.".

Syndicated 2008-09-28 08:45:00 (Updated 2008-09-28 09:30:02) from djcb

my name is nobody

It's great to see the improvements in the modest e-mail client. For most people, there should be little reason still to use the old email client. Great thanks to all involved -- my friends from Spain, Belgium, Germany and elsewhere; Vivek, Mox, and all users, contributors etc. It's good to mention contributors sometimes; I found the CNN Money-article about the N810 development team a bit off-balance in that respect - it would have been nice to include some people who write the software, too.

Anyway, back to modest. I'm sure that someone, somewhere is missing some feature that is essential to them. While usability and feature-richness are not necessarily conflicting, in practice they often are (I know, I'm a mutt-user!) But, no excuses -- in my (slightly biased) opinion, it's a nice little e-mail client and a great improvement. And with the code being open and free, there is nothing stopping people from firing up their favorite text editor and start hacking on their missing pet feature.

My personal role in the modest-project will diminish a bit. I'll be slaying some new dragons - still Nokia, open source, yadayada; I'll write a bit more about that in the near future. I feel that modest will continue its life in trusted hands, and of course I'll keep an eye on that ;-)

Syndicated 2008-05-18 10:13:00 (Updated 2008-05-18 10:57:41) from djcb

the thing that should not be

Just a short note: due to an unfortunate regression, Modest (version: W16 release) does not work with SSL/TLS, breaking providers such as Gmail. See bug 3084. The reason was that what we tested with, differs slightly with the Chinook environment, and so this one fell through the cracks. Mea culpa... Anyhow, the problem has been fixed. If you build things yourself, get the latest (tinymail and modest) and all will work fine. If you don't want to do that, you'll have to wait until Monday; unfortunately, we can't do anything before that.

Once more, apologies from the Modest team for the inconvenience.

Syndicated 2008-04-18 13:53:00 (Updated 2008-04-18 14:04:49) from djcb

images and words


These are interesting times... I just found out that the next revision of our internet tablet will have WiMAX-support. Rest assured - modest will support that as well.

Also, recently I have been studying the (very much recommended!) work of Edward Tufte, on the visualization of data, and how modern technology is great at obfuscating real meaning behind snappy graphs. Still people are trying to generate meaningful (or sometimes just pretty) pictures out of masses of data. One of the masses of data being email messages. Check these great post on FlowingData which show many different visualization of email data. For a more practical example, look at MailTrends, which analyzes the emails in your Gmail account for your.

et tu, emacs?


I was very happy to see prebuild Emacs packages for Maemo. I wonder if my instructions are still valid, especially regarding key-bindings on the N810. Anyway, I'd be interested in the next steps in integration Emacs with the platform. I'd like to connect the HW-zoom buttons to zooming the fonts in Emacs, and maybe marry the emacs-server setup with the application menu -- ie., don't use new emacs instances for new files, but instead use new buffers in the existing instance. Now all I need is a little time.

Syndicated 2008-03-31 21:31:00 (Updated 2008-04-01 20:06:59) from djcb

recreation day


Photo from Evergrey-concert, yesterday 20.03. Excellent music from the Swedish rockers; I've known them for years, but this was the first time I saw them live. Great concert, very talented band, and they were nice enough to do an autograph session afterwards; I even got into a picture with the guys -- slightly embarrassing...

Time for an update... Our beloved modest e-mail client is humming along happily. We're putting a lot of energy of testing all kinds of use cases, as well as weird error conditions. Modest/tinymail contain quite some code (in total around 240K lines), so there are a lot of things to test. Anyway, I was already quite happy with our first b├Ęta release, back in December. And modest has seen solid and consistent improvement since, every single week (with a few regressions thrown in to keep things interesting...).

Also, I have been quite happy with my emacs-on-N810. It has turned my N810 is a versatile PDA. I'm slowly capturing the power of org-mode in Emacs (see the 25 minute video), which is an amazing way to handle todo-lists, GTD and so on.

Then, there is so much happening in free software land, it's hard to keep track of it, even if just looking at the level of fundamental tools. Some things that I found quite interesting:


  • Dehydra/GCC is a plugin for gcc (Javascript!) built within the context of the Mozilla project. Mozilla uses an object system called XPCOM, which is 'inspired' by Microsoft's COM. However, times have changed, and in many place in the huge Mozilla codebase, this XPCom is seen unnecessary bloat and complexity. For example, in COM-style, one uses the return value of a method for error checking; the 'real' return value comes as an outparam. However, in many cases (see DeCOMtamination), it's much better to use a normal return value, and use e.g. exceptions for error handling. Now, try to do that automatically, taking into account possibly misuse of outparams -- sometimes, sed/awk/perl are just not enough. And that is where DeHydra comes in.
  • gold, the new & improved GNU linker. It's good to see that even classic tools like ld are still being improved -- and quite significantly in this case, esp. for speed. What we're still waiting for is link-time optimization, which can significantly speed-up programs, e.g. by making sure the most used functions are in the same memory page.
  • quagmire (giggedigig!), finally an autotools-replacement projects that seems actually capable of doing so. The initial goals is to replace automake and libtool with a bunch of GNU make macros. By simply requiring GNU make instead of a 'normal' make, a lot of the hackery autotools disappears. Another nice thing is that it understands pkg-config, which simplifies another set of problems. Apparently, the longer term goal is to replace autoconf as well. And, given designer Tom Tromey's track record, the future looks bright.

Syndicated 2008-03-21 14:37:00 (Updated 2008-03-21 17:18:58) from djcb

come together


I've returned to Helsinki after visiting the FOSDEM-conference in Brussels. Before anything else, I'd like to thank and compliment the organizers for creating a great conference. And not just the organizers, all the volunteers that made it another great FOSDEM. The amount of work that goes into something like this cannot be overestimated, and it all went very smooth. If anything, it was too succesful, so many people...

Anyway, I had the chance to meet a great many old friends, as well make new ones. It's fantastic to see all the free software projects that improve things all over the software stack. Kernel, console, X, web services, funky UI bling, end-user applications, embedded software,... So much combined brain power, pushing the envelope of free software.

I did a presentation (should be available soon) of our own little addition to that, the modest e-mail client. Although there was some delay (Murphy!), I was quite happy with my talk. And it was particularly interesting to talk to modest users - what do they like, what do they miss, and so on. Overall, we've been blessed with very helpful users, and with there assistance, we were able to kill quite a number of bugs which would have very hard to fix otherwise. See our resolved buglist in our bugzilla for some great examples of that.

Anyway, overall FOSDEM made me quite happy -- such a gathering of smart people and great software, promising a lot of good things for the free software future.

Note, screenshot is of the unstable, proof-of-concept GNOME desktop-version of modest, courtesy of dape.

Syndicated 2008-02-25 20:34:00 (Updated 2008-02-25 20:44:10) from djcb

heeding the call

This coming weekend, a big part of our multinational modest-team (Sergio, Berto, Philip and Dirk) will be at the FOSDEM conference in Brussels. There will even be a presentation about the modest e-mail client. So, if you have any questions, suggestions or even some constructive(!) criticism, this is your chance! You can use IM if you're looking for me (diggler[at]gmail.com).

Last year, I had a great time at FOSDEM, with many, many interesting people as well as the nice atmosphere (food and drinks) in Brussels. Hope to see many of you there!

Syndicated 2008-02-21 16:16:00 (Updated 2008-02-21 16:16:40) from djcb

waiting for 22


Achilles had Patroclus. Don Quixote had Sancho Panza. Michael Jackson has Bubbles. And I have emacs. On my N810. A while ago, I already wrote about it. I even showed some screenshot of emacs running in scratchbox. But, I didn't take the final step - getting it to run on an actual N810. Recently, I tried to get that to work. Well, that was frustrating... Some hackish instructions follow. They may or may not work for you -- try at your own risk :)

  • First I tried to simply rebuild the Emacs23-packages in scratchbox; that failed, because the compilation somehow crashes QEMU;
  • Then I tried Emacs 22 instead... but the problem remained;
  • So, I decided that maybe I should try to compile the package it on the N810 itself. Again, that failed. One of many problems: package building requires a real grep, and if you try to install it, it wants to remove the whole busybox environment; sigh. I fought the system - the system won...

But, all was not lost. There are prebuilt Debian packages of Emacs22 available; I took the armel-packages from there, and tried to install them. That almost worked. Almost, because the size of emacs is almost legendary. It did not fit on my root file system on the N810.

We're nearing the solution though; I copied the contents of the .debs to a directory emacs810 (with mc); then I copied this directory to the MMC-card of the N810 (/media/mmc2/). I set some symlinks, ie.


# ln -s /media/mmc2/emacs810/usr/share/emacs /usr/share/emacs
# ln -s /media/mmc2/emacs810/usr/share/emacs22 /usr/share/emacs22
# ln -s /media/mmc2/emacs810/usr/bin/emacs22-gtk /usr/bin/emacs22
# ln -s /media/mmc2/emacs810/usr/share/applications/emacs22.desktop /usr/share/applications/hildon/

Also, you'll need to install libungif4g. That should do the trick, and emacs should show up in your Extras-menu. And we can run emacs! Victory is mine!

Well, almost. In emacs, a very useful key is the Meta-key, usually mapped to the Alt-key of your keyboard. But of course, there's no alt-key on the N810-keyboard. Instead, I decided to remap the Chr-key. I'd like to remap it in my .emacs, but I haven't been able to do so.

Anyway, as a first start, I added these to /usr/share/X11/xkb/symbols/nokia_vndr/rx-44 (the xkb keyboard mapping):


key <SPCE> { [ space, space, Tab, space ] };
key <COMP> { [ Meta_L, Meta_L, Multi_key, Meta_L ] };

modifier_map Mod1 { Meta_L };

The first one will turn Fn-Spc into Tab, which is very useful for completion (for some reason, I couldn't get M-i working in the minibuffer). The second one will turn the Chr key into the M-key (obviously, you can't run emacs without that), with Fn-Chr giving the old Chr key. Not sure what it will break - it's black magic.

Ok, that's it. These steps should be cleaned-up, pre-packaged and made single-click-available. Anyway, the steps above should hopefully get you a working hand held emacs. Happy hacking!

Syndicated 2008-02-18 21:01:00 (Updated 2008-02-18 21:40:45) from djcb

be quick or be dead

Note, image has not much to do with this post, it's emacs22 running on my N810. More about that in my next post


For a while, we've been doing semi-weekly updates to modest, usually on Friday's. And every week, we're fixing many issues, big and small. Modest has grown quite a bit, and modest + tinymail/camel has about 240K lines of code. That's quite a haystack for bugs to hide, but I'm quite happy with the speed at which we've been able to squash them. The big fix of last week was adding support for maemo-launcher, due to heroic hacking efforts by dape. Maemo-launcher significantly improves startup speed. How does it do that? To answer that, let's look at what happens at application startup time.

still haven't found what i'm looking for


Few people have the time to write a UI-toolkit, or even printf(3), for every piece of software they develop. Thankfully, we can reuse libraries to do such things for us. The most common form ar dynamic libraries. With the ldd(1) utility, you case see which ones your application uses. For example, for modest:

[sbox-CHINOOK_X86: ~] > ldd /usr/local/bin/modest
linux-gate.so.1 => (0xffffe000)
libgtkhtml-3.8.so.15 => /usr/lib/libgtkhtml-3.8.so.15 (0xf7ef9000)
libtinymail-gnomevfs-1.0.so.0 => /usr/local/lib/libtinymail-gnomevfs-1.0.so.0 (0xf7ef6000)
libtinymail-maemo-1.0.so.0 => /usr/local/lib/libtinymail-maemo-1.0.so.0
.... (69 others) ....

Now, when we start modest, we must load these libraries. Suppose, somewhere in the code, we have:

magic_check = gtk_check_button_new_with_label ("Enable magic");

When starting the program, the dynamic linker will now have to figure out at what memory location gtk_check_button_new_with_label, is to be found. And not just that function... if we look at modest, we can find the number of external function (or more general, symbols), with the nm(1) utility:

[sbox-CHINOOK_X86: ~] > nm -u /usr/local/bin/modest
(...)
U gtk_check_button_new_with_label
U gtk_check_menu_item_get_active
U gtk_check_menu_item_get_type
U gtk_check_menu_item_set_active
U gtk_clipboard_get
(...)

In total, there are are almost 1300 external symbols in just modest; and this is only a fraction of the total, as GTK+ will have a lot of external symbols as well (think Glib, Pango, ...). In total, there will be many thousands. Without going into the details, it takes a significant amount of time to do the symbol lookup. Even on a fast desktop machine it can cause noticeable delays (esp. for C++), and more so on 770/N8x0.

Another factor that affects application startup significantly is initialization: in GTK+-based applications, e.g. gtk_init takes quite some time. In particular, applying the GTK-theme is slow on 770/N8x0, because the default theme file (gtkrc) is huge: the default theme on Ubuntu ("Human") contains 242 lines, but on my N810 default theme has 7046 lines, on much slower hardware. And note that the theme is very picture-heavy, and many little images must be loaded. To get an idea of how much work must be done you can use strace(1) when starting an application... scary stuff.

And finally, another slowdown is the physical loading of all these libraries into memory. This will typically only happen the first time, as Linux will keep the data around as long as there is enough memory (note, for testing, you can force a flush with echo 3 > /proc/sys/vm/drop_caches).

i remember now, i remember how it started


What can we do about all this slowness? Enter maemo-launcher.

Maemo-launcher is a daemon that loads common libraries at startup, does a gtk_init for you, and as such, the price for doing all this work (as explained above) is only paid once. Your actual application is compiled as a dynamic library. When your application is started, maemo-launcher forks, this dynamic library is loaded, and we jump to straight to its main-function. This totally gets rid of the initializations, theme loading and such mentioned above, and saves quite some startup time.

How much are the improvements you can expect? Johannes Schmid did some testing, and found improvements of about 25% for small programs. Some non-scientific testing for modest shows that the startup-time is 1-2 seconds faster. Does that matter? Well, let's look at our 10 million ;-) modest users, all of them starting modest once a day. With a 1 second improvement, every day we save almost 4 months of time! Jokes aside, application startup times are very visible to users, and really determine whether they consider your software fast and snappy, or huge and slow; it's time well spent trying to improve that.

parting thoughts


For a very practical way of how to get maemo-launcher working with your program, see the Appendix of the MaemoMM-tutorial. It also shows how to add additional libraries to the set loaded and initialized by maemo-launcher. Recommended reading.

Note; nm as shown above does not work with stripped binaries (i.e. the binaries of which the symbol got removed). And unless you compile everything yourself, most of the binaries on your system will be stripped. However, you can still get much of the information with objdump -R; however, it it's not equivalent to nm -u, and will contain some non-external symbols.

Finally, this is the kind of post that could contain embarrassing factual errors :-) please check the comments for updates.

Syndicated 2008-02-17 11:04:00 (Updated 2008-02-17 21:59:39) from djcb

battle against time


Like many people, I get a lot of e-mail every day, and I spend a very substantial part of my time processing them. In order to keep things under control, I'm following a number of practices, which I'll describe in the following. As with most good ideas, it's not my own idea -- it's inspired by a time-management method called Getting Things Done (GTD). More about that later.

inbox zero


So how to become in control of our e-mail again (instead of the other way around)?

The most important rule here is that we should always end the working day with an empty inbox. Fantastic as that may sound, it's really the key here. So, how do we do that?


  • At least once a day (and before leaving the office), review the messages in your inbox. For each message, decide what to do with it:

    1. If it requires some action:

      • if it takes less than two minutes: do it now;
      • If someone else can better handle it: delegate (forward) it;
      • otherwise, move it to a folder NextActions;

    2. If might be interesting or important later, archive it
    3. Otherwise, it's just crap - delete it.

  • After this review round, your inbox will be empty. No message stays behind - look at them, and decide where to move them.
  • Now, periodically check your NextActions, take the necessary actions, and move messages to your archive after completing them. (I've found that maybe only one-third of my incoming e-mail end up in NextActions)

That's all there's to it. Following these simple rules, it will be much easier to deal with lots of e-mail. If you have 250 e-mails in your inbox, some of which are weeks old, it's really hard not to forget something important; also, it costs a lot of time to scan those same messages again and again (I'd say handling a mailbox with n mails has at least O(n2) complexity :-).

Of course, this is just a starting point; you could add WaitingFor-folder with a copy of mails you delegated (forwarded) for tracking. You could somehow add the calendar. You can add other to-do items. You could think of some smart way to manage those items in NextActions. And so on... All of those things are discussed in GTD.

gtd


Getting Things Done (GTD) is David Allen's method of time management, which deals with much more than just e-mail. The steps above are a sort-of GTD-light that works very well for my e-mail. Anyway, I can really recommend the GTD-book for anyone interested in time-management. It's a rare gem in the sea of 'personal-productivity' books. Read an overview on the 43 Folders-website, or their excellent article on inboxzero, which has lots of additional tips.

I've been using GTD (and especially these e-mail handling practices) for quite a while now, and it has really saved me a lot of time and frustration. I'm quite sure this will work for a lot of people; but of course, the only way to really find out is to try it out yourself. On the internet, GTD enjoys an almost cult-like following on the web - don't let that scare you away. Just be skeptical, and use what works for you.

epilogue: gtd on n8x0


You could implement quite a bit of this on your N8x0 and (surprise) modest, and I have done so quite succesfully. What's still missing is some calendar integration, and an easy way to handle (search through) huge archives of messages. Anyway, being so mobile, the N810 with modest has become a great productivity tool for me.

One geeky alternative would be to use Emacs; people like Sacha Chua and others have written a lot about using Emacs for Getting Things Done. Now all we need is to port emacs to the N810 -- project maemacs to the rescue! Or maybe someone could make an N8x0 version of Chandler? (unfortunately, the book is still better than the program).

Syndicated 2008-01-26 12:34:00 (Updated 2008-01-26 14:26:14) from djcb

146 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!