Advogato blog for DV

10 Apr 2008

Thu, 10 Apr 2008 09:57:35 GMT

Releases

I pushed a bunch of releases on Tuesday, trying to catch the Fedora-9 train (I nearly missed it, it led to a not so fun curl_is_failing_to_upload debug session which led to nss3 for firefox3 is not compatible with nss3 for fedora8 curl), thanks to everybody who helped catch that train !

The releases are mostly bugfixes, libvirt-0.4.2 leading the pack, but libxml2-2.6.23 has a lot of fixes too thanks to various people reporting bug and giving patches, notably the Huawei team. Libxslt-1.1.23 includes the dozen or so fixes since last summer.

Developments

Clearly libxml2 and libxslt are in maintance mode, the focus is on libvirt, maybe I will just add support for the latest Proposed Recommendation of XML-1.0 in libxml2 before the Summer.

For libvirt, clearly we need to extend the number of hypervisor supported, maybe update and clean up the OpenVZ support too. IBM is actively contributing the Linux Container driver, I just commited a second set of patches today, you can expect good support in Fedora 10 I guess. On the high end side Sun just posted the patches for the lDOM virtualization on their Niagara based machines, lot of patch reviews those days. I also want to get a complete set of bindings for Java integrated, and now that Fedora java packaging guidelines are out, this is a good opportunity to add this.

History meme

that one is interesting, here is my contribution, as you can see I'm an old fashionned old fart, main workstation at home:

paphio:~ -> history | awk '{a[$2]++ } END{for(i in
a){print a[i] " " i}}'|sort -rn|head
319 vi
257 ssh
255 cd
156 cvs
130 make
125 ls
79 svn
60 scp
48 su
43 ping
paphio:~ ->

and on my second workstation in Annemasse:

wei:~ -> history | awk '{a[$2]++ } END{for(i in
a){print a[i] " " i}}'|sort -rn|head
362 vi
263 cd
262 make
136 cvs
115 svn
105 ssh
78 ls
67 scp
40 xmllint
38 grep
wei:~ ->

5 Feb 2008

Tue, 5 Feb 2008 11:13:26 GMT

Happy new Year

yeah it's either really late or a bit too early for Chinese New year, proof that I should blog more often !

backlog

I have been away for the last month and a half, first vacation in China, and then in Paris for Solution Linux expo. As a result an awful lot of mail piled up in various folders that I didn't really had time to process, I am slowly trying to go though them, so if you get replies from a mail in November that's normal, or if you didn't get a reply, it's probably a good idea to ask again !

libvirt and Co.

libvirt development is progressing fast thanks to DanB, Rich, Jim and the many people porting, testing and using it. I really hope we will soon have easy support for most platform - including Windows and OS-X - at least as a client to remotely access the virtualization servers, Rich did most of the work. The development of the CIM provider seems to be progressing nicely thanks to DanS and the IBM'ers :-). During last week Solution Linux i got asked a few times about P2V tools i.e. tools allowing to automatically save the state of a server to allow to start it as a virtualized domain, Rich just released virt-p2v 0.9.1, it is still experimental, what it needs now is a wider range of testing to find and remove limitations, give it a try and provide feedback, thanks !

For libvirt core we still have a number of pending extensions, like support for new hypervisors, adding APIs for storage management which have been worked on by DanB, we also need to finalize a good XML scheme for 'partition' based virtualization, probably update OpenVZ, and add sound and more USB support. I will post on the list soon to try to clarify the roadmap, much work ahead !

Solution Linux

SL 08 was last week, it's the Linux event of the year where I'm sure to go, it was good seeing people around the Fedora and Gnome booth, and of course Red Hat one. My presentation on virtualization went well, even if it was more than 2 hour long, the PDF is available and a lot of people seems to have fetched it already, I assume it's a positive feedback from the audience :-)

China

I was for a month in China, what a blast! It's impossible to try to describe in just a few lines, but we went from the very cold Dalian and Beijing to tropical Xishuanbanna, most of the time within the chinese family and environment, picking only a few selected tourist spot. Things like Shanghai museum, the Great Wall or the Forbidden city are definitely must-go, but to me the most amazing was to stay and live with the chinese inside the family. The cultural gap is huge, there is so much to learn, too bad mandarin is so hard to an occidental brain (and ears), very challenging, but also very fun and enjoyable !

29 Jul 2007

Sun, 29 Jul 2007 08:33:01 GMT

coral bleach

ncm, I think this may be an error of interpretation, the coral turns white when it dies, I mean when the organism dies. They form a symbiosis with alguaes and those give the color, white coral appears when the animal dies, you just have the skeletton left. White coral (with a few exceptions which are naturally white) is just like the bones left from a dead animal, it may result from increased acidity of water, but also from any other deadly condition. For example when overheating, if water temp reaches around 42 degrees, the coral rejects the alguae then dies, it can be quite dramatic because suddenly very large areas whithen suddenly, it's all dead and will take maybe half a century to get back in proper shape :-(

libvirt progresses

libvirt development progresses rapidly, as the traffic on the list can testify, we now have native secure remote operations, and there is ongoing work to add OpenVZ support. I hope to increase coverage of more virtualization engine, so far it looks good, we don't seems to have serious API design problems.

gamin

I made a new release 0.1.9 last week with some bug fixes and portability improvements.

19 Jun 2007

Tue, 19 Jun 2007 16:08:12 GMT

One year in review

I didn't blogged for a year precisely. No I didn't hide in a hole or something but I just didn't really feel the need. I still did a few things:

the libvirt project is my main work item, 15 releases in the last year, now supports QEmu and KVM, many contributors and well people are using it, so that's good.
the libxml2 and libxslt libraries have seen their development slowing down, partly because they are mature enough, partly because I have less time, and Kasimier one of the main contributor lost his job and left the project. But Bill is still around and helps a lot !
travelling, I had the opportunity to travel for work and also for vacations, went to Singapore twice, also Malaysia, had some fantastic time in Borneo, visited Bangkok, but didn't like Manilla too much. Also spent some vacations in Costa Rica, and went to USA and Canada for work. I really enjoy going to Asia and meet people there.
I am getting crazy about orchids, my collection is growing fast, maybe a bit too fast, but I'm rewarded by fantastic flowers, and I'm trying more and more to travel to places where I can see them growing in their natural habitat, unfortunately they are seriously endangered, and it can be fairly hard to still find them in the wild.
I also went to badminton training again this year but competition is really getting a bit too hard, and my body disagrees at time, I think I will slow this down a bit next year.

Overall quite a bit happened but no revolution, the most scary thing is that time flies faster and faster, so no promise about increase of blogging frequency !

19 Jun 2006

Mon, 19 Jun 2006 11:05:21 GMT

On broken XML processing with libxml2

The XML spec is very clear about this:

Once a fatal error is detected, however, the processor MUST NOT continue normal processing (i.e., it MUST NOT continue to pass character data and information about the document's logical structure to the application in the normal way).

That's the default behaviour of libxml2, but for the purpose of helping recover occasionally corrupted data, I provided an XML_PARSE_RECOVER parser flag allowing to correct some of the trivial errors, for example if using xmllint --recover. Now the problem is that people start using this flag for default processing, and that's something I said many time I didn't want to happen. If blog generator can't output correct well formed XML, then they need to be fixed (and that's true for all other XML generators). The intent of the drastic rule in the XML spec is cristal clear, to avoid data corruption and data loss by forcing non conformant behaviour to be fixed at the source, not at the consumer level.

I am gonna give a hard time to those who abused this feature of libxml2, because while not completely disabling the option, I will make sure it's not used repeatedly, I'm still looking at the best way. Feedback welcome on this issue, preferably on the mailing-list.

18 Jun 2006

Sun, 18 Jun 2006 21:59:52 GMT

half a decade of community knowledge gone

The archive of the xml@gnome.org and xslt@gnome.org are gone, I have no idea what happened, I didn't got a mail about it, just that going to the archives show all history is only the 4 last days. The only thing I have left right now are the words saved in the database used for the search engine on xmlsoft.org. The code is here, but the knowledge, the exchanges, all the process leading to it has vanished, I just can't express how sad this makes me...

[update] Apparently it was an indexing problem which should be relatively easy to fix and without serious effects. The post are still on-line and I now keep separate archives for more safety, the data is safe. To me OSS development is really about the process, i.e. the history, knowledge, feedback, the memory of that shared process is as important as the code itself. With that knowledge you can recreate the code, if you have just the code you miss out a large part of its semantic, and it can't be regenerated. That's why I don't believe in dumping large base of source code under a free licence can really compare to code coming from a genuine OSS/free software project.

12 Apr 2006

Wed, 12 Apr 2006 11:59:57 GMT

libvirt 0.1.0

Yup, another release from the canned penguins dept :-) . The API got extended a bit but it's mostly a large revamp of the internals to follow a driver model, so it gets easier to keep the code clean, and contribute support for other virtualization engines than Xen.

There is still quite a bit of work needed, for example acces the Xen Daemon though XML-RPC for versions of xen >= 3.0.2 (which is out apparently). I started incorporating patches coming from various contributors, I like this !

French invades Fedora Planet

it's fun, but a bit disturbing to see 50% of the posts in French. Bienvenue a bord ! Oh I don't believe blogs billets should be limited to a single language even though I stick to english only usually.

Ekiga-2.0.1 and FC5

Ekiga-2.0.1 is a clear success from my point of view, it's installed by default on Fedora Core 5, and so far I received only one bug report: I didn't switch avahi support on (I was a bit conservative, I will switch it on when pushing 2.0.2 when released). In general Fedora Core 5 looks great to me, I miss my old friends xv and xsysinfo, but rhythmbox is a great improvement, audio support is better in general. Unfortunately I can't run my desktop as a Xen Domain-0 because X crashes randomly every day or so :-\ . I also don't activate beagled by default due to the near continuous drive accesses it generates, even after a couple of days (i.e. when I would have expected its database to be initialized). Yum and Extras support works well, that is great too.

Spring is late

really cold lately here, we got snow on the mountains just a few hundred meters above Grenoble. I planted a bunch of new flowers in the balcony, but they are growing slowly due to this. There is a rose bud which is showing up anyway I guess everything will start blooming as soon as I leave for vacations in May ... On the other hand the orchids are in good shape inside, it's nice to have them blooming next to the desk. The habaneros pepers seeds germinated well and I should have some burning hot but delicious pepers this summer :-). I look forward vacations in Asia, hopefully I will be able to import new orchids, well if I manage to get through the french importation administative hassles...

13 Mar 2006

Mon, 13 Mar 2006 16:10:05 GMT

Ekiga-2.0.0

is out ! Congrats to Damien ! Packages are available for Fedora Core 4, and will be available for FC5 ... sometimes ...

libvirt

Slowly making progresses, next steps will be to drive QEmu instances and switch to XML-RPC for communication with the Xen daemon. Some serious internal code refactoring is needed.

libxml2

Went to the W3C XML plenary two weeks ago for the XML Core working group meeting, it was nice to see again a bunch people. Standard guys are a different kind of geeks, they don't hack code they hack specs (some do both !), and hacking the Web and XML can be really fun (or boring :-) ! I also went though all Coverity reports for libxml2 and libxslt, it was a PITA mostly due to the web interface to the reports, a couple of them would really lead to runtime bugs, but they forced me to be far more coherent about NULL pointers handling, static analysis is good because it can spot problem in code that your users will usually never run into (and if they do it won't be reproduceable !) So thanks to whoever paid for getting the analysis done :-)

Could someone explain locality trade-offs to Mozilla/Firefox developpers ?

Yes this may sound like a rant, it still is a very serious issue IMHO ...

Seriously guys before trying to push your stuff as a platform, fix your code. A program which balloons to half a gigabyte of virtual memory usage will not draw the web page I want to see any faster. But it managed to force me to reboot my box, linux handling of Out of Memory conditions is pathetic (I'm told there isn't good solutions to this, maybe), so give me my memory back when you don't need it for the computation of said page rendering. Oh and stop using X11 memory as a long term bitmap cache, this fools nobody, this DOES NOT HELP !

I have an http cache it's called squid and never uses more than 5Mbytes of RAM, I don't need another disk cache in Firefox to double I/O accesses and disk usage, I don't need to cache pages rendering, current processors are way fast enough to redraw them instantly WHEN NOT SWAPPING ! [Thot/Amaya was perfectly capable of instant rerendering on 10Mhz processors, large displays and complex page layouts]

For those frustrated like me by this nice looking memory pig, here is my recipe:

$  cat ~/.mozilla/firefox/default.*/user.js
user_pref("browser.cache.memory.enable", false);
$

restart firefox, you will see the following line when loading about:config in the URL :

browser.cache.memory.enable  user set   boolean  false

for good measure if you're using a cache like squid turn off browser.cache.disk.enable too by double clicking on the entry. It seems it make it a bit less glutton, but it still fuck with the X11 pixmap cache, X11 size still grows (and do not shrink when closing browser tabs !!!), if someone has a trick to beat gecko/mozilla/firefox in releasing the image pixmap as soon as they are not needed for display I would be very happy. An historical anectdote is that we had to phase out all the X11 terminals in the lab when Nescape started being popular because offloading of this pixmap cache onto the server memory led them to memory exhaustion and reboot of said terminals. This behaviour has been plaguing us for way too long, can't someone fix that code ?

14 Feb 2006

Tue, 14 Feb 2006 11:28:29 GMT

A new name and a new release

Following feedback I got, I renamed libvir into libvirt, apparently the first reaction to the old name was that it would be used to build viruses, bahh ! Rolled a new release 0.0.4 fixing the Python bindings now available on libvirt.org.

PalmOS using Gtk+ and Gstreamer

I don't know who worked on this but this looks seriously cool. Note how most of the bottom of their new stack is made of OSS components !

New beta release of Ekiga

Also in the cool section, Damien made a new beta release of Ekiga version 1.99.1, packages for Fedora Core 4 are available and it was also pushed just in time for Fedora Core 5 test 3. Good Open Source non-crippled software is the right answer to disgusting tactics used sometimes in the closed source world.

29 Jan 2006

Sun, 29 Jan 2006 15:06:07 GMT

blog

3 months without any blogging, many things happened in the meantime, let's see what is actually still worth sharing

Xen and libvir

work wise, I shifted from desktop related stuff to Xen and virtualization of the OS. I'm very happy to be back into more core OS related stuff, Xen+Linux is a disruptive technology, it just need to be stabilized and integrated into the whole system interfaces, desktop included. I'm bootstrapping a new project libvir with the intent to provide no-brainer, stable APIs to use those new virtualization features. Currently only Xen is supported, the set of APIs is not complete, but this is promising and I really hope to be able to drive more than just the Xen engine. Much fun, lot of work ahead too !

Gnome

As a result I'm less involved into GNOME, I'm not on the board anymore, I still think reducing the size was an error as there will be less manpower to tap into (I still didn't see the public minutes from the last board meeting), I wish the best to the new board and to the GUADEC team !

building stuff

I feel deep inside of me that I'm an engineer, I want to get stuff built correctly and used, software is great for that but it lacks the sense of a product, something you can touch, show, and use. That why I am so happy to have built my new kitchen, basically buying pieces of wood, planning how thing should assemble, cutting them, smoothing the angles, preparing the surfaces, drilling, glueing, assembling them, polishing the results... Took mee more than a month of my spare time plus a week of vacations, but I have built something real, that I use everyday (I do all my cooking), and which is exactly how I wanted it, that's very rewarding and compensates for the 'a bit too abstract' side of software.

Happy new Year

To our chinese friends !

Ekiga and Solution Linux

[update] I knew I forgot to post about something, first try Ekiga the replacement for Gnome Meeting, it is available into Rawhide, I made packages for Fedora Core 4 too, and while it was difficult sometimes to get GnomeMeeting working this version seems to fix the issue. Plus it's compliant with the SIP protocol, and should then allow access to a number of facilities associated with the standard.

I will also be at Solution Linux in Paris this week, presenting about Gnome (conference), and also about Xen (Red Hat booth),and will be around the Gnome booth too of course.