I've had hard drives die before, but this one seems to be doing it in an especially pathological way. It started with the kernel throwing irq timeout errors and then stepped up into actual read errors, culminating in a corrupt journal, a read-only block device and a forced fsck. It's been behaving a little better since then, though occasional io stalls (without any kernel error) suggest that it's having to repeatedly retry some sectors. SMART says it's all fine, so obviously I'm backing it all up now before a new disk arrives tomorrow and I can sort out access to the data centre. Email might be a bit spotty for me until then.

Turns out that running a 2.5" PATA drive for approximately 4 straight years may not have been the best of ideas. Who'd have guessed?

Syndicated 2009-06-25 21:38:42 from Matthew Garrett

I spent last week in the US, during which I discovered that my preconceptions of Albany as a post-industrial wasteland with no redeeming features was incorrect (it's actually a post-industrial wasteland with some decent bars), made minor contributions (mostly in the form of pesto) to a prize winning chalk picture and cycled on the wrong side of the road for the first time in a ridiculous number of years.

But more relevantly, I spent most of the week trapped in Westford. One of the things I've been looking into lately is USB autosuspend, a kernel feature which allows devices to be powered down when they're not in use. This is especially interesting for USB, since it's a poll-based protocol. If the end device is powered up then bus traffic will be generated, even if everything's idle. USB autosuspend allows this to be avoided and thus saves a worthwhile amount of power.

However, drivers need to support USB autosuspend before it's useful. Further, devices need to be able to cope with it. Some older kernel releases had autosuspend enabled by default, leading to all kinds of fun as people discovered that their scanners and printers had stopped working. This was fixed by leaving autosuspend disabled by default for most hardware. The first component that I'm working on is support for drivers to indicate whether or not a given piece of hardware supports autosuspend, allowing it to be enabled by default for that kernel. Handling this at a per-driver level means we shouldn't end up with obnoxious white or blacklists. The second is adding support for autosuspend to more drivers. Right now I'm concentrating on hardware that's common in laptops. There's support for autosuspend in the qcserial and uvc drivers, and some experimental (and unmerged) patches for bluetooth and some other modem drivers. Palm have just released their kernel code, which includes autosuspend support for the cdc-acm driver. I'm working on cleaning these up and getting them into a mergable state, and part of what I was doing in Westford was testing that various pieces of hardware work correctly. The good news is that they seem to, and with a bit of luck we'll try shipping support for a lot of this in F12. If that doesn't end up causing real world problems then it ought to be safe to merge it all to mainline.

The final component of this is handling autosuspend on devices with userspace drivers. Our only real choice there is for packages to ship udev fragments that enable it for working hardware. I've added support to the fprint package in rawhide, which means that fingerprint readers should now be powered down unless they're opened. Nobody seems to have complained yet, so I'm cautiously optimistic that this can be sent upstream without any problems.

Syndicated 2009-06-24 14:07:17 from Matthew Garrett

It's now been over 6 months since Poulsbo hardware with Intel's GMA500 graphics core started shipping in volume. And we're still utterly lacking in any sort of worthwhile driver. It's an impressive turnaround from the recent days when the straightforward recommendation for mobile Linux hardware was "anything that has lots of Intel stuff in lspci", and while the Poulsbo situation in itself doesn't change that hugely it's potentially symptomatic of a worrying trend within parts of Intel.

The first thing to realise here is that, like most large companies, Intel consists of a large number of business units with different priorities. Their open-source technology center has historically had responsibility for providing Linux support for hardware, but this obviously depends on other business units cooperating with them. And there's strong evidence that many of those business units don't get it.

There's been signs of this for some time. Back before the days of the Intel X.org driver gaining native modesetting support, some people ran the Intel embedded graphics driver. This was (is?) a closed X driver that was able to provide native modesetting on platforms that could only otherwise be run at incorrect resolutions. One business unit was shipping a driver that was more functional than the official Intel Linux driver. To the best of my knowledge, none of that code was ever used in the rewritten Intel driver that now provides the same features.

Poulsbo is another example of this. Intel wanted a low-power mobile graphics chipset and chose to buy in a 3D core from an external vendor. IP issues prevent them from releasing any significant information about that 3D core, so the driver remains closed source. The implication is pretty clear - whichever section of Intel was responsible for the design of Poulsbo presumably had "Linux support" as a necessary feature, but didn't think "Open driver" was a required part of that. There's not a lot any other body inside Intel can do once IP-limiting contracts are signed and the hardware's shipping, but it ends up tarnishing the good reputation that other parts of Intel have built up anyway.

And while Poulsbo is the most obvious example of this to date, it's not the only one. Intel recently decided to make the EFI development kit discussion lists private. Various drivers for Moorestown (the followup platform to Poulsbo) have been submitted to the Linux kernel, and while they have the advantage of being GPLed they have the disadvantage of being barely above the level of typical vendor code. Objections that chunks of them simply don't integrate into Linux correctly has done little to get these problems fixed - I still have no real idea how the runtime interface to power management on the SD driver is supposed to be used, but I suspect the answer is probably "badly".

This all makes sense if you assume that there are large groups of people in Intel who don't talk to each other. But to the casual observer it just looks schizophrenic. Explaining to an irate user that the Intel who shipped a closed Linux graphics driver is only barely the same Intel who contribute so much to architectural improvements in the Linux graphics stack doesn't make their hardware work. And while all of this confusion is going on, Intel's competitors are catching up. Atheros are now making significant contributions to the state of Linux wireless. AMD are releasing graphics chipset documentation faster than Intel, and radeon support is improving rapidly.

Is the future going to be one where we can no longer simply say that Intel hardware will Just Work? Is their work on Moblin (easily the most compelling Linux UI for netbooks) going to be wasted on the broader Linux community because it'll mostly end up running on hardware that's not supported by the mainline Linux kernel? Does Intel have a real commitment to open source, or is that being lost in the face of short-term requirements?

Intel need to demonstrate that they have a company-wide understanding of what Linux support actually means or risk losing much of what they've earned over the past few years. I'm desperately hoping that Poulsbo and what we've seen so far of Moorestown are the exception, not the future norm.

Syndicated 2009-06-23 20:41:44 from Matthew Garrett

Palm Pre

I'm in the UK and even if I weren't I wouldn't be using a CDMA phone, but the Palm Pre is undoubtedly an interesting device and so I've spent a short while looking at the root filesystem that can be extracted from the downloadable ROM image. There's some interesting things there. Bear in mind that this could be some QA internal image, so chunks of what I talk about may be wrong - on the other hand it seems to have been produced in May, so it's probably pretty close to what's shipping.
<h2>Hardware</h2>As others have figured out by now, the Pre is codenamed Castle and there's some references to another device called Pixie. This is presumably the Eos device that's expected to end up with AT&T. The other interesting thing is that the image contains modem firmware for both CDMA and UMTS. They're shipped as tarballs - extracting them gives you something that looks awfully like the firmware for the Qualcomm Gobi dual-CDMA/HSDPA chipset used in a bunch of modern laptops. The core firmware is an ELF object for 32-bit ARM. It's got references to Qualcomm in it and it's also got an embedded RSA signature which presumably cuts back on the potential for interesting hacking. The

/usr/bin/PmModemUpdater -e < /usr/lib/modem/castleumtsfw.tar

command, executed as root, should flash the UMTS firmware to the modem (castlecdma_evt1_fw.tar is the CDMA firmware). However it should be noted that the MSM6801A baseband used on the Pre seems to be a CDMA-only chip and there's no obvious place on the board for a SIM socket. My assumption is that the GSM models will have a very similar baseband with different radios. Unlocking the device will presumably be similarly difficult to the iphone - someone will need to find a flaw in the Qualcomm firmware that allows the network lock to be skipped. The PmModemFactory command will let you unlock the phone, but you'll need the appropriate code to do so and can (I guess) permanently lock it if you enter the wrong code too often.

The internal flash appears to present as mmc for some reason.
<h2>Software</h2>A lot of the software on the Pre is GPLed, and Palm are therefore obliged to provide copies of the appropriate source code to anyone who receives the software (note that receiving the device is not required - if the software is downloadable then the source offer must be made available to anyone who receives the software). The source code is not currently downloadable - instead Palm have included a written offer to supply the source code on request. This satisfies the GPL, but I can't be bothered doing that for the moment so all of this is purely based on examining the binaries.

The full list of applications extracted from the open source information documented included is here.
<h3>General</h3>The Pre appears to be an OpenEmbedded-based system. It uses ipkg for package management and the majority of the basic userspace apps are all from Busybox.
<h3>Kernel</h3>The Pre's running 2.6.24, and there's various references to Windriver. There's an internal git repository codenamed rockhopper, which is a little odd - rockhopper's the codename for a MIPS-based NEC evaluation board. It's probably coincidental, given that there's only so many species of penguin available. The version string includes "joplin" - there's evidence of this in mailing list posts dating back to last year, so it's not clear whether "joplin" is the entire WebOS kernel project or a codename that covers both the castle and pixie platforms.

The hardware-specific drivers look well-integrated, for the most part using the standard kernel interfaces (backlight, led and so on) - this is a marked contrast to Android, which tends to go its own way in this respect. There's a driver for the OMAP DSP, the exmap code for analysing application memory usage, oprofile for detailed profiling analysis, some crypto code, a driver for an SDIO-attached Marvell wifi chipset and support for being a USB gadget (ie, something that appears as a USB device when plugged into a USB host). Side note - depending on whether the system is a castle or a pixie, the g_composite gadget driver is given a different ID to let the host OS differentiate between the two. castle will appear with a USB id of 0x8002, and pixie 0x8012.

There's nothing else very interesting looking about the kernel. The source code should reveal more.
<h3>Booting</h3>The Pre uses upstart as its init application. In contrast to mainstream Linux distributions that still mostly use upstart in sysvinit emulation mode, the Pre appears to be almost entirely based on native upstart events. One of the things they've used this for is to automatically stop various services when the update service is started. Beyond that there's nothing exceptional here, but looking at etc/event.d does give insight into what's running on the device.
<h3>Userland</h3>In contast to Android, the Pre uses the standard GNU C library and associated userland. Filesystem layout is pretty typical Linux, with / containing the low level binaries and /usr containing the rest of the OS. The only slightly weird thing is that various utilities exist in both their busybox and full forms, and I'm not entirely clear on why it's shipping things like fsck.ext4 though I guess you could use an ext4 sd card or something.

Audio is handled through pulseaudio, while media is handled by gstreamer. There's a large collection of codecs for the DSP including WMA and h.264, but no obvious ogg support despite libogg and libvorbis being mentioned in the list of open source software shipped. Future plans? Pulled partway through development? Unclear.
<h3>ipod emulation</h3>The Pre pretends to be an ipod by switching into a different USB profile (and thus giving the right USB information) and setting up a filesystem that looks like an ipod. It provides a SysInfoExtended file that looks like an ipod. The only terribly interesting things about this are the list of formats (AIFF, MP3, WAV, AAC, AppleLossless, Audible, H.264, MPEG4 and H.264LC), the fact that it supports album art, the fact that it doesn't support Apple's DRM and that the firewire GUID looks like it's probably the same on all Pres. Could Apple break this support in a future version of itunes? Yes, but none of this support is hardcoded in the Pre's hardware - Palm could provide an update that matches. I don't see any real benefit to Apple in breaking the support, and if it could be shown that they were doing it deliberately then there could be potential legal issues.

The code actually looks generic enough that it could probably be made to emulate other USB mass-storage based media players if you wanted to. I have no idea why you'd want to.
<h3>misc</h3>There's a PalmOS ROM image for use in the emulator. usr/share/accountservices contains some files with interesting links in - one of them gives a full dump of the PHP setup on a Palm server, which gives some indications of their internal network setup and other things that are probably harmless but companies usually get paranoid over. The flasher application is called Trenchcoat. There's setup code for traffic shaping support - I haven't checked whether this is for application QoS on the device or whether it's intended for use with tethering.

Binutils is shipped. The Pre has an ARM assembler installed by default. The mind faintly boggles. It looks like there's tethering support via the Bluetooth PAN profile (see usr/bin/PmConnectionManager) as well as via USB. IPC is dbus-based rather than using something custom like Android's binder.
<h2>Overall</h2>I'm impressed. There's a few rough edges and some obvious short-term hacks, but overall the Pre has the appearance of being a well-engineered distribution. It's recognisably Linux in a way the Android isn't. Since it seems to be possible to gain root by entering the developer mode, I suspect that modifying the firmware image isn't especially difficult. It'll be interesting to see what happens when GSM ones appear.

There's some more information at the pre dev wiki.

Syndicated 2009-06-10 14:41:10 from Matthew Garrett

Antler customer service

In what makes a surprising change from my normal bitching about customer service, I called Antler last week after the rubber coating on the wheels of my luggage disintegrated. I've had that bag for almost 5 years and travelled something like 300,000 miles in that time, so it wasn't an especially surprising result - but the bag had a 7 year warranty covering defects in materials and manufacture, so they picked it up last week and dropped off a new one today. I'm impressed.

Syndicated 2009-06-09 15:24:03 from Matthew Garrett