Older blog entries for mjg59 (starting at number 205)

Thinkpad mixers

Older (up to the *60 series) Thinkpads have a slightly odd mixer setup. There's a mixer stage that sits between the software controllable mixer and the speakers. This is controlled by the firmware in response to the volume keys being hit and can't be prevented by the OS. This is unfortunate, since users are quite enthusiastic about seeing an on-screen display when they hit various keys and if we do the obvious thing and map the volume keys to KEY_VOLUMEUP and KEY_VOLUMEDOWN then they get the OSD but also get their software-controlled mixer changed at the same time as their firmware-controlled mixer. This causes boundless confusion.

Lorne Applebaum recently sent a patch that added an ALSA control to represent the Thinkpad mixer. This means that we can now read and control the Thinkpad-specific hardware without userspace needing any special knowledge. I've gone a little further and sent a followup patch that sends ALSA notifications on button presses. Lennart suggested that it would also be helpful to export dB values and wrote a nice app that measures them - that means that Pulseaudio will be able to set the mixer volume correctly by default.

The only missing part of the puzzle now is getting the OSD back. We still can't map the keys to KEY_VOLUMEUP and KEY_VOLUMEDOWN because that'll still result in two mixers being changed at once. We can't add new keys to indicate that it's a notification event rather than a command, since X can't deal with keycodes greater than 255 (actually 243, since it adds 8 to every kernel keycode for historical hilarity) and we've run out of space. We can't hook off the ALSA notifications because that'll result in the OSD popping up in response to someone dragging a slider in a volume control application. The easiest thing might be to add another ALSA notification that's only triggered if the notification was generated in the kernel and then have userspace listen for that.

Newer Thinkpads appear to have done away with this mixer setup entirely and just send standard volume events. This is a Good Thing.

In other news, I'm off on holiday to Australia for a couple of weeks so may be a bit slow in replying to things.

Syndicated 2009-05-04 00:53:57 from Matthew Garrett

Extended Qualcomm Gobi features

And while I'm on the subject of bitching about hardware support, I had some time to poke at the Qualcomm Gobi hardware a bit more last week. With the aid of Dan Williams of Network Manager fame we identified[1] that the hardware actually does rather more than is obvious from the qcserial driver. There's two extra ports - one is for diagnostics, and the other claims to be an NMEA port. Poking support for these into qcserial is easy enough, but they don't seem especially chatty. The more interesting factor is that the fourth interface on the endpoint is a network device. This isn't unusual in modern hardware - PPP isn't a terribly helpful layering for data transfer, so most recent high speed data cards also provide support for some kind of network device. If you're lucky it's a spec-compliant cdc device. If you've got a gobi then it claims to be a vendor defined class and type and the .inf file for the driver claims that it's a proprietary cdc device. Testing this got cut short by us not actually having an account on Verizon[2], so we don't have any data dumps to work out what the framing looks like.

Interestingly, doing USB dumps revealed that Windows seemed to do everything via the network device. We didn't see any traffic from the other ports, even when attempting to turn on the GPS. So it may also have some magic out of band configuration that doesn't simply use AT commands.

If anyone has this stuff working under Windows, it'd be interesting to try getting some dumps to try to work out what's going on here.

[1] By the simple process of opening the device manager under Windows. Ahem.

[2] The test machine was a Vaio P, which came with a gobi but no SIM slot. Thanks, Sony. Thony.

Syndicated 2009-04-19 00:32:55 from Matthew Garrett

Intel's commitment to open source driver support

By and large, working with Intel is a pleasure. In fact, a while ago I said so in a fairly obvious way. So it's a shame that the support for Poulsbo is still an absolute fucking mess with absolutely no obvious end in sight. Intel's Moblin group (the people who seem to be closest to having responsibility for providing any public Linux support for that hardware, though it's really not clear if they've got any more say in this than I do) have a kernel team that don't want or need a git tree despite having someone who's working on forward porting the driver to modern kernels - probably a job that requires more than one person given what a godawful fucking mess the last public release was, but it'd at least be nice to see what the current state is. Especially since you don't even seem to be able to get the last public release - moblin's git repositories have been rearranged and I can't find the psb-kmod one on their gitweb any more.

The most terribly depressing thing about this is that development is clearly still happening internally. Ubuntu is still being sent tarball releases by Intel, not that the code's getting any better judging by the type of diff involved. This is made even more entertaining by the bug in question being private, leaving no indication whatsoever about what bugs this code is meant to fix or why.

Closed development, random private tarball drops to partners without any public releases or changelogs, no indication of any upstream release, kernel developers entirely unrelated to the development of the driver trying to get code merged so people can actually use the hardware they bought? I'd expect this of some random far-east vendor with no experience in working with Linux, but not a company that's consistently one of the top ten contributors to the kernel and has frequently touted their level of support. There's no meaningful support for Poulsbo. There's just a thin cardboard cutout that's been carefully placed in front of a hole filled with users' hopes and money.

Seriously Intel. Sort it the fuck out.

Syndicated 2009-04-17 00:14:18 from Matthew Garrett

HP EliteBook 2530p

HP apparently decided my 2510p was cursed and replaced it with a 2530p which arrived earlier this week. A bit of playing later and it all seems pretty happy. Backlight control works fine, though you'll need hal-info git to get the hotkeys working out of the box. Sound needs alsa from 2.6.30 or for snd_hda_intel to be loaded with the "model=laptop" parameter. Suspend seems to work out of the box. Hotkeys and rfkill work with hp-wmi. The webcam is supported by the uvcvideo driver. The wwan modem needs qcserial from 2.6.30 and a tool for loading the firmware. It's happy to boot from EFI which probably saves a second or so of boot time. Wireless is iwl5100 and supported by the drivers from Intel. Video is GM45 and worked fine without any tweaking. I've no idea if the modem is supported or not, and I haven't tested the fingerprint reader yet. Grub was slightly unhappy about the idea of an EFI system partition that wasn't declared in a GPT, but that was a two line diff.

I've backported code to rawhide where necessary, so F11 should be released with full support for the hardware. It's quite a nice machine, and seems more solid than the 2510p - I'll see how it's going in a year or so.

Syndicated 2009-04-04 18:05:15 from Matthew Garrett

Qualcomm Gobi firmware loading

Shortly after I posted this entry on the qualcomm modem driver, Alexander Shumakovitch posted a shell script to do the loading. I've turned this into a C app and added checksumming support and udev integration, and now things seem to be working happily. There aren't many device IDs in the qcserial driver upstream right now, but I've just sent a patch to add a bunch. If you have a device this works for and you had to add the ID manually, let me know and I'll update things. You can get it here.

Syndicated 2009-04-04 16:26:31 from Matthew Garrett

The curious tale of the driver that did nothing

Several vendors are now shipping Qualcomm's Gobi chipset, a cunning dual CDMA-GSM wireless broadband device. There's a driver for it in the Linux kernel called qcserial which claims to support it.

Do not be fooled. This driver is a vile lie.

The hardware comes up in a dumb state and requires firmware to be loaded before it'll do anything. The only way to obtain this firmware is from a Windows driver. The only way to load this firmware is under Windows. This isn't helpful, especially given that it drops the firmware whenever you use rfkill or suspend or power down the machine. In fact, the only way you can use this driver is to boot Windows, let it load the firmware, reboot into Linux, get online and then never turn off or suspend your computer or the radio.

So, don't be like me - swearing viciously and trying to generate useful USB packet dumps in an attempt to get the hardware working. Known bad parts are the HP un2400 and the Dell wireless 5600 - sony also have a Gobi part that's used in the P-series machines, and Acer have one as well. I'll update this if I ever get anywhere with a firmware uploader, but until then remember that the presence of a driver in the kernel doesn't mean that you can actually do anything with the hardware. Fyalcomm.

Syndicated 2009-04-02 18:43:23 from Matthew Garrett

Reducing disk use

UNIX filesystems generally store three pieces of timing information about files - ctime (when the file was changed in any way), mtime (when the file contents, as opposed to its metadata, was last changed) and atime (when the file was last accessed by any process). This is a usefully flexible system, but the semantics of atime can be troublesome. atime must be updated every time a file is read, causing a read operation to instead become a read/write operation. This results in a surprising amount of io being generated in normal filesystem use, slowing the more relevant io and causing disks to spin up due to atime updates being required even if the file was read out of cache. It also results in a lot of unnecessary activity on flash media which may reduce their lifetime.

One option is to disable atime updates entirely. The problem with this approach is that certain applications depend on atime. This is especially common in mail clients which compare atime to mtime in order to determine whether a mailbox has been read since it was last modified. So, unfortunately, disabling atime entirely is impractical as a default. Back in 2006, Valerie Aurora submitted a patch that worked around this issue. The new relatime option meant that atime would only be updated if it would otherwise be older than ctime or mtime. Mail clients became happy and the world rejoiced.

Unfortunately, it turned out that there was one other common case of atime being used. Applications like tmpwatch monitor files in /tmp and delete them if they appear unused. In this case, "unused" means "has an atime older than a certain date". Since merely reading files doesn't update the ctime or mtime, relatime wouldn't cause the atime on these files to be updated and tmpwatch would happily delete them - even if users were reading them on a daily basis.

Ingo Molner submitted a patch to add a further heuristic to the relatime behaviour. With it, the atime of a file will be updated if it's older than mtime, older than ctime or (and this is the important one) more than 24 hours in the past. This deals with the tmpwatch case nicely, while still providing a significant reduction in the quantity of atime updates.

Fedora shipped this patch for several releases, and Ubuntu have used it by default since 8.04. Unfortunately there were some concerns over certain aspects of its behaviour (in respect to its interface as opposed to the relatime functionality itself) and it never got merged. I pushed a trimmed down version that purely implements the change to the relatime behaviour, and earlier today Linus merged it and a further patch that makes relatime the default behaviour on Linux.

Most users won't notice this change in behaviour at all, other than as a small improvement in io performance and a reduction in the number of drive spinups. For users that do have issues, a new strictatime mount option has been added - using this will require an updated mount command, but it's a trivial patch. I'd be surprised if there are any real world use cases that are negatively affected by this, especially since it's been default behaviour in several distributions for a while, but there's always the potential that someone will be tripped up by it. We'll see.

Syndicated 2009-03-27 00:21:51 from Matthew Garrett

Today is Ada Lovelace day, a project to celebrate the women involved in technology and make it easier to give examples of female role models in the industry. I've had the opportunity to meet many women working on Linux over the past few years, and if there's one thing that's tended to overshadow the achievements and dedication to their work it's the sheer amount of effort they've had to go to in order to gain equal recognition. And that made me realise that in many ways, the woman who's had the greatest impact on my career is Hanna Wallach[1]. About ten years ago she spent a ridiculous amount of effort teaching me that it didn't matter how much I professed to be entirely free of sexism if I then proceeded to do things that implicitly excluded women from being involved in computing communities. I've gone to some amount of effort to repay that (contains profanity and a man with a raccoon tail covering his crotch), but I'm very aware of how different my life might have been if Hanna hadn't gone to the trouble of ensuring that I knew not to be a dick.

I think the Linux community has become more welcoming since then, and Hanna and people like her have been instrumental in helping that happen. However, we still have people being driven away by the behaviour of others or arguments that technical contributions are more important than social behaviour, and while that's true we need role models for social change just as much as we need role models for technical achievement. Thanks, Hanna[2]. The Linux world's a better place because of you.

[1] And while she's a role model of mine for social reasons, she's also got a PhD in machine learning, was British computer science student of the year in 2001 and came top of her MSc class at Edinburgh. So there.

[2] "Thanna"

Syndicated 2009-03-24 23:59:00 from Matthew Garrett

ext4 and spinups

As a followup to my discussion of ext4, I did some trivial testing today. This consisted of generating a file, writing to it, closing it, opening another file, writing to it, closing it and then renaming the second file over the first. Then repeating about 10,000 times. fsync() wasn't called at any point. I used the /proc/sys/vm/block_dump knob to get an idea of when stuff was actually hitting disk. This was using the current rawhide kernel, which has the various workarounds for avoiding data corruption merged. The good news is that I was unable to get into an inconsistent state - the files always contained either the original or the new data, and the absolute worst case was having a zero length temporary file. The bad news is that while ext3 only touched disk when it reached the commit interval, ext4 touched disk for every single rename() call.

There appears to be a plan for dealing with this, but the current state of affairs is that ext4 will (under an obviously incredibly synthetic and unrealistic set of conditions) reduce the amount of time a drive can spend spun down. The next step is to attempt to measure this in a real world setup and see whether it's compensated for by the other new features.

Syndicated 2009-03-18 18:59:53 from Matthew Garrett

Important typographical update

⸘Unicode 5.1 added the inverted interrobang‽

Syndicated 2009-03-18 02:27:06 from Matthew Garrett