Older blog entries for mjg59 (starting at number 112)

Why Linux claims to be Windows

Section 5.7.2 of the ACPI specification provides for a method called _OSI. Platform firmware can query for various interfaces, which allow it to tweak its behaviour depending on the interfaces that the OS supports. Linux claims to support the following interfaces:

  • Windows 2000
  • Windows 2001
  • Windows 2001 SP1
  • Windows 2001 SP2
  • Windows 2001.1
  • Windows 2001.1 SP1
  • Windows 2006
which all correspond (unsurprisingly) to different versions of Windows. Linux used to be in there, but was removed for reasons I'll get into later.

The typical way in which this feature is used by a platform is by querying each of the interfaces in turn, starting with the most recent a firmware is aware of. After the first match, a value will be stashed somewhere and used elsewhere in the firmware when making certain decisions. An example of this may be only enabling certain devices (such as the HPET) if the OS claims to support Windows Vista.

As I mentioned, recent versions of Linux no longer claim to support the Linux interface. There's a couple of reasons for this. Firstly, some platforms break if they're told that they're running on Linux. It's usually an entirely untested codepath. Secondly, the whole point of the _OSI method is to provide the platform with information about the interface supported by an OS. "Linux" is too coarse grained - we make no guarantees that the interface between the OS and the platform will remain constant. "Linux 2.6.25" isn't all that much better, since distribution patches may alter the behaviour and BIOSes would become specific to a given kernel version. We could have something like "Linux ABI 1" and then bump that every time we altered the behaviour of the kernel in this respect, but this would become unwieldy fairly quickly (what would count as an ABI breaking change?)

So what kind of interface can we claim to provide? The sad truth of the matter is that most vendors will only test their code against Windows, and that as a result Windows provides the de-facto specification for OS/platform interaction. We could skip claiming to be Windows, but (in the best case) that means that the system will assume it's working with an OS that supports no advanced features. Rather than getting support for brightness changes being handled by the OS, we get stuck with SMM traps and unexpectedly missing time. If we want hardware to work to its full potential, we need to claim to be a recent version of Windows.

Does this mean the entire _OSI mechanism is useless? Not really. It's still useful for providing information about supported functionality. One example used in the spec is whether or not the OS implements the ACPI 3.0 thermal model. Another obvious (but unspecced) example might be whether or not the video opregion spec is implemented in the OS. _OSI strings which explicitly advertise the existence of advanced functionality are something that should be specified and encouraged. What they shouldn't be used for is as a way of advertising lacking functionality, which is effectively the only way that Linux can be usefully interpreted today. We don't want machines to automatically repost their video if they detect they're running Linux. It makes it harder to implement proper kernel-level graphics resume code. We don't want them to suddenly alter their keyboard behaviour, because that probably makes it harder to support older machines that don't have these workarounds.

Message to vendors: If we're not behaving in the same way as Windows does, let us know. We'll fix it. This has the benefit of fixing not only your latest and greatest platform, but also all the older ones that expected the same behaviour. Please don't try to work around our bugs in your BIOS - it just means that we need to special case you forever, and in the long run your hardware won't work as well as it would if we just fixed things properly.

Syndicated 2008-04-19 19:57:53 from Matthew Garrett

Another update of my AHCI force quirking patch, somewhat cleaned up to let the existing kernel code do more of the work. The only functional change in this version is that some of the chipset IDs are disabled. The Intel specs state that plain (non-RAID supporting) desktop chipsets don't support AHCI. If you find that enabling one of the disabled IDs works for you, let me know along with the output of lspci -vxn -s 1f.2 . Patch is here. Success and failure reports welcome.

Note: I've had one report of SCSI errors associated with a previous version of the patch. I'm not quite sure what's going on there, but if you're paranoid about data corruption then you probably shouldn't play with this right now.

Syndicated 2008-04-17 23:01:33 from Matthew Garrett

This week I'm in Mountain View for the X Developers' Conference, where I gave a presentation on things graphics drivers have to care about in order to make suspend/resume work reliably. Various people suggested that I write up an explanation of the current situation, along with some best practices for manufacturers.

One of the major differences between the ACPI specification and the older APM specification is that saving and restoring hardware state is generally left up to the operating system, with the firmware playing a much more passive role. Linux is now able to handle this state restoration for the majority of hardware, but we still have issues with graphics. Modern graphics hardware can be hugely complicated, which to a large extent we've been able to ignore because the platform will initialise it on boot. Many X drivers know how to switch a card from text mode to graphics mode (and vice-versa), but that's not adequate.

What can make this situation even more awkward is that the ACPI specification makes no guarantees about what state the hardware comes up in. It may still be in PCI D3. It may be powered up, but the registers may all be zeroed. It's even valid for the hardware to come up in text mode. A driver has to be able to deal with all of these situations.

So far, we've mostly tried to deal with this situation by running chunks of the video BIOS with applications like vbetool. This is hacky and often works badly - in some cases, it won't work at all (nvidia rewrite the entry point to their video BIOS after boot, in order to prevent situations where someone attempts to POST the card and it tries to jump to bits of the system BIOS that no longer exist). The only long-term option is for the kernel to handle the graphics suspend/resume. There's several levels at which this can be supported:

  • Able to restore text mode. This is basically adequate right now, since the kernel enforces a virtual terminal switch around suspend/resume.
  • Able to restore arbitrary video mode. In the kernel modesetting world, we want to be able to go straight into the mode that we're running X in, in order to reduce screen flicker. This would probably still be surrounded with a VT switch out of and into X.
  • Able to jump straight into X. Right now, on x86 the process freezer will prevent X from running until after all of the devices have been resumed. As we move to a model where we remove the freezer and expect drivers to be able to block userspace tasks, we hit a pretty obvious problem with X - right now it hits the graphics hardware directly, rather than going via the kernel. With nothing to block it, it'll hit the hardware while it's still in an undefined state and then become very unhappy. This can't really be fixed until X does all its hardware access through the kernel. For now, we can inhibit X by leaving the VT switching in place.
Since 2.6.25, i915 and later Intel hardware can manage basic mode reprogramming without requiring any external assistance. The radeon kernel modesetting code is also heading in this direction, and we ought to be able to manage something there before too long. The nouveau developers are heading towards the stage of being able to do native mode programming, so there's some hope of progress there as well.

It's not all plain sailing, though. Vendors can wire up hardware in different ways. AMD and Nvidia hardware provides scripts in the ROM that, in theory, abstract this away. The downside to this is that we'll have to put the interpreters for these scripts in kernel space, though this is probably needed for kernel modesetting work anyway. Even then, it's possible that we'll need workarounds for specific pieces of hardware that don't quite behave as we expect them to.

As far as making graphics work over suspend/resume, we have a pretty good idea where we are now (failing on anything other than modern Intel hardware), where we need to be in the future (triumph, huge success, cake) and how to get there (?). However, hardware manufacturers can still help.

Chipset vendors:

Tell us how to reinitialise your hardware. If that involves parsing BIOS tables, then please let us know how to do that. Expecting us to jump into your ROM and execute x86 code is ok if that's genuinely the only way to do it, but for any modern hardware that's clearly not the case.

System vendors:

Try not to alter your system behaviour too wildly. Requiring magic register writes in order to light up the VGA output does not classify as value add. Please don't have your firmware program text mode directly - we're trying to get away from that. And do not attempt to detect that you're running on Linux and alter your system behaviour. If it's possible for you to tell that you're not running on Windows, that's a bug in Linux. If you rely on that bug, you will break. I'm not joking here in the slightest. The closest thing to a "standard" interface between an OS and the hardware is whatever Windows does, and we're basically aiming for bug-for-bug compatibility there.

Syndicated 2008-04-17 20:06:04 from Matthew Garrett

Dear RSA,

Turing Lives? Leaving aside the fact that your biography entirely fails to mention the manner of his death (security through prejudice? Wait, hang on...), this bit really seems excessive:

There’s no telling what Turing would do if he were around today, but one thing is certain, he would attend RSA Conference to gather and share new ideas and inspiration


Speaking for the dead in order to further commercial concerns? The dead baby Jesus weeps bitter tears, so buy my t-shirts.

Syndicated 2008-04-13 22:24:35 from Matthew Garrett

Now that I've got a little more time on my hands, I've updated and done some bugfixing on my patch to quirk ICH devices over from PIIX-compatibility mode to AHCI mode. It turns out that ICH8 is a little different to ICH7, and needs an extra bit to be set. Without that being done, the AHCI BAR would still be programmed as i/o ports. Interpreting this as mmio space let to unsurprising crashes. With a bit of luck, this should also fix the "My CD drive is no longer there" issue - you'll still need to load ata_piix in order to access it, but make sure you do it after ahci has bound to the sata controller.

Patch against 2.6.24 is here - let me know if it works for you.

(Context: AHCI provides support for various features that the PIIX compatible mode doesn't, including link-level power saving. This is potentially beneficial for mobile users)

Syndicated 2008-04-06 15:15:34 from Matthew Garrett

Ext JS is some sort of Javascript framework stuff. I know nothing whatsoever about Javascript, but what interested me was the slightly odd license. If you're a non-profit you can have it under the LGPL 3.0, but if you want to use it in a library then you have to buy a commercial license for them. This seems at odds with my understanding of the LGPL, but anyway. I'm a non-profit, so I've downloaded it under the LGPL. Under my LGPL rights, I've put it up for download here. If you want to use it in a library, feel free to do so as long as you abide by version 3.0 of the LGPL.

Thanks to Decklin for pointing this out.

Syndicated 2008-03-28 15:39:17 from Matthew Garrett

Because you all love me and deeply want to know what on Earth I'm doing with my life/want to hire me, I'll put you out of your misery and let you know that I've just accepted an offer from Red Hat to save the universe work on kernel and userspace power management. With luck, it'll go some way to making up for the ridiculous amount of flying I'm still due to do this year.

Syndicated 2008-03-23 03:07:34 from Matthew Garrett

Wednesday:


Today:


Hurrah for the open source development model!

(One of these taken by someone who can take pictures, and one taken by me)

Syndicated 2008-03-14 21:17:48 from Matthew Garrett

Airports visited in the past week: 4 (rising to 6 in 12 hours or so)
Terminals visited in Heathrow: 2
Terminals enjoyed in Heathrow: 0
Vending machines in groundside of Heathrow terminal 2: 2, hidden near the men's toilets
Temperature of "ice cold" Coca Cola from vending machine: 5°C
Currencies currently carried: 4
Passports currently carried: 2
Fuses blown in airports: 1
Belonging to me: 0
Gross vm86-related hacks perpetrated: 1
Number of x86 instructions to emulate in order to achieve the above: 21
Number actually emulated as yet: 0

Those of you who haven't been too busy not paying attention will already know my whereabouts on the 13th of October and why my alibi is weak but that's another story for another time, but also that I'm in Recife from tomorrow afternoon until some time on Sunday and Porto de Galinhas from then until Wednesday in order to present at Bossa to talk about Ubuntu Mobile and the challenges involved in turning a Linux desktop distribution into something suitable for devices that are less large than your face. Sadly, logistical errors on my part mean I won't have any demo hardware with me, but there's always some potential for other synergy. Please don't beat me to death. Ow. Not the face. Please, not the face.

Syndicated 2008-03-14 01:32:54 from Matthew Garrett

Dear Mountain View,

Why is it so indescribably awkward to get from north of 101 to downtown on foot?

Fucking sort it out,

Matthew

Syndicated 2008-03-08 02:16:50 from Matthew Garrett

103 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!