Older blog entries for dkg (starting at number 28)

dd, netcat, and disk throughput

I was trying to dump a large Logical Volume (LV) over ethernet from one machine to another. I found some behavior which surprised me.

fun constraints

• I have only a fairly minimal debian installation on each machine (which fortunately includes netcat-traditional)
• The two machines are connected directly by a single (gigabit) ethernet cable, with no other network connection. So no pulling in extra packages.
• I have serial console access to both machines, but no physical access.
• The LV being transfered is 973GB in size according to lvs (fairly large, that is), and contains a LUKS volume, which itself contains a basically-full filesystem -- transferring just the "used" bytes is not going to save space/time.
• I want to be able to check on how the transfer is doing while it's happening.
• I want the LV to show up as an LV on the target system, and don't have tons of extra room on the target to play around with (so no dumping it to the filesystem as a disk image first).

(how do i get myself into these messes?)

This entry has been truncated read the full entry.

Syndicated 2009-12-21 06:21:00 from Weblogs for dkg

dealing with entropy on a virtual machine

I've been using virtual machines (KVM, these days) as isolated environments to do things like build packages as root. Unfortunately, some of these activities require decent-sized chunks of random data (pulled from /dev/random). But /dev/random pulls from the kernel's entropy pool, which in turn is replenished from "hardware" events. But a virtual machine has no actual hardware, and if it is only doing isolated package builds, there is very little activity to feed the kernel's entropy pool. So the builds and test suites that rely on this randomness all hang for a long long time. :(

My current way to get around this is to replace /dev/random with the /dev/urandom device, which does not block if the entropy pool is depleted:

mknod /dev/newrandom c 1 9
chmod --reference=/dev/random /dev/newrandom
mv -f /dev/newrandom /dev/random

This has the consequence that the "randomness" these commands use doesn't have as much "real" entropy, though some operating systems (like FreeBSD) have a non-blocking /dev/random by default (and it's also questionable what "real" entropy means for a virtual machine in the first place).

I'm also using cowbuilder within these VMs to do package builds. But cowbuilder has its own /dev tree, with its own device nodes, so this needs to be fixed too. So after you have successfully done cowbuilder --create, you need to modify the random device within the cowbuilder chroot:

mknod /var/cache/pbuilder/base.cow/dev/newrandom c 1 9
chmod --reference=/var/cache/pbuilder/base.cow/dev/random /var/cache/pbuilder/base.cow/dev/newrandom
mv -f /var/cache/pbuilder/base.cow/dev/newrandom /var/cache/pbuilder/base.cow/dev/random

Hopefully this will be useful for other people using cowbuilder (or other build strategies) on isolated virtual machines. If you've worked around this problem in other ways (or if there's a security concern about this approach), i'd be happy to hear about the details.

Syndicated 2009-12-12 18:42:00 from Weblogs for dkg

Revoking the Ubuntu Community Code of Conduct

I've just revoked my signature over the Ubuntu Code of Conduct 1.0.1. I did this because Ubuntu's CoC (perhaps jokingly?) singles out Mark Shuttleworth as someone who should be held to a super-human standard (as pointed out recently by Rhonda, as well as earlier in ubuntu bug 53848).

I think that the CoC is a good document, and good guidelines in general for reasonable participation in online communities. When i originally signed the document, i thought the Shuttleworth-exceptionalism was odd, but decided i'd be willing to hold him to a higher standard than the rest of the community, if he wanted me to. That is, i figured his position as project leader meant that he could have made the CoC different than it is, thus he was (perhaps indirectly) asking me to hold him to a higher standard.

Why does this matter to me now? Shuttleworth has apparently signed the Ubuntu Code of Conduct, but as i wrote about earlier, his recent sexist comments at LinuxCon were a Bad Thing for the community, and his apparent lack of an apology or open discussion with the community concerned about it was even worse.

So i'm asking Mark Shuttleworth to abide by the following points in the Code of Conduct that he has signed:

• Be considerate
• Be respectful [...] It's important to remember that a community where people feel uncomfortable or threatened is not a productive one.
• The important goal is not to avoid disagreements or differing views but to resolve them constructively. You should turn to the community and to the community process to seek advice and to resolve disagreements.
• When you are unsure, ask for help. Nobody knows everything, and nobody is expected to be perfect in the Ubuntu community

I've signed a revised version of the Ubuntu Code of Conduct 1.01 (with the Shuttleworth-exceptionalism clause removed), to reaffirm my commitment to these principles, and to acknowledge that, yes, the SABDFL can make a mistake, and to encourage him to address his mistakes in a fashion befitting a mature participant in this community we both care about.

UPDATE: It seems that Mako and Daniel Holbach have recently revised the CoC resulting in a new version (1.1) which has just been approved by the the Ubuntu Community Council. The new version 1.1 looks good to me (i like its broadening of scope beyond developers, and its lack of superhuman claims for Shuttleworth) and when it is available on Launchpad, i'll most likely sign it there. Thanks to the two of them for their work! I hope Shuttleworth will consider abiding by this new version.

Syndicated 2009-10-20 18:24:00 from Weblogs for dkg

sexist behavior in the free software community

So not even 3 months out from RMS's sexist Gran Canaria virgins remarks, we have another powerful leader in the Free Software Community making sexist remarks in a talk to developers (this time, it's Mark Shuttleworth). It's a shame that these two people have said stupid things that hurt their causes and their communities by perpetuating an unfriendly environment for women. And it's a bigger shame that neither leader appears to care enough about their community to issue a sincere public apology for their screwup (if i'm wrong about this, please point me to the apology — i've looked).

These guys are in a situation which is nowhere near as hard as writing good software or managing complex technical projects: if you make a stupid mistake, own up to it, apologize, and try not to make similar mistakes in the future.

Perhaps worst of all, are the remarkable number of unreasonably fucked-up comments on the blog posts discussing these unfortunate events. If you're in the habit of defending remarks like those made by RMS and Shuttleworth on the 'net, please take a minute and ask yourself a few questions:

• Do you think that the Free Software community today is overwhelmingly male (even by the standards of the male-dominated IT industry)? If not, thanks for playing. You are living in a fantasy world. Try some basic research.
• Do you think that the significant under-representation of women is a problem? Let's say there are about three answers here:

  Gender disparity in Free Software is a Good Thing

  If this is your position, please announce it explicitly so we all know. Just so you know: I don't want to be part of your all-boys club. You can stop these questions now, sorry to have bothered you.

  I don't really care about gender disparity in Free Software one way or the other

  You may not care; but a significant subset of the Free Software community thinks that it's a problem and would like to address it. Please keep this in mind as you go to the next question. Also, have you thought much about the idea of privilege and how it might apply to your situation?

  I think gender disparity in Free Software is probably a Bad Thing

  Great, glad we agree on that.

• People in our community have a problem with the current state of affairs, and point out some specific behavior that makes the bad situation worse. What should you do?

  Shout them down or attack them

  Gee, it sure is upsetting to hear people talk about problems in the community. It's almost as upsetting as getting bug reports about problems in our software. Shall we shout them down too? Maybe we should attack them! Condescension is also great. Those silly bug reporters!

  Argue them out of having a problem

  This just doesn't work very well. Someone has already volunteered to tell you about a problem that you hadn't noticed. You are unlikely to convince them that they were imagining things.

  Take them seriously

  Yes! It seems to be surprising to some commentators that this is not a witch hunt or a lynch mob (interesting that these terms often-used in defense of white men connote specific historical traditions of the exercise of male privilege and white privilege, respectively). Well-meaning people have respectfully raised good-faith concerns about the state of our community, and made very simple suggestions about what to do to make the community more welcoming to women: lay off the sexist remarks at conferences, apologize when some nonsense does slip through — we're all struggling with various kinds of internalized oppression, you won't be perfect — and try not to do it again. Why not listen to these people? Why not support them?

Please read the Geek Feminism wiki and blog. Even if you don't agree with everything on those sites (hey, it's a wiki! and a blog! you don't have to agree with everything!), people are at least trying to address the problem of sexism in our community there. Engage constructively and don't hide or ignore problems!

Syndicated 2009-10-01 20:32:00 from Weblogs for dkg

Tools should be distinct from Services

Modern daemon implementations can be run in a variety of ways, in a range of contexts. The daemon software itself can be a useful tool in environments where the associated traditional system service is neither needed nor desired. Unfortunately, common debian packaging practice has a tendency to conflate the two ideas, leading to some potentially nasty problems where only the tool itself is needed, but the system service is set up anyway.

How would i fix it? i'd suggest that we make a distinction between packages that provide tools and packages that provide system services. A package that provides the system service foo would need to depend on a package that provides the tool foo. But the tool foo should be available through the package manager without setting up a system service automatically.

Bad Examples

Here are some examples of this class of problem i've seen recently:

akonadi-server depends on mysql-server

akonadi is a project to provide extensible, cross-desktop storage for PIM data. It is a dependency of many pieces of the modern KDE4 desktop. Its current implementation relies on a private instantiation of mysqld, executed directly by the end user whose desktop is running.

This means that a sysadmin who installs a graphical calendar application suddenly now has (in addition to the user-instantiated local mysqld running as the akonadi backend) a full-blown system RDBMS service running and potentially consuming resources on her machine.

Wouldn't it be better if the /usr/sbin/mysqld tool itself was distinct from the system service?

puppetmaster depends on puppet

Puppet is a powerful framework for configuration management. A managed host installs the puppet package, which invokes a puppetd service to reconfigure the managed host by talking to a centralized server on the network. The central host installs the the puppetmaster package, which sets up a system puppetmasterd service. puppetmaster depends on puppet to make use of some of the functionality available in the package.

But this means that the central host now has puppetd running, and is being configured through the system itself! While some people may prefer to configure their all-powerful central host through the same configuration management system, this presents a nasty potential failure mode: if the configuration management goes awry and makes the managed nodes inaccessible, it could potentially take itself out too.

Shouldn't the puppet tools be distinct from the puppetd system service?

monkeysphere Build-Depends: openssh-server

The Monkeysphere is a framework for managing SSH authentication through the OpenPGP Web-of-Trust (i'm one of the authors). To ensure that the package interacts properly with the OpenSSH implementation, the monkeysphere source ships with a series of test suites that exercise both sshd and ssh.

This means that anyone trying to build the monkeysphere package must pull in openssh-server to satisfy the build-depends, thereby inadvertently starting up a potentially powerful network service on their build machine and maybe exposing it to remote access that they didn't intend.

Wouldn't it be better if the /usr/sbin/sshd tool was available without starting up the ssh system service?

Good Examples

Here are some examples of debian packaging that already understand and implement this distinction in some way:

apache2.2-bin is distinct from apache2-mpm-foo

Debian's apache packaging recently transitioned to split the apache tool into a separate package (apache2.2-bin) from the packages that provide an apache system service (apache2-mpm-foo). So apache can now be run by a regular user, for example as part of gnome-user-share.

git-core is distinct from git-daemon-run

git-core provides the git daemon subcommand, which is a tool capable of providing network access to a git repo. However, it does not set up a system service by default. The git-daemon-run package provides a way for an admin to quickly set up a "typical" system service to offer networked git access.

vblade is distinct from vblade-persist

vblade offers a simple, powerful utility to export a single file or block device as an AoE device. vblade-persist (disclosure: i wrote vblade-persist) provides a system service to configure exported devices, supervise them, keep them in service across system reboots, etc.

Tools are not Services: a Proposal

Let's consider an existing foo package which currently provides:

• the tool itself (say, /usr/sbin/foo) and
• sets up a system service running foo, including stock foo daemon configuration in /etc/foo/*, startup scripts at /etc/init.d/foo, service configs at /etc/default/foo, etc.

I suggest that this should be split into two packages. foo-bin would contain the tool itself, and foo (which Depends: foo-bin) would contain the service configuration information, postinst scripts to set it up and start it, etc. This would mean that every instance of apt-get install foo in someone's notes would retain identical semantics to the past, but packages which need the tool (but not the service) can now depend on foo-bin instead, leaving the system with fewer resources consumed, and fewer unmanaged services running.

For brand new packages (which don't have to account for a legacy of documentation which says apt-get install foo), i prefer the naming convention that the foo package includes the tool itself (after all, it does provide /usr/sbin/foo), and foo-service sets up a standard system service configured and running (and depends on foo).

Side Effects

This proposal would fix the problems noted above, but it would also offer some nice additional benefits. For example, it would make it easier to introduce an alternate system initialization process by creating alternate service packages (while leaving the tool package alone). Things like runit-services could become actual contenders for managing the running services, without colliding with the "stock" services installed by the bundled tool+service package.

The ability to create alternate service packages would also mean that maintainers who prefer radically different configuration defaults could offer their service instantiations as distinct packages. For example, one system-wide foo daemon (foo-service) versus a separate instance of the foo daemon per user (foo-peruser-service) or triggering a daemon via inetd (foo-inetd-service).

One negative side effect is that it adds some level of increased load on the package maintainers — if the service package and the tool package both come from the same source package, then some work needs to be done to figure out how to split them. If the tool and service packages have separate sources (like vblade and vblade-persist) then some coordinating footwork needs to be done between the two packages when any incompatible changes happen.

Questions? Disagreements? Next Steps?

Do you disagree with this proposal? If so, why? Unfortunately (to my mind), Debian has a long history of packages which conflate tools with services. Policy section 9.3.2 can even be read as deliberately blurring the line (though i'd argue that a cautious reading suggests that my proposal is not in opposition to policy):

Packages that include daemons for system services should place scripts in /etc/init.d to start or stop services at boot time or during a change of runlevel.

I feel like this particular conflict must have been hashed out before at some level — are there links to definitive discussion that i'm missing? Is there any reason we shouldn't push in general for this kind of distinction in debian daemon packages?

Tags: daemons, packaging, policy

Syndicated 2009-09-18 18:47:00 from Weblogs for dkg

xen etch to lenny upgrade serial console

I maintain several xen machines. Most servers that i maintain use serial consoles (i should probably write another post in more detail about why serial consoles are the One True Way to manage a server).

Trouble is, the way that xen works with the serial console has changed between etch and lenny. So what's changed? From what i can tell:

• The way that the dom0 Linux kernel interacts with the hypervisor's console has changed between Linux 2.6.18 (etch) and 2.6.26 (lenny), and
• the Xen hypervisor's method of specifying the console itself has changed between Xen 3.0.3 (etch) and 3.2.1 (lenny)

In etch systems, i had a standard GRUB specification like this (as noted earlier):

title           Xen 3.0.3-1-i386-pae / Debian GNU/Linux, kernel 2.6.18-4-xen-686
root            (hd0,0)
kernel          /xen-3.0.3-1-i386-pae.gz dom0_mem=131072 com1=115200,8n1
module          /vmlinuz-2.6.18-4-xen-686 root=/dev/mapper/vg_monkey0-dom0 ro console=ttyS0,115200n8 
module          /initrd.img-2.6.18-4-xen-686
savedefault

but in lenny, i find it necessary to do this:

title           Xen 3.2-1-i386 / Debian GNU/Linux, kernel 2.6.26-2-xen-686
root            (hd0,0)
kernel          /xen-3.2-1-i386.gz dom0_mem=131072 com1=115200,8n1 console=com1
module          /vmlinuz-2.6.26-2-xen-686 root=/dev/mapper/vg_monkey0-dom0 ro console=hvc0
module          /initrd.img-2.6.26-2-xen-686

In particular, the hypervisor itself needs an additional console=com1 argument to make sure that it uses the serial console (i suppose the com1=... argument simply specifies how to use the serial console should you need it, instead of a requirement to use the console).

And the Linux kernel itself for the dom0 needs to adopt hvc0 as its console, which i believe is an acronym for something like the "hypervisor virtual console" -- this way, the dom0 kernel will come out multiplexed over the hypervisor's console, no matter where that hypervisor's console is directed (you might not even need to specify this explicitly -- it might be set up by default).

When i tried to use the etch settings for the console with the lenny hypervisor and kernel, i saw the hypervisor try to come out on the video card, and a dom0 crash!

Tags: serial, serial console, xen

Syndicated 2009-09-13 23:33:00 from Weblogs for dkg

Wanted: Empowered, Active, Activist(?) Users

The other, better-known and wiser Dan Gillmor (disclosure: we don't just share names, we're related) has started a new project called Mediactive. His older project, We the Media was about the power and coming growth of grassroots journalism. The new project focuses on media "consumers", instead of media "producers". Economic metaphors usually leave me cold, but it resonates with me when he says:

So I’m declaring victory, albeit early, on the supply side of the equation. [...]

But that doesn’t solve what may be a bigger issue: crappy demand.

We have raised several generations of passive consumers of news and information. That’s not good enough anymore.

The directional shift he's taking is an important one, and not just one for an independent, grassroots media. It seems parallel in many ways to the situation i see around free software.

The free software community has clearly demonstrated that we can build quality, liberated tools for the public in and from the commons (though there are admittedly tons of crap tools in the commons as well). But the society-altering political goals of free software (that everyone should have the right and the ability to freely use good tools fully under their own control, a corollary of freedom 0 from the FSF's four freedoms) are still going to fail if the majority of the tool users shrug their shoulders and either:

• aren't active participants in the communities around their tools, or
• concede to be pushed into proprietary tool use by people who are more interested in being proprietors than in enabling the freedom of their users.

Software needs an active and engaged userbase if it is going to become Good Software. Your software works for you? Fine. But if you can make it work for a dozen people who are engaged enough to say Wouldn't it be great if...? or How come it acts like...?, you can build a tool capable of solving problems you couldn't have imagined on your own. And those users can help and engage each other (and still more users) too; the community makes the tool more powerful. More powerful free tools provide more freedom to their community of users.

Dan's new project seems to acknowledge that not everyone is going to become a grassroots journalist themselves. And great as the dream of free software is, we cannot realistically insist that everyone become a software developer either. For either grassroots journalism or free software to live up to their promise, we need active and engaged (or even activist) users.

This is a tough project. i'm an active and engaged software user -- i have to be if i want to be a decent software developer, protocol designer, system administrator, or participant in debian. But it takes work and mental energy just to be a user like that, let alone the other responsibilities. But frankly (sorry, Dan!), i'm a terribly passive media consumer. I know Dan's right, that i can contribute to the cause of grassroots and distributed media (a cause i believe in, without seeing myself as a grassroots journalist) by being a more engaged and activist user of the media. But i don't do it currently. Why? While i'm not sure exactly how to be an active or engaged user of the media, i have no doubt that if i threw myself at the task, i could figure it out. But this kind of learning takes time and is real work, and even if i had it all figured out, i'm sure that being active and engaged would take more time and work than being passive. And i already feel swamped by other obligations and plans.

Most software users must feel the same way about their tools. How can we reduce those barriers? How can we not only help people see that their contributions as users are vitally important, but help make those contributions and that participation easy? As far as i know, Dan (who understands the goals, tech issues, and social concerns around software freedom as well as any non-programmer i know) doesn't run a free operating system himself. And i don't participate in any significant way as an engaged user of distributed, grassroots media. That sucks.

So:

• How do you make room for one more goal you believe in?
• How do you make it so your own goals are appealing and easy enough that natural allies can participate without feeling overwhelmed?
• What do we need to do as participants in a free software culture to encourage and engage active (and even activist) users?
• What projects are there out there (by analogy with Mediactive) to encourage users of free software to be active or activist? How are they doing?

Tags: free software, politics, users

Syndicated 2009-08-27 16:34:00 from Weblogs for dkg

Canon camera lens faiure (E18) -- resolved!

I have a little old Canon SD200 point-and-shoot, which is a decent camera. unfortunately, i've (ab)used it quite a bit, lugging it everywhere in my pocket or in a bag and even sending it up in a kite (and, uh, bouncing it off the ground on the way back down sometimes).

Unsurprisingly, things occasionally fail when you subject them to harsh conditions, and for a couple months now, the lens has failed to retract, showing a message "E18" on the screen and refusing to take pictures.

It turns out i'm not the only one with this problem, and there is actually a really nice guide with good photos for the SD300, which is a very similar problem. I'd been putting off fixing it until i had time to really focus, out of fear that it would take forever, but it turned out to be a really quick fix (i did "Procedure 1" in the step-by-step guide and it worked after about a minute of fiddling with the position sensor), and now i have my camera back!

So thank you, Greg Toews, for your excellent documentation!

Syndicated 2009-08-03 16:57:00 from Weblogs for dkg

keeping ssh host keys up-to-date with monkeysphere

Enrico posted a neat trick to track the SSH host keys of debian machines, thanks to Zobel. I wanted to mention monkeysphere, a project i'm involved with which provides a more generalized structure for doing this kind of update by taking advantage of the OpenPGP Web of Trust to distribute and authenticate SSH keys.

Enrico's known_hosts update strategy is nice, but:

• it's centralized,
• it's useful only for debian developers (and only when connecting to debian machines),
• it relies on users regularly running an update they wouldn't need to do otherwise (re-fetching the file from master.debian.org via scp) and
• it relies on the maintainers of master.debian.org:
  • to avoid compromise (there are a ton of other problems if master was compromised, of course),
  • to keep /etc/ssh/ssh_known_hosts up-to-date, and
  • not to change the host key for master itself (or the user's regular scp updates would fail).

These are relatively small flaws, and as a project debian is able to work around them because we have infrastructure in place like the machines database (though checking the machines db manually is tedious and therefore error-prone). But most other projects don't have that level of organization, and the process doesn't scale to other projects we (or our users) might be involved in. And other projects (including debian, i'd think) might prefer to have a less centralized process, to minimize bottlenecks and single points of failure.

Check out Monkeysphere's documentation for a server administrator for a quick rundown about how to easily publish your SSH host keys via the Web of Trust (it's not mutually-exclusive with the technique Enrico describes).

And this is just part of what the monkeysphere can do: using the same web of trust, monkeysphere is capable of helping a host authenticate ssh users based on their OpenPGP identities, which gives full re-keying and revocation functionality for these accounts. But that's a separate discussion!

Tags: monkeysphere, openpgp, ssh

Syndicated 2009-07-23 23:04:00 from Weblogs for dkg

HOWTO prep for migration off of SHA-1 in OpenPGP

Last week at eurocrypt, a small group of researchers announced a fairly serious attack against the SHA-1 digest algorithm, which is used in many cryptosystems, including OpenPGP. The general consensus is that we should be moving in an orderly fashion toward the theater exits, deprecating SHA-1 where possible with an eye toward abandoning it soon (one point of reference: US gov't federal agencies have been directed to cease all reliance on SHA-1 by the end of 2010, and this directive was issued before the latest results).

Since Debian relies heavily on OpenPGP and other cryptographic infrastructure, i'll be blogging about how Debian users can responsibly and carefully migrate toward better digests. This post focuses on some first steps for users of gpg, and for Debian Developers and Debian Maintainers in particular.

The good news is that gpg and gpg2 both support digest algorithms from the stronger SHA-2 family: SHA512, SHA384, SHA256, and SHA224.

By using these stronger digest algorithms some of your signatures may be un-readable by users of older software. However, gpg and PGP (a proprietary implementation) have both had support for at least SHA256 for well over 5 years. Debian's gnupg packages have supported the full SHA-2 family since sarge.

However, most existing signatures in today's Web of Trust were made over the SHA-1 digest algorithm, which means that abandoning it immediately would cause the Web of Trust as we know it to evaporate. So we need to rely on SHA-1-based signatures until a reasonably-fleshed-out Web of Trust based on stronger digests is in place. Since we don't want to have to rely on SHA-1 for too much longer, we need to collectively start the transition now.

So what can you do to help facilitate the move away from SHA-1? I'll outline three steps that current gpg users can do today, and then i'll walk through how to do each one:

• start making data signatures and web-of-trust certifications using stronger digests,
• explicitly state your preferences for stronger digests when receiving private communications, and
• If you are currently using a 1024-bit DSA primary key (which relies for signatures on a 160-bit hash, traditionally SHA-1), transition to a new 2048-bit RSA key.

The first two are simple, quick, and painless actions. You'll be done with them in minutes! The third is tougher, and while you can start it today, key transitions take a little bit of time to complete. Read on for a HOWTO!

This entry has been truncated read the full entry.

Tags: cryptography, openpgp, security, sha1, tip

Syndicated 2009-05-06 06:15:00 from Weblogs for dkg