Older blog entries for richdawe (starting at number 108)

SMTP tool

In a previous blog entry, I talked about an idea for an SMTP shell. A colleague at work suggested I take a look at swaks (Swiss Army Knife SMTP). swaks looks really useful. I wonder how easily it could be extended to have a shell mode...

We've got a bunch of Perl-based tools at work for testing our SMTP server that use Net::Telnet that do plain SMTP. These use instances of stunnel to wrap the plain, unencrypted SMTP conversations in different types of TLS connections (e.g.: with client certs, without, etc.). The test tool does what it's good at (weird SMTP test cases), while stunnel does what it's good at (TLS). That could be better -- why not integrate TLS support into the test tool? At the time it saved me a lot of time, leaving me to concentrate on writing test cases. Maybe it's time to borrow code from swaks, to integrate TLS directly into our test harness.

std::string in shared memory

The Problem

A tricky problem I faced at work a while back was trying to get some code using C++ std::strings in shared memory to work with libstdc++ from gcc 3.2.x.

This didn't work because of PR/16612. The effect of this bug is that you cannot store empty strings in shared memory. The implementation of std::string in libstdc++ 3.2.x uses a shared representation of the empty string. The problem arises because this shared representation of the empty string may live at different addresses in each of the processes using the shared memory block. If one process stores the empty string in shared memory and then another accesses it, you are likely to get a segfault.

The Solution

A shared allocator had been written, to allow the std::string and other STL types to be stored in the shared memory, e.g.:

typedef std::basic_string<char, std::char_traits<char>, SomeSharedAllocator<char> > SharedString;

SomeSharedAllocator handles allocations, deallocations and accessing STL objects in the shared memory. (I didn't write any of that awesome code; I was just porting it to an older Linux version.)

The solution was to stop using the common representation of the empty string -- to actually store an empty string in shared memory. This sounds pretty straightforward, but was actually quite tricky. It involved implementing specific functions in the implementation of std::string to suppress the default common representation: the default constructor and the C-style string constructor.

From the header file:

// Default constructor
template<>
    basic_string<char, std::char_traits<char>, SomeSharedAllocator<char> >
    ::basic_string ()
    : _M_dataplus(_S_construct((size_type) 1,
                               char(' '),
                               SomeSharedAllocator<char>()),
                      SomeSharedAllocator<char>())
      {
        this->resize(0, char());
      };

// C string constructor template<gt; basic_string<char, std::char_traits<char>, SomeSharedAllocator<char> > ::basic_string (const char *__s, const SomeSharedAllocator<char>& __alloc);

From the source module:

// C string constructor
  // std::string(NULL) segfaults; preserve that behaviour.
  template<>
    basic_string<char, std::char_traits<char>, SomeSharedAllocator<char> >
    ::basic_string (const char *__s,
                    const SomeSharedAllocator<char>& __alloc)
    : _M_dataplus(*__s
                  ? _S_construct(__s,
                                 __s + traits_type::length(__s),
                                 __alloc)
                  : _S_construct((size_type) 1,
                                 char(' '),
                                 __alloc),
                  __alloc)
      {
        if (!*__s)
          this->resize(0, char());
      }

The source module was linked into the shared memory library. All access to the shared memory using strings goes via the SharedString type.

Consider an empty string created by the above code. An empty std::string created by that code has a representation stored somewhere. Assigning a std::string into a SharedString will copy the original representation into one allocated in shared memory. So the code above guarantees that multiple empty strings in shared memory would be stored multiple times, rather than using a common representation, fixing the original problem.

Prologue

This problem has a work-around in gcc 3.4.3 and later -- #define _GLIBCXX_FULLY_DYNAMIC_STRING. But how do you know whether your version needs the work-around above or the #define?

  • #if defined(__GLIBCPP__) && (__GLIBCPP__ < 20041228) -- you need the code listed above.
  • #if defined(__GLIBCXX__) && (__GLIBCXX__ < 20041228) -- you need the code listed above.
  • Otherwise you need to #define _GLIBCXX_FULLY_DYNAMIC_STRING.

Music

"Protection" by Massive Attack

Fedora Core 5, madwifi and wpa_supplicant

Upgraded my x86_64 laptop to FC5 recently. That broke my wireless, due to FC5's wpa_supplicant not shipping with madwifi support (that's not really suprising).

madwifi ticket 462 includes a patch for madwifi to add WEXT support (the Linux standard wireless extensions), so that it works "out of the box" with plain wpa_supplicant and NetworkManager. I updated my dkms-based packages to include this patch and it works fine. I have i386 (untested) and x86_64 rpms available. You'll need to update /etc/sysconfig/wpa_supplicant to specify INTERFACES and DRIVERS, otherwise wpa_supplicant will use the wrong interface and driver type. E.g.:

INTERFACES=ath0
DRIVER=-Dwext

NetworkManager currently doesn't seem to work quite right. It negotiates WPA-PSK correctly with my access point, but somehow doesn't seem to get a DHCP-assigned address. It works fine with wpa_supplicant.

Fedora Core 4 and weird DNS/nscd problems

I did a "yum update" to the latest glibc the other night. For some reason DNS was completely broken on my box today. There seemed to be a couple of problems: nscd wasn't being started; programs talking to nscd didn't seem to work, giving weird "file not found" errors.

I had a stale /etc/nsswitch.conf.rpmsave file left over, probably from the upgrade. I moved that to /etc/nsswitch.conf, did "chkconfig nscd on" and then "service nscd restart". Now DNS seems to work again.

Idea: SMTP shell

During my $DAY_JOB I find myself typing in the same stuff into SMTP sessions. Recently I got to thinking that an SMTP shell would be useful -- command-line history, ability to repeat commands, sequences of commands, search through the history, etc. Replaying DATA would not only do the DATA command, but also the data that you typed in.

So I put it on my to-do list. But you can achieve most of what I wanted using netcat and perl (Linux example):

cat<<EOT > session.smtp
MAIL FROM:<fred@company.com>
RCPT TO:<melissa@company.com>
DATA
Subject: boo!

.
QUIT
EOT
perl -pe 's/\n/\r\n/' session.smtp | nc mail.company.com smtp

It would still be useful to have an SMTP shell for recording sessions, so you choose to play back bits later. E.g.: replay this, but negotiate export-strength TLS as well. I may still write it (but don't hold your breath).

Music

"Successful Enter (Axel Konrad remix)" by Headquarter

Blah

I've not spent much time on Free Software recently, for various reasons.

I found a useful piece of software for writing out podcasts, MP3s, etc. -- mp3cd -- point it at some MP3s and off you go. Although you might want to edit the cdrdao TOC file before actually burning the CD, to add CD-Text descriptions of the tracks.

Getting closer to porting File::ExtAttr to Solaris & *BSD. I have them all running in VMware-server beta on my x86_64 box. It's pretty impressive that they all work on that combination -- i386 OSes running on x86_64 Fedora Core 4. Big up VMware!

Music

"Insides" by Orbital

madwifi and wpa_supplicant

Went to debug why my wi-fi didn't work on boot. My init scripts were starting wpa_supplicant *after* network, which explains why that didn't work. Went to fix my rpms...and found they were fine. I had the wreakage from a previous install left over. Reinstalled et voila! Now I have wi-fi from boot time.

Everything on my laptop now Just Works (er, except power saving). Time to achieve this: 1.25 years.

File::ExtAttr and non-Linux systems

I have VMware server beta going on my x86_64 box, so now I can start porting File::ExtAttr to *BSD. Also, time to have some fun with Solaris 10 / Express and DTrace. I'm itching to give DTrace a whirl.

Ruling!

I've been reading "Creating Passionate Users" and finding it inspiring. I've found myself feeling pretty jaded over the past year for a variety of reasons and it's rekindled my passion for creating computer software. The article "Re-igniting passion" hits it on the head for me. Sometimes it's hard to get away from the daily grind and do something that makes you feel like you did the first time you some out-there piece of code. Rule!

Music

"X & Y" by Coldplay

5 Feb 2006 (updated 5 Feb 2006 at 21:46 UTC) »

ipodfs

Sometime ago I bought an iPod Shuffle. Since I have no Windows or Mac OS X boxen at home, I was relieved to find that I could initialise and manage it using gtkpod. However, I found gtkpod really hard to use.

gtkpod's iPod handling is available in a library called libgpod. So using that and FUSE I built an ipod file system, ipodfs, to allow you to upload MP3s, etc. using Unix file utilities. E.g.:

ipodfs /some/mount/point
cp newalbum*.mp3 /some/mount/point

Current suckiness:

  • It assumes that your iPod is mounted automatically on /media/ipod.
  • I've only tested it with the iPod Shuffle.
  • It doesn't preserve any of the ID3 tags (or equivalent in other formats).
  • The file type guessing is based solely on file extension.

madwifi and wpa_supplicant

madwifi and wpa_supplicant are approaching the point where they "just work" for me on my x86_64 box. I can now use wifi for several hours without crashes or WPA dropping out. WPA-PSK negotiation seems to work reliably. This is a big change from a few months ago.

Admittedly, I still have to go through this after boot:

ifdown ath0
modprobe -r ath_pci
modprobe ath_pci
/etc/init.d/wpa_supplicant restart
ifup ath0

You may wish to try my Fedora Core 4 i386 or x86_64 rpms if you're having trouble.

Music

"Organ Donor" by DJ Shadown

L33t Pubs

L33t pub names resulting from a conversation that occurred, appropriately enough, down the pub at lunch:

  • Teh W00t and H4xx0r
  • Teh J0lly H4xx0r
  • The Unlucky Camper (OK, so that's a gamer one)

File::ExtAttr

File::ExtAttr 0.04 is out. Changes:

0.04 2006-01-20
- (richdawe) Add Mac OS X support; thanks to Jonathan Rockway.
- (richdawe) Add a check to Makefile.PL for libattr's headers on Linux.
- (richdawe) Requirement on Perl 5.8.5 is spurious; remove it.

Hopefully this will get a little more exposure from the CPAN testers than the last release. That probably didn't get built by many people because of the Perl 5.8.5 dependency.

Music

Talkin' Jazz Volume 3

99 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!