Older blog entries for chromatic (starting at number 230)

Making It Easier For Users to Report Test Failures:

Perl's module tools make it easy to bundle, distribute, and install software, most of the time. Other tools make it fairly easy to test software too. Of course, the tools for end-users lag somewhat behind the tools for developers, especially in places where developers are happy with their tools.

If you're a Perl-savvy developer and tests fail in a module you're installing, you know what your options are. Consider what a non-developer could do, though. Yet developers who ship their tests for end-users to run rely on receiving feedback about failures so that they can fix the code, the tests, the assumptions, or whatever's not right.

I've just released Module::Build::TestReporter which runs the tests as usual, hijacks their output, keeps a log of any failures and their diagnostics, and tells users what to do to report any failures to the developers. If you think this will solve a problem for you, give it a whirl. (I'd love to have feedback before I release it to the CPAN in a week or so.)


Why call walkoptree() yourself (see B.pm) when you have the power of XPath (at least as much as Class::XPath supports?


use strict; use warnings;

use B::XPath;

use vars qw( $foo $bar );

sub some_sub { my $x = shift; $foo = $x; print "\$x is $x\n\$foo is $foo\n"; }

my $node = B::XPath->fetch_root( \&some_sub );

for my $bar ($node->match( '//gvsv[@NAME="foo"]' )) { printf( "Found global '%s' at %s:%d\n (defined at %s:%d)\n", map { $bar->$_ } qw( NAME find_file find_line FILE LINE ) ); }

I'm sure you're on the edge of your seat for the output:

$ perl find_global_name.pl
Found global 'foo' at find_global_name.pl:13
  (defined at /usr/lib/perl5/5.8.6/vars.pm:35)
Found global 'foo' at find_global_name.pl:14
  (defined at /usr/lib/perl5/5.8.6/vars.pm:35

There are two drawbacks (besides the fact that it's a proof of concept and not releasable yet): Class::XPath has little axis support and you have to know an awful lot about the structure of the optree for which you want to search. I think the latter is solvable, but it will require more thought.

Crueler Months:

In the past month, outside of my busy day job, I:

  • Finished writing my novel. It's 72,500 words. Strangely, it's neither science fiction nor fantasy. It's just a modern novel. Now I'm letting it sit for a bit before I edit it. I can't promise when it'll be in stores though.

  • Finished writing a book with Ian Langworth. It's good stuff. It'll be out before OSCON.

  • Started applying documentation patches like mad to Parrot. At least, I've been bolder about it than before. (I also fixed a segfault, which was nice.)

  • Released Class::StorageFactory. Has it been a month already? Note that I renamed load() to fetch() and save() to store().

  • Released a (new maintainer!) version of SUPER. I volunteered to take over this module because I wanted to fix it to work with Class::Roles. That's one part down.

  • Adopted two cats. My plants downstairs are suffering a bit.

  • Learned enough about rake to write a Rakefile for Pacuby. (If that doesn't make any sense, consider that the first time I had an upcoming book deadline, I wrote tests for a fair swath of the Perl core.)

  • Wrote two actually useful testing modules I plan to release to the CPAN very soon.

  • Started brainstorming another book project.

Maybe now I can sleep again.

A Simply Serializing Factory:

I've found a nice pattern of using objects and classes for configuration information, but I don't want to go all of the way to Class::DBI with them because I like the simplicity of working with flat files.

The third or fourth time I found myself writing a factory to load and to save objects to and from YAML files, I resolved someday to write a module that does that and refactor the other attempts to use the module. Today, instead of working on one of two books, cleaning my house, or doing either of the programming projects that crossed my mind, I wrote Class::StorageFactory (and Class::StorageFactory::YAML, which is what I wanted).

If it seems useful and no one objects, I'll upload it to the CPAN in a couple of days. I looked for prior art but didn't find much in a few minutes.

I have new versions of SUPER and Test::MockObject to upload soon too.

12 Mar 2005 (updated 12 Mar 2005 at 00:56 UTC) »

People to Ignore in Software Development:

When discussing feature requests, development practices, and design issues of any public software project, reasonable people will disagree. That's fine. It's normal. It happens. It can be healthy if you deal with them appropriately.

However, sometimes some people draw lines in the sand around one position -- whatever the position -- and say something like:

Which is exactly why open source software will never replace commercial software or Microsoft.

Flip the bozo bit. Flip them all. Don't argue. Don't read any further. Above all, don't take the person seriously.

There might be merit to the argument, but you're not going to hear it from that source.

(You probably should ignore the people who continue the argument from that point too, but I don't like to flip the entire double-word of bozo bits immediately.)

Cool Testing Stuff:

Good things have happened in the test world in the past couple of weeks.

Andy has checked in a couple of patches for Test::Harness::Straps to collect diagnostic information. This means that writing a custom harness to print out (and colorize) only failed tests is easy.

Last week, Autrijus and I discussed porting Test::Builder to Perl 6 for Pugs. Schwern and I sketched out some refactorings of Test::Builder a couple of years ago and promptly didn't do it. I resurrected the idea (if not the notes, if that hard drive even exists anywhere accessible anymore) and came up with some ideas last night. Today, the idea came up again on the Perl QA mailing list for Perl 5 and it looks like we can go ahead with it.

Most of the ideas are merely internal cleanups, but they'll make it possible to do some cooler things that people have asked for in cleaner ways.

I've spent some of my off-hours documenting existing and inventing some new testing tricks. I'll explain more about this soon.

Finally, I'm fiddling with the OpenGL bindings to Parrot again. What? You didn't know they existed? Oh, they do. I'll show them off soon.

Dear Ranters:

Hey, avrietta, remember when you said:

Lastly (didn't I just say that?), ORA, again, sucks. I'm saying it again, and I can't believe it. Will they publish ANYTHING?

...and we all laughed?

I was thinking. If you want to know my editorial standards and rationales, you can kindly ask me. Contrariwise, you could fume and piddle about what the horrors and terrors of storing recipes in an non-MVCC relational database until your heart explodes. Either way.

F/OSS Testing:

zhaoway claims that the open source way to test is to make your users do it.

If I thought I could find users who ran tests completely, perfectly, and quickly, report errors accurately, never felt bored, never skipped steps, and who could perform tests in real time as I develop, I'd consider the idea.

In other words, it sounds like yet another excuse for software quality by wishful thinking. Sorry.

Dark, Dark Corners of Perl:

After tonight's PDX.pm meeting, in the bar, I bet Ovid that I could reimplement his Sub::Signatures by replacing the source filter with subroutine attributes. (Wisely, he believed me.)

A few minutes later, he asked idly if I knew of any reason why Perl couldn't call methods with spaces in their names. Offhand, I couldn't think of anything besides the tokenizer that prevents it, believing that gv.c cares more about null-byte termination than the <code>\w+</code>ness of identifiers. If you can bypass that for symbol installation and method invocation (and that's trivial), you've done it.

I hate to give away the punchline before both frightening people who didn't know such things were possible and stumping a few of the people who know it's possible but don't see the answer right away, so here are my tests. They all pass on my machine and I expect them to run just fine on at least Perl 5.8.0 and newer. (Perl 5.6 probably works, but I expect it to fail on more complicated things.) You don't need any non-core modules besides Attribute::Handlers, and I believe that's in the core in the 5.8.x family.

#!/usr/bin/perl -w

BEGIN { chdir 't' if -d 't'; use lib '../lib', '../blib/lib'; }

use strict; use Test::More tests => 4;

my $module = 'Attribute::Scary'; use_ok( $module ) or exit;

package Hello;

use strict; use warnings;

use Attribute::Scary;

sub new { my ($class, $name) = @_; bless \$name, $class; }

sub name :Method { return $$self; }

sub greet :Method { return sprintf( "Hello, %s!\n", $self->name() ); }

package main;

my $hi = Hello->new( 'Bob' ); is( $hi->greet(), "Hello, Bob!\n", ':Method attribute should autoadd invocant shift' );

my $spacey_name = 'spacey 0'; my $spacey_greet = 'spacey 1';

is( $hi->$spacey_name(), 'Bob', '... also installing first method as "spacey 0"' ); is( $hi->$spacey_greet(), "Hello, Bob!\n", '... and second as "spacey 1"' );

There are two tricks in the implementation, one of which is an "eww, evil--and CLEVER!" trick and the other is something either you know about or you don't.

Quieter Test Output:

I never really understood the point of <code>prove</code>; I write my tests as normal Perl programs, launchable from the command line. <code>perl t/testname.t</code> did most of what I wanted to do.

With more than 24 tests in a file, though, I often promised myself to write or find someday a variant test harness that ignored successes and reported only failures or an all clear message.

I looked at the code of <code>prove</code> today to find where to add it, but again, it doesn't seem to do anything I really need and there was no way to add it. I moved on to Test::Harness::Straps. I don't know if anyone's ever actually used this module (and it's admittedly a bit of a mess inside, though the long-promised Test::Builder refactoring may make it easier to write), but it took 67 lines of code to do what I needed (and a little more) in a well-factored way.

I call it qtest:


use strict; use warnings;

use Test::Harness::Straps;

my $strap = Test::Harness::Straps->new();

for my $file (@ARGV) { next unless -f $file;

my %results = $strap->analyze_file( $file );

if ($results{passing}) { report( sprintf('All (%d) tests passed in %s', $results{seen}, $file)); } elsif ($results{skip_all}) { report( sprintf('All (%d) tests skipped in %s', $results{seen}, $file)); } else { report( find_failures( $file, \%results ) ); } }

sub report { my $message = shift; print "$message\n"; }

sub find_failures { my ($file, $results) = @_; my $report = create_header($file, @{$results}{qw( max seen ok )}); my $count = 0;

for my $test ( @{ $results->{details} } ) { $count++; next if $test->{ok}; $report .= create_test_result( $count, @{ $test }{qw( name reason ) } ); }

return $report; }

sub create_header { my ($file, $expected, $seen, $passed) = @_; my $failed = $seen - $passed; return sprintf "File '%s'\nExpected %d / Seen %d / Okay %d / Failed %d\n", @_, $failed; }

sub create_test_result { my ($number, $name, $reason) = @_; $name =~ s/^-\s*//; $reason ||= ''; $reason = " ($reason)" if $reason; return sprintf "\tTest #%d: %s%s\n", $number, $name, $reason; }

In practice, I may tweak the formatting somewhat. It might be nice to report skips and surprisingly-passing TODO tests, too. I also might want a flag to end after the first failure, to solve the problem of cascading failures rolling off of the screen. Still, this was a lot simpler than I thought and already makes my life easier.

221 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!