Test::Class annoyances

[Sam Kington]

Hi,

We've been using Test::Class at $WORK a few months now, and it's been very handy indeed. But there were a couple of things that were bothering me about it, and this afternoon I finally got around to having a look at them.

One of the annoyances is the apparently-random order that Test::Class runs classes in. This is not just bizarre, it caused me problems at one point with some of my classes which seemingly for no reason refused to work with Dancer::Test. I eventually narrowed it down to inheritance and the order in which objects were constructed - which, depending on what the *test class was called*, could mean that a child object was created either before or after its parent object.

The other annoyance is that, if you've got a whole bunch of test classes, and you've decided to run them all with Test::Class::Load, there's no way of telling what failed after your test script has finished, other than a useless list of test numbers which doesn't correspond to anything in my code. Setting the environment variable TEST_VERBOSE helps a bit, but it still involves grubbing around in the output of the tests (which unfortunately, because of the use of third-party modules I have no control of, is rather chatty).

I thought to myself "Test::Class is OO, so it should be easy to fix, even if I have to be naughty and sub-class private methods" - but no. All sorts of things that look like they should be methods are called as method($self, @params) rather than $self->method(@params) - I assume to avoid test classes overriding them accidentally.

Anyway, this is what I ended up with: 1) sorting the list of classes, so they're run in a more understandable order, and 2) tracking which classes failed, so we can tell the end user when all tests are done.

# Hack: if we're running multiple test classes, run them in alphabetical
# order, rather than the apparently-random order that Test::Class
# defaults to. And remember which classes failed so we can summarise
# failures when we're done testing.
my @classes_with_failures;
INIT {
no warnings 'redefine';
local *Test::Class::_test_classes = sub {
my ($class) = shift;
return (sort @{ mro::get_isarev($class) }, $class);
};

# Report if any tests failed in a module.
my $orig_all_ok_from = \&Test::Class::_all_ok_from;
local *Test::Class::_all_ok_from = sub {
my $self = shift;
my $all_ok = $orig_all_ok_from->($self, @_);
if (!$all_ok) {
print STDERR "# Tests failed in ", ref($self), "\n";
push @classes_with_failures, ref($self);
}
return $all_ok;
};
use warnings 'redefine';

# And run the test suite.
Test::Class->runtests;
}

END {
if (@classes_with_failures) {
print STDERR "#\n# The following classes had test failures:\n# ",
join("\n# ", @classes_with_failures), "\n#\n";
}
}

Sam