Test::Builder::Tester considered harmful. Use Test::Tester.

[Michael G Schwern]

Executive Summary: If you're using Test::Builder::Tester to test your Test
module, switch to Test::Tester. It will make the transition to Test::Builder
1.5 smoother and avoid future breakage due to TAP formatting changes.

A lot of the work done fixing CPAN Test::* modules to be compatible with
Test::Builder 1.5 has been to do with small changes to the TAP format. The
TAP version header and things like the casing of "# SKIP". Stupid little
things. Testing a test library currently involves comparing the output TAP.
This sucks, even with Test::Builder::Tester.

The whole idea of testing test modules by looking at the formatted output
suuuuucks. Test::Builder::Tester is a gallant effort,

Test::Builder 1.5 is event driven, so there's no need to look at the formatted
output, you can look at the events directly and get a complete view of what's
going on. For example, this test...

ok( 1 );
is( 23, 42 );
done_testing;

Generates something like this:

# TB2::Event::TestStart
# TB2::Result
ok( 1 );

# TB2::Result
# TB2::Event::Log (for the failure diagnostics)
is( 23, 42 );

# TB2::SetPlan
# TB2::Event::TestEnd
done_testing;

There's a couple of interfaces to test via events. Test::Tester is an
existing one that works with both 0.x and 1.5, so that would be the preferred
tool to use now. TB2::Tester is the one specifically written for
Test::Builder 1.5.

My plan is to mark Test::Builder::Tester as "discouraged" and focus on making
it work as a transitional tool. That means emulating as many 0.x quirks as
possible to keep existing TBT-based tests working, but that also means
sacrificing improving it. IMO its approach is dead, so that's ok.

What say?

[Buddy Burden]

schwern,

Post by Michael G Schwern
Executive Summary: If you're using Test::Builder::Tester to test your Test
module, switch to Test::Tester. ...
What say?

Well, I say that it's a bit of a PITA, but I'll add it to my TODO
list. I recall now that the skip/SKIP thing is what was causing some
CPAN Testers failures for Test::File, so I guess I'l just switch it
over to Test::Tester and that'll solve it, eh? Hopefully it won't be
too awful, but my recollection is that TBT and TT have _completely_
different approaches when it comes to testing, so it won't be trivial
either. So I guess I'm torn between saying "yay for progress!" and
"bummer for my already overstuffed schedule." ;->

-- Buddy

[Michael G Schwern]

Post by Buddy Burden
Well, I say that it's a bit of a PITA, but I'll add it to my TODO
list. I recall now that the skip/SKIP thing is what was causing some
CPAN Testers failures for Test::File, so I guess I'l just switch it
over to Test::Tester and that'll solve it, eh? Hopefully it won't be
too awful, but my recollection is that TBT and TT have _completely_
different approaches when it comes to testing, so it won't be trivial
either. So I guess I'm torn between saying "yay for progress!" and
"bummer for my already overstuffed schedule." ;->

Test::Builder::Tester should smooth that over as of alpha 3. It's using a
special TAP formatter which emulates TB1 quirks including the skip/SKIP thing.

TBT's future is being sacrificed to work as a Test::Builder 0.x -> 1.5
compatibility layer. The behaviors you're testing with TBT will always be
stuck in the past. Test::Tester also spans 0.x -> 1.5 but it does so with
more grace.

You should be able to leave your Test::Builder::Tester stuff alone, it'll
work, but eventually you're going to want to change them.

[Buddy Burden]

schwern,

Post by Michael G Schwern
You should be able to leave your Test::Builder::Tester stuff alone, it'll
work, but eventually you're going to want to change them.

Excellent. So I'll leave it on my todo list, just bump it down in priority. ;->


-- Buddy

[Michael G Schwern]

Post by Michael G Schwern
Executive Summary: If you're using Test::Builder::Tester to test your Test
module, switch to Test::Tester. It will make the transition to Test::Builder
1.5 smoother and avoid future breakage due to TAP formatting changes.

https://metacpan.org/requires/module/Test::Builder::Tester

Sorry, I wasn't clear. In the short term, TBT will continue to work.

In the short term, TBT uses a special TAP formatter which emulates the quirks
of the existing behavior. So whatever tests you have with whatever
assumptions about the TAP format will still work.

In the long term, TBT will become further and further divorced from reality.
It will be frozen in its format and behaviors and not reflecting actual outputs.

Finally, rather than trying to interpret what happened based on the TAP, its
easier to look at the test events directly with Test::Tester or TB2::Tester.