Discussion:
Using NYTProf for code coverage?
Mark Stosberg
2013-02-27 13:49:44 UTC
Permalink
Greetings,

I was tasked with working on code coverage for a large project, but had
difficulty getting Devel::Cover to run. We had Devel::NYTProf handy,
and I realized that although that tool focuses on profiling, it produces
data that appears that it can be used for code coverage instead.

Having one tool that could be used for both profiling and code coverage
seems like a nice win. Have other people looked into using
Devel::NYTProf this way? Is there a reason why it would be undesirable?
Is Devel::Cover still the go-to tool for code coverage?

Below are some notes on how I used Devel::NYTProf for code coverage.

Mark

###

I was able to produce a report which looked like this:

0 0 Config::compile_date
0 0 Config::config_re
0 0 Config::config_sh
0 0 Config::config_vars
0 0 Config::header_files
0 0 Config::launcher
0 0 Config::local_patches
0 0 Config::myconfig
0 0 Cwd::chdir
0 0 Cwd::fast_abs_path
0 0 Data::Dumper::qquote
1 1 Cwd::getcwd
1 1 JSON::XS::encode
7 1 File::Find::CORE:closedir

It's the number of calls, places called from, and the subroutine.

To generate, in Apache add the following:

MaxClients 1
MaxRequestsPerChild 0

and this to the mod_perl startup.pl:

my $time = `/bin/date +%Y-%m-%dT%H.%M.%S.%N`;
chomp($time);
$ENV{NYTPROF} =
"file=/tmp/nytprof/nytprof.$time.out:addpid=1:endatexit=1";

require Devel::NYTProf::Apache;

Next, run your web request and a file should be created for parent and
child:

nytprof.2013-02-08T13.18.18.209426295.out.31929
nytprof.2013-02-08T13.18.18.209426295.out.31949

Now, merge the files: nytprofmerge -v --out=nytprof-merged.out
nytprof.2013-02-0*.

Create the report: covverage.pl --file nytprof-merged.out --out
merged_coverage --minimal and parse the data. Here is a script to parse:

cat *.tsv | perl -ne 'print if /\t/' | perl -a -ne
'print("$F[0]\t$F[1]\t$F[-1]\n") if $F[0] =~ /\d/' | sort -k 2,2 | sort
-u | grep -v ::BEGIN | grep -v __ANON | sort -n

This should give you something like the above.
David Cantrell
2013-02-27 16:32:18 UTC
Permalink
Post by Mark Stosberg
I was tasked with working on code coverage for a large project, but had
difficulty getting Devel::Cover to run. We had Devel::NYTProf handy,
and I realized that although that tool focuses on profiling, it produces
data that appears that it can be used for code coverage instead.
Having one tool that could be used for both profiling and code coverage
seems like a nice win. Have other people looked into using
Devel::NYTProf this way? Is there a reason why it would be undesirable?
Is Devel::Cover still the go-to tool for code coverage?
Devel::Cover tells you a lot more than just whether a line of code was
executed. Consider a line like this:

do_something if($foo || $bar);

Devel::Cover will tell you which of the three possible conditions were
covered by your tests, those being:
* neither $foo nor $bar are true;
* $foo is false, $bar is true;
* $foo is true and we don't care about $bar

I'm surprised that you can bet NYTProf working but not Cover - in my
experience, they either both work easily or both fail horribly.
--
David Cantrell | Bourgeois reactionary pig

Vegetarian: n: a person who, due to malnutrition caused by
poor lifestyle choices, is eight times more likely to
catch TB than a normal person
Mark Stosberg
2013-02-28 15:19:57 UTC
Permalink
Post by Mark Stosberg
I was tasked with working on code coverage for a large project, but had
difficulty getting Devel::Cover to run.
Can you maybe go into details as to why it won't run, or maybe condense
it to a repeatable case? PJCJ is, with help from others, actively
working on D::C and fixing bugs, so reports of broken things are quite
useful. :)
Thanks for all the replies. Sounds like the consensus is that I should
expect Devel::Cover to work, and get help if it doesn't. I'll give it
another shot.

I appreciated David Cantrell's additional detail about the additional
value that Devel::Cover provides.

Mark
Paul Johnson
2013-02-28 15:34:23 UTC
Permalink
Post by Mark Stosberg
Thanks for all the replies. Sounds like the consensus is that I should
expect Devel::Cover to work, and get help if it doesn't. I'll give it
another shot.
You can get help here or, if you've narrowed things down to a bug,
please post it to github.
--
Paul Johnson - ***@pjcj.net
http://www.pjcj.net
Christian Walde
2013-02-28 11:02:39 UTC
Permalink
Post by Mark Stosberg
I was tasked with working on code coverage for a large project, but had
difficulty getting Devel::Cover to run.
Can you maybe go into details as to why it won't run, or maybe condense it
to a repeatable case? PJCJ is, with help from others, actively working on
D::C and fixing bugs, so reports of broken things are quite useful. :)
--
With regards,
Christian Walde
Loading...