NAME¶

Test::Class - Easily create test classes in an xUnit/JUnit style

VERSION¶

version 0.51

SYNOPSIS¶

  package Example::Test;
  use base qw(Test::Class);
  use Test::More;
  # setup methods are run before every test method.
  sub make_fixture : Test(setup) {
      my $array = [1, 2];
      shift->{test_array} = $array;
  }
  # a test method that runs 1 test
  sub test_push : Test {
      my $array = shift->{test_array};
      push @$array, 3;
      is_deeply($array, [1, 2, 3], 'push worked');
  }
  # a test method that runs 4 tests
  sub test_pop : Test(4) {
      my $array = shift->{test_array};
      is(pop @$array, 2, 'pop = 2');
      is(pop @$array, 1, 'pop = 1');
      is_deeply($array, [], 'array empty');
      is(pop @$array, undef, 'pop = undef');
  }
  # teardown methods are run after every test method.
  sub teardown : Test(teardown) {
      my $array = shift->{test_array};
      diag("array = (@$array) after test(s)");
  }

later in a nearby .t file

  #! /usr/bin/perl
  use Example::Test;
  # run all the test methods in Example::Test
  Test::Class->runtests;

Outputs:

  1..5
  ok 1 - pop = 2
  ok 2 - pop = 1
  ok 3 - array empty
  ok 4 - pop = undef
  # array = () after test(s)
  ok 5 - push worked
  # array = (1 2 3) after test(s)

DESCRIPTION¶

Test::Class provides a simple way of creating classes and objects to test your code in an xUnit style.

Built using Test::Builder, it was designed to work with other Test::Builder based modules (Test::More, Test::Differences, Test::Exception, etc.).

Note: This module will make more sense, if you are already familiar with the "standard" mechanisms for testing perl code. Those unfamiliar with Test::Harness, Test::Simple, Test::More and friends should go take a look at them now. Test::Tutorial is a good starting point.

INTRODUCTION¶

TEST CLASSES¶

A test class is just a class that inherits from Test::Class. Defining a test class is as simple as doing:

  package Example::Test;
  use base qw(Test::Class);

Since Test::Class does not provide its own test functions, but uses those provided by Test::More and friends, you will nearly always also want to have:

  use Test::More;

to import the test functions into your test class.

METHOD TYPES¶

There are three different types of method you can define using Test::Class.

  package Example::Test;
  use base qw(Test::Class);
  use Test::More;
  sub subtraction : Test {
      is( 2-1, 1, 'subtraction works' );
  }

  sub addition : Test(2) {
      is(10 + 20, 30, 'addition works');
      is(20 + 10, 30, '  both ways');
  }

  sub check_class : Test(no_plan) {
      my $objects = shift->{objects};
      isa_ok($_, "Object") foreach @$objects;
  }

  sub check_class : Tests {
      my $objects = shift->{objects};
      isa_ok($_, "Object") foreach @$objects;
  }

  sub before : Test(setup)    { diag("running before test") }
  sub after  : Test(teardown) { diag("running after test") }

  sub pig : Test(setup) {
      my $self = shift;
      $self->{test_pig} = Pig->new;
  }
  sub born_hungry : Test {
      my $pig = shift->{test_pig};
      is($pig->hungry, 'pigs are born hungry');
  }
  sub eats : Test(3) {
      my $pig = shift->{test_pig};
      ok(  $pig->feed,   'pig fed okay');
      ok(! $pig->hungry, 'fed pig not hungry');
      ok(! $pig->feed,   'cannot feed full pig');
  }

  sub pig_alive : Test(teardown => 1) {
      my $pig = shift->{test_pig};
      ok($pig->alive, 'pig survived tests' );
  }

  sub db_connect : Test(startup) {
      shift->{dbi} = DBI->connect;
  }
  sub db_disconnect : Test(shutdown) {
      shift->{dbi}->disconnect;
  }

  sub example : Test(startup => 1) {
      ok(1, 'a startup method with one test');
  }

  sub example : Test(startup => no_plan) {
     ok(1, q{The first of many tests that don't want to have to count});
     ...
  }

RUNNING TESTS¶

You run test methods with runtests(). Doing:

  Test::Class->runtests

runs all of the test methods in every loaded test class. This allows you to easily load multiple test classes in a *.t file and run them all.

  #! /usr/bin/perl
  # load all the test classes I want to run
  use Foo::Test;
  use Foo::Bar::Test;
  use Foo::Fribble::Test;
  use Foo::Ni::Test;
  # and run them all
  Test::Class->runtests;

You can use Test::Class::Load to automatically load all the test classes in a given set of directories.

If you need finer control you can create individual test objects with new(). For example to just run the tests in the test class "Foo::Bar::Test" you can do:

  Example::Test->new->runtests

You can also pass runtests() a list of test objects to run. For example:

  my $o1 = Example::Test->new;
  my $o2 = Another::Test->new;
  # runs all the tests in $o1 and $o2
  $o1->runtests($o2);

Since, by definition, the base Test::Class has no tests, you could also have written:

  my $o1 = Example::Test->new;
  my $o2 = Another::Test->new;
  Test::Class->runtests($o1, $o2);

If you pass runtests() class names it will automatically create test objects for you, so the above can be written more compactly as:

  Test::Class->runtests(qw( Example::Test Another::Test ))

In all of the above examples runtests() will look at the number of tests both test classes run and output an appropriate test header for Test::Harness automatically.

What happens if you run test classes and normal tests in the same script? For example:

  Test::Class->runtests;
  ok(Example->new->foo, 'a test not in the test class');
  ok(Example->new->bar, 'ditto');

Test::Harness will complain that it saw more tests than it expected since the test header output by runtests() will not include the two normal tests.

To overcome this problem you can pass an integer value to runtests(). This is added to the total number of tests in the test header. So the problematic example can be rewritten as follows:

  Test::Class->runtests(+2);
  ok(Example->new->foo, 'a test not in the test class');
  ok(Example->new->bar, 'ditto');

If you prefer to write your test plan explicitly you can use expected_tests() to find out the number of tests a class/object is expected to run.

Since runtests() will not output a test plan if one has already been set, the previous example can be written as:

  plan tests => Test::Class->expected_tests(+2);
  Test::Class->runtests;
  ok(Example->new->foo, 'a test not in the test class');
  ok(Example->new->bar, 'ditto');

Remember: Test objects are just normal perl objects. Test classes are just normal perl classes. Setup, test and teardown methods are just normal methods. You are completely free to have other methods in your class that are called from your test methods, or have object specific "new" and "DESTROY" methods.

In particular you can override the new() method to pass parameters to your test object, or re-define the number of tests a method will run. See num_method_tests() for an example.

TEST DESCRIPTIONS¶

The test functions you import from Test::More and other Test::Builder based modules usually take an optional third argument that specifies the test description, for example:

  is $something, $something_else, 'a description of my test';

If you do not supply a test description, and the test function does not supply its own default, then Test::Class will use the name of the currently running test method, replacing all "_" characters with spaces so:

  sub one_plus_one_is_two : Test {
      is 1+1, 2;
  }

will result in:

  ok 1 - one plus one is two

RUNNING ORDER OF METHODS¶

Methods of each type are run in the following order:

1.

All of the startup methods in alphabetical order

2.

For each test method, in alphabetical order:

3.

All of the shutdown methods in alphabetical order.

Most of the time you should not care what order tests are run in, but it can occasionally be useful to force some test methods to be run early. For example:

  sub _check_new {
      my $self = shift;
      isa_ok(Object->new, "Object") or $self->BAILOUT('new fails!');
  }

The leading "_" will force the above method to run first - allowing the entire suite to be aborted before any other test methods run.

HANDLING EXCEPTIONS¶

If a startup, setup, test, teardown or shutdown method dies then runtests() will catch the exception and fail any remaining test. For example:

  sub test_object : Test(2) {
      my $object = Object->new;
      isa_ok( $object, "Object" ) or die "could not create object\n";
      ok( $object->open, "open worked" );
  }

will produce the following if the first test failed:

  not ok 1 - The object isa Object
  #   Failed test 'The object isa Object'
  #   at /Users/adrianh/Desktop/foo.pl line 14.
  #   (in MyTest->test_object)
  #     The object isn't defined
  not ok 2 - test_object died (could not create object)
  #   Failed test 'test_object died (could not create object)'
  #   at /Users/adrianh/Desktop/foo.pl line 19.
  #   (in MyTest->test_object)

This can considerably simplify testing code that throws exceptions.

Rather than having to explicitly check that the code exited normally (e.g. with "lives_ok" in Test::Exception) the test will fail automatically - without aborting the other test methods. For example contrast:

  use Test::Exception;
  my $file;
  lives_ok { $file = read_file('test.txt') } 'file read';
  is($file, "content", 'test file read');

with:

  sub read_file : Test {
      is(read_file('test.txt'), "content", 'test file read');
  }

If more than one test remains after an exception then the first one is failed, and the remaining ones are skipped.

If the setup method of a test method dies, then all of the remaining setup and shutdown methods are also skipped.

Since startup methods will usually be creating state needed by all the other test methods, an exception within a startup method will prevent all other test methods of that class running.

RETURNING EARLY¶

If a test method returns before it has run all of its tests, by default the missing tests are deemed to have been skipped; see "Skipped Tests" for more information.

However, if the class's "fail_if_returned_early" method returns true, then the missing tests will be deemed to have failed. For example,

  package MyClass;
  use base 'Test::Class';
  sub fail_if_returned_early { 1 }
  sub oops : Tests(8) {
    for (my $n=1; $n*$n<50; ++$n) {
      ok 1, "$n squared is less than fifty";
    }
  }

RETURNING LATE¶

If a test method runs too many tests, by default the test plan succeeds.

However, if the class's "fail_if_returned_late" method returns true, then the extra tests will trigger a failure. For example,

  package MyClass;
  use base 'Test::Class';
  sub fail_if_returned_late { 1 }
  sub oops : Tests(1) {
    ok 1, "just a simple test";
    ok 1, "just a simple test"; #oops I copied and pasted too many tests
  }

SKIPPED TESTS¶

You can skip the rest of the tests in a method by returning from the method before all the test have finished running (but see "Returning Early" for how to change this). The value returned is used as the reason for the tests being skipped.

This makes managing tests that can be skipped for multiple reasons very simple. For example:

  sub flying_pigs : Test(5) {
      my $pig = Pig->new;
      isa_ok($pig, 'Pig')           or return("cannot breed pigs")
      can_ok($pig, 'takeoff')       or return("pigs don't fly here");
      ok($pig->takeoff, 'takeoff')  or return("takeoff failed");
      ok( $pig->altitude > 0, 'Pig is airborne' );
      ok( $pig->airspeed > 0, '  and moving'    );
  }

If you run this test in an environment where "Pig->new" worked and the takeoff method existed, but failed when ran, you would get:

  ok 1 - The object isa Pig
  ok 2 - can takeoff
  not ok 3 - takeoff
  ok 4 # skip takeoff failed
  ok 5 # skip takeoff failed

You can also skip tests just as you do in Test::More or Test::Builder - see "Conditional tests" in Test::More for more information.

Note: if you want to skip tests in a method with "no_plan" tests then you have to explicitly skip the tests in the method - since Test::Class cannot determine how many tests (if any) should be skipped:

  sub test_objects : Tests {
      my $self = shift;
      my $objects = $self->{objects};
      if (@$objects) {
          isa_ok($_, "Object") foreach (@$objects);
      } else {
          $self->builder->skip("no objects to test");
      }
  }

Another way of overcoming this problem is to explicitly set the number of tests for the method at run time using num_method_tests() or "num_tests".

You can make a test class skip all of its tests by setting SKIP_CLASS() before runtests() is called.

TO DO TESTS¶

You can create todo tests just as you do in Test::More and Test::Builder using the $TODO variable. For example:

  sub live_test : Test  {
      local $TODO = "live currently unimplemented";
      ok(Object->live, "object live");
  }

See "Todo tests" in Test::Harness for more information.

EXTENDING TEST CLASSES BY INHERITANCE¶

You can extend test methods by inheritance in the usual way. For example consider the following test class for a "Pig" object.

  package Pig::Test;
  use base qw(Test::Class);
  use Test::More;
  sub testing_class { "Pig" }
  sub new_args { (-age => 3) }
  sub setup : Test(setup) {
      my $self = shift;
      my $class = $self->testing_class;
      my @args = $self->new_args;
      $self->{pig} = $class->new( @args );
  }
  sub _creation : Test {
      my $self = shift;
      isa_ok($self->{pig}, $self->testing_class)
              or $self->FAIL_ALL('Pig->new failed');
  }
  sub check_fields : Test {
      my $pig = shift->{pig}
      is($pig->age, 3, "age accessed");
  }

Next consider "NamedPig" a subclass of "Pig" where you can give your pig a name.

We want to make sure that all the tests for the "Pig" object still work for "NamedPig". We can do this by subclassing "Pig::Test" and overriding the "testing_class" and "new_args" methods.

  package NamedPig::Test;
  use base qw(Pig::Test);
  use Test::More;
  sub testing_class { "NamedPig" }
  sub new_args { (shift->SUPER::new_args, -name => 'Porky') }

Now we need to test the name method. We could write another test method, but we also have the option of extending the existing "check_fields" method.

  sub check_fields : Test(2) {
      my $self = shift;
      $self->SUPER::check_fields;
      is($self->{pig}->name, 'Porky', 'name accessed');
  }

While the above works, the total number of tests for the method is dependent on the number of tests in its "SUPER::check_fields". If we add a test to "Pig::Test->check_fields" we will also have to update the number of tests of "NamedPig::test->check_fields".

Test::Class allows us to state explicitly that we are adding tests to an existing method by using the "+" prefix. Since we are adding a single test to "check_fields", it can be rewritten as:

  sub check_fields : Test(+1) {
      my $self = shift;
      $self->SUPER::check_fields;
      is($self->{pig}->name, 'Porky', 'name accessed');
  }

With the above definition you can add tests to "check_fields" in "Pig::Test" without affecting "NamedPig::Test".

RUNNING INDIVIDUAL TESTS¶

NOTE: The exact mechanism for running individual tests is likely to change in the future.

Sometimes you just want to run a single test. Commenting out other tests or writing code to skip them can be a hassle, so you can specify the "TEST_METHOD" environment variable. The value is expected to be a valid regular expression and, if present, only runs test methods whose names match the regular expression. Startup, setup, teardown and shutdown tests will still be run.

One easy way of doing this is by specifying the environment variable before the "runtests" method is called.

Running a test named "customer_profile":

 #! /usr/bin/perl
 use Example::Test;
 $ENV{TEST_METHOD} = 'customer_profile';
 Test::Class->runtests;

Running all tests with "customer" in their name:

 #! /usr/bin/perl
 use Example::Test;
 $ENV{TEST_METHOD} = '.*customer.*';
 Test::Class->runtests;

If you specify an invalid regular expression, your tests will not be run:

 #! /usr/bin/perl
 use Example::Test;
 $ENV{TEST_METHOD} = 'C++';
 Test::Class->runtests;

And when you run it:

 TEST_METHOD (C++) is not a valid regular expression: Search pattern \
 not terminated at (eval 17) line 1.

ORGANISING YOUR TEST CLASSES¶

You can, of course, organise your test modules as you wish. My personal preferences is:

• Name test classes with a suffix of "::Test" so the test class for the "Foo::Bar" module would be "Foo::Bar::Test".
• Place all test classes in t/lib.

The Test::Class::Load provides a simple mechanism for easily loading all of the test classes in a given set of directories.

A NOTE ON LOADING TEST CLASSES¶

Due to its use of subroutine attributes Test::Class based modules must be loaded at compile rather than run time. This is because the :Test attribute is applied by a CHECK block.

This can be problematic if you want to dynamically load Test::Class modules. Basically while:

  require $some_test_class;

will break, doing:

  BEGIN { require $some_test_class }

will work just fine. For more information on CHECK blocks see "BEGIN, CHECK, INIT and END" in perlmod.

If you still can't arrange for your classes to be loaded at runtime, you could use an alternative mechanism for adding your tests:

  # sub test_something : Test(3) {...}
  # becomes
  sub test_something {...}
  __PACKAGE__->add_testinfo('test_something', test => 3);

See the add_testinfo method for more details.

Additionally, if you've forgotten to enable warnings and have two test subs called the same thing, you will get the same error.

GENERAL FILTERING OF TESTS¶

The use of $ENV{TEST_METHOD} to run just a subset of tests is useful, but sometimes it doesn't give the level of granularity that you desire. Another feature of this class is the ability to do filtering on other static criteria. In order to permit this, a generic filtering method is supported. This can be used by specifying coderefs to the 'add_filter' method of this class.

In determining which tests should be run, all filters that have previously been specified via the add_filter method will be run in-turn for each normal test method. If any of these filters return a false value, the method will not be executed, or included in the number of tests. Note that filters will only be run for normal test methods, they are ignored for startup, shutdown, setup, and teardown test methods.

Note that test filters are global, and will affect all tests in all classes, not just the one that they were defined in.

An example of this mechanism that mostly simulates the use of TEST_METHOD above is:

 package MyTests;
 use Test::More;
 use base qw( Test::Class );
 my $MYTEST_METHOD = qr/^t_not_filtered$/;
 my $filter = sub {
    my ( $test_class, $test_method ) = @_;
    return $test_method =~ $MYTEST_METHOD;
 }
 Test::Class->add_filter( $filter );
 sub t_filtered : Test( 1 ) {
    fail( "filtered test run" );
 }
 sub t_not_filtered : Test( 1 ) {
    pass( "unfiltered test run" );
 }

METHODS¶

HELP FOR CONFUSED JUNIT USERS¶

This section is for people who have used JUnit (or similar) and are confused because they don't see the TestCase/Suite/Runner class framework they were expecting. Here we take each of the major classes in JUnit and compare them with their equivalent Perl testing modules.

Class Assert

The test assertions provided by Assert correspond to the test functions provided by the Test::Builder based modules (Test::More, Test::Exception, Test::Differences, etc.)

Unlike JUnit the test functions supplied by Test::More et al do not throw exceptions on failure. They just report the failure to STDOUT where it is collected by Test::Harness. This means that where you have

  sub foo : Test(2) {
      ok($foo->method1);
      ok($foo->method2);
  }
    

The second test will run if the first one fails. You can emulate the JUnit way of doing it by throwing an explicit exception on test failure:

  sub foo : Test(2) {
      ok($foo->method1) or die "method1 failed";
      ok($foo->method2);
  }
    

The exception will be caught by Test::Class and the other test automatically failed.

Class TestCase

Test::Class corresponds to TestCase in JUnit.

In Test::Class setup, test and teardown methods are marked explicitly using the Test attribute. Since we need to know the total number of tests to provide a test plan for Test::Harness, we also state how many tests each method runs.

Unlike JUnit you can have multiple setup/teardown methods in a class.

Class TestSuite

Test::Class also does the work that would be done by TestSuite in JUnit.

Since the methods are marked with attributes, Test::Class knows what is and isn't a test method. This allows it to run all the test methods without having the developer create a suite manually, or use reflection to dynamically determine the test methods by name. See the runtests() method for more details.

The running order of the test methods is fixed in Test::Class. Methods are executed in alphabetical order.

To run individual test methods, see "RUNNING INDIVIDUAL TESTS".

Class TestRunner

Test::Harness does the work of the TestRunner in JUnit. It collects the test results (sent to STDOUT) and collates the results.

Unlike JUnit there is no distinction made by Test::Harness between errors and failures. However, it does support skipped and todo test - which JUnit does not.

If you want to write your own test runners you should look at Test::Harness::Straps.

OTHER MODULES FOR XUNIT TESTING IN PERL¶

In addition to Test::Class there are two other distributions for xUnit testing in perl. Both have a longer history than Test::Class and might be more suitable for your needs.

I am biased since I wrote Test::Class - so please read the following with appropriate levels of scepticism. If you think I have misrepresented the modules please let me know.

Test::SimpleUnit

A very simple unit testing framework. If you are looking for a lightweight single module solution this might be for you.

The advantage of Test::SimpleUnit is that it is simple! Just one module with a smallish API to learn.

Of course this is also the disadvantage.

It's not class based so you cannot create testing classes to reuse and extend.

It doesn't use Test::Builder so it's difficult to extend or integrate with other testing modules. If you are already familiar with Test::Builder, Test::More and friends you will have to learn a new test assertion API. It does not support todo tests.

Test::Unit

Test::Unit is a port of JUnit <http://www.junit.org/> into perl. If you have used JUnit then the Test::Unit framework should be very familiar.

It is class based so you can easily reuse your test classes and extend by subclassing. You get a nice flexible framework you can tweak to your heart's content. If you can run Tk you also get a graphical test runner.

However, Test::Unit is not based on Test::Builder. You cannot easily move Test::Builder based test functions into Test::Unit based classes. You have to learn another test assertion API.

Test::Unit implements it's own testing framework separate from Test::Harness. You can retrofit *.t scripts as unit tests, and output test results in the format that Test::Harness expects, but things like todo tests and skipping tests are not supported.

SUPPORT¶

Bugs may be submitted through GitHub issues <https://github.com/szabgab/test-class/issues>

There is also an irc channel available for users of this distribution, at "#perl-qa" on "irc.perl.org" <irc://irc.perl.org/#perl-qa>.

TO DO¶

If you think this module should do something that it doesn't (or does something that it shouldn't) please let me know.

You can see an old to do list at <http://adrianh.tadalist.com/lists/public/4798>, with an RSS feed of changes at <http://adrianh.tadalist.com/lists/feed_public/4798>.

ACKNOWLEDGMENTS¶

This is yet another implementation of the ideas from Kent Beck's Testing Framework paper <http://www.xprogramming.com/testfram.htm>.

Thanks to Adam Kennedy, agianni, Alexander D'Archangel, Andrew Grangaard, Apocalypse, Ask Bjorn Hansen, Chris Dolan, Chris Williams, Corion, Cosimo Streppone, Daniel Berger, Dave Evans, Dave O'Neill, David Cantrell, David Wheeler, Diab Jerius, Emil Jansson, Gunnar Wolf, Hai Pham, Hynek, imacat, Jeff Deifik, Jim Brandt, Jochen Stenzel, Johan Lindstrom, John West, Jonathan R. Warden, Joshua ben Jore, Jost Krieger, Ken Fox, Kenichi Ishigaki Lee Goddard, Mark Morgan, Mark Reynolds, Mark Stosberg, Martin Ferrari, Mathieu Sauve-Frankel, Matt Trout, Matt Williamson, Michael G Schwern, Murat Uenalan, Naveed Massjouni, Nicholas Clark, Ovid, Piers Cawley, Rob Kinyon, Sam Raymer, Scott Lanning, Sebastien Aperghis-Tramoni, Steve Kirkup, Stray Toaster, Ted Carnahan, Terrence Brannon, Todd W, Tom Metro, Tony Bowden, Tony Edwardson, William McKee, various anonymous folk and all the fine people on perl-qa for their feedback, patches, suggestions and nagging.

This module wouldn't be possible without the excellent Test::Builder. Thanks to chromatic and Michael G Schwern for creating such a useful module.

AUTHORS¶

Adrian Howard <adrianh@quietstars.com>, Curtis "Ovid" Poe, <ovid at cpan.org>, Mark Morgan <makk384@gmail.com>.

SEE ALSO¶

Test::Class::Load

Simple way to load "Test::Class" classes automatically.

Test::Class::Most

Test::Class with additional conveniences to reduce need for some boilerplate code. Also makes Test::Most testing functions available.

Test::Class::Moose

Testing framework allows you to write your tests in Moose and test Moose and non-Moose code. It offers reporting, extensibility, test inheritance, parallel testing and more.

Perl Testing: A Developer's Notebook by Ian Langworth and chromatic

Chapter 8 covers using Test::Class.

Advanced Perl Programming, second edition by Simon Cozens

Chapter 8 has a few pages on using Test::Class.

The Perl Journal, April 2003

Includes the article "Test-Driven Development in Perl" by Piers Cawley that uses Test::Class.

Test::Class Tutorial series written by Curtis "Ovid" Poe

Test::Builder

Support module for building test libraries.

Test::Simple & Test::More

Basic utilities for writing tests.

<https://qa.perl.org/test-modules.html>

Overview of some of the many testing modules available on CPAN.

Test::Object

Another approach to object oriented testing.

Test::Group and Test::Block

Alternatives to grouping sets of tests together.

The following modules use Test::Class as part of their test suite. You might want to look at them for usage examples:

The following modules are not based on Test::Builder, but may be of interest as alternatives to Test::Class.

Test::Unit

Perl unit testing framework closely modeled on JUnit.

Test::SimpleUnit

A very simple unit testing framework.

LICENCE¶

Copyright 2002-2010 Adrian Howard, All Rights Reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.