NAME¶

WWW::RobotRules::DB_File - Parse robots.txt files using a disk cache

SYNOPSIS¶

    require WWW::RobotRules::DB_File;
    require LWP::RobotUA;
    #Create a robot useragent that uses a disk caching RobotRules
    $ua = WWW::RobotUA->new( 'my-robot/1.0', 'me@foo.com' ,
          WWW::RobotRules::DB_File->new( 'my-robot/1.0', '/path/cachefile' ));
    #The just use $ua as usual
    $res=$ua->request($req);

DESCRIPTION¶

This is a subclass of WWW::RobotRules that uses the DB_File package to implement disk caching of robots.txt.

METHODS¶

This is a subclass of WWW::RobotRules, so it implements the same methods

$rules = WWW::RobotRules::DB_File->new('my-robot/1.0', /path/cachefile)

This is the constructor. The only difference from the original constructor from WWW::RobotRules is that you here has to specify a cache file as well.

SEE ALSO¶

WWW::RobotRules

AUTHOR¶

Hakan Ardo <hakan@munin.ub2.lu.se>