Welcome to ftp.nluug.nl Current directory: /pub/os/Linux/distr/salix/x86_64/extra-14.2/source/perl/perl-www-robotrules/

Current bandwidth utilization 25.42 Mbit/s

This module parses /robots.txt files as specified in "A Standard for
Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html>
Webmasters can use the /robots.txt file to forbid conforming robots
from accessing parts of their web site.
The parsed files are kept in a WWW::RobotRules object, and this
object provides methods to check if access to a given URL is
prohibited. The same WWW::RobotRules object can be used for one
or more parsed /robots.txt files on any number of hosts.

 Name                                                     Last modified      Size  
 Parent Directory                                                              -   
 README                                                   24-Aug-2016 02:06  488   
 WWW-RobotRules-6.02.tar.gz                               24-Aug-2016 02:43  8.8K  
 perl-www-robotrules.SlackBuild                           26-Nov-2013 10:57  2.8K  
 perl-www-robotrules.info                                 24-Aug-2016 02:06  315   
 slack-desc                                               24-Aug-2016 02:06  1.0K  

NLUUG - Open Systems. Open Standards
Become a member and get discounts on conferences and more, see the NLUUG website!