[vtk-developers] Archives are back (was: www.kitware.com/search.html)

Sebastien BARRE sebastien.barre at kitware.com
Wed May 29 10:46:07 EDT 2002

At 5/29/2002 10:04 AM, Andy Cedilnik wrote:
>Ok, indexing now works properly. We had to hack ht:/dig to ignore robots
>information, but we are all set now.

Just for info (since I will forget all of this in 2 days).
At the moment we indeed have to patch either HtDig or Mailman.

There is an option in HtDig to ignore the robots information, this option 
being just the fact that HtDig will comply with the robots.txt specifications:

A Standard for Robot Exclusion

In robots.txt, you can provide exclusion on a User-agent basis, thus 
allowing HtDig and disallowing everything else.

Sadly, mailman 2.0.x hardcodes robots exclusion in:
    <META NAME="robots" CONTENT="index,nofollow">

This will be solved in mailman 2.2, where the article body will be a 
template that can be customized for each list :
in that case, we will just have to remove the <META> line in the list 
template, and modify the robots.txt accordingly. No patch :)

Sebastien Barre

More information about the vtk-developers mailing list