[Insight-developers] Download page indexed by google

Julien Jomier jjomier at cs.unc.edu
Thu Aug 19 12:58:31 EDT 2004


Website maintainers,

I was searching on Google: "insight toolkit download", the first link is the
Download.php page that asks for user's information. However, the second link
is the Download.htm which points directly to the download archives. This
means that we can miss track of users if they go directly to the second
link. I don't know if this is the most important thing in the world :).

An easy way to fix this is to add the robots.txt (attached) into the top
level of the ITK website directory and this should tell the robots to avoid
the Download.htm page.
Full documentation regarding the robots.txt file can be found at
http://www.robotstxt.org/wc/norobots.html

Julien
-------------- next part --------------
User-agent: * 
Disallow: /HTML/Download.htm


More information about the Insight-developers mailing list