rejetto forum

Software => HFS ~ HTTP File Server => Topic started by: NewB on October 14, 2004, 09:11:52 PM

Title: robots
Post by: NewB on October 14, 2004, 09:11:52 PM

I wonder whats this:  

"Requested GET /robots.txt"

i have no robots.txt shared
I get very often on my server.

Title: robots
Post by: rejetto on October 14, 2004, 09:41:54 PM

they are "spiders", software (robots) used by search engines (google) to collect websites

since you have no robots.txt, HFS is replying "404 not found"