rejetto forum

robots

Guest · 2 · 2587
« previous next »
Pages: 1

0 Members and 1 Guest are viewing this topic.

NewB

  • Guest
on: October 14, 2004, 09:11:52 PM
I wonder whats this:  

"Requested GET /robots.txt"

i have no robots.txt shared
I get very often on my server.

Offline rejetto

  • Administrator
  • Tireless poster
  • *****
    • Posts: 13523
    • View Profile
Reply #1 on: October 14, 2004, 09:41:54 PM
they are "spiders", software (robots) used by search engines (google) to collect websites

since you have no robots.txt, HFS is replying "404 not found"

Pages: 1