The gridBOT is under development.

Robots.txt support is experimental. Please excuse bad bot behavior until we get everything ironed out, but we are happy you noticed. ;)