Google Launches Robots.txt Tester in Webmaster Tools
So the NEW VERSION of Google's Robots.txt Tester is UP and RUNNING now.
This improved version allows you to
- Highlight the errors
- Test Blocked Url's
- Show You the Earlier Version of your Robots.txt file
- Let You edit the Robots.txt file
Here is an Image , Showing some of its properties:
To guide your way through complicated directives, it will highlight the specific one that led to the final decision,You can make changes in the file and test those too, you'll just need to upload the new version of the file to your server afterwards to make the changes take effect.Additionally, you'll be able to review older versions of your robots.txt file, and see when access issues block us from crawling,For example, if Googlebot sees a 500 server error for the robots.txt file, we'll generally pause further crawling of the website.
Asaph Arnon




0 comments:
Post a Comment