Google Publishes New Robots.txt Explainer

Google publishes new Robots.txt documentation that provide SEOs and publishers with a refresher on how to control search crawler behavior

The post Google Publishes New Robots.txt Explainer appeared first on Search Engine Journal.