why search engine optimization use robots.txt

In SEO you may need to prevent search engine spiders from crawling certain pages. Such as login pages, and your own site search results pages. Place metatags…

Leave a Reply