The robots meta tag is used to tell search engine crawlers If they're allowed to index a particular web page and follow its links. Determining The most crucial webpages on your site is often fairly tough. Talk to you: which web pages would I prefer to clearly show new https://www.youtube.com/watch?v=EVeTP0aV8gY