HTML and CSS Reference
In-Depth Information
To prevent this, you can add X-Robots-Tag HTTP header tags by appending and
uncommenting the following code snippet to the .htaccess file on the staging serv-
er:
#
------------------------------------------------------------
# Disable URL indexing by crawlers (FOR
DEVELOPMENT/STAGE)
#
------------------------------------------------------------
# Avoid search engines (Google, Yahoo, etc)
indexing website's content
# http://yoast.com/prevent-site-being-indexed/
# http://code.google.com/web/controlcrawlindex/
docs/robots_meta_tag.html
# Matt Cutt (from Google Webmaster Central) on
this topic:
# http://www.youtube.com/watch?v=KBdEwpRQRD0
# IMPORTANT: serving this header is recommended
only for
# development/stage websites (or for live
websites that don't
# want to be indexed). This will avoid the
website
# being indexed in SERPs (search engines result
pages).
# This is a better approach than using
robots.txt
# to disallow the SE robots crawling your
website,
# because disallowing the robots doesn't exactly
# mean that your website won't get indexed
(read links above).
# <IfModulemod_headers.c>
# Header set X-Robots-Tag "noindex, nofollow,
Search WWH ::




Custom Search