HTML and CSS Reference
Making your site more visible on search
While the content of your website matters the most, making sure everything else sup-
ports better visibility of the content on search engines is important too. The following
sections explain some ways you can do this.
Directing search spiders to your site map
Site maps inform search engines of the existence of pages within your site that are
otherwise not discoverable; perhaps they are not linked to from other pages on your
site, or from external sites.
Some CMSs provide plugins to generate site maps, listed at code.google.com/p/
sitemap-generators/wiki/SitemapGenerators , or you can write one your-
Once you have written your site map, you can let search engine spiders discover it
when they crawl your website if you add a link to the sitemap by using the following:
You can also submit the site map to individual search engines instead of linking to the
site map within the HTML page, if you would like to make your page as small as pos-
Implementing X-Robots-Tag headers
You will likely sometimes have a staging server, such as staging.example.com for
your site example.com . If an external site links to the files on the staging server (say
you were asking a question about some feature not working on a forum and link to the
staging server), it is likely to be indexed by search engines even though the domain
name does not figure in the robots.txt file or does not hold a robots.txt file.