Watch a Video on Sitemaps for SEO or Read the Full Article Below
A sitemap of your website is essential to great optimization. It is like a roadmap for both search engines and users, providing a complete layout of how your website works. This lets web crawlers navigate your site intelligently and also provides metadata such as dates of recent updates for the engines to use in rankings.
So, how do you create a sitemap, and what type do you need? This depends on several factors, so first I’ll take you through the basics and the ways different sitemaps can work together on a website.
Sitemap for SEO Basics
A sitemap lists all links to each page on your website in a hierarchical format so that the most authoritative pages are at the top of the map. The various links and URLs are actually within the sitemap’s files, placed together. This way the web crawlers for search engines find them easily so they can quickly redirect searchers to the content they want.
Ask, Bing, Google and Yahoo all use the same protocols with sitemaps, so creating the best version for your website is your key to better visibility and rankings from all of them. Basically, they’re like your message to web crawlers: here’s where you need to crawl and what’s there. (Conversely, your robots.txt file tells the crawlers where not to go, which I’ll discuss briefly below.)
Specific kinds of content may benefit more from a certain type of sitemap. For example, video content may call for a video sitemap. Your sitemap can also provide critical metadata to web crawlers such as the date of your last update, licensing information or what kind of subject matter is on a page. Going back to the video example, a video sitemap can provide a topic or category for the video entry along with its length and age appropriateness rating.
Just having a sitemap doesn’t offer any guarantees. You can never be sure that web crawlers will index every item in your website’s sitemap; search engines use complex algorithms to manage crawling. However, almost any website will benefit from having a sitemap, and having a sitemap will never hurt your site.
Automatic Sitemap Generators
For smaller websites without frequent addition of new content automatic sitemaps can be generated. XML-Sitemaps generator is one tool you can use to generate your sitemap. WordPress sites can use a variety of sitemap plugins, including the popular Yoast. Using this kind of tool keeps you from having to enter all of your URLs yourself, and lets you determine how often and when your pages should be updated.
However, using this kind of automatic generated does curtail you in a few ways. The most important is size: five hundred pages is the limit for XML-Sitemaps generator. Second, and perhaps more troubling, is that automatic sitemap generators define all of your URLs with the similar “change frequency”. This is a problem for any site that regularly publishes new content.
SEO Sitemap Techniques
For most websites, it’s far better to create your own sitemap or use a generator that pulls directory from your database. It is true that linking your site pages skillfully increases your website’s chances of being crawled even without a sitemap, but why chance it? A sitemap almost always improves the way crawlers index your site, especially for certain kinds of websites.
Very large sites, especially those with frequent new content, sometimes get parts of that content overlooked without a sitemap. A newer site without many external links can also get overlooked; web crawlers follow links, and fewer links means fewer chances for crawlers to get there. If your website is shown by Google News or is filled with rich media content, a sitemap provides essential metadata that allows crawlers to understand what’s there and provide more information for searchers. Finally, if your site pages aren’t interlinked well or if you have many archive pages, a sitemap can ensure more of them are crawled.
Some websites benefit from having multiple sitemaps. Very large sites with thousands of URLs, for example, will typically not see all pages indexed without more than one sitemap in place. In practice most websites with regular updates benefit from having more than one sitemap; typically an XML sitemap and an HTML sitemap. That’s because they do different things that are crucial to your website’s searchability.
The XML sitemap is really there for web crawlers, while the HTML sitemap is there for users (but search engines do appreciate the internal linking it creates, which is why many SEO professionals recommend linking to your HTML sitemap in your footer). Larger sites with hundreds or thousands of pages may use multiple XML sitemaps as well. No sitemap can be more than 10MB, or hold more than 50,000 URLs. You don’t want to bog down the crawlers; give them sitemaps that are manageable.
The presence of multiple sitemaps means that you’ll also need a Sitemap Index File. Obviously this is a kind of master index or map covering all of your site’s sitemaps. So, does updating your page mean you need a new sitemap each time? No; unless you are making major changes crawlers will find minor updates and changes using your existing sitemaps. However, if you do make major changes to your site and sitemaps, you can resubmit them so crawlers will have the right information.
Types of Sitemaps for SEO and When to Use Them
Whether you are creating sitemaps to direct crawlers through your very large site or your smaller site that is destined for growth, you need to know which kinds of sitemaps will work best for you and your business. The type of content, freshness of content, depth of your pages and site sections determine which types of sitemaps and what kind of structure will prove most effective.
HTML Sitemaps
HTML sitemaps are for your human visitors. They help users navigate your site and understand the pages that are present. HTML sitemaps can be viewed in any browser and crawled by all search engines. Remember that your HTML sitemap should contain at most a few hundred links; organize it carefully so your users get what they need from it. Generally, it should link to your “hub” pages, or the main pages on your site that provide internal linking to all the sections below them. If a search engine were to crawl your HTML sitemap, they should be able to get to every page.
XML Sitemaps
The XML sitemap is the corollary to the HTML sitemap; it is for search engine web crawlers. XML sitemaps tell crawlers what pages are present, where they are, how often they are updated, and their relative importance. This is the main type of sitemap most websites add.
Crawlers use your XML sitemap to parse metadata for searches, and you use them to improve your indexing. Your XML sitemap can boost your optimization by ensuring that all of your site’s pages can be found. Every website needs at least this basic sitemap for SEO.
Text Sitemaps
Text sitemaps are simple, and list one URL per line. They can be crawled by Google and Yahoo, but not necessarily by others. Initially Yahoo crawlers were limited to text sitemaps, but now they support XML, so there is little reason to use this format. If you do have a text list you can always save it as a document and master list of all URLs.
RSS Feed Sitemap
RSS feeds can be used as sitemaps. Google and Yahoo support this use of RSS feeds, but not all search engines do. If your site has a very large RSS feed, it may be best to create a new RSS feed for each section of your site. If you publish fresh content on your site often, and you are not a news site, an RSS feed sitemap is really the best way to go. RSS feed sitemaps provide the quickest indexing capabilities out of all sitemaps. Generally, people will create an RSS feed sitemap for blog posts, articles, product feeds, etc. In some industries, there are websites that scrape other sites. In this case, an RSS feed sitemap would be ideal, as it allows the content to get indexed on the publisher’s site before any scrapers. Quick side note… I have spent years experimenting with RSS feeds and their impact on SEO. They can used to get quick indexing, build links, do social updates, etc. RSS is a very powerful too.
ROR Sitemaps
ROR stands for “Resources of a Resource.” It is actually a kind of XML format intended to describe any kind of web content in a generic way. The idea is that this allows anything—crawlers, apps, search engines—to understand the sitemap. ROR format includes various defined terms that are used to describe kinds of content and you can also use custom terms. The idea behind ROR format is to standardize the way content is classified by engines and provide better metadata. However, major search engines don’t support all of the ROR sitemap extensions, so this is not the best choice for most sites.
News Sitemap
News sitemaps are for sites that have many frequent updates and syndication feeds. A news sitemap should be limited to URLs from the most recent two days; they will stay in the Google News index for one month regardless. You also have to update this kind of sitemap as new content is published, although you should not create a new one (just add the new URLs). A news sitemap should be limited to 1,000 URLs or fewer. This is one kind of feed that is much easier to create automatically.
Image Sitemaps
You can create a separate sitemap for images or you can add information in the form of image extensions to your existing sitemap. Image sitemap information helps search engines who otherwise would not index your images in Google image search. I am a big fan of image sitemaps. Most people forget to implement them. They are incredibly easy to create and result in excellent traffic. I have seen e-commerce sites and news sites who get 30% of their search traffic from image sitemaps. In addition, having a good optimized image on the page really helps the HTML page rank as well, so it all works together.
Video Sitemaps
Much like image sitemaps, video sitemaps give search engines metadata about video content. A video sitemap (or video information within your existing sitemap) can provide Google with titles, runtimes, descriptions, categories, and intended audiences. This also allows you to show thumbnails of video content in searches. Also, letting Google know about rich video content can boost your rank. If you are running WordPress, there are some video sitemap plugins that are really easy to add.
Mobile Sitemap
There are three ways to do mobile website optimization. Vary HTTP header, responsive design and a separate website or subdomain. Here are a few posts I have written on the topic.
- Mobile SEO Checklist
- Everything Google Looks at For Mobile SEO
- Mobile SEO Basics, What you Need to Know
- Ultimate Guide to Responsive Design and SEO
If you are going to be going the subdomain/separate website route, websites will often create a mobile sitemap. Now, there are some SEO considerations with this one. Mainly, you need to make sure that Google understands the relationship between your desktop and mobile site. So if you are going to be submitting a mobile only sitemap, you need to make sure you have rel canonical in the HTML on the page of the mobile site and rel alternate on the desktop.
Conclusion on Sitemaps for SEO
There is almost never a good reason to skip a sitemap, and sometimes more than one kind is best for your website and business. If you don’t yet have this essential step towards better optimization in place, don’t wait any longer to create your sitemap. The right sitemap gives search engines much better data about your site, and boosts your rankings. So there you go! Get to adding these sitemaps to your site now.
