Google Depreciates Ajax SEO Crawling Scheme, How to Deal with It
Google sent massive shock waves through the SEO community the other day, announcing that it was discontinuing AJAX SEO guidelines!
Well, not really. We knew it was coming for months…
For those of you who don’t know, AJAX is a client-side script that relays information between a server and database without the need for a page refresh, so you don’t need to reload the page. It is a great lightweight technology.
Back in 2009, Google gave some technical fixes to make AJAX pages crawlable, but their recent announcement signals that they are moving away from this proposal entirely. In the past, if you wanted to make a page crawlable, you had to do something called the #! escape fragment optimization. It was a pretty goofy workaround. But basically, Google would crawl your URL and if it had a #! in it, they would know to look at another URL. That second URL would have the optimization on it for indexing. You can learn more about the old optimization here.
Phasing out this optimization strategy is big news because it reveals what many of us have been thinking for months; that Google wants you to use progressive enhancement web design principles to build sites instead of this old method. Personally, I have been pushing clients to move away from this technical configuration for a long time. It never ranked well anyway.
Progressive enhancement is a more user-friendly form of web design that Google has been pushing webmasters to embrace.
First, Let’s Hear From Google
Here, we see questions and answers directly from the Google blog.
This post is meant for those who had implemented the escape fragment optimization. This is how you should deal with it.
301 Redirect All Escape Fragment URLs to New, Correct URLS
Now, I know Google doesn’t say directly to do this, but I think that is a major mistake…
The best practice going forward is to set up 301 redirects for all your escape fragment URLs and all your #! URLs to the new optimized static URL. That way, you’ll be able to salvage as much of your traffic as possible.
If anyone happens to come to your old URLs, a 301 redirect will send them to the correct URL and Google will see the permanent redirect.
Google posted about the escape fragment URLs in one of their FAQs:
Q: Is moving away from the AJAX crawling proposal to industry best practices considered a site move? Do I need to implement redirects?
A: If your current setup is working fine, you should not have to immediately change anything. If you’re building a new site or restructuring an already existing site, simply avoid introducing _escaped_fragment_ urls.
As we can see, Google is not getting rid of its indexing with escape fragment immediately. They also don’t specifically say that you need to put in place redirects. However, these type of URLs never ranked well anyway. If it were my website, I would create static URLs and do 301 redirects ASAP. That way, you are leaving nothing on the table.
Make Sure you Migrate all Same Optimized Content
Just as it’s extremely important to use 301 redirects in your site migration, it’s important to migrate all the same optimized content you were using with the AJAX version of the site.
If you don’t migrate all the same optimized content to your new site, you could run into problems with your SEO rankings.
Read these two posts to learn about basic website migration.
Make Sure You Have a Basic Optimization Strategy in HTML on the Page
You can still use AJAX on your website and the good news is that Google can now execute it. We can see this clearly when we do a Fetch as Googlebot in pages that are cached. But, you cannot only have AJAX on a page or it will not rank well at all. You need all of the same basic HTML principals that make a normal page rank. Think about it, how can Google rank an AJAX overlay with no keywords? The only way is by looking at external hyperlinks, internal hyperlinks and general website theme. If you really want to get the most out of your AJAX pages, you should have an underlining based of HTML with the following.
- Static URL
- Title
- Description
- h1
- h2
- Copy
- Optimized image
- Etc.
Follow Progressive Enhancement Principles
What Google has done is pretty much explicitly said that you need to be using the principles of progressive enhancement when you’re building sites.
Here is information on Progressive Enhancement from Wikipedia. Most importantly, pay attention to the core principals.
These principles make a lot of sense to those of us in the SEO world, because they’re in-line with what Google has preached constantly with all of their algorithm updates.
For example, back in April, Google went so far as to announce “Mobilegeddon” and that is listed as the first tenet of the principles of progressive enhancement.
Summarizing Recommendations
The escape fragment #! optimization will no longer exist. So at this point, all websites that were heavily relying on this as their optimization strategy need to do the following:
- Create static URLs
- Make sure the new URLs have a baseline of HTML optimization
- Redirect all the old URLs (#! and escape fragment) to the correct new URLs
- Submit the old URLs to Google so that Google can see the redirects
- Continue to use AJAX as a UX enhancement and let Google crawl it
If you can do this with your website, you should be in good shape moving forward.
