Progressive Development & Hosting

The Problem

For Search Engine Optimization (SEO) purposes, it’s important that search engines have a consistent view of your site. This is known as URL Canonicalization. There are many ways in which to get to the same page on most sites. For example, these URLs all point to the same content:

• http://www.example.com/
• http://www.example.com/index.html
• http://example.com/
• http://example.com/index.html

You want search engines to have only one reference for the content contained in these four different URLs, which are really all the same page.

The Solution

Consolidating these down to one URL can be achieved by the use of a 301 redirect, which indicates that a page has been permanently moved to the location indicated in the redirect. The “301″ is an HTTP Response Status Code that indicates the location of a page has permanently moved. Search engines pay attention to this code, and theoretically update their indexes accordingly.

If your site runs using the Apache Web Server, 301 redirects can be implemented with the use of the mod_rewrite Module, a URL rewriting engine that is available in most installations of the Apache web server.

A complete discussion on mod_rewrite is beyond the scope of this post, but there is no shortage of information to be found on the topic.

So how do you implement this? Well, that is the main point here. Assuming your site is running on Apache, you can add the rewrite rules to a .htaccess file in the document root (the main directory) of your site. A .htaccess file can be used to set web server configuration settings that pertain to just your site. I have seen many examples of mod_rewrite rules meant to fix canonical URL issues, and most fall short somewhere. Here is a set of rules that has worked many times in the past for me.

To set up the example, we’ll assume your domain is example.com, and that you want search engines to see your site as http://www.example.com/.

The Rewrite Rules

Lines beginning with # are comments. They are ignored by Apache, but are helpful in documenting what is going on in the rewrite rules. Replace all instances of example.com with your domain name.

With the disclaimer that we cannot be held responsible for any problems making these changes might introduce, you can add these to an existing .htaccess file or create a .htaccess file in the base directory or your site if one does not already exist. It is important that this file be plain text. If this doesn’t make sense to you, contact us and maybe we can help you out.

RewriteEngine on
RewriteBase /

# add www to domain if missing
RewriteCond %{HTTP_HOST} ^example\.com [NC]
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]

# remove index.html for any request
RewriteCond %{THE_REQUEST} ^(.*?\/)index.html
RewriteRule ^(.*?)index\.html$ http://www.example.com/$1 [R=301,L]

The Result

The effect of adding these rewrite rules will be the following redirections:

• http://example.com/ –> http://www.example.com/
• http://www.example.com/index.html –> http://www.example.com/
• http://www.example.com/other.html –> no change
• http://example.com/other.html –> http://www.example.com/other.html
• http://example.com/index.html –> http://www.example.com/

This will also work properly with subdirectories:

• http://example.com/test/ –> http://www.example.com/test/
• http://www.example.com/test/index.html –> http://www.example.com/test/
• http://www.example.com/test/other.html –> no change
• http://example.com/test/other.html –> http://www.example.com/test/other.html
• http://example.com/test/index.html –> http://www.example.com/test/

Again, if you think this is a problem you need to solve with your site and would like some help in solving it, please contact us. For more information regarding SEO or for an analysis of your site, we recommend Media Wyse.

One Response to “Rewrite Rules for SEO URL Canonicalization”