Features of a 404 error in SEO

Today I want to talk about the pages with error 404 , which are in the Google index and why it’s not very good to have such pages in the index.

And in the end, I want to consider software that will help you identify and fix problems.

What is wrong with this issue?

Let's start with a few queries to Google. We are looking for pages with error 404 .

Queries: allintitle: this page cannot be found and allintitle: this page cannot be found 404 led us to what we were looking for:

Why such pages are displayed in the issue of Google?

Ignore the message that the error on the page and a warning about a 404 error. The content of the page may show that the page has errors, but in fact, such pages are indexed by search engines as normal.

And the code 200, which is contained in the response header of the server, which is generated by the host server, is to blame.

Code 200 (status "ok" and "request completed") is deceiving the search engine , forcing the page to be included in the index.

Problem and methods of its detection

Let's take a look at a few examples and use several tools to see what happens.

Kew Garden did a great job of studying the error and their standard error page is not the same as you would imagine it, the server can also generate a 404 error .

First of all, if you have not already done so, install the Live http headers for Firefox.

Live http headers is my "required" SEO tool for firefox, since it allows you to see the contents of the server response header. There are many similar Firefox add-ons that show the content of the response, but I prefer this one.

Let's return to our example. So, using Live http headers, check the error page on the Q site.

Checking the server header you will find the code 200 . As you can see, the standard error page is perfectly visible, but there is one “but”: the correct response is not 404.

Interestingly, a randomly generated URL gives the correct 404 code :

Such pages that are in the index are not ideal in terms of SEO, especially for sites with a large number of pages, and it may be that the number of blank pages will be indexed more than pages with content.

If you want to have a clean site and quick indexing of fresh content, you need to follow this example.

Checklist for 404 Error Check

Here are some tips on how to check your website for this problem:

1. If you have a standard 404 page, check the server for 404 status.
2. Generate the URL of the page that does not exist and check the status of 404.
3. Does your CMS redirect if the URL is not recognized? Personally, I think the redirect to 404 is not normal - try to avoid it.
4. Use the Google Webmaster Tools or Xenu's Link Sleuth to control the bugs on your site.

if you think that the wrong server response to a non-existing page doesn’t turn out bad for your site, I’ll hurry to disappoint. Unfortunately, such a "Makar" can greatly harm the site, I will not describe the technology, but I think that they themselves will guess in what way. This applies to both Google and Yandex.

You can also install the IIS SEO Toolkit to check for errors and much more that may affect the SEO site.