Blogging and Duplicate Content - Rank Better
For those who utilize Wordpress as their blogging platform, you may be ranking lower than your supposed to be because you are being penalized for duplicate content. This puts your article pages into the supplement index, and nobody ever sees them.
This doesn’t mean you are stealing information from other sites and posting it on yours. It means you are posting content in several different places at once! How so? Lets see on which pages your post ends up on.
- Root/Index/Homepage - Whatever you want to call it, its on the front page of your blog
- Post page - The article has its own url
- Category(ies) - You article appears on the category page(s) for each category it was published to
- Archives - The post appears in your archives
- Tags - I trust everyone is using UTW. Your post will appear on the multiple tags it was submitted to.
So your content appears on 5-15 pages - not good. Here is the code I use to only index the articles, pages, and homepage.
Now lets see what this code does:
If the content shown is a Post or a Page (e.g. About me), the spiders should index it.
If this isn’t a Post or Page, check if it is the homepage. Also, make sure that this is less than page 2 (so basically, only your homepage). If this is true, spiders should index the page.
Otherwise, spiders don’t index the page.
Through that code into the Header template inside the < head> tags and you will be avoiding all of the duplicate content penalties.
If you enjoyed this post, be sure to subscribe for updates or at least bookmark this below.
Filed under: Code Snippets
Neat, but I won’t go that far. If you get external links to the posts, duplication on main page(s) and category pages is not a big issue. Actually, that generates nice long tail traffic. The monthly archives shouldn’t be linked at all and blocked in robots.txt. My longish take on WP dupe issues.
other than the fact that there may be less code here, what’s the difference / benefit of using this code as opposed to adding the directories to robots.txt like this:
User-agent: *
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: /rss/
Disallow: /comments/feed/
Disallow: /page/
Disallow: /date/
Disallow: /comments/
Disallow: /cgi-bin/
Disallow: /2006/
Disallow: /2007/
Disallow: /*?*
Disallow: /page/
Disallow: /iframes/
Disallow: /recommends/
Disallow: /category/
Disallow: /*/feed/
I believe that when you disallow a folder in the robots.txt, this completely prevents the spiders from going to that certain directory, which prevents them from both indexing the page and following the links on the page. The meta tags above, allow for spiders to visit the page and follow all of the links, but not index the page.
Folks, Google is smart enough to ignore duplicate content in the SAME website (they stated this themselves), so why bother with all this?
Hi Dean,
By ignore you mean what? They list up both your pages in the SERPS or they choose only one? If they list both, then everyone should double up their pages. If they choose one, and choose the wrong one, we have a problem to fix.
Any link to show they said something?
Hi,
Its a great post.I found it is very interesting and more informative regarding Duplicate content.you have bring all good points.i hav created a website which page rank was good before 2 months…suddenly it had a drastic changes in page rank ..i was confused…after reading this article i gotta answers for all my questions…
Nice articles you got here, very useful for every one who need some tips or references. I should put some of your articles on my website. Maybe we can share our articles someday.