ThemeLab's Blog

Stay up to date with our newest WordPress themes, WordPress plugins, WordPress tutorials, and other announcements.

Are You Killing Your Own WordPress SEO?

Last Updated on by

On a WordPress site of mine, I was perplexed as to why it was not getting indexed in Google or other search engines.

I verified my site in Google Webmaster Tools and tried to submit a sitemap. In the webmaster tools panel, it informed me that my robots.txt file was blocking Google from grabbing my sitemap.

This confused me, as I know I would never knowingly make a robots.txt file to block search engines.

Read on to find out what the problem was, how to fix it, and how it happened in the first place…

The Problem

I checked out my robots.txt file and saw the following:

User-agent: *
Disallow: /

As if disallowing all bots wasn’t enough, I saw all the pages of my site had the following meta tag in it.

<meta name='robots' content='noindex,nofollow' />

Okay, so now my SEO was killed on two different levels – search engines can’t index or follow links, if they could somehow first get past the robots.txt.

The Fix

How did I remedy this crucial mistake? It’s a simple fix.

WordPress SEO Visible

I navigated to my privacy settings in the WordPress administration panel and selected I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archivers.

Prevention

When installing WordPress, make sure you don’t deselect the checkbox that reads: Allow my blog to appear in search engines like Google and Technorati.

Conclusion

You should always verify your site(s) in Google Webmaster Tools to see if there are any problems with indexing. I hope you found this quick WordPress tip informative, because although this is a simple mistake, it’s something to be aware of when SEO’ing WordPress sites.

About

Leland Fiegel was the original founder of ThemeLab. He is a web developer who loves WordPress and blogging.

  • Richard Paul

    Good reminder for those that don’t know about this and those that might have turned it off on purpose but forgot to turn it back on. I do like that WordPress thought to include a method to allow and disallow for folks that want to keep their blog more personal. Still – good reminder

  • http://www.doeswhat.com DoesWhat

    That’d do it.

  • http://neuracom.co.uk/ourstaff-seo.htm SEO Professionals

    Yes, I also agree for a good reminder. That can be a helpful thing for all new bloggers.

  • http://pangeran.org/ Pangeran

    Hahaha…
    Sometimes even a simple thing like that will ruined everything. I did the same mistake last time.

    For better coverage, beside submitting your website to Google Webmaster Tool, we MUST also submit our site to Site Explorer(Yahoo) and Webmaster Central(Live Search – MSN).

    And other major search engine to consider is ask.com, but I’m not sure if it have webmaster tools feature…

  • http://hotwordpresstips.com mayooresan

    Cool tip!
    Some people who make private blogs can keep their blogs private by enabling it, but for bloggers like you and me, we need more hits from Search Engines!!!

    WordPress let you do anything!!! Thats what I love with WordPress

    Thanks for the tip.

  • http://www.themelab.com/2008/06/22/are-you-killing-your-own-wordpress-seo/ webpramoter

    Yes ,i also think that such type of reminder will keep us away from this type of problem because this small mistakes create big problems.

  • http://www.myqute.com kelly

    Funny…I had written also written something about robots.txt sometime back – check it out if you want: http://www.myqute.com/2008/06/18/banning-search-engines-construction/ :)

    • http://www.myqute.com kelly

      Sorry – link down as I find a way to fix wp!

  • Leland

    Yes, I realize some people may prefer to block search engines for various reasons. This reminder was for people like me who may have accidentally blocked them, and then forgot about it.

    @Pangeran: Yep, you really can’t forget about other big search engines like Yahoo and Live. I used Google Webmaster Tools as an example since they seem to have a good “reporting” feature if there’s something wrong with your site.

    @kelly: Nice article, but there seems to be a problem with your site. There are a few PHP warning errors at the top.

    Thanks for the comments everyone.

  • http://www.myqute.com kelly

    @Leland, thanks again! Yeah, saw the errors (been there for almost 8days!) and I have not done anything yet to correct the problem. >,< GUILTY! lol

  • http://www.seo-pro-singapore.com seo singapore

    Great point. Sometime pple get too caught up with the whole post frenzy they neglect important settings in WP.

  • http://tymayn.com Tyler

    Hahah, I actually did this with my first wordpress blog myself and found out the exact same way you did, with google tools.

    Thanks for reminding me how dumb I once was. :P

  • http://complimedia.com Montana Flynn

    Good reminder for those who may not know. Personally the only WordPress install I have done that I chose that option for is my local dev blog.

  • http://complimedia.com Montana Flynn

    Good reminder for those who may not know. Personally the only WordPress install I have chose that option for is my local dev blog.

  • http://www.richardker.com Richard Ker

    No spiders visited my site and found this setting and everything went well after that!

  • http://galengidman.com/ Galen

    Just think where your site would be now if you had known this since the beginning. ;)