ThemeLab's Blog

Stay up to date with our newest WordPress themes, WordPress plugins, WordPress tutorials, and other announcements.

How To Create a Searchable Tweet Archive with WordPress

Last Updated on by

twitter fail whaleLet’s face it, the Twitter search sucks and you’ll be lucky to find tweets over a week old with it even though they still exist.

Twitter has been plagued with service issues and the fail whale (which someone coded in pure CSS) is a regular occurrence once again.

I came across a tutorial the other day on how to create a browsable, searchable archive of tweets which you can use to host your own tweets. And the best part is? It’s powered by WordPress.

In this article, I’ll go over my experience with how I set up my own @themelab tweet archive powered by WordPress, including:

  • Backing up my tweets with a third-party service
  • Importing those older tweets into WordPress
  • Setting it up so new tweets get imported automatically
  • Installing the theme to display them nicely

Backup Your Tweets

The first step is to backup your tweets. There are a few third-party services to do this, I personally used TweetBackup with good success. It’s important to remember not to expect to backup all of your tweets.

I was only able to grab 910 out of over 6000+ at the time. Even using other Twitter backup services, the same number of tweets were backed up which suggests it’s a limitation by Twitter itself. It’s best to backup as soon as possible, especially if you’re a frequent tweeter.

I had some issues logging into TweetBackup at first but eventually it worked. If you get frustrated you could also try BackupMyTweets.com. It’s worth noting that BackupMyTweets makes you “pay” by posting a tweet promoting their service. If that’s too painful for you (it’s pretty painless, I promise) you also have the option of paying $9.95 per year.

backupmytweets

After the backup process is complete, look for the “export” option and save as RSS. You should have an XML file with all the tweets that were salvageable. The next step is to import those into WordPress.

Import Tweets to WordPress

First of all, it’s suggested you create a new WordPress installation for your tweet archive so you don’t have to worry about separating it from the rest of your normal WordPress content. I personally used a subdomain so it wouldn’t interfere with any permalinks from the blog.

The next step is to browse to Tools→Import and select the “RSS” option.

WordPress RSS Import

On the next screen, upload the XML file you just grabbed from whatever tweet backer-upper service you just used. Your tweets should now be loaded in your WordPress site.

  • Obvious Tip: Assuming this is a new WordPress installation, you’ll probably want to delete the “Hello World!” post.
  • Not So Obvious Tip: Since tweets from the backup file are formatted like “username: tweet message is here” (i.e. “themelab: My actual tweet is displayed here”) you might want to use the Search Regex plugin to delete this. Just replace “username: ” in post content and remember to include the space after.

Automatically Grab New Tweets

Now that you have all your old tweets stored, you’ll want to make sure any new tweets are automatically grabbed and stored as WordPress posts. For this, we’ll be using the Twitter Tools plugin by Alex King.

After installing and verifying your Twitter account details, make sure the following two options are set on the plugin’s settings page (Settings→Twitter Tools):

  • Enable option to create a tweet when you post in your blog? NO
  • Create a blog post from each of your tweets? Yes

Twitter Tools Options

I’m not sure what would happen if you set “Yes” to both, but I don’t intend on finding out for myself for fear of annoying the crap out of my followers. Douglas Bownman, the author of the original guide, described it as an “endless loop of repeating tweets and blog posts.”

All of the other options, like whether to exclude @replies, are either not necessary or up to your personal preferences.

Note: After installing Twitter Tools I noticed a few duplicate tweets (in conjunction with my backed-up tweets) that I went back and manually deleted.

Tweet Archive Theme

Douglas Bownman of stopdesign was also nice enough to release the tweet archive theme he used for free.

Tweet Archive Theme

The theme works out of the box but it requires some *gasp* direct theme editing to personalize the hard-coded area (that I’ve highlighted in red) with your own account information. Just open up your header.php and scroll to the bottom.

tweet-archive-header

Make sure you have the following handy to copy and paste into the “user” div area in the header file:

  • A 48×48 image URL of your Twitter avatar (or whatever you want really)
  • Your name (I used “Theme Lab” for example)
  • Your Twitter username (@themelab)
  • Location and website URL (themelab.com)

Or you could just remove it completely, it’s up to you. There is also a nice display of archive links which don’t seem as dreaded on an archive-oriented site like this.

If you’re making your own custom theme make sure to display the post content and have the full date and time as the permalink to the post. I’m not sure comments are really necessary on a theme like this, but again, it’s your site now, so you can do whatever you want with it.

Update: I just noticed the search form had the /tweets/ URL hardcoded into the form action, which obviously won’t work if you’ve installed anywhere besides a /tweets/ subdirectory. To correct this, replace the following code in searchform.php:

<form method="get" id="searchform" action="/tweets/">

And replace with:

<form method="get" id="searchform" action="<?php bloginfo('url'); ?>/">

Conclusion

I really wish I set this up a long time ago. I’m a Twitter addict and have over 6000 tweets (follow me at @themelab) and unfortunately I wasn’t able to back all of them up, only about 15% of them.

Once again I’d like to thank Douglas Bowman of stopdesign for the guide on settings this up as well as the theme. It was actually a lot easier than I thought and it’s great that it can be powered by my favorite CMS, WordPress, with the help of some extra plugins.

About

Leland Fiegel was the original founder of ThemeLab. He is a web developer who loves WordPress and blogging.

  • http://www.wptheming.com Devin

    I see yet another use for custom post types. Hmmm…

    • Leland

      Yeah, definitely. This is probably just one of many possibilities you can use to integrate tweets into a WP site.

      Just have to get the older tweets imported so it’s not so dependent on Twitter anymore.

  • http://www.alephnaught.com/Blog Bob Hays (@purplehayz)

    You can use Twitter Tools to do a lot of this heavy lifting – it will load a WP database table with your new tweets from the day before, and can create a daily post, weekly post, or pretty much whatever you like. You can then search this using the same tool(s) you’re using in WP or cobble something special since the info is loaded in a custom table.

    Alternatives are good IMHO. Thanks for the nice article! – Bob

    • Leland

      Yeah I’m using Twitter Tools but each tweet is a separate post, not using the “daily digest” feature or anything similar, which might be better to integrate into an original WP blog rather than separating it into it’s own installation.

  • http://www.geniuzdesigns.de g3niuz

    thanks for this – seems to be useful …
    already waiting for your next tut ;)

    • Leland

      Thanks, glad you liked it. I’ll be sure to let you know when that tutorial you’re looking for is published.

  • http://www.palalonde.com Pierre-Armand Lalonde

    You can also use the Twitter Importer from @bradt available here: http://bradt.ca/archives/twitter-importer-for-wordpress/

    • Leland

      Thanks for the link, looks pretty useful and integrates WordPress directly with Twitter without using any third-party services.

  • http://perishablepress.com/ Jeff Starr

    The night before you posted this, I completed my own Tweet Archive. I am working on implementing new features and taking it beyond Bowman’s excellent tutorial. You can see it in its current state here.

    • Leland

      Hey Jeff, thanks for stopping by. I actually follow you (@perishable) on Twitter which is how I got tipped off to Douglas Bowman’s tutorial and how I got inspired to make my own tweet archive.

      Looking forward to see what you can do with it.

  • http://www.wptheming.com Devin

    I was originally thinking this would be useful to backup tweets, but not much else. Now I’m starting to think about possibilities for innovating the design of it.

    For instance, you could do something like this: http://human3rror.com/demos/ipseity/ that puts your tweets in large bold type and has a slider to quickly go through the posts.

    Or, since a lot of tweets have links, you could use something like the Facebook function to pull an image from the linked page and part of the text. There’s actually a lot of interesting ways this could be set up.

  • http://austinpassy.com The Frosty

    Yeah, it’s about time I did this as well.

    Now just trying to decide if I want to format my site with multi-site (WordPress 3.0), custom post-type, or an external fresh install.

    I’d really like to use the custom post-type option, but I don’t think there is an importer for CPT’s yet, nor would you be able to import new tweets with Twitter Tools to CPT’s (would be awesome though, hey Alex, feature request?).