Twitter Lists and Real Time Search deals – The Big Picture

I’ve spent the past few hours searching around the tech blogs, SEO blogs, forums and Twitter but haven’t been able to find anyone who has made the connection between the recent Twitter Lists feature and the Twitter search deals between Bing and Google. Surprising, really, so I’ll take a stab at it.

On September 30, 2009, Nick Kallen wrote an official Twitter blog that they had just rolled out a new lists feature to selected users. Rather than rewrite a summary of that blog here, I’ll quote the main points below, followed by my thoughts. Nick wrote:

“The idea is to allow people to curate lists of Twitter accounts. For example, you could create a list of the funniest Twitter accounts of all time, athletes, local businesses, friends, or any compilation that makes sense.”

Essentially, Twitter users will be able to categorize the people they follow. How does this have anything to do with the later announcements of search deals between Microsoft’s Bing search engine and the Google search deal?

Categorizing accounts adds a new layer of meta data to the enormous amount of random bits of information spewing out of Twitter at any given point in time. That meta data makes it much easier for the search engine software to quickly crunch the data for search queries. For instance, someone goes to Google or Bing and searches for “Lakers game tonight.” Rather than having to crunch all the data coming out of Twitter in the past 24 hours, then find and output relevant tweets, the software first looks for tweets by members who are on lists categorized as Sports, Basketball, Lakers, etc. and give those tweets priority.

Nick then goes on to say:

“This means lists have the potential to be an important new discovery mechanism for great tweets and accounts.”

See the paragraph I wrote above…

“We started working on this feature because of the frequent requests we received from people who were looking for a better way to organize information on Twitter. Of course, that means not just twitter.com – the Platform team will follow up in a few days with information on the Lists API. This will allow developers to add support for Lists into your favorite Twitter apps.”

And the new lists API will no doubt be used by Bing and Google as well.

There have been rumors swirling around for nearly a year now about real-time search. Pundits and tech bloggers have been wondering which of the big search engines would be the first to do “real” real-time search and do it right. The biggest problem that’s been standing in the way of real-time search, in my honest opinion, is the sheer amount of random data being produced by social media and the real-time web. Or rather, no plausable way to make that data easily discoverable on the receiving end where the search engines are.

Other big user generated content sites have turned to allowing their users to categorize and tag things in the past. Look at how Flickr encourages it’s users to tag, categorize and comment on photos and video. From the beginning, Myspace and WordPress.com have benefited tremendously by making it natural for users to categorize, comment on and tag stuff. And long before Facebook started tearing down it’s walls, much of it’s tremendous growth has been due to making it easy for users to grow content and networks by making their data easily discoverable. Of late, Google itself has been encouraging searchers to interact with it’s content by voting up and down, removing items from their search pages and add tags or descriptions to photos.

All of what I mentioned above adds meta data that can be used to deliver more relevant results to searchers, faster.

Twitter is a business with a massive amount of user generated data being produced 24 hours around the clock. Those speculating about Twitter’s business model can finally put the subject to rest. They’ve made their product more usable and at least 2 huge companies have jumped committed to using that product.

Meta data. Twitter lists are just the beginning.