Data collection and manipulation for the masses

Yahoo pipesI am increasingly impressed with the data collection and manipulation tools that are becoming available on the web for non-programmers. I first noticed Yahoo Pipes about a year ago and just noticed Dapper recently.

Yahoo Pipes allows normal people* to mash up data into combined feeds to create interesting results. Dapper provides the tools to grab virtually any data (regardless if RSS feeds are available or not) and create a feed.Dapper

I see unbelievable potential in tool like these becoming popular in the future. They are not really simple enough to bridge the gap into the mainstream population…yet. But the potential is there.

Eventually they will become more widely-used as their user interfaces evolve and more sophisticated under-the-hood as their data manipulation software adapts to virtually any scenario.

Will your grandmother** use it to combine weather reports with he price of cantaloupes at the local market to find regional produce bargains? Probably not.

Will your mother and father use it to find interesting travel deals by combining weather, event calendar, travel and exchange-rate data for an economical and interesting trip to Europe? Maybe…maybe not.

Could you use it to search all the local bargain websites (Craigslist, local newspapers, eBay, etc.) to find the cheapest DVD players within a 10 mile radius. Yep, pretty easily.

If I were a business owner I would create search-agents like this to find nearby products I use in my widget business. I would scrape local and national websites for product data integral to my business. I would also use it to find local widget buyers in the exact same way. It seems like a perfect match for small business owners, but probably not easy enough for the general public to begin using right away. For the most part, the public doesn’t use words like “data” unless they are referring to the STNG android (and even then it’s only nerds).

Within a few years we’ll see many more of these sites go mainstream as users become more and more technically savvy and eager to manage and filter the massive amount of data surrounding them everyday and find exactly what they are looking for.

Could you use tools like this in your everyday digital life or is it more hassle than it’s worth?

* Normal people do not know (or care) about the following acronyms: SQL, FTP, PHP, HTML, RSS, CSS, ETC.
** Your grandmother is probably more concerned with feeding you, than reading your feeds.

This entry was posted in Development, Software, Usability. Bookmark the permalink.

Comments are closed.