Prediction API & Big Data API

Google has branched out in many directions since their initial search-engine & adwords success. The company has had such healthy profits (mainly from their internet based advertising) that they were able to dab in pretty much every current interesting application in Computer Science - see here!

For a while they are providing cloud computing services, such as Google Storage for Developers, check out the pricing of that service here. Google maintains all the data within their own infrastructure. I think this article tries to explains how the distributed storage is implemented (of course just a very generic overview). You will notice that the service is naturally scalable and pretty smart in a number of ways.

The most recent activity of google resulted in the announcement of two new APIs (Prediction API and BigData API). The diagram below shows how these fit together.



BigData is used to query a large cloud store (using an SQL dialect over a webservice) and the Prediction API can be used on the data to train google implemented AI models for prediction. This simply seems to be a machine learning library that can be accessed over a webservice. Obviously this runs on google cloud infrastructure and that has it's advantages.

A number of Machine Learning libraries exist, such as WEKA, RapidMiner and many other. I used to write some of my own code for these algorithms, however over the last few years I noticed an amazing increase in the count of ML libraries. In most of my work these days I use open source libraries.

I am not quite sure how the pricing of these APIs works (maybe somebody can enlighten us on this issue), my impression is it is connected to the Google cloud store service, for which these APIs will present another reason to use this store.

You can check out some code samples for the API here.

Large Search-Querry datasets

On a blog I recently visited, it was pointed out in context of a web search analysis study that: "This study also highlights the current situation in web-scale research: that only companies like Microsoft, Google or Yahoo! have access to the sheer volume of data needed to do such an analaysis." (sorry I do not remember the source of this, if it is ur blog please let me know, so I can reference you) This has been brought up numerous times in the past and I couldnt agree more. What is really very important is for society (/ internet users) to realise that they are the creators of the content and they should demand access to the agregatted datasets [Tapscott&Williams in Wikinomics - How Mass Collaboration Changes Everything]. Or in other words (since most people wouldnt know what to do with the datasets) some kind of compensation. This applies less to search engines, since they provide a free service, and applies more to other collaborative web 2.0 apps. However, search engines make money on advertising, so thats the reason why they are free to use, not because users are being compensated for the datasets that these search companies decide to store and aggregate.

My criticism is that search engines shouldn't keep the raw data hidden behind proprietary domains but open up to the world research community.

News never settles down...

When rivals battle it out, news never tends to settle into the stock prices that much. In my post from Wednesday, July 8, 2009, I mentioned that Google is planning to develop an operating system and this might be strongly perceived as a long term risk to Micrsoft (as this is Microsofts core business). Since then, Microsoft has announced that it will offer Office application online and for free from 2010, further it seems that a yahoo takeover by Microsoft is agressively in the making.

J. O'Neil, the well known author and investor states that in his investigations he has found that usually news, as it comes out, creates an effect on the prices over an arbitrary period, however within some time price always tends to return to the a-priori (before) news price. Googles plans are hence maybe not so real towards microsoft shares as some people might want to think.

With Microsoft I would currently expect a drop in share prices over next few weeks. Factors: yahoo take over, Google's plans, smaller revenues if Office is to be offered for free. However over the longer term (months and next few years) I can see Microsoft on a steady footing, as Google's threat is really just more psychological than real, and Microsoft has other profitable branches that show a lot of promise, unless hindered by monopoly regulation.

Tech Companies...

According to news released yesterday, GOOGLE is planning to built an open source operating system under the name GOOGLE Chrome OS (following its successful free Chrome browser). The OS will be based on a UNIX kernel, it will however be build from the ground up. Some speculate this could have a drastic negative effect on Microsoft shares over future years.

GOOGLE certainly represents a very capable and potentially extremely dangerous competitor, especially if it taps into Microsofts' core products' business revenues.