Everybody is talking about Big Data lately. But, as usual, there is a hard and long distance between theory and an efficient practice.  The obvious benefit behind the idea of Big Data hits reality when meets important challenges in the areas of information capture, curation, storage, search, sharing, transfer, visualization and, most importantly, analysis.

Unfortunately, data produced by companies and organizations is worthless if not turned into insights and knowledge. This is why an increasing number of companies are trying to overcome those Big Data challenges asking for help to people beyond their corporate walls.

Not as popular as open innovation tool InnoCentive, Kaggle offers a similar but more focused crowdsourcing platform for predictive modelling and analytics competitions.  Considered the world’s largest community of data scientists, Kaggle aims at making data science a sport by offering a platform where its members can compete with each other to solve complex data science problems from companies worldwide.

This platform was born in 2010 based on the idea that many organizations don’t have access to the advanced machine learning techniques that provide the maximum predictive power from their data. Meanwhile, data scientists and statisticians crave real-world data to develop and refine their techniques. Kaggle corrects this mismatch by offering companies a cost-effective way to harness the ‘cognitive surplus’ of the world’s best data scientists.

Kaggle community of scientists comes from over 100 countries and 200 universities and comprises tens of thousands of PhDs from quantitative fields such as computer science, statistics, econometrics, maths and physics, and industries such as insurance, finance, science, and technology. Kaggle competitions have resulted in many successful projects including furthering the state of the art in HIV research, chess ratings or  traffic forecasting. Even several academic papers have been published on the basis of findings made in Kaggle competitions.

Learn about how companies as Amazon, Mastercard or Merk have used this ‘cognitive surplus’ offered by Kaggle. Some of them, as in the case of Facebook, have even set up a competition with the aim of detecting and recruiting new talent.

Amazon: Predict an employee’s access needs, given his/her job role

Mastercard: Improve the quality of information within transaction data

Merk: Help develop safe and effective medicines by predicting molecular activity

Facebook: Show them your talent, not just your resume

Kaggle