Data, Big and Easy

May 10, 2013 --

No, this is a not a post about The Big Easy, the famed city on the Mississippi River.  But about how Big and Easy don’t have to be antagonistic when it comes to describing data…

There is a general perception out there that big data isn’t easy. It brings many benefits, some immediately tangible and some that will become clearer in the longer run. But in order to leverage big data, organizations need expertise that few engineers or business users possess. Gartner puts it quite eloquently:

“Over the next 24 months to 36 months a main challenge that organizations will face will be recruiting, onboarding, retaining and developing people with advanced information management/analytics skills. These people include data scientists, information leaders, data stewards, chief data officers and information architects.”

I have already discussed how big data technologies – and primarily Hadoop – have matured to a point where they are now viewed as enterprise-ready. The continuous evolution of the platform and its ecosystem is making it even more powerful every month – see for example the Stinger project, or Impala, both key advances in speed and interactivity of Hadoop/Hive.

This places organizations in somewhat of a bind.  They certainly have the (big) data.  They can now get a powerful, reasonably priced and enterprise-ready technology platform to exploit this data. But they cannot get the manpower to run it!

What is the solution? Make it easy to run the platform. Instead of acquiring expertise in MapReduce and parallel programming, instead of learning several new scripting languages, leverage tools and techniques that are already widely known and accepted.

This is what Talend is delivering with version 5.3 of its next-generation integration platform. With 100% MapReduce code generation, Talend generates native Hadoop code, and runs data transformations directly inside Hadoop for maximum scalability.  The idea is to get the usability, without sacrificing the performance.  And all within Talend’s award winning platform. (Did you check the latest award by the way: CRN’s Big Data 100?)