Yesterday we announced Talend Integration Suite MPx, a highly scalable, massively parallel data integration platform that scales to the highest volumes of data. Talend Integration Suite MPx is designed to meet the highest and most demanding data integration needs.

Developed in cooperation with a team of researchers who specialize in advanced algorithmics, Talend Integration Suite MPx features some unique features to optimize high throughput data integration:

• The FileScale technology, based on the MapReduce architecture originally developed by Google for highly distributed processing of data in a data set mode. The FileScale components in Talend Integration Suite MPx allows integration processes to sort, filter and merge data, perform aggregation and arithmetic functions and transform and ensure the compliance of data. FileScale is leverages statistical algorithms that auto-configure the components to use the best algorithms for each data stream.
• Talend Integration Suite MPx also features multiple levels of massive parallelization, allowing the execution of separate sub-processes in parallel, break down of data sets into many parallel streams and the ability to leverage parallel database loaders.

Validated with some customers who have extreme scalability needs, Talend Integration Suite MPx is already getting lots of great feedback.  And of course it sill leverages the same environment as Talend Open Studio and Talend Integration Suite, and that means that all jobs are upward compatible.

For more information, please check the press release.

Fabrice