Talend Integration Suite MPx

Talend Integration Suite MPx features the unique FileScale technology which leverages the execution server hardware architecture and maximizes the performance of low-level sort algorithms.

The FileScale technology works in bulk mode on (very) large files. It takes full advantage of the execution architecture as it is not restricted by the JVM or execution engine limitations typical of traditional data integration architectures.

FileScale technology sorts and transforms data using innovative high-performance mathematical algorithms for data processing. It leverages the MapReduce architecture to automatically break down any data processing operation into a number of granular processes.

The challenges involved in processing large amounts of data are similar to those of any large-scale project. Typically, the best approach is to divide the task into as many subtasks as possible and distribute them over all available resources in order to process them in parallel.

Similarly, Talend Integration Suite MPx benefits from multi-server, multi-CPU, and multi-core architectures where code and separate sub-processes can be executed in parallel to make the most of the architecture. This massively parallel feature maximizes ¬enterprise server capabilities and the number of ¬processors available, greatly improving processing time.

Talend Integration Suite MPx also automates the break down of data sets into many parallel streams, for further acceleration of the processing, leveraging the massively parallel loaders of leading RDBMS engines.

In addition, Talend Integration Suite MPx includes support for Hadoop’s distributed file system (HDFS) that provides high throughput access to application data, as well as Hadoop’s data warehouse infrastructure (Hive) that provides data summarization and ad hoc querying.

Based on Talend Integration Suite, Talend Integration Suite MPx includes its core modules - Business Modeler, Job Designer, and Metadata Manager - as well as the teamwork, development consolidation, industrialization and monitoring-oriented features of the leading enterprise data integration platform.