Talend announced last week Talend Integration Suite 2.4, the first ever open source data integration product that includes built-in Change Data Capture.
As you may know, Change Data Capture streamlines real-time data integration for enterprises because it detects changed records in real time to allow the updated data to be immediately sent to other applications. It also cuts the time needed to load and update data during the ETL process.
Ultimately, this feature helps companies cut costs by loading only the changed data, thereby taking up a fraction of the resources that bulk ETL processes typically need.
For organizations with massive databases (for examples with several million/billion records), there are of course many challenges. One of them is that the full update of these data sets in the data warehouse takes a lot of time and uses a lot of computing resources – even with the highly optimized runtime architecture of Talend. By speeding up the refresh time of the data warehouse, CDC allows this refresh to happen more often and in some circumstances to tend toward real-time data warehousing.
In addition, whenever an updated record is detected, it is possible to trigger a data integration process. For example, the recording of a new order triggers the sending of the data to target applications (manufacturing, shipping, invoicing…). Or the update of a customer’s phone number in the CRM is automatically propagated to the Help Desk application.
Finally, a Publish/Subscribe option allows the definition of different data consumers (target applications or databases), operating with different latencies. Some consumers will get the update right away and some others, operating for example in batch mode, will get it every hour, once a day, or whenever they need it.
With the new CDC feature, options are unlimited and data integration knows no limit. And at the end of the day, enterprises win by being more agile and more reactive.
I’ll conclude noting that all the new capabilities integrated in this new version our solutions resulted from numerous interactions with our community. As Fabrice mentioned in the Press Release: “Talend relies heavily on the quality feedback from our user community for the development of features and components that are in high demand. This latest version of Talend Integration Suite incorporates many of the feature requests and suggestions that have come from our users.”
Stay tuned for more!
Yves
Out of 400 applicants, only 30 were selected. I am talking about up and coming IT companies, startups for the most, who were nominated to participate to the 2008 Morgan Stanley CTO Summit (you need to be nominated, not everyone can send their file). Hosted by Jonathan Saxe, Morgan Stanley’s Global Chief Technical Officer and Managing Director, this 2 day conference and think tank took place in San Francisco this week.
Latest Comments
Yves, Doug Laney
Joseph A. Jacks
Ted Friedman, Yves, Ted Friedman
Hideki Masagane, S. MALLERET, Peter Fisher
Killian @ Open Plus