In our last blog, we talked about developing data processing jobs using Apache Beam. This time we are going to talk about one of the most demanded things in modern Big Data world nowadays – processing of Streaming data. The principal difference between Batch ...

This blog post is part 1 of a series of blog posts on Apache Beam. Are you familiar with Apache Beam? If not, don’t be ashamed, as one of the latest projects developed by the Apache Software Foundation and first released in June 2016, Apache Beam is still relatively new in the data processing world. As a matter of fact, it wasn’t until recently when I started to work closely with Apache Beam, that I loved to learn and...

Open source software (OSS) just celebrated its 20th anniversary and not only does the community have a lot of milestones to celebrate, but also a lot to which they can look forward! OSS continues to disrupt the status quo in groundbreaking ways, but it’s also becoming increasingly mainstream. Thus, if you’re an IT leader of any-sized organization, you should be thinking about and planning for how to incorporate OSS into your infrastructure. Among the hundreds of popular open so...

This blog was originally published by Anand Iyer & Jean-Baptiste Onofré [@jbonofre] on the Apache Beam blog...

Recently, Databricks introduced Delta Lake, a new analytics platform that combines the best elements of data lakes and data warehouses in a paradigm it calls a “lakehouse.” Delta Lake expands the breadth and depth of use cases that Databricks customers can enjoy. Dat...

Guest blog post from The Digital Group. T/DG, stands for The Digital Group, has been working on Talend based data source integration for quite some time. The Digital Group recently launched 3RDi (Third Eye) Enterprise Search Discovery and Analytics Platform that utilizes capabilities of Talend for all data integration layers. 3RDi is a comprehensive suite of products to help address your...

My first blog post is on a topic which connects me to Talend in a special way - custom components. The open source nature and concept of custom components are a chance - especially for consultants/developers like me - to open spaces and make Talend a solution which does not end at the limits given by the vendor. Developers can publish these components or extensions on the Talend Exchange platform for o...

  Did you know a commercial airplane generates on average 500 GB of data per hour when flying? Taking into account the number of hours an airplane can fly per day and the flights operated on a daily basis, this creates huge volumes of data that need to be processed. We have good news for a...

A Talend Community Coders post brought to you by: Sergey Beryozkin Who could've thought that Swagger and WADL can be real friends ? Both Swagger and WADL are about describing REST APIs and while the former has a definite momentum, the latter has proved to be very capable and helpful to JAX-RS users. The important thing is that we have users who submit WADL documents  to the runtime or build time code generators which is all working fine. We a...

We recently published a benchmark comparing Talend Big Data Platform to Informatica Big Data Edition, showing the performance benefits of our native Apache Spark approach over Informatica’s solution.  Informatica responded with a rebuttal that...