Introduction Thanks for continuing to read all of our streaming data use cases during my exploration of Talend Data Streams. For the last article of this series, I wanted to walk you through a complete IoT integration scenario using a low consumption device and leveraging only cloud services. In my previous posts, I’ve used a Raspberry Pi and some sensors as my main devices. This sing...

In our last blog, we talked about developing data processing jobs using Apache Beam. This time we are going to talk about one of the most demanded things in modern Big Data world nowadays – processing of Streaming data. The principal difference between Batch ...

In the last few years, Python has become the go-to programming language for Data Scientists. In a way, this is kind of surprising. Python was not originally developed to do analytical tasks or data science, but it has evolved to become the ‘Swiss-army’ tool in the Data Scientist toolbox. The reason for this comes from a large number of third-party packages available for Data S...

Introduction I was incredibly excited when Talend finally released a new product called Talend Data Streams that you can access for on free on the AWS marketplace. From day one, I have been testing out a few use cases. <<...

The concept of cloud computing has been around for years, but cloud services truly became democratized with the advent of virtual machines and the launch of Amazon Elastic Compute in 2006. Following Amazon, Google launched Google App Engine in 2008, and then Microsoft launched Azure in 2010. At first, cloud computing offerings were not all that different from each other. But as with nearly every other market, segmentation quickly followed growth. In recent years, the c...

I am very excited to introduce Talend Data Streams, a brand new, cloud-native application that enables you to get streaming data integrations up and running in minutes all while providing unparalleled portability powered by Apache Beam. <<Download Talend Data Streams for AWS Now>> Why Talend Data Streams? One of the biggest challenges that businesses face toda...

In the past few years, there has been a shift in the data industry, leading to the emergence of a new category of data citizens: the ‘ad hoc’ or ‘citizen’ integrators. With these new personas adding to the (already long) list of data workers having access to corporate information, companies are needing to re-think the way they approach their data security and data governance strategies. Unlike data engineers, this new class of ‘citizens’ or personas, don’t necessarily utilize da...

In my previous blog, “Talend & Apache Spark: A Technical Primer” I walked you through how Talend Spark jobs equate to Spark Submit. In this blog post, I want to continue evaluating Talend Spark confiurations with Apache Spark Submit. First, we are going to look at how you can map the options in the Apache Spark Configuration tab in the Talend Spark Job, to what you can pass as arguments to Spark Submi...

Many businesses today are scrutinizing their operations to figure out how to join the digital transformation revolution. They understand that to become more competitive and customer-centric, they need processes that are flexible, integrated, insightful and scalable. They understand harnessing data and infusing business processes with it is the key to success. Unfortunately, poor data practices,...

We all know that enterprise data needs change constantly, and recently that change has come at an increasing pace. Companies that were once processing all their big data on-prem have suddenly moved into the cloud. Frameworks we used to know and love suddenly become obsolete.  However, an interesting debate that still rages on is how to get data processed faster. There are generally two heralded ways of processing data today: Batch Processing Stream Processing...