This post is co-authored by Jorge Villamariona and Anselmo Barrero at Qubole. A popular term emerging from the software industry over the last few years is serverless computing, more commonly referred to as just “...

When I hear the phrase “Data Warehouse Optimization”, shivers go down my spine.  It sounds like such a complicated undertaking.  After all, data warehouses are big, cumbersome and complex systems that can store terabytes and even petabytes of data that people depend on to make important decisions on the way their business is run.  The thought of any type of tinkering with such an integral part of a modern business w...

Let’s be honest, the ‘Data Lake’ is one of the latest buzz-words everyone is talking about. Like many buzzwords, few really know how to explain what it is, what it is supposed to do, and/or how to design and build one.  As pervasive as they appear to be, you may be surprised to learn that Gartner predicts that only 15% of Data Lake projects make it into productio...

This post was authored by Venkat Sundaram from Talend and Ramu Kalvakuntla from Clarity Insights. With the advent of Big Data technologies like Hadoop, there has been a major disruption in the information management industry. The excitement around it is not only about the three Vs –...

This post is co-authored by Jorge Villamariona and Anselmo Barrero at Qubole. A popular term emerging from the software industry over the last few years is serverless computing, more commonly referred to as just “...

Save the date! Talend Connect will be back in London and Paris in October Talend will welcome customers, partners, and influencers to its annual company conference, Talend Connect, taking place in two cities, London and Paris, in October. A must-attend event for business decision makers, CIOs, data scientists, chief architects, and developers, Talend Connect will share innovative approaches to modern cloud and big data challenges,...

Recently I had the chance to talk to an SVP of IT at one of the largest banks in North America about their digital transformation strategy. As we spoke, their approach to big data and digital transformation struck me as they described it as ever evolving. New technologies would come to market which required new pivots and approaches to leverage these capabilities for the business. It is more important than ever to have an agile architecture that can sustain and scale with your data and analyt...

Those rare companies that decide to engage in hand-coding do it either because they lack the required information (they simply don't know there are packaged solutions out there) or often mistakenly believe that all a big data project takes is to feed data into a Hadoop cluster. They figure developing a dedicated on-premise infrastructure to feed and analyze big data will mean a 20 percent savings over a packaged solution. However, according to Gartner, this approach results in a 200 percent i...

So far, our journey on using Apache Spark with Talend has been a fun and exciting one. The first three posts on my series provided an overview of how Talend works with Apache Spark, some similarities between Talend and Spark Submit, the configuration options available for Spark jobs in Talend and how to tune Spark jobs for performance. If you haven’t already read them you should do so before getting started here. Start with “...

Talend Spring ’18 (Talend 7.0) was Talend’s biggest release yet. With improvements to Cloud, big data, governance and developer productivity, Talend Spring ’18 is everything you need to manage your company’s data lifecycle. For those that missed the live webinar, you can view it here. But as highlighted in the webinar,  I realize that everyone is not going to Serverless today, or is l...