Talend Cloud How-to #1: Get to Know the Cloud Integration Platform

Talend Cloud is a secure cloud integration platform-as-a-service (iPaaS) that enables our customers to extract value from their cloud or on-premise data.  In this article, we will introduce you to our cloud integration tool and show you how easy it is to jump in and get started. We will also take you inside Talend Open Studio and illustrate how the tMap, remote engine, and more than 900 connectors help make cloud data integration easy.

Download Talend Cloud How-to #1: Get to Know the Cloud Integration Platform now.
View Now

Benefits of Talend Cloud

After the free download, the Talend Cloud environment is ready to use right away. The future-proof cloud integration platform, which can also be called an iPaaS, is a managed solution for hosting, developing, and integrating various on-premises and cloud data sources and applications. Talend Cloud helps bring agility and scalability to an enterprise by handling all of the routine non-value added tasks like backups, patches, or upgrades. In addition, Talend Cloud dramatically simplifies the integration of data, applications, security, and business compliance, giving your teams more time to focus on creating value for your organization.

Step-by-Step Application Integration

Once a job is created inside of Talend Open Studio you will publish it to the Talend Cloud. Once in the cloud integration tool, you can manage it through the software development lifecycle (SDLC).  Then, publish your job to a development environment and promote it into any environment - from test to production, the cloud integration tool can host the jobs anywhere. There is a scheduler and APIs to perform tasks like integrating with a third-party scheduler so you can stop, start, or check the status of jobs that are running. User management and permissions are present both in user roles and user groups. And Talend Cloud provides centralized monitoring and log aggregation across jobs regardless of where they run. It doesn't matter where your data, your jobs, or your infrastructure resides; Talend Cloud is a true iPaaS that can connect to, and manage, anything, anywhere. 

Download Talend Cloud How-to #1: Get to Know the Cloud Integration Platform now.
View Now

Running Jobs Through the Remote Engine

Every time you run a job you are going to publish it to a remote engine. Remote engines enable you to run the job wherever you want, whether an on-premise or cloud environment. This means that all the data that is being processed by that job is pulling directly to the remote engine that is controlled by you as a customer – it’s not flowing through the Talend Cloud. This reduces data security and privacy concerns significantly while still providing the advantages of cloud. And, for example, if you are working on two on-premise databases, you get optimal performance by having the ability to process the data close to where it's stored.

talend studio

Moving Data from Salesforce to Snowflake

Now, let’s talk about the Salesforce to Snowflake job flow we are going to demonstrate in the next article in the Talend Cloud How-to Series. We are going to leverage Talend Cloud and all the conventional components we just mentioned, and start by creating a simple job in Talend Studio and publishing it. Then we will operationalize the job by scheduling it to run inside of a remote engine to pull our Salesforce.com data and writing it into our Snowflake database. Snowflake is a very impressive cloud-based data warehouse that can run at scale.

The Talend Open Studio Landscape

To start, we go into Talend Studio, the development tool that you would use for any of our products today – including Big Data, MDM, or Application Integration – for the Salesforce to Snowflake demo in the next article, we will focus on data quality.

talend studio

Inside our Eclipse-based Studio on the left side is the “Repository” where a lot of useful items are housed.  The job designs are at the top, listing all the different jobs that we have in this environment. The repository, all the jobs, and metadata can be stored and housed within a Git repository, so we can share jobs and code and connections and effectively collaborate with coworkers. 

Also on the left side is the metadata about the source and target system's databases, files, and applications like Salesforce. In the next article, we will use our Salesforce data connections and objects to quickly build a job that's going to load account and orders into Snowflake.

Leveraging the Connectors and Components

Talend offers more than 900 components, enabling a seemingly exponential number of connections. We can see the palette that contains those on the right side of the Studio. For example, in the screenshot below you can see the considerable number of databases to which we can connect. Above “Databases” you can see all the other components Talend Cloud offers: data quality, business systems, custom code, and other useful data tools. Everything in the cloud integration platform is searchable, which is a good thing, because there are so many components and connectors it would be hard to display them all.

Starting a Talend job using the tMap

In the middle of the Talend Open Studio is where we will start dragging and dropping components to build the actual process that we refer to as a “job” which is the integration process.  The area below that is where we will configure the components of the job, such as Database connections, aggregations, and more.   

One of the most versatile components in the Talend Studio is the tMap. The wealth of functionality the tMap offers saves hours of work in an easy to use manner. Among the things we can do are map source to target output, filter input and output data, apply transformations, and do expressions and joins.

talend cloud welcome

Using Talend Cloud to Integrate Salesforce and Snowflake

When we go into the Talend Cloud environment, there are multiple applications on the page: Data Preparation, Integration Cloud, Management Console, and the latest addition – Data Stewardship. Data Preparation gives non-technical users the ability to cleanse, shape, form, enhance data through a web-based user-friendly UI.  The Management Console is the administration piece of Talend Cloud, where we set up projects, connect to Git, assign remote engines, create users, access rights, and roles.  

Now that you have a lay of the Talend Cloud land, let’s put it into action. In Part 2 of the Talend Cloud How-to series, we will demonstrate how to create a simple job that moves Salesforce data into Snowflake. Then using the cloud integration tool to run it through a remote engine or cloud engine, schedule and monitor. As you will see, Talend Cloud does the heavy lifting when we are ready to automate our jobs.

| Last Updated: August 8th, 2019