A Day In The Life of a Data Integration Developer [Part 1]

Are you are a data integration developer looking for a solution to connect your data in the fastest, most cost-efficient way possible? You need an open-source tool that is easy to use and that can synchronize data across data platforms and improve data accuracy. Talend Studio is the solution.

Within this video series, you will see how Studio's advanced features are used throughout the development cycle of a typical DI project: constructing a job, testing and debugging the process, deploying and managing jobs in a runtime environment, scheduling processes for easy development between environments, and utilizing advanced team development features.

Take a look at the step-by-step tutorials below and find out how your day as a data integration developer is about to get better.

Introduction to Talend Studio

First, we will get familiar with the look and feel of Talend Studio and get a general understanding of how to use some of the key functions and features. Although the Talend data integration platform is comprised of several different components, we’ll start with Studio's piece of the puzzle.

When I first launch Studio, I'm asked to select a connection and then either open or import an existing project or create a new one. For this demonstration, I'll use an existing project.

talend studio screen shot

It brings me to a welcome screen. I've already started a job, so let's open it from the Latest items list. The first time I open Talend Studio, it offers to guide me through the program, highlighting key points. It also opens a demo project for you to play around with to get yourself familiar with the studio. If I ever need a refresher, I can reactivate the tour within the Help dropdown.

Building a Job Flow in Talend Studio

Let's build a new Job flow to demonstrate how Talend Studio features work.

Let's first take a look at the Repository where Talend Studio gathers data related to the technical items used to design Jobs. It's here I can manage metadata, database connections, database tables as well as columns, and Jobs once I begin creating them.

I've already brought in my source component metadata by dragging and dropping it into my design workspace in the middle. The design workspace is where I lay out and design my Jobs. Then, I go to the right-hand side and view my component palette containing different technical components used to build my Jobs grouped together in families.

talend studio screen shot

A component is a preconfigured connector used to perform a specific data integration operation. I can minimize the amount of hand coding required to work on data from multiple sources. If I drill down, I can see all of the different connections, databases, and applications.

Let's bring this SalesforceOutput component into the palette to create a target for my source content.

talend studio screen shot

Now I would need to connect these two components using something like a tMap, giving me the ability to map attributes between the source and the target:

  1. Click on the source.
  2. Drag and drop.
  3. Connect to the tMap.
  4. Right-click on the tMap.
  5. Drag and drop over to the target.
  6. Give the output a name.
  7. Once that's set up, double-click on the tMap component to view the attributes of the metadata about the salesforce connection that you have established are visible.
  8. Drag these attributes from the source on the left to make the connections.

I now have a functioning process or Job within the design window. View Job details within the configuration tabs.

For complete details, watch the short video above. Next, we’ll walk through how to build your first job in Talend Studio.

| Last Updated: September 2nd, 2019