A Day In The Life of a Data Integration Developer Series: Basic Job Design Features [Part 5]

In this installment of A Day in the Life of a Data Integration Developer, we will focus on basic job design features within Talend Studio. One of the things that we want to focus in on first is the context variables.

Context variables are very useful. They can be used globally or locally. Global context variables can help make your job more dynamic, so you can migrate it from dev, test, or prod.

So let's take a look at some context variables within the dim load process.

I defined some global context variables here on the left.

talend studio screen shot

Open the context variable for the database suffix and file name. This context variable is used in all of the airline loading processes—both the dim and the fact tables. I'm going to define the suffix for the database, the file names, and the location.

Then there are different configurations within here.

talend studio screen shot

I can have a configuration for dev, prod, and QA, so that I can select which context variable I want to use, depending on which environment I want to load. The context variables, then, are part of the connection string and the file location within the Jobs themselves.

This allows me to migrate the Jobs without changing code. I'm just selecting a different configuration for parameters.

If you click on the Job itself and then go to the Context tab, you'll see where these context variables have been imported into the Job. So here are the context variables with the database suffix and the file location.

talend studio screen shot

Once you've selected or imported those contexts to select which configuration, you select the icon on the right, top, and then open a drop-down and select whether you want dev, QA, or prod. This one's using dev.

You can also see how these context variables are used. For example, this connection string is using the db suffix at the end of airline, so, depending on which configuration I'm using, it will call it airline_dev or _prod.

So now when you run this process it's going to populate the airline_dev configuration or database that is set up using the dev passwords, which I've imported from a reference context project. So as that runs it will complete and load my dev airline dim tables.

For more details on this process, watch the video above. Next, learn how to self-document any data integration Job.


| Last Updated: March 14th, 2019