Step-by-Step: Constructing a Job in Talend Open Studio
Did you know a commercial airplane generates on average 500 GB of data per hour when flying? Taking into account the number of hours an airplane can fly per day and the flights operated on a daily basis, this creates huge volumes of data that need to be processed. We have good news for airline companies: Talend can help! The data warehouse project, described in this video, details how to build a data warehouse around airline flight performance data. Airline raw files are taken to build dimension tables and then fact tables into a data warehouse schema. The Talend Open Studio for Data Integration platform allows to build, test, deploy, schedule and monitor these processes.
The construction of a Job starts with using Metadata Wizards to build metadata, connecting and mapping data flows, reusing existing Metadata connections, running the processes and finally viewing results in the developer studio.
Connecting and mapping data flows are some of the first steps when constructing a Data Integration Job with Talend! The data from the file as defined in the metadata is read and passed onto the next component. In the video, you will be able to find out more about the use of the tMap component, a very versatile component in the Talend Data Integration studio, used for all types of transformations and mapping attributes from source targets. The aggregation component is also visible in the video and used to consolidate the unique city as well as codes from the files.
Talend is also capable of reusing existing Metadata connections, which allows a developer to drop a MySQL database table into the Talend process. Once ready, the developer can run the processes and see the progress, by checking the number of records reading per second from the file in the Job. Finally, viewing results in the developer studio is easy, by opening the output in the database just written by the process.
Remember, if you don’t already have Talend installed, you can still play along with the video by downloading Talend Open Studio for free.