[TOS tutorial 08] Adding Condition-Based Filters Using the tMap Component
In this tutorial, discover how to set filters on your tMap outputs and learn how to configure them.
This tutorial uses Talend Open Studio for Data Integration version 6.
1. Create a new Job and add a tMap component
- In a new Job, add the movies metadata file as input delimited component.
- Add a tMap component to create new filters.
- Create a flow of data from the movies component to the tMap_1 component by linking the two components and rename the link as movies.
2. Add a filter to select only movies that were released in 1990 or later
- To create a new output, double-click the tMap_1 component and in the output section of the tMap wizard, click the [+] button and name the output as moviesFromThe90s.
- Select all columns from the input movies, drag, and drop them to the output moviesFromThe90s output.
- To create a new condition-based filter, click the Activate Expression Filter.
- To create a filter condition, click the releaseYear column and drop it on the expression filter field, and type >=1990.
3. Store the result in a file
- Add a tFileOutputXML component to the Job.
- Link the Row > moviesFromThe90s output to the tFileOutputXML_1 component. Rename the tFileOutputXML_1 component movies>=1990.
- To configure the movies >= 1990 component, in the Component view of the component, specify the path and name for the output file.
- To rename the root document, in the Row tag field, type “movies” and run the Job.
- To check the moviesFromThe90s.xml file, navigate to the folder in which the file was created and open the file. The file with the list of movies released in or after 1990 will be displayed.
4. Create a second filter using two conditions
- Create a second filter that will filter movies from the 80s only. Condition filters follow the Java syntax rules. You can easily create a double condition by using the java && operator (AND).
- Run the Job.
The file with the list of movies released from 1980 to 1989 will be displayed.
5. Create a filter reject output
- To create a third filter, in the tMap_1 interface, add another output named moviesBefore80s and add all columns from the input movies to it.
- To create a filter reject output, click the tMap Settings.
- Click the […] in the value field of Catch output reject property and click true. This will collect all rows of data which do not match either of the other two conditions.
- Add a tFileOutputXML component named movies<1980, and link the moviesBefore80s row from tMap_1 to it.
- To configure the movies<1980 component, in the Component view, specify the path and name for the output file.
- Run the Job.
- To check the filterRejects.xml file, navigate to the folder in which the file was created and open it.
The file with the list of movies released before 1980 will be displayed.
Ready to get started with Talend?
More related articles
- How to Get Started with Talend Open Studio for Data Integration
- [TOS tutorial 02] Reading a File
- [TOS tutorial 01] Discovering Talend Studio
- [TOS tutorial 03] Sorting a File
- [TOS tutorial 07] Configuring Joins in tMap
- [TOS tutorial 09] Using Context Variables
- [TOS tutorial 06] Joining Two Data Sources with the tMap Component
- [TOS tutorial 05] Filtering Data Using the tMap Component
- [TOS tutorial 04] Creating and Using Metadata
- [TOS tutorial 13] Running a Job on Spark
- [TOS tutorial 12] Writing and Reading Data in HDFS
- [TOS tutorial 11] Creating Cluster Connection Metadata from Configuration Files
- [TOS tutorial 10] Creating Cluster Connection Metadata
- [TOS tutorial 14] Running a Job on YARN