Creating Cluster Connection Metadata from Configuration Files

In this tutorial, create Hadoop Cluster metadata by importing the configuration from the Hadoop configuration files.
This tutorial uses Talend Data Fabric Studio version 6 and a Hadoop cluster: Cloudera CDH version 5.4.
1. Create a new Hadoop cluster metadata definition
Ensure that the Integration perspective is selected.
In the Project Repository, expand Metadata, right-click Hadoop Cluster, and click Create Hadoop Cluster to open the wizard.
In the Name field of the Hadoop Cluster Connection wizard, type MyHadoopCluster_files. In the Purpose field, type Cluster connection metadata, in the Description field, type Metadata to connect to a Cloudera CDH 5.4 cluster, and click Next.

Watch Now

Running a Job on YARN

In this tutorial, create a Big Data batch Job running on YARN, read data from HDFS, sort them and display them in the Console.

Watch Now

Running a Job on Spark

Learn how to create a Big Data batch Job using the Spark framework, read data from HDFS, sort them and display them in the Console.

Watch Now

Apache Sqoop: A Complete Guide

Apache Sqoop is an instrument expressly designed to import and export structured data into and out of Hadoop and repositories like relational databases, data warehouses, and NoSQL stores.

View Now

What is ELT?

ELT is the process by which raw data is extracted, loaded, and transformed into a data lake or warehouse. In contrast to ETL, ELT provides faster loading.

View Now

What is MapReduce?

MapReduce is a programming model or pattern within the Hadoop framework that is used to access big data stored in the Hadoop File System (HDFS). The map function takes input, pairs, processes, and produces another set of intermediate pairs as output.

View Now

ETL Testing: An Overview

ETL testing refers to tests applied throughout the ETL process to validate, verify, and ensure the accuracy of data while preventing duplicate records and data loss. Learn the 8 stages of ETL testing, 9 types of tests, common challenges, how to find the best tool, and more.

View Now

What is Hadoop?

Hadoop is an open source, Java based framework used for storing and processing big data. The data is stored on inexpensive commodity servers that run as clusters. Its distributed file system enables concurrent processing and fault tolerance.

View Now

ETL vs ELT: Defining the Difference

The difference between ETL and ELT lies in where data is transformed into business intelligence and how much data is retained in working data warehouses. Discover what those differences mean for business intelligence, which approach is best for your organization, and why the cloud is changing everything.

View Now

Data Lakes: Purposes, Practices, Patterns, and Platforms

TDWI surveyed top data management professionals to discover 12 priorities for a successful data lake implementation. With the right tools, a data lake enables self-service data access and extends programs for data warehousing, analytics, data integration, and more data-driven solutions.

Download Now

Self-Service Analytics

Organizations today are supercharged by data, but most only analyze a fraction of what they collect. Find out how self-service analytics is building a data-driven culture across a range of companies. 

Download Now