Writing and Reading Data in HDFS
In this tutorial, generate random data and write them to HDFS. Then, read the data from HDFS, sort them and display the result in the Console.
Watch NowCategories
Content Types
Talend Products
Creating Cluster Connection Metadata from Configuration Files
In this tutorial, create Hadoop Cluster metadata by importing the configuration from the Hadoop configuration files.
This tutorial uses Talend Data Fabric Studio version 6 and a Hadoop cluster: Cloudera CDH version 5.4.
1. Create a new Hadoop cluster metadata definition
Ensure that the Integration perspective is selected.
In the Project Repository, expand Metadata, right-click Hadoop Cluster, and click Create Hadoop Cluster to open the wizard.
In the Name field of the Hadoop Cluster Connection wizard, type MyHadoopCluster_files. In the Purpose field, type Cluster connection metadata, in the Description field, type Metadata to connect to a Cloudera CDH 5.4 cluster, and click Next.
Creating Cluster Connection Metadata
Learn how to create Hadoop Cluster metadata automatically by connecting to the Cloudera Manager.
Watch Now
Running a Job on YARN
In this tutorial, create a Big Data batch Job running on YARN, read data from HDFS, sort them and display them in the Console.
Watch Now
Running a Job on Spark
Learn how to create a Big Data batch Job using the Spark framework, read data from HDFS, sort them and display them in the Console.
Watch Now
Get Value Out of Your Data With a 360° Data Hub
This white paper shows you what is a data hub and why are the most forward-looking companies creating them.
Download Now
Apache Sqoop: A Complete Guide
Apache Sqoop is an instrument expressly designed to import and export structured data into and out of Hadoop and repositories like relational databases, data warehouses, and NoSQL stores.
View Now
Data Extraction Tools: Improving Data Warehouse Performance
Data extraction tools significantly expedite the collection of relevant data. Learn about the tool and the benefits achieved by leveraging the right data extraction software.
View Now
Harnessing Data Quality for Better Decisions
A good data quality solution empowers you to readily embrace enterprise-class data governance as a core discipline within your data management processes, simply and effectively.
Watch Now
Why ELT Tools Are Disrupting the ETL Market
With 50% of business data residing in the cloud, more organizations are turning to ELT tools to address their processing needs. Learn how ELT tools are impacting the future of data integration.
View Now
Machine Learning Sandbox – Data Warehouse Optimization
This video demonstrates a Data Warehouse Optimization approach that utilizes the power of Spark to perform analytics of a large dataset before loading it to the Data Warehouse.
View Now
Machine Learning Sandbox – Real-Time Risk Assessment
This video demonstration shows how an online bank is trying to mitigate their exposure and risk by targeting credit offers to only those customers whom are deemed low risk and most likely to accept the credit offer.
View Now
Machine Learning Sandbox – IoT Predictive Maintenance
This video demonstration shows how a large company with over 50,000 vending machines can use the power of IoT (Internet of Things) and machine learning to determine an individual machine’s likelihood to break down.
View Now
Machine Learning Sandbox – Building the Sandbox and Selecting a Platform
This video shows you what to expect when you start the sandbox for the first time and how to select a Big Data Platform for evaluation.
View Now
Machine Learning Sandbox – Sign Up and Download
This video shows you how to get signed up and download the Talend Big Data and Machine Learning Sandbox.
View Now
Best Practices Report: Multiplatform Data Architectures
This Multiplatform Data Architectures report explains in detail what MDAs are and do, with a focus on helping data professionals and their business counterparts worldwide architect, govern, and grow their MDAs for better business outcomes via well-integrated and unified distributed data from many sources.
Download Now
ETL in the Cloud
Since the dawn of big data, the ETL (extract, transform, and load) process has been the heart that pumps information through modern business networks. Today, cloud-based ETL is a critical tool for managing massive data sets, and one that companies will increasingly rely on in the future.
View Now
How to Move Data from Salesforce to AWS Redshift
Learn how to extract CRM data from Salesforce and load it into the Amazon Web Services (AWS) Redshift cloud data warehouse.
View Now
What is ELT?
ELT is the process by which raw data is extracted, loaded, and transformed into a data lake or warehouse. In contrast to ETL, ELT provides faster loading.
View Now
What is MapReduce?
MapReduce is a programming model or pattern within the Hadoop framework that is used to access big data stored in the Hadoop File System (HDFS). The map function takes input, pairs, processes, and produces another set of intermediate pairs as output.
View Now
ETL Testing: An Overview
ETL testing refers to tests applied throughout the ETL process to validate, verify, and ensure the accuracy of data while preventing duplicate records and data loss. Learn the 8 stages of ETL testing, 9 types of tests, common challenges, how to find the best tool, and more.
View Now
What is Hadoop?
Hadoop is an open source, Java based framework used for storing and processing big data. The data is stored on inexpensive commodity servers that run as clusters. Its distributed file system enables concurrent processing and fault tolerance.
View Now
Big Data Quality
With the advent of big data, data quality management is both more important and more challenging than ever. Fortunately the combination of Hadoop open source distributed processing technologies and Talend open source data management solutions bring big data quality operations within the reach of any organization.
View Now
ETL vs ELT: Defining the Difference
The difference between ETL and ELT lies in where data is transformed into business intelligence and how much data is retained in working data warehouses. Discover what those differences mean for business intelligence, which approach is best for your organization, and why the cloud is changing everything.
View Now
YARN in Hadoop
The recent introduction of YARN in Hadoop provides organizations that are managing big data with even greater processing speed and scalability. An acronym for Yet Another Resource Negotiator, YARN in Hadoop solves a bottleneck in the first version of Hadoop MapReduce and reduces the strict dependency of Hadoop environments on MapReduce.
View Now