What’s New in Talend Winter ‘20?
Reveal the intelligence in your data.
Every business wants confidence in their data to make smart decisions and deliver superb customer experiences. Don’t let lack of resources or legacy technologies limit your potential for growth. Talend is always introducing new efficiencies to help deliver data intelligence faster to the people who demand them.
See what’s new in Winter ’20.
Capture your Data Intelligence at first sight
Boost Data Intelligence at speed
Empower Data Intelligence at scale
Capture Data Intelligence at first sight.
Talend Data Inventory maintains a unified inventory of all datasets used across Pipeline Designer and Data Preparation. This new capability allows you to inventory, curate, consume, and share data all from a single source of trust. Automate your data assessment with a data intelligence score and make them accessible to more people. No more silos or duplicating data sets. Now everyone can automatically share data intelligence and easily find the latest data for their analysis.
Boost Data Intelligence at speed.
Pipeline Designer now features intelligent data quality capabilities. Data engineers and workers can integrate, standardize, cleanse, and enrich data in a single app. In-flight data quality and protection eliminates problems before the data is consumed or replicated. No coding or complex transformations are required, increasing development and maintenance productivity. Combined with Talend Data Inventory, you can boost your data intelligence like never before.
Empower Data Intelligence at scale.
Democratize Data Intelligence with an AI-enabled Magic Fill to shape data the way you want. Intelligent data quality with explainable AI standardizes data and allows you to deliver data intelligence at scale. Increase data reuse by crowdsourcing ratings, comments and endorsements across the enterprise. This unique automation provides greater levels of transparency enabling greater confidence in your decision thus empowering data intelligence at scale.
Enhancements
Winter ’20 includes numerous productivity enhancements for all products. Deliver trusted data faster to make smart decisions. Connect to more apps from AWS, Azure, Cloudera, Databricks, Google, Snowflake and Workday, and move faster from on-prem to Cloud. Some enhancements are coming soon. The product release notes at https://help.talend.com/ include more information and feature availability.
- A new Cloud app to inventory, curate, consume, and share data all from a single source of trust
- Dataset overview exposes all the relevant information on a given data set in a single click by accessing the dataset’s ID card
- Custom attributes to enrich and augment datasets with custom metadata
- Faceted search facilitates data sets discovery based on any available metadata – including quality and semantic awareness
- Data set provenance and destination shows the preparations and pipelines using a given data set
- Social curation brings ratings, certification and tagging of data sets
- Intelligent data preparation standardizes, cleanses, and enriches data directly in Pipeline Designer
- Join processor improvements provide a better user experience through Live Preview. This includes records flattening, pushdown from preview to sample, and how a dataset is picked to the main flow
- Testing updates improve productivity and shortens the learning experience
- Updated Databricks run profile configuration enables reuse of an existing Databricks cluster that has been set up to be used with Pipeline Designer
- Remote Engine for Pipelines on Azure is now fully supported with Azure resource template (cloud formation script for easy set up)
- Automatic documentation and sharing of any dataset used by Pipeline Designer for reuse in Data Inventory
- Support for transient usage on Databricks clusters on Azure (already available on AWS)
- Support for Databricks Delta Lake. Read and write from/to Delta Lake Bronze tables and benefit from Delta Lake ACID, time travel and partitioning capabilities
- Support for:
- Databricks 5.5 LTS (Long Term Support) on Azure and AWS
- Hortonworks Data Platform (HDP 3.1)
- Google Dataproc 1.4
- Apache Kafka 2.2.1
- Azure Data Lake Store (ADLS) Gen 2
- Support for the latest Spark API and features including Spark Datasets/Dataframes. Automatically benefit from Spark optimization.
- Monthly Studio updates provide features and bug fixes sooner
- Support of a local repository (Nexus) to proxy Talend’s hosted repository of additional Studio libraries
- Faster and better job audit and troubleshooting with support for Log4j2
- Support for Azure Data Lake Store (ADLS) Gen 2 and Workday
- Explainable AI for matching rules lets you visually understand the heuristic of a matching model generated by a grouping campaign
- DQ datamart now enables the collection of data quality statistics in Snowflake, for example related data completeness, range of values, and numbers of duplicates
- Improved anonymization for privacy by design and data privacy compliance with the ability to identify “rejected” records in masking components
- Manage exceptions for records that cannot be automatically masked in a dedicated raw output flow
- Native connectivity for Microsoft Azure Event Hubs; Google Pub/Sub, BigQuery and Cloud Storage; NetSuite and Workday
- REST API support for reading batch and streaming data
- Easily drag and drop CSV and Excel files to be stored in Talend Cloud (Cloud Engine)
- A new widget for selecting data types provides better discoverability of type options through an at-a-glance browsing of available types and constructs, as well as filtering
- API Tester – Free Edition is a freely available tool distributed through the Chrome Web Store to visually interact with HTTP APIs
- API Tester now exposes an API which allows you to manipulate your API test projects from command-line scripts and continuous integration/continuous delivery pipelines
- Easier writing from TDM to a flat Studio structure generating the map and the target TDM structure
- New Eclipse-independent runtime is more robust, universal, and cloud-friendly. It provides increased flexibility to run Talend Data Mapper in different environments and applications.
- Magic Fill data standardization and formatting improvements include handling case sensitivity and semantic transformation for US state/state code, country/country code and dates
- Outliers detection for numerical values automatically highlights anomalies outside a dynamically computed range
- Any dataset used by Cloud Data Preparation is automatically documented and shared for reuse in Data Inventory
- Cloud Data Preparation leverages Data Inventory connectivity to access a wider range of data sources
- Performance improvements for previews and interactive transformations
- Support for Data Stewardship and Data Dictionary on Azure provides improved Data Governance including pattern recognition for automatic data profiling and workflow-based capabilities for data arbitration, validation, and remediation.
- Improved user experience and unification with other Talend Apps, and improved user interface to identify and drill down on the tasks that require attention.
- UI improvements to tailor the user experience (including home page, object pages layout, navigation) to the needs of each business roles and individual users.
- Connectivity is expanded with new Metadata Bridges for Azure Data Factory, AWS Glue, Databricks Unified DataServices, Informatica Data Engineering and Denodo
- User interface and documentation in Japanese and French
- Auto-creation and auto-scaling of Remote Engines and their clusters through a Remote Engine API
- Orchestration API for creating and retrieving tasks and plans
- Improved task state management and an always up-to-date task status for data services and routes
- Easier to deploy and manage data services and routes for high availability scenarios
- In-product chat support for all Talend products
- When using a Single Sign-On provider, you can customize the logout URL of your Talend Cloud applications
- Support for MS SQL Server JDBC driver as a TAC database, leveraging security and high availability functions including: SSLProtocol, Always Encrypted, Always On Availability Groups, and Azure AD authentication
- New parameters to configure log rotation for business.log and technical.log