Announcing enhanced data observability with Data Console

By Chloe Gout & Don Pinto
Screenshot of Talend Data Console showing the Talend Trust Score, number of datasets, a bar graph for data quality, data quality rules list, data types list and popularity of datasets. The screenshot is set on a gradient purple to blue backgroundScreenshot of Talend Data Console showing the Talend Trust Score, number of datasets, a bar graph for data quality, data quality rules list, data types list and popularity of datasets. The screenshot is set on a gradient purple to blue background

During QlikWorld ’23 in Las Vegas, we were thrilled to announce the general availability of Data Console in Talend Data Inventory. With data-driven decision making becoming more crucial for organizations, it’s never been more important for users to have access to high quality data. Data practitioners can use the new data observability capabilities in the Data Console to discover new datasets and continuously monitor the quality of these datasets so they can make informed decisions, optimize their operations, and stay competitive in their industry.

While you can see the exhaustive list of capabilities in our release notes, some of the latest areas of focus in Data Console include:

Improved data discoverability

On average, organizations have over 400 data sources that are scattered across multiple departments and typically live in silos. Data Console makes it easier to discover these data sources. It allows users to tag them appropriately and add descriptions to capture more business context. With this context, other users across the organization can more easily discover and use this data for their own purposes. 

screenshot of the data console discoverability feature, showing how many datasets have tags and how many have descriptionsscreenshot of the data console discoverability feature, showing how many datasets have tags and how many have descriptions

Powerful semantic analysis

Data anomalies and outliers can significantly impact the quality of insights and decisions made based on that data. With data observability tools, you can detect these anomalies and outliers early on and chart ways to intervene and correct this data before it is too late. Depending on the use case, datasets can contain various information types (typically referred to as semantic types), from customer details such as names and geographic locations to sensitive data such as credit card details and social security numbers. When dealing with a large amount of data, it’s crucial to clearly understand the shape and structure of your data and the relevant data types. This can help you identify any inconsistencies or missing information in your data and ensure that the data is accurate and reliable. What if you could spot these inconsistencies across all your datasets at a glance?

screenshot of data console's data types across datasets feature, showing the name of the data type, the quality, and how many datasets each type is used inscreenshot of data console's data types across datasets feature, showing the name of the data type, the quality, and how many datasets each type is used in

Talend’s Data Console uses semantic types to analyze data across datasets. Using semantic types shows you exactly what the data types are used for across different datasets and the level of quality corresponding to those data types.

Evolution of Trust Score

Data quality is not a one-time activity. It needs autonomic and continuous monitoring over time. Data practitioners should be able to see the results of their efforts to improve data quality — and appropriate thresholds need to be set if the data quality falls below a certain limit.

Screenshot of the Talend Trust Score feature, showing the validity, popularity, completeness, discoverability, and usages of a dataset over a 30-day periodScreenshot of the Talend Trust Score feature, showing the validity, popularity, completeness, discoverability, and usages of a dataset over a 30-day period

Talend’s Data Console continues to build on Talend’s innovative capabilities, such as Trust Score™, making it possible for users to see at a glance how the quantified Trust Score aspects of your data have changed over time and what exact data interventions have impacted the score. This allows you to identify data drifts quickly so that remedial action can be taken before downstream actions are negatively impacted.

Business-context-aware data quality rules

Data quality impacts how well data meets the needs of its intended use, which varies by organization. Maintaining data quality involves a number of steps, including standardization and validation. User-defined data quality rules that are based on business context allow business users to validate their data and continually evaluate data quality to ensure it supports their business effectively. 

screenshot of data quality rules feature of data console showing the name of the rule, a bar graph of the quality, and how many datasets are using that rulescreenshot of data quality rules feature of data console showing the name of the rule, a bar graph of the quality, and how many datasets are using that rule

With Talend’s Data Console, business users can quickly understand which data quality rules are in play, what datasets these rules apply to, the quality level when the rules are applied, how the rules effect the Trust Score, and which datasets need review based on quality thresholds defined by the organization. Additionally, data quality rules are stored in a central repository,  pervasive across the data management platform, allowing them to be reused.

Improved data sharing

Data quality and governance efforts need teamwork. To build trust and increase transparency across stakeholders, sharing data becomes critical. The Data Console shows which datasets are shared, their popularity, and whether the datasets are certified for use by other users.

screenshot of the data sharing feature of data console, which shows how many datasets are shared by, with, and not shared with a particular userscreenshot of the data sharing feature of data console, which shows how many datasets are shared by, with, and not shared with a particular user

By operating as a team united towards the goal of improving data quality, stakeholders can certify a dataset and assign it a rating. They can share commonly used datasets as a product with individuals or other teams by specific domain. This can give stakeholders the confidence to rely on a particular dataset for future operational and analytical activities.

screenshots of the popularity rating and a bar graph of the certification workflow  as well as the average score as rating by collaboratorsscreenshots of the popularity rating and a bar graph of the certification workflow  as well as the average score as rating by collaborators

In conclusion, the Talend Data Console is a powerful, new self-service tool for data practitioners who want to proactively monitor the quality of their datasets. We are excited to offer this powerful capability to our customers and look forward to seeing how it can help them achieve their data quality objectives.

The importance of data quality and governance cannot be overstated, especially in the mortgage industry, where accuracy and compliance are critical. The data console feature of Talend's product is particularly impressive, providing a comprehensive view of the data landscape and enabling users to make informed decisions based on high-quality data. Having seen the Talend Data Inventory demo at QlikWorld, I am convinced it will be an invaluable tool for many organizations like ours.
— Julia Fryk, Data Architect at Waterstone Mortgage

To learn more about the Data Console and Talend Data Inventory, please check out our documentation or try out a free 14-day trial today.