During Summer, The Talend Blog Team will relay to share fruitful tips & to securely kick off your data project. This week, we’ll continue with the second capability: make sure you stay in control of your data pipelines from ingestion phase to consumption and beyond.
Get control over all the organization’s data and prevent shadow IT
Whenever an IT system, application or personal productivity tool is used inside an organization without explicit organizational approval, we talk about shadow IT. Shadow IT is not only a security and compliance nightmare, it creates a data sprawl where each group can create its data silos.
As per Wikipedia: Shadow IT, also known as Stealth IT or Client IT, are Information technology (IT) systems built and used within organizations without explicit organizational approval, for example, systems specified and deployed by departments other than the IT department.
According to a Cisco 2016 customer survey, there is 15-25X more used services without iT involvement in an organization. Furthermore, the cloud services explosion is likely to accelerate this trend.
Why it’s important
The more shadow IT is developing, the less easy it’s for users to access & protect data.
IDC estimates that data professionals spend 81% – and waste 24% – of their time searching, preparing and protecting data before they can actually take advantage. When data is not a team sport, everyone spends time creating silos and their version of truth, which drives up costs. Decisions are influenced by questionable data and ultimately put the organization at risk.
IDC went even deeper in the analysis in a data governance webinar, highlighting the high frequency of spreadsheet usage by business users as a data integration tool. Data silos start here, as copy/paste is the most frequently used approach to bring this data in. To avoid any shadow IT, equipping people with modern tools such as Talend Data Preparation is essential for avoiding creating those uncontrolled copies of data. Data citizens can then process data from sources to destination without keeping local storage or unknown or unprotected folders, systems, on premise storage or uncontrolled cloud-based storage.
This is not acceptable anymore with the rise of regulations (Basel II, IRFS, GDPR, CCPA, etc.), where companies are mandated to take control of their data assets. If they don’t, companies run the risk of being non-compliant and being exposed to significant regulatory fines.
When it’s important
Data Control should take place everywhere: when the data enter the system, along data pipelines and at data consumption points thru apps, api or analytics.
As more and more data professionals are getting closer to operations to drive business outcome “where the action is”, there is a growing risk of data fragmentation and misalignment. There is a need for a central organization that can enable people with data in a governed way while tracking and tracing data flows through data lineage.
Our recent data trust readiness report reveals that 46% of executives believe their organization is always in control of data. That figure falls to 28% for data practitioners. That shows that the problem seems to be controlled at top level, but data workers are less confident.
Data Trust Readiness Report now.
How Talend tools can help
Talend Data Catalog helps you to create a central, governed catalog of enriched data that can be shared and collaborated on easily. It can automatically discover, profile, organize and document your metadata and makes it easily searchable.
Imagine that you find some inconsistent data in your data systems that have been created and perpetuated in one of your datasets and you’re asked to explain it, identify it and correct it. The data lineage will dramatically accelerate your speed to resolution by helping you to spot the right problem at the right place. Moreover, if new datasets come to your data lake, establish a data lineage will help you to identify these new sources very quickly.
But data lineage provided by Talend Data Catalog is not enough. You also have to cleanse it without leaving local files on unsafe data systems. To avoid any shadow IT, equipping people with modern tools such as Talend Data Preparation is essential: it helps them to cleanse data whilst avoiding local data treatment files such as excel files. Data citizens will process data from sources to destination without keeping local storage or unknown or unprotected folders, systems, on premise storage or uncontrolled cloud-based storage.
How to get started
If you want to become a digital leader, you need to be in control of applications wherever they’re located within the company. In our survey, top leaders combine speed and integrity by reclaiming control of shadow IT while still enabling the business with data. Incorporating appropriate data controls in your data chain is vital for the success of your data governance initiatives. Establish a step-by-step approach to data governance, from discovery to sharing.
Download our Definitive Guide to Data Governance to learn how to take control of your data assets and maximize their value. This bottom up and data-driven approach to governance will foster a data-driven culture as well as the benefits of delivering trusted data at speed.
Want to explore more capabilities?
This is the second out of ten trust & speed capabilities. Cannot wait to discover our third capability?
Go and download our Trust Data Readiness Report to discover other findings and the other 9 capabilities or stay tuned to discover our next week Tip #3 : Make people accountable around a single source of trusted data.