Introducing the Talend Architecture Center – Your One Stop Resource for Best Practices, Architectures and More

Introducing the Talend Architecture Center – Your One Stop Resource for Best Practices, Architectures and More

  • Venkat Sundaram
    Venkat Sundaram is a Customer Success Architect at Talend. Prior to Talend, Venkat was focused on implementing data warehousing, data integration and business intelligence technologies to solve traditional business data processing problems for several US financial institutions.

Today, Talend is pleased to announce the release of the Talend Architecture Center. This site provides you — our customers and partners — reference architectures and guidance for building scalable and secure solutions using Talend products.

The goal of this Architecture Center is to publish logical and physical architectures in order to help you understand the various components in our products, the relationship and interactions between them, and best practices around their deployment. We also provide guidelines on scalability, availability, security and sizing. This is published on a GitHub repository — anyone is welcome to review the content, try it out in your environment and provide us your feedback.

The content start off with version 6.4 and our goal is to keep the content up to date and generate new content for every subsequent major version of the product.

Where it all began

In the past, whenever a customer or partner would request our assistance in understanding our products or architecting a solution, one of our Talend team members would provide them the required assistance. The guidance can come from any of our client-facing teams — Customer Success Architecture, Professional Services, Sales Engineering, Enablement etc.

Late last year, we decided to create an internal forum to bring all these great ideas together and create a standard reference architecture that we could share with our customers/partners. As our footprint in the enterprise scaled up quickly, we realized that for us to scale and for our customers to scale, it would be a great idea to capture and publish all architectural artifacts in a public repository that our customers could easily access. A few of us at Talend got together to brainstorm and standardize our content. The Talend Architecture Center then came about as a result of that collaborative effort.

The architectural artifacts that you’ll find are based on years of experience working in Talend deployments of all sizes and forms. We also leveraged existing architectures that we have come across and mined those architectures for best practices and patterns that work and eliminated things that don’t.

What kinds of content can I find?

Talend has several products that help organizations manage their data — Data Integration, Big Data, Cloud, Real Time, MDM, Data Quality etc. Many of our customers also leverage Talend Data Fabric that includes all the products mentioned above in a single, unified platform. We have organized our content based on our products. If you use one of our platform products (more than one product), we provide architectural guidance on the platform.

For each of the products/platforms, we provide you details on:

  • Logical Reference Architecture — an overall architecture describing the various Talend components grouped into functional blocks based on their role in their architecture and the interoperability between themselves. The various functional blocks include:
    • Clients (for building and monitoring Talend jobs)
    • Servers (for administration, management, and monitoring)
    • Databases (to store metadata and configuration information)
    • Repositories (to host project metadata and binaries) and
    • Execution Servers (to deploy and launch jobs)
  • Physical Reference Architecture — provides a template for physical deployment of the logical architecture. We cover guiding principles based on our best practices, and deployment scenarios and strategies for various environments — Development, QA, Pre-Production, and Production. This includes high availability and scalability options for your Production deployment.
  • Recommended Sizing — we provide sizing guidance for various components in the architecture. This is based on recommendations from our Product team as well as our real-world experience deploying this architecture for our customers.

Big Data Architecture Example

How do I use it in my own Talend deployment?

Our main goal is to trigger your thought process as customers/partners. You can use this content to map the reference architecture to your current and future business requirements and subsequently build an architecture for your enterprise. You can browse through and study the relevant portions of the website based on the products you have been licensed for. We also encourage you to browse through the other products so that you are ready to scale up when the need arises. If you are planning to migrate to a later version of the product, this is a good starting point to understand the architectural changes of the new version and its impact.

If you have scenarios which are not covered here, please feel free to share it us with us. It will help us improve our reference architecture based on feedback from real-world implementations of our platform.

Ready, set, build

This blog introduced you to the Talend Architecture Center and the content available there. The content captures several years of knowledge and subject matter expertise within various organizations in Talend. It includes detailed information about Logical and Physical architecture for all Talend products and platforms. In the future, we would like to include other areas of architecture e.g. architectural styles and design patterns (Batch, Real Time, Streaming, Event Driven), Solution Architecture (Data Lakes, Data Warehouse, ODS, Data Migration) and other topics that could serve as a valuable input for our customers building solutions using Talend.

We welcome your feedback on this work. You can either leave comments on this blog or at the Talend Community discussion forums.

Join The Conversation


Leave a Reply

Your email address will not be published. Required fields are marked *