7 Reasons to “Unify” Your Application and Data Integration Teams and Tools
I recently attended a Gartner presentation on the convergence of Application and Data Integration at their Application Architecture, Development and Integration conference. During the talk they stressed that “chasms exist between application- and data-oriented people and tools” and that digital businesses have to break down these barriers in order to succeed. Gartner research shows that more and more companies are recognizing this problem – in fact, 47% of respondents to a recent survey indicated they plan to create integrated teams in the next 2-3 years.
And yet, very few integration platforms, other than Talend’s, provide a single solution that supports both application integration (AI) and data integration (DI). It seems that although many people intuitively recognize the value of breaking down integration barriers, many still have a hard time pointing to the specific benefits that will result. This post outlines my top reasons organizations should take a unified approach.
Stop reinventing the wheel
Separate AI and DI teams can spend as much as 30% of their time re-inventing the wheel or re-creating similar integration jobs and meta data. With a unified integration tool, you can create your meta data once and use it over and over again. You can also often avoid re-creating the same integration job. In many situations, the requirements of an integration job can be met with either style of integration, but with separate teams, you are forced to recreate the same jobs for different projects.
Learn from Toyota, Rapid Changeover Kills Mass Production
History is full of examples where management has opted for specialization to increase throughput. This works really well in a predictable environment. A great example of this is Ford’s approach to the model T, where you could have any color you wanted as long as it was black. They could crank out the cars for less than anyone else with their assembly lines and mass-production approach. Unfortunately, I have yet to see an IT organization that could successfully predict what their business owners need. That’s why Toyota’s “one-piece flow” and flexible assembly lines have so dramatically out performed U.S. auto maker’s dedicated production lines.
Pay only for what you need
If you’re building two separate integration teams, you’re probably paying your implementation and administration “tax” twice. You’re buying two sets of hardware and you’re paying people to set up and maintain two separate systems. This tax is especially big if you need a high availability environment with live backup. With a unified integration tool, you’re only doing all of this once and the savings can be huge. At one large Talend customer, they had a team of 10 admins across AI and DI and they were able to cut that to 5 with Talend.
Train once, integrate everything
If you use two separate integration tools, it means you have to have specialists that understand each, or you have to train your people on two completely different and often highly complex tools. With a unified solution, your developers can move back and forth across integration styles with very little incremental training. This makes it much easier for your integration developers to stay current with both tools and styles of integration, even if they spend the majority of their time on a single style. This reduces training costs and employee ramp time while increasing flexibility.
Win with speed
With new types of data (web and social) and new cloud applications, data volumes are exploding in every company, large and small. This is making the ability to be data-driven a strategic differentiator that separates winners from losers. A critical part of being data driven is allowing the business to put the right data in the right places as quickly as possible. With a unified tool, you can start out with one style of integration and be ready to add other integration styles a moment’s notice. A great example of the need for this is when a data warehousing project starts out with batch data movement and transformation requirements and then later business teams realize they can use this same data to make real-time recommendations to sales. Without a unified integration solution, this would require two separate integration teams and two separate projects.
Application integration and data integration tools are often stronger at some things and weaker at others. For instance, data integration tools can include strong data quality and cleansing capabilities that application tools lack. With a unified solution, each style of integration can benefit from best that the other integration style has to offer.
It happens in almost every business. Executives show up at a meeting and each has their own reports and data that give very different views on the business and it’s almost impossible to reconcile the differences. The same thing happens with separate integration teams. Each defines separate rules around prices, revenue and product lines and as a result, it’s very hard to get a consistent view on the business and key performance indicators. A unified tool allows you to build those rules once and then apply them consistently across every integration job. This kills many data discrepancies at the root cause.
What do you think? I’m interested to hear from folks that are considering a unified approach, but believe the challenges maybe too great – equally happy to engage those with opposing viewpoints.