We needed the ability to build custom connectors and realized that Talend Open Studio for Data Integration was the only tool that would allow us to reuse existing scripts and develop our own components. In addition, it was bringing us the major advantages of open source: openness, customization, and no licensing costs.Florent Marat, Technology Analyst
A Research Aggregator made in New Zealand
An independent research and development specialist, MediaLab partners with commercial organizations, universities, and government agencies to identify specific and complex business needs. The resulting partnerships deploy project teams to develop new technologies and innovative solutions.
Primarily involved in the Information & Communications Technology industry, MediaLab manages a range of research projects and develops its own products, especially for life cycle management (optimization and modeling) and resource management. MediaLab partners with companies such as Telecom New Zealand, Alcatel Lucent, OpenCloud, Kordia, and CityLink. In addition, the company offers an extended range of professional services, ranging from the optimization of technology investments and operations research to consultancy and network testing.
Nine data sources
As part of a recent project for a major telecommunications operator in the Asia-Pacific region, MediaLab was looking for a solution that could handle complex integration challenges quickly.
"Our team of statisticians uses internally-developed modeling software to compute data and provide information for management decision support. The results of this processing needed to be integrated in the SAP system of our client. This modeling software is linked to 9 data sources and outputs need to reach two different stakeholders every month, one of them SAP," explains Florent Mara, Technology Analyst at MediaLab. "We had developed a data integration solution internally based on Perl scripts, but the development was too long and wasn't reliable enough for this highly strategic type of project. In brief, our solution did not offer sufficient stability, functionality, and scalability. We started to look for a robust data integration solution that would better meet our needs."
The timeframe to develop this project was rather short: "We only had three months to develop our integration solution - actually, more like 5 to 6 weeks when taking tests and deployment into account. Custom scripting could not possibly have worked, both in terms of timing and features," continues Florent Mara. "We looked for a solution that would be able to reuse the scripts we had developed previously, and manage all the complex parameters we handle. We also needed an important level of flexibility, to be able to meet changing requirements during the development and testing phases. Talend Open Studio for Data Integration not only met these needs, but we could also extend it with our own components."
Robustness and performance
Before executing the statistical computation, MediaLab needed to manipulate data from these 9 sources, in 6 different formats-convert format, cleanse the data, load it in a database, and then launch the integration jobs and calculations. The resulting data is then integrated into SAP via an XML format after another format conversion.
"We needed to build custom connectors to smoothly combine the integration processes with our statistical software and produce the required SAP output," explains Florent Mara. "After evaluating several integration solutions available on the market, we realized that Talend Open Studio for Data Integration was the only tool that would allow us to reuse existing scripts and develop our own components. In addition, it was bringing us the major advantages of open source: openness, customization, and no licensing costs. The other solutions we researched did not meet our criteria - they were too expensive, too inflexible, or too closed."
Time saving and risk mitigation
MediaLab is currently in the process of testing their developments, and will soon be installing them into their production environment.
Yet, the first benefits of the chosen solution can already be felt, especially with regards to time savings. "Even though the systems integration isn't complete yet, Talend Open Studio for Data Integration helped us decrease monthly development costs by about half, which translates into significant savings for our customer," rejoices Florent Mara. "In addition, because processes developed with Talend Open Studio for Data Integration are more robust than the ones we used to build internally, we are mitigating the risks linked to the project, to project changes and to its very tight timeframes - an important factor in the telecommunications industry."
"Given our initial successes and our fast ramp up with the solution, we have now incorporated it into another aspect of our project," adds Florent Mara. "Thanks to its automation capabilities, Talend Open Studio for Data Integration now manages and runs part of our statistical calculations, further increasing the reliability and time savings."
Talend Open Studio for Data Integration offers MediaLab greater maintenance flexibility. "We can easily add or modify data sources and quickly create new transformation and integration scripts. Talend Open Studio for Data Integration is becoming MediaLab's integration solution; we have started to use it for other projects, such as Network performance measurement," concludes Florent Mara. "With the time saving and the increased reliability brought by Talend's tools, we are reducing our development costs and reinforcing our credibility with our clients and prospects. We are currently initiating important projects with major global players in the telecommunications field, in which Talend's solutions might play a strategic role."