We are particularly impressed with how simple it is to expand the solution. This makes it very easy to connect additional systems.Michael Schulz, Data Warehouse Manager mobile.de
With around 65 million visitors per month, mobile.de is Germany’s most popular online marketplace for buying and selling cars. It also has the largest selection of passenger and commercial vehicles and motorbikes in Germany. Visitors to the site can currently choose from over 1.4 million vehicle listings. The site is targeted at both car dealers and private customers. It also provides additional services such as financing, insurance and advertising.
Online buying and selling is a fast-paced business. Companies in this sector have to continually adapt and improve their virtual shopfront and back-end. Web-enabled marketplaces – especially automotive exchanges such as mobile.de – also have to contend with continually changing user behavior, which means they need to constantly monitor behavior patterns and adjust their offering accordingly.
There are many factors that influence user behavior. These may stem from the company’s own marketing campaigns but can also be linked to political and economic developments. The German Government’s 2009 scheme to offer car owners a financial incentive to scrap old cars and purchase new, more environmentally friendly vehicles is a case in point here, and one that boosted new car sales. In contrast, the current euro crisis is dampening sales of new vehicles, yet stimulating the used car business. mobile.de’s IT infrastructure is designed to factor these changing dynamics into resource planning as effectively as possible. The massive flow of information from transactions and other sources is propelling mobile.de into the big data business.
mobile.de uses proprietary applications with various back-end databases to remain flexible and adapt its website and sales conditions on demand. These databases contain a wealth of information such as offers, administrative content, click metrics and search terms used by visitors to the site. All databases are consolidated in a central warehouse, which is then used to efficiently aggregate and evaluate information.
In the past, mobile.de fed data to the data warehouse using SAP Business Objects Data Integration, an extract, transform, load (ETL) software solution. In 2011, however, the company switched to Talend, using its open source platform to integrate all data. “Our previous solution comprised the data warehouse and the ETL tool but it didn’t fit our system landscape anymore,” recalls Michael Schulz, Data Warehouse Manager at mobile.de. “It didn’t support all of the databases we needed. This meant that useful data sources were not integrated in our evaluation process and so we were unable to extract value from them. In addition, the business object system didn’t support several operating systems that we used including SUSE Linux derivatives and Debian. On top of that, we weren’t able to get the results we needed in the available timeframes.” For Michael Schulz, these were compelling reasons to replace the old ETL tool. At the end of 2010, the company began looking for a replacement technology that could meet all of its needs.
Talend’s data integration platform was short-listed. A detailed assessment soon revealed that the Talend suite was an ideal fit for mobile.de’s open-source landscape and could tick all of its requirements. A pilot installation using a prototype confirmed this. However, the company still had to tick one final box. In the past, mobile.de contracted external service providers to set up the SAP-based ETL processes. The company wanted to again call on external service providers with the new solution and thus looked around for Talend experts. Michael Schulz found that there were more service providers with experience in developing and implementing Talend-based structures than there were with the previous software.
Following a three-month test phase using the open source product Talend Open Studio for Data Integration, Michael Schulz was suitably impressed: “Talend more than met all of our expectations regarding the new data warehouse and data integration processes. The Talend system fits perfectly into our infrastructure, dovetailing seamlessly with Debian and other open-source products. We were also able to develop ETL processes easily and quickly and Talend’s extensive consulting partners network make it possible for us to obtain help at better rates than was previously possible with the existing SAP solution.”
All of which sealed the decision in favor of Talend. In future, mobile.de will be using software from Talend for business analytics. In the end, the company chose the Talend Data Integration solution. Besides first-rate support from the manufacturer, the solution has the added bonus of useful enterprise functions such as team collaboration and big data support. In summer 2011, Michael Schulz and his team started migrating existing ETL processes during a phase that lasted almost one year. The transition was completed in summer 2012, establishing Talend as the data integration standard at mobile.de. The ETL processes were redesigned and redeveloped during migration. Today, the company works with three separate data warehouses built on the original underlying data models. Retaining these structures simplified migration significantly.
The Talend deployment uses a redundant server architecture to deliver the mission-critical levels of availability that mobile.de needs. Two active Talend servers support each other. The Talend Administration Console (TAC), which manages data integration processes, runs on a third server. A team of six developers developed and adapted the data integration processes and tailored the data warehouse to the requirements of mobile.de. The entire Talend environment runs on virtual machines using VMware ESX Server virtualization software. mobile.de now also uses Talend ESB and its scalable Enterprise Service Bus (ESB) to easily integrate applications, data and systems.
Michael Schulz is full of praise for Talend following the successful migration. “The Talend suite is easy to learn and it’s fast. We are particularly impressed with how simple it is to expand the solution. This makes it very easy to connect additional systems. The analysis system provides us with far more exact information, for example, on revenue, the number of people who visit our website and what they’re interested in. And it does all this at a much faster rate. Previously, we were not able to generate these metrics as the underlying data wasn’t integrated in our old system.”
In light of these positive experiences, Michael Schulz plans to make the most of his Talend investment. The company will soon be introducing a MongoDB NoSQL database system, which, thanks to Talend, will be easily integrated into the IT landscape. mobile.de also plans to introduce big data technology, and – once again – Michael Schulz and his team will be turning to Talend. “We already had our eye on big data while planning the warehouse project. Talend technology enables us to easily transition from a conventional data warehouse solution to one that supports the integration and evaluation of Hadoop sources. By choosing Talend, we have gained a powerful, forward-looking IT solution.”