Getting a Grip on Big Data with Talend

Emesa Nederland BV goes for Talend Data Integration.
We have always rated open source solutions and value, in particular, the intelligence within the community. We firmly believe that open source combined with professional support clearly outranks conventional software models.
Andre Buren, Chief Technology Officer (CTO)

Overview

Emesa Nederland BV is the Netherland’s leading e-commerce company. It provides various travel and leisure platforms where customers can search for, compare and book everything from holidays through concerts to beauty, spa and wellness packages. Emesa was named the Netherland´s website of the year in 2009 (inpakkenenwegwezen.nl) and in 2010 (vakantieveilingen.nl). The company has also been operating the online platform aladoo.de in Germany since 2011.

Every week in the Netherlands, over five million subscribers receive Emesa’s e-newsletter. Over 95 million users visit the company’s websites each year. In 2011, three million consumers completed transactions via Emesa, generating over 125 million euros in revenue for the company. Over the past five years, the company has seen its revenue rise by around 3,500 percent. At Emesa Nederland in Amsterdam, over 140 employees are dedicated to ensuring that travel customers can look forward to a unique online experience with Emesa. Headquartered in Cologne, Germany, Emesa Deutschland plans to increase headcount to around 45 by the end of the year.

Challenge

In Germany, users can register free of charge with the aladoo.de website. This gives them immediate access to a huge selection of leisure packages and activities – with everything from hotel and holiday home reservations through weekend stays at beauty farms and spa resorts to tickets for sporting events, concerts and amusement parks. Emesa’s unique auction concept sets it apart from all other providers, enabling consumers to bid for the product they want at the price they are willing to pay. Every auction starts at 0 euros. At the end of the auction, the person with the winning bid transfers the money directly to Aladoo. Unlike other providers, Aladoo is not acting as an agent and does not charge a commission. Aladoo is always the consumer’s direct contract partner and shares all financial responsibility with its cooperation partners.

Emesa’s IT infrastructure is built around an auction platform that the company developed in 2010. Even with a massive stream of data, this platform is capable of reliably handling transactions. Completed transactions are fed into an ERP system from Dutch software provider Afas. And an open-source solution from SugarCRM is used to manage customer data. These three systems all run on MySQL databases. 300,000 users and tens of thousands of auctions every day generate huge amounts of data in this interaction-intensive environment.

This big data stream is stored in a central warehouse where it can be accessed by the applications. The data warehouse is based on a database from Greenplum, a division of EMC. It currently holds over one terabyte of data and this figure is rising rapidly. Emesa’s company-wide reporting system – built on the business intelligence solution Qlikview – also draws on this data.

“Our platform brings providers and customers together – the simple, transparent way,” explains Emesa CTO Andre Buren. “Our slogan ‘Just Enjoy’, underscores our commitment to offering a unique shopping experience where simplicity is the golden rule. This simplicity, however, comes at the price of an extremely complex IT landscape. Our auction platform is the exchange at the heart of this landscape. And we have to process the big data generated by this exchange more or less in real time.”

Solution

“Our challenge was to find a solution that could exchange data between the data warehouse and MySQL databases on the one hand, and the Qlikview solution on the other – and operate reliably under extreme conditions,” recalls Arno Pont, Managing Director of WeDoBi, a systems house that specializes in business intelligence projects in the Netherlands. “Emesa was experiencing difficulties with Qlikview reporting in particular, as the software could not cope with the volume of data and the frequent updates to the data warehouse.”

An open-source enthusiast, Emesa uses the complete LAMP stack comprising Linux (operating system), Apache (web server), MySQL (database system) and PHP. “We have always rated open source solutions and value, in particular, the intelligence within the community. We firmly believe that open source combined with professional support clearly outranks conventional software models,” details Andre Buren. In light of this commitment to the open-source model, the consultants at WeDoBI recommended the Talend integration platform.

For the initial test phase, the company installed the free community version, Talend Open Studio for Data Integration, and put it through its paces under real-life conditions. Emesa then purchased the commercial version, Talend Data Integration, for its production environment. This version contains additional high-performance functions plus professional customer support from the vendor. Emesa currently has an enterprise license and several team licenses from Talend. As the central data integration platform, the Talend solution controls data exchange between the data warehouse, the MySQL databases supporting the individual applications and the Qlikview BI solution. The platform enables a bi-directional data flow. On the one hand, it retrieves data from the data warehouse for the applications and loads it to the individual databases. On the other, it collects data generated through transactions and customer interactions from the MySQL databases, converts it into a uniform format and loads it to the data warehouse. The platform has proven that it can comfortably handle the extreme requirements of big data environments.

Benefits

“We need performance, flexibility and scalability – and these are the hallmarks of Talend,” summarizes Andre Buren. “Twenty of our developers work with the Talend software. We are particularly impressed with the user-friendly graphical user interface and the modular, component-based architecture – we know that the solution will grow along with our business.” Emesa is currently testing the new big data version of the Talend platform, for example, and assessing additional data quality functionality.

The new, powerful data integration platform enables Emesa to launch targeted marketing campaigns more frequently, for example, offers aimed at older audiences or focusing on specific regions. The entire infrastructure is scalable and can thus accommodate any plans for future expansion. “We are already the clear market leader in the Netherlands. Now, we are looking to consolidate our foothold in Germany, and after that, we have our sights firmly set on the European market. Our platform is designed from the ground up to support multiple countries and languages – so from a technical perspective, there are no limits to what we can achieve,” explains Andre Buren. “Talend plays a enabling role in our infrastructure and we intend to build on this in the future.”