ScoreMD Upgrades Customer Data Using Talend Data Integration

The ASP Data Mining specialist integrates large volumes of heterogeneous data using Talend’s open source integration solution to evaluate the investment capacity of online shoppers.
The solution selected had to fit into this type of architecture and the fact that Talend Data Integration is Cloud-ready significantly accelerated our deployment.
Vincent Teyssier, BI Architect at ScoreMD

Optimize marketing operations to increase return on investment

ScoreMD, founded in 2009, provides ASP Data Mining services. The company’s clients include, Weekendesk, Relais & Châteaux, Florajet, Geny and Marketshot. With its PilotROI and Customer Insight solutions, ScoreMD helps them maximize their marketing operations to increase their return on investment. The company combines data from advertisers with behavioral and transactional data – some of which are derived from email campaigns, promotions for example – to calculate an appetence score, which defines the preferences of online shoppers and their desire to use or purchase. Advertisers use this score to adjust the commercial pressure on each individual (Send messages or not? By telephone? By SMS? etc.).

Integrate large volumes of heterogeneous data

ScoreMD integrates a multitude of heterogeneous data, originating from different customer and partner technical platforms. Each data set uses different nomenclature and information formats.

“We hope to industrialize the data transformation and loading processes, while easing the load that their management requires. Data integration represents approximately one third of our services, therefore, since our inception, it has been crucial to consider a system that enables us to preserve our efficacy as the amount of volumes processed increases,” explains Vincent Teyssier, BI Architect at ScoreMD.  “I had a good handle on Talend Open Studio for Data Integration and I even participated in the plug-in development.  Also, the team had a good knowledge of Java so it was easy to quickly become familiar with the tool.”

The financial aspect was also considered in the final selection. “With equivalent functional coverage, Talend’s solutions are clearly less expensive than other solutions on the market. We conducted tests and considered the breadth of Talend’s transformation library, which was very well-developed over time; this is not the case with other offers of the same type,” confirms Vincent Teyssier. “We finally chose Talend Data Integration to benefit on the one hand from the support services guaranteeing the vendor’s responsiveness and on the other hand, richer features, particularly for collaboration.”

Data processing provided via Web Services in the future

Today, ScoreMD is one of the only companies on the market that is able to provide in production service via both Cloud (Amazon Web Services) and a classic information-management service, the latter being used for companies who do not wish to use Cloud. The Cloud service uses a partition of Amazon’s European infrastructure (Virtual Private Cloud) and offers a “user” invoicing method via a monthly subscription that uses an orchestrator to manage instances according to the load (scale in, scale out, go, stop). Talend’s servers only operate for the time strictly necessary for the data integration operations; the charge invoiced corresponding to one quarter of the cost of using a traditional server. This system promotes rapid scalability, thanks to Amazon’s virtually unlimited extension capacity.

“The solution selected had to fit into this type of architecture and the fact that Talend Data Integration is Cloud-ready significantly accelerated our deployment,” adds Vincent Teyssier.

Using Talend Data Integration, ScoreMD loads the data into a PostgreSQL database and converts this data into ODS format (Operational Data Store). The data is then analyzed by three experts using Data Mining tools (Rapid Mining, R, SAS, etc.). The Data Mining algorithms are triggered by Talend  Data Integration when the loading and converting jobs are finalized. Once the appetence scores are calculated, Talend Data Integration retrieves the data, converts it into the original format and sends it to the client (in flat file or XML format). 

“Ultimately, we will develop Web Services which will simplify exchanges with our clients, reduce their costs and perfectly fit into their SOA initiatives,” notes Vincent Teyssier. 

Meanwhile, the loading system for reporting and campaign operation control data marts also rely on Talend Data Integration Suite.

Flexibility, minimizing costs and time to value

Aside from the cost issue, vital for a company in full development, ScoreMD appreciates the flexibility of the Talend solution, its ease of customization and its integration with other tools on the market, like the Jaspersoft tools that are used for reporting.

“In an industry like ours, flexibility is paramount.  We saw it without our Cloud offering, but more generally, all of the solutions that we use must be able to be modified quickly to respond to client requests without the need for expensive, specialized expertise, as is the case with proprietary vendors. Talend’s customization features respond to all requests and the ease of use and deployment make the tool immediately usable. We benefit from increased “time to value” which allows us to fully concentrate on the heart of our business. The flexibility of Talend Data Integration is a direct benefit that reduces the time spent on developments and unprecedented challenges that are quickly resolved,” adds Vincent Teyssier.

This flexibility is also enhanced by the simplicity of developing complementary components. “We created two Talend geocoding components (leveraging Google Maps) to quickly respond to client requests. These developments have helped enhance our offering without excessive additional costs thanks to our use of Java.  The reuse and industrialization of these components/processes is really easy,” continuesVincent Teyssier.

Finally, ScoreMD emphasizes the responsiveness and quality of Talend’s support service, which accompanied the team during the implementation and development of components.

With an eye on the future of its business, the company is currently implementing Hadoop (MapReduce) to simplify the development of applications that handle large volumes of data.  “This is consistent with Talend’s recent announcements about Hadoop’s support.  Some of our Data Mining jobs use this framework and we are now studying the deployment of these new Hadoop support features, keeping in mind the volumes that we manage and the evolution of our business,” concludes Vincent Teyssier. “Meanwhile, next year we will focus on data quality issues and concentrate on the opportunity of offering our clients a dedicated service. With Talend, we found a reliable partner that helps us offer added value to our customers.”