Sony Online Entertainment leverages Talend for its gaming analytics

Talend Integration Suite processes analysis data for newest Massively Multiplayer Online game.

The entry barrier is very low. Anyone with any mainstream database or Java experience can download Talend and start using it immediately. We got up to speed quickly and started team collaboration across the company. The transition to Talend Integration Suite-the subscription product-was seamless. Our staff is technical and Talend is a great fit for us because it can be customized as we need.

Anand Rao,
Director of Business Intelligence

Global entertainment pioneer

Sony Online Entertainment LLC. (SOE) is a recognized worldwide leader in game development and online gaming with millions of gamers around the world. Based in San Diego, SOE is a division of Sony and is best known for creating massively multiplayer online (MMO) games including EverQuest, EverQuest II, PlanetSide, Star Wars Galaxies, and Vanguard: Saga of Heroes. SOE creates, develops and provides compelling entertainment for the personal computer, online, game console, and wireless markets.

In 1999, SOE introduced EverQuest, the genre-defining online video game. In the spring of 2009, SOE launched Free Realms, a new game which brings to life a whimsical virtual world where gamers can explore, compete, raise pets, and socialize in a safe, online, family-friendly environment. Players are able to choose from jobs such as adventurer, ninja, mailman, medic, blacksmith, miner, warrior, archer, treasure hunter, wizard, kart driver, chef, brawler, explorer and switch on the fly without creating a new character. Regardless of level or class, players are able to interact, join forces and play together. Because of the social networking nature of the game, players can send links, share images and videos, and chat with friends from their own, personal, customizable social networking page, which is all optional for parents that want to restrict their child's access to user-generated content.

With a starting a membership rate of $4.99/month, Free Realms features Station Cash, a new virtual currency that players can buy to fund a virtual wallet. Once a wallet has been funded-a simple process because the wallet accesses the current credit card on the player's account-Station Cash is used to purchase premium in-game items, known as microstransactions, through the in-game marketplace.

Talend Integration Suite takes gaming to a new level

Free Realms is the first Massively Multiplayer Online game developed on open source technologies. “For the web and game databases, we use Enterprise DB as the database engine and Talend for data integration,” says Anand Rao, SOE's Director of Business Intelligence. “Both of these open source solutions boast vibrant communities and large download numbers. The native support for Enterprise DB in Talend and the close partnership between the vendors was an important factor for our decision.”

Until recently SOE depended on in-house custom development to manage their games. Data was collected and analyzed by hand-written code. The performance wasn't adequate and it was hard to maintain. Developers had to develop scripts to analyze each and every subscriber and categorize them by transaction amount and frequency. The process required a lot of raw data and a huge amount of slicing and dicing. Typically, it was taking days to write code and getting the data back. Talend Integration Suite cut this time down to a few hours.

Talend Integration Suite consumes the messages from Apache ActiveMQ to load game data into a database staging table where it is used for BI reporting. The actual process is highly parallelized and can copy the data from 50 source tables to the reporting database simultaneously. Using Oracle's Golden Gate Replication, change data capture (CDC) solution; Talend reads the changes from the OLTP system and processes them into the data warehouse. At present, Talend collects the data at predefined intervals, but the next level of evolution will be to process real-time feeds.

Low learning curve and ease of use

“The entry barrier is very low,” says Anand Rao. "Anyone with any mainstream database or Java experience can download Talend and start using it immediately. We got up to speed very quickly and started team collaboration across the company. The transition to Talend Integration Suite-the subscription product-was seamless. Our staff is technical and Talend is a great fit for us because it can be customized and optimized as we need.”

 

Talend is a leading open source software vendor that makes powerful data integration and application integration solutions accessible to organizations of all sizes. For data integration, Talend offers an easy-to-use ETL tool that makes the development of data extraction, data transformation, and data loading processes faster and more reliable, as well as flexibly supporting a wide range of data management operations such as database conversion and data synchronization. Talend's open source data quality management solution features a powerful data profiler as well as advanced data cleaning and data enrichment functionality. For enterprise application integration, Talend extends proven Apache open source technologies like Apache CXF and Camel to provide high-performance, modular solutions that enable organizations to derive more business value from their distributed systems and applications.
Apache, the Apache feather logo, and all Apache-related trademarks are the property of the Apache Software Foundation, and are used with permission. Click here for more information.