Sony Online Entertainment Leverages Talend for its Real-Time Gaming Analytics

Talend Data Integration processes analysis data in real time, boosts in-game revenue for newest Massively Multiplayer Online game.
The entry barrier is very low. Anyone with any mainstream database or Java experience can download Talend and start using it immediately. We got up to speed very quickly and started team collaboration across the company. The transition to Talend Data Integration - the subscription product - was seamless. Our staff is technical and Talend is a great fit for us because it can be customized and optimized as we need.
Anand Rao, Director of Business Intelligence

Global entertainment pioneer

Sony Online Entertainment LLC (SOE) is a recognized worldwide leader in game development and online gaming with millions of gamers around the world. Based in San Diego, SOE is a division of Sony and is best known for creating massively multiplayer online (MMO) games including EverQuest, EverQuest II, PlanetSide, Star Wars Galaxies, and Vanguard: Saga of Heroes. SOE creates, develops and provides compelling entertainment for the personal computer, online, game console, and wireless markets.

In 1999, SOE introduced EverQuest, the genre-defining online video game. In the spring of 2009, SOE launched Free Realms, a new game which brings to life a whimsical virtual world where gamers can explore, compete, raise pets, and socialize in a safe, online, family-friendly environment. Players are able to choose from jobs such as adventurer, ninja, mailman, medic, blacksmith, miner, warrior, archer, treasure hunter, wizard, kart driver, chef, brawler, explorer and switch on the fly without creating a new character. Regardless of level or class, players are able to interact, join forces and play together. Because of the social networking nature of the game, players can send links, share images and videos, and chat with friends from their own, personal, customizable social networking page, which is all optional for parents that want to restrict their child's access to user-generated content.

With a starting membership rate of $4.99/month, Free Realms features Station Cash, a new virtual currency that players can buy to fund a virtual wallet. Once a wallet has been funded - a simple process because the wallet accesses the current credit card on the player's account - Station Cash is used to purchase premium in-game items, known as microstransactions, through the in-game marketplace.

Talend Data Integration takes gaming to a new level

Free Realms is the first Massively Multiplayer Online game developed on open source technologies. "For the web and game databases, we use Enterprise DB as the database engine and Talend for data integration,"€ said Anand Rao, SOE's Director of Business Intelligence. "€œBoth of these open source solutions boast vibrant communities and large download numbers. The native support for Enterprise DB in Talend and the close partnership between the vendors was an important factor in our decision."

Until recently SOE depended on in-house custom development to manage their games. Data was collected and analyzed by hand-written code. The performance wasn't adequate and it was hard to maintain. Developers had to develop scripts to analyze each and every subscriber and categorize them by transaction amount and frequency. The process required a lot of raw data and a huge amount of slicing and dicing. Typically, it was taking days to write code and get the data back. Talend  Data Integration cut this time down to a few hours.

Talend Data Integration consumes the messages from Apache ActiveMQ to load game data into a database staging table where it is used for BI reporting. The actual process is highly parallelized and can copy the data from 50 source tables to the reporting database simultaneously. Using Oracle's Golden Gate Replication, change data capture (CDC) solution, Talend reads the changes from the OLTP system and processes them into the data warehouse. At present, Talend collects the data at predefined intervals, but the next level of evolution will be to process real-time feeds.

Low learning curve and ease of use

"The entry barrier is very low,"€ said Anand Rao. "€œAnyone with any mainstream database or Java experience can download Talend and start using it immediately. We got up to speed very quickly and started team collaboration across the company. The transition to Talend Data Integration - the subscription product - was seamless. Our staff is technical and Talend is a great fit for us because it can be customized and optimized as we need."