I had the chance to attend Apache Spark Summit last week in Amsterdam and was lucky enough to meet Matei Zaharia in person (he is my MIT Big Data online course assistant professor). Matei is indeed a modern rock star for being part of the Spark inception team. It was his experience trying to solve machine learning problems with Hadoop that led him and his teammates to create Spark.
During my trip to Amsterdam, I watched a BBC channel documentary about the mysteries X Tombs in ancient Rome and how an entire population got decimated (fun evening of entertainment!).
They asked a DNA expert and palaeogeneticist to help find out the exact disease that caused the death of these people. So they took some samples from the teeth in the remains and went fishing within the various DNA present (including those of the disease).
The problem is it will take weeks of computer calculations to decode the DNA and process all the information. In contrast, imagine if they had access to modern technologies like Hadoop + Spark and how quickly they’ll be able to find the answer.
With today’s modern technologies: Super computing power, lower costs, real-time and in-memory processing, and machine learning, scientists can not only discover drugs quicker to cure people, but they can also help us understand history and solve some of the world’s biggest mysteries faster than ever before.
Having access to more information at the speed of thought can help organizations solve some of the world’s biggest problems more quickly: hunger, global warming, resource scarcity….
Real time Big Data can lead to breakthrough innovations at speed, and help us better understand our past to build a better future.
Are you ready to improve our future and become a rock star like Matei? You too can solve world’s biggest problems and innovate by using Spark even without advanced knowledge in Spark programming, simply download our free Sandbox and get your feet wet.