Don’t Cast Away your Big Data

July 18, 2013 --

In the movie Cast Away, Tom Hanks finds himself stranded on an island in the middle of the ocean, and he needs to be very creative before he manages to return to the civilized world (this is actually a good movie, apologies for making it sound so cheesy – it isn’t).

Don’t let your big data projects be cast-aways. They don’t deserve it! Build your first Hadoop cluster in a sandbox (the one on the beach of the island…) if you’re really worried that you don’t know what you’re doing. But as soon as it works, and you’ve proven that you can load and process big data in Hadoop, bring that cluster back to the mainland.

For big data projects to really bring value, they need to be an integral part of the overall IT infrastructure and information architecture. They need access to data, and that data resides in other systems, so you actually need to connect big data applications to other systems, upstream and downstream. Big data must also become part of your IT and information governance policy.

There is really no point in viewing big data projects as isolated from the “rest” of IT. They probably deserve special attention, and dedicated skills. But not to be cast away, far from their peers.