Talend, the global integration software leader, today announced survey results that reveal a disconnect between big data’s popularity and actual big data implementations.
In the Talend-sponsored survey, less than a quarter of respondents (24%) said there was no interest in big data within their organization – a huge drop from a similar survey carried out less than a year ago, where 61% reported having no interest. At the same time, the proportion of respondents engaged in a preliminary discussion about using a big data approach has increased from 24% to 36% over the same time period.
Despite this clear growing enthusiasm, a significant percentage of organizations have not taken practical steps needed to roll-out a big data strategy. While 19% of those surveyed (up from 8% in the previous year) were at the planning and appraisal stage, the number of big data pilots remained static at 4%. Only 10% of respondents in the latest survey are engaged in a large scale roll-out - up from 2% last year, but still a relatively low figure.
“It is encouraging that the number of businesses rolling out big data strategies has increased, but overall adoption of big data strategies remains slow,” says Yves de Montcheuil, vice president, marketing, Talend. “There is still a significant gap between those businesses expressing an interest and those taking the plunge and actually implementing the approach. It is a gap that the industry needs to address and close if the promise of big data is to be fulfilled.”
Currently, significant barriers still exist to the prevalence of big data project roll-outs. Two of the top three constraints are budgetary restrictions and skills shortages, often identified as key barriers to any IT endeavor.
The potential scale of most big data projects explains the financial concerns, especially given the on-going downturn. Also, skills are key because big data requires people to be able to integrate any number of large and inflexible disparate data sources and skilled data scientists to analyze the collective data streams.
“Contrary to popular belief, big data projects do not have to come with a massive price tag,” adds de Montcheuil. “Being able to run open source databases and integration tools over an open-source Hadoop platform allows big data integration and analysis applications to be run cost-effectively across commodity server clusters, thereby reducing hardware spend.”
The report argues that tools capable of exploiting Hadoop’s MapReduce framework are available today, breaking the vendor lock-in of proprietary ETL engines and escalating license fees. Recently-developed graphical tools can also be used by Java programmers, which eliminate the need for rare and expensive data scientists.
Alternatively, a range of factors are positively driving big data adoption. For 24% of respondents to the survey, it is the increasing volume of data that is driving uptake. For 19% of the respondents, it is the business requirement to increase revenue. Pressures of compliance and product and service development motivate 11% each; 6% say big data in their organization is driven by the requirement to find new customers, keep up with competitors or collaborate more closely with business partners.
“The long-term future for big data adoption by businesses is extremely positive, based on the technology in place, the key drivers likely to become stronger over time and growing levels of interest among businesses,” says de Montcheuil. “In the short term, organizations need to address the barriers and the issues holding many of them back – and once again technology can be the key factor in streamlining the process and making it more effective to implement big data strategies.”