How big is the Big Data monster?

Posted on February 2, 2012 by Anaïs LERMA 

Big Data seems to be the IN word of this new year according to the Google trends tool.

Yet, only 40% of the 122 respondents in an online survey conducted by The Register state they have a clear understanding of what ‘Big Data’ means.

This is undoubtedly due to the lack of an agreed definition.

For some people, Big Data refers to the explosion of information. For others, it describes the technology and tools used to analyse large volumes of data.

For us, Big Data is a phenomenon caused by the multiplicity of data sources, the development of mobile devices and the rise of social media.

The following figures highlight the increase in data volumes:

Data created :
• 130 exabytes of data in 2005
• 2,720 exabytes in 2012
• 7,910 exabytes in 2015 (according to IDC)

Social Media
• 200 million tweets sent per day
• 30 billion documents added to Facebook each month
• 3 million videos watched on YouTube everyday
• 180 million blogs
• 135 million LinkedIn accounts

Attributes

In 2001, Dough Laney, Gartner Analyst described the challenge of information explosion as a three-dimensional phenomenon : Volume, Variety and Velocity.

• Volume: As the volume of data is enormous, it is difficult to capture, store and analyse. The tools and applications used to store and analyse data are not suited to that kind of volume. You don’t need zettabytes of data to meet the Big Data Monster. As well as the volume, data heterogeneity and streaming speed are two features to take into account.

• Variety: Data comes from multiple sources: emails, social media, databases, folders, internal applications, and also from the cloud. Different sources and different formats therefore often lead to compatibility issues.

• Velocity: New data is constantly being created. For Big Data, Velocity reprensents the need to ensure that data created and data modified on the Internet and within the company is instantly available. Indeed, Big Data analytics can make a business more agile, as long as the information is regularly updated.

Speaking of velocity, the slightest problem with data availability can have serious financial consequences. PolySpot Information At Work is based on an SOA, distributed architecture (distribution, replication and sharding), to ensure that the solutions offer a high level of availability.

In companies, the explosion of data can be an opportunity as well as a threat. A study conducted by Mindjet reveals that an employee wastes 21 minutes per day looking for information. These 21 minutes represent €1,500 per annum for each employee. “Data presents a double-edged sword these days. Companies can be either underprepared without big data or overloaded with too much data and no solution”, said Stephen E. Arnold.

We are drowning in data and we are no longer able to find the information we need, even when we know that it is indeed available.

How are companies coming to terms with data volume, variety and velocity?

Specialised solutions are emerging to store, manage, access, transform and index large volumes of data.

As Stephen E. Arnold observed : “When integrating multitudes of data types for analysis, a company needs a solution capable of evolution because ROI is the main goal.”

PolySpot and its cross-functional search infrastructure allow companies to provide a single point of access to all data, via which users can instantly interact with all available information resources, both inside and outside the company, and regardless of whether the data they contain is structured or not.

PolySpot Information At Work is different from standard search solutions which cannot scale to accomodate ever increasing volumes of data, it can adapt without overhauling existing servers, network devices, and other system components. PolySpot enables users to develop an innovative search environment capable of handling over-indexing.

No related posts.

Comments