Big Data is important.. for the SIs

Big data is a technology that helps customers analyse large amount of data in a short time. It is something that is being repeated by all vendors and deemed important to everyone. Some might have heard of Hadoop, the platform used to perform analysis of big data. However, is it that important?

First, big data technology is not new. In essence, it uses parallel processing to run queries concurrently using algorithms so that the processing of the queries will be quicker. Analysis of big data is never in real time, and is usually run in batches.

Therefore, I do not quite see why big data is suddenly so important to everyone when this technology existed for quite some time. Google has been using it for years.

Why is it important? Other than the importance of doing data analysis to find trends and patterns, it is also important because it causes customers to be more reliant on system integrations (SIs).

Why do I say that? It's quite obvious. Doing big data analysis requires 2 major works to be executed:

1. A consultancy study to analyse the type of information that needs to be mined and if it is available
2. A development project to develop the algorithms to mine the data and produce reports that is understandable to customers

Who does both works? The SIs.

Big data is important if customers are certain on what they require and they are already capturing the required data. If customers just want to jump on the big data wagon just because everyone is talking about it, I think the better suggestion is to re-analyse the business processes to find out what gives, or will give the business the competitive advantage.

After that, then look at if big data can help you.