Edited By Sreeparna Sengupta
Image courtsey : http://www.isymmetry.com/assets/uploads/2015/02/big-data-2.jpg
Many of us seem to believe to have knowledge on how ‘Big Data analytics’ work- but how many really understand the underlying technology, its benefits and futuristic approach? Some say it has come into picture just few years back, but the real fact suggests that firms like GE, IBMhave already taken leaps in this field. An important survey conducted by the Information Week last year was an eye opener for many organizations towards their baby steps on understanding Big Data. Only 30% of respondents from across the globe say they are very or extremely effective at identifying critical data and are using it to make decisions — that’s down from 42% in 2013. A whopping 63% say they are only moderate to slightly effective, with the remaining 7% give up and claim defeat. The lack of knowledge many have claimed in critical business lead to losses in terms of high financial crisis causing many startups to bring their shutters down. But the real question lies on whether just a scrappy research and analysis can really help the firms to bring out the true potential of their or their client’s data. The various significant aspects of making crucial decisions by using analytics on the humongous data present on the web lies on few critical knowledge areas.
The primary technological aspect of Big Data is to understand the underlying architecture of the platform for analytics. The structured or unstructured data can be stored in the distributed system which can be accessed using query language. Google in 2004 published about MapReduce architecture which was further taken to the next step by open source project from Apache called as Hadoop. To improve upon the performance of the system for faster query access using parallel processing Apache came up with different projects as Ambari, Hbase, Hive, Spark and Pig and recently Zeppelin and Flink for advanced analytics. The system have aHDFS (Hadoop Distributed File System) which has the distributed node for managing huge data sets. The underlying language doesn’t effect the overall processing of the data as traditional languages like python and java have been the backbone for the systems.
Though the technological landscape for Big Data is still considered to be in a very premature stage with lots of development happening around the globe, the empirical knowledge about the applicability of the data has to be understood by the firms. The various use cases of Big Data for firms like financial giants, manufacturing titans and other domain leaders have to be learned and analyzed to bring out true value for the customers. Business analysts who can analyze the critical data can pull logical sense from it and help to unleash the true potential of the raw data. Various real life use cases like credit card offers, dining offers, shopping offers, etc. by various financial and e-commerce players can throw ample light on how the raw data from various sources can make rational sense to enhance the businesses. The next question arises on from where the data can be pulled? The answer to this lies on our day to day activities on social media, email conversations, banking transactions and various other online sources via which we are connected. Some of the data are obtained using collaborative ventures like financial firms with credit data, the apps which are used in social media to pull personal information, survey results, marketing tools and various other platform in the form of raw data. The massive data from various data sources if properly collated and analyzed by an analytical tool can throw light on the KPIs which are used to measure the performance of the firm, or the business itself.
Though the foundation has been modest, Big Data’s future which is the IoT(Internet of Things) suggests on how the overall internet through web can be connected to make our lives much easier. Countlessphilosophies like connected smart homes, smart parking, smart lighting, waste management, etc. have been the visualization of many global firms. The idea behind it lies in simple sensors which can detect the various fluctuation in regular pattern or activity to convert it into raw data for further providing analytics. The true idea of Big Data thus lies in the modest fact of managing and analyzing huge amount of data around us which can be useful in not just putting into picture for a better living but for a better future altogether. Though the discussion on Big Data seems to be exhaustive and haywire, making sense from the data remains the master key for the lock.
