"It is a capital mistake to theorize before one has data," Sherlock Holmes had noted in
A Study in Scarlet. Data continuously aided this famous consultant detective for deductive reasoning to solve perplexing investigations. Availability and accessibility of data has helped increase the efficacy in human intuition (decision-making) in every field of science, medicine and technology. With technological advancements in storage of this large volume of data - which until recently were inaccessible for analysis - we have vast opportunity to derive at conclusions. Big Data is arriving from multiple sources at an alarming
velocity, volume and variety. And to extract meaningful value from this data, one needs optimal processing power, analytical capability and skills. In this article we will be looking at the opportunities that Big Data provides - for government agencies, private companies or social sector enterprises - (though its applications are not limited to these) - and the future of data scientists, with central focus on India's big data future. With India currently among the top 10 countries in terms of data analytics, the opportunities are limitless.
For those of you who have been living under a rock or maybe on Mars, "Big Data" simple refers to the use of predictive analysis, user behaviour analytics, etc. to examine large amounts of data to uncover hidden patterns, correlations and other insights. This analytics helps organizations and companies harness their data and use it to identify new opportunities, enabling them to develop smarter business strategies, efficient operations, and increase financial returns and customer centricity. The
UN Economic Commission for Europe's (UNECE) task team on Big Data (2013) classified this large volume of data into three categories. Human-Sourced Information - loosely structured and often ungoverned data stored everywhere from personal computers to social networks. Process-Mediated Data - structured data stored in relational database systems like the traditional business and administrative data. Machine-Generated Data - large volume of well-structured data derived from sensors and machines used to measure and record the events and situations in the physical world. To gauge the volume of this data, let us assume that each person produces 61 CD-ROMs of information. Stacking these up will touch the surface of the moon and extend to quarter of this distance beyond. But the growth in storage capacity and availability of data scientists has provided one with means to analyze these data and arrive at informed conclusions.
In his report
Big Data in Big Companies, IIA Director of Research Tom Davenport interviewed more than 50 businesses to understand how they used big data. His study summarized that companies got value through Big Data Analysis through cost-reduction, faster and better decision-making, new customer-centric products and services.
India has been making exponential growth with big data analytics being employed by government agencies and private companies, and it is slowly being imbibed by not-for-profits organizations. India's biggest auditor, the Comptroller and Auditor General (CAG), has drafted "Big Data Management Policy" for Indian audit and accounts departments in an attempt to foster the use data analysis to improve their functions. In this regard, it has inaugurated the Centre for Data Management and Analytics (CDMA) to synthesize and integrate relevant data for the auditing process. It aims to exploit the data-rich environment in the union and state governments to build capacity in the Indian audit and accounts department.
Lux Research analysts have put together real-time monitoring systems and connected platforms to integrate and deploy sensors and track different parameters to make logistics of vaccines and biological reagents safer and more efficient in India by ensuring continuous availability of data and analytical insights to optimize their services. Similarly, DISCOMS in India are capturing data from sensors installed at the last mile of power consumption to analyze along with historical power usage patterns to hypothesize preventive measures for Aggregated Technical and Commercial (AT&C) losses.
The industry is witnessing rapid growth driven by increased demand for cloud-based and predictive analytics solutions by industries such as BFSI, retail, telecom and healthcare. India has 600 data analytics firms with 100 new start-ups setup in 2015 alone. Catering to this demand would require a large supply of data scientists. According to
TeamLease Services - a staffing solutions company - by 2020, India will face a demand-supply gap of 2,00,00 data analytics professionals. The supply is short even in the US job market with only 40 out of 100 positions for data scientists being filled. To meet requirements, companies are coming up with unique programmes designed to equip graduates in mathematics, statistics, and data warehousing skills. Similarly, the National Association for Software and Services Company (NASSCOM) has proposed a curriculum upgradation to include big data and data analytics in engineering colleges. Also a Centre of Excellence was initiated to drive research in the analytics space and strengthen the ecosystem.
Currently data scientists with around 5 years' experience are earning over 75 lakh per annum as compared to 8-15 lakh for CAs and 5-8 lakh for engineers with the same experience level. With the skewed demand-supply gap, the Data Analytics industry - at a current base of $2 billion and growing at CAGR of 26 per cent - is incentivizing graduates.
The most recent and perhaps the largest entrant in the field of Big Data is Reliance Jio, unveiled recently by Reliance Industries Limited chairman Mukesh Ambani. Terming data the "oxygen of the digital era", Reliance envisions Jio as ushering in digital data abundance. With its entry-level smart phones, Jio aims to get millions of so-far left-out Indians to join the Digital India bandwagon, and allowing for a larger quantum of data to be captured, giving a better sense of customers to businesses and government programmes aimed at the public.
Although a slower entrant, not-for-profits are also steadily using more big data to increase the impact of their services and products by optimizing their finances. Akshaya Patra Foundation in Bangalore use data analytics in identifying a cost-effective solution to deliver foods to schools. Analyzing the data captured - the number of vehicles used, the time and fuel consumed on each route - enabled them in optimising their logistics and reducing the routes by five. It is therefore not a surprise that technology will play a major role in addressing the large scale social concerns of our country.
Gearing up to capture the advent of big data in optimising private businesses and public goods and services, NITI Aayog had initiated a multi-pronged approach to further the development of India's capability in Big Data. This includes incorporating skill and human resource development and developing products and platforms to compliment the rising demand. NASSCOM has anticipated India's big data industry will capture 32% of the global market to reach $16 billion by 2025 from the current level of $2 billion.
This anticipated growth and use of big data therefore has to be nurtured and enhanced to translate into financial opportunities and job creation while also making India the largest data analytics market in the world.
(Faisal Patel is a business and social entrepreneur. He is the co-founder of Axeda Analytics, a Big Data startup in India.)Disclaimer: The opinions expressed within this article are the personal opinions of the author. The facts and opinions appearing in the article do not reflect the views of NDTV and NDTV does not assume any responsibility or liability for the same.