🔖 Topic- 10: What is BIG DATA? Introduction, Types, Characteristics & Example

What is Data?

The quantities, characters, or symbols on which operations are performed by a computer, which may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical recording media.

Examples Of Big Data

Following are some examples of Big Data-

The New York Stock Exchange generates about one terabyte of new trade data per day.

Social Media

The statistic shows that 500+terabytes of new data get ingested into the databases of social media site Facebook, every day. This data is mainly generated in terms of photo and video uploads, message exchanges, putting comments etc.

A single Jet engine can generate 10+terabytes of data in 30 minutes of flight time. With many thousand flights per day, generation of data reaches up to many Petabytes.

Types Of Big Data

BigData' could be found in three forms:

1. Structured
2. Unstructured
3. Semi-structured

Structured

Any data that can be stored, accessed and processed in the form of fixed format is termed as a 'structured' data. Over the period of time, talent in computer science has achieved greater success in developing techniques for working with such kind of data (where the format is well known in advance) and also deriving value out of it. However, nowadays, we are foreseeing issues when a size of such data grows to a huge extent, typical sizes are being in the rage of multiple zettabytes.

Do you know? 10²¹ bytes equal to 1 zettabyte or one billion terabytes forms a zettabyte.

Looking at these figures one can easily understand why the name Big Data is given and imagine the challenges involved in its storage and processing.

Do you know? Data stored in a relational database management system is one example of a 'structured' data.

Examples Of Structured Data

An 'Employee' table in a database is an example of Structured Data

Unstructured

Any data with unknown form or the structure is classified as unstructured data. In addition to the size being huge, un-structured data poses multiple challenges in terms of its processing for deriving value out of it. A typical example of unstructured data is a heterogeneous data source containing a combination of simple text files, images, videos etc. Now day organizations have wealth of data available with them but unfortunately, they don't know how to derive value out of it since this data is in its raw form or unstructured format.

Examples Of Un-structured Data

The output returned by 'Google Search'

 Semi-structured

Semi-structured data can contain both the forms of data. We can see semi-structured data as a structured in form but it is actually not defined with e.g. a table definition in relational DBMS. Example of semi-structured data is a data represented in an XML file.

Examples Of Semi-structured Data

Personal data stored in an XML file-

<rec><name>Prashant Rao</name><sex>Male</sex><age>35</age></rec>
<rec><name>Seema R.</name><sex>Female</sex><age>41</age></rec>
<rec><name>Satish Mane</name><sex>Male</sex><age>29</age></rec>
<rec><name>Subrato Roy</name><sex>Male</sex><age>26</age></rec>
<rec><name>Jeremiah J.</name><sex>Male</sex><age>35</age></rec>

Data Growth over the years

 Please note that web application data, which is unstructured, consists of log files, transaction history files etc. OLTP systems are built to work with structured data wherein data is stored in relations (tables).

Characteristics Of Big Data

1. Volume:

When we talk about Big data, probably volume is the very first criteria for consideration. The range of volume justifies whether it should be considered as ‘big’ or not. Usually, if the volume of data is above gigabytes then only it is considered as big data from a volume perspective. What does measurement signifies here? It could be petabytes, terabytes, Exabyte. This volume amount is considered based on data surveys of different organizations and here are some of the examples:

Also, this is actually the purpose of differentiating such enormous size of data as Big data from traditional structured data. In addition to that, RDBMS, or traditional database systems are not efficient to process or handle this data. Because it will take extended query time, cost, reliability, etc.

Also, as per IDC estimation by 2020, business transactions on the internet for B2B and B2C will reach 450 billion per day.

2. Velocity:

Stream analytics is a popular term today where high-speed data is processed using tools. But do you know stream analytics associated with which characteristics of big data? No doubt, it is the velocity of data. Here velocity means data generation speed, how frequent it is delivered and analyzed.

Now, the amount of data generated in today’s scenario is massive. Most importantly it needs real-time processing for analysis purpose. For example, Google alone generates more than 40k search queries per second. Hence, we can imagine how fast processing is required to get insights from data.

3. Variety:

Big data deals with any data formats – structured, unstructured, semi-structured or even very complex structured. So, storing and processing unformatted data through RDBMS is not easy. However, such unstructured data provides more valuable insights on the information which we rarely get from structured data. Besides, a variety of data means different data sources. So, this characteristic of big data also provides information on the data sources.

4. Veracity:

Not that all data that come for processing are valuable. So, unless the data is cleansed correctly, it is not wise to store or process complete data. Specially when the volume is such massive. There comes this dimension of big data – veracity. This particular characteristics also helps to know whether the data is coming from a reliable source or it is the right fit for the analytic model.

5. Variability:

In Big data analysis data inconsistency is a common scenario which arises as the data is sourced from different sources. Besides, it contain different data types. Hence, to get meaningful data out of that enormous amount of data anomaly and outlier detection are essential. So, variability is considered as one of the characteristics of big data.

6. Value:

The primary interest for big data is probably for its business value. Perhaps this is the most crucial characteristic of big data. Because unless you get any business insights out of it, there is no meaning of other characteristics of big data.

Source:
https://www.guru99.com/what-is-big-data.html
https://www.motivaction.nl/en/news/blog/big-data-the-6-vs-you-need-to-look-at-for-important-insights