To operate the Blended Learning Center(BLC) at optimal level, maintenance will be performed every day at 8:30 AM and at 5:00 PM regularly which can take up to 30 minutes. Please consider scheduling your activity in the BLC platform accordingly.
Topic outline
-
Dear Students
Welcome to the Big Data and IoT Theory (CSE 412) & Lab (CSE 413) courses, I, Nasima Islam Bithi will be your co-pilot in this online journey of learning.
I care about your success in these courses. I'm glad you are here.
Nasima Islam Bithi
Lecturer,
Department of Computer Science and Engineering,
Daffodil International University
Basic Information:
Course Code: CSE 412/413
Course Title: Big Data and IoT
Program: B.Sc in Computer Science and Engineering
Faculty: Science and Information Technology
Semester: Fall; Year: 2023
Credit: Theory (1Cr) and Lab (3Cr);
Course Category: Core Engineering
Course Instructor:
Nasima Islam Bithi
Lecturer
Office: Room # 505, AB04 Building, Daffodil
Smart City
Cell Number# +880 1629665310
Email: bithi.cse@diu.edu.bd
Big Data refers to a massive set of data that no
conventional data management tool can handle. Big Data is therefore a
concept that allows access to gigantic databases in real time. Big Data's main objectives are to
improve a company's or system's responsiveness to a large amount of data
collected, increase productivity and refine knowledge of customer
behavior, so that it can offer personalized offers or advertisements and
create new trends.
The Internet of Things (IoT) is a concept that connects physical or
virtual objects to the internet. The technology very often used is the
sensor, allowing to link a physical object such as a watch, a drone or
even a speaker, to the internet. If for a long time the few objects
connected to the Internet were the telephone and the computer, this is
no longer the case today and every year new types of objects
incorporating IOT technology are born. IOT is one of the greatest technological revolutions of our era and its
potential for exploitation is immense. IOT could have a huge impact on
the cars of the future or on the new versions of smart-cities,
an urban space connected to the Internet, thus significantly improving
the lives of users, while reducing the negative impact of these on the
planet.
-
O1
Introduction with the students
O2
Introduction to Data and Information
O3
Small Data and Big Data
O5
Big Data Characteristics
O1
Learn about data and information
O2
Learn about many types of data
O3
Explore several characteristics of big data
01
Introduction to Big Data
02
Class Recorded Video 1 (Theory)
03
Class Recorded Video 1 (Theory)
-
O1
Introduction to Big Data
O2
Introduction to Python
O3
Introduction to Apache Spark
O2
Work with basic python programming
01
Install Pycharm on Ubuntu
02
Install Pycharm on Windows
03
Install Apache Spark on Ubuntu
04
Install Apache Spark on Windows
05
Install Apache Spark + Python = PySpark with Jupyter Notebook on Windows
-
O1
Why big data analytics?
O2
What is big data analytics?
O3
Lifecycle of big data analytics
O4
Types of big data analytics
O5
Tools used in big data analytics
O1
Learn about the big data analytics
O2
Learn about the application of big data analytics.
O3
Learn about the big data application domains
01
Big Data Analytics Slide
- Topic 4
Topic 4
Lab: Installtion of PySpark
O1
Installation of PySpark
O2
Introduction to PySpark
O1
Complete Setup of PySpark
O2
Knowledge about PySpark programming
01
PySpark with Jupyter Notebook on Ubuntu
02
PySpark Programming Example
-
O3
Understanding Map Reduce and Yarn
01
Introduction to Hadoop
02
Recorded Class Video 1
03
Recorded Class Video 2
- Topic 6
Topic 6
Lab: Introduction to pyspark programming
O1
Basic programming with pyspark
O2
Basic Programming with RDD
O1
To Work with RDD transformations and action
O2
Working with pyspark programming
- Week 4: Map Reduce and Yarn
Week 4: Map Reduce and Yarn
O5
Hadoop MapReduce Example
O1
Know details about MapReduce and YARN
O2
Differences between Hadoop v1.0 and v2.0
O3
Using MapReduce in word count
- Topic 8
Topic 8
Lab: Introduction to RDD and DataFrame
O1
Introduction to Dataframes
O2
WordCount through RDDs
03
Class Recording Link -1
03
Class Recording Link - 2
- Week 5: Hadoop cluster and ecosystem
Week 5: Hadoop cluster and ecosystem
Hadoop cluster and ecosystem
O3
Benefits of Hadoop Cluster
O4
Challenges of Hadoop Cluster
O1
Know details about Hadoop Cluster and Ecosystem
O2
Know about different types of tools.
- Topic 10
Topic 10
LAb: DataFrames and Spark SQL
O2
Loading and transforming data through different sources
O3
Covid- 19 example dataset
O1
Able to work on different dataset.
O2
Work with database query
01
Project Task Description
- Week 6: Mid Term
Week 6: Mid Term
Mid Syllabus: W1-W5
1. Introduction to Big Data
2. Big Data Analytics
3. Introduction to Hadoop
4. Map Reduce and Yarn
5. Apache Sqoop, Hive, Pig
- Week 8: Apache Sqoop, Hive and Pig
Week 8: Apache Sqoop, Hive and Pig
Apache sqoop, hive and pig
O5
Differences between Pig and Hive
O6
The architecture of Sqoop, Pig, and Hive
O1
Details about apache sqoop, hive, and pig
O2
Learn about the differences between them.
01
Apache Pig, Sqoop and Hive
Week 8: Lab on Feature Engineering
- Lab-Week 8: IoT Lab
Lab-Week 8: IoT Lab
O3
Work with different sensors
O1
Able to work using Arduino.
O2
Able to work with different sensors
02
Class Recording - Section I
03
Class Recording - Section J
- Week 9: Introduction to IoT
Week 9: Introduction to IoT
O3
IoT applications for industry
O1
Know Details about IoT and its importance.
O2
Know Details about IoT domains and its applications.
- Lab-Week 9-10: Implementing to IoT - 2
Lab-Week 9-10: Implementing to IoT - 2
O3
IoT applications for industry
O1
Know Details about IoT and its importance.
O2
Know Details about IoT domains and its applications.
01
Introduction to IoT - 2
O3
IoT applications for industry
O1
Know Details about IoT and its importance.
O2
Know Details about IoT domains and their applications.
- Week 10: IOT
Week 10: IOT
O3
IoT applications for industry
O1
Know Details about IoT and its importance.
O2
Know Details about IoT domains and their applications.
- Week 12: Presentation
- Week 13: Review Class
Week 13: Review Class
Review on the topics of Week 8, Week 9, Week 10, Week 11 and Week 12.
- Assignment
- Project Report
- others
- Topic 22
Topic 22
Presentation
Present any Deep Learning model architecture.
Mark: 8
Time: 5-7min
Number of Slides: 10-13
Slides must be eye catching.
Rubric of presentation (out of 100):
- getup & outfit : formal/semi formal (10%)
- Body language 10%
- Communication style : bangla/ english 10%
- eye contact 10%
- knowledge 40%
- Handling Ques/Ans 20%
- Project
- Topic 24
Topic 24
week 9 Apache Sqoop, Pig and Hive
- Apache Sqoop
-
Apache Flume
-
Apache Pig
- Apache Hive
Expected Learning Outcomes:
- Understanding the features and working principle of Apache Sqoop, Flume, Pig and Hive
Resources of Learning:
- Topic 25
Topic 25
week 10 Introduction to IoT
- What is IoT ?
- Element and characteristics of IoT
- Sensor and actuator
- Embedded system vs IoT system
- Different types of IoT level
- Green field vs Brown field of IoT
- Future factory concept and smart objects
Expected Learning Outcomes:
- Understanding the concepts of IoT to implement in real world scenario.
Resources of Learning:
- Topic 26
Topic 26
week 11 IoT protocol & communication model
- IoT protocols
- Communication models in IoT
- IoT enabling technologies
- Advantages and disadvantages of IoT
Expected Learning Outcomes:
- Understanding the concepts of protocols and communication models in IoT enabling technologies.
Resources of Learning:
- Topic 27
Topic 27
week 12 M2M and IoT
Expected Learning Outcomes:
- Understanding the concepts of IoT to implement in real world scenario.
Resources of Learning:
- Topic 28
- Topic 29
Topic 29
Final Syllabus:
Updated outline for final exam
1. 1. Apache Sqoop, Apache Pig & Apache Hive
2. Introduction to IoT
3. IoT_Part2
4. M2M and IOT
Assessment Plan
Total Marks: 40
End of the Semester.
- Topic 30
Topic 30
Lab evaluation
Mark: 30
Dataset: KMNIST
Perform the
following operation on the given dataset:
1. Load the data, check whether GPU is available
and shape of the data (5)
2. Design the CNN model (The model has 5 convolution layers, 4 fully connected layer. Include dropout, batch normalization,
pooling where needed) (10)
3. Train the model for 2epochs (5)
4. Test the model, calculate accuracy, and draw a
confusion matrix for the model (5)
5. Apply any pretratined model, calculate the accuracy (5)
Time:
1:15 hr
Don't share your code with others. Upload the code in python file format. Dont make it pdf. Rename the code by your name
- Topic 31
- Lab project