Section outline

  • Collapse all Expand all

    WELCOME LETTER

    Dear Students

    Welcome to the Big Data and IoT Theory (CSE 412) &  Lab (CSE 413) courses, I,  Nasima Islam Bithi will be your co-pilot in this online journey of learning.

    I care about your success in these courses. I'm glad you are here.

    Nasima Islam Bithi
    Lecturer,
    Department of Computer Science and Engineering,

    Daffodil International University

    Basic Information:

    Course Code: CSE 412/413
    Course Title: Big Data and IoT
    Program: B.Sc in Computer Science and Engineering
    Faculty: Science and Information Technology
    Semester: Fall; Year: 2023
    Credit: Theory (1Cr) and Lab (3Cr); 
    Course Category: Core Engineering

    Course Instructor:

    Nasima Islam Bithi
    Lecturer
    Office: Room # 505, AB04 Building, Daffodil Smart City
    Cell Number# +880 1629665310

    Email: bithi.cse@diu.edu.bd

    Course Rationals

    Big Data refers to a massive set of data that no conventional data management tool can handle. Big Data is therefore a concept that allows access to gigantic databases in real time. Big Data's main objectives are to improve a company's or system's responsiveness to a large amount of data collected, increase productivity and refine knowledge of customer behavior, so that it can offer personalized offers or advertisements and create new trends.

    The Internet of Things (IoT) is a concept that connects physical or virtual objects to the internet. The technology very often used is the sensor, allowing to link a physical object such as a watch, a drone or even a speaker, to the internet. If for a long time the few objects connected to the Internet were the telephone and the computer, this is no longer the case today and every year new types of objects incorporating IOT technology are born. IOT is one of the greatest technological revolutions of our era and its potential for exploitation is immense. IOT could have a huge impact on the cars of the future or on the new versions of smart-cities, an urban space connected to the Internet, thus significantly improving the lives of users, while reducing the negative impact of these on the planet. 

    Assessment Plan

    Final Exam
    40
    Mid-term Exam
    25
    3 Class Tests
    15
    Attendance
    07
    Presentation
    08
    Total
    100

  • Topic 1

    Week 1

    Introduction

    • Topic of Discussion

      O1
      Introduction with the students
      O2
      Introduction to Data and Information
      O3
      Small Data and Big Data
      O4
      Types of Big Data
      O5
      Big Data Characteristics
      O1
      Learn about data and information
      O2
      Learn about many types of data
      O3
      Explore several characteristics of big data

    • Resources of Learning

      01
      Introduction to Big Data
      02
      Class Recorded Video 1 (Theory)
      03
      Class Recorded Video 1 (Theory)
      03
      Reference Video
    • forum icon
      Expected Learning Outcome Forum
      Due: Tuesday, 29 August 2023, 11:59 PM
      Students must
      View
    • assign icon
      Github link Assignment
      Opened: Wednesday, 30 August 2023, 12:00 AM
      Due: Thursday, 31 August 2023, 12:00 AM
      Students must
      Mark as done

  • Topic 2

    • Week 1

      LAB : Introduction

    • Topic of Discussion

      O1
      Introduction to Big Data
      O2
      Introduction to Python
      O3
      Introduction to Apache Spark
      O1
      Install PyCharm
      O2
      Work with basic python programming
      O3
      Install Apache Spark

      Resources of Learning

      01
      Install Pycharm on Ubuntu
      02
      Install Pycharm on Windows
      03
      Install Apache Spark on Ubuntu
      04
      Install Apache Spark on Windows
      05
      Install Apache Spark + Python = PySpark with Jupyter Notebook on Windows
    • Students must
      Mark as done
      Apache spark Installation with Colab
    • forum icon
      Task submission of Installation Forum
      Due: Tuesday, 29 August 2023, 11:59 PM
      Students must
      View

  • Topic 3

    • Week 2

      Big Data Analytics

    • Topic of Discussion

      O1
      Why big data analytics?
      O2
      What is big data analytics?
      O3
      Lifecycle of big data analytics
      O4
      Types of big data analytics
      O5
      Tools used in big data analytics
      O1
      Learn about the big data analytics
      O2
      Learn about the application of big data analytics.
      O3
      Learn about the big data application domains

      Resources of Learning

      01
      Big Data Analytics Slide
      02
      Class Recorded Video 
      03
      Class Recorded Video 
    • resource icon
      lab 2 File
      Students must
      Mark as done

      Lab 02
      https://github.com/Bithi-MBSTU/Machine-Learning-Practice-Examples

  • Topic 4

    Week 2

    Lab: Installtion of PySpark

  • Topic 5

    Week 3

    Introduction to Hadoop

    • Topic of Discussion

      O1
      What is Hadoop
      O2
      Hadoop Components
      O3
      HDFS
      O4
      Master and Slave Node
      O5
      Map Reduce and Yarn
      O1
      Understand Hadoop
      O2
      Understanding HDFS
      O3
      Understanding Map Reduce and Yarn

    • Resources of Learning

      01
      Introduction to Hadoop
      02
      Recorded Class Video 1
      03
      Recorded Class Video 2
      04
      Recorded Class Video
    • forum icon
      Open Discussion Forum
      Due: Tuesday, 29 August 2023, 11:59 PM
      Students must
      View

  • Topic 6

    Week 3

    Lab: Introduction to pyspark programming

    • Topic of Discussion

      O1
      Basic programming with pyspark
      O2
      Basic Programming with RDD
      O1
      To Work with RDD transformations and action
      O2
      Working with pyspark programming

      Resources of Learning

      01
      Lab Class Recording
      02
      Lab Class Recording
      03
      Reference Link
      04
      Fifa layer CSV File
    • forum icon
      Open Discussion Forum
      Due: Tuesday, 29 August 2023, 11:59 PM
      Students must
      View

  • Week 4: Map Reduce and Yarn

    Week 4

    Map Reduce and Yarn

    • Topic of Discussion

      O1
      Map Reduce
      O2
      YARN
      O3
      YARN with Map Reduce
      O4
      Hadoop MapReduce
      O5
      Hadoop MapReduce Example
      O1
      Know details about MapReduce and YARN
      O2
      Differences between Hadoop v1.0 and v2.0
      O3
      Using MapReduce in word count

    • Resources of Learning

      01
      Map Reduce and YARN
      02
      Recorded Class Video
      03
      Recorded Class Video

  • Topic 8

    Week 4

    Lab: Introduction to RDD and DataFrame

    • Topic of Discussion

      O1
      Introduction to Dataframes
      O2
      WordCount through RDDs
      O3
      Project Related Discussion
      O1
      Work with dataset
      O2
      To update the project

      Resources of Learning

      01
      Data Set
      02
      Class Recording Link 
      03
      Class Recording Link -1 
      03
      Class Recording Link - 2 
      04
      Reference Link
    • forum icon
      Class Task-E,F Forum
      Due: Tuesday, 29 August 2023, 11:59 PM
      Students must
      View

  • Week 5: Hadoop cluster and ecosystem

    Week 5

    Hadoop cluster and ecosystem

    • Topic of Discussion

      O1
      Hadoop Cluster
      O2
      Hadoop Ecosystem
      O3
      Benefits of Hadoop Cluster
      O4
      Challenges of Hadoop Cluster
      O1
      Know details about Hadoop Cluster and Ecosystem
      O2
      Know about different types of tools.

    • Resources of Learning

      01
      Hadoop Cluster
      02
      Hadoop Ecosystem
      03
      Class Recording 
      04
      Class Recording

  • Topic 10

    Week 5

    LAb: DataFrames and Spark SQL

    • Topic of Discussion

      O1
      Creating data frames
      O2
      Loading and transforming data through different sources
      O3
      Covid- 19 example dataset
      O1
      Able to work on different dataset.
      O2
      Work with database query

      Resources of Learning

      01
      Project Task Description
      02
      Class Recording 
      03
      Class Recording 
      04
      Code
    • forum icon
      Class Task Forum
      Due: Tuesday, 29 August 2023, 11:59 PM
      Students must
      View
    • Students must
      Mark as done
      Apache Spark with SQL, Dataframes

  • Week 6: Mid Term

    Week 6

    MIDTERM


    • Mid Syllabus: W1-W5

      1. Introduction to Big Data

      2. Big Data Analytics

      3. Introduction to Hadoop

      4. Map Reduce and Yarn

      5. Apache Sqoop, Hive, Pig


    • quiz icon
      QUIZ-02
      Opened: Thursday, 14 September 2023, 11:03 AM
      Closed: Thursday, 14 September 2023, 12:40 PM
      Students must
      Mark as done
    • resource icon
      CT 02 File
      Students must
      Mark as done

  • Week 8: Apache Sqoop, Hive and Pig

    Week 8

    Apache sqoop, hive and pig

    • Students must
      Mark as done

      Topic of Discussion

      O1
      Hadoop Ecosystem
      O2
      Apache Sqoop
      O3
      Apache Hive
      O4
      Apache Pig
      O5
      Differences between Pig and Hive
      O6
      The architecture of Sqoop, Pig, and Hive
      O1
      Details about apache sqoop, hive, and pig
      O2
      Learn about the differences between them.

      Resources of Learning

      01
      Apache Pig, Sqoop and Hive
      02
      Midterm Question Related Discussion
      03
      Recorded Class Video
    • forum icon
      Open Discussion Forum
    • Students must
      Mark as done

      Week 8: Lab on Feature Engineering

    • forum icon
      Submission on Class Practise Forum
      Due: Monday, 4 September 2023, 5:30 AM
      Students must
      Mark as done
    • assign icon
      Assignment on Feature Engineering Task
      Opened: Thursday, 7 September 2023, 11:00 AM
      Due: Thursday, 14 September 2023, 11:59 PM
      Students must
      Mark as done

  • Lab-Week 8: IoT Lab

    Week 8

    IoT Lab

    • Students must
      Mark as done

      Topic of Discussion

      O1
      Basic Concept of IoT
      O2
      Work with Arduino
      O3
      Work with different sensors
      O1
      Able to work using Arduino.
      O2
      Able to work with different sensors

      Resources of Learning

      01
      Online Simulator
      02
      Class Recording - Section I
      03
      Class Recording - Section J

  • Week 9: Introduction to IoT

    Week 9

    Introduction to IoT

    • Students must
      Mark as done

      Topic of Discussion

      O1
      Introduction to IoT
      O2
      Domain Specific IoTs
      O3
      IoT applications for industry
      O1
      Know Details about IoT and its importance.
      O2
      Know Details about IoT domains and its applications.

      Resources of Learning

      01
      Introduction to IoT
      03
      Recorded Class Video
    • Students must
      Mark as done

      6.1_Hadoop Ecosystem

      6.2_Hadoop Cluster

      7.1_Introduction to IOT

  • Lab-Week 9-10: Implementing to IoT - 2


    • Students must
      Mark as done

      Topic of Discussion

      O1
      Introduction to IoT
      O2
      Domain Specific IoTs
      O3
      IoT applications for industry
      O1
      Know Details about IoT and its importance.
      O2
      Know Details about IoT domains and its applications.

      Resources of Learning

      01
      Introduction to IoT - 2
    • Students must
      Mark as done

      Topic of Discussion

      O1
      Introduction to IoT
      O2
      Domain-Specific IoTs
      O3
      IoT applications for industry
      O1
      Know Details about IoT and its importance.
      O2
      Know Details about IoT domains and their applications.

      Resources of Learning

      01
      Introduction to IoT
      03
      Recorded Class Video

  • Week 10: IOT

    Week 10

    M2M and IoT

    • Students must
      Mark as done


      Topic of Discussion

      O1
      Introduction to IoT
      O2
      Domain-Specific IoTs
      O3
      IoT applications for industry
      O1
      Know Details about IoT and its importance.
      O2
      Know Details about IoT domains and their applications.

      Resources of Learning

      01
      M2M and IoT_MMH
      03
      Recorded Class Video
    • forum icon
      Open Discussion Forum
    • quiz icon
      Quiz-3
      Opened: Thursday, 9 November 2023, 9:10 PM
      Closed: Thursday, 9 November 2023, 10:15 PM
      Students must
      Mark as done
    • Students must
      Mark as done

      7.2_IOT PATR-2

      8_M2M and IOT

  • Week 12: Presentation


  • Week 13: Review Class

    Topic of Discussion

             Review on the topics of Week 8, Week 9, Week 10, Week 11 and Week 12.




  • Assignment

    • assign icon
      Assignment-1(Who missed Quiz)
      Opened: Friday, 3 November 2023, 10:00 AM
      Due: Saturday, 4 November 2023, 4:00 PM
      Students must
      Mark as done

  • Project Report

  • others

  • Topic 22


    Presentation


    Present any Deep Learning model architecture.

    Mark: 8                
    Time: 5-7min
    Number of Slides: 10-13
    Slides must be eye catching.

    Rubric of presentation (out of 100):

    • getup & outfit : formal/semi formal (10%)
    • Body language 10%
    • Communication style : bangla/ english 10%
    • eye contact 10%
    • knowledge 40%
    • Handling Ques/Ans 20%

    • assign icon
      Section 55 - A Assignment
      Opened: Monday, 25 September 2023, 12:00 AM
      Due: Monday, 2 October 2023, 12:00 AM
      Students must
      Mark as done
    • assign icon
      Section 55-M Assignment
      Opened: Monday, 25 September 2023, 12:00 AM
      Due: Monday, 2 October 2023, 12:00 AM
      Students must
      Mark as done

  • Project

  • Topic 24

    week 9 Apache Sqoop, Pig and Hive

    Topics of Discussion:

    • Apache Sqoop
    • Apache Flume
    • Apache Pig
    • Apache Hive

    Expected Learning Outcomes:

    • Understanding the features and working principle of Apache Sqoop, Flume, Pig and Hive

    Resources of Learning:


  • Topic 25

    week 10 Introduction to IoT

    Topics of Discussion:

    • What is IoT ?
    • Element and characteristics of IoT
    • Sensor and actuator
    • Embedded system vs IoT system
    • Different types of IoT level
    • Green field vs Brown field of IoT
    • Future factory concept and smart objects

      Expected Learning Outcomes:

      • Understanding the concepts of IoT to implement in real world scenario.

      Resources of Learning:


    • Topic 26

      week 11 IoT protocol & communication model

      Topics of Discussion:

      • IoT protocols
      • Communication models in IoT
      • IoT enabling technologies
      • Advantages and disadvantages of IoT

        Expected Learning Outcomes:

        • Understanding the concepts of protocols and communication models in IoT enabling technologies.

        Resources of Learning:


      • Topic 27

        week 12 M2M and IoT

        Topics of Discussion:

        • What is IoT ?

          Expected Learning Outcomes:

          • Understanding the concepts of IoT to implement in real world scenario.

          Resources of Learning:


        • Topic 28

          Online class recording

        • Topic 29


          Final

          Final Syllabus:

          Updated outline for final exam
          1. 1. Apache Sqoop, Apache Pig & Apache Hive
          2. Introduction to IoT

          3.  IoT_Part2

          4. M2M and IOT

          Assessment Plan

          Total Marks: 40


          End of the Semester.



        • Topic 30

          Lab evaluation

          Mark: 30

          Dataset: KMNIST

          Perform the following operation on the given dataset:
          1. Load the data, check whether GPU is available and shape of the data (5)
          2. Design the CNN model (The model has 5 convolution layers, 4 fully connected layer. Include dropout, batch normalization, pooling where needed) (10)
          3. Train the model for 2epochs (5)
          4. Test the model, calculate accuracy, and draw a confusion matrix for the model (5)

          5. Apply any pretratined model, calculate the accuracy (5)

          Time: 1:15 hr
          Don't share your code with others. Upload the code in python file format. Dont make it pdf. Rename the code by your name



          • assign icon
            Final Lab test (55_M) Assignment
            Opened: Thursday, 23 November 2023, 10:00 AM
            Due: Thursday, 23 November 2023, 11:15 AM
            Students must
            Mark as done

        • Topic 31

          CT-03

          • assign icon
            CT 03 (55_A) Assignment
            Opened: Wednesday, 22 November 2023, 3:15 PM
            Due: Wednesday, 22 November 2023, 3:40 PM
            Students must
            Mark as done
          • assign icon
            CT -03 (55_M) Assignment
            Opened: Wednesday, 22 November 2023, 3:15 PM
            Due: Wednesday, 22 November 2023, 3:40 PM
            Students must
            Mark as done

        • Lab project

          • assign icon
            Lab project (55_A) Assignment
            Opened: Friday, 24 November 2023, 12:00 AM
            Due: Saturday, 25 November 2023, 11:59 PM
            Students must
            Mark as done
          • assign icon
            Lab project (55_M) Assignment
            Opened: Friday, 24 November 2023, 12:00 AM
            Due: Saturday, 25 November 2023, 11:59 PM
            Students must
            Mark as done