Watch Demo

Rs. 999  Rs. 599

Apache Flume and Apache Sqoop tutorial for Ingesting Big Data

This Flume and Sqoop tutorial is created by a Stanford alumni team and teaches how to import data to HDFS and Hive from a variety of sources including MySQL, and Twitter.

02h:05m
Lifetime access
15 learners
Introduction to the Course

Apache Flume and Apache Sqoop are two tools used to draw data from different sources and load them into HDFS. They are mainly used to transport data from sources like HBase, HDFS, and Hive. Flume Agents transport data produced by a streaming application to data stores like HDFS and Hbase, while Sqoop is used to bulk up crucial data from traditional RDBMS to the Hadoop storage architectures-Hive and HDFS. This Flume and Sqoop tutorial for ingesting Big Data covers the basics of data transportation in Hadoop.

Read more

Course Objectives

By the end of this course you will:

  • Be able to use Flume to ingest data HBase and HDFS
  • Know how Sqoop to import data from MySQL to Hive and HDFS
  • Learn how to ingest data from a variety of sources like Twitter, HTTP, and MySQL

Prerequisites and Target Audience

To take up this online course on Flume and Sqoop for Ingesting Big Data, you will require knowledge of HDFS, HBase, and Hive shells. To run most of the examples, you will also need to install HDFS.

If you do not possess knowledge in one of the required fields, you can subscribe to our course on Learn Hadoop, MapReduce for Big Data problems by Example and HBase Tutorial: Learn by Examples.

This course is designed for Engineers, in specific for those who wish to port data from legacy data stores to HDFS or want to build an application using HDFS, HBase or Hive as the data store.

Course Plan
Certificate of completion

1. Why do we need Flume and Sqoop?
1 video
2. Understanding Flume
10 videos
Example 1 : Spool to Logger 14:34

Flume Events are how data is transported 06:07

Example 2 : Spool to HDFS 09:08

Example 3: HTTP to HDFS 09:24

Example 4: HTTP to HDFS with Event Bucketing 05:40

Example 5: Spool to HBase 06:22

Example 6: Using multiple sinks and Channel selectors 09:43

Example 7: Twitter Source with Interceptors 10:48
3. Sqoop
4 videos
Installing Sqoop 04:25

Example 8: Sqoop Import from MySQL to HDFS 07:49

Example 9: Sqoop Import from MySQL to Hive 04:26

Example 10: Incremental Imports using Sqoop Jobs 05:24

Meet the Author


Loonycorn
4 Alumni of Stanford, IIM-A, IITs and Google, Microsoft, Flipkart

Loonycorn is a team of 4 people who graduated from reputed top universities. Janani Ravi, Vitthal Srinivasan, Swetha Kolalapudi and Navdeep Singh have spent years (decades, actually) working in the Tech sector across the world.

  • Janani: Graduated from Stanford and has worked for 7 years at Google (New York, Singapore). She also worked at Flipkart and Microsoft.
  • Vitthal: Studied at Stanford; worked at Google (Singapore), Flipkart, Credit Suisse, and INSEAD.
  • Swetha: An IIM Ahmedabad and IIT Madras alumnus having experience of working in Flipkart.
  • Navdeep: An IIT Guwahati alumnus and Longtime Flipkart employee.
  • More from Loonycorn
    Ratings and Reviews     5/5

    You may also like