Watch Demo

Rs. 2999  Rs. 599

Scalable programming with Scala and Spark

Learn how to use Scala and Spark for machine learning and data analytics. This course will teach you right from the basics to the advanced features on Spark Scala.

08h:54m
Lifetime access
34 learners
Course Overview

Most Data Scientists or Analysts use multiple systems like Python, Java, SQL, R etc, to work with data. Spark is a solo engine that you can use to play with and explore large chunks of data, run Machine Learning algorithms, and further use the same system to productize your code. Scala is a generic programming language equivalent to C++ and Java. Its practical programming nature, in addition to the availability of a REPL environment, are designed to specifically suit a distributed computing framework like that of Spark. Both these programming languages are used to explore and analyse data in an interactive environment that requires quick feedback. The core functionality of Spark along with its built-in libraries makes it smooth to implement complex algorithms in addition to a few lines of code.

Read more

Course Objectives

By the end of this course you will:

  • Be able to use Spark for numerous analytics and Machine Learning tasks
  • Develop an understanding on functional programming constructs in Scala
  • Be able to implement complex algorithms like Music Recommendations or PageRank
  • Learn to work with a variation of datasets like Airline delays, web graphs, Twitter, social networks and product ratings

Read more

Prerequisites and Target Audience:

All the examples that are shown in the course work either with or without Hadoop. In order to use Spark with Hadoop, it needs to be installed either in a pseudo-distributed or cluster mode. You will also require knowledge of the programming language Java or C++

Read more
Course Plan
Certificate of completion

1. You, This Course and Us
2 videos
Installing Scala and Hello World 09:43
2. Introducing Scala
11 videos
Scala - A "better Java"? 10:13

Installing Scala and Hello World 09:43

How do Classes work in Scala? 11:02

Classes in Scala - continued 15:50

Functions are different from Methods 07:30

Collections in Scala 10:12

Map, Flatmap - The Functional way of looping 11:36

First Class Functions revisited 08:46

Partially Applied Functions 07:31

Closures 08:07

Currying 10:34
3. Introduction to Spark
8 videos
Why is Spark so cool? 12:23

An introduction to RDDs - Resilient Distributed Datasets 09:39

Built-in libraries for Spark 15:37

Installing Spark 11:44

The Spark Shell 06:55

See it in Action : Munging Airlines Data with Spark 03:44

Transformations and Actions 17:06
5. Advanced RDDs: Pair Resilient Distributed Datasets
6 videos
Special Transformations and Actions 14:46

Average delay per airport, use reduceByKey(), mapValues() and join() 13:35

Average delay per airport in one step using combineByKey() 08:23

Get the top airports by delay using sortBy() 02:51

Lookup airport descriptions using lookup(), collectAsMap(), broadcast() 10:57

See it in Action: Analyzing Airlines Data with PySpark-III 04:58
6. Advanced Spark: Accumulators, Spark Submit, MapReduce , Behind The Scenes
6 videos
Get information from individual processing nodes using accumulators 09:25

Long running programs using spark-submit 07:11

Spark-Submit with Scala - A demo 06:10

Behind the scenes: What happens when a Spark script runs? 14:30

Running MapReduce operations 10:53

See it in Action: Map Reduce with Spark 10:53
7. PageRank: Ranking Search Results
5 videos
What is PageRank? 16:44

The PageRank algorithm 06:15

Implement PageRank in Spark 09:45

Join optimization in PageRank using Custom Partitioning 06:28

See it in Action: The PageRank algorithm using Spark 06:15
8. Spark SQL
1 video
Dataframes: RDDs + Tables 15:48
9. MLlib in Spark: Build a recommendations engine
4 videos
Collaborative filtering algorithms 12:20

Latent Factor Analysis with the Alternating Least Squares method 11:39

Music recommendations using the Audioscrobbler dataset 05:38

Implement code in Spark using MLlib 14:45
10. Spark Streaming
4 videos
Introduction to streaming 09:55

Implement stream processing in Spark using Dstreams 09:19

Stateful transformations using sliding windows 08:17

See it in Action: Spark Streaming 04:17
11. Graph Libraries
1 video
The Marvel social network using Graphs 14:30

Meet the Author


Loonycorn
4 Alumni of Stanford, IIM-A, IITs and Google, Microsoft, Flipkart

Loonycorn is a team of 4 people who graduated from reputed top universities. Janani Ravi, Vitthal Srinivasan, Swetha Kolalapudi and Navdeep Singh have spent years (decades, actually) working in the Tech sector across the world.

  • Janani: Graduated from Stanford and has worked for 7 years at Google (New York, Singapore). She also worked at Flipkart and Microsoft.
  • Vitthal: Studied at Stanford; worked at Google (Singapore), Flipkart, Credit Suisse, and INSEAD.
  • Swetha: An IIM Ahmedabad and IIT Madras alumnus having experience of working in Flipkart.
  • Navdeep: An IIT Guwahati alumnus and Longtime Flipkart employee.
  • More from Loonycorn
    Ratings and Reviews     3.8/5

    You may also like