CS 4641/7641: Machine Learning (Spring 2020)

Course Information

Instructor:
Mahdi Roozbahani
Head TA:
Ruijia Wang
(rwang@gatech.edu)
TA:
Jayanta Bhowmick
(jayantabhowmick@gatech.edu)
TA:
Shreeshaa Kulkarni
(shrek@gatech.edu)
TA:
Rodrigo Borela
(rborelav@gatech.edu)
TA:
Nimisha Roy
(nroy9@gatech.edu)
TA:
Kevin Tynes
(kdtynes@gatech.edu)
TA:
Kunal Chawla
(kunalchawla@gatech.edu)
TA:
Shalini Chaudhuri
(shalini.chaudhuri@gatech.edu)
TA:
Shubhangi Upasani
(shubhangi.upasani@gatech.edu)
TA:
Tongtong Xu
(txu68@gatech.edu)
TA:
Huili Huang
(hhuang413@gatech.edu)
TA:
Danrong Zhang
(dzhang373@gatech.edu)

Course Overview

This course introduces techniques in machine learning with an emphasis on algorithms and their applications to real-world data. We will investigate the following question: how to computationally extract useful knowledge from data for decision making and task support? We will focus on machine learning methods, which are organized into three parts:

  1. Basic math for data science and machine learning

    • Linear algebra
    • Probability and statistics
    • Information theory

  2. Unsupervised machine learning for data exploration

    • Clustering analysis
    • Dimension reduction
    • Kernel density estimation

  3. Supervised learning for predictive data analysis

    • Tree-based models
    • Linear classification and regression
    • Neural networks

Prerequisites for this course include 1) basic knowledge of probability, statistics, and linear algebra; 2) Basic programming experience in Python, especially Jupyter Notebook.

Schedule

Date Topic Assignment Due Readings
January 07, 2020 Course Overview; Piazza Signup GT Honor Code
January 09, 2020 Math Basics: Linear Algebra
Class Notes 		Correlation vs Covariance
Linear Algebra Review by Zico Kolter
January 14, 2020 Math Basics: Probability and Statistics
Class Notes 		Probability Theory Review by Andrew Moore
January 16, 2020 Math Basics: Information Theory;
Class Notes 		AS1 Out The Differences Between Data, Information and Knowledge
Visual Information Theory by Chris Olah
January 21, 2020 Data Analysis Toolbox - Part 1; KKT for inequality constrained optimization; Project Presentations Fall 2019 - Part 1; Project Presentations Fall 2019 - Part 2; Project Presentations Fall 2019 - Part 3;
January 23, 2020 Data Analysis Toolbox - Part 2; Project Information; GitHub Pages; YAML Configuration; NumPy Tutorial; Matplotlib Tutorial The Heilmeier Catechism; Project Examples; seaborn: statistical data visualization;
January 28, 2020 Clustering Analysis and K-Means
Class Notes; 		Curse of dimensionality (Euclidean space example) Jupyter Notbook (Kmeans and DBSCAN);
January 30, 2020 Hierarchical Clustering
Class Notes 		AS1 Due Understanding the concept of Hierarchical clustering Technique;
February 04, 2020 Density-Based Clustering Start working on Project Proposal! GitHub Student Application; Jupyter Notbook (Kmeans and DBSCAN); Overleaf for GT students, no kidding.
February 06, 2020 Gaussian Mixture Model;
Class Notes		 AS2 Out
February 11, 2020 Gaussian Mixture Model; Evaluation of Clustering Algorithms
February 13, 2020 Evaluation of Clustering Algorithms
February 18, 2020 Density Estimation;
Class Notes		 Proposal Due KDE interactive visualization ; KDE sampling ; KDE SKLearn and sampling ; Jupyter Notebook Kernel Density Example;
February 20, 2020 Dimension Reduction
Class Notes; 		Image reconstruction using PCA ; Feature extraction using PCA ; PCA for images ; PCA as linear combination of features ; PCA and Linear Discriminant Analysis ;
February 25, 2020 Midterm Review
Class Notes; 		AS2 Due BLUE EXAM BOOK just $0.7 in BARNES\& NOBLE GT or Amazon or Staples;
February 27, 2020 Midterm Exam
March 03, 2020 Linear Regression
Class Notes		 Simple Linear Regression in Matrix Format; Adding Noise to Regression Predictors
March 05, 2020 Regularization and Linear Regression AS3 Out
March 10, 2020 Regularization and Linear Regression;
Class Notes;
March 12, 2020 Naïve Bayes and Logistic Regression;
Class Notes;
March 17, 2020 No Class Spring Break
March 19, 2020 No Class Spring Break
March 24, 2020 Testing online transition;
March 26, 2020 Testing online transition;
March 31, 2020 Decision Tree and Random Forest;
Ensemble Learning and Random Forest;
Class Notes;
 Evaluating Machine Learning Methods
April 02, 2020 Support Vector Machine;
 KKT and SVM
April 07, 2020 Support Vector Machine - contd;
Class Notes;
 AS4 Out AS3 Due
April 09, 2020 Kernel Method \ SVM;
Class Notes;
April 14, 2020 Neural Networks(Forward pass and Back propagation);
Class notes 		NN Playground ;
The role of a hidden layer
Back propagation numerical example
More detailed introduction
April 16, 2020 Neural Networks - contd;
Class notes
April 21, 2020 Neural Networks and Deep learning (CNN);
 All projects (GitHub links and 5 minutes video BlueaJeans recorded screen link) should be submitted by the end of the day 11:59 pm
AS4 Due (We will give everyone one week extension for this assignment without any penalty - You are allowed to resubmit your AS4 by the end of the day April 28) 		CNN Live Demo
A guide to an efficient way to build CNN and optimize its hyper-parameters
Back Propagation in CNN
Transfer learning in CNN
Project Scoring Guidance
Project Page and Video
April 23, 2020 No Class Reading Period
April 30, 2020 No Final Exam Exam Matrix Spring 2020

Office Hours and Questions

  • Office Hours:

    • Instructor: Thursdays 10:45-11:45am in Business School lobby (access to my office is not easy)
    • Kevin and Danrong: Mondays 11:30-12:30pm
    • Rodrigo and Shreeshaa: Tuesdays 02:30-03:30pm
    • Nimisha and Kunal: Tuesdays 03:30-04:30pm
    • Huili and Shalini: Wednesdays 11:30-12:30pm
    • Ruijia and Jayanta: Thursdays 04:30-05:30pm
    • Tongtong and Shubhangi: Fridays 02:30-03:30pm

    • TA Office Hours location: in Klaus building lobby at the first floor (next to room 1325)

  • Piazza will be the main place for course discussions and announcements. If you have questions, please ask it on Piazza first because 1) other students may have the same question; 2) you will get help much faster.

  • If it’s something you do not like to discuss publicly on Piazza, you can use private messaging on Piazza.

  • Anytime you want to send a private message just to me on Piazza, please make sure to add our HEAD TA too in case I may miss your message.

Grading

  • Assignments (50%)

    • There will be four assignments. Each one is designed for testing your understanding of the taught algorithms. Assignments will have programming and written analysis.
    • You will need to submit all your assignments using ipynb. In ipynb, you can use markdown text editor. Here is a quick guideline how to use Markdown in ipynb.
    • You are required to use Markdown and Latex for the written questions.
    • All assignments follow the “no-late” policy. Assignments received after the due time will receive zero credit.
    • There are some bonus questions in assignments for Undergrad students. The bonus questions are required to be answered for all Grad students and they are not considered as bonus points.
    • All students are expected to follow the Georgia Tech Academic Honor Code.
    • You can easily export your Jupyter Notebook to a Python file and import that to your desired python IDE to debug your code for assignments.
    • You are not allowed to share any assignment codes or answers with other students whatsoever. Piazza is the best place to have discussion regarding assignments. Discussions are just for the better understanding of questions and should not directly answer the questions.

  • Project Proposal (5%)

    • A project proposal should be just one page pdf (less than 500 words single spaced)
    • A project proposal should include:
      • Introduction/Background
      • Methods
      • Potential results
      • Discussion
      • At least three references (preferably peer reviewed)
    • A checkpoint to make sure you are working on a proper machine learning related project.

  • Project (20%)

    • You are expected to complete a project on machine learning with real-life data. Your project needs to be clear about 1) the data you are using; 2) the problem you are attempting to solve; 3) the method you are using; 4) the results and conclusion you attain.
    • You will need to turn in a GitHub page for your project. The project presentation and report must be combined into one deliverable using a GitHub page. For the project presentation, you just need to scroll down on your GitHub page when you present your project (make sure you have visible images and graph).
    • Each project needs to be completed in a team of 5 people (you will be forming your team on your own. In case you can't find any team, we will randomly assign you a team). Team members need to clearly claim their contributions in the project report.
    • Each presentation cannot exceed beyond N/A minutes. If your presentation takes more than N/A minutes, you will be asked to stop the presentation at N/A minute mark. There will be N/A minute for Q/A.
    • There will be N/A judges who will grade your presentations
    • Refer to Project hints for your project's template, creating GitHub page, and also some general hints to improve the accuracy of your predictive model.
    • If you are in a Grad students team, you are required to have both unsupervised and supervised learning in your project.
    • Google colaboratory allows free access to run your Jupyter Notebook. I strongly suggest to use it for your project, specially for teams that are going to employ Deep Learning.

  • Class participation (5%)

    • Your class participation score will be graded based on attendance and possibly in-class quizzes. For some lectures, I will take attendance using CANVAS.
    • Participation in class discussions (including asking relevant questions in class, volunteering to answer questions on Piazza) will be considered when determining your final grade. It will be especially useful when you are right on the edge of two letter grades.
    • Participation is required on presentation days.

  • Midterm Exam (10%)

    • The midterm exam will only cover the math and probability and un-supervised learning parts.
    • The midterm exam will take place the regular class time slot.
    • The midterm exam will be a written and open-book exam. No electronic material can be used except calculator. Only paper material can be used in the exam (books, printed notes, etc). It would be better if you prepare a one or two page cheatsheet for yourself.
    • There will be no make-up exams. You will get zero credit for your missed midterm exam.
    • Taking the exam is required in order to receive a letter grade for this course. Students who fail this requirement will receive an Incomplete grade on their transcript.

  • Final Exam (10%)

    • The final exam will only cover the supervised learning part.
    • The final exam will be at assigened date/time for this class.
    • The final exam will be a written and open-book exam. No electronic material can be used except calculator. Only paper material can be used in the exam (books, printed notes, etc). It would be better if you prepare a one or two page cheatsheet for yourself (let's save some trees).
    • Again, there will be no make-up exams. You will get zero credit for your missed final exam.
    • Taking the exam is required in order to receive a letter grade for this course. Students who fail this requirement will receive an Incomplete grade on their transcript.

  • Bonus points

    • About Bonus points: Bonus points will be counted to always be beneficial for your final grade. What do I mean by that? it means that if for some reasons I may need to curve the grades, bonus points will be applied to your grade after curving not before curving.
    • Undergrads and grads: Piazza has statistics which give us many measurements regarding how much a student has been involved on Piazza's activities such as viewing posts, answering questions, asking questions and so on. Not only we use this to account for a minor part of the Class Participation score, we will use the statistics to give students bonus points. Bonus points will be applied to students who answer the other students' questions correctly. At the end of the semester, we will define a minimum and maximum number of involvement considering all the students, and based on those, some students will receive at most %3 bonus points. It is possible to receive less than 3% bonus based on your activities on Piazza.
    • Undergrads: As you all know, we have bonus points for hws. Bonus points will be different for different hws.For example, hw 1 may have 30 bonus points, hw 2 may have 20 bonus points and so on. If you receive all the bonus points for all your hws, we will add %5 to your final grade.

  • Course Policy

    • Regrade policy: Disputes of grading on assignments, exams, and project must be discussed within one week of their return or grade posting. Should you find yourself having an issue with a grade, contact the TA first. After you talk with your TA, if you are not satisfied you may contact the course instructor.
    • Assignment extension: Assignments are designed in a way that students would have several weeks to finish them. Assignments will not be extended under any circumstances whatsoever. You will receive 0 credit if you submit an assignment after the deadline.
    • Missed exam policy: There will be no makeups for missed exams. Any request for exceptions to this policy should be made in advance when at all possible. Requests should be due to incapacitating illness, emergency such as death in the family or something similarly serious, and should be accompanied by supporting documentation and to be submitted to Dean of Students. Excuses such as not being aware of the exam will not be considered.

Dataset Ideas (may need API, or scraping) Thanks to Polo

Office of Disability Services

The Georgia Institute of Technology has policies regarding disability accommodation, which are administered through The Office of Disability Services: http://disabilityservices.gatech.edu. For students with disabilities, please contact this Office to request classroom accommodations.

Resources

Recommended books:

Other resources, such as machine learning toolboxes and datasets, will be provided throughout the course.

Grade Calculator


Are you an undergrad or gard student?
Undergrad Grad

Assignments
Grade Undergrad bonus Genaral bonus
HW1
HW2
HW3
HW4

Project/Exam/Participation
Grade
Project Proposal
Project
Midterm
Final
Class Participation




Assignemnt Score:

Undergrad Bonus Score:

General Bonus Score:

Project Proposal:

Project:

Midterm Exam:

Final Exam:

Class Participation:

Final Grade: