CS/DATA 7200 : Machine Learning

CS 7200: Machine Learning

Semester Hours:   3.0
Contact Hours:   3
Coordinator:   TBD
Text:    1. Pattern Recognition and Machine Learning / 2. Deep Learning (Adaptive Computation and Machine Learning Series)
Authors:   1. BISHOP / 2. GOODFELLOW, BENGIO, AND COURVILLE
Year:    1. 2011 / 2. 2016

SPECIFIC COURSE INFORMATION

Catalog Description

The course provides foundations of machine learning, mathematical derivation and implementation of the algorithms and their applications. Topics include supervised learning, learning theory, graphical model, reinforcement learning, Bayesian techniques, deep learning and ethics. In addition, practical applications are considered using the machine learning algorithms. The course also requires an open-ended research program. Prerequisites: CS 5200 and STAT 5020, or permission of instructor. Credit cannot be received for both DATA 7200 and CS 7200. Approved for Distance Education.

Course type: ELECTIVE

SPECIFIC COURSE GOALS

• I am able to explain the mathematical foundations of machine learning.
• I am able to explain the differences between supervised and unsupervised learning.
• I am able to design learning model for a real-world application.
• I am able to develop and implement well-known supervised learning algorithms.
• I am able to utilize modern frameworks and tools for deep learning.
• I am able to analyze ethical issues associated with machine learning.

LIST OF TOPICS COVERED

• Introduction (~5%)
  • Overview
  • Probability review, loss function, maximum likelihood
  • Linear regression, gradient descent, Newton method
• Classification (~10%)
  • Naïve Bayes
  • Linear models
  • Kernel Methods
  • Support vector machines
• Regularization (~10%)
  • L2 regularization
  • L1 regularization, sparsity and feature selection
  • Bias-variance tradeoff, overfitting
  • Developing basic machine learning algorithms
• Learning Theory (~10%)
  • Sample complexity
  • Probably Approximately Correct (PAC) learning
  • Error bounds
• Graphical Model (~10%)
  • Bayesian networks
    • Representation, inference, maximum likelihood estimation, hidden Markov models
  • Structure learning
• Unsupervised Learning (~10%)
  • Introduction to unsupervised learning
  • Clustering
  • PCA
• Reinforcement Learning (~10%)
  • Markov Decision Processes (MDP)
  • Value function approximation
• Deep Networks & Learning (~20%)
  • Perception and multilayer perception
  • Deep feedforward networks
  • Optimization for training deep models
  • Regularization for Deep Learning
  • Convolutional networks
  • Practical methodology, applications
• Ethics of Machine Learning (~15%)
  • Introduction – What and Why?
  • Data bias and fairness
  • Privacy and security
  • Choose from: model interpretability, model accountability, or adversarial uses
  • Case study

(*Optional, if time allows.)

RECOMMENDED REFERENCES

• Machine Learning: A Probabilistic Perspective, by Kevin Murphy, MIT Press, 2012
• Elements of Statistical Learning, by Hastie, Tibshirani, Friedman, Springer, 2010
• Bayesian Reasoning and Machine Learning, by David Barber, Cambridge University Press, 2012
• Reinforcement Learning: An Introduction, by Sutton and Barto, MIT Press, 1998