Machine Learning

Slides

• Lecture 1: Introduction [Slides]
• Lecture 2: Linear Regression [Slides] [Notes]
• Lecture 3: Logistic Regression [Slides]
• Lecture 4: Bayesian Statistics and Regularization [Slides]
• Lecture 5: Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm [Slides] [Notes]
• Lecture 6: Support Vector Machine [Slides][Notes][Appendix]
• Lecture 7: K-Means [Slides]
• Lecture 8: Factor Analysis and Principle Component Analysis [Slides]
• Lecture 9: Learning Theory [Slides]
• Lecture 10: Neural Networks and Deep Learning [Slides]

Lab Exercises

Each student is supposed to submit a technical report (with source code enclosed) for every lab exercise before the due date. The template of technical report can be downloaded by clicking here.

The technical report should be in pdf format and named as follows "Student ID_Name_Lab Index.pdf". For example, for a student whose name is 李雷 with student ID being 123456, his technical report for the 4th lab exercise should be "123456_李雷_实验1.pdf". Also, the source code should be enclosed in the technical report as an appendix. Moreover, separated source codes (e.g., Octavel/Mablab or python source codes) should be submitted as separated files. All source code files should be compressed as a ZIP file (e.g., entitled as "123456_李雷_实验1.zip").

The exercises in this course use Octave1 or MATLAB, a high-level programming language well-suited for numerical computations. At the Octave/MATLAB command line, typing help followed by a function name displays documentation for a built-in function. For example, help plot will bring up help information for plotting. Further documentation for Octave functions can be found at the Octave documentation pages.. MATLAB documentation can be found at the MATLAB documentation pages. We also strongly encourage using the online Discussions to discuss exercises with other students. However, do not look at any source code written by others or share your source code with others.

• Exp 1 Linear Regression [Tutorial][Data Set] [Due Date: 8 Oct, 2024]
• Exp 2 Logistic Regression and Newton's Method [Tutorial][Data Set] [Due Date: 22 Oct, 2024]
• Exp 3 Regularization [Tutorial][Data Set] [Due Date: 5 Nov, 2024]
• Exp 4 Naive Bayes [Tutorial][Data Set] [Due Date: 19 Nov, 2024]
• Exp 5 SVM [Tutorial][Data Set] [Due Date: 3 Dec, 2024]
• Exp 6 K-Means [Tutorial][Data Set] [Due Date: 17 Dec, 2024]
• Exp 7 PCA in Face Detection [Tutorial][Data Set] [Due Date: 24 Dec, 2024]

Assignments

Students are supposed to submit their solutions before the due date.

• Problem Set 1 [PDF] [Due Date: 12 Nov, 2024]
• Problem Set 2 [PDF] [Due Date: 26 Nov, 2024]
• Problem Set 3 [PDF] [Due Date: 17 Dec, 2024]

Useful Links

• Octave: https://www.gnu.org/software/octave/
• Properties of the Trace and Matrix Derivatives (by John Duchi)
• The Matrix Cookbook (by Kaare Brandt Petersen and Michael Syskind Pedersen)
• Convex Optimization (by Stephen Boyd)
• Mathematics for Machine Learning (by Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong)
• Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Machine Learning (by Jean Gallier and Jocelyn Quaintance)
• Using LATEX for Report Writing (by Hans Fangohr)
• Linear algebra review(by Zico Kolter)
• Probability theory review(by Samuel Ieong)
• Convex optimization review(by Samuel Ieong and Chuong B. Do)
• Constructing SVM from Scratch (从零构建支持向量机(SVM))(by Hao Zhang)
• What to Do When Your Hessian Is Not Invertible: Alternatives to Model Respecification in Nonlinear Estimation(by Jeff Gill and Gray King)
• LIBSVM - A Library for Support Vector Machines(by Chih-Chung Chang and Chih-Jen Lin)
• Gradient: The Direction of Steepest Ascent(by Feng Li)
• More on Gaussians(by Chuong B. Do)
• A Brief Introduction to The Method of Lagrange Multiplier(by Feng Li)