EE381V - Large Scale Learning

Spring Semester 2013

Some Basic Information

Instructors: Constantine Caramanis / Sujay Sanghavi

Email: caramanis/sanghavi AT mail DOT utexas DOT edu

Phone: (512) 471-9269 / (512) 475-9798

Office: ENS 427/429

Constantine's Office Hours: Wednesday 1:00 pm

Sujay's Office Hours: Monday 1:00 pm

TA: Avik Ray

Email: avik001 AT gmail DOT com


Time: Tuesday and Thursday, 2:00-3:30 PM,

Location: BUR 224

Course Overview

This is the second course in a two-course sequence on Large-Scale Optimization and Learning. While the first course focused on convex optimization, with an emphasis on methods for large-scale problems, this course will focus on drawing inference from data - machine learning techniques, with a focus on methods for problems of large size and high dimensionality. Intended audience: This class is structured to be interesting and relevant to students who are using or plan to use machine learning in their research, and are interested in solving large-scale problems. The target audience is quite broad: graduate students from ECE, CS, OR, Math, DSSC, and related disciplines.

Course Prerequisites

This class does not assume previous exposure to machine learning; it also does not specifically require having taken the first part of the sequence. However, students with either or both of these background elements will benefit more. Good familiarity with Linear Algebra (at the level of, e.g., EE380K) is important, as we will freely use concepts, tools and techniques from linear algebra; the same goes for Probability and Stochastic Processes (at the level of, e.g., EE381J). Some basic knowledge of Matlab will also be needed, although basic familiarity with programming should be sufficient.

General Note: If you are concerned about the prerequisites or your background, or what the course will cover, please don't hesitate to contact us by e-mail, or come by office hours.

Grading: Homeworks, Exams and Scribing

The grade for the course will be determined by three components: Problem sets (20%), Scribing (5%), a Midterm Exam (30%) and a Final Exam (45%). Problem sets will be given out approximately weekly. The midterm and final will be in-class closed-book exams.

Scribing: Each student is expected to scribe a lecture in latex. The goal is to produce a high-quality, complete record of the material covered in class. Students that scribe a Tuesday lecture are expected to submit a high-quality, polished and complete draft to Constantine and Sujay by Friday of the same week. Students scribing a Thursday lecture should submit this by Monday. This leaves time for some iteration if required, with the goal of posting the scribed notes within a week of the class scribed. You can find the necessary scribing templates here. Please reference completely and fully, as if you were writing a paper to submit. Also, as with any paper, all the writing should be your own.

Policy on Collaboration: Discussion of homework questions is encouraged. Please be sure to submit your own independent homework solution. This includes any matlab code required for the assignments. Late homework assignments will not be accepted.

Text and References

The course will be taught from a collection of sources; some books, but mainly research papers. These will be referenced and linked to (when possible) below.

Lecture Schedule

Lecture No. Date Problem SetsProblem Set SolutionsAssigned ReadingScribed Notes
1 Tue Jan 15 paper on LSH Lecture 1
2 Thu Jan 17 Lecture 2
3 Tue Jan 22 Lecture 3
4 Thu Jan 24 unscribed
5 Tue Jan 29 Lecture 5
6 Thu Jan 31 PS1 Solution 1 Lecture 6
7 Tue Feb 5 Lecture 7
8 Thu Feb 7 PS2 Solution 2 Paper on Spectral Clustering for Mixture Models Lecture 8
9 Tue Feb 12 Lecture 9
10 Thu Feb 14
11 Tue Feb 19 Isotropic PCA Lecture 11
12 Thu Feb 21 paper on Isomap and paper on LLE Lecture 12
13 Tue Feb 26 Lecture 13-14
14 Thu Feb 28 Lecture 13-14
15 Tue Mar 5 Lecture 15
16 Thu Mar 7 Matrix completion Lecture 16
SPRING BREAK Tue Mar 12 break
SPRING BREAK Thu Mar 14 break
17 Tue Mar 19 Lecture 17
18 Thu Mar 21 Lecture 18
19 Tue Mar 26 Lecture 19
20 Thu Mar 28 PS3 Solution 3 Sparsity and RIP Lecture 20
21 Tue Apr 2 OMP and RIP Lecture 21
22 Thu Apr 4 unscribed
23 Tue Apr 9 MIDTERM
24 Thu Apr 11 Dictionary Learning Lecture 24
25 Tue Apr 16 Lecture 25
26 Thu Apr 18 Lecture 26
27 Tue Apr 23 Lecture 27
28 Tue Apr 25 Sparse and Low-rank Decompositions unscribed
29 Tue Apr 30 unscribed
30 Thu May 02 unscribed
Final Exam Thu May 09 in ENS 115 9:00-12:00