**Spring Semester 2013**

**Instructors:** Constantine Caramanis / Sujay Sanghavi

Email: caramanis/sanghavi AT mail DOT utexas DOT edu

Phone: (512) 471-9269 / (512) 475-9798

Office: ENS 427/429

Constantine's Office Hours: Wednesday 1:00 pm

Sujay's Office Hours: Monday 1:00 pm

**TA:** Avik Ray

Email: avik001 AT gmail DOT com

**Lectures:**

Time: Tuesday and Thursday, 2:00-3:30 PM,

Location: BUR 224

This is the second course in a two-course sequence on Large-Scale Optimization and Learning. While the first course focused on convex optimization, with an emphasis on methods for large-scale problems, this course will focus on drawing inference from data - machine learning techniques, with a focus on methods for problems of large size and high dimensionality. Intended audience: This class is structured to be interesting and relevant to students who are using or plan to use machine learning in their research, and are interested in solving large-scale problems. The target audience is quite broad: graduate students from ECE, CS, OR, Math, DSSC, and related disciplines.

This class does not assume previous exposure to machine learning; it also does not specifically require having taken the first part of the sequence. However, students with either or both of these background elements will benefit more. Good familiarity with Linear Algebra (at the level of, e.g., EE380K) is important, as we will freely use concepts, tools and techniques from linear algebra; the same goes for Probability and Stochastic Processes (at the level of, e.g., EE381J). Some basic knowledge of Matlab will also be needed, although basic familiarity with programming should be sufficient.

**General Note:** If you are concerned about the prerequisites or your background, or what the course will cover, please don't hesitate to contact us by e-mail, or come by office hours.

The grade for the course will be determined by three components: Problem sets (20%), Scribing (5%), a Midterm Exam (30%) and a Final Exam (45%). Problem sets will be given out approximately weekly. The midterm and final will be in-class closed-book exams.

**Scribing:** Each student is expected to scribe a lecture in latex. The goal is to produce a high-quality, complete record of the material covered in class. Students that scribe a Tuesday lecture are expected to submit a high-quality, polished and complete draft to Constantine and Sujay by Friday of the same week. Students scribing a Thursday lecture should submit this by Monday. This leaves time for some iteration if required, with the goal of posting the scribed notes within a week of the class scribed. You can find the necessary scribing templates here. Please reference completely and fully, as if you were writing a paper to submit. Also, as with any paper, all the writing should be your own.

**Policy on Collaboration:** Discussion of homework questions is encouraged. Please be sure to submit your own independent homework solution. This includes any matlab code required for the assignments. Late homework assignments will not be accepted.

The course will be taught from a collection of sources; some books, but mainly research papers. These will be referenced and linked to (when possible) below.

Lecture No. | Date | Problem Sets | Problem Set Solutions | Assigned Reading | Scribed Notes |

1 | Tue Jan 15 | paper on LSH | Lecture 1 | ||

2 | Thu Jan 17 | Lecture 2 | |||

3 | Tue Jan 22 | Lecture 3 | |||

4 | Thu Jan 24 | unscribed | |||

5 | Tue Jan 29 | Lecture 5 | |||

6 | Thu Jan 31 | PS1 | Solution 1 | Lecture 6 | |

7 | Tue Feb 5 | Lecture 7 | |||

8 | Thu Feb 7 | PS2 | Solution 2 | Paper on Spectral Clustering for Mixture Models | Lecture 8 |

9 | Tue Feb 12 | Lecture 9 | |||

10 | Thu Feb 14 | ||||

11 | Tue Feb 19 | Isotropic PCA | Lecture 11 | ||

12 | Thu Feb 21 | paper on Isomap and paper on LLE | Lecture 12 | ||

13 | Tue Feb 26 | Lecture 13-14 | |||

14 | Thu Feb 28 | Lecture 13-14 | |||

15 | Tue Mar 5 | Lecture 15 | |||

16 | Thu Mar 7 | Matrix completion | Lecture 16 | ||

SPRING BREAK | Tue Mar 12 | break | |||

SPRING BREAK | Thu Mar 14 | break | |||

17 | Tue Mar 19 | Lecture 17 | |||

18 | Thu Mar 21 | Lecture 18 | |||

19 | Tue Mar 26 | Lecture 19 | |||

20 | Thu Mar 28 | PS3 | Solution 3 | Sparsity and RIP | Lecture 20 |

21 | Tue Apr 2 | OMP and RIP | Lecture 21 | ||

22 | Thu Apr 4 | unscribed | |||

23 | Tue Apr 9 | MIDTERM | |||

24 | Thu Apr 11 | Dictionary Learning | Lecture 24 | ||

25 | Tue Apr 16 | Lecture 25 | |||

26 | Thu Apr 18 | Lecture 26 | |||

27 | Tue Apr 23 | Lecture 27 | |||

28 | Tue Apr 25 | Sparse and Low-rank Decompositions | unscribed | ||

29 | Tue Apr 30 | unscribed | |||

30 | Thu May 02 | unscribed | |||

Final Exam | Thu May 09 | in ENS 115 | 9:00-12:00 |