Edison Thomaz

Publications   Teaching   Vita   Prospective Students

EE382V Activity Sensing and Recognition (Fall 2018)

Instructor: Edison Thomaz (ethomaz@utexas.edu, or contact through Canvas)

Time: TTH, 2PM to 3:30PM

Location: ECJ 1.314

Office Hours: Tuesdays 11AM (EER 7.818) or by appointment

TA: Sarnab Bhattacharya (sarnab2008@gmail.com, or contact through Canvas)

Office Hours: TBD

Online Forum: Piazza. Sign up here.


This course is aimed at gradute students and has 2 objectives. The first one is to teach concepts and practical skills for building systems that can sense and infer human activities, context and health measures while leveraging mobile, ubiquitous and wearable computing technologies. The second aim is to examine and discuss advanced and emerging topics in the field in a seminar-style setting. Students will be expected to work on a semester-long project, will be required to read and critique a large number of papers, and present some of the papers in class. Topics covered include fundamentals of activity recognition and machine learning, sensing approaches (on-body, environmental), sensing modalities (e.g., inertial, acoustic, vision-based), sensor signal processing and applications.


Academic: Graduate standing. Undergraduate course in high-level programming language. Basic knowledge of machine learning and experience with data acquisition and/or embedded computing is useful but not required. All the programming work in this class will be in Python and the Anaconda scientific package. The instructor and TAs are happy to help with Python issues during office hours and via Piazza. If you are unsure if your background is a good match for this course, please come talk to the instructor.

Writing and Speaking: This course involves significant speaking and writing skills. Assignments must be written in English and typeset. While primary assessment will focus on course material, correct spelling and grammar (along with coherence and logic) are expected and will be assessed in grading. Also, everyone is expected to talk in every class. Class discussion each week is intended to reinforce understanding of material. UT offers individual writing consultations to Graduate Students. Take advantage of these services, particularly before handing in project assignments (this of course requires starting early to draft papers early enough to get feedback from consultations and revise drafts accordingly). Undergraduate students can make similar use of UT's Writing Center. The UT CELTA Center (Certificate in English Language Teaching to Speakers of Other Languages) offers free ESL classes at multiple levels.

Equipment: It is assumed that all students will have access to a computing system (i.e. laptop) and also a smartphone device running either iOS or Android. If you do not own a smartphone, please reach out to the instructor or TA for assistance.


There is no textbook for the course. All reading materials will be provided by the instructor or TA either in paper copy or electronically (i.e., link to PDF). However, we recommend the following books if you would like to dive deeper into the (sub) areas this class covers:


Here is a breakdown of how the final grade for each student will be computed:

Grade Distribution: A (90-100), B (80-89), C (70-79), D (60-69), Fail (<60)

Paper Readings Critique + Discussion Questions

All students in the class will be expected to read the required paper assigned for each lecture. Additionally, each student will be expected to submit a two-paragraph critique and two discussion points. A few discussion points will be selected for a class discussion. These points could be elements of the paper that you did not understand or specific questions that emerged while you were reading the paper (e.g., about the methodology, instrumentation, user study, data analysis, motivation). To write your critique and come up with the two discussion points, here are some examples of questions you could ask yourself while reading and examining papers:

You should post your critique paragraphs and discussion points to the Canvas discussion forum thread that is associated with the corresponding lecture. The deadline for these will always be 11:59PM the day before the respective lecture. You will be graded on the quality of your critique and discussion points.

Paper Presentations

Once during the semester, each student will choose a paper from one of the assigned readings and present it to the class. The presentation should be equivalent to a conference-style presentation of no more than 15 minutes. As the paper “expert”, the student presenter will be required to read the paper in detail and prepare (1) a set of slides to show in class, and (2) two quiz-type questions related to the paper with non-obvious answers. The questions will be posed to the class before the presentations. Student-paper pairings will be established in the first two weeks of class and personal preferences will be taken into account when creating these pairing. If a student does not show up to present his or her assigned paper, it will not be possible to schedule a make-up date due to the lack of available dates. Consequently, the student will get zero credit for this portion of the course grade.


Students will be asked to complete three assignments during the semester. The assignments will be due 2 weeks after they have been made available.


A substantial part of the course grade will be attributed to the semester-long class project. Students will be responsible for forming teams composed of 2-4 students. Projects should be original research or at a minimum, have elements that are original. It is ok to build on previous ideas and studies, but it is not appropriate to simply replicate previous work. Project ideas will be provided to you by the instructor. During one of the lectures, you will have an opportunity to describe one or more ideas that you would like to work on and see if you can form a team around it. If you cannot convince others to work on your idea, it will be your responsibility to find and join another team. The expectation is that the project deliverables will be of publishable quality.

Project Proposal

The first project deliverable will be the project proposal, which should be no longer than 2 pages. All the sections outlined below should be included. The project plan you include in your proposal is just an initial plan, not a binding contract. It is your best guess initially. It is ok to revise and update it as your work progress and changes occur.

Project Progress Update

The project progress update should be no longer than 2 pages. All the sections outlined below should be included:

Project Final Presentation

The final class day will be dedicated to project presentations. Bring a demo of your project to class to present to the instructor and other students. Plan on a 10 minute presentation. If you cannot physically bring your project to class, bring enough material so that you can show us what you did. You may optionally bring slides or a poster to use in your demo. Don't just show us your required video; you must be able to talk and explain your project. You can use a short video clip as part of your presentation, but do not just play your full video. Turn in any presentation materials you produce (e.g., slides, video).

Project Final Report

The report should be 4-pages long following the ACM double-column format. You may submit appendices which include design documents or other diagrams such as circuit layouts. These will not count towards your four pages. You may submit a paper longer than 4 pages if you need to. However you will not receive more credit for doing so. And if you submit a long write-up that is redundant it may actually hurt your grade. Refer to the papers we have read in class for pointers on how to present your work in writing.

Links to the Latex and Word templates for this format can be found here:

The sections we would like to see in your writeup are:

Project Grading

For the project, specific deliverables are asked and should be completed to the best of your team's ability. Teams that complete all the project deliverables satisfactorily should expect a grade of 90%. The remaining 10% of the points are reserved for work that stands out in some way, in areas such as implementation, academic contribution, human-centered design, user study, performance, novelty of approach, etc.

Class Participation

During the semester, there will be many opportunities for students to participate in class activities. A class participation grade will be derived from the level of engagement observed in these types of in-class activities. Regular attendance is expected. If for whatever reason you are absent, it is your responsibility to find out what you missed that day. Note that attendance does factor into the final grade.

Use of Laptops and Mobile Devices

Use of laptops and mobile devices can be useful in class to take notes, look up information, etc. However, if you are engrossed in your device and not clearly giving attention to the speaker and participating, you will be asked to turn off your device. The instructor may also explicit ask you to not use your device(s) at specific times during lectures. Failure to comply will cause a drop in your class participation grade.

Late Deliverables

Late deliverables will be accepted for two weeks after their due date, but at a penalty of 10 points per week -- so failure to turn in an assignment at the due date results in an immediate 10 point penalty. After two weeks, assignments will receive a 0. In the interest of fairness, there will not be any exceptions to this policy.

Absence Due to Illness

If you miss deadlines or presentations due to illness, please bring the instructor a doctor's note that indicates not only that you had a medical consultation but also reports the actual illness that was diagnosed. If such note is provided, the instructor will be happy to make accomodations and extend deadlines. Otherwise, all other policies apply (e.g., late deliverables).


There will not be a midterm or final exam for this course.

Standard UT Austin Course Information and Policies

Academic Honor Code: You are encouraged to discuss assignments with classmates, but anything submitted must reflect your own, original work. If in doubt, ask the instructor. Plagiarism and similar conduct represents a serious violation of UT's Honor Code and standards of conduct.

Students who violate University rules on academic dishonesty are subject to severe disciplinary penalties, such as automatically failing the course and potentially being dismissed from the University. **PLEASE** do not take the risk. We are REQUIRED to automatically report any suspected case to central administration for investigation and disciplinary hearings. Honor code violations ultimately harm yourself as well as other students, and the integrity of the University.

Academic honesty is strictly enforced. For more information, see the Student Judicial Services site.

Notice about students with disabilities: The University of Texas at Austin provides appropriate accommodations for qualified students with disabilities. To determine if you qualify, please contact the Dean of Students at 512-471-6529 or UT Services for Students with Disabilities. If they certify your needs, we will work with you to make appropriate arrangements.

Emergency Preparedness: Any students requiring assistance in evacuation must inform the instructor in writing of their needs during the first week of classes.

Coping with stress and personal hardships: The Counseling and Mental Health Center offers a variety of services for students, including both individual counselling and groups and classes, to provide support and assistance for anyone coping with difficult issues in their personal lives. As mentioned above, life brings unexpected surprises to all of us. If you are facing any personal difficulties in coping with challenges facing you, definitely consider the various services offered and do not be shy to take advantage of them if they might help. These services exist to be used.

Notice about missed work due to religious holy days: A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.

Electronic mail Notification Policy: In this course e-mail, Canvas and Piazza will be used as a means of communication with students. You will be responsible for checking your e-mail regularly for class work and announcements. If you are an employee of the University, your e-mail address in Canvas is your employee address.

The University has an official e-mail student notification policy. It is the student's responsibility to keep the University informed as to changes in his or her e-mail address. Students are expected to check e-mail on a frequent and regular basis in order to stay current with University-related communications, recognizing that certain communications may be time-critical.

Tentative Schedule

Week Topic Readings Assignments Project
Aug 30th Course Intro How to Read a Paper (Keshav)
How to Read an Engineering Research Paper (Griswold)
Install Anaconda
Sept 4th Fundamentals The Computer for the 21st Century (Weiser)
Sept 6th Machine Learning Introduction to Machine Learning (Smola/Chapter 1)
Sep 11th Machine Learning A Few Useful Things to Know about Machine Learning (Domingos) A1 Out Team Formation
Sep 13th Machine Learning Deep Learning (LeCun)
Sep 18th Inertial Sensing Preprocessing techniques for context recognition from accelerometer data (Figo) A1 Due Proposal
Sep 20th Inertial Sensing Let’s (not) Stick Together: Pairwise Similarity Biases (Hammerla) A2 Out
Sep 25th Project Project Workday
Sep 27th Inertial Sensing Deep Learning for Human Activity Recognition in Mobile Computing (Ploetz) A2 Due
Oct 2nd Audio Sensing Cross-Platform Mobile Acoustic Sensing (Tung) A3 Out
Oct 4th Audio Sensing DeepEar (Lane)
Oct 9th Guest Lecture Elaine Short A3 Due
Oct 11th Guest Lecture Justin Hart
Oct 16th Vision Sensing ImageNet Classification with Deep Convolutional Neural Networks (Krizhevsky)
Oct 18th Vision Sensing Predicting Daily Activities From Egocentric Images Using Deep Learning (Castro)
Oct 23rd Environmental Synthetic Sensors (Laput)
Oct 25th Environmental Electrisense (Patel)
Oct 30th Health+Behavior Passively collected sedentary behavior to predict hospital readmission (Bae/Low)
Nov 1st Health+Behavior Predicting Tomorrow's Mood (Picard) Progress Update
Nov 6th Guest Lecture Prof. Christine Julien
Nov 8th Multimodal Sensing Extra-Sensory (Vaizman)
Multimodal Deep Learning (Radu/Kawsar)
Nov 13th Active Learning Smartwatch-Based Activity Recognition with Active Learning (Shahmohammadi)
Activity Classification using Active and Semi-Supervised Learning (Estrin)
Nov 15th Activity Discovery Unsupervised Activity Discovery with Behavior Assumption (Gjoreski/Roggen)
Dense Motif Discovery (Laerhoven)
Nov 20th Project Project Workday
Nov 22nd No Class Thanksgiving
Nov 27th Privacy Mitigating Bystander Privacy Concerns with Deep Learning (Dimiccoli)
Nov 29th Project Project Workday
Dec 4th Project Project Presentations Final Report
Dec 6th Project Project Presentations

Edison Thomaz © 2018