Edison Thomaz

Publications   Teaching   Vita   Prospective Students

EE382V Human Signals: Sensing and Analytics (Fall 2021)

Unique: 17135
Time: MW, 3:00 to 4:30pm
Location: Zoom (link available on Canvas) until Sept 20th. Then, CPE 2.212

Instructor: Edison Thomaz (ethomaz at utexas dot edu, or contact through Canvas)
Office Hours: Fridays 1-2pm or by appointment
Office Hours Location: EER O's Cafe patio (outside)

TA: Dawei Liang (dawei dot liang at utexas dot edu, or contact through Canvas)
Office Hours: Mondays 5-6pm or by appointment
Office Hours Location: TBD

Online Forum: Here is our Piazza site.


This course is aimed at gradute students and has 2 objectives. The first one is to teach concepts and practical skills for building systems that can sense and infer human signals (behavioral, physiological, emotional) and their respective context while leveraging mobile, ubiquitous and wearable computing technologies. The second aim is to examine and discuss advanced and emerging topics in the field in a seminar-style setting. Students will work on assignments throughout the semester, complete a project in a topic of their choosing, read and critique papers, present papers in class, and lead discussions. Key topics covered include machine learning fundamentals, activity recognition, sensing approaches (on-body, environmental), sensing modalities (e.g., inertial, acoustic, vision), sensor signal processing, and digital phenotyping.


We will be online until Sept 20th. During this time, classes will take place over Zoom and will be recorded. Students are encouraged to keep turn on their cameras in class. After Sept 20th, we will switch to in-person.


Academic: Graduate standing (or undergraduate with instructor approval). At a basic level, students will be expected to be comfortable using a high-level programming language. Experience with machine learning and related toolkits such as scikit-learn as well as mobile programming (iOS or Android) is useful but not required; we will review key concepts in the first part of the course. All the programming work in this class will be in Python and the Anaconda scientific package. If you are unsure if your background is a good match for this course, please come talk to the instructor.

Writing and Speaking: This course involves significant speaking and writing skills. Assignments must be completed in English. While primary assessment will focus on course material, correct spelling and grammar (along with coherence and logic) are expected and will be assessed in grading. Also, everyone is expected to participate in class and should be prepared for it. Class discussion each week is intended to reinforce understanding of material. UT offers individual writing consultations to graduate students. Take advantage of these services, particularly before handing in project assignments (this of course requires starting early to draft papers early enough to get feedback from consultations and revise drafts accordingly). Undergraduate students can make similar use of UT's Writing Center. The UT CELTA Center (Certificate in English Language Teaching to Speakers of Other Languages) offers free ESL classes at multiple levels.

Equipment: It is assumed that all students will have access to a computing system (i.e. laptop) to work on assignments. If you need assistance with computing resources, please contact the instructor.

Reading Materials

This class focuses on cutting-edge research in the field. Therefore, most of what we will read and discuss are recently-published papers. These will be provided to you by the instructor.

Tentative Schedule

Week Topic and Readings Presenter Assignment Project
Aug 25th Introduction A0 Out
Aug 30th Machine Learning: Fundamentals
How to Read a Paper (Keshav)
How to Read an Engineering Research Paper (Griswold)
Sept 1st Machine Learning: Deep Learning
A Few Useful Things to Know about Machine Learning (Domingos)
Deep Learning (LeCun)
A1 Out
Sep 6th Labor Day (No class)
Sep 8th ML: Methods and Evaluation
Meaningless Comparisons: False Optimism in Medical ML (DeMasi)
Sep 13th AR: Fundamentals
A Tutorial on Human Activity Recognition using Inertial Sensors (Bulling)
A1 Due
Sep 15th AR: Data and Validation
ReVibe: A Context-Assisted Recall Approach to Improve Self-Report (Rabbi)
Sep 20th Inertial Sensing
Large-Scale Physical Activity Data Reveal Worldwide Activity Inequality (Althoff)
A2 Out
Sep 22nd Inertial Sensing II
Are Accelerometers for Activity Recognition a Dead-end? (Tong)
Sept 27th Acoustic Sensing I
SoundSense: Scalable Sound Sensing for People-Centric Applications (Lu)
JL Team
Sept 29th Acoustic Sensing II
Audio-Based ADL Recognition with Large-Scale Acoustic Embeddings from Online Videos (Liang)
EK A2 Due
Oct 4th Acoustic Sensing III
Speaker Foreground Speech Detection from Audio in Workplace from Wearable Recorders (Nadarajan)
RA A3 Out
Oct 6th Vision Sensing I
Eyes on the Road: Detecting Phone Usage by Drivers Using On-Device Cameras (Khurana)
Oct 11th Vision Sensing II
Vid2Doppler: Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition (Ahuja)
JL Proposal
Oct 13th Environmental Sensing
Activity Recognition in the Home Using Simple and Ubiquitous Sensors (Tapia)
DM, AG A3 Due
Oct 18th Multimodal Sensing
Multimodal Deep Learning for Activity and Context Recognition (Radu)
SP A4 Out
Oct 20th Interactive Activity Recognition
Automated Class Discovery for Acoustic Activity Recognition (Wu)
Oct 25th Lifelong Learning
Lifelong Learning in Sensor-Based Human Activity Recognition (Ye)
Oct 27th Health I
Using Passively Collected Behavior to Predict Hospital Readmission (Bae)
DW A4 Due
Nov 1st Health II
Seismo: Blood Pressure Monitoring using Built-in Smartphone Accelerometer and Camera (Wang)
Nov 3rd Health III
A Practical Approach for Recognizing Eating Moments (Thomaz)
Nov 8th Digital Phenotyping and Biomarkers
The Digital Phenotype (Jain)
Nov 10th Digital Phenotyping and Biomarkers II
Digital Health: Tracking Physiomes and Activity Using Wearable Biosensors (Li)
Nov 15th Privacy and Ethics
Mitigating Bystander Privacy Concerns in Egocentric AR (Dimiccoli)
Nov 17th Privacy and Ethics II
Ethical and Legal Issues of Ingestible Sensors (Gerke)
Nov 22nd Novel Sensing and Methods
Intraoral Temperature and Inertial Sensing in Automated Dietary Assessment: A Feasibility Study (Chun)
Nov 24th Thanksgiving (No class)
Nov 29th Novel Sensing and Methods
A First Step Towards On-Device Monitoring of Body Sounds in the Wild (Tailor)
Dec 1st Project Presentations Report
Dec 6th Project Presentations

Class Activities and Deliverables

At a high-level, there are 5 key activities and deliverables that students will be responsible for in this course:

Specific details about these activities and deliverables are provided in the sections below.

Paper Reading: Critique + Discussion Points

All students in the class will be expected to read the required paper assigned for each lecture. Additionally, each student will be expected to submit a one or two paragraph critique of the paper and two discussion points. A few discussion points will be selected for class discussion. These points could be elements of the paper that you did not understand or specific questions that emerged while you were reading the paper (e.g., about the methodology, instrumentation, user study, data analysis, motivation). To write your critique and come up with the two discussion points, here are some examples of questions you could ask yourself while reading and examining papers:

You should post your critique paragraphs using the following form. The deadline for these will always be 10am on the day of the lecture. You will be graded on your communication and the quality and depth of your critique and discussion points.

Perspective Papers: Presentation and Discussion

Twice during the semester, each student will be assigned a paper to present to the class. The presentation should be equivalent to a 10-minute conference-style talk, with 5 additional minutes for discussion, for a total of 15 minutes per paper. As the paper “expert”, the student presenter will be required to read the paper in detail and prepare (1) a set of slides to show in class, and (2) be ready to answer clarification questions. We will do our best to match students with their topic of choice; this assignment will be established in the first two weeks of class. Students should consider these presentations to be like exams. If a student is not available to present his or her assigned paper (without advance notification), the student will get zero credit for this portion of the course grade. Students should communicate with the instructor at least one week before the scheduled presentation if they will not be available to present on the assigned day.


Students will work on four assignments during the semester. The assignments will be due approximately 10 days after they have been made available. The assignments will be on the topics of machine learning, inertial sensing, acoustic sensing and vision sensing.


Students will be required to complete a class project, either individually or in pairs. Projects will let students choose a particular topic of their interest and study it in more depth. It is not expected to represent original research but students are encouraged to think creatively. It is ok to build on previous ideas and studies, but it is not appropriate to simply replicate previous work. Project ideas will be provided to you by the instructor. Three deliverables will be expected as part of the project: (1) a proposal, (2) a final report, and (3) a 10-minute conference-style presentation with slides. If forming a team, it is recommended that the team is in place 2 weeks before the proposal is due.


The first project deliverable will be the project proposal, which should be no longer than 2 pages and include all the sections below. The proposal will be graded on the basis of completeness and clarity for each one of the sections, as well as novelty. Students should discuss their project and proposal with the instructor and/or the TA before submission.


At the end of the semester our class will be dedicated to project presentations. Students will be expected to deliver a 15-minute conference-style presentation with slides and then answer questions afterwards for 5 minutes. The presentations will be expected to include a technical description of the project and motivation, methods useds, related work, and key take ways, e.g., what you learned. Clarity of communication and quality of the presentation slides will also be part of the grading criteria.

Final Report

The report should not be longer than 10 pages (one-column) plus references, following the ACM double-column format. Graphs and images are ok. You may submit appendices which include design documents or other diagrams such as circuit layouts. These will not count towards the page limit. Refer to the papers we have read in class for pointers on how to present your work in writing. Links to the Latex and Word templates for this format can be found here. At a minimum, your paper must include the sections below:

Class Participation

Students are expected to attend every lecture and participate in discussions. Participation is not optional; the instructor will actively engage with students throughout the semester. The class participation grade will be assigned to every student based on this engagement.


Here is a breakdown of how the final grade for each student will be computed:

Grade Distribution: A (90-100), B (80-89), C (70-79), D (60-69), Fail (<60)

Late Deliverables

Late deliverables will be accepted for one week after their due date. Within this week, deliverables will be graded but 10 points (out of 100 points) will be automatically deducted. After one week, assignments will receive a zero. In the interest of fairness, there will not be any exceptions to this policy.

Absence Due to Illness

If you miss deadlines or presentations due to illness, please bring the instructor a doctor's note that indicates not only that you had a medical consultation but also reports the actual illness that was diagnosed. If such note is provided, the instructor will be happy to make accomodations and extend deadlines. Otherwise, all other policies apply (e.g., late deliverables).


There will not be a midterm or final exam for this course.

Sharing of Course Materials is Prohibited

No materials used in this class, including, but not limited to, lecture hand-outs, videos, assessments (quizzes, exams, papers, projects, homework assignments), in-class materials, review sheets, and additional problem sets, may be shared online or with anyone outside of the class unless you have my explicit, written permission. Unauthorized sharing of materials promotes cheating. It is a violation of the University’s Student Honor Code and an act of academic dishonesty. I am well aware of the sites used for sharing materials, and any materials found online that are associated with you, or any suspected unauthorized sharing of materials, will be reported to Student Conduct and Academic Integrity in the Office of the Dean of Students. These reports can result in sanctions, including failure in the course.

FERPA and Class Recordings

Class recordings are reserved only for students in this class for educational purposes and are protected under FERPA. The recordings should not be shared outside the class in any form. Violation of this restriction by a student could lead to Student Misconduct proceedings.

COVID-19 Guidance

To help preserve our in-person learning environment, the university recommends the following:

Standard UT Austin Course Information and Policies

Academic Honor Code: You are encouraged to discuss assignments with classmates, but anything submitted must reflect your own, original work. If in doubt, ask the instructor. Plagiarism and similar conduct represents a serious violation of UT's Honor Code and standards of conduct.

Students who violate University rules on academic dishonesty are subject to severe disciplinary penalties, such as automatically failing the course and potentially being dismissed from the University. Please do not take the risk. We are required to automatically report any suspected case to central administration for investigation and disciplinary hearings. Honor code violations ultimately harm yourself as well as other students, and the integrity of the University.

Academic honesty is strictly enforced.

Notice about students with disabilities: The University of Texas at Austin provides appropriate accommodations for qualified students with disabilities. To determine if you qualify, please contact the Dean of Students at 512-471-6529 or UT Services for Students with Disabilities. If they certify your needs, we will work with you to make appropriate arrangements.

Emergency Preparedness: Any students requiring assistance in evacuation must inform the instructor in writing of their needs during the first week of classes.

Coping with stress and personal hardships: The Counseling and Mental Health Center offers a variety of services for students, including both individual counselling and groups and classes, to provide support and assistance for anyone coping with difficult issues in their personal lives. As mentioned above, life brings unexpected surprises to all of us. If you are facing any personal difficulties in coping with challenges facing you, definitely consider the various services offered and do not be shy to take advantage of them if they might help. These services exist to be used.

Notice about missed work due to religious holy days: A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.

Electronic Mail Notification Policy: In this course e-mail, Canvas will be used as a means of communication with students. You will be responsible for checking your e-mail regularly for class work and announcements. If you are an employee of the University, your e-mail address in Canvas is your employee address.

The University has an official e-mail student notification policy. It is the student's responsibility to keep the University informed as to changes in his or her e-mail address. Students are expected to check e-mail on a frequent and regular basis in order to stay current with University-related communications, recognizing that certain communications may be time-critical.

Edison Thomaz © 2021