ECE 382V Human Signals: Sensing and Analytics (Spring 2023)



Unique: 17710
Time: MW 3:00 to 4:30pm
Location: CPE 2.212

Instructor: Edison Thomaz (ethomaz at utexas dot edu, or contact through Canvas)
Office Hours: Wednesdays 1-2pm or by appointment
Office Hours Location: EER 7.818

TA: Rebecca Adaimi (rebecca dot adaimi at utexas dot edu, or contact through Canvas)
Office Hours: Mondays 4:30-5:30pm or by appointment
Office Hours Location: TBD

Online Forum: Piazza




Description

This course is aimed at graduate students and has 2 objectives. The first one is to teach concepts and practical skills for building systems that can sense and infer human signals (behavioral, physiological, emotional) and their respective context while leveraging mobile, ubiquitous and wearable computing technologies. The second aim is to examine and discuss advanced and emerging topics in the field in a seminar-style setting. Students will work on assignments throughout the semester, complete a project in a topic of their choosing, read and critique papers, present papers in class, and lead discussions. Key topics covered include machine learning fundamentals, activity recognition, sensing approaches (on-body, environmental), sensing modalities (e.g., inertial, acoustic, vision), sensor signal processing, and digital phenotyping.

Requirements

Academic: Graduate standing (or undergraduate with instructor approval). At a basic level, students will be expected to be comfortable using a high-level programming language. Experience with machine learning and related toolkits such as scikit-learn as well as mobile programming (iOS or Android) is useful but not required; we will review key concepts in the first part of the course. All the programming work in this class will be in Python and the Anaconda scientific package. If you are unsure if your background is a good match for this course, please come talk to the instructor.

Writing and Speaking: This course involves significant speaking and writing skills. Assignments must be completed in English. While primary assessment will focus on course material, correct spelling and grammar (along with coherence and logic) are expected and will be assessed in grading. Also, everyone is expected to participate in class and should be prepared for it. Class discussion each week is intended to reinforce understanding of the material. UT offers individual writing consultations to graduate students through the University Writing Center. Take advantage of these services, particularly before handing in project assignments (this of course requires starting early to get feedback from consultations and revise drafts accordingly).

Equipment: It is assumed that all students will have access to a computing system (i.e. laptop) to work on assignments. If you need assistance with computing resources, please contact the instructor.

Reading Materials

This class focuses on cutting-edge research in the field. Therefore, most of what we will read and discuss are recently-published papers. These will be provided to you by the instructor.


Tentative Schedule

Week Topic and Readings Presenter Assignment Project
Jan 9th Introduction
Jan 11th Machine Learning: Fundamentals
Jan 16th MLK Day (No class)
Jan 18th Machine Learning: Deep Learning
A Few Useful Things to Know about Machine Learning (Domingos)
Deep Learning (LeCun)
A1 Out
Jan 23rd ML: Methods and Evaluation
Meaningless Comparisons: False Optimism in Medical ML (DeMasi)
Jan 25th AR: Fundamentals
A Tutorial on Human Activity Recognition using Inertial Sensors (Bulling)
Jan 30th AR: Data and Validation
ReVibe: A Context-Assisted Recall Approach to Improve Self-Report (Rabbi)
NID A1 Due
Feb 1st Inertial Sensing
Large-Scale Physical Activity Data Reveal Worldwide Activity Inequality (Althoff)
YUY A2 Out
Feb 6th Inertial Sensing
Large-Scale Physical Activity Data Reveal Worldwide Activity Inequality (Althoff)
YUY
Feb 8th Inertial Sensing II
Are Accelerometers for Activity Recognition a Dead-end? (Tong)
SLS/STY
Feb 13th Acoustic Sensing I
SoundSense: Scalable Sound Sensing for People-Centric Applications (Lu)
NAG Team
Feb 15th Acoustic Sensing II
Audio-Based ADL Recognition with Large-Scale Acoustic Embeddings from Online Videos (Liang)
ZIT A2 Due
Feb 20th Acoustic Sensing III
Speaker Foreground Speech Detection from Audio in Workplace from Wearable Recorders (Nadarajan)
ANF A3 Out
Feb 22nd Vision Sensing I
Eyes on the Road: Detecting Phone Usage by Drivers Using On-Device Cameras (Khurana)
ASR
Feb 27th Vision Sensing II
Vid2Doppler: Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition (Ahuja)
TUP Proposal
Mar 1st Environmental Sensing
Activity Recognition in the Home Using Simple and Ubiquitous Sensors (Tapia)
ALZ
Mar 6th Multimodal Sensing
Multimodal Deep Learning for Activity and Context Recognition (Radu)
GEC/MAY A3 Due
Mar 8th Interactive Activity Recognition
Automated Class Discovery for Acoustic Activity Recognition (Wu)
SRS A4 Out
Mar 13th Spring Break (No class)
Mar 15th Spring Break (No class)
Mar 20th Lifelong Learning
Lifelong Learning in Sensor-Based Human Activity Recognition (Ye)
XIG/OLA
Mar 22nd Health I
Using Passively Collected Behavior to Predict Hospital Readmission (Bae)
KAM
Mar 27th Health II
Seismo: Blood Pressure Monitoring using Built-in Smartphone Accelerometer and Camera (Wang)
XIL
Mar 29th Health III
A Practical Approach for Recognizing Eating Moments (Thomaz)
DAJ/MAJ A4 Due Update
Apr 3rd Digital Phenotyping and Biomarkers
The Digital Phenotype (Jain)
YUS
Apr 5th Digital Phenotyping and Biomarkers II
Digital Health: Tracking Physiomes and Activity Using Wearable Biosensors (Li)
ALF
Apr 10th Privacy and Ethics
Mitigating Bystander Privacy Concerns in Egocentric AR (Dimiccoli)
ANM/JAF
Apr 12th Privacy and Ethics II
Ethical and Legal Issues of Ingestible Sensors (Gerke)
MAP
Apr 17th Novel Sensing and Methods
Intraoral Temperature and Inertial Sensing in Automated Dietary Assessment: A Feasibility Study (Chun)
KEM/HYS
Apr 19th Project Presentations Report
Apr 24th Project Presentations

Class Activities and Deliverables

At a high-level, there are 5 key activities and deliverables that students will be responsible for in this course:

Specific details about these activities and deliverables are provided in the sections below.

Paper Reading: Critique + Discussion Points

All students in the class will be expected to read the required paper assigned for each lecture. Additionally, each student will be expected to submit a one or two paragraph critique of the paper and two discussion points. A few discussion points will be selected for class discussion. These points could be elements of the paper that you did not understand or specific questions that emerged while you were reading the paper (e.g., about the methodology, instrumentation, user study, data analysis, motivation). To write your critique and come up with the two discussion points, here are some examples of questions you could ask yourself while reading and examining papers:

You should post your critique paragraphs using the following form. The deadline for these will always be 10am on the day of the lecture. You will be graded on your communication and the quality and depth of your critique and discussion points.

Perspective Papers: Presentation and Discussion

Twice during the semester, each student will be assigned a paper to present to the class. The presentation should be equivalent to a 10-minute conference-style talk, with 5 additional minutes for discussion, for a total of 15 minutes per paper. As the paper “expert”, the student presenter will be required to read the paper in detail and prepare (1) a set of slides to show in class, and (2) be ready to answer clarification questions. We will do our best to match students with their topic of choice; this assignment will be established in the first two weeks of class. Students should consider these presentations to be like exams. If a student is not available to present his or her assigned paper (without advance notification), the student will get zero credit for this portion of the course grade. Students should communicate with the instructor at least one week before the scheduled presentation if they will not be available to present on the assigned day.

Assignments

Students will work on four assignments during the semester. The assignments will be due approximately 10 days after they have been made available. The assignments will be on the topics of machine learning, inertial sensing, acoustic sensing and vision sensing.

Project

Students will be required to complete a class project, either individually or in pairs. Projects will let students choose a particular topic of their interest and study it in more depth. It is not expected to represent original research but students are encouraged to think creatively. It is ok to build on previous ideas and studies, but it is not appropriate to simply replicate previous work. Project ideas will be provided to you by the instructor. Three deliverables will be expected as part of the project: (1) a proposal, (2) a final report, and (3) a 10-minute conference-style presentation with slides. If forming a team, it is recommended that the team is in place 2 weeks before the proposal is due.

Proposal

The first project deliverable will be the project proposal, which should be no longer than 2 pages and include all the sections below. The proposal will be graded on the basis of completeness and clarity for each one of the sections, as well as novelty. Students should discuss their project and proposal with the instructor and/or the TA before submission.

Update

The project update should be no longer than 2 pages and include all the sections below. The goal of this document is to communicate how your project is progressing, whether you are running into unanticipated challenges, and what you are planning to do to mitigate these challenges.

Final Report

The report should not be longer than 10 pages (one-column) plus references, following the ACM double-column format. Graphs and images are ok. You may submit appendices which include design documents or other diagrams such as circuit layouts. These will not count towards the page limit. Refer to the papers we have read in class for pointers on how to present your work in writing. Links to the Latex and Word templates for this format can be found here. At a minimum, your paper must include the sections below:

Final Presentation

At the end of the semester our class will be dedicated to project presentations. Students will be expected to deliver a 15-minute conference-style presentation with slides and then answer questions afterwards for 5 minutes. The presentations will be expected to include a technical description of the project and motivation, methods useds, related work, and key take ways, e.g., what you learned. Clarity of communication and quality of the presentation slides will also be part of the grading criteria.

Class Participation

Students are expected to attend every lecture and participate in discussions. Participation is not optional; the instructor will actively engage with students throughout the semester. The class participation grade will be assigned to every student based on this engagement.

Grading

Here is a breakdown of how the final grade for each student will be computed:

Grade Distribution: A (90-100), B (80-89), C (70-79), D (60-69), Fail (<60)

Contesting Grades

Students may request a regrade on a deliverable only within one week of a grade being assigned.

Late Deliverables

Late deliverables will be accepted for one week after their due date. This will apply to assignments, the project proposal and the project update. Within this week, deliverables will be graded but 30 points (out of 100 points) will be automatically deducted. After one week, assignments will receive a zero. In the interest of fairness, there will not be any exceptions to this policy.

Absence Due to Illness

If you miss deadlines or presentations due to illness, please bring the instructor a doctor's note that indicates not only that you had a medical consultation but also reports the actual illness that was diagnosed. If such note is provided, the instructor will be happy to make accomodations and extend deadlines. Otherwise, all other policies apply (e.g., late deliverables).

Exams

There will not be a midterm or final exam for this course.

Sharing of Course Materials is Prohibited

No materials used in this class, including, but not limited to, lecture hand-outs, videos, assessments (quizzes, exams, papers, projects, homework assignments), in-class materials, review sheets, and additional problem sets, may be shared online or with anyone outside of the class unless you have my explicit, written permission. Unauthorized sharing of materials promotes cheating. It is a violation of the University’s Student Honor Code and an act of academic dishonesty. I am well aware of the sites used for sharing materials, and any materials found online that are associated with you, or any suspected unauthorized sharing of materials, will be reported to Student Conduct and Academic Integrity in the Office of the Dean of Students. These reports can result in sanctions, including failure in the course.

FERPA and Class Recordings

Class recordings, if available, are reserved only for students in this class for educational purposes and are protected under FERPA. The recordings should not be shared outside the class in any form. Violation of this restriction by a student could lead to Student Misconduct proceedings.

Standard UT Austin Course Information and Policies

Academic Honor Code: You are encouraged to discuss assignments with classmates, but anything submitted must reflect your own, original work. If in doubt, ask the instructor. Plagiarism and similar conduct represents a serious violation of UT's Honor Code and standards of conduct.

Students who violate University rules on academic dishonesty are subject to severe disciplinary penalties, such as automatically failing the course and potentially being dismissed from the University. Please do not take the risk. We are required to automatically report any suspected case to central administration for investigation and disciplinary hearings. Honor code violations ultimately harm yourself as well as other students, and the integrity of the University.

Academic honesty is strictly enforced.

Notice about students with disabilities: The University of Texas at Austin provides appropriate accommodations for qualified students with disabilities. Please check UT Disability and Access. If they certify your needs, we will work with you to make appropriate arrangements. Any students requiring assistance in evacuation must inform the instructor in writing of their needs during the first week of classes.

Coping with stress and personal hardships: The Counseling and Mental Health Center offers a variety of services for students, including both individual counselling and groups and classes, to provide support and assistance for anyone coping with difficult issues in their personal lives. As mentioned above, life brings unexpected surprises to all of us. If you are facing any personal difficulties in coping with challenges facing you, definitely consider the various services offered and do not be shy to take advantage of them if they might help. These services exist to be used.

Notice about missed work due to religious holy days: A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.

Electronic Mail Notification Policy: In this course e-mail, Canvas will be used as a means of communication with students. You will be responsible for checking your e-mail regularly for class work and announcements. If you are an employee of the University, your e-mail address in Canvas is your employee address.

The University has an official e-mail student notification policy. It is the student's responsibility to keep the University informed as to changes in his or her e-mail address. Students are expected to check e-mail on a frequent and regular basis in order to stay current with University-related communications, recognizing that certain communications may be time-critical.


Edison Thomaz © 2023