ECE 382V Human Signals: Sensing and Analytics (Spring 2024)



Unique: 17380
Time: TTH 11:00 to 12:30pm
Location: ECJ 1.314

Instructor: Edison Thomaz (ethomaz at utexas dot edu, or contact through Canvas)
Office Hours: Wednesdays 1-2pm or by appointment
Office Hours Location: EER 7.818

TA: Sloke Shrestha (sloke at utexas dot edu, or contact through Canvas)
Office Hours: Thursdays 2-3pm
Office Hours Location: EER 0.814

Online Forum: We will be using Ed Discussion, which you can access through Canvas.

Paper Critique Form: You should post your critique using the following form




Description

This course is aimed at gradute students and has 2 objectives. The first one is to teach concepts and practical skills for building systems that can sense and infer human signals (behavioral, physiological, emotional) and their respective context while leveraging mobile, ubiquitous and wearable computing technologies. The second aim is to examine and discuss advanced and emerging topics in the field in a seminar-style setting. Students will work on assignments throughout the semester, complete a project in a topic of their choosing, read and critique papers, present papers in class, and lead discussions. Key topics covered include machine learning fundamentals, activity recognition, sensing approaches (on-body, environmental), sensing modalities (e.g., inertial, acoustic, vision), sensor signal processing, and digital phenotyping.

Requirements

Academic: Graduate standing (or undergraduate with instructor approval). At a basic level, students will be expected to be comfortable using a high-level programming language. Experience with machine learning and related toolkits such as scikit-learn as well as mobile programming (iOS or Android) is useful but not required; we will review key concepts in the first part of the course. All the programming work in this class will be in Python and the Anaconda scientific package. If you are unsure if your background is a good match for this course, please come talk to the instructor.

Writing and Speaking: This course involves significant speaking and writing skills. Assignments must be completed in English. While primary assessment will focus on course material, correct spelling and grammar (along with coherence and logic) are expected and will be assessed in grading. Also, everyone is expected to participate in class and should be prepared for it. Class discussion each week is intended to reinforce understanding of the material. UT offers individual writing consultations to graduate students through the University Writing Center. Take advantage of these services, particularly before handing in project assignments (this of course requires starting early to get feedback from consultations and revise drafts accordingly).

Equipment: It is assumed that all students will have access to a computing system (i.e. laptop) to work on assignments. If you need assistance with computing resources, please contact the instructor.

Reading Materials

This class focuses on cutting-edge research in the field. Therefore, most of what we will read and discuss are recently-published papers. These will be provided to you by the instructor.


Tentative Schedule

Week Topic and Readings Presenter Assignment Project
Jan 16th Introduction N/A
Jan 18th Machine Learning: Fundamentals
Andrew Ng: Unbiggen AI
N/A
Jan 23rd Machine Learning: Deep Learning
Deep Learning (LeCun)
Optional: Representation Learning: A Review and New Perspectives (Bengio)
Optional: Deep Learning Is Hitting a Wall
N/A A1 Out
Jan 25th ML: Methods and Evaluation
A Few Useful Things to Know about Machine Learning (Domingos)
Optional: Meaningless Comparisons: False Optimism in Medical ML (DeMasi)
N/A
Jan 30th AR: Fundamentals
A Tutorial on Human Activity Recognition using Inertial Sensors (Bulling)
TZH
Feb 1st AR: Data and Validation
The Experience Sampling Method on Mobile Devices (Van Berkel)
TIH A1 Due
Feb 6th Inertial Sensing
AROMA: A Deep Multi-Task Learning Based Simple and Complex Human Activity Recognition Method Using Wearable Sensors (Peng)
AUJ A2 Out
Feb 8th Inertial Sensing II
Machine Learning Approaches to Predict Age from Accelerometer Records of Physical Activity at Biobank Scale (Le Goallec)
CAG
Feb 13th Inertial Sensing III
IMUTube: Automatic Extraction of Virtual On-Body Accelerometry from Video for Human Activity Recognition (Kwon)
JOC Team
Feb 15th Acoustic Sensing I
Audio-Based ADL Recognition with Large-Scale Acoustic Embeddings from Online Videos (Liang)
GEH A2 Due
Feb 20th Acoustic Sensing II
Automated Face-To-Face Conversation Detection on a Commodity Smartwatch with Acoustic Sensing (Liang)
DAJ A3 Out
Feb 22nd Vision Sensing I
Vid2Doppler: Doppler Radar Data from Videos for Training Privacy-Preserving Activity Recognition (Ahuja)
ELJ
Feb 27th Vision Sensing II
Wearable System for Personalized and Privacy-preserving Egocentric Visual Context Detection using On-device Deep Learning (Khan)
THIN/HAM Proposal
Feb 29th Environmental Sensing
Synthetic Sensors: Towards General-Purpose Sensing (Laput)
YEK
Mar 5th Multimodal Sensing
Leveraging Sound and Wrist Motion to Detect Activities of Daily Living with Commodity Smartwatches (Adaimi)
ROR/YUN A3 Due
Mar 7th Interactive Activity Recognition
Automated Class Discovery for Acoustic Activity Recognition (Wu)
THUN A4 Out
Mar 12th Spring Break (No class)
Mar 14th Spring Break (No class)
Mar 19th Lifelong Learning
Online Continual Learning for Human Activity Recognition (Schiemer)
GAK/LAA
Mar 21st Health I
GlucoScreen: A Smartphone-based Readerless Glucose Test Strip for Prediabetes Screening (Waghmare)
BOC
Mar 26th Health II
FitByte: Automatic Diet Monitoring in Unconstrained Situations Using Multimodal Sensing on Eyeglasses (Bedri)
WAC A4 Due
Mar 28th Health III
Seismo: Blood Pressure Monitoring using Built-in Smartphone Accelerometer and Camera (Wang)
VIS Update
Apr 2nd Digital Phenotyping and Biomarkers
Digital Health: Tracking Physiomes and Activity Using Wearable Biosensors (Li)
GRH
Apr 4th Digital Phenotyping and Biomarkers II
Wearable sensors enabling predictions of clinical lab measurements (Dunn)
ABB
Apr 9th Digital Phenotyping and Biomarkers III
Monitoring Changes in Depression Severity Using Wearable and Mobile Sensors (Pedrelli)
MEM
Apr 11th Privacy and Ethics
Mitigating Bystander Privacy Concerns in Egocentric AR (Dimiccoli)
HAN
Apr 16th Privacy and Ethics
Does Deidentification of Data from Wearable Devices Give us a False Sense of Security? A Systematic Review (Chikwetu)
AUU
Apr 18th Novel Sensing and Methods
Intraoral Temperature and Inertial Sensing in Automated Dietary Assessment: A Feasibility Study (Chun)
Apr 23rd Project Presentations Report
Apr 25th Project Presentations Report

Class Activities and Deliverables

At a high-level, there are 5 key activities and deliverables that students will be responsible for in this course:

Specific details about these activities and deliverables are provided in the sections below.

Paper Reading: Critique + Discussion Points

All students in the class will be expected to read the required paper assigned for each lecture. Additionally, each student will be expected to submit a one or two paragraph critique of the paper and two discussion points. A few discussion points will be selected for class discussion. These points could be elements of the paper that you did not understand or specific questions that emerged while you were reading the paper (e.g., about the methodology, instrumentation, user study, data analysis, motivation). To write your critique and come up with the two discussion points, here are some examples of questions you could ask yourself while reading and examining papers:

You should post your critique using the following form. The deadline for these will always be 7am on the day of the lecture. You will be graded on your communication and the quality and depth of your critique and discussion points.

Perspective Papers: Presentation and Discussion

Once during the semester, each student will be assigned a paper to present to the class. The presentation should be equivalent to a 10-minute conference-style talk, with 5 additional minutes for discussion, for a total of 15 minutes per paper. As the paper “expert”, the student presenter will be required to read the paper in detail and prepare (1) a set of slides to show in class, and (2) be ready to answer clarification questions. We will do our best to match students with their topic of choice; this assignment will be established in the first two weeks of class. Students should consider these presentations to be like exams. If a student is not available to present his or her assigned paper (without advance notification), the student will get zero credit for this portion of the course grade. Students should communicate with the instructor at least one week before the scheduled presentation if they will not be available to present on the assigned day.

Assignments

Students will work on four assignments during the semester. The assignments will be due approximately 10 days after they have been made available. The assignments will be on the topics of machine learning, inertial sensing, acoustic sensing and vision sensing.

Project

Students will be required to complete a class project, either individually or in pairs. Projects will let students choose a particular topic of their interest and study it in more depth. It is not expected to represent completely original research but students are encouraged to think creatively. It is ok to build on previous ideas and studies. Project ideas will be provided to you by the instructor. Three deliverables will be expected as part of the project: (1) a proposal, (2) a final report, and (3) a 15minute conference-style presentation with slides. If forming a team of 2, it is recommended that the team is in place 2 weeks before the proposal is due.

Proposal

The first project deliverable will be the project proposal, which should be no longer than 2 pages and include all the sections below. The proposal will be graded on the basis of completeness and clarity for each one of the sections, as well as novelty. Students should discuss their project and proposal with the instructor and/or the TA before submission.

Progress Report

The project update should be no longer than 2 pages and include all the sections below. The goal of this document is to communicate how your project is progressing, whether you are running into unanticipated challenges, and what you are planning to do to mitigate these challenges.

Final Report

The report should not be longer than 10 pages (one-column) plus references, following the ACM double-column format. Graphs and images are ok. You may submit appendices which include design documents or other diagrams such as circuit layouts. These will not count towards the page limit. Refer to the papers we have read in class for pointers on how to present your work in writing. Links to the Latex and Word templates for this format can be found here. At a minimum, your paper must include the sections below:

Final Presentation

At the end of the semester our class will be dedicated to project presentations. Students will be expected to deliver a 15-minute conference-style presentation with slides and then answer questions afterwards for 5 minutes. The presentations will be expected to include a technical description of the project and motivation, methods useds, related work, and key take ways, e.g., what you learned. Clarity of communication and quality of the presentation slides will also be part of the grading criteria.

Class Participation

Students are expected to attend every lecture and participate in discussions. Participation is not optional; the instructor will actively engage with students throughout the semester. The class participation grade will be assigned to every student based on this engagement.

Grading

Here is a breakdown of how the final grade for each student will be computed:

Grade Distribution: A (90-100), B (80-89), C (70-79), D (60-69), Fail (<60)

Contesting Grades

Students may request a regrade on a deliverable only within one week of a grade being assigned.

Late Deliverables

Late deliverables will be accepted for 3 days after their due date but 30 points (out of 100 points) will be automatically deducted. This will apply to the assignments, the project proposal and the project report. After 3 days, assignments will receive a zero. In the interest of fairness, there will not be any exceptions to this policy.

Absences and/or Missed Deadlines Due to Illness

If you miss deadlines or presentations due to illness, please bring the instructor a doctor's note that indicates not only that you had a medical consultation but also reports the actual illness that was diagnosed. If such note is provided, the instructor will be happy to make accomodations and extend deadlines. Otherwise, all other policies apply (e.g., late deliverables).

Absences and/or Missed Deadlines Due to Other Reasons

It is understandable that students might need to travel due to conferences, interviews, legal matters, or other reasons usually tied to their academic work or professional development. Extensions for these reasons are exceptional but might be made at the discretion of the instructor upon documentation. Even in these circumstances, students can usually plan well in advance to avoid missing deadlines. Extensions for personal reasons will not be granted.

Exams

There will not be a midterm or final exam for this course.

Sharing of Course Materials is Prohibited

No materials used in this class, including, but not limited to, lecture hand-outs, videos, assessments (quizzes, exams, papers, projects, homework assignments), in-class materials, review sheets, and additional problem sets, may be shared online or with anyone outside of the class unless you have my explicit, written permission. Unauthorized sharing of materials promotes cheating. It is a violation of the University’s Student Honor Code and an act of academic dishonesty. I am well aware of the sites used for sharing materials, and any materials found online that are associated with you, or any suspected unauthorized sharing of materials, will be reported to Student Conduct and Academic Integrity in the Office of the Dean of Students. These reports can result in sanctions, including failure in the course.

FERPA and Class Recordings

Class recordings, if available, are reserved only for students in this class for educational purposes and are protected under FERPA. The recordings should not be shared outside the class in any form. Violation of this restriction by a student could lead to Student Misconduct proceedings.

Standard UT Austin Course Information and Policies

Academic Honor Code: You are encouraged to discuss assignments with classmates, but anything submitted must reflect your own, original work. If in doubt, ask the instructor. Plagiarism and similar conduct represents a serious violation of UT's Honor Code and standards of conduct.

Students who violate University rules on academic dishonesty are subject to severe disciplinary penalties, such as automatically failing the course and potentially being dismissed from the University. Please do not take the risk. We are required to automatically report any suspected case to central administration for investigation and disciplinary hearings. Honor code violations ultimately harm yourself as well as other students, and the integrity of the University.

Academic honesty is strictly enforced.

Notice about students with disabilities: The University of Texas at Austin provides appropriate accommodations for qualified students with disabilities. Please check UT Disability and Access. If they certify your needs, we will work with you to make appropriate arrangements. Any students requiring assistance in evacuation must inform the instructor in writing of their needs during the first week of classes.

Coping with stress and personal hardships: The Counseling and Mental Health Center offers a variety of services for students, including both individual counselling and groups and classes, to provide support and assistance for anyone coping with difficult issues in their personal lives. As mentioned above, life brings unexpected surprises to all of us. If you are facing any personal difficulties in coping with challenges facing you, definitely consider the various services offered and do not be shy to take advantage of them if they might help. These services exist to be used.

Notice about missed work due to religious holy days: A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.

Electronic Mail Notification Policy: In this course e-mail, Canvas will be used as a means of communication with students. You will be responsible for checking your e-mail regularly for class work and announcements. If you are an employee of the University, your e-mail address in Canvas is your employee address.

The University has an official e-mail student notification policy. It is the student's responsibility to keep the University informed as to changes in his or her e-mail address. Students are expected to check e-mail on a frequent and regular basis in order to stay current with University-related communications, recognizing that certain communications may be time-critical.


Edison Thomaz © 2023