Edison Thomaz

Publications   Teaching   Vita   Prospective Students

EE382V Activity Sensing and Recognition (Fall 2020)

Unique: 16740 (Hybrid), 16743 (Two-Way Interactive), 16744 (Online)
Time: MW, 1:30PM to 3PM
Location: Online (the Zoom link is available on Canvas)

Instructor: Edison Thomaz (ethomaz at utexas dot edu, or contact through Canvas)
Office Hours: Wednesdays 3-4PM (the Zoom link is available on Canvas)

TA: Rebecca Adaimi (rebecca dot adaimi at utexas dot edu, or contact through Canvas)
Office Hours: Tuesdays 3-4PM (the Zoom link is available on Canvas)

Online Forum: Here is our Piazza site. Sign up here.

Description

This course is aimed at gradute students and has 2 objectives. The first one is to teach concepts and practical skills for building systems that can sense and infer human activities and context while leveraging mobile, ubiquitous and wearable computing technologies. The second aim is to examine and discuss advanced and emerging topics in the field in a seminar-style setting. Students will work on assignments throughout the semester, complete a project in a topic of their choosing, read and critique papers, present some of the papers in class, and lead discussions. Topics covered include fundamentals of activity recognition and machine learning, sensing approaches (on-body, environmental), sensing modalities (e.g., inertial, acoustic, vision-based), sensor signal processing and applications.

Requirements

Academic: Graduate standing (or undergraduate with instructor approval). At a basic level, students will be expected to be comfortable using a high-level programming language, and have experience with machine learning and related toolkits such as scikit-learn; we will review key concepts in the first part of the course. All the programming work in this class will be in Python and the Anaconda scientific package. If you are unsure if your background is a good match for this course, please come talk to the instructor.

Writing and Speaking: This course involves significant speaking and writing skills. Assignments must be completed in English and typeset. While primary assessment will focus on course material, correct spelling and grammar (along with coherence and logic) are expected and will be assessed in grading. Also, everyone is expected to participate in class. Class discussion each week is intended to reinforce understanding of material. UT offers individual writing consultations to Graduate Students. Take advantage of these services, particularly before handing in project assignments (this of course requires starting early to draft papers early enough to get feedback from consultations and revise drafts accordingly). Undergraduate students can make similar use of UT's Writing Center. The UT CELTA Center (Certificate in English Language Teaching to Speakers of Other Languages) offers free ESL classes at multiple levels.

Equipment: It is assumed that all students will have access to a computing system (i.e. laptop) to work on assignments. If you need assistance with computing resources, please contact the instructor.

Textboook

There is no textbook for the course. All reading materials will be provided by the instructor.

Tentative Schedule

Week Topic Readings or Activity Pres. Assign. Project
Aug 26th Course Introduction
Aug 31st Machine Learning How to Read a Paper (Keshav)
How to Read an Engineering Research Paper (Griswold)
The Computer for the 21st Century (Weiser)
Towards a Better Understanding of Context and Context-Awareness (Dey)
Sept 2nd Fundamentals A Few Useful Things to Know about Machine Learning (Domingos)
Deep Learning (LeCun)
A Tutorial on Human Activity Rec. using Body-Worn Inertial Sensors (Bulling)
Sep 7th Labor Day (No class)
Sep 9th Inertial Sensing I A Practical Approach for Recognizing Eating Moments (Thomaz) A1 Out
Sep 14th Inertial Sensing II Let’s (not) Stick Together: Pairwise Similarity Biases (Hammerla) GD
Sep 16th Audio Sensing I BodyScope: A Wearable Acoustic Sensor for Activity Recognition (Yatani) AP
Sep 21st Audio Sensing II SoundSense: Scalable Sound Sensing for People-Centric Applications (Lu) HC A1 Due
Sep 23rd Audio Sensing III Crowd++ Unsupervised Speaker Sount with Smartphones (Xu) DL A2 Out
Sept 28th Vision Sensing Predicting Daily Activ. From Egocentric Images Using Deep Learn. (Castro) SW, GR
Sept 30th Location and Mobility Placer: Semantic Place Labels from Diary Data (Krumm) GD, SH
Oct 5th Environmental I Synthetic Sensors: Towards General-Purpose Sensing (Laput) GL A2 Due
Oct 7th Environmental II HydroSense: IMS of Whole-Home Water Activity (Froehlich) SH A3 Out
Oct 12th Data Processing I ReVibe: A Context-Assisted Recall Approach to Improve Self-Report (Rabbi) AP, UG Team
Oct 14th Data Processing II Real-Time Crowd Labeling for Deployable Activity Recognition (Lasecki) JW
Oct 19th Discovery I Improving Class. Using Active and Semi-Supervised Learning (Longstaff) DL, HB A3 Due
Oct 21st Discovery II Multi-Task Self-Supervised Learning for Human Activity Detection (Saeed) GR, GZ Proposal
Oct 26th Interactive AR I Automated Class Discovery for Acoustic Activity Recognition (Wu) CK, MY
Oct 28th Interactive AR II Extrasensory: Data Collection In-the-Wild with Rich UI (Vaizman) CF, NR
Nov 2nd Adaptive Learning StreamAR: Incremental and Active Learning (Abdallah) UG, YZ
Nov 4th Health I FitByte: Automatic Diet Monitoring in Unconstrained Situations (Bedri) SH, GZ
Nov 9th Health II Using Passively Collected Behavior to Predict Hospital Readmission (Bae) CF, NR
Nov 11th Health III StudentLife: Assessing Mental Health Using Smartphones (Wang) SH
Nov 16th Privacy I Mitigating Bystander Privacy Concerns in Egocentric AR (Dimiccoli) HC, CK
Nov 18th Privacy II Sound Shredding: Privacy Preserved Audio Sensing (Kumar) SW, HB
Nov 23rd HCI GesturePod: Gesture-based Interaction for White Cane Users (Patil) GL, JW, YZ
Nov 25th Thanksgiving (No class)
Nov 30th Project Project Presentations
Dec 2nd Project Project Presentations
Dec 7th Project Project Presentations Web Page

Paper Readings Critique + Discussion Questions

All students in the class will be expected to read the required paper assigned for each lecture. Additionally, each student will be expected to submit a two-paragraph critique of the paper and two discussion points. A few discussion points will be selected for class discussion. These points could be elements of the paper that you did not understand or specific questions that emerged while you were reading the paper (e.g., about the methodology, instrumentation, user study, data analysis, motivation). To write your critique and come up with the two discussion points, here are some examples of questions you could ask yourself while reading and examining papers:

You should post your critique paragraphs and discussion points to the Canvas discussion forum thread that is associated with the corresponding lecture. The deadline for these will always be 11:59PM the day before the respective lecture. You will be graded on the quality and depth of your critique and discussion points.

Paper Presentations

Twice during the semester, each student will be assigned a paper to present to the class. The presentation should be equivalent to a 10-minute conference-style talk, with 5 additional minutes for discussion, for a total of 15 minutes per paper. As the paper “expert”, the student presenter will be required to read the paper in detail and prepare (1) a set of slides to show in class, and (2) be ready to answer clarification questions. Student-paper pairings will be established in the first two weeks of class and personal preferences will be taken into account when creating these pairings. If a student is not available to present his or her assigned paper, it will not be possible to schedule a make-up date due to the lack of available dates. Consequently, the student will get zero credit for this portion of the course grade.

Assignments

Students will be asked to complete three assignments during the semester. The assignments will be due 2 weeks after they have been made available.

Project

Students will also complete a class project; they will be able to work either individually or in pairs. Projects will let students choose a particular topic of their interest and study it in more depth. It is not expected to represent original research but students are encouraged to think creatively. It is ok to build on previous ideas and studies, but it is not appropriate to simply replicate previous work. Project ideas will be provided to you by the instructor. Four deliverables will be expected as part of the project: (1) a project proposal, (2) a project web page, and (3) a 10-minute conference-style presentation with slides.

Proposal

The first project deliverable will be the project proposal, which should be no longer than 2 pages and include all the sections below. The proposal will be graded on the basis of completeness and clarity for each one of the sections, as well as novelty.

Final Presentation

The last few lectures will be dedicated to project presentations. Students will be expected to deliver a 10-minute conference-style presentation with slides and then answer questions afterwards for 3-5 minutes. The presentations will be expected to include a technical description of the project and motivation, methods useds, related work, and key take ways, e.g., what you learned. Clarity of communication and quality of the presentation slides will also be part of the grading criteria.

Web Page

Students will be expected to turn in a web page documenting their projects in technical detail. At a minimum, we expect the sections below to be included on the page. Any resource can be added to the page to enrich the presentation of the project, e.g., images and videos.

Class Participation

Students are expected to attend every lecture and we will keep track of attendance. During the semester, there will be many opportunities to actively engage in class. The class participation grade will be derived from overall engagement throughout the semester and engagement during project presentations in particular.

Grading

Here is a breakdown of how the final grade for each student will be computed:

Grade Distribution: A (90-100), B (80-89), C (70-79), D (60-69), Fail (<60)

Late Deliverables

Late deliverables will be accepted for two weeks after their due date, but at a penalty of 10 points per week -- so failure to turn in an assignment at the due date results in an immediate 10 point penalty. After two weeks, assignments will receive a 0. In the interest of fairness, there will not be any exceptions to this policy.

Absence Due to Illness

If you miss deadlines or presentations due to illness, please bring the instructor a doctor's note that indicates not only that you had a medical consultation but also reports the actual illness that was diagnosed. If such note is provided, the instructor will be happy to make accomodations and extend deadlines. Otherwise, all other policies apply (e.g., late deliverables).

Exams

There will not be a midterm or final exam for this course.

Safety and Class Participation

Safety and Class Participation/Masks: We will all need to make some adjustments in order to benefit from in-person classroom interactions in a safe and healthy manner. Our best protections against spreading COVID-19 on campus are masks (defined as cloth face coverings) and staying home if you are showing symptoms. Therefore, for the benefit of everyone, this is means that all students are required to follow these important rules.

Sharing of Course Materials is Prohibited

No materials used in this class, including, but not limited to, lecture hand-outs, videos, assessments (quizzes, exams, papers, projects, homework assignments), in-class materials, review sheets, and additional problem sets, may be shared online or with anyone outside of the class unless you have my explicit, written permission. Unauthorized sharing of materials promotes cheating. It is a violation of the University’s Student Honor Code and an act of academic dishonesty. I am well aware of the sites used for sharing materials, and any materials found online that are associated with you, or any suspected unauthorized sharing of materials, will be reported to Student Conduct and Academic Integrity in the Office of the Dean of Students. These reports can result in sanctions, including failure in the course.

FERPA and Class Recordings

Class recordings are reserved only for students in this class for educational purposes and are protected under FERPA. The recordings should not be shared outside the class in any form. Violation of this restriction by a student could lead to Student Misconduct proceedings.

COVID-19 Guidance and Caveats

To help keep everyone at UT and in our community safe, it is critical that students report COVID-19 symptoms and testing, regardless of test results, to University Health Services, and faculty and staff report to the HealthPoint Occupational Health Program (OHP) as soon as possible. Please see this link to understand what needs to be reported. In addition, to help understand what to do if a fellow student in the class (or the instructor or TA) tests positive for COVID, see this University Health Services link.

Standard UT Austin Course Information and Policies

Academic Honor Code: You are encouraged to discuss assignments with classmates, but anything submitted must reflect your own, original work. If in doubt, ask the instructor. Plagiarism and similar conduct represents a serious violation of UT's Honor Code and standards of conduct.

Students who violate University rules on academic dishonesty are subject to severe disciplinary penalties, such as automatically failing the course and potentially being dismissed from the University. **PLEASE** do not take the risk. We are REQUIRED to automatically report any suspected case to central administration for investigation and disciplinary hearings. Honor code violations ultimately harm yourself as well as other students, and the integrity of the University.

Academic honesty is strictly enforced. For more information, see the Student Judicial Services site.

Notice about students with disabilities: The University of Texas at Austin provides appropriate accommodations for qualified students with disabilities. To determine if you qualify, please contact the Dean of Students at 512-471-6529 or UT Services for Students with Disabilities. If they certify your needs, we will work with you to make appropriate arrangements.

Emergency Preparedness: Any students requiring assistance in evacuation must inform the instructor in writing of their needs during the first week of classes.

Coping with stress and personal hardships: The Counseling and Mental Health Center offers a variety of services for students, including both individual counselling and groups and classes, to provide support and assistance for anyone coping with difficult issues in their personal lives. As mentioned above, life brings unexpected surprises to all of us. If you are facing any personal difficulties in coping with challenges facing you, definitely consider the various services offered and do not be shy to take advantage of them if they might help. These services exist to be used.

Notice about missed work due to religious holy days: A student who misses an examination, work assignment, or other project due to the observance of a religious holy day will be given an opportunity to complete the work missed within a reasonable time after the absence, provided that he or she has properly notified the instructor. It is the policy of the University of Texas at Austin that the student must notify the instructor at least fourteen days prior to the classes scheduled on dates he or she will be absent to observe a religious holy day. For religious holy days that fall within the first two weeks of the semester, the notice should be given on the first day of the semester. The student will not be penalized for these excused absences, but the instructor may appropriately respond if the student fails to complete satisfactorily the missed assignment or examination within a reasonable time after the excused absence.

Electronic mail Notification Policy: In this course e-mail, Canvas and Piazza will be used as a means of communication with students. You will be responsible for checking your e-mail regularly for class work and announcements. If you are an employee of the University, your e-mail address in Canvas is your employee address.

The University has an official e-mail student notification policy. It is the student's responsibility to keep the University informed as to changes in his or her e-mail address. Students are expected to check e-mail on a frequent and regular basis in order to stay current with University-related communications, recognizing that certain communications may be time-critical.

Edison Thomaz © 2020