The Tools of Programming

Update 6.2.2009: I've received the list of students for the class. If you took TOP with me last year, your wiki account is unchanged (contact me if you forgot your userid or password). Otherwise, your wiki account id is first initial concatenated with last name (e.g., Adnan Aziz is aaziz) and password is your uteid (please update it immediately). Assignment 0 is due this Friday (i.e., 6.5.2009).

Update 6.1.2008: I've been told I will have the roster tomorrow (6.2.2009 - Tuesday), and I'll create accounts and mail each of you as soon as I receive the roster.

Note: As of 5.29.2009, I still don't have a list of student enrolled in this class. I've updated the due dates for assignments for 2009, but can't create your Wiki pages, so Assignment 0 is on hold. I'll be away over the weekend (5.29-5.31) and not able to read mail.

Each of you will have a personal Wiki page with a password for uploading homework. Detailed submission notes are linked here.

This course is intended for software engineers who would like to learn how increase their productivity by learning about programming tools.

Programming is difficult—some of the problem developers face include

• How can a project be structured so that several developers can work on it concurrently?
• How can the building of a project be automated?
• How can a program be written to make it portable?
• How can a program be prototyped efficiently?
• How can a program be tested and debugged efficiently?
• How can the performance of a program be increased?

Using the right tools can solve these problems. Examples include tools for version control, documentation, program building and configuration, automatic testing, and program analysis.

Our approach will be to introduce a specific problem, show how the tool solves the problem, and the technical principles underlying the tool. We will use actual open-source tools that are freely available, and whose operation is completely transparent.

The formal course descriptor is linked here.

Have you ever…

• wasted time trying to merge your code with your project partners code?
• struggled trying to understand what your project partners code was doing?
• spent more time writing the code for reading data than for processing it?
• typed the same commands over and over again to build a project?
• pored over a file looking for certain patterns?
• tested a program but missed an obvious problem?
• spent a day looking for a memory bug the right analyzer could have found in five minutes?
• needed a program to run 10 times faster and use half the memory?

If so you, take this class!

The course will be based on the excellent text Essential Open Source Toolset, by Andreas Zeller and Jens Krinke, 2005, John-Wiley.

Your task in this class will be to read and understand ET. It is to the point (roughly 350 pages) and written. Each chapter includes examples, exercises, and further reading.

Here's a blurb from the book - ”This book offers indispensable guidance on how programming tools can be used to solve problems which occur regularly during the development of software. It will enable you to work with greater confidence and efficiency, and achieve better results.”

Option III Software Engineering graduate students should already have the programming background needed to appreciate ET.

Assignments will consist of a written component (e.g., “Your boss demands you code exclusively in Perl - tell him 5 dangers of this method”), as well as implementation. Implementation involves taking code distributed with the book, and modifying it in small ways. I expect that even if you are not comfortable with the underlying language, looking at the given code, together with the examples in the book, should be enough for you to do the implementation.

The tools used in the book are open source, and available for Unix/Linux, and Windows (e.g., via Cygwin).

I will assign homeworks consisting of exercises from ET. The homeworks will not be programming intensive—their purpose is to make you read ET in an organized manner. Grades will be assigned solely on the basis of performance on the homework.

HW is due at 11:59pm (Austin time) on the due date.

The book consists of 20 chapters, divided into 7 topics.

• Written homework is from the end of each chapter, i.e., the “review exercises”.
• Programming homework is from the exercises at the end of each topic, i.e., the exercises with the roman numbering.
  • Unless otherwise stated, you are responsible for the entire exercise.
• The number of points per question is given immediately after the question (e.g., 4.1/10 → question 4.1 is worth 10 marks). All points carry equal weight.

HW is to be submitted via uploading on the class wiki—please go through this exercise to make sure your account is working, and that you can upload/edit/create pages.

Also look quickly throught the programming component of the assignments, and make sure you have the tools. These tools are available on the LRC machines (e.g., balrog.ece.utexas.edu), are part of standard Linux installs, and you can install them on a Windows machine using Cygwin.

Due Friday, May 29, 2009

A very nice summary of version control

• Written homework: 2.4/5, 2.6/10, 3.2/5, 3.6/10, 4.2/10, 4.3/5, 4.4/10
• Exercise I/20 (page 61) - carry out program maintainence using CVS

Due Tuesday, June 9, 2009

An easy to read writeup on Yacc and Lex. Here is the source code. Another writeup that talks more about the internals of yacc/lex.

Here's a small example parser I wrote up that illustrates how you can have more than one type of result returned by the lexer. It does not use lex; here is the same example using a lex generated lexer.

• Written homework: 5.1/15, 5.6/5, 6.1/10, 6.4/15, 7.2/10, 7.6/10
• Exercise II/35 (page 119) - build programs for text analysis

Here is a richer calculator, with support for user-defined variables.

Some online guides to Yacc and Lex

• A compact guide to Lex and Yacc, by Tom Niemann
• Lexical analysis with Flex, by Paxson, Estes, and Millaway
• How do Lex and Yacc Work Internally
• Lex and Yacc primer/HOWTO (pdf)

Examples of lex and yacc source:

• AWK yacc grammar.
• CTL formula parser from VIS: lex, yacc
• BLIF-MV logic netlist format parser from VIS: lex,yacc
• You have to use the right flags to allow the two VIS parsers above to work correctly: here's the ctlp and io make commands

Due Sunday, June 21, 2009

Here is a nice short tutorial on make. The Doxygen website has comprehensive documentation. Take a look at Doxygen generated documentation for Postgres, an SQL server, and Azinix, a network monitoring and control system.

• Written homework: 8.2/5, 8.6/10, 8.7/10, 9.1/10, 9.2/5, 10.3/10
• Exercise III/30 (page 173) - automate the build of the sample program lcalc; document it using Javadoc/Doxygen
  • You are required to perform tasks 1–8 1–6 only
• Document a bitvector library with doxygen—details here

Due Sunday, June 28, 2009

Read John Ousterhout's classic classic paper on scripting. Also check out Brian Kernighan's succint language tutorials.

• Written homework: 11.1/10, 11.3/15
• Exercise IV (page 199) - scripting
  • I didn't like the scripting examples in Exercise IV, so I've made a new exercise: Parsing and analyzing with Perl (based on an example in “Perl: The Programmer's Companion”, by Nigel Chapman) 25 marks

Due Sunday, July 5, 2009

• Written homework: 12.1/10, 12.3/10, 13.1/10, 13.2/5, 13.3/5, 13.4/4, 13.5/5, 14.1/10, 14.2/5, 15.1/5
• Exercise V/30 (page 279) - test the lcalc program

Logging utilities for Java and C.

Source code: primecount.c, primelib.c, primelib.h

Due Sunday, July 12, 2009

• Written homework: 16.2/5, 16.3/5
• Exercise VI/20 (page 329) - measure performance and coverage on lcalc (parts 1-5 only)
  • There are a couple of typoes: specifically, the text should refer to Exercise II and Exercise VII. (You may find looking at Exercise VII useful too.)

Due Sunday, July 26, 2009

• Written homework: 20.1/5, 20.2/5
• Exercise VII/30 (page 363) - develop Rational example in Eclipse
  • Tasks 1–8 from “Introduction to Eclipse” (You do not have to do “Plugin development in ECLIPSE” tasks.)

Due Sunday, Aug 2, 2009

I teach another summer topics class, “The Practice of Programming” (PoP), which has been quite popular over the past few years. I developed ToP based on conversations with the students who took PoP. The two classes are meant to be complementary - PoP is more about broader principles, while ToP is focused on specific tools. For most of you, PoP would probably be the better one to take first, but they are self-contained, and you can take ToP first.

Adnan Aziz, , 1-(512) 475-9774, ACES 6.120, Office hours M 9-10 W 10-11