A Controlled Experiment Measuring the Effect of Procedure Argument Type Checking on Programmer Productivity

Type checking is considered an important mechanism for detecting programming errors, especially interface errors. This report describes an experiment to assess the error-detection capabilities of static intermodule type checking. The experiment uses ANSI C and Kernighan&Ritchie (K&R) C. The relevant difference is that the ANSI C compiler checks module interfaces (i.e., the parameter lists of calls to external functions), whereas K&R C does not. The experiment employs a counterbalanced design in which each subject writes two non-trivial programs that interface with a complex library (Motif). Each subject writes one program in ANSI C and one in K&R C. The input to each compiler run is saved and manually analyzed for errors. Results indicate that delivered ANSI C programs contain significantly fewer interface errors than delivered K&R C programs. Furthermore, after subjects have gained some familiarity with the interface they are using, ANSI C programmers remove errors faster and are more productive (measured in both time to completion and functionality implemented). This report describes the design, setup, and results of the experiment including complete source code and error lists.