Requirements for Robust Natural Language Interfaces: The LanguageCraft and XCALIBUR experiences

Natural Language interfaces to data bases and expert systems require the integration of several crucial capabilities in order to be judged habitable by their end users and productive by the developers of applications. User habitability is measured in terms of linguistic coverage, robustness of behavior and speed of response, whereas implementer productivity is measured by the amount of effort required to connect the interface to a new application, to develop its syntactic and semantic grammar, and to test and test and debug the resultant system assuring a certain level of performance• These latter criteria have not been addressed directly by natural language researchers in pure laboratory settings, with the exception of user-defined extensions to an existing interface (e.g., NanoKLAUS [4], vex [6]). But, in order to amortize the cost of developing practical, robust and efficient interfaces over multiple applications, the implementer productivity requirements are as important as user habitability. We treat each set of criteria in turn, drawing from our experience in XCALIBUR [2] and in LanguageCraft TM [5], a commercially available environment and run time module for rapid development of domain-oriented natural language interfaces. In our discussion we distill the general lessons accrued from several years of experience using these systems, and conducting several small-scale user studies.