Grounded Natural Language Generation via Interpretable Hierarchical Operations

Much recent work in natural language generation has relied on deep learning, often using neural net works with soft attention mechanisms to select salient aspects from data and then construct fluent natural  language text. However, in naturally occurring descriptions of data, humans often refer to higher-level  patterns which may require complex computations on data. In many cases, neural models using soft  attention mechanisms alone struggle to extract such patterns. Moreover, users might often find such  models to be difficult to interpret and control. In this thesis, I propose methods for inducing certain types  of discrete hierarchical operations on data and text for grounded natural language generation. Compared  to using attention alone, such hierarchical operations can better model complex patterns in data, expose  interpretable intermediate computations, and enable controllable generation. In the first half of the the sis, I will discuss adding specific discrete hierarchical operations to neural models for different grounded  natural language generation tasks, such as image and table captioning, dialog response generation, and  constructing reasoning chains for multi-hop question-answering. These tasks span various data modal ities (including images, tabular data, numerical data, and knowledge bases). In the second half, I will  describe hierarchical methods for content planning in text decoders, studying rhyming patterns in poetry  generation and discrete plans for coherent narrative text generation.