Controllable Text Generation And Ethical Implications

The 21st century is witnessing a major shift in the way people interact with technology and Natural Language Generation (NLG) is playing a central role. Users of smartphones and smart home devices now expect their gadgets to be aware of their situation, and to produce natural language outputs in interactions. This thesis identifies three aspects of human communication to make machines sound human-like - style, content and structure. This thesis provides deep learning solutions to controlling these variables in neural text generation. I first outline the various modules which could be manipulated to perform effective controllable text generation. I provide two novel solutions for style transfer – using back-translation technique, and tag and generate approach. I also introduce two new tasks for style transfer and provide datasets for further exploration – political slant transfer and politeness transfer. I establish the task of document grounded generation which leverages information from unstructured documents for the generation process. I introduce two new tasks for document grounded generation – Wikipedia Update generation and Document Grounded Dialogue Response generation. Furthermore, I build two new extensions to pre-trained encoder-decoder models to solve this task. I also design a new elegant solution for the sentence ordering task to learn effective document structures. For all three tasks of style transfer, document grounded generation and sentence order, I add importance to the human evaluation of the models. I introduce new human evaluation measures for understanding the notion of grounding and for understanding the quality of predictions in sentence ordering. At the end, I provide a discussion on the ethical considerations of the applications of controllable text generation. Specifically, I use deontological ethics to evaluate NLP systems and discuss how controllable text generation techniques can be used to make these systems ethical