Time-ordered Online Training of Convolutional Neural Networks

Learned algorithms, especially deep neural networks, have proliferated in recent years, primarily due to their adaptability and performance. For applications where humans are required to monitor a datastream, e.g., a video feed from a UAS, such algorithms can greatly reduce the cognitive load required of human operators. But these algorithms are learned, not learning. After deployment to an end device such as a UAS, they are generally unchanged until the next software update. This makes them inflexible to changing
requirements and scenarios, a sharp contrast to adaptable human operators. Furthermore, retraining requires data collection and annotation—sometimes upwards of thousands of images—and must be done offline, not while a particular mission is unfolding. In this work, we tackle the problem of online, real-time training of a deep learning algorithm. We refer to this type of problem as time-ordered online training (ToOT), of which the UAS use case is an example. We begin with the observation that in human students, learning requires both study and curiosity. A good
learner is not only good at extracting information from the data given to it, but also skilled at finding the right new information to learn from. They may have help from a mentor or teacher, but must use this assistance wisely and effectively. For a real-time learning algorithm onboard a UAS, this means being able to learn from an incoming video feed while minimizing the human annotations and time required to do so, and moving around the environment to augment its learning. We define a metric, incremental training
benefit (ITB) per annotation, that seeks to capture the value extracted for each annotation provided by the user.
We present a combined system that addresses both study and curiosity when deployed onboard a small UAS. This system has two components: ClickBAIT and autonomous curiosity. ClickBAIT is a framework designed to enable real-time training of a learning algorithm from mouse clicks on a video feed. We apply this framework to classification as well as detection models, showing that it increases the ITB of
each annotation by 3-7 times across both cases. Furthermore, we show that such techniques can be used
to enable learning from a separate low-power sensor network, without clicks at all. Autonomous curiosity
is a system, trained through deep reinforcement learning, that accomplishes the task of both guiding the UAS’ motion and choosing when to ask for annotations from the human operator. Our trained approach is able to increase average ITB by 10-14 times versus a random search. In addition, we examine how autonomous curiosity agents can be trained to consider the time-cost as well as the annotation cost of its
actions, adapting its behavior to optimize for the characteristics of a particular UAS.