Advancing Retail Robotics: A Vision-Based Out of Stock Detection System for Retail Scenes without Planograms

 One of the fundamental problems faced by retailers in today’s world is the problem of quickly restockingproducts to ensure competitive sales for the store. As simple as it sounds, ensuring fully stocked shelves amidst the several other operations taking place at a working store at all times is an extremely challenging task and would need a lot of attention from the store staff. Over the years, retailers have resorted to track- ing the point of sale (POS) data or conducting manual audits to track the out-of-stock (OOS) products in a store which is both laborious and time-consuming. With the advancement in computer vision tech- nology, in this work, we explore the possibility of automating the detection of OOS in a store to provide the retailer with real-time OOS feedback about the aisles faster than the manual process. We propose a modularized perception system that processes full panoramic aisle images with object detection, instance segmentation, and image classification coupled with various other heuristic modules that perform the downstream task of OOS detection by leveraging associations between labels and products, unlike any other work in the literature to the best of our knowledge. The system operates independently of the store planograms while being robust to multiple fixtures. We further argue that the detection of misplaced products is an implicit issue that needs to be addressed to truly solve the OOS detection problem in retail stores. We run our tests on images from multiple stores of one of the retail giants of the world and prove the effectiveness of our approach.