A Bio-sensing and Reinforcement Learning Control System for Personalized Thermal Comfort and Energy Efficiency

A comfortable indoor thermal environment plays a crucial role in preserving occupant health and productivity. In most office building today, the indoor thermal environment is regulated by heating, cooling, and air-conditioning
(HVAC) systems with static schedule-based rules. While prevalent, this control strategy has resulted in low thermal satisfaction rates and energy waste. A growing number of researchers are focusing on occupant-centric building
controls and applying various advanced control methods to improve thermal comfort and energy efficiency. However, it is still challenging to integrate occupants’ personalized requirements into a control system with a capability
of learning from the environment. This thesis has developed a bio-sensing and reinforcement learning control system for continuously integrating occupants’ bio-signals into the operation of different heating, cooling, and ventilation
systems, learning through interaction to achieve personalized thermal comfort and energy savings.
A bio-sensing and reinforcement learning control (Bio-REAL) system is comprised of a bio-sensing network, multiple Bio-REAL agents, and a negotiator. The bio-sensing network uses smart wristbands to measure occupants wrist temperature in real-time. The Bio-REAL agent initiates the best control actions on behalf of each occupant in response to the wrist temperature, subjective feedback, and environmental conditions. The negotiator resolves
conflicts in the control actions initiated by different Bio-REAL agents to maximize collective thermal comfort and minimize energy consumption. A state-of-art reinforcement learning algorithm, double Q learning with experiment
replay and neural network approximation, is applied to train the Bio-REAL agents. This thesis evaluates the Bio-REAL systems using three types of experimental techniques: simulation experiments, preliminary field and simulation
experiments, and field experiments. The simulation experiment trains a Bio-REAL system with three virtual occupants and an office room with a variable air volume (VAV) system in a heating season. The three virtual occupants are simulated using classic thermal comfort models. The room of a small-sized office building is modeled by the EnergyPlus simulation tool. The preliminary
heating season field and simulation experiments gather data from six occupants, providing inputs to create the personalized occupant models. The experimental test space is a room with water-sourced radiators for heating and modeled by the EnergyPlus tool. The co-simulation with personalized occupant models and EnergyPlus model assesses the performance of the Bio-REAL system. The cooling season field experiment evaluates the real-world
performance of the Bio-REAL systems with fourteen occupants in a tropical climate, occupying a studio with ambient temperature controls and shared controls of ceiling fans. The three types of experiments each demonstrated that the Bio-REAL system has more advantages for improving thermal comfort and energy efficiency
compared to the conventional control systems based on thermal comfort models and static schedules. With the combinations of bio-sensing and learning capability, the Bio-REAL system was able to derive dynamic and adaptive control policies, mapping occupants’ personalized requests and the changes of indoor and outdoor environmental conditions to optimum control actions. The Bio-REAL system contributes an innovative approach for controlling
building conditioning systems, to deliver thermal comfort for each individual at the lowest energy possible, with benefits for occupant health and productivity, as well as sustainability. The Bio-REAL research addresses individual
differences in thermal comfort for multi-occupant spaces with limited individual controls. It also addresses a range of heating and cooling choices from ambient to task systems. The structure and learning process of the Bio-REAL system, the strategies for the simulations, and the real-world implementation offer creative solutions for building control systems, contributing to the application of the Internet of Things and artificial intelligence in buildings.