Evolutionary multi-objective multi-agent deep reinforcement learning for sustainable maintenance scheduling

In recent years, sustainability has emerged as a major priority for businesses across various industries, and the manufacturing sector is no exception. Production and maintenance processes now need to be economically profitable while also adopting practices that adhere to the principles of environmental integrity and social responsibility. This article explores an innovative approach aimed at optimizing maintenance scheduling from an economic perspective (considering maintenance, breakdown, downtime costs), an environmental perspective (considering the carbon footprint produced during production) and a social perspective (considering the fatigue experienced by technicians during maintenance activities). To the best of our knowledge, this is the first study to propose a manufacturing scheduling approach that considers all three pillars of sustainability. Another significant contribution of this research is the innovative way in which the optimization problem is addressed. We propose an evolutionary multi-objective multi-agent Deep Q-network-based approach, where multiple agents explore the preference space to maximize the hypervolume of these sustainable objectives. Our methodology uses industrially representative data that incorporate realistic machine degradation signals, carbon intensity indicators, and technician constraints. The results demonstrate the trade-offs between these objectives when compared to traditional maintenance policies such as corrective and condition-based maintenance, as well as different Deep Q-network policies trained with various preferences. Our approach demonstrates superior performance compared to both baselines. Specifically, we observe an 11.6% improvement in hypervolume over Deep Q-network and an 18.9% improvement over Proximal Policy Optimization, resulting in significantly increased profitability within the system.