Sustainable training of high-performance models

What is it about?

This work explores the ways to make the training of deep learning models more environmentally friendly, specifically targeting convolutional neural networks (CNNs) used for learning large sequential data representations, such as in malware detection. It introduces a novel method called HotConv, which reduces the energy and resources needed for training by optimizing memory usage and processing time. By applying strategies like retroactive pruning of activations and memory-efficient backpropagation, the research demonstrates significant reductions in carbon emissions while maintaining model performance, highlighting the importance of sustainable practices in artificial intelligence.

Featured Image

Photo by Igor Omilaev on Unsplash

Photo by Igor Omilaev on Unsplash

Why is it important?

AI impacts environmental sustainability, and we cannot afford to worsen climatic changes while trying to develop AI systems. This research is vital as it offers innovative solutions to reduce energy consumption and emissions during model training, especially in sectors like cybersecurity, where energy usage is on par with large language models. By addressing the critical need for sustainable practices in AI, this work highlights how these innovations can be applied to real-world problems like malware detection, emphasizing the importance of integrating environmental considerations into the development of advanced AI systems. It also helps train CNNs in edge devices with limited GPU capacities and helps complete training CNNs in a timely fashion for detecting the latest malware.

Perspectives