Neuron importance algorithm for continual learning

Jianwen Mo; Shengyang Huang

doi:10.1117/12.3009542

16 October 2023 Neuron importance algorithm for continual learning

Jianwen Mo, Shengyang Huang

Proceedings Volume 12803, Fifth International Conference on Artificial Intelligence and Computer Science (AICS 2023); 128030U (2023) https://doi.org/10.1117/12.3009542
Event: 2023 5th International Conference on Artificial Intelligence and Computer Science (AICS 2023), 2023, Wuhan, China

Abstract

Most regularization-based continual learning methods are based on synapse importance. In contrast to them, we propose a neuron importance algorithm that uses Taylor criterion to calculate the importance of neurons. Then, for the fully connected layer and the convolutional layer, we propose two different approaches to convert neuron importance to synapse importance. Compared with existing neuron importance based method, the proposed algorithm is simple to implement, requiring only one forward propagation and one backward propagation to calculate the neuron importance. The effectiveness of the algorithm was validated on two datasets, 5-Split-MNIST and 5-Split-CIFAR10.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Jianwen Mo and Shengyang Huang "Neuron importance algorithm for continual learning", Proc. SPIE 12803, Fifth International Conference on Artificial Intelligence and Computer Science (AICS 2023), 128030U (16 October 2023); https://doi.org/10.1117/12.3009542

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available