NPNs

This repository contains the code for Neural Plasticity Networks .

Demo

1. The evolutions of decision boundaries of the learned networks on synthetic "Moons" dataset: (left) Network Sparsification, and (right) Network Expansion.

2. Visualization of part of the neurons in conv-layer and fully-connected layer of the LeNet-5-Caffe sparsified / expanded by NPNs. To achieve computational efficiency, only neuron-level (instead of weight-level) sparsification is considered.

Requirements

pytorch==1.3
tensorboard
matplotlib

Usage

python train_syn.py --k 7 --mode sparse --init_size 100 80 --stage1 500 --stage2 1000 --lambas 0.35 14
python train_syn.py --k 1 --mode expand --init_size 3 3 --stage1 100 --stage2 1000 --lambas 0.35 14
python train_lenet.py --k 7 --mode sparse --init_size 20 50 500
python train_lenet.py --k 1 --mode expand --init_size 3 3 3    
python train_resnet56.py --num_class 10 --mode expand --init_factor 0.5 0.5 0.3 0.2 0.8 --lamba 0
python train_resnet56.py --num_class 100 --mode expand --init_factor 0.5 0.5 0.3 0.2 0.8 --lamba 0
python train_resnet56.py --num_class 10 --mode sparse --init_factor -1 --lamba 1e-5
python train_resnet56.py --num_class 100 --mode sparse --init_factor -1 --lamba 1e-5

Citation

If you found this code useful, please cite our paper.

@article{npn2021,
  title={Neural Plasticity Networks},
  author={Li, Yang and Ji, Shihao},
  journal={International Joint Conference on Neural Networks (IJCNN)},
  year={2021}
}