Adding More Models

[colah]

TF Slim Models

• ✅ inception_v1
• ✅ inception_v2
• ✅ inception_v3
• ✅ inception_v4
• ✅ inception_resnet_v2
• ✅ resnet_v1_50
• ✅ resnet_v1_101
• ✅ resnet_v1_152
• ✅ resnet_v2_50
• ✅ resnet_v2_101
• ✅ resnet_v2_152
• ✅ vgg_16
• ✅ vgg_19
• ✅ mobilenet_v1
• ✅ mobilenet_v1_050
• ✅ mobilenet_v1_025
• ✅ nasnet-a_mobile
• ✅ nasnet-a_large
• ✅ pnasnet-5_large
• ✅ pnasnet-5_mobile

Couldn't create list of layers for some models.

Caffe Models

• ✅ AlexNet
• ❌ AlexNet (Places 205) -- link to model on download page broken?
• ✅ AlexNet (Places 365)
• ❌ Network In Network? -- import fails
• ✅ InceptionV1
• ✅ InceptionV1 (Places 205)
• ✅ InceptionV1 (Places 365)
• ✅ CaffeNet
• ✅ VGG_16
• 🏗️ VGG_16 (Places 205) -- plan to defer
• 🏗️ VGG_16 (Places 365) -- plan to defer
• ✅ VGG_19

Issues

• ⚠️ Some models may not have the correct list of labels.
• ⚠️ Presently assuming all caffe models are BGR but it's possible some aren't in cases where they were originally trained in a different framework.
• ⚠️ Some models may not have the correct input range.
• 🏗️ Layer lists for some models are empty.

At the moment, I'm inferring all of these from the convention of the framework the model was trained in, but this may not be reliable.

[colah]

@zanarmstrong - If you'd like to try playing around with more models, you can grab the definitions from here! :)

[zanarmstrong]

thanks @colah!

[colah]

I attempted to port all the slim models, but ran into issues. In the end, I was able to import eight and ran into significant issues with five:

• ✅ inception_v1
• ✅ inception_v2
• ✅ inception_v3
• ✅ inception_v4
• ✅ inception_resnet_v2
• ❌ resnet_v1_50
• ❌ resnet_v1_101
• ❌ resnet_v1_152
• ✅ resnet_v2_50
• ✅ resnet_v2_101
• ✅ resnet_v2_152
• ❌ vgg_16
• ❌ vgg_19

I opened an issue (tensorflow/models#4938) to track this. However, the models I failed to import actually originate with caffe, so we should be able to just grab them when I get the caffe workflow fully running.

There were also a number of models my present scripts couldn't automate. I'll take a shot at them another day.

[ludwigschubert]

As this nears completion, please take a quick look at what's making the tests fail—you can click on "Details" next to the failing travis build, and then on a Python version and it will take you to the error messages. Atm it looks like there may be a missing numpy import in vision_models?

[colah]

Every model now has a layer list, like so:

  layers = [
     {'type': 'conv', 'name': 'conv2d0', 'size': 64},
     {'type': 'conv', 'name': 'conv2d1', 'size': 64},
     {'type': 'conv', 'name': 'conv2d2', 'size': 192},
     {'type': 'conv', 'name': 'mixed3a', 'size': 256},
     {'type': 'conv', 'name': 'mixed3b', 'size': 480},
     {'type': 'conv', 'name': 'mixed4a', 'size': 508},
     {'type': 'conv', 'name': 'mixed4b', 'size': 512},
     {'type': 'conv', 'name': 'mixed4c', 'size': 512},
     {'type': 'conv', 'name': 'mixed4d', 'size': 528},
     {'type': 'conv', 'name': 'mixed4e', 'size': 832},
     {'type': 'conv', 'name': 'mixed5a', 'size': 832},
     {'type': 'conv', 'name': 'mixed5b', 'size': 1024},
     {'type': 'conv', 'name': 'head0_bottleneck', 'size': 128},
     {'type': 'dense', 'name': 'nn0', 'size': 1024},
     {'type': 'dense', 'name': 'softmax0', 'size': 1008},
     {'type': 'conv', 'name': 'head1_bottleneck', 'size': 128},
     {'type': 'dense', 'name': 'nn1', 'size': 1024},
     {'type': 'dense', 'name': 'softmax1', 'size': 1008},
     {'type': 'dense', 'name': 'softmax2', 'size': 1008},
   ]

I generated this code using a script I checked in as lucid/scratch/scripts/get_model_layers.py.

[ludwigschubert]

(An additional enhancement could be to record the output/softmax/probabilities node, too.)

[colah]

An additional enhancement could be to record the output/softmax/probabilities node, too.

Absolutely! At the moment, I include softmax layers in the layer list, but don't flag them as special.

One thing for us to consider is that the logits (ie. softmax pre-activation) are often more interesting for visualization than the softmax output. May be worth adding them to the layer list as well...

[ludwigschubert]

Yeah, we'd likely want both for different purposes.

Can you add me to the gcloud modelzoo bucket?
The newer Inception architectures need an updated labels file. (As e.g. @girving remarked in #65)

[colah]

tensorflow/models#4938 was mostly my error! Now worked around. :)

[ludwigschubert]

Great progress! Tests are currently only failing because some of the models don't yet have a layers property. While it's obviously best if all models ship with which layers we consider interesting, I don't want this to be blocking the PR—feel free to make the layer shape check conditional on there being a layers property in the first place.

[colah]

OK -- The present version isn't perfect (see issues mentioned at top), but I suspect it's better to merge things at this point to not diverge too far from master.