Questions about using several digit capsule layers

[Ang-Li-95]

I am trying to use the CapsNet and it worked well. But when I tried to add another capsule layer like this:

   # layer 3: Digits Caps Layer
   digitcaps_1 = CapsuleLayer(num_capsule=10, dim_capsule=16, routings=routings, name='digitCaps_1')(primarycaps)

   digitcaps = CapsuleLayer(num_capsule=n_class, dim_capsule=8, routings=routings, name='digitCaps')(digitcaps_1)

   # layer 4: length layer 
   out_caps = Length(name='capsnet')(digitcaps)

The loss function just doesn’t improve even after many epochs. I tried decreasing learning rate to 1e-07 and decrease the num_capsul and dim_capsule to make the model simpler to train but it still didn’t improve.

Can someone please help me solve this?

Thanks!

[imadtyx]

I am facing the same issue if you find an answer of solution please let me know