Rearrange the vectorizated weights layer by layer

[Haozhoong]

Hello! I would like to make some layer-by-layer analysis on your dataset. I attempt to rearrange the vectorization weights layer by layer, but I found that the length of vectorizated weight mismatchs what I computated accorroding the architecture. So it is possible to rearrange the vectorizated weights layer by layer? And what's the correct way to do this?

Thanks a lot.

[gabrieleilertsen]

Hi! Sorry for the late reply. I have a function that I use for this purpose. Perhaps you will find it useful:

def weight_split(x, fsize, ldepth, lwidth, bn=True):
  B = 3
  ind = np.array([0,0])

  mlt = 1
  if bn:
    mlt = 5

  sout_conv = np.empty((ldepth[0],mlt+1), dtype=object)
  sout_fc = np.empty((ldepth[1]+1,mlt+1), dtype=object)

  blockc = (np.floor(ldepth[0]/B)*np.ones(B)).astype('int64')
  for c in range(B-1,0,-1):
    if sum(blockc) < ldepth[0]:
      blockc[c] += 1

  assert(sum(blockc) == ldepth[0])

  cc = 0
  sz = np.array([32,32,3], dtype='int64')
  for c in range(B):
    w = lwidth[0]*np.power(2,c)

    # Unpack convolutional filters
    ind[1] += fsize*fsize*sz[2]*w
    sout_conv[cc,0] = np.transpose(np.reshape(x[ind[0]:ind[1]],[w,sz[2],fsize,fsize]), [2,3,1,0])
    ind[0] = ind[1]

    # bias and BN weights
    for mm in range(mlt):
      ind[1] += w
      sout_conv[cc,mm+1] = x[ind[0]:ind[1]]
      ind[0] = ind[1]
    
    sz[2] = w
    cc += 1

    for b in range(blockc[c]-1):
      # Unpack convolutional filters
      ind[1] += fsize*fsize*w*w
      sout_conv[cc,0] = np.transpose(np.reshape(x[ind[0]:ind[1]],[w,w,fsize,fsize]), [2,3,1,0])
      ind[0] = ind[1]
      
      # bias and BN weights
      for mm in range(mlt):
        ind[1] += w
        sout_conv[cc,mm+1] = x[ind[0]:ind[1]]
        ind[0] = ind[1]
      
      sz[2] = w
      cc += 1
    
    sz[:2] = sz[:2]/2

  #print('%d conv layers splitted (%d in total)'%(cc,ldepth[0]))

  sz = np.prod(sz)

  cc = 0
  for c in range(ldepth[1]):
    w = (lwidth[1]*np.power(2.0,-c)).astype('int64')

    # multiplicative weights
    ind[1] += sz*w;
    sout_fc[cc,0] = np.transpose(np.reshape(x[ind[0]:ind[1]],[w,sz]),[1,0])
    ind[0] = ind[1]
    
    # bias and BN weights
    for mm in range(mlt):
      ind[1] += w
      sout_fc[cc,mm+1] = x[ind[0]:ind[1]]
      ind[0] = ind[1]
    
    sz = w
    cc += 1

  w = 20
  ind[1] += sz*w
  sout_fc[cc,0] = np.transpose(np.reshape(x[ind[0]:ind[1]],[w,sz]),[1,0])
  ind[0] = ind[1]

  ind[1] += w
  sout_fc[cc,1] = x[ind[0]:ind[1]]
  ind[0] = ind[1]

  cc += 1

  #print('%d FC layers splitted (%d in total)'%(cc,ldepth[1]))

  assert(ind[0] == len(x))

  return (sout_conv, sout_fc)

[Haozhoong]

This function works correctly. Thank you for your nice reply~