Scaling and values for dropout

[moukle]

Wanted to create smaller EFN inspired by #26.
To first approve my understanding of the scaling, I tried to recreate the parameters used for B1-B7 as follows:

alpha = 1.2
beta = 1.1
gamma = 1.15

re = {}
for coeff in range(0,10):
    d = alpha**coeff
    w = beta**coeff
    r = gamma**coeff * 224
    re[coeff] = np.around([w,d,r], decimals=2)

But the values don't match up. Comparing my values for B7 with the original values, there's a quite a huge gap (especially for d).

w, d, res
my:  [1.95   3.58 595.84]
orig: (2.0, 3.1, 600, 0.5)

• Is my calculation wrong or is there some human fine tuning involved?

Full output:

for key, value in re.items():
    print(key, value)
    
for key, value in orig.items():
    print(key, value)

0 [  1.   1. 224.]
1 [  1.1   1.2 257.6]
2 [  1.21   1.44 296.24]
3 [  1.33   1.73 340.68]
4 [  1.46   2.07 391.78]
5 [  1.61   2.49 450.54]
6 [  1.77   2.99 518.13]
7 [  1.95   3.58 595.84]

efficientnet-b0 (1.0, 1.0, 224, 0.2)
efficientnet-b1 (1.0, 1.1, 240, 0.2)
efficientnet-b2 (1.1, 1.2, 260, 0.3)
efficientnet-b3 (1.2, 1.4, 300, 0.3)
efficientnet-b4 (1.4, 1.8, 380, 0.4)
efficientnet-b5 (1.6, 2.2, 456, 0.4)
efficientnet-b6 (1.8, 2.6, 528, 0.5)
efficientnet-b7 (2.0, 3.1, 600, 0.5)

---

Doing the same with negative coeffecients yielded following values:

-10 [ 0.39  0.16 55.37]
-9 [ 0.42  0.19 63.67]
-8 [ 0.47  0.23 73.23]
-7 [ 0.51  0.28 84.21]
-6 [ 0.56  0.33 96.84]
-5 [  0.62   0.4  111.37]
-4 [  0.68   0.48 128.07]
-3 [  0.75   0.58 147.28]
-2 [  0.83   0.69 169.38]
-1 [  0.91   0.83 194.78]

• How would one choose values for the corresponding dropout layer?