Stacked RNN in Flux.jl?

[NeroBlackstone]

Motivation and description

How to build Stacked RNN in Flux.jl?

Is the following code the correct way?

using Flux
model = Chain(GRUv3(27 => 32),GRUv3(32 => 32),Dense(32 => 27))

Chain(
  Recur(
    GRUv3Cell(27 => 32),                # 5_792 parameters
  ),
  Recur(
    GRUv3Cell(32 => 32),                # 6_272 parameters
  ),
  Dense(32 => 27),                      # 891 parameters
)         # Total: 12 trainable arrays, 12_955 parameters,
          # plus 2 non-trainable, 64 parameters, summarysize 1.938 KiB.

There is no documentation mentioning this.

Possible Implementation

No response

[CarloLucibello]

Starting with Flux v0.15, a stacked RNN can be defined as follows:

stacked_rnn = Chain(LSTM(3 => 3), Dropout(0.5), LSTM(3 => 3))

If control of the initial states is also needed, define a custom struct:

struct StackedRNN{L,S}
    layers::L
    states0::S
end

function StackedRNN(d, num_layers)
    layers = [LSTM(d => d) for _ in num_layers]
    states0 = [Flux.initialstates(l) for l in layers]
    return StackedRNN(layers, states0)
end

function (m::StackedRNN)(x)
     for (layer, state0) in zip(rnn.layers, rnn.states0)
         x = layer(x, state) 
     end
     return x
end

This stuff should be documented in the guide
https://github.com/FluxML/Flux.jl/blob/master/docs/src/guide/models/recurrence.md