Separate Transformer Encoder & Decoder modules with linear attention?

[harshakmohan]

I would like to experiment with a Transformer Encoder and Transformer Decoder with linear attention. I want to train a Transformer Encoder on some input. I want to be able to access the transformer encoder output to train a separate diffusion model.

Is this possible with this library? Thanks in advance.