From 13bc2b3bb073d52d82d8dc704499db4e1ce6ff93 Mon Sep 17 00:00:00 2001 From: TimotheeMickus Date: Mon, 25 Sep 2023 11:18:54 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20=20@=20175ac?= =?UTF-8?q?3ea9cc916abbb15cdfdb520b5924f5d68d3=20=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- _modules/index.html | 7 +- _modules/mammoth/decoders/decoder.html | 241 ------ .../mammoth/decoders/transformer_decoder.html | 741 ------------------ _modules/mammoth/encoders/encoder.html | 278 ------- _modules/mammoth/encoders/mean_encoder.html | 262 ------- .../mammoth/encoders/transformer_encoder.html | 401 ---------- _modules/mammoth/translate/translator.html | 4 +- _sources/attention_bridges.md.txt | 11 +- attention_bridges.html | 2 +- genindex.html | 38 +- mammoth.modules.html | 201 +---- objects.inv | Bin 1764 -> 1681 bytes searchindex.js | 2 +- 13 files changed, 19 insertions(+), 2169 deletions(-) delete mode 100644 _modules/mammoth/decoders/decoder.html delete mode 100644 _modules/mammoth/decoders/transformer_decoder.html delete mode 100644 _modules/mammoth/encoders/encoder.html delete mode 100644 _modules/mammoth/encoders/mean_encoder.html delete mode 100644 _modules/mammoth/encoders/transformer_encoder.html diff --git a/_modules/index.html b/_modules/index.html index 93a12d72..9eb57444 100644 --- a/_modules/index.html +++ b/_modules/index.html @@ -173,12 +173,7 @@

All modules for which code is available

-

Decoders

-
-
-class mammoth.decoders.DecoderBase(attentional=True)[source]
-

Bases: torch.nn.modules.module.Module

-

Abstract class for decoders.

-
-
Parameters
-

attentional (bool) – The decoder returns non-empty attention.

-
-
-
-
-classmethod from_opt(opt, embeddings)[source]
-

Alternate constructor.

-

Subclasses should override this method.

-
- -
-

Attention

@@ -655,112 +560,12 @@

Architecture: Transformer -
-class mammoth.encoders.TransformerEncoder(num_layers, d_model, heads, d_ff, dropout, attention_dropout, embeddings, max_relative_positions, pos_ffn_activation_fn='relu', layer_norm_module=None)[source]
-

Bases: mammoth.encoders.encoder.EncoderBase

-

The Transformer encoder from “Attention is All You Need” -[VSP+17]

-
- graph BT - A[input] - B[multi-head self-attn] - C[feed forward] - O[output] - A --> B - B --> C - C --> O -
-
Parameters
-
    -

  • num_layers (int) – number of encoder layers

    • -

  • d_model (int) – size of the model

    • -

  • heads (int) – number of heads

    • -

  • d_ff (int) – size of the inner FF layer

    • -

  • dropout (float) – dropout parameters

    • -

  • embeddings (mammoth.modules.Embeddings) – embeddings to use, should have positional encodings

    • -

  • pos_ffn_activation_fn (ActivationFunction) – activation function choice for PositionwiseFeedForward layer

    • -
-
-
Returns
-

    -

  • embeddings (src_len, batch_size, model_dim)

    • -

  • memory_bank (src_len, batch_size, model_dim)

    • -
-

-
-
Return type
-

(torch.FloatTensor, torch.FloatTensor)

-
-
-
-
-forward(src, lengths=None, skip_embedding=False, mask=None)[source]
-

See EncoderBase.forward()

-
- -
-
-classmethod from_opt(opt, embeddings, is_on_top=False)[source]
-

Alternate constructor.

-
- -
- -
-
-class mammoth.decoders.TransformerDecoder(num_layers, d_model, heads, d_ff, copy_attn, self_attn_type, dropout, attention_dropout, embeddings, max_relative_positions, aan_useffn, full_context_alignment, alignment_layer, alignment_heads, pos_ffn_activation_fn='relu', layer_norm_module=None)[source]
-

Bases: mammoth.decoders.transformer_decoder.TransformerDecoderBase

-

The Transformer decoder from “Attention is All You Need”. -[VSP+17]

-
- graph BT - A[input] - B[multi-head self-attn] - BB[multi-head src-attn] - C[feed forward] - O[output] - A --> B - B --> BB - BB --> C - C --> O -
-
Parameters
-
    -

  • num_layers (int) – number of decoder layers.

    • -

  • d_model (int) – size of the model

    • -

  • heads (int) – number of heads

    • -

  • d_ff (int) – size of the inner FF layer

    • -

  • copy_attn (bool) – if using a separate copy attention

    • -

  • self_attn_type (str) – type of self-attention scaled-dot, average

    • -

  • dropout (float) – dropout in residual, self-attn(dot) and feed-forward

    • -

  • attention_dropout (float) – dropout in context_attn (and self-attn(avg))

    • -

  • embeddings (mammoth.modules.Embeddings) – embeddings to use, should have positional encodings

    • -

  • max_relative_positions (int) – Max distance between inputs in relative positions representations

    • -

  • aan_useffn (bool) – Turn on the FFN layer in the AAN decoder

    • -

  • full_context_alignment (bool) – whether enable an extra full context decoder forward for alignment

    • -

  • alignment_layer (int) – N° Layer to supervise with for alignment guiding

    • -

  • alignment_heads (int) –

      -

    1. of cross attention heads to use for alignment guiding

      2. -
    -

    • -
-
-
-
-
-forward(tgt, memory_bank=None, step=None, memory_lengths=None, tgt_pad_mask=None, skip_embedding=False, **kwargs)[source]
-

Decode, possibly stepwise.

-
- -
-
class mammoth.modules.MultiHeadedAttention(head_count, model_dim, dropout=0.1, max_relative_positions=0)[source]

Bases: torch.nn.modules.module.Module

Multi-Head Attention module from “Attention is All You Need” -[VSP+17].

+[VSP+17].

Similar to standard dot attention but uses multiple attention distributions simulataneously to select relevant items.

@@ -907,7 +712,7 @@

Copy Attentionclass mammoth.modules.CopyGenerator(input_size, output_size, pad_idx)[source]

Bases: torch.nn.modules.module.Module

An implementation of pointer-generator networks -[SLM17].

+[SLM17].

These networks consider copying words directly from the source sequence.

The copy generator is an extended version of the standard @@ -978,7 +783,7 @@

Structured AttentionImplementation of the matrix-tree theorem for computing marginals of non-projective dependency parsing. This attention layer is used in the paper “Learning Structured Text Representations” -[LL17].

+[LL17].

forward(input)[source]
diff --git a/objects.inv b/objects.inv index 8400203f5fcc1d16d811aba80db2db5852459bf2..5f67f66112eb021707aeb74de2e99feb3a6cb8db 100644 GIT binary patch delta 1556 zcmV+v2J89c4Ur9ykblG7J!)iYF)Fg;lVk|}>u+UYu!$kYHt>)D((Z3Rw5wfNPH2)S zI}0Y7atV5vuHHw28Z#UZll9chCgQi>hoeDKr46k6bQ}rTnPN>?7F2LXEi~gw6a!&` zhx^IB`_@Rg9iAXknY1LOnxZ_=rj_Jis`Qd-)|fdHV1f?|(0??B57vUTT**593);pY zL6jw#pq4LyP(N3zRUg14p=RC_`XLX#h<2;w;yx3WKUzq;`LiA>plK>r9qio$+tKQD zbqAO>)GFQ5*b+iFWj-Y#FvewUT4}On4v8uuo;iGQAZigEZf(EzQqegU(c!k8!0 zU=IPo4bbEfsE%fJ+7)z8W+RXQ)6a5j9)H#EI1K?m0tPoc;(&#C)j_i=P(ay&%BV-x zf>5YvPGpR2E76bw-BT~yp`XH=K|)s{koBg9;S+MkKoYCwAkC@dSgK>Nu6j@TQ;$L_ zxa(cDCx7U|L%0ZL2CNJ69Uf48S$=fISa*o%J4|}E>YTD=m_E{?g4a*jRv>#^jLDX) zg9A$_*qFvWDRh%@!8*l90jBx^*$$GBS|4(4?TqsUCC0C)p-C!$EV-Q#3rXq@Kbdb- zrk}A@;0N6|p>l zeP_25N%iedgeM}5#^JlN{U(1Q7vQnardmZ{%mswwncz~gFR}6|)dq-4 z&wrX}7h2)VtgQyWe;bHNN~1Z})U(RBio|JlUWwr9?s zsIgSr^NY=7ub$2<7bf_I`%Y3E4qm*<-+$o=@L=EU0)Ce<#Cq2J`hJ&ts#UVL^>~q6 z0M+j5i0Hip!tm#qCd$br0#`4G-=YOm% znV&!&rLj=P1YhyrtxC{XxS<9P2IB3vhslpGH-AsQJlqY9W$&4ahNJPYJf2ezZ0V)$4>G*xLrnO3m0SiU?{-?^c zq{3W9O6w~cn$h^(_3P2~^_xG&Ie(;}A#URcs$uHQc6_6PZ{+w!JA9)9*zZQyZ(omJ z9gSV^)nW^tiSb+39JFAlMKb^*8NY3gRxs3}EvQ*i$xWL5jOXbwu+KP019-}xQJbss zl|gCc>*jp`lm-Sdo+KoLr>ZK&sHgwpz7DozYXKz+7tk1x28nQ!6QW9%4}TR<`42B! zWJ<=buE(##J&y26$tz~L-yZ85_6Q3O%OY>9RNLenes-_zEtQ^cGMr1y==?>#8lKIh z=l)179e%|&21j-;!0Xt{_u9jI|D#d=R0sBAe>C)+)}ZEj?;*U}SV-JE8_!}PU5$!~ zvQ6u#Q28OyT={eVLPazz%YUtz_8|Gw>)}6cV1x184b_>7!Nrt+f3$`tpV%15WtqeK zbPw0s;T8A5z&%i*8VGaYP+}~Pd!FxpBoCaye;JSFzWXjQZAU%JUz^_p4E{1@kqg5g zFa-@V!eH`uRMk~l-&?Icl+S;sSQrdQsd+TRG`{*Wb#r!GIygHt>*OOWNOlO4^m2&?HfI z8a!#rCFo(gd>;vF%y2wRims_m#BaY3M}uT@Y+&85qe#F`6>CB=r-CzTp&1o}NHNeR z_~;*QsDU~O6U8zCrWrsm(JCP-wY4(5427ylGf_GLU7)rTX@3_dz_$g}>^Q+d7J#2* zJ0sNrbQI+ts4URhiBbXRPn4FPUqMwo)=9ZtoQ^uy0k%>VJ3yG=;r{8~JvCYlwrGM# zWzv$AYKnYOx@J>-8|9Ee>WFiFKFmSW7(Q4F(sCt_a?d`HAj*&We0utz;@I+UETriQ`9QmQrQYZH(@>{ zAuz^eY+6aOCJ!J9iO8(!Do0=X1VT|pQVn?m0gTKcLUVPOZ=p1EnzjSF6Dp*lTd;3o zbhFqtn2D5bx%T8U zyE5E?QIT)8g^WkraTlhW75@buPwfNd0B&J?#aaoq4bn}dv`~>uCUT|kf*be@z&@{$ zj~25Da5vG7)mdaSjcXHKMbACOIDCH60Ij5c0kfUJ)@6Q}LqKo?Gl7a) znCb^)-AF=eeaN-dGtM1KjQ6OaNh*LWxSbLUN$QH9%qOaoOh&nTj%JYprbd>#Oeud_ zb~org$RJWh^(~d`6zDH=Fj?=tw-a&o=}&|e5ok+rH;m=^ z*OVlS$J1Uip*YMo@ zf=FOeNRl$eI;9VHnG0^L=WQ;)+})O3NO4>mFUekK^88GVrP`jKZ6@hX3dCcwhJ*#-PIVTk)#^XvO<>Vw}dyhts8YIk)+bmQ-gwjqp#mWr46R)3^D z1peX9tm#q4d$br0#`4G<;pjC{K%*$rG+MUf8BifeOzjl6S)4uA^aMlPy4w;91WgUF zykKkzAyiB(NTYP9jtu4s3RbIW)>ex&6)h?KIcrPeXOKrpER-?9SNwOY5>yuUeS@|l zzy0>`^yACT-%np2?uN#)_e@2@(SLYYZO=y!Zf(%qnlqO$S7>)SJ;3_D z`zCK5U?o4{G4VfDfQ^n7cYGcdW35tRz(SF}|EV%9sW4ZO()x;qW;A|x{d#nL{pOEx z4k>7e%Q%8+n0U1v-KeA+Il9pn-RJ=JyV3RA*W*`5W0!Qb*n($b{FXHbEq@7W(F}k{ z#&4UWl?1hDb7~e;a+791<5_wP><;B<0MGduwYjQZ8B{!9*Y5+M(lCgzl8_9ZH$^F1 z{rE5Laj+g%FsO`h0gVA^a1btXLTnDpn@Xts#q$=KlJTqS@#}DpA^fD|8)~_CkM#|6 zggJ*rId3;dttaR3v-@msp?~yjHN&~Y8J&N~S3|Sv^wb}zrNgh7#$e0#1$Z5K^q4FZoT=}|xt|A&%`POuM zko@#|_>XH?VeFb=vu8u$vde$4T|<+1td!)kO5ts~hpI66hIwG%4q#NM2Etr8R0yl> zp60u6$pdHbU&f=E&%P@}>sC+l*Xn(M!CxI&&V}I*n1BWuVKDhSs!cBy_g004^8W7> lHwFVzY938-n!OR2#AW$zcwoLQGB-3