Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Apply the attention mask in all decoding steps (LM inference) #2532

Merged
merged 18 commits into from
Dec 15, 2023
Merged

Apply the attention mask in all decoding steps (LM inference) #2532

merged 18 commits into from
Dec 15, 2023

Commits on Nov 30, 2023

  1. wip

    @l-k-11235
    l-k-11235 committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    d7b5f56 View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2023

  1. fixed attention mask

    @l-k-11235
    l-k-11235 committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    be6adab View commit details
    Browse the repository at this point in the history

  2. some code cleaning

    @l-k-11235
    l-k-11235 committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    4644bc8 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2023

  1. restore and adapt previous dec_mask

    @l-k-11235
    l-k-11235 committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    c94d8c9 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2023

  1. wip

    @l-k-11235
    l-k-11235 committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    3116271 View commit details
    Browse the repository at this point in the history

  2. wip - apply mask on context before final_linear + degugging

    @l-k-11235
    l-k-11235 committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    faae43d View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. wip - apply offset in rotaty embeddings

    @l-k-11235
    l-k-11235 committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    40caae8 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. wip

    @l-k-11235
    l-k-11235 committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    a805ad9 View commit details
    Browse the repository at this point in the history

  2. apply pad_mask for all decoding steps when batch_size is greater than… 

    … 1 - works for 'classical attention'
    @l-k-11235
    l-k-11235 committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    f3ed229 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. wip

    @l-k-11235
    l-k-11235 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    829a861 View commit details
    Browse the repository at this point in the history

  2. handle finished hypotheses

    @l-k-11235
    l-k-11235 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    8333d04 View commit details
    Browse the repository at this point in the history

  3. wip

    @l-k-11235
    l-k-11235 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    50c0754 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. use key_pad_mask in transformer LM attention layers to handle beam_si… 

    …ze > 1 thanks to map_state
    @l-k-11235
    l-k-11235 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    b50a7e1 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. differentiate between scaled-dot and scaled-dot-flash values for self… 

    …_attn_type opt
    @l-k-11235
    l-k-11235 committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    7c49902 View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. some code cleaning

    @l-k-11235
    l-k-11235 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    1bc3212 View commit details
    Browse the repository at this point in the history

  2. added warning in quickstart.md

    @l-k-11235
    l-k-11235 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    dc34150 View commit details
    Browse the repository at this point in the history

  3. removed empty lines

    @l-k-11235
    l-k-11235 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    3dcda9e View commit details
    Browse the repository at this point in the history

  4. restored proper file permissions

    @l-k-11235
    l-k-11235 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    bbc28a8 View commit details
    Browse the repository at this point in the history