Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Preserve sparsity SPARSEGPT #2282

Merged
merged 4 commits into from
May 17, 2024
Merged

Preserve sparsity SPARSEGPT #2282

merged 4 commits into from
May 17, 2024

Conversation

rahul-tuli
Copy link
Member

This PR incorporates changes from @abhinavnmagic's PR #2222 into new modifier UX

We introduce a new argument preserve_sparsity_mask in SparseGPTModifier which can be used to extend or ignore the base weight mask during SparseGPT application

Test Recipe:

test_stage:
  obcq_modifiers:
    SparseGPTModifier:
      sparsity: 0.5
      block_size: 128
      percdamp: 0.01
      mask_structure: "2:4"
      targets: ["re:model.layers.\\d*$"]
      preserve_sparsity_mask: true

@rahul-tuli rahul-tuli changed the base branch from main to preserve-sparsity-gptq May 13, 2024 14:03
@rahul-tuli rahul-tuli mentioned this pull request May 13, 2024
Merged
7 tasks
Base automatically changed from preserve-sparsity-gptq to gptq-ux-config-groups May 17, 2024 16:15
@rahul-tuli rahul-tuli merged commit 446555f into gptq-ux-config-groups May 17, 2024
@rahul-tuli rahul-tuli deleted the preserve-sparsity-sparsegpt branch May 17, 2024 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bfineran bfineran bfineran approved these changes

@Satrat Satrat Satrat approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rahul-tuli @Satrat @bfineran