Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(*): Add internlm3 config #403

Merged
merged 14 commits into from
Jan 22, 2025

Conversation

yingtongxiong
Copy link
Contributor

@yingtongxiong yingtongxiong commented Jan 16, 2025

Motivation

  1. Add internlm3 config
  2. Support load and resume internlm3 model ckpt in huggingface and internevo format
  3. Add internlm3 in feature zoo of readme
  4. Add enable_qkv_fusion switch in config
  5. Modify the 'INTERNLM2_PUBLIC" to "INTERNLM2"

Modification

  1. README.md
  2. configs/8B_internlm3.py
  3. internlm/initialize/launch.py
  4. internlm/model/modeling_internlm2.py
  5. internlm/model/registry.py
  6. internlm/utils/utils.py

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.

internlm/model/modeling_internlm2.py Outdated Show resolved Hide resolved
internlm/model/modeling_internlm2.py Show resolved Hide resolved
internlm/model/modeling_internlm2.py Outdated Show resolved Hide resolved
@gaoyang07 gaoyang07 added the enhancement New feature or request label Jan 16, 2025
@gaoyang07
Copy link
Collaborator

TODO(me): update README.md

@gaoyang07 gaoyang07 merged commit 24180aa into InternLM:develop Jan 22, 2025
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants