Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement llama and rmkv onnx #1156

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open

Implement llama and rmkv onnx #1156

wants to merge 14 commits into from

Conversation

yuki399
Copy link

@yuki399 yuki399 commented Jun 10, 2023

yuki and others added 4 commits June 10, 2023 22:02
@kyakuno
Copy link
Collaborator

kyakuno commented Mar 4, 2024

モデルアップロード済み。

@kyakuno
Copy link
Collaborator

kyakuno commented Mar 4, 2024

macでの出力。

 INFO llama.py (351) : Input : My name is Clara and I am
2024-03-04 13:38:07.355 | INFO     | llama_util:__init__:217 - Reloaded SentencePiece model from ./tokenizer.model
2024-03-04 13:38:07.356 | INFO     | llama_util:__init__:224 - #words: 32000 - BOS ID: 1 - EOS ID: 2
2024-03-04 13:38:07.357 | WARNING  | llama_util:check:78 - virtual_memory not enough, require 25.165856941603124, try `--poolsize 24`
 INFO llama.py (371) : output : 

I am a student at the

@kyakuno kyakuno changed the title Implement llama Implement llama and rmkv onnx Mar 4, 2024
@kyakuno
Copy link
Collaborator

kyakuno commented Mar 4, 2024

ailiaにmemory_modeを設定。

@kyakuno
Copy link
Collaborator

kyakuno commented Mar 4, 2024

fp16指定をしてもfp16のonnxがロードされていない。

@kyakuno
Copy link
Collaborator

kyakuno commented Mar 4, 2024

FP16のモデルが使用されるように修正。

@kyakuno
Copy link
Collaborator

kyakuno commented Mar 4, 2024

モデルとしてはマージ可能。ただ、負荷が高いのでもう少し検討。

# Conflicts:
#	README.md
#	scripts/download_all_models.sh
@kyakuno
Copy link
Collaborator

kyakuno commented Jan 1, 2025

tokenizersをailia_tokenizer.LLAMATokenizerに置き換えたい。

@kyakuno
Copy link
Collaborator

kyakuno commented Jan 1, 2025

ONNXが分割されすぎているので、統合できないか検討したい。

# Conflicts:
#	README.md
#	scripts/download_all_models.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants