Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

【WIP】update for infer config #13524

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
+16 −9
Open

【WIP】update for infer config #13524

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b5b2c65
update for infer config
tink2123 Jul 26, 2024
8f826ed
update for infer config
tink2123 Jul 26, 2024
8bfbd5a
update for infer config
tink2123 Jul 29, 2024
cda49b8
update for infer config
tink2123 Jul 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion tools/infer/utility.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,12 @@ def create_predictor(args, mode, logger):
)
)

config = inference.Config(model_file_path, params_file_path)
if paddle.__version__ == "0.0.0" or paddle.__version__ >= "3.0.0":
model_path = model_dir
model_prefix = file_name
config = inference.Config(model_path, model_prefix)
else:
config = inference.Config(model_file_path, params_file_path)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里的改动，是为了适配 PIR 启用后，存储下来的模型文件的变化吧？
正确的判断应该使用 API， 而不是版本号，因为，即便是在 3.0.0 及后续的版本下，也可以通过环境变量来设定是否使用 PIR 。

我不太确定应该用 paddle.framework.in_pir_mode() 还是用paddle.framework.in_pir_executor_mode 来判断。

另外，根据我的理解，如果使用 inference.Config(model_path, model_prefix) 的话，paddle.load 就会自己对新旧模式兼容了？

cc: @phlrain @Aurelius84

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

是的，为了适应PIR。 根据现在的说明文档，inference.Config 只在3.0 以上生效，因此在版本号小于3.0时无法自动判断。

注意：如果套件库需要兼容支持paddle2.x ,需要按照版本做分支处理(config 的新接口只在3.0 beta 版之后提供）

环境变量的使用方式我再确认一下～

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

注意：如果套件库需要兼容支持paddle2.x ,需要按照版本做分支处理(config 的新接口只在3.0 beta 版之后提供）

这个说明是来自哪儿？

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个说明是来自哪儿？

是内部的开发文档

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我查到了，是新的版本下 inference Config 才能支持传入的是模型目录名的原因吧。
真是令人头大~


if hasattr(args, "precision"):
if args.precision == "fp16" and args.use_tensorrt:
Expand Down
Loading