Update transformers test requirements #22911
Open
+17
−4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tianleiwu
force-pushed
the
tlwu/ci_python_test
branch
2 times, most recently
from
46a687e
to
b7cc8e8
Compare
tianleiwu
force-pushed
the
tlwu/ci_python_test
branch
from
b7cc8e8
to
e5fcd19
Compare
| if args.enable_transformers_tool_test and not args.disable_contrib_ops and not args.use_rocm: | ||
| # PyTorch is required for transformers tests, and optional for some python tests. | ||
| # Install cpu only version of torch when cuda is not enabled in Linux. | ||
| extra = [] if args.use_cuda and is_linux() else ["--index-url", "https://download.pytorch.org/whl/cpu"] |
There was a problem hiding this comment.
Can we always pass the --index-url? From the Get Started page, it is used for Linux CUDA and there is a link for all three CUDA versions.
- CUDA 11.8: https://download.pytorch.org/whl/cu118
- CUDA 12.1: https://download.pytorch.org/whl/cu121
- CUDA 12.4: https://download.pytorch.org/whl/cu124
There was a problem hiding this comment.
The first commit actually used that. For some reason, there was error in importing torch in Windows GPU ci pipeline.
Later, I changed it since --index-url will change if pytorch changes cuda version (like to 12.6). If we do not set index URL, it will always be the latest version in pypi.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Motivation and Context
Currently, transformers tests are enabled in the following CI pipelines:
For ROCm CI Pipeline, transformer tests are enabled but skipped since onnx package is not installed in CI.
Previously, torch was not installed before python tests, so some tests depending on torch were skipped like test_bind_onnx_types_not_supported_by_numpy or test user_compute_stream.
In this PR, we changed build.py to install torch before running python tests.