Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Newer GPU arch support errors #397

Open
Allen920 opened this issue Feb 24, 2025 · 0 comments
Open

Newer GPU arch support errors #397

Allen920 opened this issue Feb 24, 2025 · 0 comments

Comments

@Allen920
Copy link

Hi, I had some errors when running the repo on GH200 (Hopper architecture). It seems the logs indicate hardcoding certain GPU configs which does fit into the newer GPU type. But I cannot find where such hardcode comes from. Could you please give any suggestions to avoid such errors? Thanks for the help!

The running logs are as below. And after a while, the code will be ended with "CUDA error: an illegal memory access was encountered":
"Starting work on task 0
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 105 SMs (15 CGAs with 7 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 105 SMs (15 CGAs with 7 CTAs each)"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant