You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I had some errors when running the repo on GH200 (Hopper architecture). It seems the logs indicate hardcoding certain GPU configs which does fit into the newer GPU type. But I cannot find where such hardcode comes from. Could you please give any suggestions to avoid such errors? Thanks for the help!
The running logs are as below. And after a while, the code will be ended with "CUDA error: an illegal memory access was encountered":
"Starting work on task 0
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 105 SMs (15 CGAs with 7 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 105 SMs (15 CGAs with 7 CTAs each)"
The text was updated successfully, but these errors were encountered:
Hi, I had some errors when running the repo on GH200 (Hopper architecture). It seems the logs indicate hardcoding certain GPU configs which does fit into the newer GPU type. But I cannot find where such hardcode comes from. Could you please give any suggestions to avoid such errors? Thanks for the help!
The running logs are as below. And after a while, the code will be ended with "CUDA error: an illegal memory access was encountered":
"Starting work on task 0
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 117 SMs (39 CGAs with 3 CTAs each)
Running with some CTAs in CGA Out of bounds
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (132 CGAs with 1 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 132 SMs (66 CGAs with 2 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 105 SMs (15 CGAs with 7 CTAs each)
Skyline(sm_counts_gpc) specified by user = 816161616161818000200200202000000000000
=> Using 105 SMs (15 CGAs with 7 CTAs each)"
The text was updated successfully, but these errors were encountered: