Skip to content

Commit

Permalink
Use IB_MERGE_VFS argument when detecting PCI path
Browse files Browse the repository at this point in the history
When running in a cloud-hypervisor guest, IB VFs are exposed as a
RCiEP. If the IB VFs are merged, NCCL does not correctly detect
PCI topology.
  • Loading branch information
Thomas Barrett committed Jan 28, 2024
1 parent e453866 commit e6f477e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/p2p_plugin.c
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@ ncclResult_t nccl_p2p_ib_pci_path(nccl_ib_dev_t *devs, int num_devs, char* dev_n
// Merge multi-port NICs into the same PCI device
p[strlen(p)-1] = '0';
// Also merge virtual functions (VF) into the same device
p[strlen(p)-3] = '0';
if (ncclParamIbMergeVfs()) p[strlen(p)-3] = p[strlen(p)-4] = '0';
// And keep the real port aside (the ibv port is always 1 on recent cards)
*real_port = 0;
for (int d=0; d<num_devs; d++) {
Expand Down

0 comments on commit e6f477e

Please sign in to comment.