Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Divide 1D tensor into more than 2 TPC instances #22

Open
mcisowsk opened this issue Feb 5, 2024 · 0 comments
Open

Divide 1D tensor into more than 2 TPC instances #22

mcisowsk opened this issue Feb 5, 2024 · 0 comments

Comments

@mcisowsk
Copy link

mcisowsk commented Feb 5, 2024

I noticed that for a 1D input tensor, we can define index space in such a way, that max 2 TPC cores are utilized (as in example https://docs.habana.ai/en/latest/TPC/TPC_User_Guide/TPC_Programming_Model.html#index-space-mapping). To use 4 TPCs, tensor must be 2D. What I want to achieve is to have a 1D tensor and divide the load equally into all TPC cores. So for a 1D tensor of shape size 512 want each TPC core to handle 64 elements. But all I can accomplish is 2 TPC each handling 256 elements. Why is that?

int elementsInVec = 64;
unsigned depthIndex = (outputSizes[0] + (elementsInVec - 1)) / elementsInVec;
kernel->indexSpaceGeometry.dims = 1;
kernel->indexSpaceGeometry.sizes[0] = depthIndex;

kernel->inputTensorAccessPattern[0].dim[0].dim      = 0;
kernel->inputTensorAccessPattern[0].dim[0].start_a  = elementsInVec;
kernel->inputTensorAccessPattern[0].dim[0].end_a    = elementsInVec;
kernel->inputTensorAccessPattern[0].dim[0].start_b  = 0;
kernel->inputTensorAccessPattern[0].dim[0].end_b    = elementsInVec - 1;

I defined the mapping as:

  • startF(x) = 64*x + 0
  • endF(x) = 64*x+63

but it seems that it is ignored and instead it behaves more as if the mapping was:

  • startF(x) = 256*x + 0
  • endF(x) = 256*x+255

What values is x actually gonna be? [0,1] ? What is wrong with my code? Is it even possible to launch 8 TPC for a data layout like this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant