Add boilerplate code #1635
Add boilerplate code #1635
Conversation
facebook-github-bot
added
the
CLA Signed
* Fix ZeroPointDomain.NONE support & make it default for da8w8 weights * Fix bug & apply review recommendations * Throw exceptions when None zero_point_domain is used * Use ZeroPointDomain.NONE for weight in int8_dynamic_activation_int8_weight * Rebase with the latest main branch * Fix typo
torchao/utils.py
Outdated
| aten = torch.ops.aten | ||
|
|
||
|
|
||
| @implements(aten.detach.default) |
There was a problem hiding this comment.
one thing here is what would happen when a tensor subclasses TorchAOBaseTensor and tried to overwrite these functions, to allow child tensor classes to overwrite these functions I think we'd have to copy the table
Line 402 in 7b0d2ce
def _implements(cls, aten_ops_or_torch_fns):
# making sure we are querying the attribute from current class, not parent
# please check if this works
if "_ATEN_OP_OR_TORCH_FN_TABLE" not in dir(cls):
# copy the table from parent
def _dispatch__torch_function__(cls, func, types, args=(), kwargs=None):
# making sure we are querying the attribute from current class, not parent
# please check if this works
if "_ATEN_OP_OR_TORCH_FN_TABLE" not in dir(cls):
# copy the table from parent if it exists
torchao/utils.py
Outdated
| class TorchAOBaseTensor(torch.Tensor): | ||
| """A util tensor subclass that provides commonly used functions | ||
| new tensor subclass can inherit it to get all the utility functions | ||
| new tensor subclass can inherit it to get all the utility functions, and |
There was a problem hiding this comment.
what are the pros and cons of using inheritance here versus just having utility functions and tensors using what they need from utility functions, without inheritance? It might be hard to come up with a TorchAOBaseTensor which is generic enough to truly handle all the important use cases in torchao.
There was a problem hiding this comment.
makes sense, we could have both I think, e.g. _get_to_kwargs can be a standalone util function
There was a problem hiding this comment.
As per my understanding:
Option 1: We can move out all the util functions, and make an independent tensor-subclass (inheriting from torch.Tensor), then developer will have to define/inherit util functions.
Option 2: We can add the minimum needed util functions in TorchAOBaseTensor, so that it gives a base start to developer and build on top of it.
There was a problem hiding this comment.
depends on how much can be reused by other tensor subclasses I think, if the TorchAOBaseTensor is very small then it may not make sense to have it any more and it will not be useful for existing inference tensor subclasses. I feel it might be better to have both the current TorchAOBaseTensor and some utils that can be reused by other tensor subclasses as a starting point and then adapt based on use cases
Pass all args to pytest.main to propage user options like -k Tested locally with `python test/test_ops.py -k test_dequantize_tensor_core_tiled_layout_correctness_quant_dequant` which previously just ran all the tests but after this PR will run 60, the same number as `pytest test/test_ops.py -k test_dequantize_tensor_core_tiled_layout_correctness_quant_dequant`
only run docs CI jobs when docs have changed
…sion Differential Revision: D68726705 Pull Request resolved: #1630
There's a lot of content in the contributor guide that belongs better to "Quantization Overview", so here we split the content and put them in the right pages.
Update [ghstack-poisoned]
lint
jainapurva
force-pushed
the
gh/jainapurva/3/head
branch
from
49c8514 to
d42c725
Compare
jainapurva
added
topic: for developers
Summary: Adds the workaround from pytorch/pytorch#141881 to the torchao float8 rowwise recipe, to reduce memory usage when FSDP is on. Test Plan: tested in torchtitan, LLaMa 3 8B 8H100 training with rowwise peak memory decreased from 67GiB to 59GiB Reviewers: Subscribers: Tasks: Tags:
* more stringent test for CPUOffloadOptimizer * fix missing sync
* synchronize param H2D * let CPU offload inherits Optimizer * add scheduler to test
stack-info: PR: #1658, branch: drisspg/stack/32
|
|
||
| def _get_to_kwargs(self, *args, **kwargs): |
There was a problem hiding this comment.
�this should be preserved I think, since it's called in child classes, we can just call the util function above
Stack from ghstack (oldest at bottom):