Add support for ragged inputs to model #666
Conversation
Co-authored-by: Marc Romeyn <[email protected]>
Documentation previewhttps://nvidia-merlin.github.io/Transformers4Rec/review/pr-666 |
| ) | ||
| model_output = model(inference_inputs) | ||
|
|
||
| # if the model is traced with ragged inputs it can only be called with ragged inputs |
There was a problem hiding this comment.
note that when tracing the model, the representation used as the input determines what the inputs to the traced model expects. (padded vs ragged)
| batch_padded = {} | ||
| for col_name, col in TensorTable(batch).items(): |
There was a problem hiding this comment.
TensorTable is not currently compatible with torch.jit.script compilation.
example of one of the errors that shows up (I don't think it prints out all errors, only the first it encounters -> there may be more unsupported things apart from the below example)
E torch.jit.frontend.UnsupportedNodeError: SetComp aren't supported:
E File "/workspace/merlin/core/merlin/table/tensor_table.py", line 61
E def _validate_columns(self, cols_dict):
E col_types = {type(col_obj) for col_obj in cols_dict.values()}
E ~ <--- HERE
E if len(col_types) >= 2:
E raise TypeError(
E '__torch__.merlin.table.tensor_table.TensorTable' is being compiled since it was called from 'pad_batch'
| ] | ||
| ), | ||
| ) | ||
| assert torch.equal( |
There was a problem hiding this comment.
dense sequence inputs are not padded as part of this pad_inputs currently. Assuming we'll either have ragged or padded sequence inputs, not a mix of both
| @@ -481,6 +482,7 @@ def __init__( | |||
| head_reduction: str = "mean", | |||
| optimizer: Type[torch.optim.Optimizer] = torch.optim.Adam, | |||
| name: str = None, | |||
| max_sequence_length: Optional[int] = None, | |||
There was a problem hiding this comment.
Added a max_sequence_length to limit the size of the padding when receiving ragged inputs.
|
|
||
|
|
||
| @torch.jit.script | ||
| def pad_inputs(inputs: Dict[str, torch.Tensor], max_sequence_length: Optional[int] = None): |
There was a problem hiding this comment.
| def pad_inputs(inputs: Dict[str, torch.Tensor], max_sequence_length: Optional[int] = None): | |
| def pad_inputs( | |
| inputs: Dict[str, torch.Tensor], max_sequence_length: Optional[int] = None | |
| ) -> Dict[str, torch.Tensor]: |
Part of NVIDIA-Merlin/Merlin#255
Goals ⚽
Implementation Details 🚧
{feature}__values{feature}__offests.Testing Details 🔍