Automated PR: Downstream develop rebase new changes #60
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…#31395) * Add llama3-llava-next-8b to llava_next conversion script Adds support for the lmms-lab/llama3-llava-next-8b model to the convert_llava_next_weights_to_hf.py script, along with an example prompt generated from the llava_llama_3 conv_template in the LLaVA-NeXT repo. * Exclude <|begin_of_text|> from prompt example This token gets added automatically, so it should not be included in the prompt example. * Add llava-next-72b and llava-next-110b Adds the Qwen-based LLaVA-Next models to the conversion script, along with changes to load the models on multiple GPUs for inference. * Add llama3 and qwen prompt formats to docs * Chat prompt and padding side left for llama3 batched * update * Update src/transformers/models/llava_next/convert_llava_next_weights_to_hf.py Co-authored-by: amyeroberts <[email protected]> * Update src/transformers/models/llava_next/convert_llava_next_weights_to_hf.py Co-authored-by: amyeroberts <[email protected]> * remove code * better naming --------- Co-authored-by: raushan <[email protected]> Co-authored-by: Raushan Turganbay <[email protected]> Co-authored-by: amyeroberts <[email protected]>
* pad on right if training * docs * add tests
* [whisper integration] use parquet dataset for testing * propagate to others * more propagation * last one
…ace#31749) * [whisper] remove un-necessary transpose for fa2 attention * propagate
* fix mask creation of gpt2 and gpt_neox caused by me * forgot the reshape of masks when shape > 2 * add tests for gpt neox and gpt2 * nit on a comment
encapsulate chat template logic
* Add YaRN and Dynamic-YaRN RoPE Scaling Methods YaRN (Yet another RoPE extension method) combines the NTK-By-Parts Interpolation and Attention Scaling methods, improving upon existing RoPE interpolation methods for longer context window sizes. Fine-tuned models maintain their original performance across benchmarks while enabling efficient extrapolation and transfer learning for quicker convergence, especially in compute-limited environments. We implement YaRN and Dynamic-YaRN for the following list of models: - LLaMA - Falcon - GPT-NeoX - Olmo - Persimmon - Phi - StableLM - OpenLLaMA New unit tests are added to assert YaRN's correct behavior on both short and long sequence inputs. For more details, please refer to https://arxiv.org/abs/2309.00071. Co-authored-by: Miguel Almeida <[email protected]> * Refactor YaRN implementation for LLaMA Iterate on YaRN implementation for LLaMA and remove diff from remaining models for increased PR modularity. This commit includes the following changes: - Merge 'yarn_rope_scaling' and 'rope_scaling' dictionaries - Remove unnecessary attributes ('extrapolation_factor' and 'finetuned') from YaRN classes - Inherit 'forward' method in YaRN classes from superclass - Rename 'yarn' method to 'compute_yarn_scaling' - Extend YaRN tests with further assertions - Fix style inconsistencies Co-authored-by: Miguel Monte e Freitas <[email protected]> * Refactor Tensor Building Logic for YaRN - Comply with the the tensor building logic introduced in huggingface#30743 - Add referencing to the optimized Attention Factor equation - Remove Dynamic YaRN for a more agile deployment Co-authored-by: mig-mfreitas <[email protected]> * remove unwanted file --------- Co-authored-by: Miguel Almeida <[email protected]> Co-authored-by: mig-mfreitas <[email protected]> Co-authored-by: Joao Gante <[email protected]>
add attribute to model Signed-off-by: Daniel Lok <[email protected]>
…huggingface#31979) * Change resize_token_embeddings to make it return same Class that is passed to it * Add explanatory comment as requested in review * Add explanatory comments for add resizing function in lxmert * Add comment for padding_idx and moving _resize_bias in lxmert to LxmertForPreTraining --------- Co-authored-by: Prashanth Sateesh <[email protected]> Co-authored-by: Prashanth Sateesh <[email protected]>
Co-authored-by: amyeroberts <[email protected]> Co-authored-by: Arthur <[email protected]>
* gguf conversion forces add_prefix_space=False for llama3, this is not required and forces from_slow, which fails. changing to None + test * typo * clean test
Add the lru_cache for speed
--------- Co-authored-by: Merve Noyan <[email protected]>
* Update README.md * tests: forward ok * backward test done * done testing * removed check. scripts * Update README.md * added use_mambapy arg * fixed typo in warning * protected imports w/ mambapy package * delete pscan.py + raise rather than assert * Update import_utils.py * fix whitespaces and unused import * trailing whitespace + import block unformatted * Update modeling_mamba.py * transpose before pscan * shape comment * ran make style * use_mambapy=False by default Co-authored-by: Arthur <[email protected]> * ran make fix-copies --------- Co-authored-by: Arthur <[email protected]>
* renamed phi3 rope_scaling type * fixed trailing whitespaces * fixed test * added warning * fixed format
…e#32148) Revert "Incorrect Whisper long-form decoding timestamps (huggingface#32003)" This reverts commit cd48553.
…ingface#31857) * feat(cache): StaticCache uses index_copy_ to avoid useless copy Using index_copy_ allows for explicit in-place change of the tensor. Some backends (XLA) will otherwise copy the tensor, making the code slower and using more memory. Proposed implementation will end up using less memory and on XLA will result in less compilation, but the change is also quite generic, making no change whatsoever on CUDA or CPU backend. * feat(cache): SlidingWindowCache uses index_copy_ to avoid useless copy Applying the same change done in StaticCache. * fix(cache): fallback of index_copy_ when not implemented * fix(cache): in index_copy_ ensure tensors are on same device * [run slow] llama * fix(cache): add move of cache_position to same device in SlidingWindowCache * Revert "[run slow] llama" This reverts commit 02608dd.
…r search (huggingface#31924) Update integration_utils.py Added additional kwarg
…ith Position IDs (huggingface#31629) * add DataCollatorBatchFlattening * Update data_collator.py * change name * new FA2 flow if position_ids is provided * add comments * minor fix * minor fix data collator * add test cases for models * add test case for data collator * remove extra code * formating for ruff check and check_repo.py * ruff format ruff format tests src utils * custom_init_isort.py
* Updated ruff version and fixed the required code accorindg to the latest version. * Updated ruff version and fixed the required code accorindg to the latest version. * Added noqa directive to ignore 1 error shown by ruff
Co-authored-by: Arthur Zucker <[email protected]>
…face#32160) Fixed an if condition always evaluating to true.
…eights in the layer (huggingface#32171) * adds: extra_repr() to MambaRMSNorm to include the hidden size of the layer * style fix with ruff:
…than the ones present at import time. (huggingface#32153) * fix: default value reflects the runtime environment variables rather than the ones present at import time. * Fix: Change `deterministic` to None by default; use env var if None
* Update qwen2.md outdated description * Update qwen2.md amended * Update qwen2.md Update * Update qwen2.md fix wrong version code, now good to go
Remove conversation pipeline tests
…#31913) Add accelerate version check, needs accelerate>=0.33.0
…uggingface#32678) * Fixed failing tests in tests/utils/test_add_new_model_like.py * Fixed formatting using ruff. * Small nit.
update list of people to tag
* Add TorchAOHfQuantizer Summary: Enable loading torchao quantized model in huggingface. Test Plan: local test Reviewers: Subscribers: Tasks: Tags: * Fix a few issues * style * Added tests and addressed some comments about dtype conversion * fix torch_dtype warning message * fix tests * style * TorchAOConfig -> TorchAoConfig * enable offload + fix memory with multi-gpu * update torchao version requirement to 0.4.0 * better comments * add torch.compile to torchao README, add perf number link --------- Co-authored-by: Marc Sun <[email protected]>
JetMoeIntegrationTest Co-authored-by: ydshieh <[email protected]>
…ingface#32669) * Update the Kubernetes CPU training example * Add namespace arg Signed-off-by: Dina Suehiro Jones <[email protected]> --------- Signed-off-by: Dina Suehiro Jones <[email protected]>
…2475) Fixed unknown config option doctest_glob.
Unpin deepspeed
Updated few workflows to the latest versions.
…32837) Corrected the model checkpoint.
* fix beam search in video llava * [run-slow] video_llava
* add back the position ids * fix failing test
* use head_dim if in config for RoPE * typo * simplify with getattr
* fix on xpu * [run_all]
* more precise name * better docstrings * Update src/transformers/cache_utils.py Co-authored-by: Arthur <[email protected]> --------- Co-authored-by: Arthur <[email protected]>
…e#32844) * Fix: fix all model_type of Llava-Next-Video to llava_next_video * Fix doc for llava_next_video * * Fix formatting issues * Change llava-next-video.md file name into llava_next_video.md to make it compatible with implementation * Fix docs TOC for llava-next-video
* improve _get_is_as_tensor_fns * format
…enamed, and provide a step forward (huggingface#32656) * Fin * Modify msg * Finish up nits
…uggingface#32674) * Fix beam_constraints.Constraint.advance() docstring * Update src/transformers/generation/beam_constraints.py Co-authored-by: Steven Liu <[email protected]> --------- Co-authored-by: Joao Gante <[email protected]> Co-authored-by: Steven Liu <[email protected]>
* Update testing_utils.py * changes * from env var * name change * debug * name change
* skip failures * navi31 skip * mi300 skips * conversational test backwards compatability * mi300 skips
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR was created automatically by the Fork Maintenance System to sync changes from the downstream main into downstream develop.