Skip to content

Commit

Permalink
[SPARK-50823][PYTHON] Upgrade cloudpickle from 3.1.0 to 3.1.1
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR proposes to upgrade Cloudpickle to 3.1.1

### Why are the changes needed?

To leverage bug fixes.

### Does this PR introduce _any_ user-facing change?

No for now. It contains several changes for Python 3.14 support preparation.

### How was this patch tested?

Existing unittests should cover.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#49496 from HyukjinKwon/upgrade-cloudpickle2.

Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
  • Loading branch information
HyukjinKwon committed Jan 15, 2025
1 parent 6f3b778 commit d2ecfd7
Show file tree
Hide file tree
Showing 2 changed files with 12 additions and 4 deletions.
2 changes: 1 addition & 1 deletion python/pyspark/cloudpickle/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

__doc__ = cloudpickle.__doc__

__version__ = "3.1.0"
__version__ = "3.1.1"

__all__ = [ # noqa
"__version__",
Expand Down
14 changes: 11 additions & 3 deletions python/pyspark/cloudpickle/cloudpickle.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
import logging
import opcode
import pickle
from pickle import _getattribute
from pickle import _getattribute as _pickle_getattribute
import platform
import struct
import sys
Expand Down Expand Up @@ -192,6 +192,14 @@ def _is_registered_pickle_by_value(module):
return False


if sys.version_info >= (3, 14):
def _getattribute(obj, name):
return _pickle_getattribute(obj, name.split('.'))
else:
def _getattribute(obj, name):
return _pickle_getattribute(obj, name)[0]


def _whichmodule(obj, name):
"""Find the module an object belongs to.
Expand Down Expand Up @@ -219,7 +227,7 @@ def _whichmodule(obj, name):
):
continue
try:
if _getattribute(module, name)[0] is obj:
if _getattribute(module, name) is obj:
return module_name
except Exception:
pass
Expand Down Expand Up @@ -293,7 +301,7 @@ def _lookup_module_and_qualname(obj, name=None):
return None

try:
obj2, parent = _getattribute(module, name)
obj2 = _getattribute(module, name)
except AttributeError:
# obj was not found inside the module it points to
return None
Expand Down

0 comments on commit d2ecfd7

Please sign in to comment.