Ax is incompatible with SQLAlchemy 2.0+; incompatibility errors arise when using only JSON storage, too #1697
Assignees
Labels
Comments
|
I have the same issue. When I use Am I missing something? Thank you very much for your help. |
|
Sorry the two of you are running into this issue -- this process could certainly better documented in our tutorials. In order to encode custom metrics and runners you must ensure they are included in Ax's storage registries. These are objects that tell Ax how to handle the encoding and decoding of objects. We have a helper abstraction called the RegistryBundle that handles all this for you under the hood; see it in use in the following snippet: https://ax.dev/tutorials/scheduler.html#6.-Leveraging-SQL-storage-and-experiment-resumption @ekurtgl for you the corrected code will look like this: @fernexda yours will look the same as above but with |
|
Thank you for your help @mpolson64 , however the below code throws the following error: Do you have any idea on what might be the reason? Thanks. |
|
This looks like a bug on our end -- I will investigate this and report back once I have a repro. In the meantime, could you provide a list of all the packages you're using in this environment as well as their versions? This is typically best done via |
|
They you very much, @mpolson64 , your solution works like a charm. I have two objectives, and I tinkered a bit to load the experiment using 'decoder' instead of 'encoder'. This is my code: Thanks again for your quick response! |
|
It still doesn't work for me. My package list is here: |
|
Hi @ekurtgl, this is just a short-term work-around, but I was able to resolve this problem by downgrading to SQLAlchemy v1. E.g. But in my opinion this is definitely a bug because if someone only wants to save experiments to JSON, they shouldn't have to install sql alchemy. |
|
Thank you @16acdg -- yes, Ax is currently incompatible with SQLAlchemy 2+. We are working on getting a fix out for this soon. We will also intend to look into solving this behavior where JSON storage is broken due to an incompatible SQLAlchemy version; currently this code is too tightly coupled and users should be able to save custom JSON objects without importing SQLAlchemy code at all. |
lena-kashtelyan
changed the title
|
Upgrading SQLAlchemy to 2.0 will likely take some substantial investment on our part, moving this task to our master tracking Issue so it can make it on to our team's roadmap #566. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I am tryinng to save the NAS experiment presented here to a file, so that I can load it later on to do further analysis and visualization based on the results. I use the following code:
import ax.storage ax.storage.json_store.save.save_experiment(experiment, 'ax_nas_results.json')However, it throws error:
`
JSONEncodeError Traceback (most recent call last)
Cell In[71], line 2
1 import ax.storage
----> 2 ax.storage.json_store.save.save_experiment(experiment, 'ax_nas_results.json')
3 # dir(ax.storage)
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/save.py:45, in save_experiment(experiment, filepath, encoder_registry, class_encoder_registry)
42 if not filepath.endswith(".json"):
43 raise ValueError("Filepath must end in .json")
---> 45 json_experiment = object_to_json(
46 experiment,
47 encoder_registry=encoder_registry,
48 class_encoder_registry=class_encoder_registry,
49 )
50 with open(filepath, "w+") as file:
51 file.write(json.dumps(json_experiment))
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:83, in object_to_json(obj, encoder_registry, class_encoder_registry)
81 if _type in encoder_registry:
82 obj_dict = encoder_registry_type
---> 83 return {
84 k: object_to_json(
85 v,
86 encoder_registry=encoder_registry,
87 class_encoder_registry=class_encoder_registry,
88 )
89 for k, v in obj_dict.items()
90 }
92 # Python built-in types +
typingmodule types93 if _type in (str, int, float, bool, type(None)):
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:84, in (.0)
81 if _type in encoder_registry:
82 obj_dict = encoder_registry_type
83 return {
---> 84 k: object_to_json(
85 v,
86 encoder_registry=encoder_registry,
87 class_encoder_registry=class_encoder_registry,
88 )
89 for k, v in obj_dict.items()
90 }
92 # Python built-in types +
typingmodule types93 if _type in (str, int, float, bool, type(None)):
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:83, in object_to_json(obj, encoder_registry, class_encoder_registry)
81 if _type in encoder_registry:
82 obj_dict = encoder_registry_type
---> 83 return {
84 k: object_to_json(
85 v,
86 encoder_registry=encoder_registry,
87 class_encoder_registry=class_encoder_registry,
88 )
89 for k, v in obj_dict.items()
90 }
92 # Python built-in types +
typingmodule types93 if _type in (str, int, float, bool, type(None)):
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:84, in (.0)
81 if _type in encoder_registry:
82 obj_dict = encoder_registry_type
83 return {
---> 84 k: object_to_json(
85 v,
86 encoder_registry=encoder_registry,
87 class_encoder_registry=class_encoder_registry,
88 )
89 for k, v in obj_dict.items()
90 }
92 # Python built-in types +
typingmodule types93 if _type in (str, int, float, bool, type(None)):
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:83, in object_to_json(obj, encoder_registry, class_encoder_registry)
81 if _type in encoder_registry:
82 obj_dict = encoder_registry_type
---> 83 return {
84 k: object_to_json(
85 v,
86 encoder_registry=encoder_registry,
87 class_encoder_registry=class_encoder_registry,
88 )
89 for k, v in obj_dict.items()
90 }
92 # Python built-in types +
typingmodule types93 if _type in (str, int, float, bool, type(None)):
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:84, in (.0)
81 if _type in encoder_registry:
82 obj_dict = encoder_registry_type
83 return {
---> 84 k: object_to_json(
85 v,
86 encoder_registry=encoder_registry,
87 class_encoder_registry=class_encoder_registry,
88 )
89 for k, v in obj_dict.items()
90 }
92 # Python built-in types +
typingmodule types93 if _type in (str, int, float, bool, type(None)):
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:96, in object_to_json(obj, encoder_registry, class_encoder_registry)
94 return obj
95 elif _type is list:
---> 96 return [
97 object_to_json(
98 x,
99 encoder_registry=encoder_registry,
100 class_encoder_registry=class_encoder_registry,
101 )
102 for x in obj
103 ]
104 elif _type is tuple:
105 return tuple(
106 object_to_json(
107 x,
(...)
111 for x in obj
112 )
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:97, in (.0)
94 return obj
95 elif _type is list:
96 return [
---> 97 object_to_json(
98 x,
99 encoder_registry=encoder_registry,
100 class_encoder_registry=class_encoder_registry,
101 )
102 for x in obj
103 ]
104 elif _type is tuple:
105 return tuple(
106 object_to_json(
107 x,
(...)
111 for x in obj
112 )
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:83, in object_to_json(obj, encoder_registry, class_encoder_registry)
81 if _type in encoder_registry:
82 obj_dict = encoder_registry_type
---> 83 return {
84 k: object_to_json(
85 v,
86 encoder_registry=encoder_registry,
87 class_encoder_registry=class_encoder_registry,
88 )
89 for k, v in obj_dict.items()
90 }
92 # Python built-in types +
typingmodule types93 if _type in (str, int, float, bool, type(None)):
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:84, in (.0)
81 if _type in encoder_registry:
82 obj_dict = encoder_registry_type
83 return {
---> 84 k: object_to_json(
85 v,
86 encoder_registry=encoder_registry,
87 class_encoder_registry=class_encoder_registry,
88 )
89 for k, v in obj_dict.items()
90 }
92 # Python built-in types +
typingmodule types93 if _type in (str, int, float, bool, type(None)):
File ~/anaconda3/envs/tpot/lib/python3.10/site-packages/ax/storage/json_store/encoder.py:185, in object_to_json(obj, encoder_registry, class_encoder_registry)
178 return {"_type": f"torch{_type.name}", "value": torch_type_to_str(obj)}
180 err = (
181 f"Object {obj} passed to
object_to_json(of type {_type}, module: "182 f"{_type.module}) is not registered with a corresponding encoder "
183 "in ENCODER_REGISTRY."
184 )
--> 185 raise JSONEncodeError(err)
JSONEncodeError: Object MyTensorboardMetric('val_acc') passed to
object_to_json(of type , module: main) is not registered with a corresponding encoder in ENCODER_REGISTRY`
As far as I understand, I need to encode my metrics, but I am not sure how to do it. Any ideas on this? Is this the best way to export the experiments after the optimization process is done? Thank you.
The text was updated successfully, but these errors were encountered: