Skip to content

Commit

Permalink
refine
Browse files Browse the repository at this point in the history
Signed-off-by: Weichen Xu <[email protected]>
  • Loading branch information
WeichenXu123 committed Nov 29, 2024
1 parent e196604 commit a0291eb
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions python/ray/util/spark/databricks_hook.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,13 +189,13 @@ def custom_environment_variables(self):
conf[DATABRICKS_CLIENT_ID] = os.environ[DATABRICKS_CLIENT_ID]
conf[DATABRICKS_CLIENT_SECRET] = os.environ[DATABRICKS_CLIENT_SECRET]
else:
# Ray nodes runs as subprocess of spark UDF or spark driver,
# in databricks, it doens't have MLflow service credentials
# so it can't use MLflow.
# 'DISABLE_MLFLOW_INTEGRATION' is the environmental variable to disable
# huggingface transformers MLflow integration.
conf["DISABLE_MLFLOW_INTEGRATION"] = "TRUE"

warn_msg = (
"MLflow integration is disabled in Ray tasks. To enable MLflow integration, "
"MLflow integration is not available in Ray tasks."
"To enable MLflow integration, "
"you need to set environmental variables DATABRICKS_HOST + DATABRICKS_TOKEN,"
"or set environmental variables "
"DATABRICKS_HOST + DATABRICKS_CLIENT_ID + DATABRICKS_CLIENT_SECRET "
Expand Down

0 comments on commit a0291eb

Please sign in to comment.