Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement migration sequencing (phase 2) #3009

Open
wants to merge 33 commits into
base: main
Choose a base branch
from

Conversation

ericvergnaud
Copy link
Contributor

@ericvergnaud ericvergnaud commented Oct 17, 2024

Changes

Implement migration steps for notebooks and python files

  • Test cycles with notebooks referencing other notebooks

Linked issues

Progresses #1415

Functionality

None

Tests

  • added unit tests

@ericvergnaud ericvergnaud requested a review from a team as a code owner October 17, 2024 17:39
@ericvergnaud ericvergnaud marked this pull request as draft October 17, 2024 17:40
def register_workflow_job(self, job: jobs.Job) -> MigrationNode:
job_node = self._nodes.get(("JOB", str(job.job_id)), None)
job_node = self._nodes.get(("WORKFLOW", str(job.job_id)), None)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

align with ObjectInfo type enum

Copy link

github-actions bot commented Oct 17, 2024

✅ 68/68 passed, 4 skipped, 59m43s total

Running from acceptance #7064

@ericvergnaud ericvergnaud changed the title Migration sequencing phase 2 Implement migration sequencing (phase 2) Oct 18, 2024
@ericvergnaud ericvergnaud marked this pull request as ready for review October 18, 2024 13:11
…-phase-2

# Conflicts:
#	src/databricks/labs/ucx/assessment/sequencing.py
#	tests/unit/assessment/test_sequencing.py
# Conflicts:
#	src/databricks/labs/ucx/assessment/clusters.py
#	src/databricks/labs/ucx/assessment/jobs.py
…-phase-2

# Conflicts:
#	src/databricks/labs/ucx/framework/owners.py
ws_path = WorkspacePath(self._ws, object_id)
object_owner = WorkspacePathOwnership(self._admin_locator, self._ws).owner_of(ws_path)
else:
raise ValueError(f"{object_type} not supported yet!")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where this exception is caught? it'll crash the assessment workflow if unhandled.

@@ -78,8 +78,8 @@ def as_message(self) -> str:


class WorkflowTask(Dependency):
def __init__(self, ws: WorkspaceClient, task: jobs.Task, job: jobs.Job):
loader = WrappingLoader(WorkflowTaskContainer(ws, task, job))
def __init__(self, ws: WorkspaceClient, task: jobs.Task, job: jobs.Job, cache: WorkspaceCache | None = None):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't add | None as constructor dependencies - it leads to non-deterministic logic and subtle bugs that are harder to diagnose later.

Base automatically changed from migration-sequencing-phase-1 to main November 1, 2024 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants