Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "[COR-2297/] Fix nested offloaded type validation (#552) (#5996)" #6045

Merged
merged 2 commits into from
Nov 22, 2024

Conversation

eapolinario
Copy link
Contributor

This reverts commit f20b8aa.

Tracking issue

Related to #5103

Why are the changes needed?

In flyteorg/flytekit#2872 we're handling the case of offloading literals in map tasks single executions explicitly. The impact of this is that we can simplify the handling of this case in the backend, invalidating #5996.

What changes were proposed in this pull request?

Revert #5996.

How was this patch tested?

Confirmed that the original test workflow works:

import logging
from typing import List
from flytekit import map_task, task, workflow,LaunchPlan

logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger("flytekit")
logger.setLevel(logging.DEBUG)

@task(cache=True, cache_version="1.1")
def my_30mb_task(i: str) -> str:
    return f"Hello world {i}" * 30 * 100 * 1024

@task(cache=True, cache_version="1.1")
def generate_strs(count: int) -> List[str]:
    return ["a"] * count

@workflow
def my_30mb_wf(mbs: int) -> List[str]:
  strs = generate_strs(count=mbs)
  return map_task(my_30mb_task)(i=strs)

@workflow
def big_inputs_wf(input: List[str]):
   noop()

@task(cache=True, cache_version="1.1")
def noop():
    ...

big_inputs_wf_lp = LaunchPlan.get_or_create(name="big_inputs_wf_lp", workflow=big_inputs_wf)

@workflow
def ref_wf(mbs: int):
  big_inputs_wf_lp(input=my_30mb_wf(mbs))

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Copy link

codecov bot commented Nov 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 37.06%. Comparing base (e13babb) to head (20ecd7f).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6045      +/-   ##
==========================================
- Coverage   37.06%   37.06%   -0.01%     
==========================================
  Files        1316     1316              
  Lines      132191   132178      -13     
==========================================
- Hits        49002    48988      -14     
- Misses      78926    78928       +2     
+ Partials     4263     4262       -1     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.07% <ø> (ø)
unittests-flytecopilot 22.23% <ø> (ø)
unittests-flytectl 62.46% <ø> (-0.05%) ⬇️
unittests-flyteidl 7.25% <ø> (ø)
unittests-flyteplugins 53.72% <ø> (ø)
unittests-flytepropeller 42.63% <100.00%> (-0.03%) ⬇️
unittests-flytestdlib 57.59% <ø> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

Signed-off-by: Eduardo Apolinario <[email protected]>
Copy link
Contributor

@wild-endeavor wild-endeavor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 this is just a pure git revert

@eapolinario eapolinario merged commit b220c9c into master Nov 22, 2024
52 checks passed
@eapolinario eapolinario deleted the revert-5996 branch November 22, 2024 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants