Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tempo causes collect() deprecation warning when used in DLT pipeline #408

Open
BradLotsberg opened this issue May 28, 2024 · 1 comment
Open
Assignees
Labels
enhancement New feature or request good first issue Good for newcomers performance Issues with the time to execute operations or other performance issues

Comments

@BradLotsberg
Copy link

There are a number of collect()[0][0] instances in the Tempo code which trigger deprecation warnings if used in DLT pipelines. Perhaps replace collect()[0][0] with head()[0]. Accomplishes the same thing while avoiding the deprecation warning and might even bring a marginal performance boost since only the top row instead of the whole data frame would move to the driver node.

@tnixon tnixon self-assigned this May 29, 2024
@tnixon tnixon added enhancement New feature or request good first issue Good for newcomers performance Issues with the time to execute operations or other performance issues labels May 29, 2024
@tnixon
Copy link
Contributor

tnixon commented May 29, 2024

This is a good suggestion, thanks @BradLotsberg.
@yuriymargulis-db - perhaps you can pick this one up?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers performance Issues with the time to execute operations or other performance issues
Projects
None yet
Development

No branches or pull requests

3 participants