Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pipe_input target #1173

Merged
merged 5 commits into from
Oct 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions docs/reference/decorators/config_when.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,13 @@ Note the following:

* ``@config`` If you're feeling adventurous, you can pass in a lambda function that takes in the entire configuration and resolves to ``True`` or ``False``. You probably don't want to do this.

* To always exclude a function (such as helper functions) from the DAG you can also use ``@hamilton_skip``.
* To always exclude a function (such as helper functions) from the DAG the most straightforward and preferred pattern is to prefix it with "_", but you can also use ``@hamilton_exclude``.

----

**Reference Documentation**

.. autoclass:: hamilton.function_modifiers.config
:members: when, when_in, when_not, when_not_in
:special-members: __init__

.. autoclass:: hamilton.function_modifiers.hamilton_skip
.. autoclass:: hamilton.function_modifiers.configuration.hamilton_exclude

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -2,37 +2,37 @@

import pandas as pd

from hamilton.function_modifiers import hamilton_skip, pipe_output, source, step, value
from hamilton.function_modifiers import hamilton_exclude, pipe_output, source, step, value


# data1 and data2
@hamilton_skip
@hamilton_exclude
def filter_(some_data: pd.DataFrame) -> pd.DataFrame:
return some_data.dropna()


@hamilton_skip
@hamilton_exclude
def test_foo(a, b, c):
return a + b + c


# data 2
# this is for value
@hamilton_skip
@hamilton_exclude
def add_missing_value(some_data: pd.DataFrame, missing_row: List[Any]) -> pd.DataFrame:
some_data.loc[-1] = missing_row
return some_data


# data 2
# this is for source
@hamilton_skip
@hamilton_exclude
def join(some_data: pd.DataFrame, other_data: pd.DataFrame) -> pd.DataFrame:
return some_data.set_index("col_2").join(other_data.set_index("col_1"))


# data1 and data2
@hamilton_skip
@hamilton_exclude
def sort(some_data: pd.DataFrame) -> pd.DataFrame:
columns = some_data.columns
return some_data.sort_values(by=columns[0])
Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,24 @@
from typing import Dict

from hamilton.function_modifiers import extract_fields, hamilton_skip, pipe_output, step
from hamilton.function_modifiers import (
extract_fields,
hamilton_exclude,
pipe_output,
step,
)


@hamilton_skip
@hamilton_exclude
def pre_step(something: int) -> int:
return something + 10


@hamilton_skip
@hamilton_exclude
def post_step(something: int) -> int:
return something + 100


@hamilton_skip
@hamilton_exclude
def something_else(something: int) -> int:
return something + 1000

Expand Down
2 changes: 1 addition & 1 deletion hamilton/function_modifiers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

# The config decorator
config = configuration.config
hamilton_skip = configuration.hamilton_skip()
hamilton_exclude = configuration.hamilton_exclude()

# Dependency Specification
# Helper functions to specify dependency sources for parameterization
Expand Down
9 changes: 5 additions & 4 deletions hamilton/function_modifiers/configuration.py
Original file line number Diff line number Diff line change
Expand Up @@ -257,15 +257,16 @@ def LEAD_LOG_BASS_MODEL_TIMES_TREND(
return config(resolver, config_used=list(resolver.optional_config))


class hamilton_skip(base.NodeResolver):
class hamilton_exclude(base.NodeResolver):
"""Decorator class that excludes a function from the DAG.

This is useful for decorating helper functions without the need to prefix them with "_" and
use them either inside other nodes or in conjunction with ``step`` or ``apply_to``.
The preferred way to hide functions from the Hamilton DAG is to prefix them with "_". However,
for the exceptional case, it can be useful for decorating helper functions without the need to prefix
them with "_" and use them either inside other nodes or in conjunction with ``step`` or ``apply_to``.

.. code-block:: python

@hamilton_skip
@hamilton_exclude
def helper(...) -> ...:
'''This will not be part of the DAG'''
...
Expand Down
Loading