Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up internal column logic in _run_classifier_helper function #457

Merged
merged 3 commits into from
Jan 3, 2025
Merged

Clean up internal column logic in _run_classifier_helper function #457

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5712395
first commit
sarahyurick Dec 23, 2024
498910d
working code and pytest
sarahyurick Dec 24, 2024
5377c19
Merge branch 'main' into cleanup_run_classifier_helper
sarahyurick Jan 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 10 additions & 25 deletions nemo_curator/classifiers/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,44 +121,29 @@ def _run_classifier_helper(
prob_col: str = None,
) -> "dask_cudf.DataFrame":

keep_prob = prob_col is not None
prob_internal_col = "_prob"
# TODO: Make crossfit handle this cleanly
pred_internal_col = "labels"
df["sliced_text"] = df[text_field].str.slice(0, max_chars)
if prob_col:
df[prob_col] = 0
Comment on lines +124 to +125
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted CrossFit to internally create the prob_col, but I was having trouble getting this to work. Specifically, I was able to get the entire classification pipeline to work, but then at the end prob_col would be dropped somewhere and not returned with the result.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the PR. I like the changes, thanks for cleaning it up.

Would you mind creating a issue on crossfit (with a MRE), we can merge this PR and then fix it in crossfit.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have opened an issue here: rapidsai/crossfit#108. Please LMK if there is anything else I can clarify there. Thanks!

else:
prob_col = "_prob"

columns_to_keep_list = df.columns.to_list()
columns_to_keep_list.remove("sliced_text")

classifier_pipe = op.Sequential(
op.Tokenizer(model, cols=["sliced_text"], tokenizer_type="default"),
op.Tokenizer(
model, cols=[text_field], tokenizer_type="default", max_chars=max_chars
),
op.Predictor(
model,
sorted_data_loader=True,
batch_size=batch_size,
pred_output_col=prob_internal_col,
pred_output_col=prob_col,
),
op.Labeler(labels, cols=[prob_col], suffix=label_col),
VibhuJawa marked this conversation as resolved.
Show resolved Hide resolved
repartition=df.npartitions,
keep_cols=columns_to_keep_list,
)
df = classifier_pipe(df)

# TODO: Make crossfit handle this cleanly
# to prevent the labeler from dropping the prob_internal_col
# and combine it into a single step
labeling_pipe = op.Sequential(
op.Labeler(labels, cols=[prob_internal_col]),
keep_cols=columns_to_keep_list + [prob_internal_col],
)
df = labeling_pipe(df)

if keep_prob:
df = df.rename(
columns={prob_internal_col: prob_col, pred_internal_col: label_col},
)
else:
df = df.rename(columns={pred_internal_col: label_col})
df = df.drop(columns=[prob_internal_col])

return df


Expand Down
41 changes: 25 additions & 16 deletions tests/test_classifiers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import os

import pytest
from distributed import Client

Expand Down Expand Up @@ -48,24 +46,35 @@ def domain_dataset():


@pytest.mark.gpu
def test_domain_classifier(gpu_client, domain_dataset):
@pytest.mark.parametrize("keep_prob", [True, False])
def test_domain_classifier(gpu_client, domain_dataset, keep_prob):
from nemo_curator.classifiers import DomainClassifier

classifier = DomainClassifier()
result_dataset = classifier(dataset=domain_dataset)
result_pred = result_dataset.df.compute()["domain_pred"]
if keep_prob:
prob_column = "domain_prob"
else:
prob_column = None

expected_pred = cudf.Series(
[
"Computers_and_Electronics",
"Finance",
"Health",
"Jobs_and_Education",
"Travel_and_Transportation",
]
)
classifier = DomainClassifier(prob_column=prob_column)
result_dataset = classifier(dataset=domain_dataset)

assert result_pred.equals(expected_pred)
if keep_prob:
result_df = result_dataset.df.compute()
assert "domain_prob" in result_df.columns
else:
result_pred = result_dataset.df.compute()["domain_pred"]

expected_pred = cudf.Series(
[
"Computers_and_Electronics",
"Finance",
"Health",
"Jobs_and_Education",
"Travel_and_Transportation",
]
)

assert result_pred.equals(expected_pred)


@pytest.mark.gpu
Expand Down
Loading