Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 1.0.0 #3

Merged
merged 31 commits into from
Sep 7, 2024
Merged

Release 1.0.0 #3

merged 31 commits into from
Sep 7, 2024

Conversation

x-tabdeveloping
Copy link
Member

@x-tabdeveloping x-tabdeveloping commented Aug 21, 2024

Changed the API completely to make the library more comfortable to use.
Main changes:

Trooper

The brand new Trooper interface allows you not to have to specify what model type you wish to use.
Stormtrooper will automatically detect the model type from the specified name.

from stormtrooper import Trooper

# This loads a setfit model
model = Trooper("all-MiniLM-L6-v2")

# This loads an OpenAI model
model = Trooper("gpt-4")

# This loads a Text2Text model
model = Trooper("google/flan-t5-base")

Unified zero and few-shot classification

You no longer have to specify whether a model should be a few or a zero-shot classifier when initialising it.
If you do not pass any training examples, it will be automatically assumed that the model should be zero-shot.

# This is a zero-shot model
model.fit(None, ["dog", "cat"])

# This is a few-shot model
model.fit(["he was a good boy", "just lay down on my laptop"], ["dog", "cat"])

Reimplemented SetFit

The SetFit library caused me many headaches, and has unfixed bugs, and I didn't want to downgrade everything.
I implemented the SetFit algorithm from scratch using training utilities in sentence-transformers.

New docs in MKDocs

Switched ti MKDocs, because it's simpler to deal with and looks better.

Test suite

Added tests to ensure that the library will not break overnight.

Multiple GPU support

Run models on multiple GPU's by passing: Trooper(model_name, device_map="auto")

Copy link

@KennethEnevoldsen KennethEnevoldsen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next time I would probably split the pr into multiple.

CI:

  • It might be a good idea to make a makefile to unify CI across both local and CI

I would like to know more about how it selects models. - Mixing the APIs with HF models is also a bit confusing. I would split it up.

You are also hiding a lot of complexity from the user, it might be worth letting people know what is going on below the hood.

README.md Show resolved Hide resolved
README.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
README.md Show resolved Hide resolved
mkdocs.yml Outdated Show resolved Hide resolved
stormtrooper/set_fit.py Show resolved Hide resolved

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename test_trooper.py

stormtrooper/trooper.py Show resolved Hide resolved
@x-tabdeveloping x-tabdeveloping merged commit 5e9757b into main Sep 7, 2024
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants