Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[python-package] Fix mypy errors for fit() incompatible signature #5679

Merged
merged 5 commits into from
Jan 31, 2023
Merged

[python-package] Fix mypy errors for fit() incompatible signature #5679

merged 5 commits into from
Jan 31, 2023

Conversation

IdoKendo
Copy link
Contributor

Contributes to #3867.

mypy currently raises the following errors regarding the signature of the fit() method:

python-package\lightgbm\sklearn.py:987: error: Signature of "fit" incompatible with supertype "LGBMModel"  [override]
python-package\lightgbm\sklearn.py:987: note:      Superclass:
python-package\lightgbm\sklearn.py:987: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., group: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., eval_class_weight: Any = ..., eval_init_score: Any = ..., eval_group: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any      
python-package\lightgbm\sklearn.py:987: note:      Subclass:
python-package\lightgbm\sklearn.py:987: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., eval_init_score: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, 
bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any
python-package\lightgbm\sklearn.py:1033: error: Signature of "fit" incompatible with supertype "LGBMModel"  [override]
python-package\lightgbm\sklearn.py:1033: note:      Superclass:
python-package\lightgbm\sklearn.py:1033: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., group: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., eval_class_weight: Any = ..., eval_init_score: Any = ..., eval_group: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any     
python-package\lightgbm\sklearn.py:1033: note:      Subclass:
python-package\lightgbm\sklearn.py:1033: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., eval_class_weight: Any = ..., eval_init_score: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any
python-package\lightgbm\dask.py:1164: error: Signature of "fit" incompatible with supertype "LGBMClassifier"  [override]
python-package\lightgbm\dask.py:1164: note:      Superclass:
python-package\lightgbm\dask.py:1164: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., 
eval_class_weight: Any = ..., eval_init_score: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any
python-package\lightgbm\dask.py:1164: note:      Subclass:
python-package\lightgbm\dask.py:1164: note:          def fit(self, X: Union[Any, Any], y: Union[Any, Any, Any], sample_weight: Union[Any, Any, None] = ..., init_score: Union[Any, Any, Any, None] = ..., eval_set: Optional[List[Tuple[Union[Any, Any], Union[Any, Any, Any]]]] = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Optional[List[Union[Any, Any]]] = ..., eval_class_weight: Optional[List[Union[Dict[Any, Any], str]]] = ..., eval_init_score: Optional[List[Union[Any, Any, Any]]] = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., **kwargs: Any) -> DaskLGBMClassifier
python-package\lightgbm\dask.py:1164: error: Signature of "fit" incompatible with supertype "LGBMModel"  [override]
python-package\lightgbm\dask.py:1164: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., group: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., eval_class_weight: Any = ..., eval_init_score: Any = ..., eval_group: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any        
python-package\lightgbm\dask.py:1164: note:          def fit(self, X: Union[Any, Any], y: Union[Any, Any, Any], sample_weight: Union[Any, Any, None] = ..., init_score: Union[Any, Any, Any, None] = ..., eval_set: Optional[List[Tuple[Union[Any, Any], Union[Any, Any, Any]]]] = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Optional[List[Union[Any, Any]]] = ..., eval_class_weight: Optional[List[Union[Dict[Any, Any], str]]] = ..., eval_init_score: Optional[List[Union[Any, Any, Any]]] = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., **kwargs: Any) -> DaskLGBMClassifier
python-package\lightgbm\dask.py:1336: error: Signature of "fit" incompatible with supertype "LGBMRegressor"  [override]
python-package\lightgbm\dask.py:1336: note:      Superclass:
python-package\lightgbm\dask.py:1336: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., 
eval_init_score: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any
python-package\lightgbm\dask.py:1336: note:      Subclass:
python-package\lightgbm\dask.py:1336: note:          def fit(self, X: Union[Any, Any], y: Union[Any, Any, Any], sample_weight: Union[Any, Any, None] = ..., init_score: Union[Any, Any, None] = ..., eval_set: Optional[List[Tuple[Union[Any, Any], Union[Any, Any, Any]]]] = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Optional[List[Union[Any, Any]]] = ..., eval_init_score: Optional[List[Union[Any, Any]]] = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., **kwargs: Any) -> DaskLGBMRegressor
python-package\lightgbm\dask.py:1336: error: Signature of "fit" incompatible with supertype "LGBMModel"  [override]
python-package\lightgbm\dask.py:1336: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., group: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., eval_class_weight: Any = ..., eval_init_score: Any = ..., eval_group: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any        
python-package\lightgbm\dask.py:1336: note:          def fit(self, X: Union[Any, Any], y: Union[Any, Any, Any], sample_weight: Union[Any, Any, None] = ..., init_score: Union[Any, Any, None] = ..., eval_set: Optional[List[Tuple[Union[Any, Any], Union[Any, Any, Any]]]] = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Optional[List[Union[Any, Any]]] = ..., eval_init_score: Optional[List[Union[Any, Any]]] = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., **kwargs: Any) -> DaskLGBMRegressor
python-package\lightgbm\dask.py:1489: error: Signature of "fit" incompatible with supertype "LGBMRanker"  [override]
python-package\lightgbm\dask.py:1489: note:      Superclass:
python-package\lightgbm\dask.py:1489: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., group: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., eval_init_score: Any = ..., eval_group: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., eval_at: Union[List[int], Tuple[int, ...]] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any
python-package\lightgbm\dask.py:1489: note:      Subclass:
python-package\lightgbm\dask.py:1489: note:          def fit(self, X: Union[Any, Any], y: Union[Any, Any, Any], sample_weight: Union[Any, Any, None] = ..., init_score: Union[Any, Any, None] = ..., group: Union[Any, Any, None] = ..., eval_set: Optional[List[Tuple[Union[Any, Any], Union[Any, Any, Any]]]] = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Optional[List[Union[Any, Any]]] = ..., eval_init_score: Optional[List[Union[Any, Any]]] = ..., eval_group: Optional[List[Union[Any, Any]]] = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., eval_at: Union[List[int], Tuple[int, ...]] = ..., **kwargs: Any) -> DaskLGBMRanker
python-package\lightgbm\dask.py:1489: error: Signature of "fit" incompatible with supertype "LGBMModel"  [override]
python-package\lightgbm\dask.py:1489: note:          def fit(self, X: Any, y: Any, sample_weight: Any = ..., init_score: Any = ..., group: Any = ..., eval_set: Any = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Any = ..., eval_class_weight: Any = ..., eval_init_score: Any = ..., eval_group: Any = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., feature_name: Any = ..., categorical_feature: Any = ..., callbacks: Any = ..., init_model: Union[str, Path, Booster, LGBMModel, None] = ...) -> Any        
python-package\lightgbm\dask.py:1489: note:          def fit(self, X: Union[Any, Any], y: Union[Any, Any, Any], sample_weight: Union[Any, Any, None] = ..., init_score: Union[Any, Any, None] = ..., group: Union[Any, Any, None] = ..., eval_set: Optional[List[Tuple[Union[Any, Any], Union[Any, Any, Any]]]] = ..., eval_names: Optional[List[str]] = ..., eval_sample_weight: Optional[List[Union[Any, Any]]] = ..., eval_init_score: Optional[List[Union[Any, Any]]] = ..., eval_group: Optional[List[Union[Any, Any]]] = ..., eval_metric: Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]], List[Union[str, Union[Callable[[ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]], Callable[[ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any], ndarray[Any, Any]], Union[Tuple[str, float, bool], List[Tuple[str, float, bool]]]]]]], None] = ..., eval_at: Union[List[int], Tuple[int, ...]] = ..., **kwargs: Any) -> DaskLGBMRanker

Notes for Reviewers

This was tested by running mypy as documented in #3867.

mypy \
    --exclude='python-package/compile/|python-package/build' \
    --ignore-missing-imports \
    python-package/

jameslamb
jameslamb requested changes Jan 18, 2023
View reviewed changes
Copy link
Collaborator

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping with this!

But I'm sorry, I disagree with most of the proposed changes. The fit() methods of the estimators in lightgbm.sklearn should not be identical to each other...they expect different arguments for the different machine learning tasks (e.g., group is only relevant to ranking).

That same principle applies for the fit() methods on classes in lightgbm.dask, which are intended to mimic them. In addition, the lightgbm.dask estimators cannot simply have the same types for their arguments as the underlying lightgbm.sklearn things that they inherit from...special consideration has to be given there to how distributed training works and how to orchestrate moving data around multiple processes / machines.

If you haven't used Dask or LightGBM before, I recommend not closing this PR and not trying to attempt to fix the mypy warnings related to these fit() methods at this time.

python-package/lightgbm/dask.py Outdated
@@ -1166,14 +1167,20 @@ def fit(
X: _DaskMatrixLike,
y: _DaskCollection,
sample_weight: Optional[_DaskVectorLike] = None,
init_score: Optional[_DaskCollection] = None,
init_score: Optional[_DaskVectorLike] = None,
group: Optional[_DaskVectorLike] = None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

group is not supported by DaskLGBMClassifier. That argument is only relevant for ranking problems, not for classification or regression. Please remove this.

python-package/lightgbm/dask.py Outdated
@@ -1166,14 +1167,20 @@ def fit(
X: _DaskMatrixLike,
y: _DaskCollection,
sample_weight: Optional[_DaskVectorLike] = None,
init_score: Optional[_DaskCollection] = None,
init_score: Optional[_DaskVectorLike] = None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is not correct. init_score for multi-class classification can be multidimensional (e.g. 1 row per observation, 1 column per class), so _DaskVectorLike (which only has 1-dimensional types) is not appropriate.

python-package/lightgbm/dask.py Outdated
eval_metric: Optional[_LGBM_ScikitEvalMetricType] = None,
**kwargs: Any
eval_at: Union[List[int], Tuple[int, ...]] = (1, 2, 3, 4, 5),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eval_at is only relevant for ranking applications, not classification or regression. This doesn't belong on DaskLGBMClassifier.

python-package/lightgbm/dask.py Outdated
feature_name: str = 'auto',
categorical_feature: str = 'auto',
callbacks: Optional[List[Callable]] = None,
init_model: Optional[Union[str, Path, Booster, "LGBMModel"]] = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding support for training continuation in the Dask interface isn't as simple as just adding this argument with the same types as the underlying scikit-learn estimators.

For example, what is the user experience if this is a str or Path (implying a pretrained model in a local file)? Should the client code expect to find such a file on its local storage and broadcast it to all workers? Should every worker be expected to already have an identical copy of that file at an identical filepath on its local storage?

That is it's own feature work that should be done (with new docs and tests) in #4063.

python-package/lightgbm/dask.py Outdated
**kwargs: Any
feature_name: str = 'auto',
categorical_feature: str = 'auto',
callbacks: Optional[List[Callable]] = None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supporting callbacks in the Dask interface isn't as simple as just adding this argument to the signatures of these methods.

For example, unlike in the sklearn estimators these inherit from, in Dask all callbacks must be pickleable so they can be sent from the client to workers. (See the discussion in #5080 for some background).

I don't support adding this argument to the signature like this.

python-package/lightgbm/sklearn.py Outdated
@@ -1201,6 +1212,7 @@ def fit(
eval_set=None,
eval_names: Optional[List[str]] = None,
eval_sample_weight=None,
eval_class_weight=None,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

eval_class_weight is not relevant for the ranking task... it only matters for classification.

Please remove this.

python-package/lightgbm/sklearn.py Outdated
eval_metric: Optional[_LGBM_ScikitEvalMetricType] = None,
eval_at: Union[List[int], Tuple[int, ...]] = (1, 2, 3, 4, 5),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

group, eval_at, and eval_group are not relevant for the classification task. They're specific to ranking. Please revert these changes adding them to LGBMClassifier.fit().

@IdoKendo
Copy link
Contributor Author

Thanks for helping with this!

But I'm sorry, I disagree with most of the proposed changes. The fit() methods of the estimators in lightgbm.sklearn should not be identical to each other...they expect different arguments for the different machine learning tasks (e.g., group is only relevant to ranking).

Since mypy does not accept different signatures for child classes, these errors regarding differences in the signature are unsolvable then. Would you rather just silence them with # type: ignore?

@jameslamb
Copy link
Collaborator

Would you rather just silence them with # type: ignore?

I'm willing to look at a proposal for that! I'd want to be sure that #type: ignore is at the lowest level possible... e.g., if it's possible I'd prefer to not stop type checking at LGBMClassifier.fit() and all the way down, so mypy could still catch issues like "y should be a pandas Series or numpy array but you passed a list".

If it's possible to specifically opt out of these signature-mismatch warnings, that would be even better.

@IdoKendo
Copy link
Contributor Author

If it's possible to specifically opt out of these signature-mismatch warnings, that would be even better.

It is possible! I've reverted the previous changes and added the override ignore tag to the fit method to silence these errors specifically.

@IdoKendo IdoKendo requested review from jameslamb and removed request for jmoralez, shiyu1994 and StrikerRUS January 26, 2023 07:48
jameslamb
jameslamb approved these changes Jan 29, 2023
View reviewed changes
Copy link
Collaborator

@jameslamb jameslamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, thank you so much! I didn't know about # type: ignore[override] before. That's exactly what I was hoping for.

The failing CI jobs aren't related to your change. Once we fix them (in #5689), I'll re-run them for this PR and merge it.

@jameslamb jameslamb merged commit ed0a7f2 into microsoft:master Jan 31, 2023
@jameslamb
Copy link
Collaborator

Thanks again for the help @IdoKendo !

@IdoKendo IdoKendo deleted the python-package/fix-mypy-fit-errors branch March 9, 2023 15:20
@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed.
To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues
including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@jameslamb jameslamb jameslamb approved these changes

Assignees
No one assigned
Labels
maintenance
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@IdoKendo @jameslamb